A memory allocation optimization method and device for ATS cache

Abstract

The present invention discloses a method, device and storage medium for optimizing memory allocation of an ATS cache, and relates to the technical field of content distribution networks. The method comprises configuring relevant threshold parameters, and selecting a memory allocation strategy of tcmalloc or a memory pool memory strategy of ATS to allocate memory according to the relevant threshold parameters and the file size. When allocating memory in ATS, a threshold for use is set between the memory allocation method of the native freelist of ATS and the tcmalloc generally recognized by the industry, so as to give full play to the advantages of both and minimize the disadvantages of both; in the case of high concurrency requests for large and small files, it is achieved that both high concurrency requests for large files and high concurrency requests for small files can be met, memory can be reasonably allocated, high CPU usage can be avoided, and response speed can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of content distribution networks, and in particular to a method, device and storage medium for optimizing memory allocation of an ATS cache. Background Art

[0002] The principle of content distribution network is to use nodes at all levels of CDN to cache resources from remote source stations to nodes closer to users. Users can obtain resources nearby, thereby reducing the pressure on the source station and improving access speed.

[0003] The ATS cache system, or Apache Traffic Server, is a high-performance, modular HTTP proxy and cache server similar to Nginx and Squ id. When processing concurrent requests for large files like videos and downloads, as well as high-concurrency requests for small files like website files, cache systems typically cache large files in shards to improve response speed, reduce back-to-origin requests, and increase hit rates. However, when concurrent requests are high, the use of sharded caching increases concurrency, which can lead to memory and CPU usage spikes and slow responses.

[0004] The ATS cache system uses free list as a memory allocation method, which constantly occupies the global space. This may lead to a bad situation where a thread occupies a large amount of global space when performing special operations, resulting in uneven memory distribution between threads. The optimized ATS cache system has more reasonable control when the file is large, but in the case of small files and high concurrency, it is still easy for the CPU to reach a high level, but the concurrency number is not ideal. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a memory allocation optimization method, device and storage medium for ATS cache, which solves the problem that the existing technology is difficult to meet high-concurrency requests for large files and small files at the same time. It can meet high-concurrency requests for large files and small files at the same time, reduce the probability of high CPU and memory usage, and improve response speed.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for optimizing memory allocation of an ATS cache, comprising the following steps:

[0007] S1. Configure relevant threshold parameters;

[0008] S2. Based on the relevant threshold parameters and file size, select the memory allocation strategy of tcmall oc or the memory pool memory strategy of ats to allocate memory.

[0009] Furthermore, the relevant threshold parameters in S1 include a file size threshold of the usage policy and a freelist memory pool size threshold of ATS.

[0010] Furthermore, the default value of the file size threshold of the usage policy is set to 512K.

[0011] Furthermore, the file size threshold of the usage policy is dynamically adjusted according to the proportion of small files on the cache machine.

[0012] Furthermore, the free list memory pool size threshold of the ATS is based on the total memory size of the machine and dynamic statistics of memory applications.

[0013] Furthermore, the specific steps of S2 include:

[0014] S21. Determine whether the file size is smaller than the file size threshold of the usage policy. If so, execute the memory allocation policy of tcmall oc; otherwise, execute the memory pool memory policy of ats;

[0015] S22. After executing the ATS memory pool memory policy, determine whether the memory pool saved by ATS has reached the freelist memory pool size threshold of ATS. If so, release the memory directly; otherwise, save it to the ATS memory pool.

[0016] Furthermore, when using the free list memory pool, the secondary thread memory cache size is increased to reduce the number of times the memory is released to the free list at a time.

[0017] Furthermore, when there is free space in the ATS memory pool, it is used directly; when there is no free space, it is allocated from the tcma ll oc.

[0018] A device for optimizing memory allocation of an ATS cache includes a configuration module and a policy execution module, wherein:

[0019] Configuration module, used to configure relevant threshold parameters;

[0020] The policy execution module is used to allocate memory by selecting the memory allocation policy of tcmall oc or the memory pool memory policy of ats based on relevant threshold parameters and file size.

[0021] A computer-readable storage medium is used to store a program, which, when executed by a processor, implements a memory allocation optimization method for an ATS cache.

[0022] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0023] When allocating memory in ATS, a threshold is set between the native free list memory allocation method of ATS and the industry-wide recognized tcmall oc. When the file is large, the native free list memory allocation method of ATS is used to avoid the problem of tcmall oc's spin lock seriously preempting the CPU. When the file is small, the tcmall oc memory allocation method is used to fully utilize the advantage of tcmall oc's advantage of basically no lock contention for allocating small objects, avoiding the problem of unbalanced memory allocation between threads of the native free list of ATS. This achieves the effect of meeting both high-concurrency requests for large files and high-concurrency requests for small files, reducing the probability of high CPU and memory usage, and improving response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of a method for optimizing memory allocation of an ATS cache according to the present invention;

[0025] Figure 2 This is a schematic diagram of the CPU occupancy distribution of ATS including TCMALL OC in the prior art of the present invention;

[0026] Figure 3 This is a complete example diagram of a memory allocation optimization method for ATS cache according to the present invention;

[0027] Figure 4 3M file configuration for concurrent request testing of the present invention Figure 1 ;

[0028] Figure 5 3M file configuration for concurrent request testing of the present invention Figure 2 ;

[0029] Figure 6 3M file configuration for concurrent request testing of the present invention Figure 3 ;

[0030] Figure 7 0.5M file configuration for concurrent request testing of this invention Figure 1 ;

[0031] Figure 8 0.5M file configuration for concurrent request testing of this invention Figure 2 ;

[0032] Figure 9 0.5M file configuration for concurrent request testing of this invention Figure 3 ;

[0033] Figure 10 This is a structural diagram of a memory allocation optimization device for ATS cache according to the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0035] The present invention proposes a memory allocation optimization method, device and storage medium for an ATS cache, which solves the problem that the existing technology is difficult to meet high-concurrency requests for large files and small files at the same time. It can meet high-concurrency requests for large files and small files at the same time, reduce the probability of high CPU and memory usage, and improve response speed.

[0036] The technical solution in the embodiments of the present application is to solve the problem that the above-mentioned prior art is difficult to meet the high concurrency requests for large files while also meeting the high concurrency requests for small files. The overall idea is as follows:

[0037] By comprehensively utilizing the advantages and disadvantages of the native free list memory allocation method of ATS and the industry-recognized tcmall oc memory allocation method, the native free list memory allocation method of ATS is adopted for large files to avoid the problem of tcmall oc's spin lock seriously preempting the CPU. The memory allocation method of tcmall oc is adopted for small files to fully utilize the advantage of tcmall oc's advantage of basically no lock contention for small object allocation, thus avoiding the problem of uneven memory allocation between threads of the native free list of ATS.

[0038] A dynamically adjustable threshold is provided. It can be adjusted dynamically between the native free list of ATS and the tcmall oc based on the actual business scenario. If there are no special requirements, the default threshold can be used directly.

[0039] tcmall oc is a memory manager widely recognized in the industry. Its full name is Thread-Cache Malloc, which means thread-cached malloc. It implements efficient multi-threaded memory management and is used to replace the system's memory allocation-related functions. It is expected to reduce performance losses caused by frequent memory allocation and release, and effectively control memory fragmentation. tcmall oc is particularly optimized for multi-threading. There is basically no lock contention for small object allocations, while large objects use fine-grained, efficient spin locks. However, when the version of ATS containing tcmall oc requested a 3M file in an offline concurrent test, the spin lock of tcmall oc had a high CPU usage, such as Figure 2 shown.

[0040] like Figure 1 As shown in FIG, a method for optimizing memory allocation of an ATS cache provided in an embodiment of the present application is applied to a device for optimizing memory allocation of an ATS cache, and includes the following steps:

[0041] S1. Configure relevant threshold parameters;

[0042] S2. Based on the relevant threshold parameters and file size, select the memory allocation strategy of tcmall oc or the memory pool memory strategy of ats to allocate memory.

[0043] Furthermore, the relevant threshold parameters in S1 include the file size threshold of the usage policy and the freelist memory pool size threshold of ATS;

[0044] Specifically, the default value of the file size threshold of the usage policy is set to 512K;

[0045] Specifically, the file size threshold of the usage policy is dynamically adjusted based on the proportion of small files on the cache machine;

[0046] Specifically, the free list memory pool size threshold of the ATS is based on the total memory size of the machine and dynamic statistics of memory applications.

[0047] In this implementation, before performing memory allocation optimization, the file size threshold a of the usage policy is configured. The default value of the file size threshold a of the usage policy is set to 512K. This threshold can then be dynamically adjusted based on the proportion of small files on the cache machine. For example, if statistics show that most cached files over a period of time are distributed between 0 and 512K, the file size threshold a of the usage policy will be dynamically adjusted to 256K to avoid high CPU usage.

[0048] In addition, you also need to configure the free list memory pool size threshold B of ATS. The free list memory pool size threshold B of ATS is calculated based on the total memory size of the machine and dynamic statistics of memory requests. The average memory size is calculated per unit time to avoid memory exhaustion at the highest point, resulting in OOM. The threshold obtained by statistical calculation is used to determine whether to use the lock-free free list memory pool.

[0049] Further, such as Figure 3 As shown, the specific steps of S2 include:

[0050] S21. Determine whether the file size is smaller than the file size threshold of the usage policy. If so, execute the memory allocation policy of tcmall oc; otherwise, execute the memory pool memory policy of ats;

[0051] S22. After executing the ATS memory pool memory policy, determine whether the memory pool saved by ATS has reached the freelist memory pool size threshold of ATS. If so, release the memory directly; otherwise, save it to the ATS memory pool.

[0052] In this implementation scheme, for files smaller than the file size threshold a of the usage policy, the memory allocation strategy of tcma l loc is directly used, and the memory is directly released after completing this request. If the file size is larger than the file size threshold a of the usage policy, the memory pool memory strategy of ATS is executed. After the request is completed, the memory is prepared to be released. Then, it is determined whether the total size of the ATS memory pool reaches the free list memory pool size threshold b of ATS. If so, the memory is directly released after completing this request. If not, the memory used for this request will not be released and will be directly stored in the ATS memory pool.

[0053] Specifically, when using the free list memory pool, increase the secondary thread memory cache size and reduce the number of times the memory is released to the free list at a time;

[0054] When there is free space in the ATS memory pool, it is used directly. When there is no free space, it is allocated from the tcma ll oc.

[0055] In this implementation scheme, when using the free list memory pool, in order to reduce the number of requests and releases, the secondary thread memory cache size is increased to reduce the number of times the memory is released to the free list at a time. In addition, if there is free memory in the ATS memory pool, the free memory will be used directly. If there is no free memory, it will be allocated from the tcmal loc.

[0056] To test the free list optimization algorithm configuration items and tcmalloc for configuring the ATS itself, first test 3M concurrent requests. The specific steps are as follows:

[0057] The first step, such as Figure 4 As shown, configure thread_free list_size to 1024 and thread_free list_low_watermark to 1023, that is,

[0058] CONF IG proxy.conf ig.all ocator.thread_free li st_s i ze INT1024

[0059] CONF IG proxy.conf ig.all ocator.thread_free li st_l ow_watermarkINT1023;

[0060] The second step is Figure 5 As shown, configure thread_free list_size to 512 and thread_free list_low_watermark to 511, that is,

[0061] proxy.conf ga ll ocator.thread_free li st_s i ze INT 512

[0062] CONF IG proxy.conf i ga ll ocator.thread_free li st_l ow_watermark I NT511;

[0063] The third step, such as Figure 6 As shown, memory sharding uses Tcmall oc, which is not the native free list of ATS.

[0064] analyze Figure 4-Figure 6 It can be seen that in the scenario of concurrently requesting 3M files, configuring thread_free list_size to 1024 and thread_free list_low_watermark to 1023 achieves the best performance, with a stress test bandwidth of about 4.4G and a CPU usage of only about 22%. The performance of using tcmall oc for memory is the worst, with a stress test bandwidth of about 4.1G, but a CPU usage of about 62%.

[0065] Test concurrent requests for 0.5M files:

[0066] The first step, such as Figure 7 As shown, configure thread_free list_size to 1024 and thread_free list_low_watermark to 1023, that is,

[0067] CONF IG proxy.conf i ga ll ocator.thread_free li st_s i ze I NT1024

[0068] CONF IG proxy.conf i ga ll ocator.thread_free li st_l ow_watermark I NT1023;

[0069] The second step is Figure 8 As shown, configure thread_free list_size to 512 and thread_free list_low_watermark to 511, that is,

[0070] proxy.conf i ga ll ocator.thread_free li st_s i ze I NT 512

[0071] CONF IG proxy.conf i ga ll ocator.thread_free li st_l ow_watermark I NT511;

[0072] The third step, such as Figure 9 As shown, memory sharding uses Tcmall oc, which is not the native free list of ATS.

[0073] analyze Figure 7-Figure 9 It can be seen that in the scenario of concurrently requesting 0.5M files,

[0074] Using tcmall oc for memory achieved the best performance, with a stress test bandwidth of approximately 3.4 GB and a CPU utilization of approximately 70%. Setting thread_free list_size to 512 and thread_free list_low_watermark to 511 resulted in the worst performance, with a stress test bandwidth of approximately 2.6 GB and a CPU utilization of approximately 57%. The bandwidth of tcmall oc was 1.31 times that of free list, and the CPU utilization was 1.23 times that of free list.

[0075] Therefore, when the native free list memory allocation method of ATS takes effect, it is recommended to use the configuration:

[0076] CONF IG proxy.conf ig.all ocator.thread_free li st_s i ze INT1024

[0077] CONF IG proxy.conf ig.all ocator.thread_free li st_l ow_watermarkINT1023

[0078] ATS's memory allocation optimization algorithm recommends setting a threshold for the memory allocation algorithm so that ATS's native free list memory allocation method takes effect in larger files and tcmall oc's memory allocation method takes effect in files smaller than 1M.

[0079] The embodiment of the present application provides a memory allocation optimization device for ATS cache, such as Figure 10 As shown, it includes a configuration module and a policy execution module, where:

[0080] Configuration module, used to configure relevant threshold parameters;

[0081] The policy execution module is used to allocate memory by selecting the memory allocation policy of tcmall oc or the memory pool memory policy of ats based on relevant threshold parameters and file size.

[0082] In this implementation scheme, the configuration module configures relevant parameters, including the file size threshold of the usage policy and the free list memory pool size threshold of ATS. After that, the policy execution module will first compare the file size with the file size threshold of the usage policy to determine whether to use the memory allocation strategy of tcmail oc or the memory pool memory strategy of ATS. Finally, based on the relationship between the total size of the ACT memory pool and the free list memory pool size threshold of ATS, it will choose to directly release the memory or save the memory.

[0083] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0084] When allocating memory in ATS, a threshold is set between the native free list memory allocation method of ATS and the industry-recognized tcmal l oc. When the file is large, the native free list memory allocation method of ATS is used to avoid the problem of tcmal l oc's spin lock seriously preempting the CPU. When the file is small, the tcmal l oc memory allocation method is used to fully utilize the advantage of tcmal l oc's advantage in allocating small objects without basically no lock contention, avoiding the problem of unbalanced memory allocation between threads of the native free list of ATS.

[0085] A computer-readable storage medium is used to store a program, which, when executed by a processor, implements a memory allocation optimization method for an ATS cache.

[0086] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0090] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0091] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for optimizing memory allocation of an ATS cache, characterized in that: The following steps are involved: S1. Configure relevant threshold parameters; S2. Based on the relevant threshold parameters and file size, select the tcmalloc memory allocation strategy or the ATS memory pool memory strategy to allocate memory; The specific steps of S2 include: S21. Determine whether the file size is smaller than the file size threshold of the usage policy. If so, execute the memory allocation policy of tcmalloc; otherwise, execute the memory pool memory policy of ats. S22. After executing the ATS memory pool memory policy, determine whether the memory pool stored in ATS has reached the ATS freelist memory pool size threshold. If so, release the memory directly; otherwise, save it to the ATS memory pool.

2. The method for optimizing memory allocation of an ATS cache according to claim 1, wherein: The relevant threshold parameters in S1 include the file size threshold of the usage policy and the freelist memory pool size threshold of the ATS.

3. The method for optimizing memory allocation of an ATS cache according to claim 2, wherein: The default value of the file size threshold of the usage policy is set to 512K.

4. The method for optimizing memory allocation of an ATS cache according to claim 3, wherein: The file size threshold of the usage policy is dynamically adjusted according to the proportion of small files on the cache machine.

5. The method for optimizing memory allocation of an ATS cache according to claim 2, wherein: The freelist memory pool size threshold of the ATS is based on the total memory size of the machine and dynamic statistics of memory requests.

6. The method for optimizing memory allocation of an ATS cache according to claim 5, wherein: When using the freelist memory pool, increase the secondary thread memory cache size and reduce the number of times memory is released to the freelist at a time.

7. The method for optimizing memory allocation of an ATS cache according to claim 1, wherein: When there is free space in the ATS memory pool, it is used directly. If there is no free space, it is allocated from tcmalloc.

8. A memory allocation optimization device for ATS cache, characterized in that: It includes configuration module and policy execution module, including: Configuration module, used to configure relevant threshold parameters; The policy execution module is used to allocate memory based on relevant threshold parameters and file size, selecting the tcmalloc memory allocation strategy or the ATS memory pool memory strategy. Specifically, it includes: Determine whether the file size is less than the file size threshold of the usage policy. If so, execute the memory allocation policy of tcmalloc; otherwise, execute the memory pool memory policy of ats; After executing the ATS memory pool memory policy, it is determined whether the memory pool saved by ATS has reached the ATS freelist memory pool size threshold. If so, the memory is released directly; otherwise, it is saved in the ATS memory pool.

9. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, a memory allocation optimization method for an ATS cache according to any one of claims 1 to 7 is implemented.

Images