Preempt_rt-based memory real-time enhancement method

Abstract

The application discloses a memory real-time enhancement method based on Preempt_RT, and comprises the following steps: step S1, constructing a user-mode memory pool without page fault exception: adjusting kernel start parameter memmap, reserving continuous physical address, downloading jemalloc source code and modifying configuration, writing LD_PRELOAD shared library, and replacing application memory allocation library through LD_PRELOAD; step S2, deploying and configuring Cache Color: configuring MPAM for a specified CPU, creating MPAM partition and configuring MPAM resource quota, and binding a target application to the specified CPU and MPAM partition; and step S3, system configuration optimization: disabling swap, closing ASLR, giving special permission to the target application, optimizing kswapd background recovery parameters and closing transparent large page background merging.

Description

Technical Field

[0001] This invention relates to the field of operating system technology, and in particular to a method for enhancing memory real-time performance based on Preempt_RT. Background Technology

[0002] The uncertainty of Linux kernel memory management stems from the design logic of dynamic on-demand allocation, global lock synchronization, and asynchronous background optimization. This conflict with the requirements of industrial control scenarios for control cycle stability, timely command response, and motion axis synchronization accuracy. Consequently, this leads to uncertainties in core industrial scenarios such as PLC logic control, motion control, and real-time sensor data processing, as specifically manifested in the following ways:

[0003] In the memory allocation process, the partner system needs to dynamically merge fragmented free pages. The number of merging times and the time taken vary with the degree of memory fragmentation in the system, which directly affects the execution accuracy of PLC cycle tasks (such as 1ms-level scan cycles) and leads to control deviations. After the per-CPU cache of the SLUB / SLAB allocator is exhausted, it needs to compete for a global lock to replenish it. The lock waiting time is affected by the concurrent load of multiple devices in the industrial field, which may cause lag in sensor data acquisition.

[0004] In the address translation process, the probability of TLB miss and the processing time are unpredictable due to the memory access patterns of industrial tasks (such as high-frequency IO data interaction) and address space randomization (ASLR) interference. TLB refresh operations triggered by context switching or page table modification need to synchronize the TLB status of multiple cores. In the multi-core architecture of industrial controllers, the synchronization delay increases with the number of cores, which destroys the synchronization accuracy of multi-axis linkage.

[0005] In the CPU cache interaction process, cache conflicts caused by set-associative mapping, false sharing caused by multi-core sharing of industrial control data, and dynamic selection of cache replacement strategies can cause uncontrollable timing and frequency of cache failure, which in turn leads to fluctuations in the execution time of control instructions and affects the real-time response of servo drives.

[0006] The uncertainty of memory reclamation mechanisms poses a particularly prominent threat to industrial control: the wake-up time of the background reclamation thread kswapd is determined by the vm.min_free_kbytes threshold and the real-time load of the industrial site (such as batch data upload and concurrent access of multiple devices). The scanning range and page cleanup time of the reclamation process are not fixed. Direct memory reclamation will block the current control task, and the time taken depends on the number of reclaimable pages and the strength of lock contention. If swap partition pages are involved, it will introduce completely unpredictable disk I / O delays, which may lead to emergency stop signal response timeouts.

[0007] When handling page faults, the disk data read latency of hard page faults is affected by the performance of industrial storage media (such as industrial-grade SSDs and CF cards) and the length of the field IO queue. The page table mapping operation of soft page faults requires competition for mmap_lock, and the latency fluctuates with the concurrency of industrial tasks (such as multi-station synchronous control), which disrupts the strict timing of control instructions.

[0008] In addition, in the scenario of continuous 24 / 7 operation of industrial equipment, the dynamic accumulation of memory fragmentation will continue to amplify the uncertainty of allocation and reclamation. Although kernel defragmentation can alleviate fragmentation, the process of moving physical page data will occupy CPU and memory bandwidth. Its execution timing and time cannot be predicted in advance, which may interfere with the real-time control process in the industrial field (such as production line cycle synchronization).

[0009] For hard real-time scenarios such as industrial control, existing Linux memory management technologies have significant shortcomings: not only are there unpredictable latency issues in memory allocation, address translation, cache interaction, garbage collection, and page fault handling, but related optimization solutions often require modifications to business application code, making adaptation to existing mature software costly and time-consuming. Furthermore, system-level optimization methods are fragmented and lack end-to-end coordination, making it difficult to meet the deterministic latency requirements of real-time scenarios. Currently, there is no good overall solution to these technical shortcomings. Summary of the Invention

[0010] To address the aforementioned issues, this invention provides a memory real-time performance enhancement method based on Preempt_RT, which solves the problems of uncertain memory access latency, high application adaptation costs, and interference from dynamic system operations in existing hard real-time scenarios.

[0011] This invention is implemented as follows:

[0012] A memory real-time performance enhancement method based on Preempt_RT constructs a deterministic memory management system through a multi-cooperative approach of hardware isolation, memory reservation, user-mode management, and system optimization to achieve memory real-time performance enhancement. Memory reservation and user-mode management are implemented through step S1, hardware isolation is implemented through step S2, and system optimization is implemented through step S3.

[0013] Step S1: Construct a user-mode memory pool free from page faults: Adjust the kernel startup parameter memmap, reserve contiguous physical addresses, download the jemalloc source code and modify its configuration, write the LD_PRELOAD shared library, and seamlessly replace the application memory allocation library with LD_PRELOAD to implement a user-mode memory pool management mechanism, eliminate memory allocation page faults, and avoid memory contention and fragmentation interference.

[0014] Step S2, Deploy and configure Cache Color: Configure MPAM for a specified CPU, create an MPAM partition and configure MPAM resource quotas, bind the target application to the specified CPU and MPAM partition, and improve the resource exclusivity and latency determinism of core real-time tasks;

[0015] Step S3, System Configuration Optimization: Disable swap, turn off ASLR, grant special permissions to the target application, optimize kswapd background recycling parameters and turn off transparent large page background merging, eliminate underlying interference, and compress unpredictable overhead caused by background processes.

[0016] The process of downloading the jemalloc source code and modifying the configuration includes: disabling mmap and brk, enabling thread-local caching, disabling background garbage collection and memory statistics, and compiling jemalloc into the dynamic library libjemalloc.so.

[0017] The process of writing the LD_PRELOAD shared library includes: using mmap to map the memory at the starting address, binding the memory allocation and deallocation functions of jemalloc to the mapped memory region at the starting address, rewriting multiple function bodies, and compiling them into the rt_jemalloc_hook.so dynamic library.

[0018] Specifically, configuring MPAM for a specified CPU in step S2 includes the following steps:

[0019] Step S211: Configure CPU0 to be isolated via grub, and prevent the system default task from being scheduled to this core;

[0020] Step S212: Add mpam.partid_base=100 to the kernel boot parameters, which sets the starting ID for the MPAM partition.

[0021] Creating an MPAM partition includes the following steps:

[0022] Step S221: Create the rt_cpu0_mpam folder in the / sys / fs / cgroup folder to create a cgroup V2 partition for the specified CPU;

[0023] Step S222: Restrict this partition to use only isolated CPU0;

[0024] Step S223: Write “+mpam +cache +memory.bandwidth” to the rt_cpu0_mpam file to enable MPAM, caching, and bandwidth control.

[0025] The optimization of kswapd background recycling parameters includes: binding kswapd to a core other than CPU0, limiting kswapd recycling speed, fixing the wake-up threshold of kswapd, and not actively recycling anonymous pages.

[0026] Specifically, 64 is written to / proc / sys / vm / pageout_io_blocks to limit the kswapd recycling speed.

[0027] Specifically, 1 is written to / proc / sys / vm / watermark_scale_factor to fix the wake-up threshold of kswapd.

[0028] Specifically, 0 is written to / proc / sys / vm / swappiness to prevent anonymous pages from being actively reclaimed.

[0029] Specifically, `never` is written to ` / sys / kernel / mm / transparent_hugepage / khugepaged / defrag` to disable transparent big page back-end merging.

[0030] The beneficial effects of this invention are:

[0031] The memory real-time performance enhancement method based on Preempt_RT of this invention can effectively eliminate latency fluctuations caused by page faults, lock contention, and memory fragmentation, reduce TLB miss probability and refresh latency, and compress unpredictable overhead caused by swap partition disk I / O and kswapd background reclamation. At the same time, it enables seamless access to business software with zero code modification, taking into account both hard real-time requirements and application compatibility. On hardware platforms that support MPAM, it can also enhance the exclusive use of cache and memory bandwidth resources of specified CPU cores, further improving the latency stability of core real-time tasks. Attached Figure Description

[0032] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0034] This invention primarily addresses core issues in hard real-time scenarios, such as uncertain memory access latency, high application adaptation costs, and interference from dynamic system operations. The overall approach involves a multi-layered collaborative solution encompassing hardware isolation, memory reservation, user-mode management, and system optimization to construct a deterministic memory management system, thereby enhancing memory real-time performance. This includes:

[0035] First, dedicated contiguous physical memory is reserved through kernel memmap startup parameters, thus bypassing the kernel's dynamic memory management mechanism. Then, a customized jemalloc library is used to disable mmap / brk and enable thread caching. Combined with LD_PRELOAD function interception technology, the application's standard memory operations are seamlessly routed to the reserved memory pool, avoiding the uncertainty of dynamic allocation and eliminating page faults, lock contention, and fragmentation interference during memory allocation.

[0036] For hardware that supports MPAM, dedicated cache and bandwidth resources are allocated to the core CPU through CPU isolation and cgroup V2 partition configuration, which addresses the issue of cache and memory bandwidth resource contention of the CPU core and improves the resource exclusivity and latency determinism of the core real-time tasks.

[0037] Meanwhile, by optimizing system configurations such as reducing TLB Miss probability and refresh latency, disabling ASLR, disabling swap, optimizing kswapd, and prohibiting transparent large page background operations, we can eliminate underlying interference and compress unpredictable overhead caused by swap partition disk I / O and background processes such as kswapd.

[0038] Through the three main inventions mentioned above, a seamless replacement of the memory management mechanism is achieved without modifying the business software code, taking into account both hard real-time requirements and application compatibility, forming a deterministic guarantee across the entire chain of hardware isolation, memory reservation, user-mode management, and system optimization.

[0039] like Figure 1 The present invention illustrates a memory real-time performance enhancement method based on Preempt_RT, comprising steps S1 to S3.

[0040] This invention is based on the Phytium D2000 hardware platform, using the Phytium open-source kernel version 5.10-RT. Because memory allocation, address translation, CPU cache, memory reclamation, page faults, and memory fragmentation are unpredictable, this invention utilizes a user-mode memory pool management technology to mitigate interference from memory sharding, reclamation, page faults, and memory fragmentation. It also reduces CPU cache uncertainty through Cache Color technology, reduces memory reclamation uncertainty by disabling kswapd and swap, and reduces address translation uncertainty by disabling ASLR and large memory pages. The specific steps include steps S1 to S3 as described below.

[0041] Step S1, constructing a user-mode memory pool free from page faults: Adjust the kernel startup parameter memmap, reserve contiguous physical addresses, download the jemalloc source code and modify its configuration, write the LD_PRELOAD shared library, and use LD_PRELOAD user-mode function interception technology to seamlessly replace the application memory allocation library through LD_PRELOAD, thereby implementing a user-mode memory pool management mechanism, eliminating memory allocation page faults, and avoiding memory contention and fragmentation interference. This ensures that business software is unaffected and improves the real-time performance of memory access. Specifically, this includes the following steps S11 to S13.

[0042] Step S11, reserve consecutive physical addresses:

[0043] (1) Adjust the kernel boot parameters, for example, reserve 2GB and the physical address starts from 0x100000000, memmap=2G$0x100000000;

[0044] (2) An entry of 0x100000000 was observed in the reserved field of / proc / iomem.

[0045] Step S12: Download the jemalloc source code and modify the configuration as follows:

[0046] (1) Disable mmap and brk, and only use the reserved memory provided by this invention (disable-mmap=true, disable-brk=true).

[0047] (2) Enable thread-local caching to reduce lock contention and improve real-time performance (enable-tcache=true).

[0048] (3) Disable background recycling to avoid uncertainty introduced by asynchronous operations (disable-background-thread=true);

[0049] (4) Disable memory statistics to reduce overhead (disable-stats=true);

[0050] (5) Compile jemalloc into a dynamic library libjemalloc.so.

[0051] Step S13: Writing the LD_PRELOAD shared library includes: using mmap to map the memory at the starting address, binding the jemalloc memory allocation and deallocation functions to the mapped starting address memory region, rewriting multiple function bodies, and compiling them into the rt_jemalloc_hook.so dynamic library. Specifically, it is set up as follows:

[0052] (1) Use mmap to map a memory with a starting address of 0x100000000 and a size of 2GB, and define the region as rt_mmap;

[0053] (2) Bind the memory allocation and deallocation functions of jemalloc to the rt_mmap memory region;

[0054] (3) Rewrite the malloc function body, and call the jemalloc function at the underlying level;

[0055] (4) Rewrite the free function body, which calls the je_free function at the underlying level;

[0056] (5) Rewrite the calloc function body, which calls the je_calloc function at the underlying level;

[0057] (6) Rewrite the realloc function body, which calls the je_realloc function at the underlying level;

[0058] (7) Compile the above content into the rt_jemalloc_hook.so dynamic library.

[0059] Then, the LD_PRELOAD variable is used to set the rt_jemalloc_hook.so and libjemalloc.so libraries, enabling seamless replacement of the application memory allocation library.

[0060] Step S2, Deploy and Configure Cache Color: Configure MPAM for the specified CPU, create MPAM partitions and configure MPAM resource quotas, bind the target application to the specified CPU and MPAM partition, and improve the resource exclusivity and latency determinism of core real-time tasks.

[0061] This technology refers to the Cache QoS and Memory Bandwidth QoS features introduced by the ARMv8.4-A instruction set. Most chips lack this capability, such as the Phytium D2000 device in the current test platform. The following steps can be verified on the Kunpeng 920 high-performance server device:

[0062] Specifically, configuring MPAM for a specified CPU in step S2 includes the following steps:

[0063] Step S211: Configure CPU0 to be isolated via grub, and prevent the system default task from being scheduled to this core;

[0064] Step S212: Add mpam.partid_base=100 to the kernel boot parameters, which sets the starting ID for the MPAM partition.

[0065] The creation of the MPAM partition in step S2 specifically includes the following steps:

[0066] Step S221: Create the folder rt_cpu0_mpam in the / sys / fs / cgroup folder. Create a cgroup V2 partition for the specified CPU, i.e., create a cgroup V2 partition for CPU0. At this time, the system will automatically create cgroup.subtree_control, cache.max, cgroup.procs, mpam.partid, cgroup.procs, memory.bandwidth.min, memory.bandwidth.max, and mpam.partid in the rt_cpu0_mpam folder.

[0067] Step S222: Restrict this partition to use only isolated CPU0;

[0068] Step S223: Write “+mpam +cache +memory.bandwidth” to the rt_cpu0_mpam file to enable MPAM, caching, and bandwidth control.

[0069] Specifically, configuring MPAM resource quotas in step S2 is done through the following steps:

[0070] The current device has a 16-way L3 cache and a total memory bandwidth of 20GB / s. Dedicated resources are allocated to CPU0 as follows:

[0071] (1) Write "l3:ways=4" to the newly created cache.max file in the rt_cpu0_mpam folder to allocate 4-way L3 Cache to CPU0;

[0072] (2) Write “8G” to the newly created memory.bandwidth.min file in the rt_cpu0_mpam folder, that is, reserve 8GB / s of memory bandwidth;

[0073] (3) Write “10G” to the newly created memory.bandwidth.max in the rt_cpu0_mpam folder, which is the maximum memory bandwidth limit of 10GB / s;

[0074] (4) Write “100” into the newly created mpam.partid in the rt_cpu0_mpam folder, that is, assign PARTID (consistent with the kernel parameter base).

[0075] In step S2, binding the target application to the specified CPU and MPAM partition involves the following steps:

[0076] (1) Use the taskset command to bind the specified application to CPU0;

[0077] (2) Write the process PID to the newly created cgroup.procs file in the rt_cpu0_mpam folder, that is, add the task to the MPAM partition.

[0078] Step S3, System Configuration Optimization: Disable swap, turn off ASLR, grant special permissions to the target application, optimize kswapd background recycling parameters and turn off transparent large page background merging, eliminate underlying interference, and compress unpredictable overhead caused by background processes.

[0079] Specifically, use swapoff -a to disable swap, write 0 to / proc / sys / kernel / randomize_va_space, disable ASLR, reduce TLB miss probability and flush latency, and grant the process mlock and access permissions to / dev / mem, namely IPC_LOCK, CAP_SYS_RAWIO+ep permissions.

[0080] The optimization of kswapd background reclamation parameters includes: using the taskset command to bind kswapd to a core other than CPU0, such as CPU1; writing 64 to / proc / sys / vm / pageout_io_blocks to limit the kswapd reclamation speed; writing 1 to / proc / sys / vm / watermark_scale_factor to fix the kswapd wake-up threshold; and writing 0 to / proc / sys / vm / swappiness to prevent the active reclamation of anonymous pages.

[0081] Specifically, `never` is written to ` / sys / kernel / mm / transparent_hugepage / khugepaged / defrag` to disable transparent big page back-end merging.

[0082] This invention employs a collaborative design between memmap reserving contiguous physical memory and a customized jemalloc library. By disabling the system memory allocation path of jemalloc, it forces the library to use only reserved memory, achieving deterministic management of the user-mode memory pool. A seamless replacement mechanism based on LD_PRELOAD, by rewriting standard functions such as malloc / free and routing them to the customized jemalloc, enables zero-code modification adaptation for business applications. A combined optimization scheme for system configuration, namely disabling swap, turning off ASLR, optimizing kswapd, and disabling transparent large-page background operations, completely eliminates interference from dynamic system operations. A CPU-level resource isolation scheme supporting MPAM hardware is implemented by isolating the target CPU through kernel parameters, configuring MPAM resource quotas (cache path count, bandwidth upper and lower limits) for cgroup V2 partitions, and binding real-time tasks to dedicated partitions, achieving exclusive hardware-level resources. The binding mechanism between reserved memory mapping and the jemalloc memory pool, through mmap mapping of reserved physical memory and locking it to physical pages, ensures no page faults.

[0083] This invention improves the performance of core real-time tasks by employing three technical means: user-mode memory pool management without page faults, cache color deployment and configuration technology, and system configuration optimization. These technologies address the interference of memory sharding, memory reclamation, page faults, and memory fragmentation, reduce the uncertainty of CPU cache, and reduce the uncertainty of memory reclamation and address translation.

[0084] In summary, this invention can effectively eliminate latency fluctuations caused by page faults, lock contention, and memory fragmentation, reduce TLB miss probability and refresh latency, and compress unpredictable overhead caused by swap partition disk I / O and kswapd background reclamation. Simultaneously, it enables seamless access to business software with zero code modification, balancing hard real-time requirements with application compatibility. On hardware platforms supporting MPAM, it can also enhance the exclusive access of cache and memory bandwidth resources for designated CPU cores, further improving the latency stability of core real-time tasks.

[0085] While the present invention discloses preferred embodiments to achieve the above objectives, these are not intended to limit the structural features of the invention. Anyone skilled in the art should know that any easily conceived variations or modifications are possible within the technical spirit of the invention and are covered by the claims of the present invention.

Claims

1. A memory real-time performance enhancement method based on Preempt_RT, characterized in that, A deterministic memory management system is constructed through a multi-cooperative approach of hardware isolation, memory reservation, user-mode management, and system optimization to enhance memory real-time performance. Specifically, memory reservation and user-mode management are implemented through step S1, hardware isolation is implemented through step S2, and system optimization is implemented through step S3. Step S1: Construct a user-mode memory pool free from page faults: Adjust the kernel startup parameter memmap, reserve contiguous physical addresses, download the jemalloc source code and modify its configuration, write the LD_PRELOAD shared library, and seamlessly replace the application memory allocation library with LD_PRELOAD to implement a user-mode memory pool management mechanism, eliminate memory allocation page faults, and avoid memory contention and fragmentation interference. Step S2, Deploy and configure Cache Color: Configure MPAM for a specified CPU, create an MPAM partition and configure MPAM resource quotas, bind the target application to the specified CPU and MPAM partition, and improve the resource exclusivity and latency determinism of core real-time tasks; Step S3, System Configuration Optimization: Disable swap, turn off ASLR, grant special permissions to the target application, optimize kswapd background recycling parameters and turn off transparent large page background merging, eliminate underlying interference, and compress unpredictable overhead caused by background processes.

2. The memory real-time performance enhancement method based on Preempt_RT according to claim 1, characterized in that, Download the jemalloc source code and modify the configuration, including: disabling mmap and brk, enabling thread-local caching, disabling background garbage collection and memory statistics, and compiling jemalloc into the dynamic library libjemalloc.so.

3. The memory real-time performance enhancement method based on Preempt_RT according to claim 2, characterized in that, Writing the LD_PRELOAD shared library involves: using mmap to map the memory at the starting address, binding the jemalloc memory allocation and deallocation functions to the mapped starting address memory region, rewriting multiple function bodies, and compiling them into the rt_jemalloc_hook.so dynamic library.

4. The memory real-time performance enhancement method based on Preempt_RT according to claim 1, characterized in that, Configuring MPAM for a specified CPU in step S2 includes the following steps: Step S211: Configure CPU0 to be isolated via grub, and prevent the system default task from being scheduled to this core; Step S212: Add mpam.partid_base=100 to the kernel boot parameters, which sets the starting ID for the MPAM partition.

5. The memory real-time performance enhancement method based on Preempt_RT according to claim 1, characterized in that, Creating an MPAM partition involves the following steps: Step S221: Create the rt_cpu0_mpam folder in the / sys / fs / cgroup folder to create a cgroupV2 partition for the specified CPU; Step S222: Restrict this partition to use only isolated CPU0; Step S223: Write "+mpam +cache +memory.bandwidth" to the rt_cpu0_mpam file to enable MPAM, caching, and bandwidth control.

6. The memory real-time performance enhancement method based on Preempt_RT according to claim 4, characterized in that, Optimizing kswapd background recycling parameters includes: binding kswapd to a core other than CPU0, limiting kswapd recycling speed, fixing the wake-up threshold of kswapd, and not actively recycling anonymous pages.

7. The memory real-time performance enhancement method based on Preempt_RT according to claim 6, characterized in that, Write 64 to / proc / sys / vm / pageout_io_blocks to limit the kswapd reclamation speed.

8. The memory real-time performance enhancement method based on Preempt_RT according to claim 6, characterized in that, Write 1 to / proc / sys / vm / watermark_scale_factor to fix the wake-up threshold of kswapd.

9. The memory real-time performance enhancement method based on Preempt_RT according to claim 6, characterized in that, Write 0 to / proc / sys / vm / swappiness to prevent the active reclamation of anonymous pages.

10. The memory real-time performance enhancement method based on Preempt_RT according to claim 1, characterized in that, Write `never` to ` / sys / kernel / mm / transparent_hugepage / khugepaged / defrag` to disable transparent big page back-end merging.

Images