A memory compression management system for embedded device table storage

Abstract

The application discloses a memory compression management system for embedded device table storage, and has the characteristics that the system comprises an evicted page compression selector, a partition design and a repeated page filter, the evicted page compression selector uses an entropy sampling algorithm to estimate the compression rate of the evicted page and determines the compressed page, the partition design uses zRAM memory space for frequently swapped-in / swept-out pages and uses auxiliary storage space for infrequently swapped-in / swept-out pages, and the repeated page filter adds two design page tables, namely, a Swap Index Table and a Swap Page Table, to construct a Swap page index. Compared with the prior art, the application has the advantages of high memory utilization, reduced system energy consumption caused by the swap partition, reduced influence on application caused by the swap, further improved storage performance of the table storage system, relieved memory pressure of the embedded device, and good application prospect and commercial value.

Description

Technical Field

[0001] This invention relates to the field of memory development technology, and in particular to a memory compression management system for table storage in embedded devices. Background Technology

[0002] Small embedded devices, due to their limited memory, frequently need to swap pages in and out when handling large tasks. Because of the significant differences in storage speeds between hardware devices, frequent page swapping degrades I / O performance between memory and backend storage, impacting user experience. Physical memory is a scarce resource in embedded devices. Traditional Linux memory optimization schemes for terminal devices and embedded devices typically employ the SWAP mechanism and zRAM memory scheme. The built-in SAPP mechanism in the operating system is not suitable for the memory capacity of embedded devices. This swapping method can actually significantly impact the I / O performance of small embedded devices, increasing the communication cost between memory and backend storage. The SWAP mechanism utilizes the principle of virtual memory. Physical memory is the computer's actual memory, composed of RAM (Random Access Memory). Virtual memory is virtualized, using disk space to represent physical memory. The advent of virtual memory partially solves the problem of insufficient machine memory. When a program runs, the operating system handles the actual swapping and loading of virtual memory into physical memory. It uses a disk space or a local file as memory, involving two actions: swapping in and swapping out. Swapping out involves moving temporarily unused memory to disk and releasing the memory occupied by that data; swapping in occurs when a process accesses that memory again, reading data from disk into memory. Small embedded devices can function using traditional memory-to-disk swapping mechanisms, but their performance depends heavily on the size of their physical memory. However, embedded devices typically don't carry large amounts of physical memory, and the cost of their hardware resources also plays a role. Terminal devices generally use flash memory-based devices, such as Secure Digital Cards (SD cards) and Universal Flash Storage (UFS), for low-power, non-volatile data storage. Due to hardware cost and power consumption considerations, they are usually equipped with a small number of flash memory chips and very limited embedded RAM. Furthermore, flash memory has a severe durability problem. Flash memory cells have a limited number of programming / erase (P / E) operations; to extend the lifespan of flash memory, it is necessary to minimize I / O amplification.

[0003] Therefore, memory compression technology emerged. Simply put, memory compression involves allocating a portion of the system's physical memory as a zRAM partition when physical memory is insufficient. Infrequently used anonymous pages are then compressed and placed in the zRAM partition, essentially sacrificing some CPU efficiency to increase the available memory for currently running programs. When needed, the data is decompressed from the zRAM partition. The main principle is to allocate a region from memory as a swap partition. Whenever memory is insufficient, instead of killing the application, the memory data occupied by the application is copied to the swap partition. When switching back, this data can be directly restored to memory, saving the time required for restarting. The memory occupied by applications placed in the swap partition is compressed. For example, if WeChat occupies 50MB of space in normal memory, with a compression rate of 0.4, only 20MB of space is needed in the swap partition. This allows the swap partition to store more background applications that are not currently in use, effectively expanding the available memory. zRAM was added to the kernel's staging directory in 2009 and was officially merged into the Linux kernel mainline in version 3.14 released on March 30, 2014. Ubuntu began using zRAM in version 13.10. In December 2012, Ubuntu considered enabling zRAM by default for computers with limited memory. Google uses zRAM in Chrome OS, and it has also become an option for Android 4.4 and later devices.

[0004] While existing memory compression methods offer novel solutions to memory compression to some extent, they are all solutions proposed for specific problems. For memory-constrained embedded devices, traditional memory management (a simplified version of Linux memory management) is still used, which is not well adapted to the current mainstream ubiquitous operating system scenarios and cannot meet the storage requirements of ubiquitous applications. It directly builds table structures on the storage device, removes file abstraction, and involves cross-layer design between the database and file system. This inevitably leads to new problems in new scenarios. Native table storage systems emerged as a new storage method in response to the advent of ubiquitous operating system scenarios. However, for structured data storage under ubiquitous operating systems, there are problems such as severe I / O amplification, excessively large integrated system size, failure to fully integrate storage hardware characteristics, software stack redundancy, and low memory resource utilization of table storage systems in ubiquitous operating system scenarios. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a memory compression management system for table-based storage in embedded devices. It employs a partitioned hybrid swap layout method and uses an entropy sampling algorithm to quickly estimate the compression ratio of swapped-out pages. Based on the estimated compression ratio, reuse probability, and eviction cost, it determines whether a page should be accepted as a compressed page. The system includes an eviction page compression selector, partitioned memory regions, and auxiliary storage regions. Pages with low compression ratios and high performance advantages are selectively stored with minimal overhead. The partitioned hybrid swap layout method combines memory compression and memory swapping techniques. Its main idea is that frequently swapped-in / out pages use zRAM memory space, while infrequently swapped-in / out pages use auxiliary storage space. A duplicate page filter solves the problem of failing to identify redundant identical pages stored in the compressed swap area. Two newly added page tables, Swap Index Table and Swap Page Table, enable faster partitioning of swap memory, reducing the occurrence and compression probability of duplicate pages. A Swap page index is constructed to reduce the impact of swap on applications, ultimately achieving improved performance through memory management of the table-based storage system. This invention improves the memory utilization of table storage systems, further refines the management of embedded device memory space, and addresses key aspects such as selecting more useful pages, establishing appropriate indexes, reducing duplicate page selection, minimizing system power consumption caused by swap partitioning, reducing the impact of swap on applications, and further enhancing the storage performance of table storage systems. This reduces memory pressure on embedded devices. Building upon table storage systems in new scenarios, this invention improves memory utilization and memory space management, further enhancing system response time, reducing unnecessary I / O operations, and minimizing swapping between memory and flash storage, thereby effectively reducing wear and tear on flash-based storage. By reducing I / O loss and extending its duration, this memory compression management system significantly reduces the I / O required for system swapping, lowering the triggering of I / O bottlenecks. Combining the hardware characteristics of embedded devices with the characteristics of native table storage systems, this system achieves a high-efficiency, easy-to-use memory compression management system for table storage systems with small memory. It effectively solves the problem of low memory resource utilization of table storage systems in ubiquitous operating system scenarios caused by existing memory compression technologies, thereby improving the memory utilization and memory space management of table storage systems, further enhancing the storage performance and response time of table storage systems, and reducing unnecessary I / O operations. It has excellent application prospects and commercial value.

[0006] The specific technical solution for implementing this invention is: a memory compression management system for embedded device table storage. The system is characterized by consisting of an evicted page compression selector, a partitioned memory region, and an auxiliary storage region. It uses a hybrid swap layout method with partitioned design to achieve memory compression management for embedded device table storage. The evicted page compression selector, unlike most existing memory compression techniques which only consider compressing anonymous pages containing information recorded in the stack and heap, and blindly compress all pages evicted from main memory, stores the compressed pages (zpages) in the swap pages of the compressed swap pool in RAM. When a zpage is requested from main memory, it is decompressed and moved back to main memory, and the decompressed zpage is deleted from its swap pages. The evicted page compression selector considers not only anonymous pages but also file-mapped pages. File-mapped pages specifically refer to unmodified pages; modified pages become dirty pages and do not require compression because they will eventually be written back to the backend storage device. The eviction page compression selector uses an entropy sampling algorithm to quickly estimate the compression ratio, reuse probability, and eviction cost of the page to be eviction, determining whether to accept it into the compression swap pool (zpool). Acceptance is based solely on the expectation that the page will remain in the zpool will be equal to or equal to the average benefit of existing zpages in the zpool. The entropy sampling algorithm calculates the expected benefit of the target page and the average benefit of zpages in the zpool using equations (a) to (b) below, respectively.

[0007] ExpectedBenefit=Cost÷Compression Ratio (a);

[0008]

[0009] Here, ExpectedBenefit represents the expected benefit of the target page; Cost is the sum of the costs of discarding the target page and reloading it from storage in the future. A higher Cost value is more beneficial for keeping the target page in the zpool. Generally, random writes in flash storage are two to four times slower than random reads. The Cost for file-mapped pages is set to 1 because they require one read operation, and the Cost for anonymous pages is set to 4 because discarding anonymous pages requires both a read operation and an additional write operation. Compression Ratio is the expected compression ratio of the target page. When the page compression ratio distribution is wide, it is expected that more pages can be stored in the same capacity swap pool by selectively accommodating only pages with low compression ratios. Assuming the zpool stores an average of 200 zpages, half of which are swapped out with a compression ratio of 1.0 and the other half with a compression ratio of 0.5, if the swap pool only keeps pages with a compression ratio of 0.5, the number of zpages stored in the swap pool will increase to 300. Therefore, selectively accommodating highly compressed pages will increase the data density of the swap pool; Avg Cost and Avg Compression Ratio are the average eviction cost and compression ratio of zpages already stored in the zpool, respectively; Hit Rate is the hit rate of the most recent page load request, which can be calculated using a 256-bit map of the hit window to track the number of hits during the last 256 accesses; Capacity is the ratio of the number of used chunks in the zpool to the total number of chunks. A chunk represents the number of blocks in the zpool. Calculating this ratio provides feedback that when the zpool is less occupied, it reduces the average benefit of zpages in the zpool, thus encouraging more aggressive page acceptance into the zpool. (The last sentence is a repetition of the first and can be omitted.) This is to avoid meaningless compression and thus encourage more active acceptance of pages into the zpool; AverageBenefit is the average benefit of zpages in the zpool. Information entropy has a strong positive correlation with compression ratio. To save computational resources and avoid wasting time, and to avoid meaningless compression, the information entropy of the target page is used to estimate its compression ratio, because information entropy has a strong positive correlation with compression ratio; Pi is the probability that a randomly sampled byte has an integer value i. To suppress overhead, we only use the first 512 bytes of the page to approximate the estimate; AvgCompression Ratio is the compression ratio estimated using the information entropy of the target page; Pi Given the probability that a randomly sampled byte has an integer value i, to suppress overhead, only the first 512 bytes in the page are used to approximate its P. i Furthermore, it was verified that this approximation method can produce sufficiently accurate estimates, as the entropy sampling calculation method is much faster than compression and requires relatively fewer CPU cycles.

[0010] The hybrid swapping layout method of the partition design uses an SD card as the back-end storage device, which interacts with the embedded device. Partitioning is performed on the SD card as an auxiliary storage swapping space. The main idea behind the well-designed partitions and the introduction of the back-end storage device is that frequently swapped-in / out pages use zRAM memory space, while infrequently swapped-in / out pages use auxiliary storage space. This design references the principles of the original memory compression technologies zRAM and zSWAP, but it is not simply a single RAM partition. It combines memory compression and memory swapping technologies, and customizes a reasonable and efficient scheduling management strategy and interactive layout scheme to ensure that compression and swapping work efficiently and in balance. A resident process ZSWAPD is created in the kernel to swap out anonymous pages compressed in zRAM to the back-end storage partition, thereby completely freeing up a block of usable memory to maintain the Memavailable level. Simultaneously, the ZSWAPD module can also record the hot / cold characteristics of each anonymous page and store this data according to correlation and hot / cold order, so that the anonymous pages stored consecutively in the back-end swap area have temporal and spatial locality. Therefore, when swapping in anonymous pages, adjacent anonymous pages in the swap area can be read into zRAM together to ensure data access speed and improve I / O performance.

[0011] The duplicate page filter swaps out infrequent pages from the swap partition within the auxiliary storage area, moving infrequent pages into the zRAM space. Traditional memory compression techniques cannot identify redundant identical pages stored in their compressed swap area. Many identical anonymous pages often exist within a single process or even between different processes, possibly due to libraries being shared between different applications. Analysis shows that under the maximum swap memory usage, approximately 10-15% of swapped-out pages are identical. Therefore, this section introduces two new page tables: a Swap Index Table and a Swap Page Table, to achieve faster partitioning of swap memory, reduce the occurrence and compression of duplicate pages, build a swap page index, and reduce the impact of swap on applications. The index number (subscript) in the Swap Index Table represents the page size; pages of the same size exist in the same linked list. The Swap Index Table manages the location information of swapped-out pages in the swap page table, such as the index and offset of the swapped-out page. An index (sp_idx) and an offset (sp_offset) are required to access the swapped-out page. This index identifies an entry in the swap page table, and the index is also the size of the indicated swapped-out page. The offset is used to obtain the starting address of a specific swapped-out page in the swap page table entry; all swapped-out pages are stored in the Swap Page table. The position in the Swap Page table is indicated by the sp_idx and sp_offset fields in the Swap Index table. This invention incorporates the maintenance of these two page tables to record specific information about compressed swapped-out pages. The purpose of this is to find pages of the same size at the corresponding position based on the page size information when using same-page search, making comparison faster, without needing to compare all compressed swapped-out pages, thus saving comparison time.

[0012] Compared with existing technologies, this invention improves the memory utilization and memory space management of table storage systems in new scenarios, further enhances the storage performance and response time of table storage systems, reduces unnecessary I / O operations, and significantly reduces swapping between memory and flash storage. This effectively reduces wear and tear on flash-based storage and extends its lifespan, significantly reduces I / O that requires swapping, lowers the triggering of I / O bottlenecks, and effectively solves many problems of traditional memory compression technologies in considering special conditions, such as the selection of compressed pages and the ignoring of duplicate pages. Furthermore, traditional solutions do not consider the use of mixed interaction spaces, thus relieving memory pressure on small embedded devices to a greater extent. It has excellent application prospects and commercial value. Attached Figure Description

[0013] Figure 1 This is a framework diagram of the present invention;

[0014] Figure 2 Flowchart of the entropy sampling algorithm for the page compression selector;

[0015] Figure 3 A framework diagram for designing a hybrid switching layout for partitioning;

[0016] Figure 4 This is a framework diagram for the duplicate page filter. Detailed Implementation

[0017] This invention combines the hardware characteristics of embedded devices with the characteristics of native table storage systems. Addressing the issues of low memory utilization and I / O amplification caused by the limited memory space in table storage of small embedded devices, it introduces and optimizes memory compression technology. This results in a highly efficient, easy-to-use memory compression management system for table storage in embedded devices with limited memory. It effectively solves the problem of low memory resource utilization in table storage systems under ubiquitous operating system scenarios using existing memory compression technologies, thereby improving the memory utilization and memory space management of the table storage system, further enhancing its storage performance, improving system response time, and reducing unnecessary I / O operations. The memory compression management system for embedded device table storage includes: an eviction page compression selector, a partitioned hybrid swapping layout scheme, and a duplicate page filter. The eviction page compression selector avoids blindly compressing all evicted pages by using an entropy sampling algorithm to quickly estimate the compression ratio of the swapped-out pages. It determines whether to accept a page as a compressed page based on the estimated compression ratio, reuse probability, and eviction cost. This method can identify pages with low compression ratios and high performance advantages, selectively storing them with minimal overhead. The partitioned hybrid swapping layout scheme combines memory compression and memory swapping technologies. Its main idea is that frequently swapped-in / out pages use zRAM memory space, while infrequently swapped-in / out pages use auxiliary storage space. The duplicate page filter addresses the inability of traditional memory compression technologies to identify redundant identical pages stored in the compressed swapping area. This invention also incorporates two new page tables: a Swap Index Table and a Swap Page. Tables enable faster partitioning and swapping of memory, reducing the likelihood of duplicate pages and compression, building swap page indexes, and minimizing the impact of swap on applications. Ultimately, this achieves memory management for table storage systems to further improve performance.

[0018] The present invention will be further described in detail through the following specific embodiments.

[0019] Example 1

[0020] See Figure 1The present invention comprises three key components: an eviction page compression selector, a hybrid swap layout scheme with partition design, and a duplicate page filter.

[0021] See Figure 2 The eviction page compression selector uses an entropy sampling algorithm to quickly estimate the compression ratio, reuse probability, and eviction cost of the page to be swapped out to determine whether to accept it into the compression swap pool (zpool). Acceptance is based solely on the condition that the expected benefit of keeping the page in the zpool is greater than or equal to the average benefit of keeping an existing zpage in the zpool.

[0022] The entropy sampling algorithm calculates the expected return of the target page and the average return of zpages in the zpool using equations (a) and (b) below, respectively:

[0023] ExpectedBenefit=Cost÷Compression Ratio (a);

[0024]

[0025] Here, ExpectedBenefit represents the expected benefit of the target page; Cost is the sum of the costs of discarding the target page and reloading it from storage in the future. A higher Cost value is more beneficial for keeping the target page in the zpool. Generally, random writes in flash storage are two to four times slower than random reads. The Cost for file-mapped pages is set to 1 because they require one read operation, and the Cost for anonymous pages is set to 4 because discarding anonymous pages requires both a read operation and an additional write operation. Compression Ratio is the expected compression ratio of the target page. When the page compression ratio distribution is wide, it is expected that more pages can be stored in the same capacity swap pool by selectively accommodating only pages with low compression ratios. Assuming the zpool stores an average of 200 zpages, half of which are swapped out with a compression ratio of 1.0 and the other half with a compression ratio of 0.5, if the swap pool only keeps pages with a compression ratio of 0.5, the number of zpages stored in the swap pool will increase to 300. Therefore, selectively accommodating highly compressible pages will increase the data density of the swap pool; Avg Cost and Avg Compression Ratio are the average eviction cost and compression ratio of zpages already stored in the zpool, respectively; Hit Rate is the hit rate of the most recent page load request, which can be a 256-bit hit window. The `map` property allows us to track the number of hits during the last 256 accesses. `Capacity` is the ratio of used chunks to the total number of chunks in the zpool. A chunk represents the number of blocks in the zpool. Calculating this ratio provides feedback: when the zpool is less occupied, it reduces the average benefit of zpages in the zpool, thus encouraging more active page inclusion. `AverageBenefit` is the average benefit of zpages in the zpool. Information entropy has a strong positive correlation with compression ratio. To save computational resources and avoid wasting time, and to avoid meaningless compression, we use the information entropy of the page to roughly estimate its compression ratio. Pi is the probability that a randomly sampled byte has an integer value i. To suppress overhead, we only use the first 512 bytes of the page to approximate its probability. AvgCompression Ratio is the compression ratio estimated using the information entropy of the target page. P i The probability that a randomly sampled byte has an integer value i.

[0026] See Figure 3 The hybrid switching layout scheme of the partition design features a reasonably partitioned layout, introducing a back-end storage device. Partitioning is performed on the SD card as an auxiliary storage switching space for the embedded device. When multiple programs run simultaneously in the hybrid switching system, if a page to be referenced is not in main memory and has not been swapped to the auxiliary storage switching space or zRAM switching space, it will be loaded from the program repository on the auxiliary storage into main memory using the Fin arrow. Conversely, if the referenced page is swapped out (Sout arrow) to the auxiliary storage switching space, it will be swapped back into main memory using the Sin arrow. If the referenced page is swapped out to the zRAM switching space, it will be swapped into main memory using the Zin arrow. Finally, if there is no available free space in main memory (usually representing page frames), the longest unreferenced anonymous page in the auxiliary storage switching space or zRAM switching space will be swapped out using the Sout or Zout arrows, respectively. Finally, a resident process ZSWAPD is created in the kernel. The ZSWAPD I / O arrow points to swap the anonymous pages compressed by zRAM to the backend storage partition, thereby freeing up a block of available memory to maintain the available memory Memavailable level.

[0027] See Figure 4 The duplicate page filter swaps out infrequently used pages from the auxiliary storage area to the swap partition and into the zRAM space. The duplicate page filter incorporates two newly designed page tables, the Swap Index Table and the Swap Page Table, to record specific information about the compressed swapped-out pages. The index in the Swap Index table represents the page size; pages of the same size are stored in the same linked list. The Swap Index table manages the location information of swapped-out pages in the swap page table, such as the index and offset. An index (sp_idx) and an offset (sp_offset) are required to access the swapped-out page. The index identifies the entry in the swap page table and is also the size of the indicated swapped-out page. The offset is used to obtain the starting address of a specific swapped-out page in the swap page table entry. All swapped-out pages are stored in the Swap Page table. The location in the Swap Page table is represented by the sp_idx and sp_offset fields in the Swap Index table.

[0028] See Figure 4The duplicate page filter has four swapped-out pages: P1, P2, P3, and P4. These pages originally had different sizes, but because all pages are compressed, they are all stored in the Swap Page table. The position in the Swap Page table is indicated by the sp_idx and sp_offset fields in the Swap Index table, and the compressed page size is the index number in the Swap Index table. Therefore, P1 and P4 are the same size, both 2KB. Thus, the sp_idx and sp_offset values ​​of P1 are written to index number 2048 in the Swap Index page table, and the value of P4 is appended to the value of P1.

[0029] Similarly, the page sizes of P2 and P3 are also the index numbers of their respective Swap Index page tables. The information recorded in the Swap Index page table is the position index number of the Swap Page page table where the page is located, as well as its offset.

[0030] The steps for using the memory compression management system are as follows:

[0031] Step A-0: When the table storage system is started on an embedded device and memory utilization is extremely low, memory compression is initiated.

[0032] Step A-1: ​​All pages evicted from memory enter the evicted page compaction selector, which selects pages that meet the requirements according to the entropy sampling algorithm, compresses them, and puts them into the compaction pool zpool;

[0033] Step A-2: Once all the compressed pages in the zRAM partition are full, activate the hybrid swap layout scheme designed for the partition, add auxiliary storage space, and enable both compression and swapping to work efficiently and in a balanced manner.

[0034] Step A-3: Simultaneously start the duplicate page filter and start the Swap Index Table and Swap Page Table to record the specific information of the compressed swapped-out pages, reducing the chance of duplicate pages being compressed;

[0035] Step A-4: The memory compression system calls these management optimizations to complete the table storage system data operations.

[0036] The above is merely a further description of the present invention and is not intended to limit the scope of this patent. Any equivalent implementation of the present invention should be included within the scope of the claims of this patent.

Claims

1. A memory compression management system for table storage in embedded devices, characterized in that, The system consists of an eviction page compression selector and a partitioned memory region and auxiliary storage region. It uses a hybrid swap layout method with partitioned design to achieve memory compression management of the embedded device's table storage. The memory region includes available memory and a zRAM partition; the auxiliary storage region includes a duplicate page filter and a Swap partition. The eviction page compression selector evicts frequently accessed pages that satisfy the algorithm from the memory region, and these frequently accessed pages are compressed and moved into the zRAM partition. The duplicate page filter swaps infrequent pages from the auxiliary storage region out of the Swap partition and into the zRAM space. The hybrid switching layout of the partition design uses an SD card from the back-end storage device to interact with the embedded device. Partitioning is performed on the SD card as auxiliary storage and swap space. The arrows pointing to each partition must be consistent. When multiple programs run simultaneously in the hybrid switching system, if a page to be referenced is not in main memory and has not been swapped to the auxiliary storage or zRAM swap space, it will use F... in The arrow points to loading the program repository from secondary storage into main memory; if a referenced page is swapped out, S out If the arrow points to the secondary storage swap space, then via S... in The arrow points to swapping it into main memory; if the referenced page is swapped out to the zRAM swap space, then via Z... in The arrow points to swapping in main memory; if there is no free space available in main memory at this time, then swapping is performed via S. out Arrow pointing or Z out The arrows point to the anonymous pages that have not been referenced for the longest time in the secondary storage swap space or zRAM swap space, respectively; finally, a resident process ZSWAPD is created in the kernel. The ZSWAPD I / O arrows point to the anonymous pages compressed in zRAM and swap them to the backend storage partition, thus freeing up a block of usable memory. The eviction page compression selector uses an entropy sampling algorithm to estimate the compression ratio, reuse probability, and eviction cost of all eviction pages to determine whether to accept them into the compression exchange pool (zpool). Acceptance is based on the condition that the expected benefit of keeping the target page in the zpool is greater than or equal to the average benefit of keeping it in the zpool. All pages include anonymous pages and file-mapped pages. The entropy sampling algorithm calculates the expected benefit of the target page and the average benefit of zpages in the zpool using equations (a) to (b) below. ExpectedBenefit = Cost ÷ Compression Ratio (a); AverageBenefit = Capacity × (AvgCost × Hit Rate) ∕ AvgCompression Ratio (b); Where ExpectedBenefit is the expected benefit of the target page; Cost is the sum of the costs of abandoning the target page and reloading it from storage in the future; Avg Cost and Avg Compression Ratio are the average eviction cost and compression ratio of zpages stored in the zpool, respectively; Hit Rate is the hit rate of the most recent page load request; Capacity is the ratio of the number of used chunks to the total number of chunks in the zpool; AverageBenefit is the average benefit of zpages in the zpool; AvgCompression Ratio is the compression ratio estimated using the information entropy of the target page; and Pi is the probability that a randomly sampled byte has an integer value i.

2. The memory compression management system for table storage in embedded devices according to claim 1, characterized in that, The duplicate page filter incorporates two newly designed page tables, the Swap Index Table and the Swap Page Table, to record specific information about compressed swapped-out pages. The Swap Index Table manages the index and offset position information of swapped-out pages in the swap page table. The index identifies the entry in the swap page table and indicates the size of the swapped-out page; the index number represents the page size, and pages with the same size are stored in the same linked list. The offset is used to obtain the starting address of a specific swapped-out page in the swap page table entry. The position in the Swap Page Table is represented by the sp_idx and sp_offset fields in the Swap Index Table, and all swapped-out pages are stored in the Swap Page Table.

3. The memory compression management system for table storage of embedded devices according to claim 1, characterized in that, When the embedded device enables table storage system operation, when the memory storage volume reaches a set threshold, the memory compression management system is activated. This system uses a page eviction and compression selector to select pages that meet the requirements based on an entropy sampling algorithm. These pages are then compressed and added to the compression pool (zpool) in the zRAM partition. Once the zRAM partition is full of compressed pages, a hybrid swapping layout method designed for the partition is activated. When multiple large tasks or programs are running in the embedded device's table storage system, they are added to auxiliary storage space. Simultaneously, a duplicate page filter is activated, using two page tables to record the specific information of the compressed swapped-out pages, thus achieving memory compression management of the embedded device's table storage.

Images