Mobile device memory optimization method based on hot file page exchange and compression prediction

Abstract

The invention relates to the field of memory optimization, in particular to a mobile device memory optimization method based on hot file page exchange and compression prediction. Enabling the page recovery process to support the migration of the file page between the DRAM and the compressed memory space; judging whether a page which is switched out from the FLASH and is to be sent into the compressed memory space is an anonymous page or a hot file page, if so, transmitting the page to the compressed memory space, and otherwise, discarding or writing back the page to the FLASH; for the file page, dynamically predicting the compressibility of the page transmitted to the compressed memory space by using the sliding window; for an anonymous page, the compressibility of the page is preliminarily screened through the page average value and the page byte value variance, and then the compressibility of the whole page is evaluated through a compression result of sampling prediction; and compressing and storing the page of which the prediction result is compressible. According to the method, higher benefits can be obtained, and unnecessary compression overhead brought by compression of the memory space to some incompressible pages is reduced.

Description

Technical Field

[0001] The present invention relates to the field of memory optimization, and in particular to a mobile device memory optimization method based on hot file page exchange and compression prediction. Background Art

[0002] Sufficient memory is one of the important prerequisites to ensure smooth system operation. When the system does not have enough available memory for allocation, it usually relies on the kswapd (periodic memory recycling mechanism) thread to perform memory cleanup to release system resources. In this process, compressed memory space is often used as a swap partition to improve memory utilization efficiency.

[0003] The compressed memory space usually includes ZRAM and ZSWAP. Its working process is as follows: First, the compression module performs compression operations on the incoming pages. Generally, the page compression value μ = compressed page size / original page size is used to define the compressibility of the page. If the compressed μ value exceeds the preset threshold μ T , then the page is determined to be incompressible, so the page is directly copied to the compressed memory space storage module. Conversely, if the compressed μ value is less than the set threshold μ T , it is considered as a compressible page and its compressed version is moved to the compressed memory space storage module. The main function of the compressed memory space storage module is to store compressed pages, or directly save those pages that cannot be fully compressed.

[0004] Compressing memory space is a processing method that aims to minimize memory usage and improve memory utilization. It dynamically determines whether a page is compressible and determines whether to compress it based on a threshold.

[0005] However, this method will only benefit performance when the data is compressible. If the swapped pages are all incompressible, not only will it not save memory, but it will also cause unnecessary overhead due to compressing these incompressible pages, resulting in performance loss.

[0006] Currently, compressed memory space technology is widely used in many Android mobile devices to effectively expand memory capacity, but there are the following problems:

[0007] 1. Due to the design of the kernel code, the compressed memory space on most Android mobile devices currently only supports the storage of anonymous pages, but not file pages. The original intention of this design is that anonymous pages are relatively easy to compress and have a low compression rate, making them more suitable for storage in compressed memory space. However, the speed of reading file pages from FLASH is slow, which may affect the user experience. Moreover, when the compressed memory space is still large after storing anonymous pages, it will lead to significant space waste and performance loss.

[0008] 2. Compressed memory space cannot determine the compressibility of pages, and it will perform indiscriminate compression operations on the pages that enter. If most of the pages that enter the compressed memory space are incompressible, not only will it not save memory, but it will also bring unnecessary overhead because of compressing these incompressible pages.

[0009] 3. After using the phone fully, it was found that under some Android phone configurations (when the ratio of DRAM to ZRAM size exceeds 50%), the compressed memory space is not fully utilized, resulting in a huge waste of space.

[0010] In response to problem 1, a study proposed the FLASH swap space technology, which swaps out infrequently used pages to the FLASH swap space and swaps them back in when they are used. However, this technology has problems such as slow I / O speed, FLASH write wear, and I / O bandwidth contention.

[0011] In response to question 2, a study proposed a two-level compression prediction interface, which uses the average byte value of the page and page sampling to predict the compressibility of the page. However, when deploying this interface to an Android test machine, it was found that it had a large performance overhead, which indicates that this interface may have some problems in practical applications and needs further research and improvement to improve its performance. Summary of the invention

[0012] In view of the above problems, the present invention proposes a memory optimization method for Android mobile devices. The method supports sending anonymous pages and high-frequency file pages into compressed memory space at the same time, aiming to improve the utilization rate of compressed memory space, increase memory data density, and improve system performance.

[0013] In order to achieve the above-mentioned purpose, the mobile device memory optimization method based on hot file page exchange and compression prediction in the present invention comprises the following steps:

[0014] Step 1: Enable the page recycling process to support the migration of file pages between DRAM and compressed memory space;

[0015] Step 2: Determine whether the page to be swapped out from FLASH and sent to the compressed memory space is an anonymous page or a hot file page. If so, transfer it to the compressed memory space; otherwise, the page will be discarded or written back to FLASH.

[0016] Step 3: For the file page, the compressibility of the page transferred to the compressed memory space is dynamically predicted using the sliding window;

[0017] For anonymous pages, the compressibility of the pages is preliminarily screened by the average page size and the page byte value variance, and then the compression results of the sample predictions are used to evaluate the compressibility of the entire page;

[0018] Step 4: compress and save the pages whose prediction results are compressible and are transferred to the compression memory space.

[0019] Furthermore, in step 1, the modification of the file page swapping process includes: when the memory is insufficient, the file page is preferentially swapped out to the compressed memory space;

[0020] The modifications to the file page swap-in process include: when a page fault occurs, the missing file page is searched for in the compressed memory space first.

[0021] Furthermore, the process of determining whether the file page is a hot file page in step 2 specifically includes:

[0022] For the file page to be sent to the compressed memory space, if the page is a dirty file page, it is written back to FLASH. If it is a clean file page, its access frequency is determined, including:

[0023] Step 2.1 first determines whether the page access frequency is greater than or equal to the first threshold. If the condition is met, proceed to the next step; if not, discard the page;

[0024] Step 2.2: Determine whether the page access frequency is greater than or equal to a second threshold. If the condition is met, it is determined to be a hot page. If not, proceed to step 2.3.

[0025] Step 2.3: Determine whether this page enters the compressed memory space for the first time. If the conditions are met, it is determined to be a hot page, and a flag indicating that the page is not swapped out for the first time is set; if not, the page is discarded.

[0026] Furthermore, in step 3, the compressibility prediction of the file page includes:

[0027] Step 3.1 Set the initial value of the sliding window m and retain the information on whether the first n historical pages are compressible;

[0028] For pages to be sent to the compressed memory space, if both of the following conditions are met:

[0029] ①The first n history pages cannot be compressed.

[0030] ②This page is continuous with the last history page.

[0031] ③The sliding window m value is greater than 0,

[0032] If the page is not compressible, the value of m is reduced by 1, the information of the history page is updated and the system exits.

[0033] Otherwise, go to step 3.2;

[0034] Step 3.2: Compress the page. If the page remains continuous, reset the sliding window value to m and update the historical page information; otherwise, go to step 3.3;

[0035] Step 3.3: If the page is not compressible, update the historical page information and exit. If the page is compressible, exit directly.

[0036] Furthermore, in step 3, the compressibility prediction for the anonymous page includes:

[0037] First level: based on the average byte value of some pages, the judgment is made: if the average byte value is less than a certain low average byte value or greater than a certain high average byte value, it is directly determined to be a compressible page. Pages that are not in these two areas enter the second level prediction. If the page is predicted to be compressible, it exits directly;

[0038] Second level: Compression prediction based on page byte value variance: If the byte value variance is less than a certain low byte value variance or greater than a certain high byte value variance, it is directly determined as a compressible page. Pages that are not in these two areas enter the third level prediction. If it is predicted that the page can be compressed, it will exit directly;

[0039] Level 3: Sample-based sampling prediction: Take the first part of the page as a sample and perform compression prediction on the sample to reflect the compressibility of the complete page.

[0040] The present invention modifies the page recycling process and introduces hot file page identification to screen out hot pages, which are more likely to be accessed again. Later, two page compressibility predictions are introduced, one is a page compressibility prediction module based on time locality, and the other is a page compressibility prediction module based on space locality. By supporting the swapping out of clean file pages with a higher access frequency to the compressed memory space swap space, the utilization rate of the compressed memory space is improved and I / O scheduling is reduced. Since hot pages are more likely to be accessed again, retaining them in the compressed memory space can obtain higher benefits. Through compressed row prediction, the compressibility of the page can be judged quickly and accurately, reducing the unnecessary compression overhead of the compressed memory space on some incompressible pages. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a mobile device memory optimization method for hot file page swapping and compression prediction in a specific implementation example of the present invention;

[0042] Figure 2 It is a working flow diagram of a hot file page identification module in a specific implementation example of the present invention;

[0043] Figure 3This is a working diagram of a page compressibility prediction module based on time locality in a specific implementation example of the present invention;

[0044] Figure 4 This is a working diagram of a page compressibility prediction module based on spatial locality in a specific implementation example of the present invention;

[0045] Figure 5 This is a diagram showing the accuracy and performance improvement of a time-based page compressibility prediction module in a specific implementation example of the present invention;

[0046] Figure 6 This is a comparison chart of the accuracy of a space-based page compressibility prediction module and other working secondary compression prediction modules in a specific implementation example of the present invention;

[0047] Figure 7 This is a comparison diagram of the interface call time between a space-based page compressibility prediction module and other working secondary compression prediction modules in a specific implementation example of the present invention;

[0048] Figure 8 This is a diagram showing the effect of improving the hot start speed of an application in a specific implementation example of the present invention. DETAILED DESCRIPTION

[0049] The specific implementation manner and working principle of the present invention are further described in detail below with reference to the accompanying drawings.

[0050] For the compression prediction algorithm of the present invention, there are four possible combinations of prediction and actual compression. If y is compressible and n is incompressible, then the possible cases are yy, yn, ny and nn. For example, yn means that a page is predicted to be compressible, but in fact it is incompressible.

[0051] Prediction overhead is added to all four cases, but affects each case differently. First, the yy and yn cases simply have additional prediction overhead because the actual compression is run after the prediction. Although these two cases do not gain any benefit from prediction, this is not too important as long as prediction is fast enough than compression. Second, this case not only adds prediction overhead, but also increases memory usage. This is the worst case that needs to be avoided as much as possible. Finally, the nn case improves performance by removing unnecessary compression resources, which is the most ideal case.

[0052] Assume the total number of pages is defined as:

[0053] N all =N yy +N yn +N ny +N nn

[0054] The accuracy of the prediction is:

[0055]

[0056] Where N yy 、N yn 、N ny and N nn Indicates the number of pages for each case.

[0057] In addition, the total compression time savings and memory loss can be summarized as:

[0058]

[0059]

[0060] Where Comp(X) and Pred(X) are the actual compression and predicted computational overhead of page X, respectively. OrigSZ(X) and CompSZ(X) are the original size and compressed size of page X, respectively.

[0061] Please refer to Figure 1 , Figure 1 The flowchart of a mobile device memory optimization method for hot file page swapping and compression prediction is shown below. The specific process includes:

[0062] Step 1: Modify the kernel code and change the priority of file page swapping, from the original priority of swapping out file pages to FLASH to the priority of swapping out file pages to compressed memory space;

[0063] Step 2: Start the hot file page identification module;

[0064] Step 3: Start the page compressibility prediction module based on time locality;

[0065] Step 4: Start the page compressibility prediction module based on spatial locality;

[0066] Specifically, when a page is swapped out, it is first judged according to the page attributes. If the page is an anonymous page, the page is sent to the compressed memory space B for compression prediction; if the page is a dirty file page, it is written back to FLASH; if it is a clean file page, it is sent to the compressed memory space A to judge the hotness of the page. The goal of the present invention is to support anonymous pages and some qualified file pages to be sent to the compressed memory space, and to minimize the system delay caused by compressed pages through the compression prediction module, thereby improving the data utilization rate of the compressed memory space, increasing the memory data density, reducing I / O scheduling, and ultimately improving system performance.

[0067] Figure 2The following is the workflow diagram of the hot file page identification module. The working steps are as follows:

[0068] Step 1: For the file page that is about to enter the compressed memory space, the module first determines whether the page access frequency is greater than or equal to the first threshold. If the condition is met, it proceeds to the next step; if not, the page is discarded and the module exits.

[0069] Step 2: Determine whether the page access frequency is greater than or equal to a second threshold. If the condition is met, it is determined to be a hot page and the module is exited; if not, proceed to step 3.

[0070] Step 3: Determine whether this page enters the compressed memory space for the first time. If the conditions are met, it is determined to be a hot page, and the page is set to be swapped out for the first time, and the module is exited; if not, the page is discarded and the module is exited.

[0071] In steps 3 and 4, the page passes through the page compressibility prediction module. If the page is predicted to be incompressible, the page compression module is skipped and the page is directly copied to the page storage module; if the page is predicted to be compressible, the page enters the page compression module to compress the page.

[0072] Figure 3 The following is a workflow diagram of the page compressibility prediction module based on temporal locality. The working steps are as follows:

[0073] Step 1: For the page compressibility prediction module, first access the storage of n historical page information and the size of the current sliding window m value. If ① the first n historical pages are not compressible, ② this page is continuous with the last historical page, and ③ the sliding window m value is greater than 0, then the page is determined to be incompressible, the m value is reduced by 1, the historical page information is updated, and the module is exited; otherwise, enter step 2.

[0074] Step 2: Compress the page. If the page remains continuous, reset the sliding window value to m and update the historical page information; otherwise, go to step 3.

[0075] Step 3: If the page is not compressible, update the historical page information and exit the module; if the page is compressible, exit the module.

[0076] Figure 4 The following is a workflow diagram of the page compressibility prediction module based on spatial locality. The working steps are as follows:

[0077] Step 1: For the page compressibility prediction module, first perform compression prediction on the average byte value of the page: This prediction method is mainly based on the average byte value size of some pages. If the average byte value is less than a low average byte value or greater than a high average byte value, the page within this range is directly determined to be a compressible page. Pages not in these two areas proceed to the next step. If it is predicted that the page is compressible, exit the module directly.

[0078] Step 2: Compression prediction based on page byte value variance: This prediction method is mainly based on the byte value variance of some pages. If the byte value variance is less than a low byte value variance or greater than a high byte value variance, the pages within this range are directly judged as compressible pages. Pages not in these two areas enter the next step method. If it is predicted that the page can be compressed, the module is directly exited.

[0079] Step 3: Sample-based sampling prediction: Partial sampling of a page can reflect the compressibility of the entire page to a certain extent, that is, taking the first part of the page as a sample and performing compression prediction on the sample to reflect the compressibility of the complete page.

[0080] The page compressibility prediction module is designed to quickly and accurately determine the compressibility of a page, effectively reduce the unnecessary compression overhead of the compression module on some incompressible pages, and reduce the system latency caused by supporting file pages to be sent to the compressed swap memory.

[0081] Figure 5 The figure shows the accuracy and performance improvement of the time-based page compressibility prediction module. Due to the principle of temporal locality, the consecutive file pages that are recycled have similar compressibility. By simulating user behavior, the time-based page compressibility prediction module is deployed on the 940,000 recycled file pages. n = 2, m = 3, and the page compression threshold μ is set. T Set to 0.75, the module effect is as follows Figure 5 As shown in the figure, the accuracy of predicting incompressible pages is as high as 85.8%, and the reduction of invalid compression ratio (number of correctly predicted incompressible pages / total number of incompressible pages) reaches 47.8%.

[0082] Figure 6 Shown is a comparison chart of the accuracy of using a space-based page compressibility prediction module (three-level prediction) and a two-level compression prediction module (two-level prediction) of other works.

[0083] Figure 7The comparison chart of the interface call time between the space-based page compressibility prediction module and the two-level compression prediction module of other works shows that the three-level prediction proposed in this example can save 79.6% of the interface call time. It is concluded that the space-based page compressibility prediction module proposed in this study can greatly reduce the interface call time without losing the prediction accuracy.

[0084] Figure 8 The following figure shows the improvement of the hot start speed of 7 applications on mobile devices when the above optimization method is used and not used. The test method is to start 11 commonly used apps in the background, of which 4 apps are background processes that occupy memory, and 7 apps are manually switched and the hot start time is recorded. This is repeated 10 times, and the median is taken as the final data for comparison. It is observed that the hot start speed of some applications (such as gallery and calculator) has increased by about 15%. The average application hot start speed has increased by 11.7%.

[0085] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by the present invention shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A mobile device memory optimization method based on hot file page swapping and compression prediction, characterized in that: The following steps are involved: Step 1: Enable the page recycling process to support the migration of file pages between DRAM and compressed memory space; Step 2: Determine whether the page to be swapped out from FLASH and sent to the compressed memory space is an anonymous page or a hot file page. If so, transfer it to the compressed memory space; otherwise, the page will be discarded or written back to FLASH. Step 3: For the file page, the compressibility of the page transferred to the compressed memory space is dynamically predicted using the sliding window; For anonymous pages, the compressibility of the pages is preliminarily screened by the average page size and the page byte value variance, and then the compression results of the sample predictions are used to evaluate the compressibility of the entire page; Step 4: compress and save the pages whose prediction results are compressible and are transferred to the compression memory space.

2. The method according to claim 1, characterized in that: In step 1, the modification of the file page swapping process includes: when the memory is insufficient, the file page is swapped out to the compressed memory space first; The modifications to the file page swap-in process include: when a page fault occurs, the missing file page is searched for in the compressed memory space first.

3. The method according to claim 2, characterized in that The process of determining whether the file page is a hot file page in step 2 specifically includes: For the file page to be sent to the compressed memory space, if the page is a dirty file page, it is written back to FLASH. If it is a clean file page, its access frequency is determined, including: Step 2.1 first determines whether the page access frequency is greater than or equal to the first threshold. If the condition is met, proceed to the next step; if not, discard the page; Step 2.2: Determine whether the page access frequency is greater than or equal to a second threshold. If the condition is met, it is determined to be a hot page. If not, proceed to step 2.

3. Step 2.3: Determine whether this page enters the compressed memory space for the first time. If the conditions are met, it is determined to be a hot page, and a flag indicating that the page is not swapped out for the first time is set; if not, the page is discarded.

4. The method according to claim 3, characterized in that In step 3, the compressibility prediction of the file page includes: Step 3.1 Set the initial value of the sliding window m and retain the information on whether the first n historical pages are compressible; For pages to be sent to the compressed memory space, if both of the following conditions are met: ①The first n history pages cannot be compressed. ②This page is continuous with the last history page. ③The sliding window m value is greater than 0, If the page is not compressible, the value of m is reduced by 1, the information of the history page is updated and the system exits. Otherwise, go to step 3.2; Step 3.2: Compress the page. If the page remains continuous, reset the sliding window value to m and update the historical page information; otherwise, go to step 3.3; Step 3.3: If the page is not compressible, update the historical page information and exit. If the page is compressible, exit directly.

5. The method according to claim 3, characterized in that: In step 3, the compressibility prediction for anonymous pages includes: First level: judge based on the average byte value of some pages: if the average byte value is less than a certain low average byte value or greater than a certain high average byte value, it is directly judged as a compressible page. Pages that are not in these two areas enter the second level prediction. If it is predicted that the page can be compressed, it will exit directly; Second level: Compression prediction based on page byte value variance: If the byte value variance is less than a certain low byte value variance or greater than a certain high byte value variance, it is directly determined as a compressible page. Pages that are not in these two areas enter the third level prediction. If it is predicted that the page can be compressed, it will exit directly; Level 3: Sample-based sampling prediction: Take the first part of the page as a sample and perform compression prediction on the sample to reflect the compressibility of the complete page.

Images