Database page replacement method capable of adaptively adjusting weight
By introducing a page replacement method with adaptively adjusting weights in the database, the problem of poor performance of database page replacement algorithms under high concurrency and skewed workloads is solved, achieving higher throughput and hit rate, adaptability and memory efficiency.
Patent Information
- Application Number
- CN202411005185.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-25
AI Technical Summary
Existing database page replacement algorithms perform poorly under high concurrency and skewed workloads, resulting in reduced throughput and low hit rate, especially in Zipfian distributed data access mode, where existing algorithms cannot effectively handle this unevenness. Data access features.
The page replacement method of adaptively adjusting weights is adopted. By introducing page values into the database, the page priority is indicated, and atomic operations are used to avoid concurrent locks, dynamically update the weights to improve hit rate, adapt to high concurrent scenarios, and reduce memory space consumption.
Improves the throughput and hit rate of the database, enhances adaptability to skewed workloads, reduces memory space consumption, and improves the overall performance of the database.
Smart Images

Figure CN118939670A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of databases, and in particular to a database page replacement method for adaptively adjusting weights. Background Art
[0002] Database is an indispensable cornerstone of modern information system. It is responsible for storing, managing and retrieving large amounts of data and providing data support for various applications. It organizes, stores and manages data according to data structure to ensure data security, integrity and reliability. Database not only has a huge storage space and can store data from millions to hundreds of millions, but also its structured data tables are interrelated and can reflect the essential connection between objective things. Database technology has gone through multiple stages of development from hierarchical database, network database to relational database, especially relational database has become the most important one in current database products. However, no matter how database technology develops, a common challenge it faces is how to efficiently store and access data in limited memory resources. At this time, the database page replacement algorithm becomes particularly important.
[0003] The database page replacement algorithm is a key technology in computer storage management, which is mainly used to solve the problem of insufficient memory during process operation. When there are no free pages in the memory, the page replacement algorithm will select a page for replacement to make room for loading the required page.
[0004] Common traditional page replacement algorithms include the First In First Out (FIFO) algorithm, the Clock Replacement (CLOCK) algorithm, the Least Recently Used (LRU) algorithm, etc. These algorithms have their own characteristics. For example, the FIFO algorithm selects the oldest page for replacement, while the LRU algorithm selects the page that has not been accessed for the longest time for elimination. The quality of the page replacement algorithm directly affects the performance of the system. A good page replacement algorithm should be able to reduce the frequency of page replacement and improve system efficiency. Therefore, in the memory management of databases and operating systems, it is very important to choose a suitable page replacement algorithm. These traditional replacement algorithms are rarely directly applied in database systems due to various limitations. Existing database systems often use more advanced replacement algorithms.
[0005] However, the most advanced page replacement algorithms have certain limitations. The LRU-K algorithm requires the use of an LRU linked list, which is affected by concurrent locks and leads to a decrease in throughput under high concurrency conditions (Elizabeth J. O'Neil, Patrick E. O'Neil, and Gerhard Weikum. 1993. The LRU-K Page Replacement Algorithm for Database Disk Buffering. In SIGMOD. 297–306.); the ARC algorithm uses multiple queues, consumes a lot of memory space, and is also affected by concurrent locks (Nimrod Megiddo and Dharmendra S. Modha. 2003. ARC: A Self-Tuning, Low Overhead Replacement Cache. In FAST.); the LIRS algorithm will cause high memory space consumption in extreme cases, and the parameters need to be carefully selected and configured (Demian and Viktor Leis. Write-Aware Timestamp Tracking: Effective and Efficient Page Replacement for Modern Hardware. PVLDB, 16(11):3323-3334, 2023.).
[0006] In addition, the specific data access pattern should also be considered when considering the specific design of the algorithm. In scenarios such as data processing, storage, and access, there is a special data access pattern, and databases also have this access pattern. This pattern is characterized by a small number of data items (or "hot data") being frequently accessed, while most data items are relatively rarely accessed. This distribution pattern is similar to the Zipfian distribution, which is often used to describe the data access characteristics of certain natural and artificial systems (SciPy.2023.SciPy Documentation for Zipfian.Retrieved July 10,2023from https: / / docs.scipy.org / doc / scipy / reference / generated / scipy.stats.zipfian.html). Specifically, a skewed workload similar to the Zipfian distribution has the following characteristics. In a skewed workload, the distribution of data access is uneven. A small number of data items (such as 20%) will be frequently accessed (such as 80% of the access volume), while most data items are rarely accessed. This access pattern is similar to the word frequency distribution in Zipfian distribution, where only a few words (high-frequency words) are frequently used.
[0007] In order to handle skewed workloads like Zipfian distribution, page replacement algorithms need to deal with them accordingly. However, existing page replacement strategies do not explicitly handle this situation, resulting in poor hit rate performance of existing algorithms under skewed workloads. Summary of the invention
[0008] The purpose of the present invention is to provide a database page replacement method with adaptive weight adjustment in view of the deficiencies in the prior art.
[0009] The object of the present invention is achieved through the following technical solution: a database page replacement method for adaptively adjusting weights, comprising the following steps:
[0010] 1. When the database is started, the database initializes N free data page slots in the buffer to store specific data pages; each data page slot is assigned a corresponding page slot ID and page value V, where ID∈{0, N-1}, and V is initialized to 0; before running the replacement algorithm, the following parameters are defined and initialized in the database: initialize the atomic variable pointer P to 0, the weight W to 2, the action A to 1, the replacement number R to 0, the hit number H to 0, and the hit rate HR to 0;
[0011] 2. When the service layer of the database accesses the target data page in the buffer, if a page slot in the buffer contains the target data page, the page value V of the data page slot is increased by W, the hit number H is increased by 1, and the data page slot ID is obtained, and the subsequent steps are skipped to step 4; if there is no target data page in the buffer, the target data page is imported from the disk into the buffer;
[0012] 3. When importing the target data page into the buffer, if there is an idle data page slot in the buffer, import the target data page into the idle data page slot and obtain the data page slot ID; if there is no idle data page slot in the buffer, find a non-idle data page slot ID, import the target data page into the non-idle data page slot, and obtain the data page slot ID;
[0013] 4. According to the obtained data page slot ID, read the data page in the page slot and return the data to the service layer.
[0014] Furthermore, in step 3, a non-free data page slot ID is found and a page replacement method with adaptive weight adjustment is adopted, and the method is specifically as follows:
[0015] 3.1. Use atomic operations to obtain the value of pointer P: If the value of P is N, set P to 0; if the value of P is between 0 and N-1, return the value of P; using atomic operations can avoid the use of concurrent locks, thereby improving the concurrency of this method in high concurrency situations and improving the throughput of the database;
[0016] 3.2. Read the page value Vp of the corresponding page slot ID according to the value of P;
[0017] 3.3. Determine the page value Vp: If the page value Vp is 0, obtain the page slot ID and proceed to step 3.4; if the page value Vp is greater than 0, reduce the page value Vp of the page slot by 1, increase P by 1, and return to step 3.1;
[0018] 3.4. Increase the replacement number R by 1 and determine the replacement number R: if the replacement number R is greater than or equal to N, proceed to step 3.5; if the replacement number R is less than N, skip the subsequent steps and proceed to step 3.8;
[0019] 3.5. Calculate the temporary hit rate HRp in this time period: The temporary hit rate calculation formula is: HRp = H / (H+R); set H to 0 and R to 0; use a replacement number R as a period to periodically calculate the hit rate in a period of time, which is used to count the hit rate in the past period of time;
[0020] 3.6. Update weight W: If the hit rate HRp is less than HR, change action A to the opposite number and add A to W; if the hit rate HRp is greater than HR, keep action A unchanged and add A to W; use the hit rate during this period to compare with the last hit rate. If the hit rate this time is greater than the last hit rate, it means that action A during this period is beneficial to the replacement method, so take the same action as last time to change weight W; if the hit rate is less than the last hit rate, it means that action A during this period is harmful to the replacement method, so take the opposite action as last time to change weight W; by dynamically changing weight W in this way, the hit rate of the database can be made as high as possible, thereby increasing the hit rate of the buffer;
[0021] 3.7. Update hit rate HR: set HR to HRp;
[0022] 3.8. Read the page slot ID and read the target data page from the disk to the page slot.
[0023] The beneficial effects of the present invention are as follows: by introducing a page value to represent the priority of a page, the page priority can be updated quickly and efficiently without using a specific data structure and a concurrent lock of the data structure to adapt to read requests in a high-concurrency scenario, thereby improving the throughput of the database; using a single page value to represent the priority of a page greatly reduces the consumption of memory space; using a method of adaptively adjusting weights can make it easier to retain hot data in memory, increase adaptability to tilted workloads, thereby increasing the hit rate of the database buffer and increasing the throughput of the database. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 It is a flow chart of the database service layer of the present invention reading the data page of the database buffer;
[0025] Figure 2 It is a flow chart of the database page replacement method for adaptively adjusting weights of the present invention. DETAILED DESCRIPTION
[0026] The present invention will be described in detail below with reference to the accompanying drawings.
[0027] like Figure 1 As shown, a database page replacement method for adaptively adjusting weights includes the following steps:
[0028] Step 1: When the database is started, all necessary variables are initialized; the database will initialize N free data page slots in the buffer to store specific data pages; each data page slot is assigned a corresponding page slot ID (from 0 to N-1) and a page value V (initialized to 0); the pointer variable P is initialized to 0, the weight variable W is 2, the action variable A is 1, the replacement number variable R of the buffer within a period of time is 0, the hit number variable H of the buffer within a period of time is 0, and the hit rate variable HR of the buffer within a period of time is 0. These variables can be saved by defining a class object and using integer or floating point types to save these data.
[0029] Step 2: If Figure 1 As shown in the figure, after entering the initial process, when the database service layer initiates a request to read the target data page to the buffer, the buffer starts to check whether the target data page is in the buffer. If a page slot in the buffer contains the target data page, the page value V of the data page slot is increased by W, the hit number H is increased by 1, and the data page slot ID is obtained, skipping the subsequent steps and entering step 4, that is, reaching step 5. Figure 1 Find the ID of the data page in the buffer; if there is no target data page in the buffer, you need to import the target data page from the disk into the buffer and enter step three, that is, reach Figure 1 In the query to see if there is a free data page slot. Here, adding W to V can use atomic operations and must be thread-safe.
[0030] Step 3: When the target data page needs to be imported into the buffer, the buffer starts to check whether there is a free page slot, corresponding to Figure 1 In the step 4, the target data page is imported into the free data page slot, and the data page slot ID is obtained, and the subsequent steps are skipped to enter step 4; Regarding how to determine whether there are free data page slots, the prior art provides many solutions, for example, these page slots can be logically saved in the FreeList, which is a common operation in the database.
[0031] When there is no free data page slot in the buffer, a page replacement method with adaptive weight adjustment is used to find a non-free data page slot ID, and the target data page is imported into the non-free data page slot, and the data page slot ID is obtained.
[0032] like Figure 2 As shown, the page replacement method for adaptively adjusting weights is divided into the following sub-steps:
[0033] 3.1. Use atomic operations to obtain the value of pointer P, that is Figure 2 Get the value of P in Figure 2In the above code, we determine whether the value of P is greater than N: if the value of P is N, P is set to 0; if the value of P is between 0 and N-1, the value of P is returned. Using atomic operations here can avoid the use of concurrent locks, thereby improving the concurrency of the method in high concurrency situations and improving the throughput of the database.
[0034] 3.2. Read the page value Vp of the corresponding page slot ID according to the value of P, that is, read the page value Vp of the page slot ID equal to P.
[0035] 3.3, determine the page value Vp, that is Figure 2 Determine whether Vp is 0: If the page value Vp is 0, obtain the page slot ID and go to step 3.4; if the page value Vp is greater than 0, reduce the page value Vp of the page slot by 1, increase P by 1, and return to step 3.1.
[0036] 3.4. Increase the replacement number R by 1 and determine the replacement number R: if the replacement number R is greater than or equal to N, proceed to step 3.5; if the replacement number R is less than N, skip the subsequent steps and proceed to step 3.8.
[0037] 3.5. Calculate the hit rate HRp: The formula for calculating the hit rate is: HRp = H / (H+R). Set H to 0 and R to 0. Here, the hit rate within a period of time is periodically calculated with a replacement number R as a period, which is used to count the hit rates within a period of time in the past. There are many calculation methods for the calculation of the hit rate in the industry. Using the statistical hit number and replacement number to calculate is simple and easy to operate.
[0038] 3.6. Update weight W: If the hit rate HRp is less than HR, change action A to the opposite number and add A to W; if the hit rate HRp is greater than HR, keep action A unchanged and add A to W. Use the hit rate during this period to compare with the last hit rate. If the hit rate this time is greater than the last hit rate, it means that action A during this period is beneficial to the replacement method, so take the same action as last time to change the weight W; if the hit rate is less than the last hit rate, it means that action A during this period is harmful to the replacement method, so take the opposite action of last time to change the weight W. By dynamically changing the weight W in this way, the hit rate of the database can be made as high as possible, thereby increasing the hit rate of the buffer.
[0039] 3.7. Update hit rate HR: Set HR to HRp, that is, reach Figure 2 Let HR=HRp.
[0040] 3.8. Read the page slot ID and arrive at Figure 2 The last step of the process, when it is finished Figure 2 process, it will be transferred to Figure 1The next step in the process is to use the adaptive weighted multi-page replacement method to select the appropriate page slot ID.
[0041] Step 4: Read the data page in the page slot according to the obtained data page slot ID, and return the data page information to the database service layer, that is, reach Figure 1 The last process of this step is to complete the process of the database service layer reading the target data page.
[0042] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.
Claims
1. A database page replacement method with adaptive weight adjustment, characterized in that: The following steps are involved: (1) When the database is started, the database initializes N free data page slots in the buffer to store specific data pages; each data page slot is assigned a corresponding page slot ID and page value V, where ID∈{0, N-1} and V is initialized to 0; before running the replacement algorithm, the following parameters are defined and initialized in the database: the atomic variable pointer P is initialized to 0, the weight W is 2, the action A is 1, the replacement number R is 0, the hit number H is 0, and the hit rate HR is 0; (2) When the service layer of the database accesses the target data page in the buffer, if a page slot in the buffer contains the target data page, the page value V of the data page slot is increased by W, the hit number H is increased by 1, and the data page slot ID is obtained, and the subsequent steps are skipped to step (4); if there is no target data page in the buffer, the target data page is imported from the disk into the buffer; (3) When importing the target data page into the buffer, if there is an idle data page slot in the buffer, the target data page is imported into the idle data page slot and the data page slot ID is obtained; if there is no idle data page slot in the buffer, a non-idle data page slot ID is found and the target data page is imported into the non-idle data page slot and the data page slot ID is obtained; (4) Based on the obtained data page slot ID, read the data page in the page slot and return the data to the service layer.
2. The method for replacing a database page with adaptive weight adjustment according to claim 1, characterized in that: In the step (3), a non-idle data page slot ID is found and a page replacement method with adaptive weight adjustment is adopted. The method is specifically as follows: (3.1) Use atomic operations to obtain the value of pointer P: if the value of P is N, set P to 0; if the value of P is between 0 and N-1, return the value of P; (3.2) Read the page value Vp of the corresponding page slot ID according to the value of P; (3.3) Determine the page value Vp: If the page value Vp is 0, obtain the page slot ID and proceed to step (3.4); if the page value Vp is greater than 0, reduce the page value Vp of the page slot by 1, increase P by 1, and return to step (3.1); (3.4) Increase the replacement number R by 1 and determine the replacement number R: if the replacement number R is greater than or equal to N, proceed to step (3.5); if the replacement number R is less than N, skip the subsequent steps and proceed to step (3.8); (3.5) Calculate the temporary hit rate HRp during this period: The temporary hit rate calculation formula is: HRp = H / (H+R); set H to 0 and R to 0; (3.6) Update weight W: If the hit rate HRp is less than HR, change action A to its opposite number and add W to A; If the hit rate HRp is greater than HR, then action A remains unchanged and W is added to A; (3.7) Update the hit rate HR: set HR to HRp; (3.8) Read the page slot ID and read the target data page from the disk to the page slot.
Citation Information
Patent Citations
LRU flash memory cache management method based on dynamic page weight
CN109857680A
Cache prefetching method and system based on NVM hybrid memory
CN111143243A
Isolated ordered regions (IOR) prefetching and page replacement
US20050018152A1
Using global memory information to manage memory in a computer network
US6167490A
Persistence of page access heuristics in a memory centric architecture
US9652405B1
