An LRU-based Memory Release Scheduling System and Method for Data Buffers

Abstract

The present invention provides an LRU-based data buffer memory release scheduling system and method for handling cache release in data transmission tasks and hierarchically retaining important information file data; it solves the problem that some important resources in data caching are easily terminated incorrectly, resulting in a large amount of time consumption for subsequent rescheduling; and it realizes the relatively balanced utilization of resources in the memory field while shortening the overall execution time of data transmission tasks. The system and method of the present invention partition the memory area, preferentially place corresponding cached data files according to levels respectively, and first release the data in the low-level cache area when the memory is released, avoiding the cleaning of important data; through the partitioning algorithm, the overall time of memory data scheduling is significantly improved.

Description

Technical Field

[0001] The present invention belongs to the field of in-memory cache management of databases, and particularly relates to an LRU-based data buffer memory release scheduling system and method. Background Art

[0002] LRU (Least Recently Used) is a common page replacement algorithm. In computing, all file operations need to be carried out in memory. However, the size of computer memory is fixed, so it is impossible for us to load all files into memory. Therefore, we need to formulate a strategy to select the files added to memory.

[0003] Using the principle of locality, it infers future behavior based on the past page access history of a job during execution. It believes that pages that have not been accessed in the past period are likely not to be accessed in the near future. Therefore, the essence of this algorithm is: when a page needs to be replaced, always select the page that has not been used for the longest time in the recent period for replacement. That is, replace the page that has not been accessed for the longest time recently. Summary of the Invention

[0004] In view of this, the present invention aims to propose an LRU-based data buffer memory release scheduling system and method to solve the situation of insufficient effective utilization of resources on the buffer memory in a computing environment, and to shorten the overall execution time of computer data scheduling on the basis of relatively balanced effective utilization rate on the buffer memory.

[0005] To achieve the above object, the technical solution of the present invention is realized as follows:

[0006] An LRU-based data buffer memory release scheduling method includes the following steps:

[0007] S1. Roughly divide the memory area into three levels: high, medium, and low;

[0008] S2. Sort the three levels from high to low and name them respectively;

[0009] S3. Set the priority of the called data file, and optimize the size of the level area corresponding to the called data file according to the memory size occupied by the called data file;

[0010] S4. Use the status interface to obtain the status of each memory partition in real time;

[0011] S5. Allocate a reasonable memory area according to the priority of the called data file, and release the data in the memory according to the memory release method;

[0012] The memory release method includes the following: Judging the memory partition level according to the priority of the called data file, preferentially occupying the memory area consistent with this priority. If this memory area is full and not released, occupy the next level or any level of the next level of this priority.

[0013] Further, the sizes of the three areas in step S1 are set by the administrator.

[0014] Further, the naming method in step S2 is as follows: Sort the three-level areas from high to low and name them in turn according to the sorting result. The naming names include: the memory name of this memory, the level of this memory, and the actual size of the memory partition of this memory. The naming format is specifically as follows: BUFFER_a_[size], BUFFER_b_[size], BUFFER_c_[size], BUFFER_d_[size]... The level sizes are a>b>c>d and so on. [size] represents the actual size of the divided memory partition. BUFFER is the unified memory name, and the letter represents the data level stored in its partition.

[0015] Further, the memory release method in step S5 includes the following: Judging the memory partition level according to the priority of the data file, preferentially occupying the memory area consistent with this priority. If this memory area is full and not released, occupy the next level or any level of the next level of this priority;

[0016] Specifically as follows:

[0017] The memory release method includes the high-level memory release method, the intermediate-level memory release method, and the low-level memory release method. Specifically as follows:

[0018] 1) The high-level memory release method includes: Preferentially occupying the memory partitions at the highest level and below, not releasing memory when the overall memory resources are tense, and occupying the next-level memory partition;

[0019] 2) The intermediate-level memory release method includes:

[0020] In the first case, occupying the memory partitions at the intermediate level and below, when the low-level and intermediate-level memory resources are tense, the highest-level memory partition can be occupied, and memory is released when the overall or high-level memory is tense;

[0021] In the second case, occupying the memory partitions at the intermediate level and below, and releasing memory when the low-level, intermediate-level, and overall memory are tense;

[0022] 3) The low-level memory release method includes: Occupying the low-level memory partition, the intermediate-level memory partition can be occupied when the low-level partition is tense, and memory is released at any time according to the LRU principle.

[0023] A data buffer memory release scheduling system based on LRU, comprising: a memory partition manager, a data task manager, a data buffer allocator, a memory status scanner, and a memory resource manager;

[0024] Memory partition manager: used to manage the merging, splitting, naming, and grading of memory;

[0025] Data task manager: used to set the memory level and memory release method for special data;

[0026] Data buffer allocator: used to parse the data priorities set by the data task manager, arrange them in reasonable memory areas according to the data priorities, and release other data according to the setting principles of the data task manager when memory resources are in short supply;

[0027] Memory status scanner: used to obtain the status of the data buffer allocator for different memory partitions in real time;

[0028] Memory resource manager: used to release legal data when resources are in short supply according to the release principles set by the data task manager for different data, or temporarily raise the priority to ensure that the highest-level data is not affected.

[0029] Furthermore, the status of the memory partition includes partition name, partition size, partition level, data name, data size, data occupation time, and data release principle.

[0030] Compared with the prior art, the data buffer memory release scheduling system and method based on LRU of the present invention have the following advantages:

[0031] (1) The data buffer memory release scheduling system and method based on LRU of the present invention meet more data transmission requirements in computing tasks, make more full use of resources during the operation of each task; avoid the phenomenon of resource preemption of important data during data transmission; avoid the incorrect release of important data; greatly improve the utilization rate of memory resources, and further balance the occupation of different data in memory.

[0032] (2) The data buffer memory release scheduling system and method based on LRU of the present invention can avoid the phenomenon that the memory resources occupied by important data are incorrectly preempted frequently when calculating data tasks. This enables customers or administrators to raise the priority [reduce the degree of deference] of their own more important data, store it in a higher-level memory area, thus largely avoiding the situation that due to the sudden influx of a large amount of data, it is directly released by the system incorrectly, resulting in an increase in the time to re-call the important data next time.

[0033] (3) The data buffer memory release scheduling system and method based on LRU according to the present invention can ensure that the overall data information is reasonably allocated in the memory, ensure that the data buffer is not occupied and released, and avoid the systematic overall collapse caused by incorrect release. By making the memory management more meticulous and precise, the calculation time in the overall progress is shortened and the number of calculation crashes is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings forming a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0035] Figure 1 It is a process schematic diagram of a data buffer memory release scheduling system and method based on LRU according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0037] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise stated, the meaning of "a plurality" is two or more.

[0038] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0039] The present invention will be described in detail below with reference to the drawings and in combination with the embodiments.

[0040] A data buffer memory release scheduling method based on LRU includes the following steps:

[0041] S1. The memory area is roughly divided into three levels: high, medium and low;

[0042] S2. Sort the three level areas from high to low and name them respectively;

[0043] S3, setting the priority of the called data file, and optimizing the size of the level area corresponding to the called data file according to the size of the memory occupied by the called data file;

[0044] S4. Use the status interface to obtain the status of each memory partition in real time;

[0045] S5. Allocate a reasonable memory area according to the priority of the called data file, and release the data in the memory according to the memory release method.

[0046] The sizes of the three areas in step S1 are set by the administrator.

[0047] The naming method in step S2 is as follows: The naming method in step S2 is as follows: Sort the three level areas from high to low, and name them in sequence according to the sorting results. The naming names include: the memory name of this memory, the level of this memory, the actual size of the memory partition of this memory. The specific naming is as follows: BUFFER_a_[size], BUFFER_b_[size], BUFFER_c_[size], BUFFER_d_[size]..., the level size is a>b>c>d and so on, [size] represents the actual size of the divided memory partition, BUFFER is a unified memory name, and the letters represent the level of data stored in its partition.

[0048] The memory release method in step S5 includes the following contents: judging the memory partition level according to the priority of the data file, and preferentially occupying the memory area consistent with the current priority; if the current memory area is full and not released, occupying the next level of the current priority or any level of the next level of the current priority;

[0049] like Figure 1 As shown, the memory release method includes a high-level memory release method, a medium-level memory release method, and a low-level memory release method;

[0050] 1) The advanced memory release method includes: giving priority to occupying the highest level and lower memory partitions, not releasing memory when the overall memory resources are tight, and occupying the next level memory partition;

[0051] 2) Intermediate memory release methods include:

[0052] In the first case, it occupies memory partitions at the intermediate level and below. When the low-level and intermediate-level memory resources are tight, it can occupy the highest-level memory partition and release memory when the overall or high-level memory is tight.

[0053] In the second case, it occupies memory partitions at the intermediate level and below and releases memory when the low-level, intermediate-level, and overall memory are tight.

[0054] 3) The low-level memory release method includes: occupying the low-level memory partition, which can occupy the intermediate-level memory partition when the low-level partition is tight, and releasing memory at any time according to the LRU principle.

[0055] Use the status interface to obtain the real-time status of each memory partition:

[0056] Through the status interface, scan the data information of each partition in real time to obtain the actual situation, and perform level differentiation and memory allocation on the data newly entering the memory.

[0057] Allocate a reasonable memory area according to the priority of the calling task:

[0058] According to the priorities of different tasks set in advance, the system puts them into the corresponding memory areas according to the judgment. The allocation principle is downward compatibility. If there is no space, memory is released according to the memory release method.

[0059] An LRU-based data buffer memory release scheduling system includes: a memory partition manager, a data task manager, a data buffer allocator, a memory status scanner, and a memory resource manager;

[0060] Memory partition manager: used to manage the merging, splitting, naming, and grading of memory;

[0061] Data task manager: used to set the memory level and memory release method for special data;

[0062] Data buffer allocator: used to parse the data priorities set by the data task manager, arrange them in a reasonable memory area according to the data priorities, and release other data according to the setting principles of the data task manager when memory resources are tight;

[0063] Memory status scanner: used to obtain the status of the data buffer allocator for different memory partitions in real time;

[0064] Memory resource manager: used to release legal data when resources are tight according to the release principles set by the data task manager for different data, or temporarily raise the priority to ensure that the highest-level data is not affected.

[0065] The states of memory partitions include partition name, partition size, partition level, data name, data size, data occupancy time, and data release principle.

[0066] For example:

[0067] On a very wide road, there are trucks, cars, bicycles, and pedestrians moving. They only differ in size. When a large number of bicycles and pedestrians suddenly pour in, due to limited space, the traffic police [memory allocation mechanism] force some cars and large trucks to stop. Some of these cars and trucks are frequently used, and starting them again will consume a lot of time. Or some cars, although not frequently used, carry important data information. This is a situation that developers and customers do not want to see.

[0068] It can avoid the phenomenon that the memory resources occupied by important data are wrongly preempted frequently when calculating data tasks. This enables customers or administrators to prioritize [reduce deference] and promote their own relatively important data, storing it in a higher-level memory area, thus largely avoiding the situation where, due to a sudden influx of a large amount of data, it is directly released by the system wrongly, increasing the time for re-calling this important data next time.

[0069] Specific implementation process:

[0070] The memory layering process of the data buffer area based on LRU is completed by the following components working together:

[0071] 1. Memory partition manager: Responsible for managing the merging, splitting, naming, and grading of memory.

[0072] 2. Data task manager: Responsible for setting the memory level and release principle of special data.

[0073] 3. Data buffer allocator: Responsible for parsing the data priority and arranging it in a reasonable memory area, and releasing other data according to the setting principle of the data task manager when the memory is resource-constrained.

[0074] 4. Memory status scanner: Real-time obtains the status of different memory partitions, including but not limited to partition name, partition size, partition level, data name, data size, data occupancy time, data release principle, etc.; administrators or authorized users can set the memory environment for different data through the scanner and the data task manager.

[0075] 5. Memory resource manager: Releases legal data according to the set release principle of different data when resources are tense, or temporarily promotes the priority to ensure that the highest-level data is not affected.

[0076] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for scheduling memory release of a data buffer based on LRU, characterized in that it includes the following steps: S1. Divide the memory area into three levels: high, medium, and low; S2. Sort the three levels of areas from high to low and name them respectively; S3. Set the priority of the called data file, and optimize the size of the level area corresponding to the called data file according to the memory size occupied by the called data file; S4. Use the status interface to obtain the status of each memory partition in real time; S5. Allocate a reasonable memory area according to the priority of the called data file, and release the data in the memory according to the memory release method; The memory release method includes the following: Judge the memory partition level to which it belongs according to the priority of the called data file, and preferentially occupy the memory area consistent with this priority. If this memory area is full and not released, occupy the next level or any level of the next level of this priority; The memory release method includes a high-level memory release method, a medium-level memory release method, and a low-level memory release method, which are specifically as follows: 1) The high-level memory release method includes: Preferentially occupy the memory partitions at the highest level and below, and do not release memory when the overall memory resources are tense, and occupy the next-level memory partition; 2) The medium-level memory release method includes: In the first case, occupy the memory partitions at the medium level and below. When the low-level and medium-level memory resources are tense, the highest-level memory partition can be occupied, and memory is released when the overall or high-level memory is tense; In the second case, occupy the memory partitions at the medium level and below, and release memory when the low-level, medium-level, and overall memory are tense; 3) The low-level memory release method includes: Occupy the low-level memory partition, and can occupy the medium-level memory partition when the low-level partition is tense, and release memory at any time according to the LRU principle.

2. A method for scheduling memory release of a data buffer based on LRU according to claim 1, characterized in that: In step S1, the sizes of the three areas are set by the administrator.

3. A method for scheduling memory release of a data buffer based on LRU according to claim 1, characterized in that: In step S2, the naming method is as follows: Sort the three levels of areas from high to low, and name them in sequence according to the sorting result. The naming names include: the memory name of this memory, the level of this memory, and the actual size of the memory partition of this memory.

4. A system for scheduling memory release of a data buffer based on LRU, characterized in that it includes: A memory partition manager, a data task manager, a data buffer allocator, a memory status scanner, and a memory resource manager; The memory partition manager: used to manage the merging, splitting, naming, and grading of memory; The data task manager: used to set the memory level and memory release method in claim 1; The data buffer allocator: used to analyze the data priority set by the data task manager, arrange it in a reasonable memory area according to the data priority, and release other data according to the setting principle of the data task manager when the memory is in resource tension; The memory status scanner: used to obtain the status of different memory partitions by the data buffer allocator in real time; Memory resource manager: used to release legal data when resources are scarce or temporarily raise the priority according to the release principles set by the data task manager for different data, ensuring that the highest-level data is not affected.

5. A memory release scheduling system for an LRU-based data buffer according to claim 4, characterized in that: The state of the memory partition includes partition name, partition size, partition level, data name, data size, data occupancy time, and data release principle.

Images