A dual storage pool data elimination method, device and equipment

Abstract

The present invention provides a dual storage pool data elimination method, device, and equipment, the method comprising the following steps: after flushing the cache pool data to the data pool, obtaining the cache pool water level; judging whether the cache pool water level is greater than the preset warning water level; if so, directly eliminating the clean objects in the cache pool; wherein the clean objects are the data in the cache pool that have been flushed to the data pool; if not, placing the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy; periodically obtaining the clean objects in the set data range from the LRU algorithm queue for elimination processing. Ensure that the cache pool is not full. After the data is flushed, the data in the cache pool is eliminated under the premise of ensuring the data reading efficiency, so as to avoid the storage system being unavailable due to the cache pool being full.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed storage systems, and in particular to a dual storage pool data elimination method, device and equipment. Background Art

[0002] In a distributed storage system, there are two strategies for data preservation: replica and erasure. For small blocks of data, the data storage speed is fast in the replica mode, and the read operation only needs to select one replica for synchronous reading. However, since the data is copied and saved, space is wasted. The erasure mode divides the written data into erasure stripes and calculates the verification data blocks, so the writing speed is slow. When reading, multiple shards must be read and merged for calculation. Using asynchronous reading, the reading speed is also slower than the replica mode. However, the erasure mode only needs to save more verification data blocks, so it uses less space.

[0003] In order to improve the read and write performance of small blocks of data and take into account the storage space utilization rate of large blocks of data, a replica pool and an erasure pool are established for data storage, and the replica pool is used as a cache pool for the erasure pool. All data operations must first pass through the replica pool, and the storage pool is selected according to the data size. At the same time, in order to improve the performance of the replica pool, SSD is used as the storage medium of the replica pool.

[0004] However, SSDs are more expensive than mechanical hard disks. Considering the cost factor, the cache pool is generally smaller. In order to avoid the cache pool being full and causing the entire storage system to be unavailable, the data written to the cache pool needs to be flushed to the data pool. After the flush is completed, the clean data in the cache pool is eliminated. Summary of the invention

[0005] In order to prevent the cache pool from being filled up and causing the entire storage system to be unavailable, the data written into the cache pool needs to be flushed to the data pool. After the flush is completed, the clean data in the cache pool is eliminated. The present invention provides a dual storage pool data elimination method, device, and equipment.

[0006] The technical solution of the present invention is:

[0007] In a first aspect, the technical solution of the present invention provides a dual storage pool data elimination method, wherein the dual storage pool includes a cache pool and a data pool, and the method includes the following steps:

[0008] After flushing the cache pool data to the data pool, obtain the cache pool water level;

[0009] Determine whether the water level of the cache pool is greater than the preset warning water level;

[0010] If yes, the clean object in the cache pool is directly eliminated; the clean object is the data in the cache pool that has been flushed to the data pool;

[0011] If not, put the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy;

[0012] Periodically obtain the clean objects in the set data range from the LRU algorithm queue for elimination.

[0013] Furthermore, the step of placing the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy also includes:

[0014] Get the clean object tag information in the cache pool;

[0015] The clean objects with the mark information of false in the cache pool are directly eliminated; among them, the mark information of the data that is directly eliminated after the flush is completed is set to false.

[0016] Put the remaining clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy.

[0017] Furthermore, in the step of periodically obtaining clean objects with a set data elimination range from the LRU algorithm queue for elimination processing, the step of determining the data elimination range includes:

[0018] Get the scope of the clean object in the cache pool;

[0019] Get the property information of the clean object;

[0020] The calculation attribute information is the range of the clean object of dirty data;

[0021] The range of clean objects in the cache pool is subtracted from the range of clean objects whose attribute information is dirty data to determine the data elimination range.

[0022] Since data may be reused after being flushed, the data cannot be classified as trusted after being reused, that is, the data cannot be eliminated after being used again.

[0023] Furthermore, when the node is restarted or the OSD process is restarted, the queue in the OSD memory is lost, and the step of periodically obtaining the clean object of the set data range from the LRU algorithm queue for elimination includes:

[0024] Check whether the LRU algorithm queue exists.

[0025] If so, the clean object of the set data range is obtained from the LRU algorithm queue periodically for elimination;

[0026] If not, scan all clean objects in the cache pool and re-determine the clean objects in the cache pool;

[0027] Put the re-determined clean object into the corresponding LRU algorithm queue according to the set strategy;

[0028] Further, the steps of scanning all clean objects in the cache pool and re-determining the clean objects in the cache pool include:

[0029] Scan all clean objects in the cache pool;

[0030] Get the scope of clean objects in the cache pool;

[0031] The range of the obtained clean objects is subtracted from the range of the clean objects whose attribute information is dirty data, and the clean objects in the cache pool are re-determined.

[0032] Furthermore, the step of placing the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy specifically includes:

[0033] Get the length of data flushed to the data pool;

[0034] Compare the length of the data flushed to the data pool with the erasure length of the data pool;

[0035] If the length of data flushed to the data pool is greater than or equal to one-fourth of the erasure length of the data pool, the clean object in the cache pool is placed in the first LRU algorithm queue;

[0036] If the length of data flushed to the data pool is greater than or equal to one eighth of the erasure length of the data pool and less than one quarter of the erasure length of the data pool, the clean object in the cache pool is placed in the second LRU algorithm queue;

[0037] If the data length flushed to the data pool is greater than 0 and less than one eighth of the erasure length of the data pool, the clean object in the cache pool is placed in the third LRU algorithm queue; wherein, the data length of the second LRU algorithm queue is less than the data length of the first LRU algorithm queue and greater than the data length of the third LRU algorithm queue.

[0038] Furthermore, in the step of periodically obtaining clean objects of a set data range from the LRU algorithm queue for elimination, objects with large clean data volumes are obtained first; when there are no objects in the first LRU algorithm queue and the second LRU algorithm queue, clean objects are obtained from the third LRU algorithm queue.

[0039] In a second aspect, the technical solution of the present invention provides a dual storage pool data elimination device, the dual storage pool includes a cache pool and a data pool, the device includes a cache pool capacity acquisition module, a cache pool capacity judgment module, an elimination module, a preprocessing module, and a processing elimination module;

[0040] The cache pool capacity acquisition module is used to obtain the cache pool water level after the data is flushed;

[0041] The buffer pool capacity judgment module is used to judge whether the buffer pool water level is greater than the preset warning water level;

[0042] The elimination module is used to directly eliminate the clean objects in the cache pool if the water level in the cache pool is greater than the preset warning water level; wherein the clean objects are the data in the cache pool that have been flushed to the data pool;

[0043] The pre-processing module is used to put the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy if the water level of the cache pool is not greater than the preset warning water level;

[0044] The elimination processing module is used to periodically obtain clean objects in a set data range from the LRU algorithm queue for elimination processing.

[0045] Furthermore, the preprocessing module includes a marking information acquisition unit and a queue insertion unit;

[0046] A tag information acquisition unit, used to acquire the tag information of the clean object in the cache pool;

[0047] The elimination module is also used to directly eliminate clean objects in the cache pool whose mark information is false;

[0048] The insertion queue unit is used to put the remaining clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy.

[0049] Furthermore, the processing elimination module includes an object range acquisition unit, an attribute information acquisition unit, a calculation unit, and a data elimination range determination unit;

[0050] The object range acquisition unit is used to obtain the range of the clean object in the cache pool;

[0051] An attribute information acquisition unit, used to acquire attribute information of a clean object;

[0052] A calculation unit is used to calculate the range of clean objects whose attribute information is dirty data;

[0053] The data elimination range determination unit is used to subtract the range of the clean objects in the cache pool from the range of the clean objects whose attribute information is dirty data to determine the data elimination range.

[0054] Furthermore, the device also includes a queue status detection module and a scanning module;

[0055] Queue status detection module, used to detect whether the LRU algorithm queue exists;

[0056] The scanning module is used to scan all clean objects in the cache pool and re-determine the clean objects in the cache pool when the queue status detection module detects that the LRU algorithm queue does not exist;

[0057] The preprocessing module is also used to put the re-determined clean objects into the corresponding LRU algorithm queue according to the set strategy;

[0058] Further, the scanning module includes a scanning unit, an object range acquisition unit, and an object determination unit;

[0059] Scanning unit, used to scan all clean objects in the cache pool;

[0060] The object range acquisition unit is used to obtain the range of clean objects in the cache pool;

[0061] The object determination unit is used to subtract the range of the obtained clean objects from the range of the clean objects whose attribute information is dirty data, and re-determine the clean objects in the cache pool.

[0062] Furthermore, the insertion queue unit includes a data length acquisition submodule, a data length comparison submodule, and a selection insertion submodule;

[0063] The data length acquisition submodule is used to obtain the length of data flushed to the data pool;

[0064] A data length comparison submodule is used to compare the length of the data obtained and flushed to the data pool with the erasure length of the data pool;

[0065] Select an insertion submodule to place the clean objects in the cache pool into the first LRU algorithm queue if the length of data flushed to the data pool is greater than or equal to one-fourth of the erasure length of the data pool; place the clean objects in the cache pool into the second LRU algorithm queue if the length of data flushed to the data pool is greater than or equal to one-eighth of the erasure length of the data pool and less than one-fourth of the erasure length of the data pool; place the clean objects in the cache pool into the third LRU algorithm queue if the length of data flushed to the data pool is greater than 0 and less than one-eighth of the erasure length of the data pool; wherein the data length of the second LRU algorithm queue is less than the data length of the first LRU algorithm queue and greater than the data length of the third LRU algorithm queue.

[0066] Furthermore, the processing elimination module preferentially obtains clean objects with large data volume for elimination; when there are no objects in the first LRU algorithm queue and the second LRU algorithm queue, clean objects are obtained from the third LRU algorithm queue for elimination.

[0067] In a third aspect, the technical solution of the present invention further provides an electronic device, comprising a processor and a memory, wherein the processor and the memory communicate with each other via a communication bus;

[0068] A memory for storing a data elimination program;

[0069] The processor is used to implement the steps of the dual storage pool data elimination method as described in the first aspect when executing the data elimination program.

[0070] From the above technical solutions, it can be seen that the present invention has the following advantages: using the replica pool as the cache pool of the erasure pool, designing a set of clean data elimination methods to ensure that the cache pool is not full. After the data is flushed, the data in the cache pool is eliminated under the premise of ensuring data reading efficiency, to avoid the storage system being unavailable due to the cache pool being full.

[0071] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect.

[0072] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0074] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0075] Figure 2 is a schematic block diagram of an apparatus according to an embodiment of the present invention.

[0076] In the figure, 22-cache pool capacity acquisition module, 33-cache pool capacity judgment module, 44-elimination module, 55-preprocessing module, 66-processing and elimination module, 77-queue status detection module, 88-scanning module. DETAILED DESCRIPTION

[0077] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0078] LRU, Last Recent Used, the most recently used objects are discarded last.

[0079] like Figure 1 As shown, the technical solution of the present invention provides a dual storage pool data elimination method, the dual storage pool includes a cache pool and a data pool, and the method includes the following steps:

[0080] Step 11: After flushing the cache pool data to the data pool, obtain the cache pool water level;

[0081] Step 12: Determine whether the water level of the buffer pool is greater than the preset warning water level; if so, execute step 13; otherwise, execute step 14;

[0082] Step 13: directly eliminate the clean objects in the cache pool; the clean objects are the data in the cache pool that have been flushed to the data pool;

[0083] Step 14: Put the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy;

[0084] Step 15: Periodically obtain the clean objects of the set data range from the LRU algorithm queue for elimination.

[0085] LRU algorithm queue, used to store objects with different amounts of clean data.

[0086] In some embodiments, the step of placing the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy further includes:

[0087] Get the clean object tag information in the cache pool;

[0088] The clean objects with the mark information of false in the cache pool are directly eliminated; among them, the mark information of the data that is directly eliminated after the flush is completed is set to false.

[0089] Put the remaining clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy.

[0090] In some embodiments, in the step of periodically obtaining clean objects with a set data elimination range from the LRU algorithm queue for elimination processing, the step of determining the data elimination range includes:

[0091] Get the scope of the clean object in the cache pool;

[0092] Get the property information of the clean object;

[0093] The calculation attribute information is the range of the clean object of dirty data;

[0094] The range of clean objects in the cache pool is subtracted from the range of clean objects whose attribute information is dirty data to determine the data elimination range.

[0095] Since data may be reused after being flushed, the data cannot be classified as trusted after being reused, that is, the data cannot be eliminated after being used again.

[0096] In some embodiments, when a node is restarted or an OSD process is restarted, there is a situation where the queue in the OSD memory is lost, and the step of periodically obtaining a clean object of a set data range from the LRU algorithm queue for elimination processing includes:

[0097] Check whether the LRU algorithm queue exists;

[0098] If so, the clean object of the set data range is obtained from the LRU algorithm queue periodically for elimination;

[0099] If not, scan all clean objects in the cache pool and re-determine the clean objects in the cache pool;

[0100] Put the re-determined clean object into the corresponding LRU algorithm queue according to the set strategy;

[0101] In some embodiments, the step of scanning all clean objects in the buffer pool and re-determining the clean objects in the buffer pool includes:

[0102] Scan all clean objects in the cache pool;

[0103] Get the scope of clean objects in the cache pool;

[0104] The range of the obtained clean objects is subtracted from the range of the clean objects whose attribute information is dirty data, and the clean objects in the cache pool are re-determined.

[0105] In some embodiments, the step of placing the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy specifically includes:

[0106] Get the length of data flushed to the data pool;

[0107] Compare the length of the data flushed to the data pool with the erasure length of the data pool;

[0108] If the length of data flushed to the data pool is greater than or equal to one-fourth of the erasure length of the data pool, the clean object in the cache pool is placed in the first LRU algorithm queue;

[0109] If the length of data flushed to the data pool is greater than or equal to one eighth of the erasure length of the data pool and less than one quarter of the erasure length of the data pool, the clean object in the cache pool is placed in the second LRU algorithm queue;

[0110] If the data length flushed to the data pool is greater than 0 and less than one eighth of the erasure length of the data pool, the clean object in the cache pool is placed in the third LRU algorithm queue; wherein, the data length of the second LRU algorithm queue is less than the data length of the first LRU algorithm queue and greater than the data length of the third LRU algorithm queue.

[0111] In some embodiments, in the step of periodically obtaining clean objects of a set data range from the LRU algorithm queue for elimination, objects with large clean data volumes are obtained first; when there are no objects in the first LRU algorithm queue and the second LRU algorithm queue, clean objects are obtained from the third LRU algorithm queue.

[0112] The specific implementation process of this method is illustrated by example:

[0113] Design an elimination thread CacheEvictThread and a pg scanning thread PGScanThread, and add three LRU algorithm queues for eliminating clean objects, including the first LRU algorithm queue m_clean_big_lru, the second LRU algorithm queue m_clean_mid_lru, and the third LRU algorithm queue m_clean_small_lru, which are used to insert clean objects with different flushing ranges respectively. At the same time, add a bool configuration item cache_tier_evict_data_when_flush to mark whether to directly eliminate the data in the clean range after the flushing is completed. The default value of this configuration item is false;

[0114] Design a warning water level for the cache pool. After the data is flushed, obtain the cache pool water level. If the water level is greater than the warning water level, the flushed data is directly eliminated.

[0115] If the water level of the cache pool does not exceed the warning water level, the flushed object is inserted into the LRU algorithm queue. The LRU algorithm queue selection strategy is: the flushed data length is [1 / 4,1] of the data pool erasure length stripe_width, select the first LRU algorithm queue m_clean_big_lru, the flushed data length is [1 / 8,1 / 4) of the data pool erasure length stripe_width, select the second LRU algorithm queue, m_clean_mid_lru, the flushed data length is (0,1 / 8) of the data pool erasure length stripe_width, select the third LRU algorithm queue m_clean_small_lru;

[0116] Data elimination is implemented. The elimination thread periodically obtains objects from the LRU algorithm queue for elimination. Each time, a specified number of clean objects are obtained, and objects with large clean data volumes are obtained first. Only when there are no objects in m_clean_big_lru and m_clean_mid_lru, objects are obtained from m_clean_small_lru.

[0117] Data elimination range: Use fiemap to obtain the range of the object in the cache pool, and subtract the dirty_range range recorded in the attribute to get the data range that needs to be eliminated in this operation.

[0118] Exception handling: When a node is restarted or the OSD process is restarted, the LRU algorithm queue in the OSD memory is lost, and the elimination thread cannot eliminate the clean data in the cache pool, which may cause the cache pool water level to be too high. Use the pg scanning thread to scan all objects in the pg, compare the range obtained by the fiemap of the object with the dirty_range range, and reinsert it into the LRU algorithm queue based on the difference.

[0119] like Figure 2 As shown, the HIA of the embodiment of the present invention provides a dual storage pool data elimination device, the dual storage pool includes a cache pool and a data pool, the device includes a cache pool capacity acquisition module 22, a cache pool capacity judgment module 33, an elimination module 44, a pre-processing module 55, and a processing elimination module 66;

[0120] The buffer pool capacity acquisition module 22 is used to obtain the buffer pool water level after the data flush is completed;

[0121] The buffer pool capacity determination module 33 is used to determine whether the buffer pool water level is greater than a preset warning water level;

[0122] The elimination module 44 is used to directly eliminate the clean objects in the cache pool if the water level of the cache pool is greater than the preset warning water level; wherein the clean objects are the data in the cache pool that have been flushed to the data pool;

[0123] The pre-processing module 55 is used to put the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy if the water level of the cache pool is not greater than the preset warning water level;

[0124] The elimination processing module 66 is used to periodically obtain clean objects in a set data range from the LRU algorithm queue for elimination processing.

[0125] In some embodiments, the preprocessing module 55 includes a marking information acquisition unit and a queue insertion unit;

[0126] A tag information acquisition unit, used to acquire the tag information of the clean object in the cache pool;

[0127] The elimination module is also used to directly eliminate clean objects in the cache pool whose mark information is false;

[0128] The insertion queue unit is used to put the remaining clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy.

[0129] In some embodiments, the processing elimination module 66 includes an object range acquisition unit, an attribute information acquisition unit, a calculation unit, and a data elimination range determination unit;

[0130] The object range acquisition unit is used to obtain the range of the clean object in the cache pool;

[0131] An attribute information acquisition unit, used to acquire attribute information of a clean object;

[0132] A calculation unit is used to calculate the range of clean objects whose attribute information is dirty data;

[0133] The data elimination range determination unit is used to subtract the range of the clean objects in the cache pool from the range of the clean objects whose attribute information is dirty data to determine the data elimination range.

[0134] In some embodiments, the apparatus further includes a queue status detection module 77 and a scanning module 88;

[0135] Queue status detection module 77, used to detect whether the LRU algorithm queue exists;

[0136] The scanning module 88 is used to scan all clean objects in the cache pool and re-determine the clean objects in the cache pool when the queue state detection module detects that the LRU algorithm queue does not exist;

[0137] The preprocessing module 55 is also used to put the re-determined clean object into the corresponding LRU algorithm queue according to the set strategy;

[0138] In some embodiments, the scanning module 88 includes a scanning unit, an object range acquisition unit, and an object determination unit;

[0139] Scanning unit, used to scan all clean objects in the cache pool;

[0140] The object range acquisition unit is used to obtain the range of clean objects in the cache pool;

[0141] The object determination unit is used to subtract the range of the obtained clean objects from the range of the clean objects whose attribute information is dirty data, and re-determine the clean objects in the cache pool.

[0142] In some embodiments, the insertion queue unit includes a data length acquisition submodule, a data length comparison submodule, and a selection insertion submodule;

[0143] The data length acquisition submodule is used to obtain the length of data flushed to the data pool;

[0144] A data length comparison submodule is used to compare the length of the data obtained and flushed to the data pool with the erasure length of the data pool;

[0145] Select an insertion submodule to place the clean objects in the cache pool into the first LRU algorithm queue if the length of data flushed to the data pool is greater than or equal to one-fourth of the erasure length of the data pool; place the clean objects in the cache pool into the second LRU algorithm queue if the length of data flushed to the data pool is greater than or equal to one-eighth of the erasure length of the data pool and less than one-fourth of the erasure length of the data pool; place the clean objects in the cache pool into the third LRU algorithm queue if the length of data flushed to the data pool is greater than 0 and less than one-eighth of the erasure length of the data pool; wherein the data length of the second LRU algorithm queue is less than the data length of the first LRU algorithm queue and greater than the data length of the third LRU algorithm queue.

[0146] In some embodiments, the processing elimination module 66 preferentially obtains clean objects with large data volumes for elimination; when there are no objects in the first LRU algorithm queue and the second LRU algorithm queue, clean objects are obtained from the third LRU algorithm queue for elimination.

[0147] The value and / or the storage time reaches the preset time length and / or a running status query request is received.

[0148] Accordingly, an embodiment of the present invention further discloses an electronic device, including a processor and a memory; wherein the processor implements the following steps when executing a data elimination program stored in the memory:

[0149] After flushing the cache pool data to the data pool, obtain the cache pool water level;

[0150] Determine whether the water level of the cache pool is greater than the preset warning water level;

[0151] If yes, the clean object in the cache pool is directly eliminated; the clean object is the data in the cache pool that has been flushed to the data pool;

[0152] If not, put the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy;

[0153] Periodically obtain the clean objects in the set data range from the LRU algorithm queue for elimination.

[0154] In some specific embodiments, when the processor executes the data elimination subroutine stored in the memory, the following steps can be specifically implemented: obtaining the marking information of the clean objects in the cache pool; directly eliminating the clean objects in the cache pool whose marking information is false; and placing the remaining clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy.

[0155] In some specific embodiments, when the processor executes the data elimination subroutine stored in the memory, the following steps may be implemented: obtaining a range of the clean object in the cache pool;

[0156] Get the property information of the clean object;

[0157] The calculation attribute information is the range of the clean object of dirty data;

[0158] The range of clean objects in the cache pool is subtracted from the range of clean objects whose attribute information is dirty data to determine the data elimination range.

[0159] In some specific embodiments, when the processor executes the data elimination subroutine stored in the memory, the following steps may be implemented: detecting whether an LRU algorithm queue exists;

[0160] If so, the clean object of the set data range is obtained from the LRU algorithm queue periodically for elimination;

[0161] If not, scan all clean objects in the cache pool and re-determine the clean objects in the cache pool;

[0162] The re-determined clean object is placed into the corresponding LRU algorithm queue according to the set strategy.

[0163] In some specific embodiments, when the processor executes the data elimination subroutine stored in the memory, the following steps may be implemented:

[0164] Get the length of data flushed to the data pool;

[0165] Compare the length of the data flushed to the data pool with the erasure length of the data pool;

[0166] If the length of data flushed to the data pool is greater than or equal to one-fourth of the erasure length of the data pool, the clean object in the cache pool is placed in the first LRU algorithm queue;

[0167] If the length of data flushed to the data pool is greater than or equal to one eighth of the erasure length of the data pool and less than one quarter of the erasure length of the data pool, the clean object in the cache pool is placed in the second LRU algorithm queue;

[0168] If the data length flushed to the data pool is greater than 0 and less than one eighth of the erasure length of the data pool, the clean object in the cache pool is placed in the third LRU algorithm queue; wherein, the data length of the second LRU algorithm queue is less than the data length of the first LRU algorithm queue and greater than the data length of the third LRU algorithm queue.

[0169] Furthermore, the electronic device in this embodiment may further include:

[0170] The input interface is used to obtain the data elimination program imported from the outside and save the obtained data elimination program to the memory. It can also be used to obtain various instructions and parameters transmitted by the external terminal device and transmit them to the processor so that the processor can use the above various instructions and parameters to carry out corresponding processing. In this embodiment, the input interface can specifically include but is not limited to a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, etc.

[0171] The output interface is used to output various data generated by the processor to the terminal device connected thereto, so that other terminal devices connected to the output interface can obtain various data generated by the processor. In this embodiment, the output interface may specifically include but is not limited to a USB interface, a serial interface, etc.

[0172] The communication unit is used to establish a remote communication connection between the running state management device and the external server so that the running state management device can mount the image file to the external server. In this embodiment, the communication unit may specifically include but is not limited to a remote communication unit based on wireless communication technology or wired communication technology.

[0173] The keyboard is used to obtain various parameter data or instructions input by the user by real-time typing on the keyboard.

[0174] A display is used to display relevant information of the running data elimination process in real time.

[0175] The mouse can be used to assist users in inputting data and simplify user operations.

[0176] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0177] Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A dual storage pool data elimination method, characterized in that: The dual storage pool includes a cache pool and a data pool. The method includes the following steps: After flushing the cache pool data to the data pool, obtain the cache pool water level; Determine whether the water level of the cache pool is greater than the preset warning water level; If yes, the clean object in the cache pool is directly eliminated; the clean object is the data in the cache pool that has been flushed to the data pool; If not, put the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy; Periodically obtain the clean objects in the set data elimination range from the LRU algorithm queue for elimination processing; In the step of periodically obtaining clean objects with a set data elimination range from the LRU algorithm queue for elimination processing, the step of determining the data elimination range includes: Get the scope of the clean object in the cache pool; Get the property information of the clean object; The calculation attribute information is the range of the clean object of dirty data; Subtract the range of clean objects in the cache pool from the range of clean objects whose attribute information is dirty data to determine the data elimination range; The step of placing the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy specifically includes: Get the length of data flushed to the data pool; Compare the length of the data flushed to the data pool with the erasure length of the data pool; If the length of data flushed to the data pool is greater than or equal to one-fourth of the erasure length of the data pool, the clean object in the cache pool is placed in the first LRU algorithm queue; If the length of data flushed to the data pool is greater than or equal to one eighth of the erasure length of the data pool and less than one quarter of the erasure length of the data pool, the clean object in the cache pool is placed in the second LRU algorithm queue; If the data length flushed to the data pool is greater than 0 and less than one eighth of the erasure length of the data pool, the clean object in the cache pool is placed in the third LRU algorithm queue; wherein, the data length of the second LRU algorithm queue is less than the data length of the first LRU algorithm queue and greater than the data length of the third LRU algorithm queue.

2. The dual storage pool data elimination method according to claim 1, characterized in that: The step of placing the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy also includes: Get the clean object tag information in the cache pool; Eliminate clean objects in the cache pool whose mark information is false directly; Put the remaining clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy.

3. The dual storage pool data elimination method according to claim 2, characterized in that: The step of periodically obtaining a clean object with a set data elimination range from the LRU algorithm queue for elimination processing includes: Check whether the LRU algorithm queue exists; If so, the clean object with the set data elimination range is obtained from the LRU algorithm queue periodically for elimination; If not, scan all clean objects in the cache pool and re-determine the clean objects in the cache pool; The re-determined clean object is placed into the corresponding LRU algorithm queue according to the set strategy.

4. The dual storage pool data elimination method according to claim 3, characterized in that: The steps of scanning all clean objects in the cache pool and re-determining the clean objects in the cache pool include: Scan all clean objects in the cache pool; Get the scope of clean objects in the cache pool; The range of the obtained clean objects is subtracted from the range of the clean objects whose attribute information is dirty data, and the clean objects in the cache pool are re-determined.

5. The dual storage pool data elimination method according to claim 1, characterized in that: In the step of periodically obtaining clean objects with a set data elimination range from the LRU algorithm queue for elimination processing, objects with large clean data volume are obtained first; when there are no objects in the first LRU algorithm queue and the second LRU algorithm queue, clean objects are obtained from the third LRU algorithm queue.

6. A dual storage pool data elimination device, characterized in that: The dual storage pool includes a buffer pool and a data pool, and the device includes a buffer pool capacity acquisition module, a buffer pool capacity judgment module, an elimination module, a pre-processing module, and a processing elimination module; The cache pool capacity acquisition module is used to obtain the cache pool water level after the data is flushed; The buffer pool capacity judgment module is used to judge whether the buffer pool water level is greater than the preset warning water level; The elimination module is used to directly eliminate the clean objects in the cache pool if the water level in the cache pool is greater than the preset warning water level; wherein the clean objects are the data in the cache pool that have been flushed to the data pool; The pre-processing module is used to put the clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy if the water level of the cache pool is not greater than the preset warning water level; The elimination processing module is used to periodically obtain the clean objects in the set data elimination range from the LRU algorithm queue for elimination processing; The processing elimination module includes an object range acquisition unit, an attribute information acquisition unit, a calculation unit, and a data elimination range determination unit; The object range acquisition unit is used to obtain the range of the clean object in the cache pool; An attribute information acquisition unit, used to acquire attribute information of a clean object; A calculation unit is used to calculate the range of clean objects whose attribute information is dirty data; A data elimination range determination unit, used to determine the data elimination range by subtracting the range of the clean objects in the cache pool from the range of the clean objects whose attribute information is dirty data; The preprocessing module includes an insert queue unit, which is used to put the remaining clean objects in the cache pool into the corresponding LRU algorithm queue according to the set strategy; The insertion queue unit includes a data length acquisition submodule, a data length comparison submodule, and a selection insertion submodule; The data length acquisition submodule is used to obtain the length of data flushed to the data pool; A data length comparison submodule is used to compare the length of the data obtained and flushed to the data pool with the erasure length of the data pool; Select an insertion submodule to place the clean objects in the cache pool into the first LRU algorithm queue if the length of data flushed to the data pool is greater than or equal to one-fourth of the erasure length of the data pool; place the clean objects in the cache pool into the second LRU algorithm queue if the length of data flushed to the data pool is greater than or equal to one-eighth of the erasure length of the data pool and less than one-fourth of the erasure length of the data pool; place the clean objects in the cache pool into the third LRU algorithm queue if the length of data flushed to the data pool is greater than 0 and less than one-eighth of the erasure length of the data pool; wherein the data length of the second LRU algorithm queue is less than the data length of the first LRU algorithm queue and greater than the data length of the third LRU algorithm queue.

7. The dual storage pool data elimination device according to claim 6, characterized in that: The device also includes a queue status detection module and a scanning module; Queue status detection module, used to detect whether the LRU algorithm queue exists; The scanning module is used to scan all clean objects in the cache pool and re-determine the clean objects in the cache pool when the queue status detection module detects that the LRU algorithm queue does not exist; The preprocessing module is also used to put the re-determined clean object into the corresponding LRU algorithm queue according to the set strategy.

8. An electronic device, characterized in that: It includes a processor and a memory, wherein the processor and the memory communicate with each other via a communication bus; A memory for storing a data elimination program; A processor is used to implement the steps of the dual storage pool data elimination method as described in any one of claims 1 to 5 when executing the data elimination program.

Images