Data block management method, device, equipment and medium
Through heat index calculation and binary heap management strategy, the problem of inaccurate data hot and cold degree assessment is solved, and the performance and life of solid-state drives are improved.
Patent Information
- Application Number
- CN202410621819.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-20
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2044-05-20
AI Technical Summary
Existing technologies are unable to accurately assess the hotness or coldness of data, resulting in low search efficiency and the inability to implement differentiated garbage collection and wear leveling strategies, affecting the performance and lifespan of solid-state drives.
The heat index calculation formula is used to determine the initial heat index of the data block, and it is dynamically managed through a binary heap structure (hot heap and cold heap). The data blocks are sorted and migrated according to the heat index, and a periodic garbage collection program is executed.
It improves the efficiency of data block search, optimizes the reading and writing speed, extends the service life of the solid-state drive, and realizes wear-leveling management.
Smart Images

Figure CN118535092B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of solid-state hard disks, and in particular to a data block management method, apparatus, device, and medium. Background Art
[0002] In the development of solid-state drive technology, optimizing strategies for identifying hot and cold data plays a crucial role in improving storage performance and extending device life. SSD firmware often requires differentiated garbage collection and wear leveling strategies for storage cells with varying temperatures, necessitating unified management of all storage cells based on their temperature.
[0003] Traditional hot and cold block management schemes use an unordered linear table to store the heat information of all data blocks, sorted by the unique index of the storage unit. With the continuous growth of data volumes and the increasing complexity of application scenarios, higher requirements are placed on the strategies for identifying hot and cold data. In solid-state drives, data is typically organized into blocks and pages, and each erase and write operation causes physical damage to the storage unit. Existing technologies cannot accurately assess the heat and coldness of data, resulting in low search efficiency. Summary of the Invention
[0004] In order to solve the above technical problems, embodiments of the present application provide a data block management method, apparatus, system, device and medium.
[0005] In a first aspect, an embodiment of the present application provides a data block management method, the method comprising:
[0006] Determine the initial heat index of each data block according to the heat index calculation formula;
[0007] Updating each of the initial heat indexes within a preset update period to obtain a plurality of updated heat indexes;
[0008] Comparing each of the update heat indexes with the hot and cold thresholds of the data block to obtain a plurality of comparison results;
[0009] Determining a sorting position of each data block in a binary heap according to each comparison result and each update heat index;
[0010] Putting each of the data blocks into the corresponding binary heap according to the sorting position;
[0011] A garbage collection process is performed by periodically accessing the data block at the top of the binary heap.
[0012] In one embodiment, the heat index calculation formula is:
[0013] HI-f×(ω r ·R|ωw ·W)×e -λt ×(1-e n )
[0014] Where HI is the heat index, f is the access frequency, R is the number of reads, W is the number of writes, ω r To read the weight factor, ω w is the write weight factor, λ is the time decay constant, t is the historical time, e n is the normalized number of erase and write cycles.
[0015] In one embodiment, the binary stack includes a hot stack and a cold stack, and each data block is placed in the cold stack.
[0016] In one embodiment, each data block is placed into the corresponding binary heap according to each comparison result, and the corresponding data blocks are sorted in the binary heap according to the order of the update heat index.
[0017] In one embodiment, determining whether each of the update heat indexes is greater than the hot and cold thresholds;
[0018] If the update heat index is greater than the cold / hot threshold, moving each data block corresponding to each update heat index from the cold stack to the hot stack;
[0019] If the update heat index is less than the cold / hot threshold, the corresponding data blocks are placed in the cold pile according to the update heat index.
[0020] In one embodiment, each of the data blocks to be moved is determined to obtain data nodes of a plurality of thermal stack data blocks;
[0021] Deleting the data nodes of each of the hot stack data blocks in the cold stack;
[0022] The data nodes of each of the thermal pile data blocks are added to the end of the thermal pile, and a floating operation is performed on the data nodes of each of the thermal pile data blocks.
[0023] In one embodiment, each of the thermal pile data blocks is sorted according to the thermal index;
[0024] determining whether each of the thermal pile data blocks is the thermal pile data block with the lowest thermal index;
[0025] If so, the thermal pile data block does not need to perform the float operation.
[0026] In a second aspect, an embodiment of the present application provides a data block management device, the data block management device comprising:
[0027] A determination module, configured to determine an initial heat index of each data block according to a heat index calculation formula;
[0028] An updating module, configured to update each of the initial heat indexes within a preset update period to obtain a plurality of updated heat indexes;
[0029] a comparison module, configured to compare each of the update heat indexes with the hot and cold thresholds of the data blocks to obtain a plurality of comparison results;
[0030] A sorting module, configured to determine a sorting position of each data block in the binary heap according to each comparison result and each update heat index;
[0031] an inserting module, configured to insert each of the data blocks into the corresponding binary heap according to the sorting position;
[0032] An execution module is configured to execute a garbage collection program by periodically accessing the data block at the top of the binary heap.
[0033] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the computer program executes the data block management method provided in the first aspect when the processor is running.
[0034] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which executes the data block management method provided in the first aspect when running on a processor.
[0035] The data block management method, device, equipment and medium provided by the present application above determine the initial heat index of each data block according to the heat index calculation formula; update each of the initial heat indexes within a preset update period to obtain multiple updated heat indexes; compare each of the updated heat indexes with the hot and cold thresholds of the data block to obtain multiple comparison results; determine the sorting position of each of the data blocks in the binary heap according to each of the comparison results and each of the updated heat indexes; put each of the data blocks into the corresponding binary heap according to each of the sorting positions; and execute the garbage collection program by periodically accessing the data blocks at the top of the binary heap. The heat index is obtained by calculation, and the data blocks are dynamically migrated in the binary heap in real time according to the heat index. The heat index can be searched by managing the binary heap, thereby improving the overall read and write speed and overall performance, and extending the service life of the solid-state hard disk. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solution of this application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of this application and should not be regarded as limiting the scope of protection of this application. In each of the drawings, similar components are numbered similarly.
[0037] Figure 1 A schematic flow chart of a data block management method is shown;
[0038] Figure 2 A schematic diagram of hot and cold stack management provided by an embodiment of the present application is shown;
[0039] Figure 3 A schematic structural diagram of a data block management device provided in an embodiment of the present application is shown;
[0040] Figure 4 A structural schematic diagram of an electronic device provided in an embodiment of the present application is shown.
[0041] Icons: 300 - data block management device, 301 - determination module, 302 - update module, 303 - comparison module, 304 - sorting module, 305 - insertion module, 306 - execution module, 400 - electronic device, 401 - transceiver, 402 - processor, 403 - memory. DETAILED DESCRIPTION
[0042] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.
[0043] The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but rather merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.
[0044] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present application, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.
[0045] Furthermore, the terms “first,” “second,” “third,” etc., are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.
[0046] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.
[0047] Example 1
[0048] An embodiment of the present application provides a data block management method.
[0049] See also Figure 1 , the data block management method includes:
[0050] S101, determining the initial heat index of each data block according to a heat index calculation formula.
[0051] It's important to note that traditional data hotness and coldness assessment schemes use either the LRU (Least Recently Used) algorithm or the temporal locality principle. Some of these schemes fail to consider the varying impacts of data operations on hardware lifespan, some lack effective implementation mechanisms, and others suffer from inadequate data management.
[0052] In this embodiment, to optimize traditional data hotness and coldness assessments and comprehensively assess data hotness and coldness, this solution uses superblocks as management units, combining the cold and hot stacks in a dual binary heap structure to dynamically manage the hotness and coldness of superblocks. A superblock is a set of data blocks with the same offset across all physical layers of all logical units in the solid-state drive.
[0053] In addition, this scheme proposes a data block heat index to linearly measure the activity of each super block to guide data migration and erase and write allocation, thereby optimizing storage performance and extending the service life of the SSD.
[0054] In one embodiment, the heat index calculation formula is:
[0055] HI=f×(ω r ·R+ω w ·W)×e -λt ×(1-e n )
[0056] Where HI is the heat index, f is the access frequency, R is the number of reads, W is the number of writes, ω r To read the weight factor, ω w is the write weight factor, λ is the time decay constant, t is the historical time, e n is the normalized number of erase and write cycles.
[0057] It should be noted that the heat index calculation formula can linearly reflect the heat index (HI) of a data block. Since write operations cause greater wear on the SSD, write operations are given a higher weight than read operations. The time decay constant is determined based on factors such as the type of SSD particles used and the load during use.
[0058] Combining these factors, the Heat Index is designed to provide a balanced and comprehensive measure of data heat, based not only on frequency but also on data recency, the type of operations performed, and the impact on hardware lifespan. This multi-dimensional scoring approach can help the SSD's firmware manage data more intelligently, for example by retaining hot data in faster storage areas while migrating cold data to higher-endurance storage areas, thereby extending the overall lifespan of the device and improving performance.
[0059] For example, to calculate the heat index of a data block, read the weight factor ω r =1; write weight factor ω w = 8; time decay constant λ = 0.1; number of reads of the data block R = 200, number of writes W = 1, access frequency f = 0.01, time since the last access to the unit t = 20s, normalized normalized erase and write times e n =0.5. The access frequency is the total number of times the medium is accessed divided by the number of times the data block is accessed.
[0060] In summary, HI spb 1 =0.01×(1×200+8×1)×e -2 ×(1-0.5)=0.14.
[0061] In one embodiment, the binary stack includes a hot stack and a cold stack, and each data block is placed in the cold stack.
[0062] It should be noted that in the firmware technology of solid-state drives, it is often necessary to implement differentiated garbage collection and wear leveling strategies for storage units with different temperatures, which requires unified management of all storage units based on temperature.
[0063] Traditional hot and cold block management schemes use an unordered linear table to store the heat information of all data blocks, sorted by the unique index of the storage unit. This traditional scheme has the following limitations: 1. Inefficient search: In the worst case, to find the coldest or hottest information, the entire table may need to be traversed, with a time complexity of O(n). 2. Priority operations are not supported: unordered linear tables cannot support calls based on the hot and cold priority of data blocks.
[0064] To address the issues of traditional unordered linear table solutions, a hot / cold heap management strategy (HCH) based on heat index was designed. This strategy improves the SSD's management efficiency for all storage units with different heat indices, optimizes the search speed for cold and hot blocks, and more conveniently finds the data blocks with the highest or lowest heat. It also supports priority-based garbage collection and wear leveling strategies.
[0065] The hot and cold heap management strategy uses two binary heaps, the hot heap and the cold heap, to manage hot and cold data blocks, respectively. These heaps are organized as a max heap and a min heap, respectively. The nodes in the heaps store the unique index and heat index of the data block.
[0066] S102: Update each of the initial heat indexes within a preset update period to obtain a plurality of updated heat indexes.
[0067] In this embodiment, each time a data block is accessed, the following operations are performed: the read count or write count, the time since the last access to the block, and the access count for the data block are updated. When a controller becomes relatively idle, it calculates the access frequency of each data block using the previously updated read count, historical time, and access count, and updates the heat index for each data block.
[0068] It should be noted that the update cycle of the heat index needs to be set according to the actual usage scenario. For example, the update cycle of the garbage collection program and the heat index table should be synchronized, and the garbage collection program should be started after the heat index table is updated.
[0069] S103: Compare each of the update heat indexes with the hot and cold thresholds of the data block to obtain a plurality of comparison results.
[0070] It should be noted that the hot and cold thresholds need to be determined through field testing based on the primary SSD usage scenarios. Black box testing can be used to find the thresholds that optimize hot and cold data management performance. For example, if the heat index of a data block after an update is 0.14 and the hot and cold threshold is 10, the updated data block should be placed in the cold pile.
[0071] It is further explained that when the entire thermal index table is updated, the update of the hot and cold piles should be completed synchronously, and the timestamp of the most recent update should be recorded after the update is completed.
[0072] S104 : Determine the sorting position of each data block in the binary heap according to each comparison result and each update heat index.
[0073] It's important to note that when a data block's position in the heap needs to be adjusted, the heap's properties are maintained by sinking and floating operations. For example, if the heat index of a data block in the hot heap increases, it will move up in the hot heap, and vice versa. The same applies to decreasing heat indexes.
[0074] In one embodiment, each data block is placed into the corresponding binary heap according to each comparison result, and the corresponding data blocks are sorted in the binary heap according to the order of the update heat index.
[0075] In one embodiment, it is determined whether each of the update heat indexes is greater than the cold and hot thresholds; if the update heat index is greater than the cold and hot thresholds, the data blocks corresponding to each of the update heat indexes are moved from the cold pile to the hot pile; if the update heat index is less than the cold and hot thresholds, the corresponding data blocks are placed in the cold pile according to each of the update heat indexes.
[0076] It should be noted that if Figure 2 As shown in Figure 1, a hot heap is a max heap where the top of the heap holds the hottest data blocks, i.e., those with the highest heat index. Data blocks in the hot heap are sorted by their heat index, with blocks with higher heat indexes moving toward the top of the heap.
[0077] A cold heap is a minimum heap where the top of the heap holds the coldest data blocks, those with the lowest heat index. Data blocks in a cold heap are arranged according to their heat index, with blocks with lower heat indexes moving toward the top of the heap.
[0078] S105 , placing each of the data blocks into the corresponding binary heap according to the sorting position.
[0079] It should be noted that by monitoring the performance of the hot and cold piles, the parameters for calculating the thermal index and the hot and cold thresholds for distinguishing between the hot and cold piles are dynamically adjusted according to the workload and wear of the solid-state drive.
[0080] It is further explained that the present application has the following features and advantages: (1) Performance optimization: Through the hot and cold pile management strategy, rapid heat index search can be performed, enabling the solid-state drive to implement a priority-based dynamic garbage collection program under high load, improving read and write speeds and the overall performance of the solid-state drive. (2) Wear balancing: By reasonably allocating the weight of write operations in the hot and cold data evaluation, the wear of the solid-state drive is balanced and the service life is extended. (3) Adaptability: The ability to dynamically adjust parameters such as hot and cold thresholds and weight factors according to actual usage patterns can improve the ability of the solid-state drive to adapt to different workloads.
[0081] Overall, this solution provides an intelligent data management solution for solid-state drives, optimizing the physical layout and access efficiency of data by comprehensively analyzing data usage patterns and predicting future access trends.
[0082] In one embodiment, each of the data blocks to be moved is determined to obtain data nodes of multiple hot pile data blocks; the data nodes of each of the hot pile data blocks in the cold pile are deleted; the data nodes of each of the hot pile data blocks are added to the end of the hot pile, and a floating operation is performed on the data nodes of each of the hot pile data blocks.
[0083] For example, the heat index of a data block node in the cold pile rises and exceeds the cold and hot thresholds, and needs to be moved from the cold pile to the hot pile; delete the node from the cold pile: swap the node with the last node in the cold pile and delete the node, and perform a sinking operation on the new node now located at the original node position, which maintains the integrity of the pile while simplifying the deletion operation; add the node to the hot pile: add the node to the end of the hot pile and perform a floating operation. After the operation is completed, the hot pile will be rebuilt according to the properties of the maximum heap to ensure that the heat index of all parent nodes is higher than the heat index of their child nodes.
[0084] In one embodiment, the thermal pile data blocks are sorted according to the heat index; and it is determined whether each thermal pile data block is the thermal pile data block with the lowest heat index; if so, the thermal pile data block does not need to perform the floating operation.
[0085] S106: Execute a garbage collection program by periodically accessing the data block at the top of the binary heap.
[0086] It's important to note that the data block management approach proposed in this solution can integrate machine learning algorithms to automatically adjust scoring parameters, predict future data types and load conditions, and continuously optimize performance based on historical data and future forecasts. Strategies can identify different workload patterns, such as database operations and file server operations, and optimize parameters to suit specific patterns.
[0087] It is further explained that by executing the garbage collection program from the top of the binary heap, the present application can always obtain the hottest or coldest data at the fastest speed (time complexity is O(1)).
[0088] The data block management method provided in this embodiment determines the initial heat index of each data block according to the heat index calculation formula; updates each of the initial heat indexes within a preset update cycle to obtain multiple updated heat indexes; compares each of the updated heat indexes with the hot and cold thresholds of the data block to obtain multiple comparison results; determines the sorting position of each of the data blocks in the binary heap according to each of the comparison results and each of the updated heat indexes; places each of the data blocks in the corresponding binary heap according to each of the sorting positions; and executes a garbage collection program by periodically accessing the data blocks at the top of the binary heap. The heat index is obtained by calculation, and the data blocks are dynamically migrated in the binary heap in real time according to the heat index. The heat index search can be performed by managing the binary heap, thereby improving the overall read and write speed and overall performance, and extending the service life of the solid-state hard disk.
[0089] Example 2
[0090] In addition, an embodiment of the present application provides a data block management device.
[0091] like Figure 3 As shown, the data block management device 300 includes:
[0092] Determination module 301, for determining the initial heat index of each data block according to the heat index calculation formula;
[0093] An updating module 302 is configured to update each of the initial heat indexes within a preset update period to obtain a plurality of updated heat indexes;
[0094] A comparison module 303 is configured to compare each of the update heat indexes with the hot and cold thresholds of the data block to obtain a plurality of comparison results;
[0095] A sorting module 304 is configured to determine a sorting position of each data block in the binary heap according to each comparison result and each update heat index;
[0096] The placing module 305 is used to place each of the data blocks into the corresponding binary heap according to the sorting position;
[0097] The execution module 306 is configured to execute a garbage collection program by periodically accessing the data block at the top of the binary heap.
[0098] The data block management device 300 provided in this embodiment can implement the data block management method provided in Example 1, and will not be described again here to avoid repetition.
[0099] The data block management device provided in this embodiment determines the initial heat index of each data block according to the heat index calculation formula; updates each of the initial heat indexes within a preset update period to obtain multiple updated heat indexes; compares each of the updated heat indexes with the hot and cold thresholds of the data block to obtain multiple comparison results; determines the sorting position of each of the data blocks in the binary heap according to each of the comparison results and each of the updated heat indexes; places each of the data blocks in the corresponding binary heap according to each of the sorting positions; and executes a garbage collection program by periodically accessing the data blocks at the top of the binary heap. The heat index is obtained by calculation, and the data blocks are dynamically migrated in the binary heap in real time according to the heat index. The heat index search can be performed by managing the binary heap, thereby improving the overall read and write speed and overall performance, and extending the service life of the solid-state hard disk.
[0100] Example 3
[0101] In addition, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, the data block management method provided in Example 1 is executed.
[0102] For details, see Figure 4 The electronic device 400 includes: a transceiver 401, a bus interface and a processor 402, wherein the processor 402 is used to: determine the initial heat index of each data block according to a heat index calculation formula; update each of the initial heat indexes within a preset update period to obtain multiple updated heat indexes; compare each of the updated heat indexes with the hot and cold thresholds of the data block to obtain multiple comparison results; determine the sorting position of each of the data blocks in the binary heap according to each of the comparison results and each of the updated heat indexes; place each of the data blocks in the corresponding binary heap according to each of the sorting positions; and execute a garbage collection program by periodically accessing the data block at the top of the binary heap.
[0103] In one embodiment, the processor 402 is further configured to: calculate the heat index using the formula:
[0104] HI=f×(ω r ·R+ω w ·W)×e -λt ×(1-e n )
[0105] Where HI is the heat index, f is the access frequency, R is the number of reads, W is the number of writes, ω r To read the weight factor, ω w is the write weight factor, λ is the time decay constant, t is the historical time, e nis the normalized number of erase and write cycles.
[0106] In one embodiment, the processor 402 is further configured to: the binary heap includes a hot heap and a cold heap, and place each of the data blocks into the cold heap.
[0107] In one embodiment, the processor 402 is further configured to: place each data block into the corresponding binary heap according to each comparison result, and sort the corresponding data blocks in the binary heap in order of the update heat index.
[0108] In one embodiment, the processor 402 is further used to: determine whether each of the update heat indexes is greater than the cold and hot thresholds; if the update heat index is greater than the cold and hot thresholds, move each of the data blocks corresponding to each of the update heat indexes from the cold pile to the hot pile; if the update heat index is less than the cold and hot thresholds, place each of the corresponding data blocks in the cold pile according to each of the update heat indexes.
[0109] In one embodiment, the processor 402 is further used to: determine each of the data blocks to be moved to obtain data nodes of multiple hot pile data blocks; delete the data nodes of each of the hot pile data blocks in the cold pile; add the data nodes of each of the hot pile data blocks to the end of the hot pile, and perform a floating operation on the data nodes of each of the hot pile data blocks.
[0110] In one embodiment, the processor 402 is further configured to: sort the thermal stack data blocks according to the thermal index; determine whether each thermal stack data block is the thermal stack data block with the lowest thermal index; if so, the thermal stack data block does not need to perform the floating operation.
[0111] In the embodiment of the present application, the electronic device 400 further includes a memory 403. Figure 4 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits such as one or more processors represented by processor 402 and memory represented by memory 403. The bus architecture may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 401 may be a plurality of components, i.e., a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium. The processor 402 is responsible for managing the bus architecture and general processing, and the memory 403 may store data used by the processor 402 when performing operations.
[0112] The electronic device 400 provided in the embodiment of the present application can execute the steps of the data block management method provided in the above-mentioned method embodiment 1, and will not be described again here to avoid repetition.
[0113] The electronic device provided by this embodiment determines the initial heat index of each data block according to the heat index calculation formula; updates each of the initial heat indexes within a preset update period to obtain multiple updated heat indexes; compares each of the updated heat indexes with the hot and cold thresholds of the data block to obtain multiple comparison results; determines the sorting position of each of the data blocks in the binary heap according to each of the comparison results and each of the updated heat indexes; places each of the data blocks into the corresponding binary heap according to each of the sorting positions; and executes a garbage collection program by periodically accessing the data blocks at the top of the binary heap. The heat index is obtained by calculation, and the data blocks are dynamically migrated in the binary heap in real time according to the heat index. The heat index search can be performed by managing the binary heap, thereby improving the overall read and write speed and overall performance, and extending the service life of the solid-state hard disk.
[0114] Example 4
[0115] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the data block management method provided in Example 1 is implemented.
[0116] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0117] The computer-readable storage medium provided in this embodiment can implement the data block management method provided in Example 1, and will not be described again here to avoid repetition.
[0118] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.
[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0120] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.
Claims
1. A data block management method, characterized in that: The method comprises: Determine the initial heat index of each data block according to the heat index calculation formula; Updating each of the initial heat indexes within a preset update period to obtain a plurality of updated heat indexes; Comparing each of the update heat indexes with the hot and cold thresholds of the data block to obtain a plurality of comparison results; Determining a sorting position of each data block in a binary heap according to each comparison result and each update heat index; Putting each of the data blocks into the corresponding binary heap according to the sorting position; Performing a garbage collection procedure by periodically accessing the data block at the top of the binary heap; Determining the heat index of each data block includes: the heat index calculation formula is: in, is the heat index, is the access frequency, is the number of reads, is the number of writes, To read the weight factor, To write the weight factor, is the time decay constant, For historical time, is the normalized number of erase and write times; Determining the sorting position of each data block in the binary heap according to each comparison result and each update heat index includes: placing each data block in the corresponding binary heap according to each comparison result, and sorting the corresponding data blocks in the binary heap in order of the size of each update heat index.
2. The data block management method according to claim 1, characterized in that: The binary heap includes a hot heap and a cold heap. Before determining the initial heat index of each data block according to the heat index calculation formula, the following steps are performed: Each of the data blocks is placed in the cold stack.
3. The data block management method according to claim 2, characterized in that: The step of placing each data block into the corresponding binary heap according to each comparison result includes: Determining whether each of the update heat indexes is greater than the hot and cold thresholds; If the update heat index is greater than the cold / hot threshold, moving each data block corresponding to each update heat index from the cold stack to the hot stack; If the update heat index is less than the cold / hot threshold, the corresponding data blocks are placed in the cold pile according to the update heat index.
4. The data block management method according to claim 3, characterized in that: The moving the data blocks corresponding to the updated heat indexes from the cold stack to the hot stack includes: Determine each of the data blocks to be moved, and obtain data nodes of multiple thermal stack data blocks; Deleting the data nodes of each of the hot stack data blocks in the cold stack; The data nodes of each of the thermal pile data blocks are added to the end of the thermal pile, and a floating operation is performed on the data nodes of each of the thermal pile data blocks.
5. The data block management method according to claim 4, characterized in that: The performing a float operation on the data nodes of each of the thermal pile data blocks includes: sorting the thermal pile data blocks according to the thermal index; determining whether each of the thermal pile data blocks is the thermal pile data block with the lowest thermal index; If so, the thermal pile data block does not need to perform the float operation.
6. A data block management device, characterized in that: The device comprises: A determination module, configured to determine an initial heat index of each data block according to a heat index calculation formula; An updating module, configured to update each of the initial heat indexes within a preset update period to obtain a plurality of updated heat indexes; a comparison module, configured to compare each of the update heat indexes with the hot and cold thresholds of the data blocks to obtain a plurality of comparison results; A sorting module, configured to determine a sorting position of each data block in the binary heap according to each comparison result and each update heat index; an inserting module, configured to insert each of the data blocks into the corresponding binary heap according to the sorting position; an execution module, configured to execute a garbage collection program by periodically accessing the data block at the top of the binary heap; Determining the heat index of each data block includes: the heat index calculation formula is: in, is the heat index, is the access frequency, is the number of reads, is the number of writes, To read the weight factor, To write the weight factor, is the time decay constant, For historical time, is the normalized number of erase and write times; Determining the sorting position of each data block in the binary heap according to each comparison result and each update heat index includes: placing each data block in the corresponding binary heap according to each comparison result, and sorting the corresponding data blocks in the binary heap in order of the size of each update heat index.
7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program, the method for managing data blocks according to any one of claims 1 to 5 is executed.
8. A computer-readable storage medium, characterized in that The device stores a computer program, which executes the data block management method according to any one of claims 1 to 5 when the computer program is run on a processor.
Citation Information
Patent Citations
Cold and hot data management method and device, electronic equipment and computer readable medium
CN109407995A
Data processing method, device, equipment and medium
CN112286459A