Low-overhead data placement method and system

Abstract

The invention belongs to the field of data storage, and discloses a low-overhead data placement method and system. The method comprises the following steps: calculating a difference between an old logic address and a newly allocated logic address of a data block to obtain heat information of the data block; and placing the data block in a preset temperature region according to the heat value of the heat information, and storing the data block according to a unidirectional linear growth mode. According to the method, the difference between the logic old address and the newly allocated logic address during block updating in the log structure storage is used as the heat information, so that the global lock overhead is effectively reduced; and meanwhile, one-way linear growth is adopted for the temperature section group, so that the read-write performance is improved, the space and time overhead is remarkably reduced, the throughput is greatly improved, and the performance is superior to that of an existing data placement method.

Description

Technical Field

[0001] The present invention belongs to the field of data storage, and more specifically, relates to a low-overhead data placement method and system. Background Art

[0002] Log-structured storage has been widely used in today's mainstream storage fields, such as database management systems, distributed log systems, etc. It writes data to the storage medium in an append manner, avoiding the overhead of random writes and significantly improving the write performance. At the same time, through regular merge operations, multiple small log segments can be merged into larger and more efficient storage structures. However, this write method also brings the problem of write amplification. Since subsequent merge operations may need to merge multiple small segments into large segments, involving data copying and sorting, the amount of data actually written to the storage medium is much larger than the amount of data requested by the user to be written. This not only reduces the write performance but also accelerates the wear of the storage medium and shortens its lifespan, especially for flash memory devices, because flash memory needs to erase the old data block before writing, and the erase operation is costly.

[0003] In terms of data placement strategies, SEP BIT uses the skewness of write data in the storage workload to infer the invalid time (BIT) of each written block, and then places blocks with similar estimated invalid times into the same group. For example, data is divided into user data and garbage collection (GC) write data, and user data is classified and stored according to its lifespan, and GC write data is also classified according to its source and characteristics. MiDAS monitors the long-term trend of block updates by using the update interval distribution (UID) and the Markov chain-based analysis model (MCAM), and dynamically optimizes the number and size of groups according to the workload characteristics to reduce the number of valid block copies between segments.

[0004] Although existing data placement algorithms such as SEP BIT and MiDAS help to reduce write amplification to a certain extent, there are still storage and performance loss problems. Specifically, the existing data placement strategies need to allocate additional internal and external memory spaces to store and manage the heat information, which brings additional storage overhead. In addition, during each identification process, the heat information needs to be indexed to achieve operations such as addition, deletion, modification, and query, which harms the system performance. Summary of the Invention

[0005] In view of the above defects or improvement requirements of the prior art, the present invention provides a low-overhead data placement method and system, aiming to use the difference between the logical old address and the newly allocated logical address of data block writing in log-structured storage as heat information, effectively reducing the global lock overhead, and replacing the heat storage overhead in memory with negligible computational overhead, thereby solving the technical problems of low traditional read and write performance and low resource utilization rate.

[0006] To achieve the above object, according to one aspect of the present invention, a low-overhead data placement method is provided. The method includes:

[0007] Obtain a data block, and calculate the difference between the logical old address and the newly allocated logical address of the data block to obtain the heat information of the data block.

[0008] Place the data block in a preset temperature region according to the heat value of the heat information, and store it in a unidirectional linear growth manner.

[0009] Further, the method further includes:

[0010] Set a buffer region; the buffer region is between two adjacent preset temperature regions;

[0011] Obtain the remaining space of each preset temperature region. When the remaining space of the preset temperature region is less than a first preset value, allocate the space adjacent to the end position of the preset temperature region in the buffer region to the preset temperature region.

[0012] Further, the method further includes: recycling the victim segment of the preset temperature region, including automatically enabling recycling or forcibly triggering the enabling of recycling; wherein, the victim segment starts from the logical start segment of the preset temperature region, analyzes and determines the failure degree of the data block using a Bayesian model, and determines it in descending order of the failure degree.

[0013] Further, the automatic enabling of recycling includes: when there is no IO operation for a preset time, recycle the victim segment of the preset temperature region.

[0014] Further, the forced triggering of enabling recycling includes: scanning each preset temperature region. When the remaining space of the preset temperature region is less than a second preset value, recycle the victim segment of the preset temperature region.

[0015] Further, when the unidirectional linear growth method is a rollback unidirectional linear growth method, it includes:

[0016] When there is no remaining space at the end position of the preset temperature region, start from the logical start segment of the preset temperature region, and sequentially append and store newly written data blocks at idle positions.

[0017] Further, the data block is an initially written data block or an updated written data block; when the data block is an initially written data block, the logical old address of the data block is defined as infinite.

[0018] Further, the method further includes: initializing the log-structured storage during initial operation, including at least dividing a preset temperature region and determining a placement rule for the preset temperature region.

[0019] Further, the dividing of the preset temperature region further includes:

[0020] Dividing the preset temperature region according to the historical load information data of the log-structured storage.

[0021] According to another aspect of the present invention, there is provided a low-overhead data placement system, including:

[0022] An initialization module for initializing the log-structured storage and dividing a preset temperature region;

[0023] An acquisition module for acquiring a data block and calculating the difference between the logical old address and the newly allocated logical address of the data block to obtain the heat information of the data block;

[0024] A placement module for placing the data block in the preset temperature region according to the heat value of the data block and storing it in a unidirectional linear growth manner.

[0025] Generally speaking, compared with the prior art by the above technical solutions conceived by the present invention, the following technical advantages are mainly possessed:

[0026] (1) The present invention utilizes native information and relies only on the logical address changes generated by the log-structured storage system itself during block updates to obtain heat information. Compared with the traditional method of using a global clock to maintain the distance difference, the present invention avoids the additional heat information storage overhead and query overhead brought by the traditional method. The present invention uses the difference between the logical old and new addresses to represent the heat information, which coincides with the idea of the global clock. Based on the fact that the system returns the difference between the old and new addresses of the block during each I / O, when a block is written, only the difference between these two values needs to be calculated to represent the usage frequency of the information and distinguish the different heats of the blocks; at the same time, combined with the unidirectional linear growth method, it ensures the sequential append of each temperature region. The strict sequential append writing is an effective guarantee for the heat information. Therefore, the heat information management mechanism is simplified by the method of the present invention, avoiding the resource consumption and the increase in system complexity caused by maintaining additional variables and clocks, reducing the global lock and heat storage overhead, and reducing the system performance loss.

[0027] (2) The present invention has an orderly temperature region management and dynamic boundary adjustment, improving the utilization rate of storage resources. This dynamic adjustment mechanism enables the storage system to automatically optimize the storage structure according to the actual load situation and adapt to the needs of different application scenarios and different loads.

[0028] (3) The improved GC strategy of the present invention selects the victim segment from the logical starting segment, and combines the heat information to ensure that the invalidation degree of data blocks decreases, thereby improving the garbage collection efficiency. Specifically, the victim segment selection mechanism based on heat information and Bayesian model ensures that the garbage collection operation can preferentially process the blocks with a high invalidation degree, improving the accuracy and efficiency of garbage collection and reducing the impact on valid data.

[0029] (4) The data blocks of the present invention grow in a strict rollback unidirectional linear growth manner, enabling the logical address in each temperature region to grow infinitely, ensuring the orderliness and continuity of data within the temperature segment group, enabling read and write operations to quickly locate according to the logical address, improving the response speed of the storage system and the read and write performance; at the same time, by reducing the time for data search and positioning, optimizing the data read and write operation process, and enhancing the data access efficiency.

[0030] (5) The present invention effectively reduces the write amplification problem, reduces the number of write operations to the storage medium, and prolongs the life of the storage medium, especially suitable for flash memory devices. Through precise data placement and an efficient garbage collection strategy, unnecessary data replication and rewriting are reduced, the number of erasures of the flash memory is decreased, thereby prolonging its service life. Brief Description of the Drawings

[0031] Figure 1 It is a flowchart of a low-overhead data placement method provided by an embodiment of the present invention.

[0032] Figure 2 It is a schematic diagram of native information provided by an embodiment of the present invention.

[0033] Figure 3 It is a schematic diagram of the architecture of a method for reducing write amplification without loss of performance for log-structured storage provided by an embodiment of the present invention. Detailed Embodiments

[0034] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0035] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0036] To solve the technical problems in log-structured storage that the traditional data placement and management methods lead to serious write amplification problems, and reduce the read / write performance and resource utilization rate of the storage system, the present invention provides a low-overhead data placement method. The overall idea is as follows: making full use of the characteristics of the log-structured storage system itself, using the difference between the logical old address and the newly allocated logical address of the data block write as the heat information (Native_info), and replacing the traditional heat storage overhead with computational overhead in memory, so as to reduce the performance loss caused by the global lock; at the same time, adopting a strict unidirectional linear growth placement and storage for the preset temperature region, so as to optimize data placement and management, avoid unnecessary data replication and rewriting, reduce write amplification, and further improve the read / write performance and resource utilization rate of the storage system.

[0037] To achieve the above object, in the first aspect of the present invention, as Figure 1 shown, a low-overhead data placement method is provided, including the following steps: obtaining a data block, and calculating the difference between the logical old address and the newly allocated logical address of the data block to obtain the heat information of the data block;

[0038] Placing the data block in the preset temperature region according to the heat value of the data block, and storing it in a unidirectional linear growth manner.

[0039] The difference between the logical addresses generated by the log-structured storage system itself during block update is used as the change distance difference to reflect how often this information is used. Through this interval, the usage frequency of this information is represented, and it is used to distinguish the different heats of the blocks. As Figure 2The left side shows the writing and storage of data blocks by traditional methods; the right side shows the method of the present invention. Specifically, when a block written by a user is updated, the system automatically records the logical old address and the newly allocated logical address, calculates the popularity information based on the change of the old and new logical addresses, and does not require a complex popularity information collection and maintenance mechanism, reducing the consumption of system resources. It should be noted that when this solution is applied to the writing of initial data blocks, since there is no logical old address for the initial data blocks, the logical old address of the initial data blocks is defined as infinity for calculation.

[0040] In some embodiments, the method of the present invention further includes the application of a buffer area. Specifically, the low-overhead data placement method of the present invention further includes:

[0041] Scanning each preset temperature region to obtain the remaining space of each preset temperature region;

[0042] When the remaining space of the preset temperature region is less than a first preset value, the space adjacent to the end position of the preset temperature region in the buffer area is allocated to the preset temperature region. Among them, the buffer area is determined during the initialization of the log-structured storage and is placed between two adjacent preset temperature regions.

[0043] Specifically, the logical region boundaries of each temperature region are dynamically managed by the buffer area to flexibly respond to different loads and dynamically adjust the partition size of the corresponding temperature region. Specifically: A buffer area is set between two adjacent preset temperature regions, and the boundary between the buffer area and the next log segment is dynamically adjusted according to the load situation. For example, at high load, the buffer area is appropriately expanded to accommodate more possible updated data and avoid frequent adjustment of the log segment boundary; at low load, the buffer area is reasonably reduced to release storage resources and improve storage utilization.

[0044] In some embodiments, the method of the present invention further includes the application of a data block recycling mechanism. Specifically, the victim segments of the preset temperature region are recycled, including automatically enabling recycling or triggering the enabling of recycling; among them, the victim segments are determined in sequence from high to low according to the failure degree by analyzing the failure degree of the data blocks starting from the logical start segment of the preset temperature region using a Bayesian model.

[0045] It should be noted that the Native_info of the data block and other relevant factors are analyzed using a Bayesian model to determine its failure degree, and the victim segments are defined according to the failure degree. Other relevant factors here include the last access time, update frequency, etc.

[0046] In the garbage collection (GC) strategy improved by the present invention, the victim segment selection starts strictly from the logical starting segment of each temperature region. By using the Bayesian model and experimental verification, while the heat information is sufficiently representative, it is ensured that the block failure degree of each temperature segment group decreases from front to back. When selecting the victim segment, factors such as heat information and the position of the block in the temperature region are comprehensively considered. Through the analysis of heat information by the Bayesian model, the failure possibility of the data block can be judged more accurately. Starting from the logical starting segment of the temperature region, it is ensured that the blocks with a high failure degree are preferentially recycled, thus improving the garbage collection efficiency.

[0047] In this embodiment, the automatic enabling of recycling includes: scanning each preset temperature region, and when there is no IO operation for a preset time, recycling the victim segments of the preset temperature region. That is, within a certain time period, when the system of the present invention is idle, the automatic recycling mechanism is triggered, so that the victim segments in the preset temperature region perform recycling operations.

[0048] In this embodiment, the triggered enabling of recycling includes: scanning each preset temperature region, and when the remaining space of the preset temperature region is less than the second preset value, recycling the victim segments of the preset temperature region.

[0049] Among them, the first preset value is greater than the second preset value, which indicates that it is preferred to replenish the space through the buffer mechanism.

[0050] In some embodiments, the data blocks are stored in a one-way linear growth manner, preferably through a rollback one-way linear growth manner, specifically including:

[0051] When there is no remaining space at the end position of the preset temperature region, starting from the logical starting segment of the preset temperature region, the newly written data blocks are sequentially stored in the idle positions. Or, when there is no remaining space at the end position of the preset temperature region and when the remaining space of the preset temperature region is less than the second preset value, first recycle the victim segments of the preset temperature region, and store the newly written data blocks at the corresponding positions of the victim segments. Specifically, a strict rollback one-way linear growth is adopted for each temperature region, so that the logical addresses of each temperature region grow infinitely. This growth method ensures the orderliness and continuity of the data blocks in the temperature region, which is beneficial to improving the read and write performance. The data blocks are written in sequence within the temperature region, which is convenient for quick positioning and reading, reduces the data search time, and optimizes the read and write operation processes.

[0052] In some embodiments, before the implementation of the entire invention, a log-structured storage device is determined and the device system is initialized, preset temperature regions are divided, and the placement rules of the preset temperature regions are determined;

[0053] Initialize the log-structured storage and divide the preset temperature regions. When dividing the preset temperature regions, it includes determining the types, numbers, sizes of each temperature region, and the placement rules of the preset temperature regions. The placement rules of the preset temperature regions are used to place the heat information of data blocks according to these rules in the subsequent process, that is, according to the heat value of the heat information and the placement rules of the preset temperature regions, place the data blocks in the preset temperature regions and store them in a unidirectional linear growth manner.

[0054] In this embodiment, divide the preset temperature regions according to the historical load information data of the log-structured storage. Specifically: after the previous system ends its operation, the operation and maintenance personnel can dynamically adjust the basic parameters before the next system starts running according to the previous historical load ratio. For example, the sizes and boundaries of each temperature region, the size of the buffer region, and the threshold for performing garbage collection operations, etc. Optimize the entire framework structure adaptively to further optimize the storage efficiency.

[0055] It should be noted that during the operation of the system, regularly check and optimize the stored data. For example, in the application of the above-mentioned recycling mechanism and buffer region, it is necessary to periodically scan the corresponding preset temperature region or buffer region, obtain the corresponding information, and then make adjustments in combination with the buffer mechanism and / or recycling mechanism. And in the division of temperature regions, regularly perform periodic scans on the stored data and storage space.

[0056] In the second aspect of the present invention, a low-overhead data placement system is provided, which is characterized by including:

[0057] An initialization module for initializing the log-structured storage and dividing the preset temperature regions;

[0058] An acquisition module for acquiring data blocks, and calculating the difference between the logical old address and the newly allocated logical address of the data blocks to obtain the heat information of the data blocks;

[0059] A placement module for placing the data blocks in the preset temperature regions according to the heat value of the data blocks and storing them in a unidirectional linear growth manner.

[0060] Furthermore, the method of the present invention is applied to data placement in a log-structured storage system, especially focusing on the evaluation of data placement based on the log file system (f2fs) and flash-based solid-state drives (ssd). In actual applications, it is optimized according to the characteristics of the f2fs file system and ssd storage devices. For example, consider the file organization structure and metadata management method of the f2fs file system, and the flash writing characteristics of the ssd, so that the algorithm can better adapt to different storage environments.

[0061] When implementing the method of the present invention, the sources of database blocks include at least the following two implementation manners. The following are specific embodiments.

[0062] Embodiment 1:

[0063] The implementation process of a data cold and hot separation method with low storage overhead (NDNE) includes:

[0064] As Figure 3 shown, when the log-structured storage system is initialized, determine the temperature regions for storing data blocks, and set the initial parameters of each temperature region, such as the number / types of temperature segment regions, the initial size of each temperature region, the growth rule, etc. In the figure, starting from segment 0, segments 0 to v are hot temperature regions, segments m to n are cold temperature regions, and segments s to k are warm temperature regions; at the same time, determine the memory regions for storing heat information and related data management structures.

[0065] When the system obtains an initially written or updated written data block, the system automatically records the logical old address and the newly allocated logical address of the data block, and calculates the difference between the two to obtain the heat information.

[0066] Based on the heat value of each data block, place the data block into the corresponding temperature region. For example, divide the temperature segment region into a low-temperature region, a medium-temperature region, and a high-temperature region. Set the heat value to be less than the threshold T1, and place the data block into the low-temperature region; if it is between T1 and T2 (T2 > T1), place it into the medium-temperature region; if it is greater than T2, place it into the high-temperature region.

[0067] Within each temperature region, the data blocks are stored in a strict rollback-style unidirectional linear growth manner. The new data blocks are appended after the existing data blocks in sequence to ensure the logical order of the data within the temperature region.

[0068] In terms of the logical region boundary management of the temperature region, adopt a buffer dynamic management mechanism. In the case of low load, appropriately reduce the buffer region to reduce unnecessary memory occupation; in the case of high load, such as when a large amount of data updates are concentrated in a certain temperature segment group, expand the buffer region to accommodate more possible updated data and avoid frequent adjustment of the temperature region boundary.

[0069] When garbage collection (GC) operations are required, strictly select the victim segments starting from the logical start segment of each temperature region, and at the same time use the Bayesian model to analyze the heat information of the data blocks and other related factors to determine their failure degree. Give priority to selecting the data blocks with a high failure degree as the victim segments for recycling. After recycling, readjust the logical addresses of the remaining data blocks within the temperature region to maintain the unidirectional linear growth characteristic.

[0070] In this embodiment, it further includes: during the operation of the system, regularly check and optimize the stored data. The specific steps are as follows:

[0071] (S1) Scan each temperature segment group to obtain the distribution of data blocks and the statistical information of heat information, and understand whether the distribution of data blocks in different temperature segment groups is reasonable and the distribution trend of heat information.

[0072] (S2) According to the scanning results, determine whether it is necessary to adjust the temperature region division strategy or the size of the boundary buffer. If it is found that the data volume in a certain temperature region grows too fast or too slow, it may be necessary to re-divide the range of the temperature segment group; if the buffer utilization rate is too high or too low, adjust the buffer size accordingly.

[0073] (S3) Check the execution effect of the garbage collection strategy, evaluate whether the utilization rate of storage resources has increased after recycling, and whether there is over-recycling or under-recycling. If there are problems, adjust the parameters in the Bayesian model and optimize the victim segment selection strategy.

[0074] (S4) According to the performance metrics of the system (such as read / write latency, throughput, etc.), determine whether it is necessary to adjust other parameters in the algorithm (such as heat information threshold, temperature segment group growth step, etc.) to further optimize the system performance.

[0075] Generally speaking, this embodiment can effectively solve the technical problems of serious write amplification, low read / write performance, and low resource utilization rate caused by the existing data placement and management strategies in log-structured storage. While ensuring the effectiveness and reliability of data storage, it reduces the write amplification factor, reduces the system resource overhead, and significantly improves the overall performance of the log-structured storage system.

[0076] Embodiment 2:

[0077] The implementation process of a data cold and hot separation method with low storage overhead (NDNE) includes: deploying on multiple client nodes to perform read and write operations on files simultaneously;

[0078] System initialization and configuration. When the distributed file system starts, the server side initializes the number, size range, and initial growth parameters of the temperature regions according to the expected load of the system and the characteristics of the storage device. For example, set the initial size of the low-temperature region to 100MB, the medium-temperature region to 200MB, the high-temperature region to 300MB, and set the growth step of the low-temperature region to 10MB, the medium-temperature region to 20MB, and the high-temperature region to 30MB. At the same time, allocate space in the server memory for storing heat information and related data structures (such as temperature region index table, block status record table, etc.).

[0079] When the client node establishes a connection with the server, it obtains the configuration information of the server regarding temperature region division and data management policies, so as to perform corresponding data processing and request sending locally.

[0080] Data block writing and heat information calculation (client operation). When the client node writes a new file data block to the distributed file system, assume that client A writes a data block with a size of 4KB, and the initial logical address assigned by the file system for it is 5000. Subsequently, this data block is updated due to content modification, and the new logical address becomes 5050. Client A locally records the old logical address 5000 and the newly assigned logical address 5050 of this data block, and calculates the heat information as 5050 - 5000 = 50.

[0081] Based on the heat value (50) of client A and the temperature segment group division rule obtained from the server, it is determined that this data block should belong to the medium-temperature region. Then, client A sends a write request containing the data block content, logical address information, and heat information to the server side.

[0082] Data placement and temperature region management (server operation). After the server side receives the write request from client A, it places the data block into the medium-temperature region. Within the medium-temperature region, in accordance with a strict rollback-style unidirectional linear growth method, the new data block is sequentially stored after the existing data blocks. For example, if the logical address range of the existing data blocks in the medium-temperature region is currently 500 - 540, then the new data block (logical address 5050) will be sequentially stored in the appropriate position.

[0083] The server side simultaneously monitors the load conditions of each temperature region, including the number of data blocks, data volume size, and data update frequency, etc. Assume that the data in the medium-temperature region is updated frequently within a period of time and the load gradually increases. According to the buffer dynamic management policy, the server side increases the buffer size of the medium-temperature region from the default 5MB to 8MB to accommodate more possible update operations and avoid frequently adjusting the temperature segment group boundary.

[0084] Garbage collection (GC) operation (server operation). When the server side detects that the storage resource utilization rate reaches a certain threshold or meets the predetermined GC trigger condition, it starts the garbage collection process. First, it analyzes the data blocks starting from the logical start segment of each temperature region. For example, for the medium-temperature region, it starts from the data block with the lowest logical address.

[0085] The server - side uses a Bayesian model to comprehensively consider factors such as the heat information of data blocks, the last access time (recorded in the block status record table), and the relative position within the temperature region, etc., to evaluate the failure degree of each data block. Suppose the data block at the logical start segment of the medium - temperature region, although its heat information is within the medium - temperature range, has not been accessed for a long time (the last access time is more than 1 hour ago), while a subsequent data block has slightly higher heat information but has been frequently accessed recently (the last access time is within 10 minutes). Then, the data block at the start segment that has not been accessed for a long time is preferentially selected as the victim segment for recycling.

[0086] After recycling the victim - segment data block, the server - side re - adjusts the logical addresses of the remaining data blocks within the medium - temperature region to ensure that the data - block order still conforms to the strict one - way linear growth characteristic. At the same time, relevant data structures (such as the temperature - segment - group index table, block status record table, etc.) are updated to reflect the new state of the data blocks.

[0087] Data reading and system optimization (client - server interaction), when client B needs to read the data blocks of a certain file, it sends a read request containing the file identifier and logical - address range to the server - side. The server - side quickly locates the data blocks in the corresponding temperature region according to the request. Since the data is stored in order within the temperature region and there is heat information to assist in management, the server - side can efficiently find the target data block and return it to client B.

[0088] During the operation of the system, the server - side regularly (for example, every hour) conducts performance evaluation and optimization of the entire distributed file system. It scans the data distribution in each temperature region, checks whether the distribution of heat information is reasonable, and whether the size of the temperature region and buffer settings are suitable for the current load. If it is found that the heat information distribution of the data blocks in a certain temperature region is too concentrated in a certain interval, it may mean that the temperature - region division is not fine enough. The server - side will consider adjusting the temperature - region division strategy, such as increasing the number of temperature - segment groups or adjusting the threshold. At the same time, if the buffer utilization rate of a certain temperature - segment group is lower than 20% for a long time, the buffer size is appropriately reduced to release memory resources.

[0089] According to the storage - resource utilization rate and system - performance indicators (such as read - write request response time, throughput, etc.) after garbage collection, the server - side may adjust the GC trigger condition (such as lowering the resource - utilization threshold to perform GC more frequently) or optimize the parameters in the Bayesian model to improve the garbage - collection efficiency and the overall system performance.

[0090] In this embodiment, for the specific implementation manners of each module, reference can be made to the description in Embodiment 1 above, and will not be repeated here.

[0091] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of protection of the present invention and the scope of equivalent technologies thereof, the present invention also intends to include these modifications and variations. The above-described embodiments are only preferred embodiments given to fully illustrate the present invention, and the scope of protection is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the scope of protection of the present invention.

Claims

1. A low-overhead data placement method, characterized in that The method includes: Obtain a data block, and calculate the difference between the logical old address and the newly allocated logical address of the data block to obtain the heat information of the data block; Place the data block in a preset temperature region according to the heat value of the heat information, and store it in a one-way linear growth manner.

2. The low-overhead data placement method according to claim 1, wherein The method further includes: Set a buffer region; the buffer region is between two adjacent preset temperature regions; Obtain the remaining space of each preset temperature region. When the remaining space of the preset temperature region is less than a first preset value, allocate the space adjacent to the end position of the preset temperature region in the buffer region to the preset temperature region.

3. The low-overhead data placement method according to claim 1, characterized in that The method further includes: recycling the victim segment of the preset temperature region, including automatically enabling recycling or forcibly triggering the enabling of recycling; wherein, the victim segment starts from the logical starting segment of the preset temperature region, analyzes and determines the failure degree of the data block using a Bayesian model, and determines it in descending order of the failure degree.

4. The low-overhead data placement method according to claim 3, wherein The automatic enabling of recycling includes: when there is no IO operation for a preset time, recycle the victim segment of the preset temperature region.

5. The low-overhead data placement method according to claim 3, wherein The forced triggering of enabling recycling includes: scanning each preset temperature region. When the remaining space of the preset temperature region is less than a second preset value, recycle the victim segment of the preset temperature region.

6. The low-overhead data placement method according to claim 1, wherein When the one-way linear growth manner is a rollback one-way linear growth manner, it includes: When there is no remaining space at the end position of the preset temperature region, start from the logical starting segment of the preset temperature region and sequentially append and store newly written data blocks at idle positions.

7. The low-overhead data placement method according to claim 1, characterized in that The data block is an initially written data block or an updated written data block; when the data block is an initially written data block, the logical old address of the data block is defined as infinite.

8. The low-overhead data placement method according to claim 1, characterized in that The method further includes: during initial operation, initialize the log-structured storage, at least including dividing the preset temperature region and determining the placement rule of the preset temperature region.

9. The low-overhead data placement method according to claim 8, wherein The dividing of the preset temperature region further includes: Divide the preset temperature region according to the historical load information data of the log-structured storage.

10. A low-overhead data placement system, characterized in that, It includes: An initialization module, used to initialize the log-structured storage and divide the preset temperature region; An acquisition module, used to obtain a data block, and calculate the difference between the logical old address and the newly allocated logical address of the data block to obtain the heat information of the data block; A placement module, used to place the data block in a preset temperature region according to the heat value of the heat information, and store it in a one-way linear growth manner.

Images