Data storage method and storage system

Abstract

The present invention provides a data storage method and a storage system, which relate to the technical field of data storage. The method is applied to a controller in a storage system, and the storage system also includes a memory. The controller is used to use the memory in a host as a host memory buffer, and receive read and write requests for data stored in the memory, including the following steps: reading a first replacement weight of first data; reading a second replacement weight of second data; comparing the first replacement weight and the second replacement weight; if the first replacement weight is greater than the second replacement weight, storing the first data in the host memory buffer and deleting the second data from the host memory buffer. The above method replaces low-access frequency data in the host memory buffer with high-access frequency data, which is beneficial to improving the data reading speed and reducing the number of accesses to the memory.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a data storage method and a storage system. Background Art

[0002] HMB (Host Memory Buffer) is a technology used in storage systems that allows RAID (Redundant Array of Independent Disks) cards or SSD (Solid State Drive) controllers to use part of the system memory as cache to improve data access performance. By using host memory as cache, HMB technology can expand the cache capacity of storage devices, thereby accelerating data read and write operations.

[0003] In the prior art, the cache replacement method in the HMB is usually to delete the earliest data stored in the HMB when insufficient storage space is detected in the HMB, freeing up cache space to store data. This method will cause the data that users frequently read to be deleted, while the newly stored data is only occasionally read by the user. Summary of the invention

[0004] The present invention provides a data storage method and a storage system, which perform cache replacement according to the number of requests, timely eliminate data with low access frequency from the cache, timely release cache space, and timely add data with high access frequency into the cache, which is beneficial to improving the data reading speed and reducing the number of accesses to the memory.

[0005] The present invention provides a data storage method, which is applied to a controller in a storage system, wherein the storage system also includes a memory, and the controller is used to use the memory in a host as a host memory buffer, and receive read and write requests for data stored in the memory, including the following steps: reading a first replacement weight of first data, wherein the first data is only stored in the memory, and the replacement weight is calculated based on the number of requests for the data, and the replacement weight is positively correlated with the number of requests; reading a second replacement weight of second data, wherein the second data is stored in the host memory buffer and the memory; comparing the first replacement weight and the second replacement weight; if the first replacement weight is greater than the second replacement weight, storing the first data in the host memory buffer and deleting the second data from the host memory buffer.

[0006] According to a data storage method provided by the present invention, the method further includes: updating the replacement weight of each data according to the access time of the data, and the access time is the time when the data was last accessed.

[0007] According to a data storage method provided by the present invention, the replacement weight of each data is updated according to the access time of the data, including: calculating the time difference of each data, wherein the time difference is the difference between the access time and the current time; calculating the current replacement weight of each data according to the time difference and the read and write performance parameters of the host memory buffer, wherein the current replacement weight is the replacement weight of the data at the current moment; and modifying the historical replacement weight of each data to the current replacement weight, wherein the historical replacement weight is the replacement weight of the data before the current moment.

[0008] According to a data storage method provided by the present invention, a replacement weight of each data is calculated based on a time difference and the read-write parameters of the memory, including: determining an update period based on the read-write parameters of the memory; dividing the time difference by the update period to obtain an update value of each data; and obtaining a current replacement weight of each data based on the update value of the data and a historical replacement weight.

[0009] According to a data storage method provided by the present invention, the current replacement weight of each data is obtained according to the update value of the data and the historical replacement weight, including: obtaining the gain multiple of the data according to the threshold interval in which the update value of the data is located, and the threshold interval is preset; subtracting the product of the gain multiple and the update value from the historical replacement weight of the data to obtain the current replacement weight of each data.

[0010] According to a data storage method provided by the present invention, the method further comprises: removing data with a current replacement weight less than 0 in the host memory buffer.

[0011] According to a data storage method provided by the present invention, the method also includes: receiving a read request; determining read data based on the read request; judging whether the read data is stored in the host memory buffer; if the read data is not stored in the host memory buffer, obtaining the read data from the memory; judging whether the host memory buffer can store the read data; if the host memory buffer can store the read data, storing the read data in the host memory buffer; calculating a replacement weight for the read data, and saving the replacement weight for the read data.

[0012] According to a data storage method provided by the present invention, the method also includes: if the host memory buffer cannot store the read data, determining whether the read data has been marked as the second data; if the read data has been marked as the second data, updating the replacement weight of the read data; if the read data has not been marked as the second data, marking the read data as the second data, and calculating and saving the replacement weight of the read data.

[0013] According to a data storage method provided by the present invention, the method also includes: receiving a write request; determining write data based on the write request, the write data being data written in the memory when the write request is executed; judging whether historical data corresponding to the write request is stored in the host memory buffer, the historical data being data modified when the write request is executed; if the historical data is already stored in the host memory buffer, modifying the historical data to the write data; calculating a replacement weight based on the write data, and saving the replacement weight of the write data.

[0014] According to a data storage method provided by the present invention, the method also includes: if the historical data is not stored in the host memory buffer, determining whether the host memory buffer is capable of storing the write data; if the host memory buffer is capable of storing the write data, storing the write data in the host memory buffer; calculating a replacement weight based on the write data, and saving the replacement weight of the write data.

[0015] According to a data storage method provided by the present invention, the method also includes: if the host memory buffer cannot store the write data, obtaining second data with a minimum replacement weight; if the minimum replacement weight is greater than a preset threshold, deleting the minimum replacement weight from the host memory buffer and storing the write data in the host memory buffer; if the minimum replacement weight is less than the preset threshold, marking the write data as the first data; calculating the replacement weight of the write data, and saving the replacement weight of the write data.

[0016] According to a data storage method provided by the present invention, the method also includes: if the host memory buffer cannot store the write data, determining whether the write data has been marked as the second data; if the write data has been marked as the second data, updating the replacement weight of the write data; if the write data has not been marked as the second data, marking the write data as the second data, and calculating and saving the replacement weight of the write data.

[0017] According to a data storage method provided by the present invention, the method also includes: if the read / write request has no corresponding historical data, determining whether the host memory buffer is capable of storing the write data; if the host memory buffer is capable of storing the write data, storing the write data in the host memory buffer; calculating a replacement weight based on the write data, and saving the replacement weight of the write data.

[0018] According to a data storage method provided by the present invention, a replacement weight of data is obtained from a data structure based on a data identifier, wherein the data structure is pre-set, and the data structure is used to store a data identifier and a replacement weight of each data, and the data identifier is a unique identifier of the data in the storage system.

[0019] According to a data storage method provided by the present invention, the data structure includes a first data structure and a second data structure, the first data structure is used to store the data identification and replacement weight of the first data, and the second data structure is used to store the data identification and replacement weight of the second data.

[0020] According to a data storage method provided by the present invention, the data structure includes multiple data nodes, each data node stores a data identifier and a replacement weight of a data, each data node in the second data structure is associated with a storage address, the storage address is a data storage unit of the data corresponding to the data identifier in the data node in a host memory buffer, and the data identifier is a logical block address of the data in the memory.

[0021] The present invention also provides a storage system, which includes a controller and a memory, wherein the controller uses the memory in the host as a host memory buffer, and receives a read request for data stored in the memory; the controller is configured to read a first replacement weight of first data, the first data is only stored in the memory, the replacement weight is calculated based on the number of requests for the data, and the replacement weight is positively correlated with the number of requests; read a second replacement weight of second data, the second data is stored in the host memory buffer and the memory; compare the first replacement weight and the second replacement weight; if the first replacement weight is greater than the second replacement weight, store the first data in the host memory buffer and delete the second data from the host memory buffer.

[0022] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned data storage methods when executing the computer program.

[0023] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any of the data storage methods described above is implemented.

[0024] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned data storage methods.

[0025] The data storage method and storage system provided by the present invention calculate the replacement weight of data according to the number of requests, compare the replacement weights of first data and second data, and when the replacement weight of the first data is greater than the replacement weight of the second data, implement cache replacement according to the number of requests, timely eliminate data with low access frequency from the cache, release cache space in time, and timely add data with high access frequency into the cache, thereby realizing the replacement of low access frequency data with high access frequency data in cache replacement, and replacing low access frequency data in the host memory buffer with high access frequency data is beneficial to improving the data reading speed and reducing the number of accesses to the memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 It is a schematic diagram frame of the storage system and host provided by the present invention.

[0028] Figure 2 It is a flow chart of the data storage method provided by the present invention.

[0029] Figure 3 It is a structural schematic diagram of the storage system provided by the present invention.

[0030] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Combine the following Figure 1-Figure 4 The data storage method and storage system of the present invention are described.

[0033] Figure 1 is a schematic diagram of a first storage system 100 and a host 200 provided by the present invention, such as Figure 1As shown, the first storage system 100 includes the following: a first controller 110 and a first memory 120; preferably, the first memory 120 is a non-volatile memory composed of one or more non-volatile memory tube cores. As described in the present invention, the term tube core refers to a collection of non-volatile memory cells formed on a single semiconductor substrate, and associated circuits for managing the physical operations of those non-volatile memory cells. The first controller 110 interacts with the host and transmits command sequences for read operations, program operations, and erase operations to the non-volatile memory tube cores.

[0034] The first controller 110, which may be a non-volatile memory controller, such as a flash memory, a resistive random access memory (ReRAM), a phase change memory (PCM), or a magnetoresistive random access memory (MRAM) controller), may take the following forms: for example, a processing circuit, a microprocessor or a processor, and a computer-readable medium storing a computer-readable program code (such as firmware) executable by a (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. The first controller 110 may be configured with hardware and / or firmware to perform the various functions described below and shown in the flow chart. In addition, some components shown as being inside the controller may also be stored outside the controller, and other components may be used. In addition, the phrase "operably communicates with..." may mean communicating directly or indirectly (wired or wirelessly) with one or more components, communicating through one or more components, which may or may not be shown in the present invention. In one embodiment of the present invention, the first controller 110 is a RAID card, which is a board used to implement RAID (Redundant Array of Independent Disks) functions. It combines multiple independent hard disks (physical hard disks) in different ways to form a hard disk group (logical hard disk), thereby providing higher storage performance and data redundancy than a single hard disk.

[0035] A nonvolatile memory controller is a device that manages data stored on a nonvolatile memory and communicates with a host such as a computer or electronic device. In addition to the specific functions described in the present invention, a nonvolatile memory controller can have various functions. For example, a nonvolatile memory controller can format a nonvolatile memory to ensure that the memory is operating correctly, mark bad nonvolatile memory cells, and allocate spare cells to replace future faulty cells. Some parts in the spare cells can be used to accommodate firmware to operate the nonvolatile memory controller and implement other features. In operation, when the host needs to read data from the nonvolatile memory or write data to the nonvolatile memory, it can communicate with the nonvolatile memory controller. If the host provides a logical address to read / write data, the nonvolatile memory controller can convert the logical address received from the host into a physical address in the nonvolatile memory. Alternatively, the host can provide a physical address. The nonvolatile memory controller may also perform various memory management functions such as, but not limited to, wear leveling (distributing writes to avoid wearing out specific memory blocks that would otherwise be written repeatedly) and garbage collection (moving only valid data pages to new blocks after a block is full so the complete block can be erased and reused).

[0036] The non-volatile memory die may include any suitable non-volatile storage medium, including resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), phase change memory (PCM), NAND flash memory cells, and / or NOR flash memory cells. The memory cells may be in the form of solid-state (e.g., flash) memory cells, and may be programmable once, programmable a few times, or programmable multiple times. The memory cells may also be single-level cells (SLC), multi-level cells (MLC), triple-level cells (TLC), or use other memory cell level technologies now known or later developed. In addition, the memory cells may be manufactured in two dimensions or in three dimensions.

[0037] The interface between the first controller 110 and the non-volatile memory die may be any suitable flash memory interface. In one embodiment, the first storage system 100 may be a card-based system, such as a secure digital card (SD) or a micro secure digital (micro SD) card. In an alternative embodiment, the first storage system 100 may be a plurality of independent hard disks (physical hard disks) combined in different ways to form a hard disk group (logical hard disk).

[0038] The host 200 may take any suitable form, including but not limited to a computer, a mobile phone, a tablet computer, a wearable device, a digital video recorder, a surveillance system, etc. In an embodiment of the present invention, the host 200 (eg, a computing device) includes a first processor 210 and a second memory 220 .

[0039] In some cases, the memory in the first storage system 100 may be augmented with volatile memory. The volatile memory may be used as a cache for data otherwise stored in non-volatile memory (e.g., a logical-to-physical address mapping table) and / or temporary data (e.g., data used in error handling or error correction, write data to be stored in non-volatile memory, etc.).

[0040] However, since the storage capacity in the volatile memory is limited, the space in the volatile memory may be exhausted, or some storage systems are not equipped with volatile memory. For example, in a storage system composed of an NVMe interface SSD disk and a Raid card, the access latency of the SSD disk is very small and the rate is very high. If the access rate is only to be improved a little, the Raid card needs a large DDR as a read cache, which will increase the hardware cost very much. Therefore, general NVMe only Raid cards generally do not carry DDR (DDR SDRAM, double rate synchronous dynamic random access memory) as a read cache.

[0041] This may adversely affect the performance of the first storage system 100. In one embodiment, the first storage system 100 uses the second memory 220 in the host 200 as an extension of the volatile memory in the first storage system 100. When the second memory 220 in the host 200 is used for such a purpose, it may be referred to as a "host memory buffer (HMB)".

[0042] Figure 2 It is a flow chart of the data storage method provided by the present invention. Figure 2 As shown, the method includes the following:

[0043] Step 201: read a first replacement weight of first data, where the first data is only stored in the memory, and the replacement weight is calculated based on the number of requests for the data, and the replacement weight is positively correlated with the number of requests.

[0044] Step 202: Read a second replacement weight of second data, where the second data is stored in the host memory buffer and the memory.

[0045] The first replacement weight of the first data is read, and the first data is a data stored only in the memory of the storage system. Then the second replacement weight of the second data is read, and the second data is a data stored simultaneously in the host memory buffer of the host and the memory of the storage system. The replacement weight can be calculated based on the number of times the data is requested. For example, a data is requested to be read by the host. When the data is the first data, the data is read from the memory and returned. This request is the first request for the data to be recorded, and an initial replacement weight is set for the data. Alternatively, when the data is the second data, after the data is read from the host memory buffer and returned, the existing replacement weight of the data is added with a preset value to generate a new replacement weight.

[0046] It should be noted that the replacement weight is positively correlated with the number of requests. The more requests there are, the greater the replacement weight is. Furthermore, the increase in the replacement weight under write requests is greater than the increase in the replacement weight under read requests. For example, the relationship between the replacement weight and the number of requests is set to P=xN, where P is the replacement weight and N is the number of requests. Then x for write requests is 6 and x for read requests is 2. Alternatively, the initial replacement weight of the data is set to 0. When the data is read once through the data identifier, the replacement weight is increased by 1. When the data is read once through the data identifier, the replacement weight is increased by 10.

[0047] In one embodiment of the present invention, the replacement weight of data is obtained from a data structure based on the data identifier, wherein the data structure is pre-set, and the data structure is used to store the data identifier and replacement weight of each data, and the data identifier is a unique identifier of the data in the storage system.

[0048] The present invention is pre-set with a data structure, and the data structure is used to store the data identification and replacement weight of each data. In computer science, a data structure is a data organization and storage format for efficient access to data. More specifically, a data structure is a collection of data values, the relationships between them, and functions or operations that can be applied to the data values. The forms of data structures include: array, linked list, record, tree, etc.

[0049] For example, a linked list may be pre-set, in which each node stores a data identifier and a replacement weight of a data.

[0050] The storage location of the set data structure can be a memory or other location. For example, when the controller is a Raid card, the set data structure is stored in the Raid card. The data identifier in the data structure is the unique identifier of the data in the storage system, which can be the logical block address of the data, a hash value, etc.

[0051] In the above embodiment, the replacement weight of the data can be quickly read from the data structure by indexing through the data identifier of the data, which facilitates the management and reading of the replacement weight of the data.

[0052] In one embodiment of the present invention, the data structure includes a first data structure and a second data structure, the first data structure is used to store the data identification and replacement weight of the first data, and the second data structure is used to store the data identification and replacement weight of the second data.

[0053] Two data structures are preset: a first data structure and a second data structure. For data stored only in the memory but not in the host memory buffer, the data identifiers and replacement weights of the data are saved in the first data structure.

[0054] In one embodiment of the present invention, only information related to specified data that is not stored in the host memory buffer and is only stored in the memory is stored in the first data structure. For example, the first data structure is used to store the data identifier and replacement weight of the request data, and the request data is the data requested by the read and write request and is only stored in the memory.

[0055] For the data requested by the read or write request, if the requested data is not stored in the host memory buffer, it is only stored in the memory, and the data identification and replacement weight of the data are saved in the first data structure.

[0056] The above embodiment solves the problem that the data with high access frequency cannot be found in time when the cache area is full through the second data structure.

[0057] After the data is saved in the host memory buffer, the data identifier and replacement weight of the data are saved in a second data structure.

[0058] For example, the data structure adopts a statistical tree structure, and two statistical trees are pre-set: a buffer statistical tree and a spare area statistical tree. The buffer statistical tree stores relevant information of the data stored in the host memory buffer, such as data identification, replacement weight, and the timestamp of the last access. The spare area statistical tree stores relevant information of the data stored only in the memory.

[0059] When performing cache replacement on the host memory buffer, data of a node is read from the first data structure to obtain first data and a replacement weight of the data, and data of a node is read from the second data structure to obtain second data and a weight of the data.

[0060] Furthermore, the data structure includes multiple data nodes, each data node stores a data identifier and a replacement weight of a data, each data node in the first data structure is associated with a storage address, the storage address is a data storage unit in a host memory buffer corresponding to the data identifier in the data node, and the data identifier is a logical block address of the data in the memory.

[0061] Each node in the statistical tree stores relevant information about a piece of data, including: the starting lba (logical block address) address, the access timestamp, and the replacement weight.

[0062] The starting lba address is the starting logical block address of the data in the memory. Using the starting lba address of the data as the identifier of the data in the storage system can not only play the role of unique identification, but also facilitate the rapid search for the storage location of the data in the memory. The access timestamp is the time when the data was last accessed. It should be noted that no matter whether the data is in the host memory buffer or in the memory, the access timestamp in the statistical tree must be updated as long as the data is accessed.

[0063] In the host memory buffer, each cacheline (data storage unit) corresponds to a node, and all nodes form a statistical tree, called a buffer statistical tree. Through this association relationship, data with too small replacement weights in the host memory buffer can be quickly modified in a timely manner. For example, if the first replacement weight is greater than the second replacement weight, the storage location of the second data can be quickly found through the node storing the second replacement weight, and the second data can be deleted from the host memory buffer to free up storage space.

[0064] In the above embodiment, the data identifiers waiting to be stored in the host memory buffer are read from the first data structure, and the data identifiers stored in the host memory buffer are read from the second data structure, so that the new and old data are read quickly, which facilitates cache replacement.

[0065] Step 203: Compare the first replacement weight and the second replacement weight.

[0066] Step 204: If the first replacement weight is greater than the second replacement weight, store the first data in the host memory buffer and delete the second data from the host memory buffer.

[0067] Compare the first replacement weight and the second replacement weight.

[0068] If the first replacement weight is greater than the second replacement weight, the second data is deleted from the host memory buffer, and the first data is stored in the host memory buffer, thereby implementing cache replacement.

[0069] For example, if the first replacement weight is greater than the second replacement weight, the node of the second data in the buffer statistics tree is obtained, and the node is replaced, such as clearing the node, and the relevant information of the first data is saved in the node. The second data is deleted in the host memory buffer, and the first data is saved in the same data storage unit after the deletion.

[0070] The above-mentioned data storage method calculates the replacement weight of the data according to the number of requests, compares the replacement weights of the first data and the second data, and when the replacement weight of the first data is greater than the replacement weight of the second data, realizes cache replacement according to the number of requests, eliminates the data with low access frequency from the cache in time, releases the cache space in time, and adds the data with high access frequency into the cache in time, thereby realizing the replacement of low access frequency data with high access frequency data in cache replacement. By replacing the low access frequency data in the host memory buffer with high access frequency data, it is beneficial to improve the data reading speed and reduce the number of accesses to the memory.

[0071] When the above method is applied to a storage system composed of an SSD disk and a Raid card, a read cache is added to the Raid card without DDR, and the data previously read from the SSD disk is cached in the host memory. The life of DRAM is not limited by the number of reads and writes, and the host memory read cache is used to intercept some disk requests to access the SSD, thereby improving the service life of the SSD disks mounted on the Raid card without DDR. The memory used by the read cache is the HMB allocated by the host according to the memory usage, which is the free memory of the host. There is no need to add new hardware, and it is implemented by software, which will not increase additional hardware overhead. The software algorithm is used to reasonably allocate existing hardware resources, thereby improving the life of the SSD disk mounted on the Raid card at a low cost. And compared with the Raid card without read cache, the delay of the host reading data will also be reduced.

[0072] In one example of the present invention, the method further includes: updating the replacement weight of each data according to the access time of the data, wherein the access time is the time when the data was last accessed.

[0073] Traverse each node in the statistical tree, and for each node, read the access time data stored in the node, such as the access timestamp. The access timestamp is the time when the data was last accessed, so as to obtain the access time of the data corresponding to the node (that is, the data corresponding to the data identifier in the node). Update the replacement weight in the node based on the access timestamp.

[0074] In one embodiment of the present invention, updating the replacement weight of each data based on the access time of the data includes: calculating the time difference of each data, wherein the time difference is the difference between the access time and the current time; calculating the current replacement weight of each data based on the time difference and the read and write performance parameters of the host memory buffer, wherein the current replacement weight is the replacement weight of the data at the current moment; and modifying the historical replacement weight of each data to the current replacement weight, wherein the historical replacement weight is the replacement weight of the data before the current moment.

[0075] For each node, the difference between the current time and the timestamp accessed within the node is calculated to obtain the time difference value of the data corresponding to the node.

[0076] The current replacement weight of the data is calculated based on the time difference of the data and the read and write performance parameters of the host memory buffer. The read and write performance parameters refer to the read and write performance indicators of the memory buffer such as IOPS (Input / Output Operations Per Second).

[0077] In one embodiment of the present invention, the replacement weight of each data is calculated based on the time difference and the read and write parameters of the memory, including: determining the update period based on the read and write parameters of the memory; dividing the time difference by the update period to obtain the update value of each data; and obtaining the current replacement weight of each data based on the update value of the data and the historical replacement weight.

[0078] N = (current time – access timestamp of statistical block) / T, where N is the update value and T is the update period. The period T is dynamically adjusted according to IOPS. The higher the IOPS, the shorter the period, and the lower the IOPS, the longer the period.

[0079] The current replacement weight of the data is calculated based on N and the historical replacement weight.

[0080] In one embodiment of the present invention, the current replacement weight of each data is obtained based on the update value of the data and the historical replacement weight, including: obtaining the gain multiple of the data based on the threshold interval in which the update value of the data is located, and the threshold interval is preset; subtracting the product of the gain multiple and the update value from the historical replacement weight of the data to obtain the current replacement weight of each data.

[0081] When N is less than or equal to 5, the statistic block replacement weight = statistic block replacement weight – N.

[0082] When N is greater than 5 and less than 10, the statistics block replacement weight = statistics block replacement weight – 2N.

[0083] When N is greater than 10 and less than 20, the statistics block replacement weight = statistics block replacement weight – 8N.

[0084] When N is greater than 20, the statistics block replacement weight = statistics block replacement weight – 64N.

[0085] In the above embodiment, the replacement weight is updated according to the access time, so that the host memory buffer data that has not been accessed for a long time can be replaced in time, and the space of the host memory buffer can be released in time.

[0086] In one embodiment of the present invention, the method further comprises: removing data having a current replacement weight less than 0 in the host memory buffer.

[0087] After the replacement weight is updated, if the replacement weight of a node in the second data structure is less than 0, the data corresponding to the node is removed from the host memory buffer, the buffer space is released, and the node is released.

[0088] If the replacement weight in a node in the first data structure is less than 0, the node of the data in the first data structure is released.

[0089] In one embodiment of the present invention, the method also includes: receiving a read request; determining read data based on the read request; determining whether the read data is stored in the host memory buffer; if the read data is not stored in the host memory buffer, obtaining the read data from the memory; determining whether the host memory buffer can store the read data; if the host memory buffer can store the read data, storing the read data in the host memory buffer; calculating a replacement weight for the read data, and saving the replacement weight for the read data.

[0090] When the host sends a read request, the data to be read by the read request is determined, that is, the read data.

[0091] The read cache is queried first, for example, whether the read data exists in the host memory buffer, or whether the read data exists in the second data structure.

[0092] If found, a cache read success event is generated after the data is returned to the host.

[0093] After receiving a read cache success event, the replacement weight and access time are updated, such as adding 1 to the replacement weight in the corresponding statistical tree node and updating the access timestamp to the current time.

[0094] If the data is not found, the memory is queried, and after the data is returned to the host, an event of cache read failure but disk read success is generated.

[0095] After receiving the event of failed read cache but successful disk read, determine whether the host memory buffer has enough space to store the read data. For example, deploy a Memory Analyze process on the machine to analyze the host memory usage, calculate the free memory based on the security threshold set by the user, and send the HMB information to the Raid card through the set features command. After receiving the Set features command, the Raid card adjusts the size of the host memory buffer. During the use of the Raid card, you can also use commands to view the space usage in the host memory buffer.

[0096] If it is found that there is still space in the current host memory buffer, a cacheline is established in the host memory buffer based on the read data, and the read data is stored in it. At the same time, a node is initialized for the data in the buffer statistics tree, and the data identification, replacement weight and other information of the data are stored in the node.

[0097] In one embodiment of the present invention, the method further includes: if the host memory buffer cannot store the read data, determining whether the read data has been marked as the second data; if the read data has been marked as the second data, updating the replacement weight of the read data; if the read data has not been marked as the second data, marking the read data as the second data, and calculating and saving the replacement weight of the read data.

[0098] If there is no space in the current host memory buffer, it is determined whether the read data has been marked as the second data, for example, it is determined whether the read data can be found in the spare area statistics tree.

[0099] If the read data has been marked as the second data, such as finding the read data in the spare area statistics tree, the replacement weight of the read data in the spare area statistics tree node is updated, and the access timestamp is updated at the same time.

[0100] If the read data is not marked as the second data, such as the read data is not found in the spare area statistics tree, a node is initialized for the data in the spare area statistics tree, the replacement weight of the read data is calculated, and the data identification, replacement weight, access timestamp and other information of the data are stored in the node.

[0101] Under specified conditions, for example, after detecting that a node of a statistical number is updated, the statistical tree is sorted from large to small according to the replacement weight. Then, cache replacement is performed through the above steps 101 to 104. For example, nodes are read from the spare area statistical tree in order from small to large replacement weights.

[0102] For a node weight read, it is compared with the nodes in the buffer statistics tree in order of replacement weight from small to large. If the replacement weight is greater than the replacement weight of the buffer statistics tree, the data corresponding to the replacement weight of the buffer statistics tree is deleted, and the data corresponding to the replacement weight is stored in the host memory buffer.

[0103] The above embodiment provides a method for data caching using replacement weights under read requests, which promptly adds data with increased access frequency into the cache and promptly eliminates data with low access frequency from the cache, which is beneficial to improving the data reading speed and reducing the number of accesses to the memory.

[0104] In one embodiment of the present invention, the method also includes: receiving a write request; determining write data based on the write request, the write data being data written in the memory when executing the write request; determining whether historical data corresponding to the write request is stored in the host memory buffer, the historical data being data modified when executing the write request; if the historical data is already stored in the host memory buffer, modifying the historical data to the write data; calculating a replacement weight based on the write data, and saving the replacement weight of the write data.

[0105] After receiving a write request from the host, it executes the write request and writes it to the hard disk, then notifies the host that the write request is completed and generates a write request event.

[0106] Determine the data written by the write request, and judge whether the historical data corresponding to the read / write request is stored in the host memory buffer. For example, the write request is to change the data at the first storage location from 0000 to 0001. Then the write data of this write request is 0000, and the historical data is 0001.

[0107] After receiving the write request event, determine whether the historical data is stored in the host memory buffer. If the historical data is already in the host memory buffer, modify the historical data in the host memory buffer to write data, that is, execute the write request in the host memory buffer. Calculate the replacement weight of the write data, for example, add 10 to the replacement weight in the node corresponding to the historical data, and update the access timestamp to the current time. In the present invention, the replacement weight calculated for the write data will be greater than the replacement weight for the read data.

[0108] In one embodiment of the present invention, the method further includes: if the historical data is not stored in the host memory buffer, determining whether the host memory buffer is capable of storing the write data; if the host memory buffer is capable of storing the write data, storing the write data in the host memory buffer; calculating a replacement weight based on the write data, and saving the replacement weight of the write data.

[0109] If the historical data is not in the host memory buffer, it is determined whether there is enough space in the host memory buffer to store the historical data. If the host memory buffer has space to store the write data, the write data is stored in the host memory buffer, a node is initialized for the data in the buffer statistics tree, a replacement weight based on the write data is calculated, and the replacement weight of the write data is stored in the node.

[0110] In one embodiment of the present invention, the method further includes: if the host memory buffer cannot store the write data, obtaining second data with a minimum replacement weight; if the minimum replacement weight is greater than a preset threshold, deleting the minimum replacement weight from the host memory buffer and storing the write data in the host memory buffer; if the minimum replacement weight is less than the preset threshold, marking the write data as the first data; calculating the replacement weight of the write data, and saving the replacement weight of the write data.

[0111] The host memory buffer cannot store the write data, and the minimum replacement weight is taken out in the buffer statistics tree.

[0112] The minimum replacement weight is compared with a preset threshold. For example, the preset threshold is the replacement weight of the write data. The preset threshold is also an initial value assigned to a write data. The initial value is the value assigned when the replacement weight of the write data is calculated for the first time. For example, when a data is written to the memory for the first time, it will be assigned the initial value, or when a data has never been read / written before, it will be assigned the initial value when the data is written.

[0113] See if the minimum replacement weight can be greater than a preset threshold, such as 10.

[0114] If the minimum replacement weight is less than a preset threshold, the written data is marked as the second data, such as initializing a node corresponding to the minimum replacement weight, storing the data identifier and the replacement weight of the written data in the node, thereby marking the written data as the second data. The minimum replacement weight is deleted from the host memory buffer, and the written data is stored in the host memory buffer.

[0115] If it is greater than a preset threshold, a new node is generated in the spare area statistical tree, and relevant information of the data to be written, such as data identification and replacement weight, is stored in the node, thereby marking the written data as the first data.

[0116] Alternatively, in one embodiment of the present invention, the method further includes: if the host memory buffer cannot store the write data, marking the write data as the second data; calculating a replacement weight of the write data, and saving the replacement weight of the write data.

[0117] When the host memory buffer cannot store the written data, a new node can be directly generated in the spare area statistical tree, and relevant information of the data to be written, such as data identification and replacement weight, is stored in the node, thereby marking the written data as the first data.

[0118] In one embodiment of the present invention, the method further includes: if the read / write request has no corresponding historical data, determining whether the host memory buffer is capable of storing the write data; if the host memory buffer is capable of storing the write data, storing the write data in the host memory buffer; calculating a replacement weight based on the write data, and saving the replacement weight of the write data.

[0119] If the write request has no corresponding historical data, for example, the write request is to write new data 0000, then the write data of this write request is 0000, and there is no corresponding historical data.

[0120] If there is no historical data, determine whether there is enough space in the host memory buffer to store the write data, store the write data in the host memory buffer, initialize a node in the buffer statistics tree for the data, calculate the replacement weight based on the write data, and store the replacement weight of the write data in the node.

[0121] When the host memory buffer cannot store the written data, a new node can be directly generated in the spare area statistical tree, and relevant information of the data to be written, such as data identification and replacement weight, is stored in the node, thereby marking the written data as the first data.

[0122] The above-mentioned embodiment provides a method for data caching using replacement weights under read requests, which promptly adds data with increased access frequency into the cache and promptly eliminates data with low access frequency from the cache, thereby realizing the replacement of low access frequency data with high access frequency data in cache replacement. By replacing low access frequency data in the host memory buffer with high access frequency data, it is beneficial to improve the data reading speed and reduce the number of accesses to the memory.

[0123] When the above data storage method is applied to a storage system composed of an SSD hard disk and a Raid card, the life of the SSD hanging under the Raid card can be improved, the host's idle memory is used as the HMB of the Raid card, the current host memory usage is analyzed, and the size of the Raid card HMB is dynamically adjusted, thereby realizing low-cost addition of read cache on a Raid card without DDR, reducing the number of times the SSD disk is directly read, and using the life of DRAM that is not limited by the number of reads and writes, thereby improving the life of the SSD disk hanging under the Raid card.

[0124] Different from the traditional read cache of Raid card, the main purpose is to reduce the latency of reading data, and some pre-reading will be performed to increase the number of disk accesses. The cache replacement algorithm of the above method is to extend the life of the SSD disk, so that the data blocks with high access frequency are placed in the cache as much as possible, and the spare area statistics are added to solve the problem that the data with high access frequency cannot be found in time when the cache area is full. The weight is updated periodically, and the period is dynamically adjusted according to IOPS. The low access frequency can be replaced in time, reducing the number of accesses to the entire disk, and not actively increasing the number of disk reads.

[0125] The second storage system 300 provided by the present invention is described below. The second storage system 300 described below and the data storage method described above can be referenced to each other.

[0126] Figure 3 An example of a physical structure diagram of a second storage system 300 is shown as follows: Figure 3 As shown, the second storage system 300 includes: a second controller 310 and a third memory 320, the second controller 310 uses the memory in the host as a host memory buffer, and the second controller 310 receives a read request for data stored in the third memory 320;

[0127] The second controller 310 is configured to read a first replacement weight of the first data, the first data is only stored in the memory, the replacement weight is calculated based on the number of requests for the data, and the replacement weight is positively correlated with the number of requests; read a second replacement weight of the second data, the second data is stored in the host memory buffer and the memory; compare the first replacement weight and the second replacement weight; if the first replacement weight is greater than the second replacement weight, store the first data in the host memory buffer and delete the second data from the host memory buffer.

[0128] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a second processor (processor) 410, a communication interface (Communications Interface) 420, a fourth memory (memory) 430 and a communication bus 440, wherein the second processor 410, the communication interface 420, and the fourth memory 430 complete mutual communication through the communication bus 440. The second processor 410 can call the logic instructions in the fourth memory 430 to execute the data storage method, which includes: reading the first replacement weight of the first data, the first data is only stored in the memory, the replacement weight is calculated according to the number of requests for the data, and the replacement weight is positively correlated with the number of requests; reading the second replacement weight of the second data, the second data is stored in the host memory buffer and the memory; comparing the first replacement weight and the second replacement weight; if the first replacement weight is greater than the second replacement weight, the first data is stored in the host memory buffer, and the second data is deleted from the host memory buffer.

[0129] In addition, the logic instructions in the fourth memory 430 mentioned above can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0130] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data storage method provided by the above methods, which includes: reading a first replacement weight of the first data, the first data is only stored in the memory, the replacement weight is calculated based on the number of requests for the data, and the replacement weight is positively correlated with the number of requests; reading a second replacement weight of the second data, the second data is stored in the host memory buffer and the memory; comparing the first replacement weight and the second replacement weight; if the first replacement weight is greater than the second replacement weight, storing the first data in the host memory buffer and deleting the second data from the host memory buffer.

[0131] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the data storage method provided by the above-mentioned methods, the method comprising: reading a first replacement weight of first data, the first data being stored only in the memory, the replacement weight being calculated based on the number of requests for the data, and the replacement weight being positively correlated with the number of requests; reading a second replacement weight of second data, the second data being stored in the host memory buffer and the memory; comparing the first replacement weight and the second replacement weight; if the first replacement weight is greater than the second replacement weight, storing the first data in the host memory buffer and deleting the second data from the host memory buffer.

[0132] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data storage method, characterized in that: The method is applied to a controller in a storage system, the storage system further comprising a memory, the controller being used to use the memory in a host as a host memory buffer and to receive a read or write request for data stored in the memory, the method comprising: Reading a first replacement weight of first data, where the first data is only stored in the memory, the replacement weight is calculated based on the number of requests for the data, and the replacement weight is positively correlated with the number of requests; reading a second replacement weight for second data, the second data being stored in the host memory buffer and the memory; comparing the first replacement weight and the second replacement weight; If the first replacement weight is greater than the second replacement weight, storing the first data in the host memory buffer and deleting the second data from the host memory buffer; Wherein, the method further comprises: Update the replacement weight of each data according to the access time of the data, where the access time is the time when the data was last accessed; The updating of the replacement weight of each data according to the access time of the data includes: Calculate the time difference of each data, where the time difference is the difference between the access time and the current time; Calculating a current replacement weight of each data according to the time difference and the read / write performance parameter of the host memory buffer, wherein the current replacement weight is the replacement weight of the data at the current moment; The historical replacement weight of each data is modified to the current replacement weight, where the historical replacement weight is the replacement weight of the data before the current moment.

2. The method according to claim 1, characterized in that Calculate the replacement weight of each data according to the time difference and the read / write parameters of the memory, including: Determining an update cycle according to the read and write parameters of the memory; Divide the time difference by the update period to obtain an update value for each data; According to the updated value of the data and the historical replacement weight, the current replacement weight of each data is obtained.

3. The method according to claim 2, characterized in that According to the updated value of the data and the historical replacement weight, the current replacement weight of each data is obtained, including: According to the threshold interval in which the updated value of the data is located, obtaining the gain multiple of the data, wherein the threshold interval is preset; Subtract the product of the gain multiplier and the update value from the historical replacement weight of the data to obtain the current replacement weight of each data.

4. The method according to claim 1, characterized in that: The method further comprises: Data having a current replacement weight less than 0 is removed in the host memory buffer.

5. The method according to claim 1, characterized in that The method further comprises: Receive a read request; Determining read data according to the read request; determining whether the read data is stored in the host memory buffer; If the read data is not stored in the host memory buffer, obtaining the read data from the memory; determining whether the host memory buffer can store the read data; If the host memory buffer is capable of storing the read data, storing the read data in the host memory buffer; The replacement weight of the read data is calculated, and the replacement weight of the read data is saved.

6. The method according to claim 5, characterized in that The method further comprises: If the host memory buffer cannot store the read data, determining whether the read data has been marked as the second data; If the read data has been marked as the second data, updating the replacement weight of the read data; If the read data is not marked as the second data, the read data is marked as the second data, and a replacement weight of the read data is calculated and saved.

7. The method according to claim 1, characterized in that The method further comprises: Receive write request; Determining write data according to the write request, the write data being data written into the memory when executing the write request; Determining whether historical data corresponding to the write request is stored in the host memory buffer, the historical data being data modified when executing the write request; If the historical data has been stored in the host memory buffer, modifying the historical data to the write data; A replacement weight based on the written data is calculated, and the replacement weight of the written data is stored.

8. The method according to claim 7, characterized in that The method further comprises: If the historical data is not stored in the host memory buffer, determining whether the host memory buffer is capable of storing the write data; If the host memory buffer is capable of storing the write data, storing the write data in the host memory buffer; A replacement weight based on the written data is calculated, and the replacement weight of the written data is stored.

9. The method according to claim 8, characterized in that The method further comprises: If the host memory buffer cannot store the write data, obtaining second data with a minimum replacement weight; If the minimum replacement weight is greater than a preset threshold, deleting the minimum replacement weight from the host memory buffer and storing the write data in the host memory buffer; If the minimum replacement weight is less than a preset threshold, marking the written data as the first data; A replacement weight of the written data is calculated, and the replacement weight of the written data is saved.

10. The method according to claim 8, characterized in that The method further comprises: If the host memory buffer cannot store the write data, determining whether the write data has been marked as the second data; If the written data has been marked as the second data, updating the replacement weight of the written data; If the written data is not marked as the second data, the written data is marked as the second data, and a replacement weight of the written data is calculated and saved.

11. The method according to claim 7, characterized in that The method further comprises: If the read / write request has no corresponding historical data, determining whether the host memory buffer is capable of storing the write data; If the host memory buffer is capable of storing the write data, storing the write data in the host memory buffer; A replacement weight based on the written data is calculated, and the replacement weight of the written data is stored.

12. The method according to any one of claims 1 to 10, characterized in that: The replacement weight of the data is obtained from the data structure according to the data identifier, wherein the data structure is preset, and the data structure is used to store the data identifier and the replacement weight of each data, and the data identifier is a unique identifier of the data in the storage system.

13. The method according to claim 12, characterized in that The data structure includes a first data structure and a second data structure, the first data structure is used to store the data identification and replacement weight of the first data, and the second data structure is used to store the data identification and replacement weight of the second data.

14. The method according to claim 12, characterized in that The data structure includes multiple data nodes, each data node stores a data identifier and a replacement weight of a data, each data node in the second data structure is associated with a storage address, the storage address is a data storage unit of the data corresponding to the data identifier in the data node in the host memory buffer, and the data identifier is a logical block address of the data in the memory.

15. A storage system, characterized in that: The storage system includes a controller and a memory, wherein the controller uses the memory in the host as a host memory buffer, and the controller receives a read request for data stored in the memory; The controller is configured to read a first replacement weight of first data, the first data is only stored in the memory, the replacement weight is calculated according to the number of requests for the data, and the replacement weight is positively correlated with the number of requests; read a second replacement weight of second data, the second data is stored in the host memory buffer and the memory; comparing the first replacement weight and the second replacement weight; If the first replacement weight is greater than the second replacement weight, storing the first data in the host memory buffer and deleting the second data from the host memory buffer; The controller is further configured to update the replacement weight of each data according to the access time of the data, wherein the access time is the time when the data was last accessed; The updating of the replacement weight of each data according to the access time of the data includes: Calculate the time difference of each data, where the time difference is the difference between the access time and the current time; Calculating a current replacement weight of each data according to the time difference and the read / write performance parameter of the host memory buffer, wherein the current replacement weight is the replacement weight of the data at the current moment; The historical replacement weight of each data is modified to the current replacement weight, where the historical replacement weight is the replacement weight of the data before the current moment.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the data storage method according to any one of claims 1 to 14 is implemented.

17. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data storage method according to any one of claims 1 to 14 is implemented.

18. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data storage method according to any one of claims 1 to 14 is implemented.

Images