Data storage method, data query method and device

Abstract

Embodiments of the present specification provide a data storage method, a data query method and apparatus, wherein the data storage method is applied to a management platform in a data storage system, the data storage system comprising the management platform, a cache layer and a capacity layer; the method comprising: receiving a data storage request, wherein the data storage request carries to-be-stored data; adding the to-be-stored data to an idle cache block in the cache layer to obtain a valid cache block, wherein the idle cache block refers to a cache block available in the cache layer; in a case where the number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, sequentially writing data in the valid cache block to a buffer area in the cache layer to generate aggregated data; and in a case where the amount of data stored in the buffer area reaches a second preset threshold, storing the aggregated data to the capacity layer. The overall performance and system durability of the data storage system are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and particularly relate to a data storage method. One or more embodiments of the present specification also relate to a data query method, a data storage device, a data query device, a computing device, a computer-readable storage medium, and a computer program. BACKGROUND

[0002] Common storage devices include hybrid hard drives (HHD), solid state drives (SSD), and traditional hard drives (HDD). A hybrid hard drive is a large-capacity storage device that contains both a traditional hard drive and a flash module. A solid state drive is a hard drive made of an array of solid state electronic storage chips, consisting of a control unit and a storage unit.

[0003] With the development of computer technology, the capacity of industry storage devices such as HDD / SSD can be continuously improved by increasing density. Taking the solid state drive SSD as an example, the SSD gradually evolves from a trinary-level cell (TCL) to a quad-level cell (QLC). The QLC has the characteristics of large capacity and low cost, but the disadvantages are low endurance and short life, and the random write performance is only one hundredth of the sequential write performance. Therefore, there is an urgent need for a durable and efficient data storage solution. SUMMARY

[0004] In view of this, embodiments of the present specification provide a data storage method. One or more embodiments of the present specification also relate to a data query method, a data storage device, a data query device, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects in the prior art.

[0005] According to a first aspect of embodiments of the present specification, a data storage method is provided, applied to a management platform in a data storage system, the data storage system comprising a management platform, a cache layer, and a capacity layer; the method comprises:

[0006] receiving a data storage request, wherein the data storage request carries to-be-stored data;

[0007] adding the to-be-stored data to an idle cache block in the cache layer to obtain a valid cache block, wherein the idle cache block refers to a cache block available in the cache layer;

[0008] In a case where the number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, data in the valid cache blocks is sequentially written into the buffer area in the cache layer to generate aggregated data;

[0009] In a case where the amount of data stored in the buffer area reaches a second preset threshold, the aggregated data is stored to the capacity layer.

[0010] According to a second aspect of an embodiment of the present specification, a data query method is provided, applied to a management platform in a data storage system, and the method comprises:

[0011] receiving a data query request, wherein the data query request carries a logical block address of to-be-queried data;

[0012] finding a storage address corresponding to the logical block address;

[0013] finding the to-be-queried data according to the storage address, wherein the to-be-queried data is stored by the data storage method of any one of claims 1-7.

[0014] According to a third aspect of an embodiment of the present specification, a data storage device is provided, applied to a management platform in a data storage system, and the data storage system comprises the management platform, a cache layer, and a capacity layer; the device comprises:

[0015] a first receiving module configured to receive a data storage request, wherein the data storage request carries to-be-stored data;

[0016] an adding module configured to add the to-be-stored data to an idle cache block in the cache layer to obtain a valid cache block, wherein the idle cache block refers to a cache block available in the cache layer;

[0017] a generating module configured to, in a case where the number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, sequentially write data in the valid cache block into a buffer area in the cache layer to generate aggregated data;

[0018] a storage module configured to, in a case where the amount of data stored in the buffer area reaches a second preset threshold, store the aggregated data to the capacity layer.

[0019] According to a fourth aspect of an embodiment of the present specification, a data query device is provided, applied to a management platform in a data storage system, and the device comprises:

[0020] a second receiving module configured to receive a data query request, wherein the data query request carries a logical block address of to-be-queried data;

[0021] a first finding module configured to find a storage address corresponding to the logical block address;

[0022] The second searching module is configured to search the to-be-queried data according to the storage address, wherein the to-be-queried data is stored by the data storage method of the first aspect.

[0023] In a fifth aspect of the embodiments of the present disclosure, a data storage system is provided, including a management platform, a cache layer, and a capacity layer.

[0024] The management platform is configured to receive a data storage request, wherein the data storage request carries to-be-stored data; and transmit the to-be-stored data to the cache layer.

[0025] The cache layer is configured to store the to-be-stored data to an idle cache block to obtain an effective cache block, wherein the idle cache block refers to a cache block available in the cache layer.

[0026] The management platform is further configured to monitor a number of cache blocks of stored data in the cache layer, and send a write instruction to the cache layer when the number reaches a first preset threshold.

[0027] The cache layer is further configured to sequentially write data in the effective cache block to a buffer based on the write instruction to generate aggregated data.

[0028] The management platform is further configured to monitor a data amount stored in the buffer, and send a forwarding instruction to the cache layer when the data amount reaches a second preset threshold.

[0029] The cache layer is further configured to forward the aggregated data to the capacity layer based on the forwarding instruction.

[0030] The capacity layer is configured to store the aggregated data.

[0031] According to a sixth aspect of the embodiments of the present disclosure, a computing device is provided, including:

[0032] a memory and a processor;

[0033] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, which implement the steps of the data storage method of the first aspect or the data query method of the second aspect when executed by the processor.

[0034] According to a seventh aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer executable instructions, which implement the steps of the data storage method of the first aspect or the data query method of the second aspect when executed by the processor.

[0035] According to an eighth aspect of the embodiments of the present disclosure, a computer program is provided, which, when executed in a computer, causes the computer to perform the steps of the data storage method of the first aspect or the data query method of the second aspect.

[0036] The data storage method provided by one embodiment of the present specification is applied to a management platform in a data storage system, and the data storage system comprises the management platform, a cache layer and a capacity layer. The method comprises: receiving a data storage request, wherein the data storage request carries to-be-stored data; adding the to-be-stored data to an idle cache block in the cache layer to obtain a valid cache block, wherein the idle cache block refers to a cache block available in the cache layer; in a case where the number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, sequentially writing the data in the valid cache block to a buffer area in the cache layer to generate aggregated data; and in a case where the amount of data stored in the buffer area reaches a second preset threshold, storing the aggregated data to the capacity layer. By sequentially writing the data in the valid cache block to the buffer area in the cache layer and batch storing the aggregated data stored in the buffer area to the capacity layer, the write amplification and wear of the capacity layer are reduced, and the overall performance of the data storage system and the durability of the data storage system are further improved. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1a FIG. 1 is a schematic diagram of a data storage method provided by one embodiment of the present specification;

[0038] Figure 1b FIG. 2 is a structural schematic diagram of a data storage system provided by one embodiment of the present specification;

[0039] Figure 2 FIG. 3 is an architecture diagram of a data storage system applied in a shopping scenario provided by one embodiment of the present specification;

[0040] Figure 3 FIG. 4 is a flowchart of a data storage method provided by one embodiment of the present specification;

[0041] Figure 4 FIG. 5 is an architecture diagram of a cache layer provided by one embodiment of the present specification;

[0042] Figure 5 FIG. 6 is an architecture diagram of a logical-to-physical address table provided by one embodiment of the present specification;

[0043] Figure 6 FIG. 7 is a processing process schematic diagram of a data storage method provided by one embodiment of the present specification;

[0044] Figure 7 FIG. 8 is a processing process schematic diagram of another data storage method provided by one embodiment of the present specification;

[0045] Figure 8 FIG. 9 is a garbage collection process schematic diagram in a data storage method provided by one embodiment of the present specification;

[0046] Figure 9 is a flow chart of a data query method provided by one embodiment of the present specification;

[0047] Figure 10 is a processing block diagram of a data storage method provided by one embodiment of the present specification;

[0048] Figure 11a is a processing process flow chart of a data storage method provided by one embodiment of the present specification;

[0049] Figure 11b is a cache block schematic diagram provided by one embodiment of the present specification;

[0050] Figure 11c is a buffer schematic diagram provided by one embodiment of the present specification;

[0051] Figure 12 is a structural schematic diagram of a data storage device provided by one embodiment of the present specification;

[0052] Figure 13 is a structural schematic diagram of a data query device provided by one embodiment of the present specification;

[0053] Figure 14 is a structural block diagram of a computing device provided by one embodiment of the present specification. DETAILED DESCRIPTION

[0054] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present specification. However, the present specification can be practiced without the specific details, other than in the examples, and it is understood that the scope of the present specification is not limited to the details below.

[0055] The terminology used in one or more embodiments of the present specification is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present specification. As used in one or more embodiments of the present specification and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0056] It should be understood that, although the terms first, second, etc. can be employed in describing various information in one or more embodiments of the present specification, the information should not be limited to such terms. These terms are only used to differentiate one piece of information from another piece of information. For example, without departing from the scope of one or more embodiments of the present specification, first can also be referred to as second, and similarly, second can also be referred to as first. Depending on the context, the word "if' as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0057] First, the noun terms related to one or more embodiments of the present specification are explained.

[0058] Cloud disk: a disk instance built on a distributed storage system. It can be used for reading and writing in the cloud server as a computer disk.

[0059] IO request: the basic operation of a user using a cloud disk is to read data from the disk and write data to the disk. Each of these read and write requests is called an IO request (abbreviated as IO).

[0060] NVM Express (NVMe): or Non-Volatile Memory Host Controller Interface Specification (NVMHCIS), is a logical device interface specification. It is a bus transmission protocol specification based on device logical interface, used to access non-volatile memory media attached through high-speed serial computer expansion bus (PCIe, Peripheral Component Interconnect Express). NVMe is a protocol, a set of software and hardware standards that allow SSDs to use PCIe bus, while PCIe is the actual physical connection channel.

[0061] TLC: Trinary-Level Cell, also known as three-layer storage unit.

[0062] QLC: Quad-Level Cell, also known as four-layer storage unit.

[0063] WSC: Write Shaping Cache, also known as write shaping cache.

[0064] Nand flash: a type of flash memory, which uses a non-linear macro cell mode inside, providing a cheap and effective solution for the realization of solid-state mass storage.

[0065] In the ZNS (Zoned Namespace) specification, the entire LBA (Logical Block Address) range of the SSD namespace is divided into several equal-length intervals, and the equal-length LBA interval becomes a zone, and writing is performed in a sequential manner in each zone.

[0066] In the present specification, a data storage method is provided. One or more embodiments of the present specification also relate to a data query method, a data storage device, a data query device, a computing device, a computer-readable storage medium, and a computer program, which are described in detail in the following embodiments.

[0067] With the development of computer technology, the capacity of storage devices such as HDD / SSD in the industry can be continuously improved by increasing the density. Taking the SSD as an example, the SSD gradually evolves from the TCL (Trinary-Level Cell) to the QLC (Quad-Level Cell). Among them, the storage density of the TCL is relatively high, so the capacity is large, the cost is low, but the life is relatively low, and the theoretical erase-write times are different between 1000-3000 times. It is the mainstream flash memory particle on the market at present. In the QLC, the more data each standard cell (Cell) stores, the higher the capacity per unit area, but at the same time, it will cause more different voltage states and be more difficult to control. Therefore, although the SSD using QLC particles has larger capacity and cheaper price, it has poor stability and low life, and the slowest speed. In terms of performance, the sequential read and write performance of QLC is 7000MB / s and 3200MB / s respectively, and the random read and write performance is 3200MB / s and 32MB / s respectively. It can be seen that the random write performance of QLC is only one percent of the sequential write performance. Therefore, a durable and efficient data storage scheme is urgently needed to solve the problems of low random read and write performance and low life of QLC medium.

[0068] In the embodiments of the present specification, an architecture and method for high-speed medium acceleration are proposed, mainly aiming at the characteristics of QLC, providing an architecture in the block layer access field, reducing QLC write amplification, avoiding the low performance of random write, and maximizing the value of QLC new medium.

[0069] Specifically, a data storage request is received, wherein the data storage request carries to-be-stored data; the to-be-stored data is added to a free cache block in a cache layer to obtain a valid cache block, wherein the free cache block refers to a cache block available in the cache layer; in a case where a quantity of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, data in the valid cache block is sequentially written into a buffer in the cache layer to generate aggregated data; and in a case where a quantity of data stored in the buffer reaches a second preset threshold, the aggregated data is stored to a capacity layer. By sequentially writing data in the valid cache block into the buffer in the cache layer and storing the aggregated data stored in the buffer to the capacity layer in batches, write amplification and wear of the capacity layer are reduced, and the overall performance of the data storage system and the durability of the data storage system are further improved.

[0070] It should be noted that the scheme of the embodiments of the present specification can utilize a mixed medium of a high-speed medium and a low-speed medium to solve the problems faced by QLC, and the embodiments of the present specification are based on physical machine performance + cloud operation and maintenance, and the storage stack maintains the principle of simple and transparent transmission to improve the richness of the data storage system.

[0071] Referring to Figure 1a , Figure 1a A schematic diagram of a data storage method provided by one embodiment of the present specification is shown, and the data storage system includes a management platform, a cache layer and a capacity layer;

[0072] The management platform is configured to receive a data storage request, wherein the data storage request carries to-be-stored data; and transmit the to-be-stored data to the cache layer.

[0073] The cache layer is configured to store the to-be-stored data to a free cache block to obtain a valid cache block, wherein the free cache block refers to a cache block available in the cache layer.

[0074] The management platform is further configured to monitor a quantity of cache blocks in which data has been stored in the cache layer, and send a write instruction to the cache layer in a case where the quantity reaches a first preset threshold.

[0075] The cache layer is further configured to sequentially write data in the valid cache block to a buffer based on the write instruction to generate aggregated data.

[0076] The management platform is further configured to monitor a quantity of data stored in the buffer, and send a forwarding instruction to the cache layer in a case where the quantity reaches a second preset threshold.

[0077] The cache layer is further configured to forward the aggregated data to the capacity layer based on the forwarding instruction.

[0078] The capacity layer is configured to store the aggregated data.

[0079] In an optional embodiment of the present specification, the management platform is further configured to receive a data query request, wherein the data query request carries a logical block address of to-be-queried data; find a storage address corresponding to the logical block address; and find the to-be-queried data according to the storage address.

[0080] According to the scheme of the embodiment of the present specification, a data storage request is received, wherein the data storage request carries to-be-stored data; the to-be-stored data is added to an idle cache block in a cache layer to obtain a valid cache block, wherein the idle cache block refers to a cache block available in the cache layer; in a case where a number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, data in the valid cache block is sequentially written into a buffer area in the cache layer to generate aggregated data; and in a case where an amount of data stored in the buffer area reaches a second preset threshold, the aggregated data is stored to a capacity layer. By sequentially writing data in the valid cache block into the buffer area in the cache layer and batch storing aggregated data stored in the buffer area to the capacity layer, write amplification and wear of the capacity layer are reduced, and the overall performance of the data storage system and the durability of the data storage system are further improved.

[0081] Referring to Figure 1b , Figure 1b A structure diagram of a data storage system provided by an embodiment of the present specification is shown, the data storage system is implemented in a block layer, and the data storage system can be understood as a write shaping cache system (WSC, Write Shaping Cache). The WSC is an abstract layer optimized for a high-speed medium-low-speed medium hierarchy structure. The WSC uses a high-speed medium as a write cache to store data randomly written by a user, maximally reduces system-level write amplification and hard disk wear, and improves the overall performance of the data storage system and the system durability. The WSC provides a write shaping abstraction layer for general block I / O (Block I / O) services, and the write shaping abstraction layer includes a storage layer, and the storage layer includes a cache layer and a capacity layer. The high-speed medium can be used as a non-volatile cache (NvCache, Nonvolatile Cache) in the cache layer, and a flash memory (NAND) based SSD can be used as a large-capacity main storage in the capacity layer, which can also be understood as a pooled capacity storage (Pooled Capacity Storage).

[0082] It should be noted that the write shaping abstraction layer also includes a flash translation layer (FTL), which includes a logical to physical address table (L2P) and a metadata area (Metadata). The logical to physical address table manages a complete associative address mapping between logical block addresses (LBAs) and physical locations where data is stored in the capacity layer.

[0083] The scheme provided by one or more embodiments of the present specification can be applied to various data storage or query scenarios, such as shopping scenarios, regulatory scenarios, and the like. The specific selection is made according to the actual situation, and the embodiments of the present specification do not make any limitation thereto.

[0084] Referring to Figure 2 , Figure 2 An architecture diagram of a data storage system applied in a shopping scenario is shown, which is provided by one embodiment of the present specification. The system can include a server 210 and a plurality of clients 200. The server 210 is also a management platform in the data storage system. The plurality of clients 200 can establish a communication connection through the server 210. In the shopping scenario, the server 210 is used to provide data storage services or data query services between the plurality of clients 200. The plurality of clients 200 can be respectively used as a sending end or a receiving end to achieve data storage or data query through the server 210.

[0085] A user can interact with the server 210 through the client 200 to query data stored by other clients 200 or send data to other clients 200, and the like. In the shopping scenario, the user can publish a data query request to the server 210 through the client 200. The server 200 performs data query based on the data query request and pushes the query result to other clients that establish a communication connection.

[0086] The connection between the client 200 and the server 210 is established through a network. The network provides a medium for a communication link between the client and the server. The network can include various connection types, such as wired, wireless communication links, or fiber optic cables, and the like.

[0087] The client 200 can be a browser, an application (APP), or a web application such as an H5 (HyperText Markup Language 5) application, or a light application (also known as a mini-program, a lightweight application), or a cloud application, etc. The client 200 can be developed based on a software development kit (SDK) of a corresponding service provided by the server, such as an RTC SDK, etc. The client 200 can be deployed in an electronic device, and needs to rely on the device or some app in the device, etc. The electronic device can have a display screen and support information browsing, etc., such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0088] Referring to Figure 3 , Figure 3 A flowchart of a data storage method provided by one embodiment of the present specification is shown, the data storage method is applied to a management platform in a data storage system, the data storage system includes the management platform, a cache layer, and a capacity layer; specifically including the following steps:

[0089] Step 302: receiving a data storage request, wherein the data storage request carries to-be-stored data.

[0090] In one or more embodiments of the present specification, when the management platform in the data storage system receives the data storage request carrying the to-be-stored data, it is determined to start storing the to-be-stored data, that is, the data storage request is the premise of starting data storage.

[0091] Specifically, the data storage request refers to a request to store to-be-stored data, and the data storage request can be a data storage instruction carrying to-be-stored data, of course, the data storage request can also be an encrypted data storage instruction, further ensuring the security of the to-be-stored data, and the specific selection is based on the actual situation, and the present specification does not make any limitation on this. The to-be-stored data refers to the data that needs to be stored, and can also be understood as the data that is about to be stored. In actual application, the to-be-stored data can be transaction data, or user information, etc., and the specific selection is based on the actual situation, and the present specification does not make any limitation on this.

[0092] It should be noted that there are various ways to receive the data storage request. In an optional implementation, the management platform can receive the data storage request sent by the client actively. In another optional implementation, the management platform can send inquiry information to the client regularly to inquire whether the client needs to store data, and receive the data storage request sent by the client in the case that the client needs to store data.

[0093] Step 304: Adding the to-be-stored data to the idle cache block in the cache layer to obtain an effective cache block, wherein the idle cache block refers to a cache block available in the cache layer.

[0094] In one or more embodiments of the present specification, after receiving the data storage request, the to-be-stored data carried in the data storage request can be further added to the idle cache block in the cache layer to obtain an effective cache block.

[0095] Specifically, the effective cache block refers to a cache block in which the to-be-stored data has been stored, and the cache layer works at the front end of the capacity layer, which can be understood as a high-speed cache layer, and is used to process the data written by the user, write the data written by the user into the cache layer, and further write the effective data into the capacity layer in sequence. Referring to Figure 4 , Figure 4 An architecture diagram of a cache layer provided by one embodiment of the present specification is shown, which can be divided into two parts, namely a metadata area (Metadata) and a data area (Data). The data area is organized as a group of cache blocks (Chunk), for example, the data area includes cache block 0, cache block 1, cache block 2, cache block 3,..., and cache block N. Each cache block manages a plurality of sub-cache blocks (Block) with variable sector size (VSS, variable sector size), for example, cache block 2 includes sub-cache block 0, VSS, sub-cache block 1, VSS, sub-cache block 2, VSS,..., sub-cache block X, and VSS. The metadata area is used to persist the metadata page of the L2P table, wherein the metadata includes but is not limited to superblock metadata, block metadata, etc., which is selected according to the actual situation, and the present specification does not make any limitation on this.

[0096] In an optional implementation of the present specification, an available cache block, namely an idle cache block, can be found in the cache layer, and after obtaining the idle cache block, the to-be-stored data can be stored in any sub-cache block in the idle cache block to complete the storage of the to-be-stored data in the cache layer. That is, the above step of adding the to-be-stored data to the idle cache block in the cache layer to obtain an effective cache block can include the following steps:

[0097] The free cache block is searched in the cache layer, the to-be-stored data is stored into a first sub-cache block in the free cache block, and a valid cache block is obtained, wherein the first sub-cache block is any sub-cache block in the free cache block.

[0098] Exemplarily, the cache layer includes three cache blocks, namely, cache block 1, cache block 2 and cache block 3. Among the three cache blocks, cache block 1 and cache block 3 are available cache blocks, that is, cache block 1 and cache block 3 are determined as free cache block 1 and free cache block 3. At this time, one free cache block 3 can be determined from the above two free cache blocks, and the to-be-stored data is stored into any sub-cache block in the free cache block 3. Since the to-be-stored data is stored in the free cache block 3, the free cache block 3 is a valid cache block.

[0099] In actual application, when the number of free cache blocks in the cache layer is greater than or equal to two, one can be randomly selected from the free cache blocks as a free cache block for storing data. Further, according to the arrangement order of the cache blocks in the cache layer, the free cache block with a higher order can be preferentially selected. After the free cache block is determined, the to-be-stored data can also be stored in the sub-cache block with a higher order in the free cache block.

[0100] By applying the scheme of the embodiments of the present specification, the free cache block is searched in the cache layer, the to-be-stored data is stored into a first sub-cache block in the free cache block, and a valid cache block is obtained, thereby avoiding storing the to-be-stored data directly into the SSD, reducing the write amplification of the SSD, and improving the service life of the SSD.

[0101] It is worth noting that after the step of adding the to-be-stored data to the free cache block in the cache layer to obtain the valid cache block, the following steps can also be included:

[0102] The cache address of the valid cache block is determined.

[0103] The cache address is used as a storage address corresponding to the logical block address of the to-be-stored data.

[0104] Specifically, the cache address refers to the address of the valid cache block, and the range of the cache address is [size of the capacity layer, size of the capacity layer + size of the cache layer - 1]. After the to-be-stored data is stored into the valid cache block, the cache address of the valid cache block can be used as a storage address corresponding to the logical block address, and the storage address records the storage position of the to-be-stored data.

[0105] In an optional implementation of the present specification, a logical-to-physical address table (L2P) can be used as an index of the cached data to be stored, to indicate whether the data to be stored is cached in the cache layer and the specific cache address in the cache layer. After the data to be stored is written into the cache layer, the L2P table is updated to point the logical block address of the data to be stored to the cache address of the valid cache block.

[0106] Specifically, the L2P table is actually an L2P cache with two levels, which manages the full associative address mapping between the logical block address (LBA) and the physical location of the data storage in the capacity layer. Each LBA is associated with an L2P table metadata page, which maintains a set of entries for a series of LBAs, and each entry represents a physical address of an LBA, which directly points to the capacity layer, with a value range of [0, NAND SSD size-1], generally 32 bits, or a larger address range such as 64 bits, where NAND SSD is the capacity layer. The L2P table manages all pages using a page pointer array. Since the range of the LBA depends on the capacity of the capacity layer, using the L2P cache can avoid consuming a large amount of system memory (DRAM) for all LBA pages. All pages are supported on the capacity layer, and only hot pages that are frequently accessed are resident in the DRAM. See Figure 5 , Figure 5 An architecture diagram of a logical-to-physical address table provided by an embodiment of the present specification is shown, in which, among N logical block addresses 0-N, logical block address A, logical block address B, logical block address C, and logical block address D are taken as examples, logical block address A corresponds to page 1 in the L2P table, page 1 in the L2P table points to page 1 in the cache layer; logical block address B corresponds to page N in the L2P table, page N in the L2P table points to page N in the cache layer; logical block address C corresponds to page N+1 in the L2P table, page N+1 in the L2P table points to page N+1 in the cache layer; and logical block address D corresponds to page X in the L2P table, page X in the L2P table points to page X in the cache layer, where the system memory (DRAM) is used as the cache of the L2P table.

[0107] In practical applications, when a user's request for a specific LBA is received, the page number of the given LBA can be calculated and arranged by page number. If the page pointer is not empty, the L2P table directly returns the storage address corresponding to the LBA. Otherwise, the WSC first submits a read I / O to the cache layer, and then fills the page using the metadata from the cache layer. Finally, the L2P table returns the storage address corresponding to the LBA.

[0108] Since the size of the DRAM L2P cache configuration is limited, a page replacement policy is used to exchange pages between the DRAM and the cache tier. The basic idea is based on the Least Recently Used (LRU) page replacement algorithm. The WSC maintains a ranked list of DRAM resident pages, in which each page is sorted according to the page hit rate. The popular page will penetrate to the head of the ranked list, and the cold page will be evicted from the tail of the ranked list. When the required metadata page is not in the L2P cache and the L2P cache is full, the L2P table will exchange the metadata page between the DRAM and the cache tier. Among them, the popular page can be understood as the page frequently accessed, and the cold page can be understood as the page rarely accessed. Based on this, the L2P cache is a trade-off between performance and system resources. By adjusting the cache of the L2P, the performance of the data storage system can be adjusted. The larger the configured L2P cache, the fewer the triggered page evictions, and the higher the system performance.

[0109] By applying the scheme of the embodiments of the present specification, the cache address of the effective cache block is determined, and the cache address is taken as the storage address corresponding to the logical block address of the to-be-stored data, so that the data storage process is looped, and in the subsequent data query process, data query can be directly performed according to the storage address corresponding to the logical block address.

[0110] In an optional implementation manner of the present specification, when the to-be-stored data is stored into any sub-cache block in the idle cache block, the logical block address (LBA, Logical Block Addresses) of the to-be-stored data can be stored into the variable sector of the sub-cache block to record the storage address of the to-be-stored data.

[0111] Specifically, the variable sector (VSS) is used to record the logical block address given by the user for each sub-cache block. The logical block address is a general mechanism for describing the block where the data is located on the computer storage device, and can mean the address of a certain data block. Generally, the logical block address is numbered from 0 to locate the block, for example, the first block LBA=0, the second block LBA=1, and so on.

[0112] It should be noted that the VSS is used to record the LBA given by the user for each sub-cache block, which is the reverse mapping of the L2P entry. This reverse mapping is used to rebuild the L2P table when the WSC crashes unexpectedly. In addition, the WSC manages a physical-to-logical address table (P2L, Physical address to Logical Address) to the reverse mapping in the DRAM. This DRAM resident P2L is used to effectively find the related L2P entry of the block in the compression process.

[0113] Exemplarily, referring toFigure 6 , Figure 6 A user requests to update logical block address A and logical block address B in logical block address 0-N, and the data corresponding to logical block address A is stored in sub-cache block 4, and the data corresponding to logical block address B is stored in sub-cache block 5. The cache layer includes at least sub-cache blocks 0-6, and the L2P table includes at least pages 0-X, page 1 corresponds to logical block address A, and the storage address corresponding to page 1 is updated from the cache address of sub-cache block 0 to the cache address of sub-cache block 4. Similarly, page N corresponds to logical block address B, and the storage address corresponding to page N is updated from the cache address of sub-cache block 2 to the cache address of sub-cache block 5. That is, the entry of logical block address A from sub-cache block 0 to sub-cache block 4 and the entry of logical block address B from sub-cache block 2 to sub-cache block 5 in the L2P table are updated.

[0114] Step 306: In the case where the number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, the data in the valid cache blocks is sequentially written into the buffer area in the cache layer to generate aggregated data.

[0115] In one or more embodiments of the present specification, after receiving a data storage request, adding the data to be stored to the idle cache blocks in the cache layer, and obtaining the valid cache blocks, further triggering a data compression process in the case where the number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold. In the data compression process, the frequent update or temporary storage of data will be terminated, and the data in the valid cache blocks is read from the cache layer and written into the capacity layer. By caching the frequently updated or temporarily stored data on the cache layer, the writing to the NAND SSD in the capacity layer is maximized, and the service life of the SSD is improved. In practical applications, the data compression process can be to sequentially write the data in the valid cache blocks into the buffer area in the cache layer to generate aggregated data, and in the case where the amount of data stored in the buffer area reaches a second preset threshold, the aggregated data is stored to the capacity layer.

[0116] Specifically, the first preset threshold is a critical condition for writing the data in the cache block to the buffer area. If the number of cache blocks in which data has been stored in the cache layer reaches the first preset threshold, the data in the cache block can be written to the buffer area, and if it does not reach, no operation is performed. For example, the first preset threshold can be the proportion of the number of cache blocks in which data has been stored to the number of idle cache blocks, such as 80%, or a specific numerical value, which is selected according to actual conditions, and the present specification does not make any limitation on this. The aggregated data refers to the data generated by aggregating the data written to the buffer area.

[0117] For example, assuming the first preset threshold is 80%, there are 100 free cache blocks in the cache layer. The data to be stored is added to the free cache blocks in the cache layer. After obtaining the valid cache blocks, there are 86 cache blocks with stored data. The ratio of the number of cache blocks with stored data to the number of free cache blocks is 86%. Since 86% > 80%, it can be determined that the number of cache blocks with stored data in the cache layer has reached the first preset threshold. The data in the valid cache blocks is then written into the buffer in the cache layer in sequence to generate aggregated data.

[0118] In one optional implementation of this specification, the buffer includes multiple buffer blocks. Data from the valid cache blocks can be sequentially written into the buffer to generate aggregated data. That is, the step of sequentially writing data from the valid cache blocks into the buffer in the cache layer to generate aggregated data can include the following steps:

[0119] Traverse multiple buffer blocks in the buffer to determine the set of free buffer blocks in the buffer, wherein the set of free buffer blocks includes multiple free buffer blocks arranged in order;

[0120] Based on the order of the free buffer blocks, the data to be stored in the valid cache blocks is sequentially stored into the free buffer blocks to generate aggregated data.

[0121] Specifically, a free buffer block refers to an available buffer block. Assuming the buffer contains 10 buffer blocks, 0-9, where buffer blocks 3-6 are not free buffer blocks, the set of free buffer blocks can be determined as {buffer block 0, buffer block 1, buffer block 2, buffer block 7, buffer block 8, buffer block 9}.

[0122] See Figure 7 , Figure 7 This diagram illustrates the processing flow of another data storage method provided in one embodiment of this specification. It allows for periodic checks of used cache blocks. When the number of cache blocks containing stored data reaches a first preset threshold, the data to be stored in cache block N is sequentially written from sub-cache blocks 1, 2, and 4 into buffer blocks 0, 1, and 2 of buffer X. Similarly, the data to be stored in cache block N+1 is sequentially written from sub-cache blocks 1, 3, and 4 into buffer blocks 7, 8, and 9 of buffer X. Cache block N includes at least sub-cache blocks 0-6, and cache block N+1 includes at least sub-cache blocks 0-6.

[0123] The application of the scheme of the embodiment of the present specification traverses the plurality of buffer blocks in the buffer area, determines a set of idle buffer blocks in the buffer area, sequentially stores the to-be-stored data in the valid cache block into the idle buffer blocks according to the arrangement order of the idle buffer blocks, generates aggregated data, adjusts the random write of the user into sequential write, and batch generates the aggregated data, thereby reducing the write amplification and the hard disk wear.

[0124] In actual application, after the step of sequentially storing the to-be-stored data in the valid cache block into the idle buffer blocks and generating aggregated data, the following steps can be further included:

[0125] updating the valid cache block into an invalid cache block and erasing the data in the invalid cache block;

[0126] taking the erased invalid cache block as an idle cache block.

[0127] Specifically, the invalid cache block refers to a valid cache block that has written data into the buffer area, since the data in the valid cache block has been written into the buffer area, the data in the valid cache block can be erased, and the valid cache block can be updated into an invalid cache block.

[0128] It should be noted that after the to-be-stored data in the valid cache block is stored into the idle buffer block, the valid bitmap can be updated. The bitmap is used to indicate the use and state of the block in each region. For example, Figure 7 In the embodiment, after the data in the sub-cache block 1, the sub-cache block 2 and the sub-cache block 4 in the buffer block N is written into the buffer area X, the data in the sub-cache block 1, the sub-cache block 2 and the sub-cache block 4 is erased, and the sub-cache block 1, the sub-cache block 2 and the sub-cache block 4 are changed into invalid, at this time, the buffer block 0, the buffer block 1 and the buffer block 2 in the buffer area X are changed into valid.

[0129] The application of the scheme of the embodiment of the present specification updates the valid cache block into an invalid cache block, erases the data in the invalid cache block, and takes the erased invalid cache block as an idle cache block, which can timely release the invalid cache block, reset the invalid cache block as available space, and improve the resource utilization.

[0130] In actual application, after the to-be-stored data in the valid cache block is stored into the idle buffer block and the aggregated data is generated, since the storage position of the to-be-stored data has changed, the storage address corresponding to the logical block address of the to-be-stored data can be updated, that is, after the above step of sequentially storing the to-be-stored data in the valid cache block into the idle buffer block and generating the aggregated data, the following steps can be further included:

[0131] determining the buffer address of the idle buffer block;

[0132] The buffer address is used as a storage address corresponding to a logical block address of the data to be stored.

[0133] Specifically, the buffer address refers to an address of a free buffer block. In the embodiments of the present specification, the buffer address can be directly used as a storage address corresponding to a logical block address of the data to be stored. Further, an entry in the L2P table can be updated to refer to the buffer address. Figure 7 For example, in the L2P table, the logical block addresses corresponding to the sub-cache block 1, the sub-cache block 2, and the sub-cache block 4 are updated from the cache addresses of the sub-cache block 1, the sub-cache block 2, and the sub-cache block 4 to the buffer addresses of the buffer block 0, the buffer block 1, and the buffer block 2.

[0134] By applying the scheme of the embodiments of the present specification, the buffer address of the free buffer block is determined, and the buffer address is used as a storage address corresponding to a logical block address of the data to be stored, so that the data storage process is looped, and in the subsequent data query process, data can be directly queried according to the storage address corresponding to the logical block address.

[0135] Step 308: In a case where the amount of data stored in the buffer area reaches a second preset threshold, the aggregated data is stored to the capacity layer.

[0136] In one or more embodiments of the present specification, a data storage request is received, and the data to be stored is added to a free cache block in the cache layer. The valid cache block is obtained. In a case where the number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, the data in the valid cache block is sequentially written to the buffer area in the cache layer. After the aggregated data is generated, further, in a case where the amount of data stored in the buffer area reaches a second preset threshold, the aggregated data is stored to the capacity layer.

[0137] Specifically, the amount of data can be understood as the size of the stored data, and the second preset threshold is a critical condition for storing the aggregated data to the capacity layer. If the amount of data stored in the buffer area reaches the second preset threshold, the aggregated data can be stored to the capacity layer. If not, no operation is performed. For example, the second preset threshold can be a specific numerical value, such as 64 KB. It can also be a proportion of the amount of stored data to the total storage amount of the buffer area. The specific selection is made according to the actual situation, and the present specification does not make any limitation thereon. When the aggregated data is stored to the capacity layer, it can be directly stored, or the aggregated data can be compressed, and the compressed aggregated data is stored to the capacity layer, thereby reducing resource consumption.

[0138] For example, it is assumed that the second preset threshold is 64 KB, and the amount of data stored in the buffer area is 77 KB. Since 77 KB > 64 KB, it is determined that the amount of data stored in the buffer area reaches the second preset threshold, and the aggregated data is further stored to the capacity layer.

[0139] It should be noted that, since the period of user writing data is different, the L2P table can be used as a host-side flash translation layer (FTL, Flash Translation Layer) to serve as a buffer in the data storage process, and adjust the user's random write to sequential write to the cache layer. The regions of the capacity layer SSD are divided into two types. One is a compression region for compression write; the other is a garbage collection region for garbage collection, which is used to improve the efficiency of internal garbage collection, also known as GC region (Garbage Collection zone). When the invalid blocks in the garbage collection region reach the garbage collection threshold, defragmentation will be triggered. The valid blocks in the capacity layer will be moved to the GC region, and the original region will be reset as available space.

[0140] Referring to Figure 8 , Figure 8 A schematic diagram of a garbage collection process in a data storage method provided by one embodiment of the present specification is shown. In this example, two regions (region N and N+1) in the capacity layer have reached the garbage collection threshold. Among them, region N includes blocks 0-6, region N+1 includes blocks 0-6, and the GC region includes blocks 0-6. According to the valid bitmap, read the valid blocks in regions N and N+1, and append the valid blocks to the GC region. Then, update the valid bitmap. For example, blocks 3, 4, and 6 of region N are invalid; blocks 0, 1, and 2 in the GC region will become valid. Finally, update the L2P entries of these blocks. For example, the LBA entries associated with blocks 3, 4, and 6 of region N will be updated from blocks 3, 4, and 6 of region N to blocks 0, 1, and 2 of the GC region. Similarly, blocks 2, 4, and 5 of region N+1 are invalid; blocks 4, 5, and 6 in the GC region will become valid, and the L2P entries of these blocks will be updated. For example, the LBA entries associated with blocks 2, 4, and 5 of region N+1 will be updated from blocks 2, 4, and 5 of region N+1 to blocks 4, 5, and 6 of the GC region. Since the SSD needs to be erased before writing data, regions N and N+1 are reset to inform the SSD to erase these regions and add these regions to the tail of the available region list. With the processing of garbage collection, the GC region will be full. Then, the GC region will be marked as a common region (common zone), and another blank region will be selected as a new GC region.

[0141] The application of the scheme of the embodiment of the present specification receives a data storage request, wherein the data storage request carries to-be-stored data; the to-be-stored data is added to an idle cache block in a cache layer to obtain a valid cache block, wherein the idle cache block refers to a cache block available in the cache layer; in a case where the number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, data in the valid cache block is sequentially written into a buffer in the cache layer to generate aggregated data; in a case where the amount of data stored in the buffer reaches a second preset threshold, the aggregated data is stored to a capacity layer. By sequentially writing data in the valid cache block into the buffer in the cache layer and batch storing aggregated data stored in the buffer to the capacity layer, the write amplification and wear of the capacity layer are reduced, and the overall performance of the data storage system and the durability of the data storage system are further improved.

[0142] In actual application, after the aggregated data is stored to the capacity layer, the following steps can be further included:

[0143] The physical address of each data storage in the capacity layer is determined.

[0144] The physical address is taken as a storage address corresponding to a logical block address of the to-be-stored data.

[0145] Specifically, the physical address refers to an address of data storage in the capacity layer. In the embodiment of the present specification, the physical address can be directly taken as a storage address corresponding to a logical block address of the to-be-stored data. Further, an entry can be further updated in the L2P table.

[0146] The application of the scheme of the embodiment of the present specification determines the physical address of each data storage in the capacity layer, and takes the physical address as a storage address corresponding to a logical block address of the to-be-stored data, so that the data storage process is loopable, and in a subsequent data query process, data can be directly queried according to the storage address corresponding to the logical block address.

[0147] Referring to Figure 9 , Figure 9 A flowchart of a data query method provided by one embodiment of the present specification is shown, the data query method is applied to a management platform in a data storage system, and specifically includes the following steps:

[0148] Step 902: receiving a data query request, wherein the data query request carries a logical block address of to-be-queried data.

[0149] Step 904: finding a storage address corresponding to the logical block address.

[0150] Step 906: finding the to-be-queried data according to the storage address, wherein the to-be-queried data is stored by the above-mentioned data storage method.

[0151] Specifically, the data query request refers to a request for querying the to-be-queried data, and can also be understood as a request for querying target data. The data query request carries a logical block address of the to-be-queried data. In an actual query process, the management platform can use the logical block address to find a storage address corresponding to the logical block address, and further use data corresponding to the storage address as the to-be-queried data.

[0152] According to the scheme of the embodiment of the present specification, a data query request is received, wherein the data query request carries a logical block address of to-be-queried data; a storage address corresponding to the logical block address is found; and the to-be-queried data is found according to the storage address, wherein the to-be-queried data is stored by using the data storage method. By finding the storage address corresponding to the logical block address and finding the to-be-queried data according to the storage address, the data query efficiency is improved.

[0153] The following describes the data storage method in combination with the accompanying Figure 10 The data storage method provided in the present specification is further described by taking the application of the data storage method in a transaction scenario as an example. Among them, Figure 10 A processing block diagram of a data storage method provided by an embodiment of the present specification is shown, and specifically includes:

[0154] The data storage system receives I / O requests from the transaction program. The data written by the user will be appended to the free cache blocks available in the cache layer. The data storage system updates the L2P table, and the cache address in the cache layer will be inserted into the entry for the given LBA of the user. Then, the transaction program is confirmed for data writing. The usage of the cache layer is checked regularly, the data is compressed, and the data is moved from the cache layer to the ZNS (Zoned Namespace) flash area in the capacity layer, and the related L2P entry will be updated to now point to the location on the ZNS flash. At the same time, the usage of the capacity layer is checked, and if there is a region in the capacity layer that reaches the GC threshold, the defragmentation is performed. During the process of compressing data to defragmentation, when the data is moved to a new region, the related L2P entry will be updated to the new storage address.

[0155] According to the scheme of the embodiment of the present specification, in the scenario of small I / O requests, the hard disk write amplification is effectively eliminated by aggregating small I / O, the hard disk performance is improved, in the scenario that the write I / O has local hot spots, the cache layer absorbs the hot spot data, and the system performance is further improved, effectively solving the QLC write amplification and life problem, and solving the problem that the random write performance is only 1% of the sequential write performance.

[0156] Referring to FIG. 11, Figure 11a A processing process flow diagram of a data storage method provided by an embodiment of the present specification is shown, and specifically includes the following steps:

[0157] Step 1102: receiving a data storage request, wherein the data storage request carries to-be-stored data.

[0158] Step 1104: searching for a free cache block in the cache layer, storing the to-be-stored data into a first sub-cache block in the free cache block, and obtaining a valid cache block.

[0159] Specifically, referring to Figure 11b , Figure 11b a schematic diagram of a cache block is shown, wherein Figure 11b The cache block 1, the cache block 2, the cache block 3, and the cache block 4 are included in the cache layer, and the shaded part is the to-be-stored data.

[0160] Step 1106: in a case where a number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, traversing a plurality of buffer blocks in the buffer area to determine a set of free buffer blocks in the buffer area.

[0161] Step 1108: according to an arrangement order of the free buffer blocks, sequentially storing the to-be-stored data in the valid cache block into the free buffer blocks to generate aggregated data.

[0162] Specifically, referring to Figure 11c , Figure 11c a schematic diagram of a buffer area is shown, and the to-be-stored data in the cache block 1, the cache block 2, the cache block 3, and the cache block 4 in the cache layer are sequentially stored into free buffer blocks in the buffer area to obtain the buffer area as shown in Figure 11b The shaded part is the to-be-stored data, Figure 11c The data in the 2*4 shaded part in the buffer area is aggregated data. Figure 11c

[0163] Step 1110: in a case where an amount of data stored in the buffer area reaches a second preset threshold, storing the aggregated data into the capacity layer.

[0164] Specifically, the specific implementation process of the above steps 1102-1110 can be referred to Figure 3 embodiment, and details are not described herein again.

[0165] By applying the scheme of the embodiment of the present specification, the data in the valid cache block is sequentially written into the buffer area in the cache layer, and the aggregated data stored in the buffer area is batch-stored into the capacity layer, thereby reducing the write amplification and wear of the capacity layer, and further improving the overall performance of the data storage system and the durability of the data storage system.

[0166] Corresponding to the above data storage method embodiment, the present specification also provides a data storage device embodiment, Figure 12 ​A structural diagram of a data storage device is shown. As shown in the figure, Figure 12 The data storage device is applied to a management platform in a data storage system, and the data storage system includes the management platform, a cache layer, and a capacity layer. The device includes:

[0167] A first receiving module 1202 is configured to receive a data storage request, wherein the data storage request carries to-be-stored data.

[0168] An adding module 1204 is configured to add the to-be-stored data to a free cache block in the cache layer to obtain a valid cache block, wherein the free cache block refers to a cache block available in the cache layer.

[0169] A generating module 1206 is configured to, in a case where a number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, sequentially write data in the valid cache block to a buffer area in the cache layer to generate aggregated data.

[0170] A storing module 1208 is configured to, in a case where an amount of data stored in the buffer area reaches a second preset threshold, store the aggregated data to the capacity layer.

[0171] Optionally, the buffer area includes a plurality of buffer blocks; the generating module 1206 is further configured to traverse the plurality of buffer blocks in the buffer area to determine a set of free buffer blocks in the buffer area, wherein the set of free buffer blocks includes a plurality of free buffer blocks arranged in sequence; and sequentially store the to-be-stored data in the valid cache block to the free buffer blocks according to an arrangement order of the free buffer blocks to generate the aggregated data.

[0172] Optionally, the device further includes a first determining module configured to determine a buffer address of the free buffer block; and the buffer address is used as a storage address corresponding to a logical block address of the to-be-stored data.

[0173] Optionally, the device further includes an erasing module configured to update the valid cache block to an invalid cache block, erase data in the invalid cache block, and use the erased invalid cache block as the free cache block.

[0174] Optionally, the cache block includes a plurality of sub-cache blocks with variable sectors; the adding module 1204 is further configured to find the free cache block in the cache layer, store the to-be-stored data to a first sub-cache block in the free cache block to obtain the valid cache block, wherein the first sub-cache block is any sub-cache block in the free cache block; and store a logical block address of the to-be-stored data to the variable sector of the sub-cache block.

[0175] Optionally, the apparatus further comprises a second determining module configured to determine a cache address of the valid cache block; and take the cache address as a storage address corresponding to the logical block address of the data to be stored.

[0176] Optionally, the apparatus further comprises a third determining module configured to determine a physical address of each data storage in the capacity layer; and take the physical address as a storage address corresponding to the logical block address of the data to be stored.

[0177] According to the scheme of the embodiment of the present specification, a data storage request is received, wherein the data storage request carries data to be stored; the data to be stored is added to a free cache block in a cache layer to obtain a valid cache block, wherein the free cache block refers to a cache block available in the cache layer; in a case where a number of cache blocks in which data has been stored in the cache layer reaches a first preset threshold, data in the valid cache block is sequentially written into a buffer in the cache layer to generate aggregated data; and in a case where an amount of data stored in the buffer reaches a second preset threshold, the aggregated data is stored to a capacity layer. By sequentially writing data in the valid cache block into the buffer in the cache layer and storing the aggregated data stored in the buffer to the capacity layer in batches, the write amplification and wear of the capacity layer are reduced, and the overall performance of the data storage system and the durability of the data storage system are further improved.

[0178] The above is a schematic scheme of a data storage apparatus of the embodiment. It should be noted that the technical scheme of the data storage apparatus belongs to the same concept as the technical scheme of the data storage method described above, and the details of the technical scheme of the data storage apparatus that are not described in detail can be referred to the description of the technical scheme of the data storage method.

[0179] Corresponding to the data query method embodiment described above, the present specification also provides a data query apparatus embodiment, Figure 13 A structural schematic diagram of a data query apparatus provided by one embodiment of the present specification is shown. As shown in the figure, Figure 13 The data query apparatus is applied to a management platform in a data storage system, and the apparatus comprises:

[0180] A second receiving module 1302 is configured to receive a data query request, wherein the data query request carries a logical block address of data to be queried;

[0181] A first finding module 1304 is configured to find a storage address corresponding to the logical block address;

[0182] A second finding module 1306 is configured to find the data to be queried according to the storage address, wherein the data to be queried is stored by the data storage method described above.

[0183] According to the scheme of the embodiment of the present specification, a data query request is received, wherein the data query request carries a logical block address of to-be-queried data; a storage address corresponding to the logical block address is found; and the to-be-queried data is found according to the storage address, wherein the to-be-queried data is stored by using the data storage method. By finding the storage address corresponding to the logical block address and finding the to-be-queried data according to the storage address, the data query efficiency is improved.

[0184] The above is a schematic scheme of a data query device according to the embodiment. It should be noted that the technical scheme of the data query device belongs to the same concept as the technical scheme of the data query method described above, and the details of the technical scheme of the data query device that are not described in detail can be referred to the description of the technical scheme of the data query method.

[0185] Figure 14 A structural block diagram of a computing device is shown, which is provided by an embodiment of the present specification. The components of the computing device 1400 include but are not limited to a memory 1410 and a processor 1420. The processor 1420 is connected to the memory 1410 through a bus 1430, and a database 1450 is used to save data.

[0186] The computing device 1400 further includes an access device 1440, which enables the computing device 1400 to communicate via one or more networks 1460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1440 can include one or more of any type of network interface (for example, a network interface card (NIC)), such as a IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and the like.

[0187] In an embodiment of the present specification, the above components of the computing device 1400 and Figure 14Other components not shown can also be connected to each other, such as through a bus. It should be understood that Figure 14 The illustrated computing device structural block diagram is merely for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced by those skilled in the art as needed.

[0188] The computing device 1400 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. The computing device 1400 can also be a mobile or stationary server.

[0189] The processor 1420 is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the data storage method or the data query method.

[0190] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device belongs to the same concept as the technical scheme of the data storage method or the data query method, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the data storage method or the data query method.

[0191] An embodiment of the present specification further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps of the data storage method or the data query method.

[0192] The above is a schematic scheme of the computer-readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium belongs to the same concept as the technical scheme of the data storage method or the data query method, and the details of the technical scheme of the storage medium that are not described in detail can be referred to the description of the technical scheme of the data storage method or the data query method.

[0193] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer program causes the computer to execute the steps of the data storage method or the data query method.

[0194] The above is a schematic scheme of the computer program of the embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the data storage method or the data query method described above belong to the same concept, and the technical scheme of the computer program which is not described in detail can be referred to the description of the technical scheme of the data storage method or the data query method.

[0195] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order described in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or possible.

[0196] The computer instructions include computer program code, which can be in the form of source code, object code, executable code, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0197] It should be noted that for the foregoing method embodiments, in order to facilitate description, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the embodiments of the present specification are not limited by the order of the described actions, because according to the embodiments of the present specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the embodiments of the present specification.

[0198] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0199] The preferred embodiments of the present specification disclosed above are only used to help explain the present specification. Alternative embodiments do not describe all the details and limit the present application to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of the embodiments of the present specification. The present specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present specification, so that those skilled in the art can well understand and utilize the present specification. The present specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data storage method applied to a management platform in a data storage system, the data storage system comprising the management platform, a cache layer, and a capacity layer; the method comprising: Receive a data storage request, wherein the data storage request carries data to be stored; The data to be stored is added to a free cache block in the cache layer to obtain a valid cache block, wherein the free cache block refers to a cache block that is available in the cache layer; When the number of cache blocks containing stored data in the cache layer reaches a first preset threshold, the data in the valid cache blocks is sequentially written into the buffer in the cache layer to generate aggregated data. The buffer includes multiple buffer blocks, which are used to store the data to be stored in the valid cache blocks to generate aggregated data. When the amount of data stored in the buffer reaches a second preset threshold, the aggregated data is stored in the capacity layer.

2. The method according to claim 1, wherein the buffer comprises a plurality of buffer blocks; the step of sequentially writing data from the valid cache blocks into the buffers of the cache layer to generate aggregated data comprises: Traverse multiple buffer blocks in the buffer to determine a set of free buffer blocks in the buffer, wherein the set of free buffer blocks includes multiple free buffer blocks arranged in order; According to the arrangement order of the free buffer blocks, the data to be stored in the valid cache blocks are sequentially stored into the free buffer blocks to generate aggregated data.

3. The method according to claim 2, after the step of sequentially storing the data to be stored in the effective cache block into the free buffer block to generate aggregated data, further comprising: Determine the buffer address of the free buffer block; The buffer address is used as the storage address corresponding to the logical block address of the data to be stored.

4. The method according to claim 2, after the step of sequentially storing the data to be stored in the effective cache block into the free buffer block to generate aggregated data, further comprising: Update the valid cache block to an invalid cache block and erase the data in the invalid cache block; Use the erased invalid cache blocks as free cache blocks.

5. The method according to claim 1, wherein the cache block comprises a plurality of sub-cache blocks having variable sectors; The step of adding the data to be stored to a free cache block in the cache layer to obtain a valid cache block includes: In the cache layer, a free cache block is searched, and the data to be stored is stored into the first sub-cache block in the free cache block to obtain a valid cache block, wherein the first sub-cache block is any sub-cache block in the free cache block; The logical block address of the data to be stored is stored in the variable sector of the sub-cache block.

6. The method according to claim 1, after the step of adding the data to be stored to a free cache block in the cache layer to obtain a valid cache block, further comprising: Determine the cache address of the valid cache block; The cache address is used as the storage address corresponding to the logical block address of the data to be stored.

7. The method according to claim 1, further comprising, after the step of storing the aggregated data in the capacity layer: Determine the physical address of each data storage in the capacity layer; The physical address is used as the storage address corresponding to the logical block address of the data to be stored.

8. A data query method, applied to a management platform in a data storage system, the method comprising: Receive a data query request, wherein the data query request carries the logical block address of the data to be queried; Find the memory address corresponding to the logical block address; The data to be queried is located according to the storage address, wherein the data to be queried is stored by the data storage method according to any one of claims 1-7.

9. A data storage system, the system comprising a management platform, a cache layer, and a capacity layer; The management platform is used to receive data storage requests, wherein... The data storage request carries the data to be stored; The data to be stored is transferred to the cache layer; The caching layer is used to store the data to be stored into a free cache block to obtain a valid cache block, wherein the free cache block refers to a cache block available in the caching layer; The management platform is also used to monitor the number of cache blocks containing stored data in the cache layer, and send a write command to the cache layer when the number reaches a first preset threshold. The cache layer is further configured to write the data in the effective cache block into a buffer sequentially based on the write instruction to generate aggregated data. The buffer includes multiple buffer blocks, which are used to store the data to be stored in the effective cache block to generate aggregated data. The management platform is also used to monitor the amount of data stored in the buffer, and send a forwarding instruction to the cache layer when the amount of data reaches a second preset threshold. The caching layer is also used to forward the aggregated data to the capacity layer based on the forwarding instruction; The capacity layer is used to store the aggregated data.

10. The system according to claim 9, wherein the management platform is further configured to receive data query requests, wherein, The data query request carries the logical block address of the data to be queried; the storage address corresponding to the logical block address is found; and the data to be queried is found based on the storage address.

11. A data storage device, applied to a management platform in a data storage system, the data storage system including the management platform, a cache layer, and a capacity layer; the device comprising: The first receiving module is configured to receive a data storage request, wherein the data storage request carries data to be stored; The addition module is configured to add the data to be stored to a free cache block in the cache layer to obtain a valid cache block, wherein the free cache block refers to a cache block available in the cache layer; The generation module is configured to, when the number of cache blocks containing stored data in the cache layer reaches a first preset threshold, sequentially write the data in the valid cache blocks into a buffer in the cache layer to generate aggregated data. The buffer includes multiple buffer blocks, which are used to store the data to be stored in the valid cache blocks to generate aggregated data. The storage module is configured to store the aggregated data to the capacity layer when the amount of data stored in the buffer reaches a second preset threshold.

12. A data query device, applied to a management platform in a data storage system, the device comprising: The second receiving module is configured to receive a data query request, wherein the data query request carries the logical block address of the data to be queried; The first lookup module is configured to look up the storage address corresponding to the logical block address; The second search module is configured to search for the data to be queried based on the storage address, wherein the data to be queried is stored using the data storage method according to any one of claims 1-7.

13. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7 or claim 8.

14. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method of any one of claims 1 to 7 or 8.

Images