Data processing method, system, device, storage system and medium

Abstract

The embodiments of the present application provide a data processing method, system, device, storage system and medium. The method includes: receiving a logical address corresponding to the data to be processed; storing the logical address as a first access unit in a first storage pool; when the number of first access units stored in the first storage pool meets the requirement, combining multiple first access units into a logical address group, storing the logical address group as a second access unit in a second storage pool, so as to process the data based on the second access unit. Since the first access unit in the first storage pool has a small access unit and a mapping table, when writing or reading data, it can be implemented based on the first storage pool, while the second storage block in the second storage pool supports a large access unit and a large mapping table, avoiding partial data writing or reading based on the second storage pool, thereby effectively alleviating problems such as write amplification and poor service quality. While meeting data storage needs, it can reduce the hardware cost of the storage system.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to data processing methods, systems, devices, storage systems and media. Background Art

[0002] With the development of cloud storage technology and demand, the capacity of solid-state drives (SSDs) is increasing. This requires a larger cache to store address mapping tables. The more cache used, the higher the cost. Therefore, it is necessary to find ways to reduce cache costs.

[0003] In the prior art, by introducing Large Access Units (Large AUs) into solid-state drives, the number of logical address to physical address (L2P) secondary mapping tables managed by the flash translation layer (FTL) can be reduced. However, when a Large Access Unit is read or written, the entire Large Access Unit (Large AU) must first be read, merged with the newly received data block, and then written back to the flash memory. If the Large Access Unit contains valid data and the newly received data block is smaller than the space of a Large Access Unit, this will result in greater write amplification (WA). To address these issues, the prior art utilizes a cache to store the last Large Access Unit by deploying a sequential stream. Generally, an application may open hundreds of logical blocks in parallel, requiring a lot of cache space to store the corresponding sequential streams. Although a large space is used, when writing data directly, a small access unit size data writing method is still used, which in turn causes greater write amplification. Summary of the Invention

[0004] To solve or improve the problems existing in the prior art, the embodiments of the present application provide data processing methods, systems, devices, storage systems and media.

[0005] In one aspect, in one embodiment of the present application, a data processing method is provided. The method includes:

[0006] Receive the logical address corresponding to the data to be processed;

[0007] storing the logical address as a first access unit in a first storage pool;

[0008] When the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process data based on the second access unit.

[0009] In a second aspect, in one embodiment of the present application, a data processing system is provided. The system includes:

[0010] a first storage pool for receiving a logical address corresponding to data to be processed; storing the logical address as a first access unit in the first storage pool; and when a required number of first access units stored in the first storage pool is met, combining a plurality of first access units into a logical address group, and storing the logical address group as a second access unit in the second storage pool;

[0011] The second storage pool is used to store second access units obtained by combining logical addresses in the multiple first access units, and process data based on the second access units.

[0012] In a third aspect, in one embodiment of the present application, an electronic device is provided, including a memory and a processor;

[0013] The memory includes: a non-solid-state hard disk and a solid-state hard disk, the non-solid-state hard disk stores a data processing program, and the solid-state hard disk is used to store data to be processed;

[0014] The processor is coupled to the memory and configured to execute the output processing program for:

[0015] Receive the logical address corresponding to the data to be processed sent by the host;

[0016] storing the logical address as a first access unit in a first storage pool;

[0017] When the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process data based on the second access unit.

[0018] In a fourth aspect, in one embodiment of the present application, a storage system based on a solid-state drive is provided, comprising: a host and a solid-state drive mounted on the host;

[0019] The solid-state hard disk is used to receive the logical address corresponding to the data to be processed sent by the host;

[0020] storing the logical address as a first access unit in a first storage pool;

[0021] When the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process data based on the second access unit.

[0022] In a fifth aspect, in one embodiment of the present application, a computer-readable medium is provided on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the method described in the first aspect.

[0023] The technical solution provided by the embodiment of the present application is that when the host has data that it wants to store in the solid-state drive, it is necessary to establish a mapping table for managing the mapping relationship between the logical address and the physical address of the data to be processed. When storing, the logical address of the data to be processed can be first stored in the first access unit in the first storage pool; when the data volume of the logical address in the first storage pool meets the requirements (for example, when the data volume of the logical address in the first storage pool is an integer multiple of the data capacity of the second access unit in the second storage pool), the logical addresses in the multiple first access units can be transferred and stored in at least one second access unit. Since the density of flash memory particles in the second storage pool is greater than that of flash memory particles in the first storage pool, in other words, the first access unit in the first storage pool has a smaller access unit and mapping table. When data is written or read, it can be implemented based on the first storage pool, while the second storage block in the second storage pool supports a larger access unit and a larger mapping table, avoiding partial data writing or reading based on the second storage pool as much as possible, thereby effectively alleviating problems such as write amplification and poor service quality. Furthermore, while ensuring data storage needs, the hardware cost of the storage system can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0026] Figure 2 A schematic flow chart of a method for performing storage based on a first storage pool provided in an embodiment of the present application;

[0027] Figure 3A schematic diagram of the structure of the first storage pool provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of a flow chart of a combined storage method provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of the structure of the second storage pool provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of the structure of a data processing system provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of the structure of a storage system based on a solid-state drive provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to enable people skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0034] In some processes described in the specification, claims and the above-mentioned figures of this application, multiple operations appearing in a specific order are included. These operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit "first" and "second" to different types. In addition, the embodiments described below are only some of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0035] With the advancement of storage technology, especially with the widespread adoption of cloud storage technology, the cost of cloud storage devices has become increasingly prominent. Since the development of NAND Flash, flash memory particles have undergone four innovations. Based on the density of the electronic cells arranged within the NAND Flash particles, flash memory particles are now classified as SLC, MLC, TLC, and QLC flash memory. The storage density of these flash memory particles has been greatly improved, and the cost per bit has been significantly reduced. On large-capacity QLC drives, it is desirable to use larger access units to reduce DRAM usage within solid-state drives (SSDs). However, choosing a larger access unit also brings certain drawbacks. For example, a larger access unit can lead to read-modify-write (RMW) operations, resulting in additional write amplification (WA), which can impact the performance and lifespan of the SSD. Therefore, a device is needed that can support data processing with larger access units while avoiding the adverse effects of write amplification. This will be illustrated below through specific embodiments.

[0036] First, let’s explain the main technical terms:

[0037] SSD (Solid State Drive): Solid-state drives mainly include two categories, one is based on flash memory particles (such as SLC, MLC, TLC, QLC, PLC), and the other is based on DRAM. The technical solution of this application is an SSD based on flash memory particle technology. Among them, SLC (single-level cell) has the best stability and the longest lifespan (theoretically can be erased and written 100,000 times), but also the highest cost. MLC dual-bit cells (each Cell stores 2 data, there are 22=4 states, 00 / 01 / 10 / 11, so there are four different voltage states), that is, there are only two different voltage states) have lower performance, lifespan (theoretically can be erased and written 10,000 times), and cost than SLC. TLC triple-bit cells (each Cell stores 3 data, there are 23=8 states, and there are eight different voltage states) are low in cost. QLC and PLC will be specifically described in the following embodiments. The more data stored per Cell, the higher the capacity per unit area.

[0038] AU (Access Unit): Access unit, also the mapping management unit within the storage disk.

[0039] QoS (Quality of Service): Quality of Service. QoS parameters mainly include: transmission bandwidth, transmission delay and jitter, packet loss rate, etc.

[0040] WA (Write Amplification): Write amplification generally describes the ratio of the amount of data written to the user data written in NAND. When writing new data, if the SSD controller cannot find a page to write to, it will perform garbage collection. The garbage collection mechanism will then merge the valid data in some data blocks into other data blocks, then erase the invalid data in these blocks, and then write the new data into these blocks. During this entire process, in addition to writing the user data, the SSD actually also writes some data merged from other data blocks. In other words, the actual written data is greater than the user data, thus causing write amplification.

[0041] FTL (Flash Translation Layer): A general term for SSD mapping management and related algorithms. It essentially represents a mapping between LBAs and PBAs. When the file system sends a command to write or update a specific logical page, the FTL actually writes the data to a different, free physical page, updates the mapping table (the data that associates LBAs and PBAs), and marks the "old data" on this page as "invalid." (The updated data has been written to the new PBA, so the data at the old address is invalid.)

[0042] LBA (Logical Block Address): Logical block address is a common mechanism used in data storage devices to indicate the location of data. It refers to the physical address of a data block or the data block pointed to by a physical address.

[0043] L2P (Logical to Phsycal): Mapping table from logical address to physical address.

[0044] GC (Garbage Collection): Garbage collection. The solid-state drive controller will first copy all valid data (data still in use) and write it to blank pages in different data areas, erase all data units in the current data area, and then start writing new data to the data area that has just been erased.

[0045] like Figure 1 This is a flow chart of a data processing method provided in an embodiment of the present application. In actual application scenarios, the execution subject of this method can be a storage device (e.g., a local storage device, a cloud storage device, etc.). The method specifically includes the following steps:

[0046] 101: Receive the logical address corresponding to the data to be processed.

[0047] 102: Store the logical address as a first access unit in a first storage pool.

[0048] 103: When the number of first access units stored in the first storage pool meets the requirement, combine multiple first access units into a logical address group, and store the logical address group as a second access unit in the second storage pool to process data based on the second access unit.

[0049] The data to be processed here can be data to be read from or written to a storage device. The data to be processed will be stored at a corresponding physical address. However, when reading or writing the data to be processed, the operating system needs to find the corresponding physical address by accessing the logical address in the first access unit corresponding to the mapping table, thereby performing the corresponding data processing operation.

[0050] It should be noted that the first storage pool mentioned here can be an SLC (single-level cell) pool. SLC stores 1 bit of data in each storage cell of the NAND flash memory. Whether the stored data represents "0" or "1" is determined by the V th The voltage threshold is used to determine the voltage. For NAND flash memory, writing (programming) is to charge it so that its voltage exceeds the voltage judgment point. The storage cell indicates 0 - programmed. If it is not charged or the voltage threshold is lower than the voltage judgment point, it indicates 1 - erased. The first access unit mentioned here is the mapping management unit in the SLC pool with a unit of 4KB. When performing data read and write operations based on the first storage pool, the first access unit is used as the basic operation unit, while when performing erase operations, the logical block is used as the basic operation unit.

[0051] The second storage pool can be a QLC (Quad-Level cell) pool and / or a PLC (Penta-level cell) pool. QLC stores 4 bits of data per storage cell, while PLC stores 5 bits of data per storage cell. The stored data is also determined based on the voltage threshold. In other words, the second access unit can store four or five times the data flow of the first access unit (depending on the second access unit).

[0052] Although the second access unit of the second storage pool can store more data, when managing the mapping relationship, it is also managed based on the second access unit that supports larger data capacity as the basic unit. For example, during the reading process, since the physical address of the 4KB logical block can be calculated based on the starting physical position of the second access unit and its relative offset in the second access unit, there is no QoS penalty for random 4KB reads. When the application updates a 4KB data in a second access unit, read-modify-write (RMW) is required. The FTL needs to read the entire second access unit (N*4KB), merge it with the incoming 4KB data, and then write it back to the NAND flash memory. This means that N times of write amplification (WA) are introduced. Therefore, when the host needs to process the data to be processed, it can be implemented based on the first storage pool, because the first access unit in the first storage pool is based on 4KB as the basic management unit, which is consistent with the mainstream logical block size of the user host, both 4KB, which can well solve the problems caused by write amplification.

[0053] In actual applications, the logical address is first stored in the first storage pool as a first access unit. When the number of first access units stored in the first storage pool meets the requirements, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit. In other words, the logical address of the data to be processed is first stored in the first access unit in the first storage pool. When the number of first access units can meet the storage requirements of the second access unit, the physical addresses stored in the multiple first access units are directly transferred to the second access unit. Based on this combination of the first storage pool and the second storage pool, after investing a small amount of high-cost first storage pool, problems such as write amplification and substandard quality of service (QoS) can be alleviated, and the low-cost advantage of the second access unit in supporting the management of larger amounts of data can be fully realized.

[0054] like Figure 2 A flow chart of a method for storing data based on a first storage pool provided in an embodiment of the present application. Figure 2 As can be seen from FIG, storing the logical address as the first access unit in the first storage pool specifically includes the following steps:

[0055] 201: Store the logical address corresponding to the data to be processed in the first access unit.

[0056] 202: Determine a sequential flow identifier of the logical address and a first unit sequence of the first access unit.

[0057] 203: Based on the first unit sequence, group multiple first access units into a first logical block.

[0058] 204: Store the plurality of first logical blocks into the first storage pool.

[0059] It should be noted that, as described in step 201, after storing the logical address corresponding to the data to be processed in the first access unit, it also includes: determining the mapping relationship between the physical address of the data to be processed and the logical address; and generating a first mapping table corresponding to the first access unit based on the mapping relationship.

[0060] In actual applications, when transferring the logical address in the first access unit to the second access unit, it is not a simple matter of moving valid data from the first storage pool to the second storage pool, but it is necessary to maintain the valid logical blocks for each data stream in the first storage pool. The first mapping table of the first logical block in the first storage pool is still managed at a granularity of 4KB. In order to facilitate data management, when storing the first access unit, it is necessary to store it in each first logical block in the order of the first unit. Then, a first storage pool consisting of multiple first logical blocks storing logical addresses in sequence can be obtained. In order to control costs, the number of first storage pools is limited. When certain conditions are met, the logical addresses in the first storage pool will be transferred to the second storage pool with lower costs according to the access unit or logical block.

[0061] To facilitate understanding, specific examples are given below.

[0062] like Figure 3 A schematic diagram of the structure of the first storage pool provided in the embodiment of the present application. Figure 3 As can be seen in FIG, assuming that the first storage pool has four logical blocks, namely SLC BLK0, SLC BLK1, SLC BLK2, and SLC BLK3. Each logical block contains multiple first access units AU. For example,

[0063] In SLC BLK0, the first access units are AU0, AU1, AU2, AU3, AU4, AU5, AU6, and AU7;

[0064] The first access units in SLC BLK1 are AU8, AU9, AU10, AU11, AU12, AU13, AU14, and AU15.

[0065] The logical addresses in the first access unit in SLC BLK0 are also marked with corresponding sequential stream identifiers, namely: stream1_0, stream2_0, stream3_0, stream4_0, stream1_1, stream2_1, stream3_1, stream4_1;

[0066] The logical addresses in the first access unit in SLC BLK1 are also marked with corresponding sequential stream identifiers, namely: stream1_2, stream2_2, stream3_2, stream4_2, stream1_3, stream2_3, stream3_3, stream4_3. Specifically, the first access units in SLC BLK2 and SLC BLK3 are as follows: Figure 3 As shown, no further details are given here. The multiple first access units in each first logical block are stored in the order of the first units. Moreover, the logical addresses between the multiple first logical blocks also comply with the order rules of the sequential flow identifiers, which facilitates the subsequent transfer of the logical addresses according to the sequential flow.

[0067] In one or more embodiments of the present application, when the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process the data based on the second access unit, including: if the amount of data of the logical address in the first storage pool is greater than the first threshold; or the amount of data of the first access unit is an integer multiple of the data capacity of one second access unit; then multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process the data based on the second access unit.

[0068] As previously mentioned, the second access unit supports storage with a larger data capacity. To mitigate issues such as write amplification when reading and writing data to the second access unit, restrictions can be set for the second access unit corresponding to the logical address group formed by combining the logical addresses in the first access unit. The requirements mentioned here can be of various types. For example, a first threshold can be set for the first storage pool to process logical addresses, that is, when the amount of data of the logical addresses in the first storage pool or the number of first access units generated in the first storage pool containing logical addresses is reached. Alternatively, the first threshold can be omitted. When the number of first access units or the amount of data of the logical addresses is an integer multiple of the data capacity of at least one second access unit, the logical addresses in the first storage pool are transferred to the second access unit. This approach can reduce the data capacity requirements of the first storage pool and is equivalent to achieving real-time transfer. As previously mentioned, when reading and writing data in the second storage pool, the second access unit is used as the minimum basic unit for read and write operations. Assuming that the data capacity of the second access unit is N times the data capacity of the first access unit, when there are N first access units in the first storage pool storing logical addresses, the logical addresses in the N first access units can be combined into one second access unit. After the transfer is completed, the logical addresses in multiple first access units in the first storage pool become invalid and can be erased, thereby reducing the number of access units to one-N of the original first storage pool. In addition, the number of first mapping tables corresponding to the first access units is also reduced to one-N of the original first storage pool, which can significantly save storage space.

[0069] For example, assume that the size of the first storage pool is 512KB, which contains 16 first logical blocks, each of which contains 8 first access units, and each first access unit is 4KB. Assume that the size of the second storage pool is 2048KB, which contains 16 second logical blocks, each of which contains 8 second access units, and each second access unit is 16KB. It can be seen that the data capacity of the second access unit is 4 times that of the first access unit. In other words, the logical addresses stored in 4 first access units can be combined into one second access unit. If the first threshold is set to 256KB, then when 64 first access units in the first storage pool store logical addresses, the logical addresses in these 64 first access units will be combined and stored in the corresponding 16 second access units. Combined storage can also be performed without setting the first threshold. Specifically, when there are four first access units in the first storage pool that all store logical addresses, that is, the amount of data in the first access unit is 1 times the data capacity of the second access unit (16KB), the logical addresses in these four first access units can be combined into the second access unit.

[0070] Furthermore, whether transfer is required can be determined based on the relationship between the number of first logical blocks and the number of second logical blocks. Specifically, if the first threshold is set to 256 KB, then when 16 first logical blocks in the first storage pool store logical addresses, the logical addresses in these 16 first logical blocks are combined and stored in the second access units of the corresponding four second logical blocks.

[0071] like Figure 4 A flow chart of the combined storage method provided in the embodiment of the present application. Figure 4 As can be seen in FIG, combining multiple first access units into a logical address group, and storing the logical address group as a second access unit in the second storage pool includes:

[0072] 401: According to the sequential flow identifiers of the logical addresses in the first access unit, sequentially combine the logical addresses into a logical address group, and use the logical address group as a second access unit.

[0073] 402: Store the second logical block formed by the second access unit into the second storage pool.

[0074] When storing according to the sequential stream label, it can ensure that the physical addresses of the data are adjacent. When the storage device deletes data in blocks, the corresponding garbage logical blocks will also converge, thereby alleviating the write amplification problem. The sequential stream identifier mentioned here can be marked according to the order of the data to be processed provided by the host. Specifically, the garbage collector (GC) will select a sequential stream from the first storage pool, read multiple logical addresses in multiple first access units from the first storage pool and program them into the second storage pool (for example, QLC / PLC).

[0075] like Figure 5 A schematic diagram of the structure of the second storage pool provided in the embodiment of the present application. Figure 5As can be seen, it is assumed that the second storage pool has 1 logical block, QLC BLK0. And the second logical block contains multiple second access units AU. For example, AU0, AU1, AU2, AU3, AU4, AU5, AU6, AU7. As mentioned above, the data capacity of the second access unit is an integer multiple of the first access unit (in this embodiment, the second access unit is 4 times the first access unit). In the second access unit AU0, the logical addresses and their corresponding sequential stream identifiers are: stream1_0, stream1_1, stream1_2, stream1_3; in the second access unit AU1, the logical addresses and their corresponding sequential stream identifiers are: stream1_4, stream1_5, stream1_6, stream1_7. It can be seen that not only are the logical addresses in the second access unit AU0 continuous according to the sequential stream identifiers, but the sequential stream identifiers between the second access units AU0 and AU1 are also continuous. The same combination processing method as AU0 and AU1 is used, and the combination processing is continued according to the sequence stream identifier to obtain AU2, AU3, AU4, AU5, AU6, and AU7. The logical addresses and corresponding sequence stream identifiers in AU2, AU3, AU4, AU5, AU6, and AU7 are as follows: Figure 5 As shown, I will not go into details here.

[0076] In one or more embodiments of the present application, sequentially combining the logical addresses into a logical address group based on the sequential flow identifiers of the logical addresses in the first access unit, and using the logical address group as the second access unit, includes: if multiple first access units are respectively stored in multiple first logical blocks, respectively obtaining the sequential flow identifiers of the logical addresses in the multiple first logical units. Based on the order of the sequential flow identifiers, the logical addresses are grouped according to the data capacity of the second access unit to generate the second access unit.

[0077] from Figure 3 As can be seen from FIG, in the first logical block, each first access unit is grouped and stored according to the first unit sequence. Figure 5 As can be seen in the second logic block, the storage is based on the sequence flow identifier. It can be seen that the first unit sequence and the sequence flow identifier are different, which may cause the following Figure 3In that case, multiple sequential flow identifiers of continuous logical addresses are allocated to multiple first logical blocks. Therefore, when using garbage collection (GC) to combine the logical addresses in the first access unit into the second access unit, it is necessary to obtain the sequential flow identifiers corresponding to each logical address in the multiple first logical blocks. For example, it is necessary to combine the logical addresses in 4 first logical blocks into one second logical block, and use garbage collection to obtain the sequential flow identifiers of all logical addresses in these 4 first logical blocks. After sorting according to the sequential flow identifiers, the logical addresses are grouped and combined at a time according to the data capacity of each second access unit to obtain the following: Figure 5 The eight second access units are shown as follows. Furthermore, these eight second access units are stored in the second storage pool as a second logical block.

[0078] In one or more embodiments of the present application, after storing the second logical block composed of the second access unit into the second storage pool, the method further includes: determining multiple first mapping tables corresponding to multiple first access units used to combine the second access unit, deleting the multiple first mapping tables from the first storage unit pool, and generating a second mapping table corresponding to the second access unit based on the mapping relationships contained in the multiple first mapping tables.

[0079] It should be noted that when establishing a mapping table between physical addresses and logical addresses, it is established based on the access unit. This means that when establishing a mapping table for the amount of data to be processed, the larger the data capacity of the access unit, the fewer mapping tables are required; and the less flash memory storage space the mapping table needs to occupy. Therefore, when storing logical addresses based on the first storage pool, the amount of data in the mapping table generated based on the first access unit is also relatively large. It will occupy a large amount of storage space in the flash memory, which means that it will cost a lot to provide more storage space to store the mapping table. Therefore, in the technical solution of the present application, after completing the combination operation of the logical addresses in the first access unit, it is necessary to reclaim the storage space where the first access unit is located and the storage space of its corresponding mapping table. This is so that more storage space can be released to store the logical addresses and mapping tables of the newly provided data to be processed by the host.

[0080] In one or more embodiments of the present application, deleting the multiple first mapping tables from the first storage unit pool includes: determining a first logical block marked as deleted and the corresponding multiple first mapping tables; determining a recyclable first logical block in the first storage pool based on the first logical block marked as deleted and the corresponding multiple first mapping tables, and performing garbage collection on the recyclable first logical block so as to delete the multiple first mapping tables from the first storage unit pool.

[0081] It is easy to understand that while the first mapping table is being erased, a corresponding second mapping table is also created for the second access unit. As mentioned above, assuming the first access unit is 4KB and the second access unit is 16KB, after completing the combination operation on the 256KB logical address, the number of mapping tables is reduced from 64 to 16, and the storage space occupied by the mapping tables is reduced to one-quarter of the original. This allows the convergence of garbage logical blocks and corresponding mapping tables to be achieved during garbage collection.

[0082] In order to facilitate understanding of the embodiments of the present application, the garbage collection process is explained below. For solid-state drives, garbage collection (GC) refers to the process of transferring existing data to other flash memory locations and completely deleting some useless data. As described above, the way data is written is written in units of access units, but if you want to delete data, you need to do it in units of blocks. Therefore, to delete useless data, the solid-state drive first needs to copy and paste the useful data in a block into a page in a new block, so that the useless data contained in the original block can be deleted in units of blocks. Only after deletion can new data be written, and new data cannot be written before erasing.

[0083] When a file is deleted in the operating system, the operating system simply marks the file in its internal file table as deleted (indicating that the corresponding first logical block and multiple first mapping tables are also marked as deleted). Since NAND flash memory devices cannot overwrite existing data, currently invalid data on solid-state drives (SSDs) remains on the hard drive. This will generate a large amount of invalid data (including invalid logical addresses and corresponding mapping tables). In order to improve the utilization efficiency of solid-state drives, it is necessary to perform "garbage collection" in a timely manner to erase the logical addresses in the first access unit and their corresponding mapping tables.

[0084] In one or more embodiments of the present application, the method further includes: if the amount of data of the logical address in the first storage pool is smaller than the data capacity of the second access unit, waiting for the host to send a logical address corresponding to new data to be processed.

[0085] As previously mentioned, the second access unit is the smallest read / write unit in the second storage pool. If the amount of data in the first storage pool is less than the data capacity of the second access unit, writing the logical address to the second access unit will result in the second access unit not being fully written, resulting in idle and wasted storage space. Therefore, it is necessary to ensure that the amount of data at the logical address in the first storage pool meets the data capacity of at least one second access unit before performing the logical address combination operation. Otherwise, the host will continue to wait for the logical address corresponding to the new data to be processed.

[0086] For example, assume that the amount of logical address data in the current first storage pool is 516KB, and assume that these logical addresses are stored in 16 complete first logical blocks and one first logical block storing 4KB logical addresses, and each first logical block contains 8 first access units, and each first access unit stores a 4KB logical address. Therefore, when performing the first access unit combination processing, the logical addresses in the 16 complete first access units are combined to obtain 4 complete second access units. However, since there is still 1KB of logical addresses that neither meet the data capacity of the first access unit nor the data capacity of the second access unit, this last 1KB of logical addresses will temporarily be unable to be combined and can only continue to wait until the new logical addresses are stored in the first access unit before being combined into the second access unit.

[0087] It should be noted that the flash memory density of the second storage pool is greater than that of the first storage pool. The data capacity of the second logical block is greater than that of the first logical block, and the data capacity of the second access unit is greater than that of the first access unit. Furthermore, after the combination processing of the logical addresses is completed, the first access unit and the corresponding first mapping table are converged.

[0088] Based on the same idea, the embodiment of the present application also provides a data processing system. Figure 6 This is a structural diagram of a data processing system provided in an embodiment of the present application. The data processing system is a system for performing data reading and writing processing on a solid-state hard disk, such as Figure 6 As shown, the system includes:

[0089] The first storage pool 61 is used to receive the logical address corresponding to the data to be processed; store the logical address as a first access unit in the first storage pool; when the number of first access units stored in the first storage pool meets the requirement, combine multiple first access units into a logical address group, and store the logical address group as a second access unit in the second storage pool.

[0090] The second storage pool 62 is used to store second access units obtained by combining the logical addresses in the multiple first access units, and process data based on the second access units.

[0091] In order to understand the working principle of the above system, the following will give specific examples. Figure 6 As can be seen in the figure, the host provides multiple logical addresses. First, the logical addresses are stored in the first unit order in the four first logical blocks of the first storage pool. Since the number of logical addresses in the first storage pool can meet the data capacity of one second logical block, the logical addresses are combined into the second logical block according to the sequence flow identifiers corresponding to the logical addresses.

[0092] An embodiment of the present application also provides an electronic device. Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a memory 701, a processor 702, and a communication component 703. The memory 701 includes: a non-solid-state hard disk and a solid-state hard disk. The non-solid-state hard disk stores a data processing program, and the solid-state hard disk is used to store the logical address corresponding to the data to be processed;

[0093] The processor is coupled to the memory and configured to execute the output processing program for:

[0094] Receive the logical address corresponding to the data to be processed sent by the host;

[0095] storing the logical address as a first access unit in a first storage pool;

[0096] When the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process data based on the second access unit.

[0097] The memory 701 can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0098] Furthermore, the processor 702 in this embodiment may specifically be: a programmable switching processing chip, which is configured with a data replication engine and can replicate received data.

[0099] When the processor 702 executes the program in the memory, in addition to the above functions, it can also realize other functions, for details, please refer to the description of the above embodiments. Figure 7 As shown, the electronic device further includes: a display component 704 and other components.

[0100] Based on the same idea, the embodiment of the present application also provides a storage system based on a solid state drive. Figure 8 A schematic diagram of a storage system based on a solid-state drive provided in an embodiment of the present application. The data processing system includes: a host 81 and a solid-state drive 82 mounted on the host;

[0101] The solid state drive 82 is used to receive the logical address corresponding to the data to be processed sent by the host;

[0102] storing the logical address as a first access unit in a first storage pool;

[0103] When the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process data based on the second access unit.

[0104] Receive the logical address corresponding to the data to be processed;

[0105] storing the logical address as a first access unit in a first storage pool;

[0106] When the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process data based on the second access unit.

[0107] Optionally, the solid state drive 82 is further configured to store the logical address corresponding to the data to be processed in the first access unit;

[0108] determining a sequential flow identifier of the logical address and a first unit sequence of the first access unit;

[0109] Based on the first unit sequence, grouping a plurality of the first access units into a first logical block;

[0110] The plurality of first logical blocks are stored in the first storage pool.

[0111] Optionally, the solid state drive 82 is further configured to determine a mapping relationship between the physical address of the data to be processed and the logical address;

[0112] Based on the mapping relationship, a first mapping table corresponding to the first access unit is generated.

[0113] Optionally, the solid state drive 82 is further configured to: if the amount of data at the logical address in the first storage pool is greater than a first threshold; or if the amount of data at the first access unit is an integer multiple of the data capacity of one second access unit;

[0114] The plurality of first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit, so as to process the data based on the second access unit.

[0115] Optionally, the solid state drive 82 is further configured to sequentially combine the logical addresses into a second access unit according to the sequential flow identifiers of the logical addresses in the first access unit;

[0116] The second logical block formed by the second access unit is stored in the second storage pool.

[0117] Optionally, the solid state drive 82 is further configured to respectively obtain sequential flow identifiers of logical addresses in a plurality of first logical units if the plurality of first access units are respectively stored in a plurality of first logical blocks;

[0118] According to the sequence flow identification order, the logical addresses are grouped and combined according to the size of the data capacity of the second access unit;

[0119] A second access unit is generated.

[0120] Optionally, the solid state drive 82 is further configured to determine a plurality of first mapping tables corresponding to a plurality of the first access units used to combine the second access unit;

[0121] Deleting the plurality of first mapping tables from the first storage unit pool;

[0122] Based on the mapping relationships included in the multiple first mapping tables, a second mapping table corresponding to the second access unit is generated.

[0123] Optionally, the solid state drive 82 is further configured to determine a first logic block marked as being in a deleted state and the corresponding plurality of first mapping tables;

[0124] According to the first logical block marked as deleted and the corresponding multiple first mapping tables, a reclaimable first logical block in the first storage pool is determined, and garbage collection is performed on the reclaimable first logical block so as to delete the multiple first mapping tables from the first storage unit pool.

[0125] Optionally, the solid state drive 82 is further configured to wait for the host to send a logical address corresponding to new data to be processed if the amount of data at the logical address in the first storage pool is smaller than the data capacity of the second access unit.

[0126] Optionally, the flash memory particle density of the second storage pool in the solid state drive 82 is greater than the flash memory particle density of the first storage pool.

[0127] The embodiment of the present application also provides a computer-readable medium on which is stored at least one instruction, at least one program, code set or instruction set, which is loaded and executed by a processor to implement the following Figures 1 to 5 The method described.

[0128] Based on the above embodiment, when the host has data that it wants to store in the solid-state drive, it is necessary to establish a mapping table for managing the mapping relationship between the logical address and the physical address of the data to be processed. When storing, the logical address of the data to be processed can be first stored in the first access unit in the first storage pool; when the data volume of the logical address in the first storage pool meets the requirements (for example, when the data volume of the logical address in the first storage pool is an integer multiple of the data capacity of the second access unit in the second storage pool), the logical addresses in multiple first access units can be transferred and stored in at least one second access unit. Since the density of flash memory particles in the second storage pool is greater than that of flash memory particles in the first storage pool, in other words, the first access unit in the first storage pool has a smaller access unit and mapping table. When writing or reading data, it can be implemented based on the first storage pool, while the second storage block in the second storage pool supports a larger access unit and a larger mapping table, avoiding partial data writing or reading based on the second storage pool as much as possible, thereby effectively alleviating problems such as write amplification and poor service quality. Furthermore, while ensuring data storage needs, the hardware cost of the storage system can be reduced.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0130] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, comprising: Receive the logical address corresponding to the data to be processed; Storing the logical address as a first access unit in a first storage pool includes: storing the logical address in the first access unit in the first storage pool; After the logical address is stored in the first storage pool, determining a mapping relationship between the physical address of the data to be processed and the logical address; Based on the mapping relationship, generating a first mapping table corresponding to the first access unit; When the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process data based on the second access unit; and the first mapping table corresponding to the combined first access unit in the first storage pool and the logical address in the combined first access unit are deleted to generate a second mapping table corresponding to the second access unit.

2. The method according to claim 1, wherein storing the logical address as the first access unit in the first storage pool comprises: Storing the logical address corresponding to the data to be processed in the first access unit; determining a sequential flow identifier of the logical address and a first unit sequence of the first access unit; Based on the first unit sequence, grouping a plurality of the first access units into a first logical block; The plurality of first logical blocks are stored in the first storage pool.

3. The method according to claim 2, further comprising: after storing the logical address corresponding to the data to be processed in the first access unit; Determining a mapping relationship between the physical address of the data to be processed and the logical address; Based on the mapping relationship, a first mapping table corresponding to the first access unit is generated.

4. The method according to any one of claims 1 to 3, comprising: when the number of first access units stored in the first storage pool meets a requirement, combining multiple first access units into a logical address group, storing the logical address group as a second access unit in the second storage pool, and processing data based on the second access unit, comprising: If the amount of data at the logical address in the first storage pool is greater than a first threshold; Alternatively, the data volume of the first access unit is an integer multiple of the data capacity of the second access unit; The plurality of first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit, so as to process the data based on the second access unit.

5. The method according to claim 4, combining the plurality of first access units into a logical address group, and storing the logical address group as a second access unit in the second storage pool, comprising: Combining the logical addresses in sequence into a logical address group according to the sequential flow identifiers of the logical addresses in the first access unit, and using the logical address group as a second access unit; The second logical block formed by the second access unit is stored in the second storage pool.

6. The method according to claim 5, wherein the sequentially combining the logical addresses into a logical address group according to the sequential flow identifiers of the logical addresses in the first access unit, and using the logical address group as the second access unit, comprises: If the plurality of first access units are respectively stored in a plurality of first logical blocks, obtaining the sequential flow identifiers of the logical addresses in the plurality of first logical blocks respectively; According to the sequence flow identification order, the logical addresses are grouped and combined according to the size of the data capacity of the second access unit; A second access unit is generated.

7. The method according to claim 5, after storing the second logical block formed by the second access unit into the second storage pool, further comprising: Determine a plurality of first mapping tables corresponding to a plurality of the first access units used to combine the second access unit; Deleting the plurality of first mapping tables from the first storage pool; Based on the mapping relationships included in the multiple first mapping tables, a second mapping table corresponding to the second access unit is generated.

8. The method according to claim 7, wherein deleting the plurality of first mapping tables from the first storage pool comprises: Determining a first logic block marked as being in a deleted state and the corresponding plurality of first mapping tables; According to the first logical block marked as deleted and the corresponding multiple first mapping tables, a reclaimable first logical block in the first storage pool is determined, and garbage collection is performed on the reclaimable first logical block so as to delete the multiple first mapping tables from the first storage pool.

9. The method according to claim 1, further comprising: If the amount of data at the logical address in the first storage pool is smaller than the data capacity of the second access unit, the host is waited for to send a logical address corresponding to new data to be processed. 10 . The method according to claim 1 , wherein the flash memory particle density of the second storage pool is greater than the flash memory particle density of the first storage pool.

11. A data processing system comprising: The first storage pool is used to receive the logical address corresponding to the data to be processed; Storing the logical address as a first access unit in a first storage pool includes: storing the logical address in the first access unit in the first storage pool; after the logical address is stored in the first storage pool, determining a mapping relationship between the physical address of the to-be-processed data and the logical address; generating a first mapping table corresponding to the first access unit based on the mapping relationship; and when the number of first access units stored in the first storage pool meets a requirement, combining multiple first access units into a logical address group, and storing the logical address group as a second access unit in a second storage pool; The second storage pool is used to store the second access unit obtained by combining the logical addresses in the multiple first access units, and to process data based on the second access unit; and to delete the first mapping table corresponding to the combined first access unit in the first storage pool and the logical address in the combined first access unit, and generate a second mapping table corresponding to the second access unit.

12. An electronic device comprising: memory and processor; The memory includes: a non-solid-state hard disk and a solid-state hard disk, the non-solid-state hard disk stores a data processing program, and the solid-state hard disk is used to store data to be processed; The processor is coupled to the memory and configured to execute the processing program for: Receive the logical address corresponding to the data to be processed sent by the host; Storing the logical address as a first access unit in a first storage pool includes: storing the logical address in the first access unit in the first storage pool; After the logical address is stored in the first storage pool, determining a mapping relationship between the physical address of the data to be processed and the logical address; Based on the mapping relationship, generating a first mapping table corresponding to the first access unit; When the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process data based on the second access unit; and the first mapping table corresponding to the combined first access unit in the first storage pool and the logical address in the combined first access unit are deleted to generate a second mapping table corresponding to the second access unit.

13. A storage system based on a solid-state drive, comprising: A host and a solid-state drive mounted on the host; The solid-state hard disk is used to receive the logical address corresponding to the data to be processed sent by the host; Storing the logical address as a first access unit in a first storage pool includes: storing the logical address in the first access unit in the first storage pool; After the logical address is stored in the first storage pool, determining a mapping relationship between the physical address of the data to be processed and the logical address; Based on the mapping relationship, generating a first mapping table corresponding to the first access unit; When the number of first access units stored in the first storage pool meets the requirement, multiple first access units are combined into a logical address group, and the logical address group is stored in the second storage pool as a second access unit to process data based on the second access unit; and the first mapping table corresponding to the combined first access unit in the first storage pool and the logical address in the combined first access unit are deleted to generate a second mapping table corresponding to the second access unit.

14. A computer-readable medium having stored thereon at least one instruction, at least one program, code set or instruction set, wherein the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the method according to any one of claims 1 to 10.

Images