Partition allocation and full-parallelism writing method based on ZNS SSD file system

Abstract

The invention relates to the technical field of computer storage, in particular to a ZNS SSD file system-based partition allocation and full-parallelism writing method, which comprises the following steps of: dividing ZNS SSD partitions into a plurality of partition parallel groups according to a mapped chip group, and performing unified management through a linked list; partition parallel groups are distributed for write caches with different heat degrees of the file system, the partition parallel groups of the different write caches are mutually independent, and when the partition parallel groups are fully written, a new partition parallel group is redistributed; identifying the data popularity according to the source of the I / O request, and then putting the data into the write cache with the corresponding popularity according to the popularity; writing the data in the write cache into the partition parallel group in parallel in a polling manner; according to the method and the device, the parallelism resources in the device are fully utilized when the file system write cache data fall to the ZNS SSD device, so that the I / O (Input / Output) performance of the file system is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, and in particular to a partition allocation and full parallel writing method based on a ZNS SSD file system. Background Art

[0002] Zoned Namespace Space (ZNS) has emerged as the next generation interface for flash-based SSDs. ZNS SSD devices divide their storage space into multiple fixed-size partitions, which only allow sequential writes. Compared to traditional block-based SSDs, ZNS SSDs show lower DRAM cost and over-provisioning. In addition, a partition can be mapped to multiple chips to gain the parallelism of these chips. The write / read operations of a partition can be performed in parallel on multiple chips mapped to the partition.

[0003] In the latest research, ZNS SSDs are usually equipped with a series of small partitions, which can bring a series of advantages compared to large partitions, such as high space utilization, excellent isolation under multi-tenant conditions, and higher recycling efficiency during garbage collection. However, the number of chips mapped to a small partition is limited, thus limiting the parallelism within a single partition. In addition, most host-side applications ignore the use of parallelism between partitions. These host applications are unaware of the size of ZNS SSD partitions, and their write structure and partition relationship are usually 1 to 1, resulting in a large amount of idle parallelism resources in the ZNS device, which in turn leads to lower I / O performance. Summary of the invention

[0004] In order to make full use of the idle parallel resources in the device to improve the overall performance of the device, the present invention proposes a partition allocation and full parallel writing method based on the ZNS SSD file system, which specifically includes the following steps:

[0005] The partitions of the ZNS SSD are divided into multiple partition parallel groups according to the mapped chipset. All partitions in each partition parallel group just occupy the full parallelism of the ZNS SSD device and are uniformly managed through a linked list.

[0006] Assign partition parallel groups to write caches of different popularity in the file system. The partition parallel groups of different write caches are independent of each other. When a partition parallel group is full, a new partition parallel group needs to be reallocated.

[0007] When an I / O request reaches the file system, the data temperature is identified based on the source of the I / O request, and then the data is placed in the write cache of the corresponding temperature according to the temperature;

[0008] When the write cache needs to actually write data to the ZNS SSD device, the data in the write cache is written in parallel to the partition parallel group through polling.

[0009] Furthermore, the process of dividing the partitions of the ZNS SSD into a plurality of partition parallel groups according to the mapped chipsets includes:

[0010] On the device side, number each chipset according to the mapping table, then find all the groups mapped to each chipset, and set a flag bit for these partitions to indicate the chipset they belong to;

[0011] According to the chipset information to which the partitions belong, a group of parallel partitions with different chipset identifiers are selected in sequence, and these partitions are divided into a partition parallel group;

[0012] Repeat the above steps until all partitions are grouped; after the grouping is completed, insert the partition parallel groups into the free linked list in sequence.

[0013] Furthermore, the process of allocating partition parallel groups to write caches of different hotnesses of the file system includes the following steps:

[0014] If it is the file system initialization phase, empty partition parallel groups are taken out from the head of the free linked list in turn and assigned to write caches of different hotness. The write cache maintains the partition parallel groups assigned to it through the linked list, and the newly assigned partition parallel group will be inserted into the tail of the linked list of the write cache;

[0015] If a partition parallel group of a hot write cache is full and there are free partition parallel groups, a partition parallel group is taken from the head of the free list for allocation;

[0016] If the partition parallel group of a certain hot write cache is full and there is no idle partition parallel group, a partition parallel group that can be written is searched for from the partition parallel group linked list of other hot write caches for allocation, otherwise the device is informed that the space is full.

[0017] Furthermore, the process of identifying data heat according to the source of the I / O request includes:

[0018] If the I / O data includes a specific heat identifier, the data heat is classified according to the corresponding heat identifier;

[0019] If the data comes from data migration after garbage collection, or from resources that are not easily modified, the data is considered cold data;

[0020] If the data comes from a folder or a document that is easily edited, it is considered hot data;

[0021] Others are considered warm data by default.

[0022] Furthermore, when the write cache needs to actually write data to the ZNS SSD device, the data in the write cache is written in parallel to the partition parallel group in a polling manner, specifically including the following steps:

[0023] When a hot write cache needs to actually write data to the ZNS SSD device, if the current partition parallel group is full, an available partition parallel group is allocated to the current write cache;

[0024] Find the partition that was last written in the partition parallel group of the write cache, check the amount of data that has been written to the partition in the current write round, and if the amount of data that has been written does not reach the size of a stripe, continue to write the data in the write cache to the partition and continuously update the amount of data that has been written to the partition;

[0025] If the amount of data written has reached the size of a stripe, the next partition is found from the current partition parallel group through polling, and then the write cache data continues to be written to the partition;

[0026] Repeat the above steps until all the write cache data is written to the ZNS SSD device.

[0027] The present invention proposes a partition allocation and full parallel writing method based on the ZNS SSD file system. Aiming at the problem that most file systems ignore the use of parallelism between small partitions, the present invention adopts the strategy of partition parallel grouping and full parallel writing. When the file system writes the cache and the data is stored on the ZNS SSD device, the parallel resources inside the device are fully utilized, thereby improving the I / O performance of the file system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is the overall architecture diagram of the present invention;

[0029] Figure 2 It is a schematic diagram of the partitioned parallel grouping of the present invention;

[0030] Figure 3 It is an overall flow chart of the partition parallel group allocation method of the present invention;

[0031] Figure 4 The overall flow chart of the partitioned parallel writing of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The present invention proposes a partition allocation and full parallel writing method based on the ZNS SSD file system, which specifically includes the following steps:

[0034] The partitions of the ZNS SSD are divided into multiple partition parallel groups according to the mapped chipset. All partitions in each partition parallel group just occupy the full parallelism of the ZNS SSD device and are uniformly managed through a linked list.

[0035] Assign partition parallel groups to write caches of different popularity in the file system. The partition parallel groups of different write caches are independent of each other. When a partition parallel group is full, a new partition parallel group needs to be reallocated.

[0036] When an I / O request reaches the file system, the data temperature is identified based on the source of the I / O request, and then the data is placed in the write cache of the corresponding temperature according to the temperature;

[0037] When the write cache needs to actually write data to the ZNS SSD device, the data in the write cache is written in parallel to the partition parallel group through polling.

[0038] like Figure 1 During initialization, the device side actively performs partition identification, that is, divides each chip into unused areas; after the device side completes the partitioning, the host side sends a request to the device side to obtain the parallel partition grouping information of the device side. The host side groups the chips according to the parallel partition grouping information, that is, the host obtains the information of which groups are partition parallel groups; during the process of writing data, the host side assigns the specified partition parallel group to it according to the data type, and executes the data type for parallel writing in the partition parallel group assigned to it.

[0039] The strategy of the present invention for performing partition parallel group division is as follows Figure 2 ,First, the chip is divided into multiple chipsets, each chip can perform write operations, and chips in the same chipset can ,be written at the same time. Each partition is mapped to all chips in a chipset, and ,partitions of different chipsets further form a partition parallel group, ,e.g. Figure 2 Three chipsets are shown in the figure. Each chipset is divided into four partitions, among which partition Zone0, partition Zone4, and partition Zone8 can be written at the same time. These partitions are divided into a parallel partition group.

[0040] This embodiment proposes a partition allocation and full parallel writing method based on the ZNS SSD file system, which specifically includes the following steps:

[0041] S1: To facilitate the management and allocation of parallel partitions, a partition parallel grouping strategy is proposed. The partitions of the ZNS SSD are divided into multiple partition parallel groups according to the mapped chipset. All partitions in each partition parallel group just occupy the full parallelism of the ZNS SSD device and are uniformly managed through a linked list.

[0042] S2: To address the problem that the file system cannot fully utilize the parallelism of ZNS SSD partitions, a partition allocation strategy based on partition parallel groups is proposed. Partition parallel groups are allocated to write caches of different popularity of the file system in turn. Partition parallel groups of different write caches are independent of each other. When a partition parallel group is full, a new partition parallel group needs to be reallocated.

[0043] S3: A heat identification method is proposed for the I / O data to be written. When the I / O request reaches the file system, the data heat is identified according to the source of the I / O request, and then the data is placed in the write cache of the corresponding heat according to the heat;

[0044] S4: In order to utilize the full parallelism of the partition parallel group, a partition parallel write strategy is proposed. When the write cache needs to actually write data to the ZNS SSD device, the data in the write cache is written to the partition parallel group in parallel through polling, utilizing the full parallelism of the device.

[0045] To facilitate the management and allocation of parallel partitions, the present invention proposes a partition parallel grouping strategy which specifically includes the following steps:

[0046] S11: On the device side, number each chipset according to the mapping table, then find all the groups mapped to each chipset, and further set a flag bit for these partitions to indicate the chipset to which they belong;

[0047] S12: selecting a group of parallel partitions with different chipset identifiers in sequence according to the chipset information to which the partitions belong, and dividing the partitions into a partition parallel group;

[0048] S13: Repeat the step of S12 until all partitions are grouped. After the grouping is completed, the partition parallel groups are inserted into the free linked list in sequence.

[0049] After the partition parallel partitioning is completed, the partition parallel group is assigned to the write cache of the file system. In step S2, the partition allocation strategy based on the partition parallel group specifically includes the following steps:

[0050] S21: If it is the file system initialization phase, then the empty partition parallel groups are taken out from the head of the free linked list in turn and assigned to write caches of different hotness. The write cache maintains the partition parallel groups assigned to it through a linked list, and the newly assigned partition parallel group will be inserted into the tail of the linked list of the write cache;

[0051] S22: If a partition parallel group of a certain hot write cache is full and there are idle partition parallel groups, a partition parallel group is taken from the head of the idle linked list for allocation;

[0052] S23: If the partition parallel group of a write cache of a certain heat is full and there is no idle partition parallel group, try to find a partition parallel group that can still be written to from the partition parallel group linked list of the write cache of other heats for allocation, otherwise inform the device that the space is full.

[0053] In step S3, the heat identification method includes the following steps: when an I / O request reaches the file system, analyzing the data source of the I / O request, and dividing the data into different heats according to the data source.

[0054] Specifically, if the I / O data contains a specific heat identifier, the data heat is classified according to the corresponding heat identifier; if the data comes from data migration after garbage collection, or from resources that are not easily modified, such as pictures or audio and video, it is regarded as cold data; if the data comes from a folder or an easily edited document, it is regarded as hot data; otherwise, it is regarded as warm data by default; the classified data is added to the write cache of the corresponding heat.

[0055] When I / O data reaches the write cache and needs to be written to disk, the partition parallel write strategy includes the following steps:

[0056] S41: When a hot write cache needs to actually write data to the ZNS SSD device, if the current partition parallel group is full, an available partition parallel group is allocated to the current write cache through step S3;

[0057] S42: Find the partition that was written last in the partition parallel group of the write cache, check the amount of data that has been written to the partition in the current write round, and if the amount of data that has been written does not reach the size of a stripe, continue to write the data of the write cache to the partition, and continuously update the amount of data that has been written to the partition;

[0058] S43: If the amount of written data has reached the size of a stripe, find the next partition from the current partition parallel group by polling, and then continue to write the write cache data to the partition;

[0059] The formula for selecting the next partition is:

[0060]

[0061] Among them, OZ represents the partition number of the last written data; K represents the number of partitions in a partition parallel group; the formula indicates that if the partition number OZ written last time is not the number of the last partition in the current partition parallel, then the next write cache data is written to the next partition of the current partition, that is, the next partition number is the current partition number incremented; if the partition number OZ written last time is the number K of the last partition in the current group, it means that the current partition OZ is already the last partition of the current partition parallel group, and the next partition parallel group is selected according to polling. The number of the partition written next time is the partition numbered 0 in the newly selected partition parallel group, that is, the number of the available partition is updated to 0, and the writing is continued. The K partitions of a partition parallel group of the present invention are numbered 0 to K-1;

[0062] S44: Repeat S41 to S43 until all the write cache data is written to the ZNS SSD device.

[0063] In order to achieve the separation of hot and cold I / O data and enable the file system write cache to fully utilize the parallelism of ZNS SSD devices, such as Figure 3 As shown, the specific steps of I / O data heat identification and partition parallel group allocation are:

[0064] In step 101, the I / O request currently arriving at the file system is obtained;

[0065] In step 102, the I / O data is classified into corresponding heats according to the source of the I / O data and the heat identifier it carries;

[0066] In step 103, according to the divided heat levels, the I / O data is added to the write cache of the corresponding heat;

[0067] In step 104, it is determined whether the write cache has been assigned a partition parallel group. If so, step 105 is executed; otherwise, step 106 is executed.

[0068] In step 105, it is determined whether there is any remaining space in the partition parallel group of the write cache. If there is any remaining space, step 107 is executed, otherwise step 106 is executed;

[0069] In step 106, a new partition parallel group is taken from the head of the idle partition parallel group linked list and assigned to the current write cache;

[0070] At step 107, end.

[0071] In order to utilize the full parallelism of the partition parallel group, the present invention proposes a partition parallel writing strategy, such as Figure 4 As shown, the specific steps include:

[0072] In step 201, the size N of the stripe in the device is obtained. s , the partition parallel group PG of the write cache, the size of the data to be written to the write cache N, the number of partitions K in the partition parallel group PG, OZ represents the partition number of the last written data, and the value of the partition number represents the partition number, for example, OZ = 0, which means the first partition in the current partition parallel group;

[0073] In step 202, it is determined whether there is free space available for writing in the partition parallel group PG. If so, step 204 is executed, otherwise step 203 is executed;

[0074] In step 203, a new partition parallel group PG is allocated for the write cache, and the partition number OZ (initialized to OZ=0) of the last written data in the partition parallel group PG and the size N of the data written to the partition are initialized. oz ;

[0075] In step 204, the partition number OZ of the last written data in the partition parallel group PG and the size N of the data written to the partition are obtained. oz , the write pointer position P of the partition oz And the maximum capacity N of the partition z ;

[0076] In step 205, it is determined whether the current partition is not full and whether the amount of data written to the current partition in this round is less than the stripe size (i.e., it is determined whether N oz <N z And N oz %N s ≠0, % indicates a remainder operation), if the current partition is not full and the amount of data written to the current partition in this round is less than the stripe size limit, execute step 208, otherwise execute step 206;

[0077] In step 206, it is determined whether there is free space available for writing in the partition parallel group PG. If so, step 207 is executed, otherwise step 203 is executed;

[0078] In step 207, the partition number of the last written data is updated. The partition number is the number corresponding to the next available partition of the partition parallel group PG. The update formula is OZ = (OZ + 1)% K. After updating the partition number of the last written data, the corresponding write pointer position P is updated. oz And the size of the data written N oz ;

[0079] In step 208, the write pointer position P of the partition corresponding to the partition number OZ where data was written last time isoz Write data; update the partition number of the last written data after writing data, and its corresponding write pointer position P oz 、The size of the data written is N oz And the size N of the data to be written into the write cache;

[0080] In step 209, it is determined whether there is any data to be written (ie, whether N>0 is established). If so, step 205 is executed to determine whether the next data can be written in the current partition. Otherwise, step 210 is executed to end the writing.

[0081] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A partition allocation and full parallel writing method based on ZNS SSD file system, characterized in that: The specific steps include: The partitions of the ZNS SSD are divided into multiple partition parallel groups according to the mapped chipset. All partitions in each partition parallel group just occupy the full parallelism of the ZNS SSD device and are uniformly managed through a linked list. Assign partition parallel groups to write caches of different popularity in the file system. The partition parallel groups of different write caches are independent of each other. When a partition parallel group is full, a new partition parallel group needs to be reallocated. When an I / O request reaches the file system, the data temperature is identified based on the source of the I / O request, and then the data is placed in the write cache of the corresponding temperature according to the temperature; When the write cache needs to actually write data to the ZNS SSD device, the data in the write cache is written in parallel to the partition parallel group through polling.

2. According to claim 1, a partition allocation and full parallel writing method based on ZNS SSD file system is characterized in that: Each partition parallel group is independent of each other, and all partitions in each partition parallel group just occupy the entire parallelism of the ZNS SSD device. That is, during the parallel writing process of each partition parallel group, its parallelism is the parallelism of the ZNS SSD device.

3. According to claim 1, a partition allocation and full parallel writing method based on ZNS SSD file system is characterized in that: The process of dividing the ZNS SSD partitions into multiple partition parallel groups according to the mapped chipset includes: On the device side, number each chipset according to the mapping table, then find all the groups mapped to each chipset, and set a flag bit for these partitions to indicate the chipset they belong to; According to the chipset information to which the partitions belong, a group of parallel partitions with different chipset identifiers are selected in sequence, and these partitions are divided into a partition parallel group; Repeat the above steps until all partitions are grouped; after the grouping is completed, insert the partition parallel groups into the free linked list in sequence.

4. According to claim 1, a partition allocation and full parallel writing method based on ZNS SSD file system is characterized in that: The process of assigning partition parallel groups to the write caches of different file system hotnesses includes the following steps: If it is the file system initialization phase, empty partition parallel groups are taken out from the head of the free linked list in turn and assigned to write caches of different hotness. The write cache maintains the partition parallel groups assigned to it through the linked list, and the newly assigned partition parallel group will be inserted into the tail of the linked list of the write cache; If the partition parallel group of a hot write cache is full and there are free partition parallel groups, a partition parallel group is taken from the head of the free list for allocation; If the partition parallel group of a certain hot write cache is full and there is no idle partition parallel group, a partition parallel group that can be written is searched for from the partition parallel group linked list of other hot write caches for allocation, otherwise the device is informed that the space is full.

5. According to claim 1, a partition allocation and full parallel writing method based on ZNS SSD file system is characterized in that: The process of identifying data heat based on the source of I / O requests includes: If the I / O data includes a specific heat identifier, the data heat is classified according to the corresponding heat identifier; If the data comes from data migration after garbage collection, or from resources that are not easily modified, the data is considered cold data; If the data comes from a folder or a document that is easily edited, it is considered hot data; Others are considered warm data by default.

6. According to claim 1, a partition allocation and full parallel writing method based on ZNS SSD file system is characterized in that: When the write cache needs to actually write data to the ZNS SSD device, the data in the write cache is written in parallel to the partition parallel group in a polling manner, which specifically includes the following steps: When a hot write cache needs to actually write data to the ZNS SSD device, if the current partition parallel group is full, an available partition parallel group is allocated to the current write cache; Find the partition that was last written in the partition parallel group of the write cache, check the amount of data that has been written to the partition in the current write round, and if the amount of data that has been written does not reach the size of a stripe, continue to write the data in the write cache to the partition and continuously update the amount of data that has been written to the partition; If the amount of data written has reached the size of a stripe, the next partition is found from the current partition parallel group through polling, and then the write cache data continues to be written to the partition; Repeat the above steps until all the write cache data is written to the ZNS SSD device.

7. A partition allocation and full parallel writing method based on ZNS SSD file system according to claim 6, characterized in that: The process of finding the next partition from the current partition parallel group includes: Among them, OZ represents the partition number where data was last written; K represents the number of partitions in a partition parallel group.

8. According to claim 7, a partition allocation and full parallel writing method based on ZNS SSD file system is characterized in that: If the partition number written last time is not the last partition in the current group, then the incremental OZ number is the next partition; otherwise, it means that the current partition is already the last partition in the current partition parallel group. According to polling, the partition number written last time is updated to the partition numbered 0 in the next partition parallel group to continue writing.

Images