A resource allocation method based on ZNS SSD system

Abstract

The present invention relates to the field of computer storage technology, and in particular to a resource allocation method based on a ZNS SSD system. The method comprises: a host puts IO flow requests of different application programs into different SQs for sending; the IO flow requests in different SQs are stored in corresponding IO queues through a diversion module; a resource allocation module is used to periodically calculate the strength of each IO queue, and resources are allocated to each IO queue according to the ratio of the strengths of all IO queues; IO flow requests are taken out from all IO queues in a polling manner and responded to, and returned to CQ after successful response; the present invention quantifies the strength of each IO flow by periodically calculating the total number of bytes of each IO flow request, and reasonably allocates SSD internal resources according to the strength, thereby isolating the resources used by each IO flow to reduce interference between flows, and effectively improving the fairness within the ZNS SSD device.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, and in particular to a resource allocation method based on a ZNS SSD system. Background Art

[0002] NAND flash-based SSDs (Solid-State Drives) are widely used in various application scenarios due to their advantages such as high throughput, light weight, and low energy consumption. However, with the continuous upgrading of storage media, the operating costs of traditional block-based SSDs (such as over-capacity configuration, DRAM required for mapping tables, garbage collection overhead, etc.) have been increasing. These costs are caused by the mismatch between the block interface and the current storage medium characteristics. In order to avoid these problems, ZNS (Zoned NameSpace) has emerged as a new storage interface that divides the logical address space into fixed-size areas, each of which must be written sequentially. SSDs using the ZNS interface are called ZNS SSDs.

[0003] ZNS SSD still uses NVMe's multi-queue protocol, allowing multiple applications to send IO requests to access the device at the same time. However, when multiple applications access a single ZNS SSD at the same time, the IO (input and output) stream with higher intensity will cause serious interference to the IO stream with lower intensity, causing the response delay of the IO stream with lower intensity to increase sharply, which is the fairness problem in ZNS SSD. Due to the limitations of sequential writing and partitioned writing, previous solutions to the fairness problem cannot be effectively utilized. And in modern multi-threaded computers and multi-tenant cloud environments, fairness has become an important indicator. Therefore, the fairness problem in ZNS SSD needs to be effectively solved. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to propose a resource allocation method based on the ZNS SSD system based on the ZNS interface characteristics, so as to improve the fairness within the ZNS SSD device as much as possible without losing other performance as much as possible.

[0005] The present invention provides a resource allocation system based on ZNS SSD. First, a ZNS SSD system is constructed. The system includes: a host and a ZNS SSD device; the ZNS SSD device is provided with a flow distribution module, a resource allocation module, a dynamic mapping module and a response request module; the ZNS SSD device opens a corresponding IO queue for the IO flow of each application program;

[0006] The resource allocation method based on the ZNS SSD system includes the following steps:

[0007] The host puts the IO stream requests of different applications into different SQs (Submission Queues) for sending;

[0008] The IO flow requests in different SQs are stored in the corresponding IO queues through the diversion module; the strength of each IO queue is periodically calculated by the resource allocation module, and resources are allocated to each IO queue according to the ratio of the strengths of all IO queues;

[0009] Dynamically map specific physical blocks to each zone based on the block-level mapping mechanism and the allocation results of the resource allocation module;

[0010] The IO stream request is taken out from all IO queues through polling and responded. The response method adopted is different depending on the operation (read operation or write operation) required by the IO stream request. After a successful response, it is returned to CQ (CompletionQueue).

[0011] Furthermore, the specific process of resource allocation by the resource allocation module is as follows: the total number of bytes of all IO flow requests in each IO queue is calculated every 10ms, the strength of the IO queue is quantified by the total number of bytes, the resource allocation ratio is calculated according to the ratio between the strengths of all IO queues, and resources are allocated to each IO queue according to the resource allocation ratio.

[0012] Furthermore, the main resource allocated by the resource allocation module for the IO flow request is Chip / DIE, and each IO queue is allocated at least one Chip / DIE.

[0013] Furthermore, the idleness of resources allocated to each IO stream request is similar.

[0014] Furthermore, in the ZNS SSD system, a physical block-level address mapping table and an internal construction table are established for each zone of the ZNS SSD.

[0015] Furthermore, the specific mapping process of the block-level mapping mechanism is as follows: if the received IO stream request is a write request, when responding to the write request, first confirm whether there is enough physical space in the target zone required by the write request to write all the data in the write request; if yes, write directly; if not, confirm the IO queue to which the write request belongs, allocate a new physical block from the allocated resources of the IO queue, and fill the ID of the new physical block allocated this time into the address mapping table of the target zone.

[0016] Furthermore, the number of physical blocks allocated each time is determined according to the parallelism of the zone and the size of the request. One allocation of the zone is regarded as an interval, which is numbered in sequence. The number of physical blocks allocated to each interval and the starting logical block are recorded in the internal construction table.

[0017] Furthermore, if the IO stream request is a read request, the response process under the block-level mapping mechanism includes: S11. parsing the starting logical address and request size of the read request to obtain the partition sequence number and the target logical page;

[0018] S12. Determine the interval number of the interval corresponding to the target logical page through the internal construction table of the zone corresponding to the partition number;

[0019] S13. Obtain the logical block address and the offset within the block according to the interval sequence number and the starting logical block index of the interval;

[0020] S14. In the address mapping table of the zone, the logical block address is indexed to the physical block address, and then the offset within the block is indexed to the physical page;

[0021] S15. Return the data read from the physical page to the host. Beneficial effects of the present invention:

[0022] The present invention proposes a method for improving the fairness of the ZNS SSD device level, by periodically calculating the total number of bytes requested by each IO stream to quantify the strength of each IO stream, and reasonably allocating the internal resources of the SSD according to the strength, thereby isolating the resources used by each IO stream to reduce the interference between streams. At the same time, a new dynamic block-level mapping mechanism is proposed to reasonably utilize the allocated resources. The mechanism enables the internal resources of the SSD to be effectively utilized under the premise of ensuring the ZNS interface characteristics (such as sequential writing, block-level address mapping, etc.), thereby effectively improving the fairness within the ZNS SSD device without losing other performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a structural comparison diagram of an embodiment of the present invention;

[0024] Figure 2 An overview diagram of the ZNS SSD system according to an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of resource allocation according to an embodiment of the present invention;

[0026] Figure 4 The address mapping table and internal structure of the embodiment of the present invention are schematically shown;

[0027] Figure 5 is a read operation response flow chart of an embodiment of the present invention;

[0028] Figure 6 This is a write operation response flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] like Figure 1 As shown, Figure 1 (a) is the SSD structure of the traditional block interface. Figure 1 (b) is an SSD structure based on the ZNS interface; compared with the traditional block interface SSD, the ZNS SSD divides the internal space of the flash memory into multiple partitions of equal fixed size, and the partitions are called zones. Each zone can only be written and erased in a sequential manner, and each zone has its own state machine to allow the host to manage it. At the same time, the ZNS SSD transfers functions such as load balancing and garbage collection to the host side, which can transfer the responsibility of data management to the host, but hides the device-specific reliability characteristics and the complexity of device management from the host software. Since the host can more conveniently use data characteristics to make more reasonable data placement, the ZNS SSD can minimize garbage collection overhead and write amplification. The present invention is based on Figure 1 (b) A resource allocation method based on ZNS SSD system is proposed.

[0031] Example 1

[0032] like Figure 2 As shown, the ZNS SSD system includes: a host and a ZNS SSD device; the ZNS SSD device is provided with a diversion module, a resource allocation module, a dynamic mapping module and a response request module; the ZNS SSD device opens a corresponding IO queue for the IO flow of each application program, and the set of all IO queues is called APP Queue, and APPQueue = {APP0 Queue, APP1 Queue, ..., APPX Queue}, X+1 represents the total number of IO queues;

[0033] The host is used to put the IO stream requests of different applications into different SQs for sending; different applications will put the IO stream requests into different SQs, so this embodiment identifies different IO stream requests according to the SQ;

[0034] The traffic distribution module is used to identify the IO stream requests sent by different applications through different SQs in the SSD, and store the different IO stream requests in the corresponding IO queues based on the identification results;

[0035] The resource allocation module is used to periodically calculate the strength of each IO queue and allocate resources to each IO queue according to the ratio of the strengths of all IO queues;

[0036] Dynamic mapping module, used to dynamically map specific physical blocks to each zone based on the block-level mapping mechanism and the allocation results of the resource allocation module;

[0037] The response request module is used to poll and take out IO flow requests from all IO queues and respond to them, and return them to CQ after successful response.

[0038] Specifically, the resource allocation method based on the ZNS SSD system includes the following steps:

[0039] The host puts the IO stream requests of different applications into different SQs for sending;

[0040] The IO flow requests in different SQs are stored in the corresponding IO queues through the diversion module;

[0041] Dynamically map specific physical blocks to each zone based on the block-level mapping mechanism and the allocation results of the resource allocation module;

[0042] The IO stream request is taken out from all IO queues through polling and responded to, and returned to CQ after successful response.

[0043] Example 2

[0044] The speed at which the application sends IO stream requests and the size of the IO stream requests will eventually reflect the strength of the IO stream requests. The present invention uses the total number of bytes of IO stream requests within a period of time to represent the strength of the IO stream requests. The main resource for responding to requests within the SSD is Chip / DIE. When all resources are fully or mostly occupied by IO stream requests with higher intensity, requests from other IO streams can only be blocked and wait for the previous requests to be responded to before they can be responded to, which will cause fairness issues. When resources are reasonably allocated to each IO stream, resource isolation will prevent the above phenomenon from occurring, thereby achieving the purpose of improving fairness.

[0045] like Figure 3As shown in the figure, there are two IO queues, APP0 Queue and APP1 Queue, there are 4 IO stream requests in APP0 Queue, and 7 IO stream requests in APP1 Queue. The specific process of resource allocation by the resource allocation module is as follows: the total number of bytes of IO stream requests in the two IO queues, APP0 Queue and APP1 Queue, is calculated every 10ms, recorded as SMB1 and SMB2 respectively, the resource allocation ratio (X:Y) is calculated according to the ratio of SMB1 and SMB2, and resources are allocated to each IO stream request according to the resource allocation ratio.

[0046] Specifically, the main resource for responding to requests inside the SSD is Chip / DIE, so the ratio (X:Y) is used as the basis for allocating the number of Chips / DIEs to each IO queue. The greater the strength, the more Chips / DIEs are allocated, but it is also necessary to ensure that the IO queues with lower strength are allocated at least one Chip / DIE, and at the same time, it should be ensured that the idle level of the resources allocated to each IO queue is similar.

[0047] After determining the specific amount of resource allocation, this embodiment will sort all Chips / DIEs according to their next idle time, and then bind the allocated resources to the IO queue based on the next idle time of the Chip / DIE and the resource allocation ratio. After that, the IO stream requests of the IO queue can only be responded to using the allocated resources.

[0048] Example 3

[0049] All requests sent by the host are based on logical addresses. Address mapping is required inside the SSD to correspond the logical address to the physical address to ensure that the data can be read correctly. The mapping mechanism used in the ZNS SSD is based on the physical block as the basic unit. In view of this, the present invention designs a dynamic block-level mapping mechanism.

[0050] A block-level address mapping table and an internal construction table are established for each zone. The mapping table is empty at the beginning. If the received IO stream request is a write request, when responding to the write request, first confirm whether there is enough physical space in the target zone of the write request to write all the data in the write request; if so, write directly; if not, confirm the IO queue to which the write request belongs, allocate a new physical block from the Chip / DIE resources bound to the IO queue for mapping, and fill the ID of the new physical block into the address mapping table of the target zone.

[0051] Specifically, each time a new physical block is allocated, only one physical block is selected from a Chip / Die, and the number of physical blocks allocated this time is determined based on the parallelism of the zone and the size of the request. First of all, the parallelism of the zone refers to the number of physical units that can perform parallel operations contained in the zone, which refers to the number of Chips / Dies here. The parallelism of the zone is generally set in advance, and the parallelism of the zone should be maximized during subsequent mapping. If the parallelism is 2, then the current zone needs to allocate a physical block from each of the two different Chips / Dies, that is, two physical blocks will be allocated at a time; but if the IO queue to which the IO stream request belongs is only allocated to one Chip / Die at this time, then the current zone can only allocate one physical block at most, so the number of physical blocks allocated by the zone each time will be different. Secondly, if the write data size of the request is large, a physical block of appropriate size will be allocated to it. Therefore, the number of physical blocks allocated by the zone each time may be different. In order to ensure that the written data is correctly indexed, the zone allocation is regarded as an interval. Each time a physical block is allocated, the number of allocated physical blocks and the starting logical block are recorded in the internal construction table. The starting logical block is the ID of the first logical block in the allocation.

[0052] like Figure 4 As shown, LBA1, LBA2, ..., LBA10 are logical block numbers, PBA10, PBA1, PBA85, etc. are physical block numbers, N1 represents the first allocation, that is, the number of physical blocks in the first interval, and N2 and N3 are the same. S1 represents the number of the starting logical block at the first allocation, and S2 and S3 are the same. When the partition zone is reset to the initial state, the address mapping table and the internal construction table are empty, which means that the partition has not been mapped to a specific physical address. In order to reduce the overhead caused by garbage collection, the ZNS interface stipulates that the partition mapping needs to be aligned with the physical media erase granularity, that is, block-level mapping is used. As requests continue to arrive, the partition will allocate specific physical blocks in sequence from the Chip / DIE allocated by the IO queue to which the IO stream request belongs, and fill them into the block-level mapping table. The next allocation can only be made when the space of the physical block allocated this time has been used up.

[0053] Example 4

[0054] Based on Examples 1-3, Figure 5 As shown, the read operation flow after adopting the dynamic mapping mechanism includes:

[0055] S11. parse the starting logical address slba and the request size nlb of the read request to obtain the partition number zone_id and the target logical page lpn;

[0056] S12. Determine the interval number Nx of the interval corresponding to the target logical page lpn through the internal construction table of the zone corresponding to the partition number zone_id;

[0057] S13. Obtain the offset offset_section within the interval according to the interval sequence number Nx and the index of the starting logical block Sx of the interval, and obtain the logical block address lbn and the offset offset_block within the block according to the offset offset_section within the interval, where:

[0058] offset_section=lpn-(Sx×pages_per_block);

[0059] lbn = offset_section % Nx;

[0060] offset_block=offset_section%Nx;

[0061] pages_per_block is the number of physical pages contained in each physical block, and % indicates a remainder operation.

[0062] S14. In the address mapping table of the zone, the logical block address lbn is indexed to the physical block address, and then the offset offset_block in the block is indexed to the physical page ppn;

[0063] S15. Return the data read from the physical page ppn to the host.

[0064] Specifically, the process of determining the interval sequence number Nx of the interval corresponding to the target logical page lpn includes:

[0065] S100. Determine whether N1×pages_per_block is greater than or equal to lpn. If so, the interval number of the interval corresponding to lpn is N1. Otherwise, execute step S101.

[0066] S102. Determine whether (N1+N2)×pages_per_block is greater than or equal to lpn. If so, the interval number of the interval corresponding to lpn is N2. Otherwise, traverse in sequence until the corresponding interval is found.

[0067] Example 5

[0068] Based on Examples 1-3, Figure 6 As shown, the write operation process after adopting the dynamic mapping mechanism includes:

[0069] S21. parse the starting logical address and request size of the write request to obtain the partition number and the write pointer wp inside the zone;

[0070] S22. Check the remaining space of the physical block allocated to the zone corresponding to the partition number to determine whether the space required for writing the write request is met. If so, execute step S25, otherwise execute step S24;

[0071] S23. Allocate physical blocks to the zone according to the dynamic block-level mapping mechanism, and fill the address mapping table and internal construction table of the zone;

[0072] S24. Write the data of the write request sequentially into the corresponding physical page.

[0073] In the present invention, unless otherwise clearly stipulated and limited, the terms such as "installation", "setting", "connection", "fixation" and "rotation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0074] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A resource allocation method based on a ZNS SSD system, characterized in that: Constructing a ZNS SSD system, the system includes: a host and a ZNS SSD device; the ZNS SSD device is provided with a diversion module, a resource allocation module, a dynamic mapping module and a request response module; the ZNS SSD device opens a corresponding IO queue for each IO stream request of an application program; The resource allocation method based on the ZNS SSD system includes the following steps: The host puts the IO stream requests of different applications into different SQs for sending; The IO flow requests in different SQs are stored in the corresponding IO queues through the diversion module; The resource allocation module is used to periodically calculate the strength of each IO queue and allocate resources to each IO queue based on the ratio of the strengths of all IO queues. Dynamically map specific physical blocks to each zone based on the block-level mapping mechanism and the allocation results of the resource allocation module; Take out IO stream requests from all IO queues through polling and respond to them. If the response is successful, return it to CQ. The block-level mapping mechanism includes: establishing a physical block-level address mapping table and an internal construction table for each zone. The mapping table is empty at the beginning. If the received IO stream request is a write request, when responding to the write request, first confirm whether there is enough physical space in the target zone of the write request to write all the data in the write request; if so, write directly; if not, confirm the IO queue to which the write request belongs, allocate a new physical block from the Chip / DIE resources bound to the IO queue for mapping, and fill the ID of the new physical block into the address mapping table of the target zone.

2. According to a resource allocation method based on a ZNS SSD system according to claim 1, it is characterized in that: The specific process of resource allocation by the resource allocation module is as follows: the total number of bytes of IO flow requests in each IO queue is calculated every 10ms, the strength of the IO queue is quantified by the total number of bytes, the resource allocation ratio is calculated according to the ratio between the strengths of all IO queues, and resources are allocated to each IO queue according to the resource allocation ratio.

3. A resource allocation method based on a ZNS SSD system according to claim 2, characterized in that: The main resource allocated by the resource allocation module for IO flow requests is Chip / DIE, and each IO queue is allocated at least one Chip / DIE.

4. A resource allocation method based on a ZNS SSD system according to claim 2, characterized in that: The idleness of resources allocated to each IO stream request is similar.

5. A resource allocation method based on a ZNS SSD system according to claim 1, characterized in that: The number of physical blocks allocated each time is determined according to the parallelism of the zone. One allocation of the zone is regarded as an interval, which is numbered in sequence. The number of physical blocks allocated to each interval and the starting logical block are recorded in the internal construction table.

6. A resource allocation method based on a ZNS SSD system according to claim 1, characterized in that: If the IO stream request is a read request, the response process under the block-level mapping mechanism includes: S11. parse the starting logical address and request size of the read request to obtain the partition number and target logical page; S12. Determine the interval number of the interval corresponding to the target logical page through the internal construction table of the zone corresponding to the partition number; S13. Obtain the logical block address and the offset within the block according to the interval sequence number and the starting logical block index of the interval; S14. In the address mapping table of the zone, the logical block address is indexed to the physical block address, and then the offset within the block is indexed to the physical page; S15. Return the data read from the physical page to the host.

Images