Method for managing a storage system, electronic device, and computer program product

Abstract

Embodiments of the present disclosure provide a method, an electronic device, and a computer program product for managing a storage system. The method includes flushing a target page in a cache device of the storage system to a persistent storage device of the storage system. The method further includes releasing resources of a page descriptor storing the target page to a resource pool. The resource pool is configured to provide resources to store page descriptors of pages to be flushed in the cache device. The method further includes removing the secondary descriptor of the target page from the queue if it is determined that the secondary descriptor of the target page is at the tail of a queue of secondary descriptors of the pages to be flushed. The secondary descriptor of the page to be flushed is configured to describe the page descriptor of the page to be flushed. In this way, the page flushing performance of the storage system can be improved, and thus the input / output performance can be improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of data processing, and more particularly, to methods, electronic devices, and computer program products for managing a storage system. Background Art

[0002] With the development of data storage technology, various data storage devices have been able to provide users with increasingly high data storage capabilities, and the data access speed has also been greatly improved. While improving the data storage capacity, users have also put forward higher and higher requirements for the performance of the storage system. Therefore, it is desirable to effectively manage the pages of user data in the storage system to better serve access requests from users. Summary of the Invention

[0003] Embodiments of the present disclosure provide a solution for managing a storage system.

[0004] In a first aspect of the present disclosure, a method for managing a storage system is provided. The method includes flushing a target page in a cache device of the storage system to a persistent storage device of the storage system. The method further includes releasing the resources of the page descriptor storing the target page to a resource pool. The resource pool is configured to provide resources to store the page descriptors of the pages to be flushed in the cache device. The method further includes removing the auxiliary descriptor of the target page from the queue if it is determined that the auxiliary descriptor of the target page is at the tail of the queue of the auxiliary descriptors of the pages to be flushed. The auxiliary descriptor of the page to be flushed is configured to describe the page descriptor of the page to be flushed.

[0005] In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes a processor and a memory coupled to the processor, the memory having instructions stored therein that, when executed by the processor, cause the device to perform actions. The actions include flushing a target page in a cache device of the storage system to a persistent storage device of the storage system. The actions further include releasing the resources of the page descriptor storing the target page to a resource pool. The resource pool is configured to provide resources to store the page descriptors of the pages to be flushed in the cache device. The actions further include removing the auxiliary descriptor of the target page from the queue if it is determined that the auxiliary descriptor of the target page is at the tail of the queue of the auxiliary descriptors of the pages to be flushed. The auxiliary descriptor of the page to be flushed is configured to describe the page descriptor of the page to be flushed.

[0006] In a third aspect of the present disclosure, a computer program product is provided, the computer program product being tangibly stored on a computer-readable medium and including machine-executable instructions that, when executed, cause the machine to perform the method according to the first aspect.

[0007] The Summary of the Invention section is provided to introduce, in simplified form, a selection of concepts that are further described below in the Detailed Description. The Summary of the Invention section is not intended to identify key or essential features of the present disclosure, nor is it intended to limit the scope of the present disclosure. Brief Description of the Drawings

[0008] The above and other objects, features, and advantages of the present disclosure will become more apparent from the following more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which like reference numerals generally represent like components in the exemplary embodiments of the present disclosure. In the drawings:

[0009] Figure 1 A schematic diagram of a storage system in which embodiments of the present disclosure may be implemented is shown;

[0010] Figure 2 A schematic diagram of a data structure of a storage system according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram of an auxiliary descriptor according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A schematic diagram of a page descriptor according to some embodiments of the present disclosure is shown;

[0013] Figure 5 A flowchart of an example method for managing a storage system according to an embodiment of the present disclosure is shown;

[0014] Figure 6 A schematic diagram of a data structure of a storage system after flushing a page according to some embodiments of the present disclosure is shown;

[0015] Figure 7 A flowchart of an example method for selecting a target page according to some embodiments of the present disclosure is shown;

[0016] Figure 8 A schematic diagram of a mapping tree according to some embodiments of the present disclosure is shown;

[0017] Figure 9 A schematic diagram of a grouping result according to some embodiments of the present disclosure is shown;

[0018] Figure 10 A schematic diagram of a sorting result according to some embodiments of the present disclosure is shown; and

[0019] Figure 11 A block diagram of an example device that may be used to implement embodiments of the present disclosure is shown. Detailed Description

[0020] The principles of the present disclosure will now be described with reference to several example embodiments shown in the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the description of these embodiments is only for enabling those skilled in the art to better understand and then implement the present disclosure, rather than limiting the scope of the present disclosure in any way.

[0021] As used herein, the term "comprising" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. There may be other explicit and implicit definitions hereinafter.

[0022] Figure 1 A schematic diagram of a storage system 100 in which embodiments of the present disclosure can be implemented is shown. As Figure 1 shown, the storage system 100 generally includes multi-level storage devices, such as a volatile storage device 110, a cache device 130, and a persistent storage device 140. It should be understood that the architecture and functions in the environment 100 are described only for exemplary purposes, without implying any limitation on the scope of the present disclosure. Embodiments of the present disclosure can also be applied to environments involving data protection systems with different structures and / or functions.

[0023] The volatile storage device 110 may include any suitable volatile memory, such as dynamic random access memory (DRAM). The cache device 130 may include any suitable non-volatile cache, such as non-volatile memory express (NVMe) cache and non-volatile dual in-line memory module (NVDIMM) cache. The persistent storage device 140 may include one or more storage disks, such as solid state disks (SSDs), magnetic disks, or optical disks, etc. It should be understood that the implementations of the volatile storage device 110, the cache device 130, and the persistent storage device 140 described herein are only exemplary and are not intended to limit the scope of the present disclosure. Any type of memory can be selected according to the specific application scenario to implement the volatile storage device 110, the cache device 130, and the persistent storage device 140.

[0024] The storage system 100 serves access requests 115 from user devices to provide data to users or store data for users. When a read request is received from a user device, the data to be read can be retrieved from the persistent storage device 140 (or other cache devices), and the retrieved data is returned to the user device. When a write request is received, to improve the response speed of the storage system 100, the data to be written can be first written to the cache device 130, and a "successful write" response is returned to the user device.

[0025] At this time, the data in the cache device 130 is "dirty data", which means that the data has been modified but not yet flushed to the persistent storage device 140. Inside the storage system 100, the dirty data in the cache device 130 can be managed in the background. As the storage system 100 runs, the dirty data in the cache device 130 may continue to increase. The dirty data can be flushed to the persistent storage device 140 at a predetermined time interval (or according to other triggering conditions).

[0026] For this reason, the cache device 130 usually processes the data to be flushed to the persistent storage device 140 in units of pages. Figure 1 Pages 120-1 to 120-8 are shown, which can also be collectively or individually referred to as page 120 or the page to be flushed 120. To manage such pages to be flushed 120, a page descriptor (PD) for each page is maintained inside the storage system 100. The page descriptor can describe the attributes of the page, which storage volume the page belongs to, which node the page comes from, etc.

[0027] Conventionally, the page descriptors of different pages are stored in a circular queue in the storage system according to the time when the corresponding pages are written. The page descriptor of the latest written page is added to the head of the queue, while the page descriptor of the oldest page is at the tail of the queue. When the oldest page is flushed to the backend persistent storage device, its page descriptor is removed from the queue. The tail of the queue can move forward to release the storage resources used to store the page descriptor of the oldest page to the storage system.

[0028] However, the order in which the storage system flushes the pages in the cache device does not follow the time order in which the pages are written to the storage system. This makes the circular queue of page descriptors move slowly, and in some cases, the circular queue becomes saturated. This will lead to delays in data input / output (I / O) and severe I / O throttling. Therefore, it is desirable to be able to manage the page descriptors of pages more effectively to improve the performance of the storage system.

[0029] Embodiments of the present disclosure propose a solution for managing a storage system to address one or more of the above problems and other potential problems. In this solution, instead of using a queue, a resource pool is used to manage the resources for storing page descriptors. In this way, after a page is flushed to a persistent storage device, the resources for storing the descriptor of this page can be released to the resource pool in a timely manner for use by subsequent written pages. Additionally, in the storage system, a queue is used to manage the resources for storing auxiliary descriptors of pages, where the auxiliary descriptors are configured to describe the page descriptors of the pages. In this way, the arrangement of the auxiliary descriptors in the queue can reflect the time when the pages are written or modified, thus preventing the loss of time information. The storage space of the queue assigned to the auxiliary descriptors can be redundant. In this way, the moving speed of the queue of the auxiliary descriptors will no longer be the primary influencing factor. In this manner, the page flushing performance of the storage system can be improved, thereby enhancing the I / O performance.

[0030] The following will be combined with Figures 2 to 11 to elaborate on the exemplary embodiments of the present disclosure in detail. Figure 2 FIG. shows a schematic diagram of the data structure of a storage system 100 according to some embodiments of the present disclosure. The resource pool 215 in the cache device 130 is configured to provide resources in the cache device 130 for storing the page descriptors of the pages 120. Figure 2 Page descriptors 210-1 to 210-8 of the page 120 are shown therein, which may also be collectively or individually referred to as "page descriptor 210". The page descriptor 210 may have a size of 512 bytes. For illustrative purposes only, the page descriptor 210 and the page 120 it describes are identified with the same letter. For example, page descriptors 210-1, 210-2, 210-3, 210-4, 210-5, 210-6, 210-7, 210-8 are respectively used to describe pages 120-1, 120-2, 120-3, 120-4, 120-5, 120-6, 120-7, 120-8.

[0031] The volatile storage device 110 stores page descriptors 220-1 to 220-8 of page 120, which may also be collectively or individually referred to as "page descriptor 220". The page descriptor 220 can be regarded as a copy of the page descriptor 210. For this purpose, the resource pool 225 in the volatile storage device 110 is configured to provide resources in the volatile storage device 110 for storing the page descriptor 220 of page 120. The page descriptor 220 can have a size of 512 bytes. For illustrative purposes only, the page descriptor 220 and the page 120 it describes are identified with the same letter. For example, page descriptors 220-1, 220-2, 220-3, 220-4, 220-5, 220-6, 220-7, 220-8 are respectively used to describe pages 120-1, 120-2, 120-3, 120-4, 120-5, 120-6, 120-7, 120-8.

[0032] Compared with a queue, by managing page descriptors 210 and 220 with a resource pool, more flexible and efficient resource allocation and release can be achieved. However, the resource pool cannot maintain the chronological order between the pages described by the page descriptors. The chronological order is crucial when dealing with dependencies between pages (such as those generated by snapshots). For this purpose, secondary descriptors 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8 of page 120 are stored in the volatile storage device 110, which may also be collectively or individually referred to as "secondary descriptor 230". The secondary descriptor 230 of page 120 is configured to describe the page descriptor 210 and the page descriptor 220 of page 120. In other words, the secondary descriptor described herein is a descriptor of the page descriptor. For illustrative purposes only, the secondary descriptor 230 and the page descriptors 220, 210, and page 120 it describes are identified with the same letter. For example, secondary descriptors 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8 respectively correspond to pages 120-1, 120-XX, 120-3, 120-4, 120-5, 120-6, 120-7, 120-8.

[0033] In the queue 235 (e.g., a circular queue) of the auxiliary descriptor 230, the respective auxiliary descriptors 230 are arranged in the chronological order of writing to or modifying the storage system 100 by the corresponding pages 120. For example, the auxiliary descriptor 230-8 is at the head of the queue, which means that the corresponding page 120-8 is the latest page. The auxiliary descriptor 230-1 is at the tail of the queue, which means that the corresponding page 120-1 is the oldest page. With the auxiliary descriptor, the storage system 110 can still maintain the chronological order between pages, thus not affecting operations related to dependencies between pages.

[0034] In some embodiments, the size of the auxiliary descriptor 230 can be smaller than the sizes of the page descriptors 220 and 230. For example, when the page descriptors 220 and 230 have a size of 512 bytes, the auxiliary descriptor 230 can have a size of 16 bytes. It should be understood that the numerical values described herein are merely exemplary and are not intended to limit the scope of the present disclosure. The sizes of different page descriptors and auxiliary descriptors can be set according to specific application scenarios.

[0035] To address the problem of slow queue movement, the queue 235 can be overprovisioned. That is, the storage space of the queue 235 allocated to the auxiliary descriptor 230 can be redundant. It can be understood that the greater the redundancy of the queue 235, the smaller the possibility of the queue 235 becoming saturated.

[0036] In some embodiments, the number of auxiliary descriptors that the queue 235 can accommodate can be a multiple N of the number of page descriptors that the resource pool 225 and the resource pool 215 can store. This multiple N is greater than 1 and can be, for example, 2 times, 4 times, 8 times, 10 times, etc. Again, these numerical values are merely exemplary.

[0037] Since the size of the auxiliary descriptor 230 can be much smaller than the page descriptors 210 and 220, the additional storage overhead for overprovisioning the queue 235 is limited. Table 1 shows the additional storage overhead for overprovisioning the queue 235 with different multiples N when the auxiliary descriptor 230 has a size of 16 bytes.

[0038] Table 1 Additional storage overhead for overprovisioning

[0039] Multiplier N of over-configuration Extra storage overhead 2 6.25％ 4 12.5％ 8 25％

[0040] Although Figure 2Shown are the page descriptor 220 in the volatile storage device 110 and the page descriptor 210 in the cache device 130, but it should be understood that this is merely exemplary and not intended to limit the scope of the present disclosure. The page descriptor may be stored in any suitable storage device of the storage system 100. The page descriptor may be maintained in only one type of storage device. For example, in some embodiments, the page descriptor 210 may be maintained only in the cache device 130 without the page descriptor 220 in the volatile storage device 110.

[0041] The auxiliary descriptor 230 may have any suitable elements to describe the page descriptor. In some embodiments, the auxiliary descriptor 230 may indicate the position of the auxiliary descriptor 230 itself in the queue 235 and the position of the described page descriptor 220 in the resource pool 225. For example, the auxiliary descriptor 230-1 may indicate the position of the auxiliary descriptor 230-1 itself in the queue 235 and the position of the described page descriptor 220-1 in the resource pool 225. Figure 3 Shown is a schematic diagram of the auxiliary descriptor 230 according to some embodiments of the present disclosure. The "ring_offset" ("ring offset") element 310 indicates the offset of the auxiliary descriptor 230 itself in the queue 235, e.g., the offset relative to the tail. The "pointer_to_the_PD_buffer" ("pointer to the PD buffer") element 320 indicates the position of the described page descriptor 220 in the resource pool 225, e.g., a pointer pointing to the page descriptor 220.

[0042] Additionally, in some embodiments, the auxiliary descriptor 230 may also indicate the position of the described page descriptor 210 in the resource pool 215. For example, the auxiliary descriptor 230-1 may also indicate the position of the described page descriptor 210-1 in the resource pool 215. Further, in embodiments without the page descriptor 220 in the volatile storage device 110, the auxiliary descriptor 230 may indicate the position of the auxiliary descriptor 230 itself in the queue 235 and the position of the described page descriptor 210 in the resource pool 215. For example, the auxiliary descriptor 230-1 may indicate the position of the auxiliary descriptor 230-1 itself in the queue 235 and the position of the described page descriptor 210-1 in the resource pool 215.

[0043] Accordingly, the page descriptor 220 can have any suitable elements to indicate the auxiliary descriptor that describes it, another page descriptor that describes the same page as it. In some embodiments, the page descriptor 220 can indicate the position (e.g., logical position) of the auxiliary descriptor 230 that describes it in the queue 235 and the position of the page descriptor 210 that describes the same page as the page descriptor 220 in the resource pool 215. For example, the page descriptor 220-1 can indicate the position of the auxiliary descriptor 230-1 in the queue 235 and the position of the page descriptor 210-1 in the resource pool 215. Figure 4 FIG. shows a schematic diagram of a page descriptor 220 according to some embodiments of the present disclosure. The "logical_offset_in_the_memory_ring" ("logical offset in the memory ring") element 410 indicates the position of the auxiliary descriptor 230 that describes it in the queue 235, e.g., a logical offset. The "physical_offset_in_the_on_disk_pool" ("physical offset in the disk-based pool") element 420 indicates the position of the page descriptor 210 that describes the same page as the page descriptor 220 in the resource pool 215, e.g., a physical offset in the resource pool 215.

[0044] In embodiments where both the page descriptor 220 and the page descriptor 210 are maintained in the storage system 110, the page descriptor 210 can have the same elements as the page descriptor 220. In embodiments without the page descriptor 220, the page descriptor 210 can indicate the position (e.g., logical position) of the auxiliary descriptor 230 that describes it in the queue 235 and the position of the page descriptor 210 itself in the resource pool 215.

[0045] It should be understood that Figure 3 and Figure 4 the elements of the auxiliary descriptor and the page descriptor shown are merely exemplary. In embodiments of the present disclosure, the auxiliary descriptor and the page descriptor can include any suitable elements to indicate associated descriptors, pages, etc.

[0046] Figure 5 FIG. shows a flowchart of an example method 500 for managing a storage system according to an embodiment of the present disclosure. The method 500 can be implemented, for example, at a storage system 100 as shown in Figure 1 using a data structure as shown in Figure 2 It should be understood that the method 500 can also include additional actions not shown and / or can omit the actions shown, and the scope of the present disclosure is not limited in this regard. The following will describe the method 500 in detail in conjunction with Figure 1 and Figure 2 .

[0047] At block 510, the target page in the cache device 130 of the storage system 110 is flushed to the persistent storage device 140 of the storage system 110. The target page described herein can refer to any one of the pages flushed in a single flush cycle. The target page can be selected from Figure 2 the to-be-flushed pages 120 shown. How to select the target page from the pages 120 will be described below with reference to Figures 7 to 10 . Now refer to Figure 6 . As Figure 6 shown, pages 120-1, 120-2, 120-6, and 120-8 are selected as target pages and flushed to the persistent storage device 140.

[0048] In response to the target page being flushed to the persistent storage device 140, at block 520, the resources storing the page descriptor of the target page are released to the resource pool. For example, as Figure 6 shown, the resources for storing page descriptors 210-1, 210-2, 210-6, and 210-8 are released to the resource pool 215. Similarly, the resources for storing page descriptors 220-1, 220-2, 220-6, and 220-8 are released to the resource pool 225.

[0049] By managing the resources for storing page descriptors 210 and 220 in a resource pool manner, once a page is flushed to the backend, the resources for storing the page descriptor of that page can be directly released without considering the chronological order of the flushed pages and the unflushed pages. The resources released to the resource pool can be immediately used for other transactions, such as storing the page descriptors of newly written pages.

[0050] Continue to refer to Figure 5 . In response to the target page being flushed to the persistent storage device 140, at block 530, it is determined whether the secondary descriptor of the target page is at the tail of the queue 235 of secondary descriptors. If it is determined at block 530 that the secondary descriptor of the target page is at the tail of the queue 235 of secondary descriptors, the method 500 proceeds to block 540. At block 540, the secondary descriptor of the target page is removed from the queue 235. That is, in this case, the tail of the queue 235 can move forward. If it is determined at block 530 that the secondary descriptor of the target page is at the tail of the queue 235 of secondary descriptors, the tail of the queue 235 does not move forward.

[0051] Continue to refer to Figure 6Example. After page 120-1 is flushed, the auxiliary descriptor 230-1 of page 120-1 is removed from the queue 235. Thus, the tail of the queue 235 moves forward. The storage space originally used to store the auxiliary descriptor 230-1 becomes available. Subsequently, after page 120-2 is flushed, the auxiliary descriptor 230-2 of page 120-2 is removed from the queue 235. Thus, the tail of the queue 235 moves forward again. The storage space originally used to store the auxiliary descriptor 230-2 becomes available.

[0052] Since pages 120-3, 120-4, and 120-5 are not flushed, the corresponding auxiliary descriptors 230-3, 230-4, and 230-5 still remain in the queue 235. Therefore, after page 120-6 is flushed, the tail of the queue 235 cannot continue to move. In this case, the storage space used to store the auxiliary descriptor 230-6 is still unavailable. Similarly, after page 120-8 is flushed, the storage space used to store the auxiliary descriptor 230-8 is still unavailable. Nevertheless, in the case where the queue 235 is over-provisioned, there may be sufficient storage space for newly arriving data.

[0053] It should be understood that Figure 5 the order of the respective blocks shown in Figure 5 is merely exemplary and is not intended to limit the scope of the present disclosure. In some embodiments, the respective blocks may not be performed in the

[0054] order shown. For example, block 520 may be performed in parallel with the combination of blocks 530 and 540. As another example, blocks 530 and 540 may be performed before block 520.

[0055] As mentioned above with reference to block 510, the target page to be flushed is selected from the pages to be flushed 120. Some embodiments of how to select the target page to be flushed to the persistent storage device 140 will be described below. Figure 7FIG. 700 is a flowchart showing an example method for selecting a target page according to some embodiments of the present disclosure. Method 700 can be regarded as an additional step of Method 500. It should be understood that Method 700 may further include additional actions not shown and / or actions shown may be omitted, and the scope of the present disclosure is not limited in this regard. The following will be described in detail in conjunction with Figure 1 and Figure 2 to describe Method 700 in detail.

[0056] In block 710, based on the respective logical addresses of a plurality of pages to be flushed 120 in the cache device 130, the plurality of pages to be flushed 120 are divided into a plurality of groups. The logical addresses of the pages to be flushed in the same group are within the same address range, and the plurality of groups correspond to different address ranges.

[0057] The logical address of page 120 can be, for example, a logical block address (LBA). Now refer to Figure 8 . Figure 8 FIG. 800 is a schematic diagram showing a mapping tree according to some embodiments of the present disclosure. As Figure 8 shown, the mapping tree 800 includes a root level 810, a top level 820, an intermediate level 830, and a leaf level 840, which map consecutive LBAs to a virtual large block (VLB) 850 and a physical large block (PLB) 860.

[0058] As Figure 8 shown, the address range 811 of the root level can be divided into a plurality of address ranges of the top level, for example, including the address ranges 821 and 822 of the top level. Each address range of the top level can be further divided into a plurality of address ranges of the intermediate level. For example, the address range 821 of the top level can include the address ranges 831, 832, etc. of the intermediate level. Another example is that the address range 822 of the top level can include the address ranges 833, 834, etc. of the intermediate level. The address ranges of the intermediate level can be further divided into a plurality of address ranges of the leaf level. For example, the address range 831 of the intermediate level includes the address ranges 841, 842, 843, etc. of the leaf level. By way of example only and not intended to be limiting, the address range of the top level can be a continuous 512 GB logical space, the address range of the intermediate level can be a continuous 1 GB logical space, and the address range of the leaf level can be a continuous ........

[0059] It should be noted that there seems to be an incomplete sentence in the original text at the end of the translation of ID=20. Please check and supplement it if necessary.Based on the LBA of a page, it can be determined which top-level address range, which middle-level address range, and which leaf-level address range the page belongs to. When a page is flushed to the persistent storage device 130, the mapping of LBA to VLB and PLB needs to be updated. Therefore, it is necessary to traverse from the root level 810 of the mapping tree 800 down through the top level 820, the middle level 830, and the leaf level 840 until the VLB 850 and the PLB 860.

[0060] In view of this, if the target pages to be flushed are as much as possible within the same address range, the overhead of searching the mapping tree 800 can be reduced. To this end, based on the LBAs of the multiple pages 120 to be flushed, the multiple pages 120 to be flushed can be divided into multiple groups such that the LBAs of the pages to be flushed in the same group are within the same address range, and the multiple groups correspond to different address ranges.

[0061] In some embodiments, the LBAs of the pages to be flushed in the same group can be within the same middle-level address range. In other words, the pages to be flushed in the same group belong to the same middle-level subtree.

[0062] In some embodiments, the LBAs of the pages to be flushed in the same group can be within the same top-level address range. In other words, the pages to be flushed in the same group belong to the same top-level subtree. The advantages of such an embodiment will be described in detail below.

[0063] Reference Figure 9 . Figure 9 FIG. shows a schematic diagram of the grouping result 900 according to some embodiments of the present disclosure. In Figure 9 the example, the pages 120 to be flushed are divided into four groups, including a first group 901, a second group 902, a third group 903, and a fourth group 904. The logical addresses of the pages 120-1, 120-6, and 120-8 in the first group 901 are within the first address range. The logical addresses of the pages 120-2 and 120-3 in the second group 902 are within the second address range. The logical addresses of the pages 120-4 and 120-5 in the third group 903 are within the third address range. The logical address of the page 120-7 in the fourth group 904 is within the fourth address range. It can be understood that the first, second, third, and fourth address ranges are different from each other.

[0064] In some embodiments, the first, second, third, and fourth address ranges can be top-level address ranges. For example, the logical addresses of the pages 120-1, 120-6, and 120-8 in the first group 901 can be within the top-level address range 821, while the logical addresses of the pages 120-2 and 120-3 in the second group 902 can be within the top-level address range 822.

[0065] Return reference Figure 7 After grouping multiple pages 120 to be flushed, a predetermined number of pages can be selected from at least one of the multiple groups for flushing to the persistent storage device 140 based on the positions of the auxiliary descriptors 230 of the pages to be flushed in each group in the queue 235. The predetermined number described herein can refer to the number of pages to be flushed in one flushing cycle.

[0066] In some embodiments, the average offset of the auxiliary descriptors 230 of the pages to be flushed in each group relative to the tail of the queue 235 can be determined for each group. Then, based on the value of the average offset, the pages in the multiple groups can be flushed sequentially. For example, the pages in the group with the smallest average offset can be flushed first, and then the pages in the group with the second smallest average offset can be flushed until the predetermined number is reached.

[0067] In some embodiments, the multiple groups can be sorted based on the offsets of the oldest pages to be flushed in each group relative to the tail of the queue 235. Figure 7 Boxes 720 to 740 in [reference] describe such an embodiment.

[0068] In box 720, for each of the multiple groups, the offset of the auxiliary descriptor 230 of the oldest page to be flushed in the queue 235 relative to the tail can be determined. In box 730, the multiple groups can be sorted based on the offsets determined for the multiple groups respectively. In box 740, based on the sorting, a predetermined number of pages to be flushed are selected from at least one of the multiple groups as target pages in sequence.

[0069] Reference Figure 9 and Figure 10 to describe such an example. With reference to Figure 2 it can be seen that in the first group 901, page 120-1 is the oldest page to be flushed, and its offset relative to the tail of the queue 235 is 0. In the second group 902, page 120-2 is the oldest page to be flushed, and its offset relative to the tail of the queue 235 is 1. In the third group 903, page 120-4 is the oldest page to be flushed, and its offset relative to the tail of the queue 235 is 3. In the fourth group 904, page 120-7 is the oldest page to be flushed, and its offset relative to the tail of the queue 235 is 6.

[0070] Based on the above offset values determined for the first group 901, the second group 902, the second group 903, and the fourth group 904 respectively, sorting these groups gives Figure 10 the sorting result 1000 shown. Figure 10 FIG. [reference] shows a schematic diagram of the sorting result 1000 according to some embodiments of the present disclosure. Figure 10The sorting result is shown in the form of a binary tree, which should be understood as being merely exemplary and not intended to be limiting. In the embodiments of the present disclosure, the sorting result may also be maintained in other ways (e.g., a linked list).

[0071] As Figure 10 indicated by the arrows in, the first group 901 is sorted at the front, and the fourth group 904 is sorted at the end. Accordingly, the flushing can be performed sequentially starting from the pages in the first group 901. After the current group is empty, move to the next group to perform the flushing operation until a predetermined number of pages are flushed. For example, first flush the pages in the first group 901, i.e., pages 120-1, 120-6, and 120-8. After the first group 901 is empty, move to the second group 902. In the case where the predetermined number is 4, the oldest page to be flushed in the second group 902 (i.e., the page with the auxiliary descriptor closest to the tail), i.e., page 120-2, is flushed to the persistent storage device 130. Such an example is as Figure 6 shown. In the case where the predetermined number is greater than 4, the remaining pages in the second group 902, such as page 120-3, will continue to be flushed.

[0072] The embodiments of flushing pages based on group sorting are described above. This can make the pages to be flushed come from the same address range as much as possible. In this way, the consumption of updating the mapping tree can be reduced. In particular, in the case where the address range is the top-level address range, such embodiments have additional advantages.

[0073] If the addresses of the pages flushed within a single flushing cycle are located in different top-level address ranges, then different transactions are required to handle the flushing of these pages. For example, if the logical address of the first page to be flushed is within the top-level address range 821, and the logical address of the next second page to be flushed is within the top-level address range 822, then two different transactions are required to handle the flushing of the first page and the second page. In this case, the locality of the pages to be flushed is poor. In addition, due to the increase in the number of transactions, the flushing performance will decrease.

[0074] In the conventional scheme, moving the tail of the queue of page descriptors forward after flushing takes precedence over the flushing efficiency, and the flushing efficiency depends on the locality of the pages to be flushed. In contrast, in the embodiments of the present disclosure, the queue of auxiliary descriptors can be over-provisioned. Therefore, moving the tail of the queue of auxiliary descriptors forward as much as possible is no longer the primary influencing factor. In this case, the pages to be flushed can be grouped according to the top-level address range. Thereby, the locality of the pages to be flushed is improved. In this way, the number of flushing transactions can be minimized to further improve the performance of the storage system.

[0075] Take the test where the queue of auxiliary descriptors is over-provisioned by 5 times as an example. When the flushed pages are grouped according to the address range of the middle level, the average number of transactions submitted per flush cycle is 23. In contrast, when the flushed pages are grouped according to the address range of the top level, the average number of transactions submitted per flush cycle is reduced to 13.

[0076] Figure 11 FIG. 4 shows a schematic block diagram of an example device 1100 that can be used to implement embodiments of the present disclosure. As Figure 11 shown, the device 1100 includes a central processing unit (CPU) 1101, which can execute various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 1102 or computer program instructions loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The CPU 1101, ROM 1102, and RAM 1103 are connected to each other through a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0077] A plurality of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a magnetic disk, an optical disc, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0078] The various processes and treatments described above, such as methods 500 and 700, can be executed by the processing unit 1101. For example, in some embodiments, methods 200 and 700 can be implemented as computer software programs, which are tangibly included in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the CPU 1101, one or more actions of methods 200 and 700 described above can be executed.

[0079] The present disclosure may be a method, an apparatus, a system, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for performing various aspects of the present disclosure.

[0080] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium can be, for example—but not limited to—an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein is not construed as being a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0081] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or can be downloaded to an external computer or an external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0082] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or alternatively, may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0083] Aspects of the present disclosure are described herein with reference to the flowchart and / or block diagram of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer - readable program instructions.

[0084] These computer - readable program instructions can be provided to a processing unit of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that, when the instructions are executed by the processing unit of the computer or other programmable data - processing apparatus, a device is created that implements the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions includes a manufacture, which includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0085] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other devices implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0086] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0087] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A method for managing a storage system, comprising: Dividing a plurality of pages to be flushed in a cache device of the storage system into a plurality of groups based on the respective logical addresses of the plurality of pages to be flushed, wherein the logical addresses of the pages to be flushed in the same group are within the same address range, and the plurality of groups correspond to different address ranges; Selecting a predetermined number of target pages from at least one of the plurality of groups based on the positions of the auxiliary descriptors of the pages to be flushed in each group in a queue; Flushing the target pages in the cache device to a persistent storage device of the storage system; Releasing the resources storing the page descriptors of the target pages to a resource pool, the resource pool being configured to provide resources to store the page descriptors of the pages to be flushed in the cache device; And If it is determined that the auxiliary descriptor of the target page is at the tail of the queue of the auxiliary descriptors of the pages to be flushed, removing the auxiliary descriptor of the target page from the queue, the auxiliary descriptor of the page to be flushed being configured to describe the page descriptor of the page to be flushed.

2. The method according to claim 1, wherein the number of auxiliary descriptors that the queue can accommodate is a multiple of the number of page descriptors that the resource pool can store, and the multiple is greater than 1.

3. The method according to claim 1, wherein flushing the pages to be flushed with logical addresses in different address ranges is processed by different transactions.

4. The method according to claim 1, wherein selecting a predetermined number of the target pages from at least one of the plurality of groups comprises: For each of the plurality of groups, determining the offset of the auxiliary descriptor of the oldest page to be flushed in the queue relative to the tail; Sorting the plurality of groups based on the offsets respectively determined for the plurality of groups; And Based on the sorting, sequentially selecting the predetermined number of pages to be flushed from at least one of the plurality of groups as the target pages.

5. The method according to claim 1, wherein the auxiliary descriptor at least indicates the following items: The position of the auxiliary descriptor in the queue, and The position of the page descriptor in the resource pool.

6. The method according to claim 1, wherein the queue is stored in a volatile storage device of the storage system, and the resource pool is provided by at least one of the following: The volatile storage device, or The cache device.

7. The method according to claim 1, further comprising: If it is determined that additional pages are stored in the cache device, allocating resources from the resource pool to store the page descriptors of the additional pages; And Adding the auxiliary descriptors of the additional pages to the head of the queue.

8. The method according to claim 1, wherein the auxiliary descriptor is smaller than the page descriptor.

9. An electronic device, comprising: A processor; And A memory coupled to the processor, the memory having instructions stored therein, the instructions when executed by the processor causing the device to perform actions, the actions including: Based on the logical addresses of multiple pages to be flushed in a cache device of a storage system, divide the multiple pages to be flushed into multiple groups, where the logical addresses of the pages to be flushed in the same group are within the same address range, and the multiple groups correspond to different address ranges; Based on the positions of the auxiliary descriptors of the pages to be flushed in each group in the queue, select a predetermined number of target pages from at least one of the multiple groups; Flush the target pages in the cache device to the persistent storage device of the storage system; Release the resources storing the page descriptors of the target pages to a resource pool, where the resource pool is configured to provide resources to store the page descriptors of the pages to be flushed in the cache device; and If it is determined that the auxiliary descriptor of the target page is at the tail of the queue of the auxiliary descriptors of the pages to be flushed, remove the auxiliary descriptor of the target page from the queue, where the auxiliary descriptors of the pages to be flushed are configured to describe the page descriptors of the pages to be flushed.

10. The electronic device according to claim 9, wherein the number of auxiliary descriptors that the queue can accommodate is a multiple of the number of page descriptors that the resource pool can store, and the multiple is greater than 1.

11. The electronic device according to claim 9, wherein the flushing of the pages to be flushed with logical addresses in different address ranges is processed by different transactions.

12. The electronic device according to claim 9, wherein selecting a predetermined number of the target pages from at least one of the multiple groups includes: For each of the multiple groups, determine the offset of the auxiliary descriptor of the oldest page to be flushed in the queue relative to the tail; Based on the offsets respectively determined for the multiple groups, sort the multiple groups; And Based on the sorting, sequentially select the predetermined number of pages to be flushed from at least one of the multiple groups as the target pages.

13. The electronic device according to claim 9, wherein the auxiliary descriptor at least indicates the following items: The position of the auxiliary descriptor in the queue, and The position of the page descriptor in the resource pool.

14. The electronic device according to claim 9, wherein the queue is stored in a volatile storage device of the storage system, and the resource pool is provided by at least one of the following: The volatile storage device, or The cache device.

15. The electronic device according to claim 9, the actions further include: If it is determined that additional pages are stored in the cache device, allocate resources from the resource pool to store the page descriptors of the additional pages; And Add the auxiliary descriptors of the additional pages to the head of the queue.

16. The electronic device according to claim 9, wherein the auxiliary descriptor is smaller than the page descriptor.

17. A computer program product, the computer program product being tangibly stored on a computer-readable medium and including machine-executable instructions that, when executed, cause a machine to perform the method according to any one of claims 1 to 8.

Images