A method and apparatus for managing a deduplicated volume space

Abstract

The application discloses a deduplication volume space management method and device, and relates to the technical field of computers, which comprises the following steps: when any storage controller in a multi-control storage system receives a deduplication volume space recovery request, determining a to-be-recovered logical address corresponding to the space recovery request, querying a target physical address corresponding to the to-be-recovered logical address from a mapping table of <logical address pointing to physical address>, and deleting the to-be-recovered logical address corresponding to the target physical address from a second mapping table of <physical address pointing to logical address> through a first storage controller to which the target physical address belongs; and after the to-be-recovered logical address is successfully deleted, recovering the space of a sub-volume corresponding to the target physical address in the deduplication volume, so that the space of the sub-volume corresponding to a single storage controller can be reduced in real time, the space of the entire deduplication volume does not need to be recovered, and therefore the space recovery efficiency of the deduplication volume is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and device for deduplication space management. Background Technology

[0002] Currently, deduplication has become a standard feature of multi-controller storage systems and is a key technology for reducing storage costs. Deduplication is a coarse-grained compression technique. Thanks to lower computational overhead, deduplication can find duplicate data units (data blocks or files) throughout the entire storage system, thus achieving higher compression efficiency compared to traditional byte-level compression techniques.

[0003] Currently, to improve data deduplication rates, multi-controller storage systems typically deploy global deduplication technology across multiple controllers and allocate a portion of space from the storage pool as the initial space for the global deduplication volume. This storage space is assigned a number (i.e., volume number). As data is continuously written, the storage pool continuously allocates storage space to the global deduplication volume. One of the most pressing issues in managing the space of the global deduplication volume is how to achieve real-time space reduction. However, currently, when reclaiming space from the global deduplication volume, the entire volume is typically reclaimed at once, failing to reduce the occupied space in real time, thus affecting the space reclamation efficiency of the storage pool. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a deduplication volume space management method and device, which can shrink the sub-volume space corresponding to a single storage controller in real time without reclaiming the space of the entire deduplication volume, thereby effectively improving the space reclamation efficiency of the global deduplication volume and thus improving the space reclamation efficiency of the storage pool. The specific solution is as follows:

[0005] In a first aspect, this application discloses a deduplication volume space management method, applied to any storage controller in a multi-controller storage system, comprising:

[0006] Receive a space reclamation request for the target deduplication volume and determine the logical address to be reclaimed corresponding to the space reclamation request;

[0007] The first mapping table is used to query the target physical address corresponding to the logical address to be reclaimed; the first mapping table is used to record the mapping relationship between different logical addresses and different physical addresses.

[0008] The first storage controller to which the target physical address belongs is determined, and the space reclamation request, the logical address to be reclaimed, and the target physical address are sent to the first storage controller. Based on the type of space reclamation request, the logical address to be reclaimed corresponding to the target physical address is deleted from the second mapping table of the first storage controller. The second mapping table is used to record the mapping relationship between different physical addresses and different logical addresses.

[0009] When a first successful deletion response is received for the logical address to be reclaimed, space reclamation is performed on the subvolume corresponding to the target physical address in the target de-deleted volume.

[0010] Secondly, this application discloses an electronic device, including a processor and a memory; wherein the processor implements the aforementioned deduplication space management method when executing a computer program stored in the memory.

[0011] As can be seen, this application is applied to any storage controller in a multi-controller storage system. When a space reclamation request for a target deduplicated volume is received, the logical address to be reclaimed corresponding to the space reclamation request is first determined. Then, the target physical address corresponding to the logical address to be reclaimed is queried from the first mapping table. The first storage controller to which the target physical address belongs is then determined, and the space reclamation request, the logical address to be reclaimed, and the target physical address are sent to the first storage controller. Based on the type of space reclamation request, the logical address to be reclaimed corresponding to the target physical address is deleted from the second mapping table of the first storage controller. The second mapping table is used to record the mapping relationship between different physical addresses and different logical addresses. When a first deletion success response for the logical address to be reclaimed is received, space reclamation is performed on the sub-volume in the target deduplicated volume corresponding to the target physical address.

[0012] In this application, when any storage controller in a multi-controller storage system receives a deduplication volume space reclamation request, it first determines the logical address to be reclaimed corresponding to the current space reclamation request. Then, it queries the first mapping table of <logical address pointing to physical address> to find the target physical address corresponding to the logical address to be reclaimed. Through the first storage controller to which the target physical address belongs, it deletes the logical address to be reclaimed corresponding to the target physical address from the second mapping table of <physical address pointing to logical address>. Finally, after the logical address to be reclaimed is successfully deleted, the space of the sub-volume corresponding to the target physical address in the deduplication volume is reclaimed. In this way, the space of the sub-volume corresponding to a single storage controller can be reduced in real time without reclaiming the space of the entire deduplication volume, thereby effectively improving the space reclamation efficiency of the global deduplication volume and thus improving the space reclamation efficiency of the storage pool. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0014] Figure 1 This application discloses a flowchart of a deduplication space management method.

[0015] Figure 2 This application discloses a specific <logical address, physical address> mapping representation.

[0016] Figure 3 This application discloses a specific representation of a sub-request recycling chain.

[0017] Figure 4 This application discloses a specific <physical address, logical address> mapping representation.

[0018] Figure 5 This application discloses a specific <physical address, hash value> mapping representation.

[0019] Figure 6 This is a schematic diagram of a specific deduplication space reclamation process disclosed in this application;

[0020] Figure 7 This application discloses a flowchart of a specific deduplication space management method. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] This application discloses a deduplication volume space management method, applicable to any storage controller in a multi-controller storage system. See [link to relevant documentation]. Figure 1 As shown, the method includes:

[0023] Step S11: Receive a space reclamation request for the target deduplication volume and determine the logical address to be reclaimed corresponding to the space reclamation request.

[0024] It should be noted that the deduplication space management scheme proposed in this application is applied to any storage controller in a multi-controller storage system. A multi-controller storage system refers to a storage system composed of multiple storage devices, each of which may include one or more storage controllers. The storage controller is the core component of the storage system, used for data transmission and reception, data storage, and data protection.

[0025] It's important to note that before reclaiming space from a deduplicated volume, it's necessary to create one first. A deduplicated volume is the sum of sub-volumes corresponding to all storage controllers in a multi-controller storage system. The creation process includes: constructing the target hard drive as an independent redundant array of disks using the multi-controller storage system; integrating these independent redundant arrays to obtain a storage pool; allocating a storage space of a preset size from the storage pool; using this preset storage space as the initial space for the target deduplicated volume; dividing the initial space into multiple partitioned spaces; and allocating these partitioned spaces to various storage controllers in the multi-controller storage system to obtain sub-volumes corresponding to each storage controller. Understandably, multi-controller storage systems typically construct multiple RAID arrays using a preset RAID (redundant array of independent disks) algorithm, then integrate and manage these RAID arrays to form a "storage pool," providing users with a unified storage space and enabling storage devices to be converted into logical data storage. In this context, a storage pool refers to a group of local disks available for storage within a storage system. It can consist of one or more RAID arrays. The storage pool not only coordinates the overall functionality of the local storage devices but also manages them, coordinates their usage, and integrates and allocates their space. Furthermore, the storage pool hides the characteristics of each individual storage device, forming a unified model that provides users with virtual disks, or "volumes," thereby improving resource utilization and flexibility. A volume is a logical disk provided by the system to the connected host; applications actually access volumes rather than directly accessing the hard drive. In this embodiment, the target hard drive can first be constructed into multiple RAID arrays using a multi-controller storage system. These RAID arrays are then integrated to obtain a storage pool. A storage space of a preset size is then allocated from this pool and used as the initial space for the target deduplication volume. This initial space is then divided according to a preset partitioning method (such as equal partitioning, preset partitioning ratio, or storage controller priority) to obtain multiple partitioned spaces. Each partitioned space is then allocated to a storage controller in the multi-controller storage system, resulting in a sub-volume corresponding to each storage controller, such as storage controller 0 corresponding to sub-volume 0. By creating subvolumes for each storage controller, it is convenient to reduce the subvolume space corresponding to a single storage controller in real time, thus facilitating precise subvolume space reclamation.

[0026] It should be noted that each subvolume in this application maintains four mapping tables: a <logical address, physical address> mapping table, a <physical address, logical address> mapping table, a <hash value, physical address> mapping table, and a <physical address, hash value> mapping table. The hash value is the value obtained by calculating a fixed-length data (e.g., 8KB of data) using a hash algorithm; the physical address is the storage location of the 8KB of data in the storage pool; and the logical address is the address presented to the user by the storage system. As data is continuously written, these four mapping tables and the space of the deduplication volume are constantly updated.

[0027] In this embodiment, when any storage controller in the multi-controller storage system (such as storage controller 0) receives a space reclamation request for a target deduplication volume (i.e., a global deduplication volume), it first counts the logical address to be reclaimed corresponding to the space reclamation request, that is, counts the logical address of the space to be reduced.

[0028] Specifically, space reclamation requests can be split into multiple sub-reclamation requests according to a preset byte length, and then the logical addresses to be reclaimed corresponding to each sub-reclamation request can be counted. It is understandable that when a user needs to reduce the space of a globally deduplicated volume, a space reclamation request will be generated through commands or other means. Each space reclamation request will cover a certain range of logical addresses and will be sent to the corresponding storage controller in the multi-controller storage system. In one specific implementation, the multi-controller storage system has a total of 4 storage controllers: storage controller 0, storage controller 1, storage controller 2, and storage controller 3. When storage controller 0 receives a space reclamation request for the target deduplication volume, it divides the space reclamation request into several sub-reclamation requests in 8KB granularity according to the logical address range covered by the space reclamation request. Specifically, when the starting LBA (Logical Block Address) of the logical address range to be reclaimed is Start_LBA and the ending LBA is End_LBA, the number of sub-reclamation requests that can be divided is N=(End_LBA-Start_LBA) / 8, and the logical address range of the nth sub-reclamation request is [Start_LBA+8*n, Start_LBA+8*(n+1)], where n = 0,1,2,…,N-1. For example, when the logical address range to be reclaimed is 0-31KB (i.e., the length is 32KB), the number of sub-reclaimed requests is N = 32 / 8 = 4. These 4 sub-reclaimed requests can be named Sub-reclaimed Request 0, Sub-reclaimed Request 1, Sub-reclaimed Request 2, and Sub-reclaimed Request 3, respectively, with logical address ranges of [0,7], [8,15], [16,23], and [24,31], respectively. By splitting the space reclamation request into multiple sub-reclaimed requests, subsequent reclamation operations can be performed in parallel, thereby improving the efficiency of space reclamation.

[0029] Step S12: Query the target physical address corresponding to the logical address to be reclaimed from the first mapping table; the first mapping table is used to record the mapping relationship between different logical addresses and different physical addresses.

[0030] In this embodiment, after determining the logical address to be reclaimed corresponding to the space reclamation request, the target physical address corresponding to the logical address to be reclaimed can be queried from the first mapping table; wherein the first mapping table (denoted as <logical address, physical address> mapping table) is used to record the mapping relationship between different logical addresses and different physical addresses.

[0031] It should be noted that since a logical address can only correspond to one physical address, there is a one-to-one mapping relationship between logical addresses and physical addresses in the <logical address, physical address> mapping table.

[0032] For details, see Figure 2 As shown, if the logical address to be reclaimed is logical address 0, then by querying entry 0 in the <logical address, physical address> mapping table, it can be found that the target physical address that has a mapping relationship with logical address 0 is physical address 0.

[0033] Step S13: Determine the first storage controller to which the target physical address belongs, and send the space reclamation request, the logical address to be reclaimed, and the target physical address to the first storage controller, so as to delete the logical address to be reclaimed corresponding to the target physical address from the second mapping table of the first storage controller based on the type of the space reclamation request; the second mapping table is used to record the mapping relationship between different physical addresses and different logical addresses.

[0034] In this embodiment, after querying the first mapping table (i.e., the <logical address, physical address> mapping table) to find the target physical address (physical address 0) corresponding to the logical address to be reclaimed (e.g., logical address 0), the storage controller corresponding to the target physical address (physical address 0) is further determined, i.e., the first storage controller (e.g., storage controller 1) to which physical address 0 belongs. Then, the space reclamation request, the logical address to be reclaimed (logical address 0), and the target physical address (physical address 0) are sent to the first storage controller (i.e., storage controller 1).

[0035] Next, when the first storage controller (i.e., storage controller 1) receives a space reclamation request, a logical address to be reclaimed (such as logical address 0), and a target physical address (physical address 0), it first parses the type of the space reclamation request. If the request type is a reclamation request type, it deletes the logical address to be reclaimed (logical address 0) corresponding to the target physical address (physical address 0) from the second mapping table (denoted as the <physical address, logical address> mapping table) located in the first storage controller (i.e., storage controller 1). The second mapping table (<physical address, logical address> mapping table) is used to record the mapping relationship between different physical addresses and different logical addresses.

[0036] In another implementation, if the parsed request type is a non-recovery request type (such as a read request type or a write request type), then the corresponding data read and write operations are performed.

[0037] Specifically, determining the first storage controller to which the target physical address belongs can include: counting the number of storage controllers in a multi-controller storage system to obtain the total number of controllers; performing a modulo operation on the target physical address and the total number of controllers to obtain the controller number of the first storage controller to which the target physical address belongs; and determining the first storage controller based on the controller number. For example, when there are 4 storage controllers in the multi-controller storage system, performing a modulo operation on the target physical address (physical address 0) and the total number of controllers (i.e., 4) to obtain the controller number of the first storage controller (storage controller 1), specifically calculated as: controller number = physical address 0 % 4. The first storage controller (storage controller 1) to which the target physical address (physical address 0) belongs can be directly determined by the calculated controller number.

[0038] In this embodiment, deleting the logical address to be reclaimed corresponding to the target physical address from the second mapping table of the first storage controller can specifically include: storing the sub-reclaimment request in the sub-reclaimment list corresponding to the first storage controller; querying the total number of logical addresses corresponding to each sub-reclaimment request in the sub-reclaimment list from the second mapping table of the first storage controller; the total number of logical addresses is the total number of different logical addresses corresponding to a single physical address; reordering the multiple sub-reclaimment requests in the sub-reclaimment list according to the total number of logical addresses in descending order to obtain sorted reclamation requests; and deleting the logical addresses corresponding to each sub-reclaimment request in the sorted reclamation requests from the second mapping table according to a preset priority. For example, sub-reclaimment request 0 can be stored in the sub-reclaimment list (sub_req_list) corresponding to the first storage controller (storage controller 1), see [link to relevant documentation]. Figure 3 As shown, the sub-recycling request list (sub_req_list) includes sub-recycling request k (i.e., sub_req k), sub-recycling request c (i.e., sub_req c), sub-recycling request a (i.e., sub_reqa), ..., sub-recycling request 0 (i.e., sub_req 0) in sequence. Next, the total number of logical addresses corresponding to each sub-recycling request in the sub-recycling request list is queried from the second mapping table (<physical address, logical address> mapping table) located in the first storage controller (storage controller 1). It should be noted that since one physical address can correspond to multiple logical addresses, the physical address and logical address in the <physical address, logical address> mapping table have a one-to-many mapping relationship. To facilitate obtaining the number of logical addresses corresponding to a single physical address, statistical information on the number of logical addresses can be added to the second mapping table (<physical address, logical address> mapping table), that is, the total number of different logical addresses corresponding to a single physical address. For example, see [link to documentation]. Figure 4As shown, when the target physical address corresponding to sub-recycling request 0 is physical address 0, by querying entry 0 in the second mapping table (<physical address, logical address> mapping table), it can be found that the logical addresses corresponding to physical address 0 include logical address 0, logical address 1, ..., logical address k. The total number of all logical addresses corresponding to physical address 0 can also be directly obtained from the mapping table. Further, according to the total number of logical addresses corresponding to each sub-recycling request (sub_req), the multiple sub-recycling requests in the sub-recycling request list (sub_req_list) are reordered in descending order, resulting in sorted recycling requests. Then, according to a preset priority, the logical addresses corresponding to each sub-recycling request in the sorted recycling requests are deleted from the second mapping table (<physical address, logical address> mapping table). For example, logical address 0 corresponding to sub-recycling request 0 (i.e., sub_req 0) in the sorted recycling requests is deleted from the second mapping table. Additionally, the total number of corresponding logical addresses in the second mapping table after deletion needs to be updated. In this embodiment, all received sub-reclaiming requests are managed uniformly through a sub-reclaiming request chain. The second mapping table (<physical address, logical address> mapping table) used to query logical addresses not only records the mapping relationship between physical addresses and logical addresses, but also records the total number of all logical addresses corresponding to a single physical address. This makes it convenient to perform reclamation operations on different sub-reclaiming requests based on the total number of logical addresses, thereby meeting more flexible volume space reclamation scenarios and improving the overall space reclamation rate of the deduplicated volume.

[0039] Specifically, storing the sub-recycling request in the sub-recycling request list corresponding to the first storage controller can include: identifying the request type of the currently received sub-recycling request to obtain the sub-request type; if the sub-request type is a recycling request type, then storing the sub-recycling request in the sub-recycling request list corresponding to the first storage controller. In this embodiment, considering that the storage controller can not only be used for data recycling operations, but also for other data operations, such as data transmission and reception, data reading and writing, and data protection operations, when a sub-recycling request is received, the type of the request can be identified first. If the request type is a recycling request type (i.e., a request used to perform a recycling operation), then the corresponding sub-recycling request (sub_req) is stored in the sub-recycling request list (sub_req_list) corresponding to the first storage controller (storage controller 1).

[0040] Specifically, querying the second mapping table of the first storage controller for the total number of logical addresses corresponding to each sub-recycling request in the sub-recycling list can include: querying the second mapping table of the first storage controller for the total number of logical addresses corresponding to each sub-recycling request in the sub-recycling list according to a preset period; correspondingly, deleting logical addresses corresponding to each sub-recycling request in the sorted recycling requests from the second mapping table according to a preset priority can include: deleting logical addresses corresponding to each sub-recycling request in the sorted recycling requests in the second mapping table in ascending order of the total number of logical addresses. For example, every five minutes, storage controller 1 re-sorts all sub-recycling requests (sub_req) in the sub-recycling list (sub_req_list) according to the total number of logical addresses corresponding to each sub-recycling request (sub_req), and then prioritizes processing the sub-recycling request with the smallest total number of logical addresses. Assuming that the sub-recycling request with the smallest total number of logical addresses is sub-recycling request 0, then logical address 0 corresponding to sub-recycling request 0 is deleted from the second mapping table first. By obtaining the total number of logical addresses from the second mapping table, reordering the sub-recycling requests in the sub-recycling chain based on the total number of logical addresses, and deleting the corresponding logical addresses in the second mapping table in ascending order of the total number of logical addresses, requests with relatively small request workloads can be processed first, thereby improving the efficiency of the overall logical address deletion operation and the user experience.

[0041] Furthermore, after sequentially deleting the logical addresses corresponding to each sub-recycling request in the sorted recycling requests from the second mapping table, the process may further include: determining whether a logical address corresponding to the target physical address exists in the second mapping table; if a logical address corresponding to the target physical address exists in the second mapping table, then sending a first deletion success response for the logical address to be recycled to any storage controller. In this embodiment, after performing a deletion operation on the corresponding logical address in the second mapping table (e.g., deleting logical address 0 corresponding to physical address 0), the total number of logical addresses corresponding to physical address 0 can be used to determine whether a logical address corresponding to the target physical address (physical address 0) exists in the second mapping table. If a logical address corresponding to the target physical address (physical address 0) exists in the second mapping table, that is, the total number of logical addresses corresponding to physical address 0 is not 0, then a deletion success response for the logical address to be recycled (logical address 0) can be sent to storage controller 0.

[0042] In another specific implementation, it may further include: if no logical address corresponding to the target physical address exists in the second mapping table, then querying the target hash value corresponding to the target physical address from the third mapping table located in the first storage controller; the third mapping table is used to record the mapping relationship between different physical addresses and different hash values, and the hash value is the value obtained after hashing the target data corresponding to different physical addresses; determining the second storage controller to which the target hash value belongs, and generating a derived sub-request with the request type of recycling request; sending the derived sub-request, the target physical address, and the target hash value to the second storage controller to identify the request type of the derived sub-request, deleting the mapping relationship corresponding to the target hash value and the target physical address from the fourth mapping table of the second storage controller, and sending a second deletion success response for the target hash value and the target physical address to the first storage controller; the fourth mapping table is used to record the mapping relationship between different hash values ​​and different physical addresses; when the first storage controller receives the second deletion success response sent by the second storage controller, it forwards the second deletion success response to any storage controller. In this embodiment, if the second mapping table (<physical address, logical address> mapping table) does not contain a logical address corresponding to the target physical address (physical address 0), that is, the total number of logical addresses corresponding to physical address 0 is 0 (i.e., all logical addresses corresponding to physical address 0 have been deleted), then the target hash value corresponding to the target physical address (physical address 0) can be queried from the third mapping table (denoted as <physical address, hash value> mapping table) located in the first storage controller (storage controller 1). For example, see... Figure 5 As shown, by querying entry 0 in the third mapping table (denoted as the <physical address, hash value> mapping table), it can be seen that the target hash value corresponding to physical address 0 is hash value 0. It should be noted that the hash values ​​recorded in the third mapping table are the values ​​obtained after hashing the target data (referring to fixed-length data, such as 8KB, which can be user data or metadata, etc.) corresponding to different physical addresses. Next, see... Figure 6 As shown, the second storage controller (such as storage controller 3) to which the target hash value (hash value 0) belongs is determined, and a derived sub-request with request type of recycling request is generated. Then, the derived sub-request, the target physical address (physical address 0) and the target hash value (hash value 0) are sent to the second storage controller (such as storage controller 3).

[0043] When the second storage controller (e.g., storage controller 3) receives the aforementioned information sent by the first storage controller (storage controller 1), it parses it and first identifies the request type of the derived sub-request. If the request type is identified as a recycling request type, it deletes the mapping relationship corresponding to the target hash value (hash value 0) and the target physical address (physical address 0) from the fourth mapping table (denoted as the <hash value, physical address> mapping table) of the second storage controller (e.g., storage controller 3) (i.e., simultaneously deleting hash value 0 and physical address 0). Then, it sends a deletion success response for the target hash value (hash value 0) and the target physical address (physical address 0) to the first storage controller (e.g., storage controller 1). The fourth mapping table (<hash value, physical address> mapping table) is used to record the mapping relationship between different hash values ​​and different physical addresses, for example, hash value 0 points to physical address 0. Furthermore, when the first storage controller (storage controller 1) receives the deletion success response sent by the second storage controller (e.g., storage controller 3), it forwards the deletion success response to any storage controller (i.e., storage controller 0). After detecting that all logical addresses corresponding to physical address 0 in the second mapping table (<physical address, logical address> mapping table) have been deleted, this application determines the target hash value corresponding to the current physical address 0 by querying the third mapping table (<physical address, hash value> mapping table), then determines the corresponding storage controller 3 based on the target hash value, and deletes the corresponding mapping relationship in the fourth mapping table (<hash value, physical address> mapping table) of the storage controller 3, thereby realizing the update of the fourth mapping table (<hash value, physical address> mapping table) and laying the foundation for subsequent real-time space reclamation operations of the volume.

[0044] Specifically, determining the second storage controller to which the target hash value belongs can include: performing a modulo operation on the target hash value and the total number of controllers to obtain the controller number of the second storage controller to which the target hash value belongs; and determining the second storage controller based on the controller number. For example, when there are a total of 4 storage controllers in a multi-controller storage system, performing a modulo operation on the target hash value (hash value 0) and the total number of controllers (i.e., 4) to obtain the controller number of the second storage controller (such as storage controller 3) to which hash value 0 belongs. The specific calculation formula is: controller number = hash value 0 % 4. The second storage controller (storage controller 3) to which the target hash value (hash value 0) belongs can be directly determined by the calculated controller number.

[0045] Step S14: When a first successful deletion response is received for the logical address to be reclaimed, space reclamation is performed on the sub-volume corresponding to the target physical address in the target de-deleted volume.

[0046] In this embodiment, when the storage controller 0 receives the first successful deletion response sent by the storage controller 1 for the logical address to be reclaimed (logical address 0), it can reclaim space in the sub-volume corresponding to the target physical address (physical address 0) in the target de-deleted volume according to a preset byte length (such as 8KB).

[0047] For details, see Figure 6 As shown, when storage controller 0 receives the second successful deletion response (i.e., from storage controller 1), it can reclaim space from the sub-volume corresponding to the target physical address (physical address 0) in the target deduplication volume according to a preset byte length (e.g., 8KB). Since the target physical address (physical address 0) corresponds to storage controller 1, reclaiming space from the sub-volume corresponding to the target physical address (physical address 0) is equivalent to reclaiming space from the sub-volume corresponding to storage controller 1.

[0048] Additionally, to avoid space reclamation (shrinkage) failures or problems, a complete snapshot or backup of each subvolume of the target deduplicated volume can be created upon receiving a space reclamation request. This way, if space reclamation of a subvolume at a particular physical address / storage controller fails or anomalies, a rollback operation can be performed on the snapshot or backup to restore the volume to its state before shrinkage. For example, if an anomaly occurs, the shrunken subvolume can be deleted, and a new subvolume can be quickly restored from the snapshot. Before restoration, it's necessary to determine if there is sufficient space to perform the rollback operation. Furthermore, the reclaimed subvolumes can be verified and monitored. For example, data integrity can be verified by running application health checks, performing database consistency checks, or sampling critical files and data. Performance metrics, latency, and error counts of the subvolumes can be continuously monitored for a preset time period to confirm whether the available space in the storage pool has increased as expected. After confirming everything is normal, the temporary snapshots or backups created during the preparation phase can be deleted to release the space they occupied.

[0049] As can be seen, in this embodiment, when any storage controller in the multi-controller storage system receives a deduplication volume space reclamation request, it first determines the logical address to be reclaimed corresponding to the current space reclamation request. Then, it queries the first mapping table of <logical address pointing to physical address> to find the target physical address corresponding to the logical address to be reclaimed. Through the first storage controller to which the target physical address belongs, it deletes the logical address to be reclaimed corresponding to the target physical address from the second mapping table of <physical address pointing to logical address>. Finally, after the logical address to be reclaimed is successfully deleted, the space of the sub-volume corresponding to the target physical address in the deduplication volume is reclaimed. Through this method, the space of the sub-volume corresponding to a single storage controller can be reduced in real time without reclaiming the space of the entire deduplication volume, thereby effectively improving the space reclamation efficiency of the global deduplication volume and thus improving the space reclamation efficiency of the storage pool. Simultaneously, it can improve data deduplication efficiency, reduce the actual storage space of the data, and thus save storage resources of the storage device.

[0050] This application discloses a specific deduplication volume space management method, applicable to any storage controller in a multi-controller storage system. See [link to relevant documentation]. Figure 7 As shown, the method includes:

[0051] Step S21: Receive a space reclamation request for the target deduplication volume, split the space reclamation request into multiple sub-reclamation requests according to a preset byte length, and determine the logical address to be reclaimed for each sub-reclamation request.

[0052] Step S22: Query the target physical address corresponding to the logical address to be reclaimed from the first mapping table; the first mapping table is used to record the mapping relationship between different logical addresses and different physical addresses.

[0053] Step S23: Determine the first storage controller to which the target physical address belongs, and send the space reclamation request, the logical address to be reclaimed, and the target physical address to the first storage controller, so as to delete the logical address to be reclaimed corresponding to the target physical address from the second mapping table of the first storage controller based on the type of the space reclamation request; the second mapping table is used to record the mapping relationship between different physical addresses and different logical addresses.

[0054] Step S24: When a first successful deletion response is received for the logical address to be reclaimed, space reclamation is performed on the sub-volume corresponding to the target physical address in the target de-deleted volume.

[0055] Step S25: Update the occupied space size of the sub-volume corresponding to the first storage controller through the first storage controller and based on the preset byte length.

[0056] In this embodiment, after the space is reclaimed from the sub-volume corresponding to the target physical address (i.e., storage controller 1), the space occupied by the sub-volume corresponding to the first storage controller (i.e., storage controller 1) can be updated based on a preset byte length (e.g., 8KB). For example, if the space occupied by the sub-volume corresponding to storage controller 1 before space reclamation is size1, then when controller 1 deletes the <physical address 0, logical address 0> corresponding to sub-reclamation request 0 from the <physical address, logical address> mapping table, it indicates that physical address 0 can be reused. At this time, the space occupied by the sub-volume on storage controller 1 becomes (size1–8).

[0057] Step S26: Generate a space change request for a preset byte length and send the space change request to the multi-controller storage system to update the free space size of the storage pool corresponding to the current multi-controller storage system based on the preset byte length.

[0058] In this embodiment, the storage controller 1 can also generate a space change request for subvolume 1 that has been reduced by 8KB, and send the request to the multi-controller storage system so that the storage system can update the free space size of the storage pool. If the free space size of the storage pool before space reclamation is total_size, then it becomes pool_size+8 after space reclamation.

[0059] For more detailed processing procedures of steps S21 to S24, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0060] As can be seen, in this embodiment, after reclaiming space in the sub-volume corresponding to the target physical address in the target deduplication volume, the first storage controller updates the occupied space size of the sub-volume corresponding to the first storage controller based on a preset byte length, and generates a space change request for the preset byte length. Then, the space change request is sent to the multi-controller storage system so that the storage system updates the free space size of the storage pool corresponding to the multi-controller storage system based on the preset byte length. By updating the occupied space of the sub-volume and the free space of the storage pool, users can easily obtain and view the latest space distribution information, promptly grasp the available space size of the current sub-volume and storage pool, and it is beneficial to calculate the data deduplication rate based on the updated data.

[0061] Accordingly, this application also discloses a deduplication volume space management device, applied to any storage controller in a multi-controller storage system, the device comprising:

[0062] The request receiving module is used to receive space reclamation requests for the target deduplication volume;

[0063] The address determination module is used to determine the logical address to be reclaimed corresponding to the space reclamation request.

[0064] The address query module is used to query the target physical address corresponding to the logical address to be reclaimed from the first mapping table; the first mapping table is used to record the mapping relationship between different logical addresses and different physical addresses;

[0065] The controller determination module is used to determine the first storage controller to which the target physical address belongs;

[0066] The sending module is used to send a space reclamation request, a logical address to be reclaimed, and a target physical address to the first storage controller, so as to delete the logical address to be reclaimed corresponding to the target physical address from the second mapping table of the first storage controller based on the type of the space reclamation request; the second mapping table is used to record the mapping relationship between different physical addresses and different logical addresses;

[0067] The space reclamation module is used to reclaim space in the sub-volume corresponding to the target physical address in the target de-deleted volume when a first successful deletion response for the logical address to be reclaimed is received.

[0068] The specific workflow of each of the above modules can be found in the relevant content disclosed in the foregoing embodiments, and will not be repeated here.

[0069] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the deduplication space management method.

[0070] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the deduplication volume space management method when it is run.

[0071] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0072] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the deduplication volume space management method.

[0073] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above embodiments of the deduplication volume space management method.

[0074] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0075] The above provides a detailed description of the deduplication space management method and device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A deduplication volume space management method, characterized in that, Any storage controller used in a multi-controller storage system, including: Receive a space reclamation request for the target deduplication volume and determine the logical address to be reclaimed corresponding to the space reclamation request; The first mapping table is used to query the target physical address corresponding to the logical address to be reclaimed; the first mapping table is used to record the mapping relationship between different logical addresses and different physical addresses. The first storage controller to which the target physical address belongs is determined, and the space reclamation request, the logical address to be reclaimed, and the target physical address are sent to the first storage controller. Based on the type of the space reclamation request, the logical address to be reclaimed corresponding to the target physical address is deleted from the second mapping table of the first storage controller. The second mapping table is used to record the mapping relationship between different physical addresses and different logical addresses, and the mapping relationship between physical addresses and logical addresses is one-to-many. When a first successful deletion response is received for the logical address to be reclaimed, space reclamation is performed on the sub-volume in the target de-deleted volume corresponding to the target physical address. The method further includes: constructing the target hard disk as an independent redundant disk array through the multi-controller storage system, and integrating the independent redundant disk array to obtain a storage pool; dividing a storage space of a preset size from the storage pool; using the storage space of the preset size as the initial space of the target deduplication volume; the target deduplication volume is the sum of the sub-volumes corresponding to multiple storage controllers in the multi-controller storage system; dividing the initial space to obtain multiple divided spaces, and allocating the multiple divided spaces to each storage controller in the multi-controller storage system to obtain the sub-volumes corresponding to each storage controller.

2. The deduplication space management method according to claim 1, characterized in that, The first storage controller to which the target physical address belongs includes: The total number of storage controllers in the multi-controller storage system is obtained by counting the number of controllers. The controller number of the first storage controller to which the target physical address belongs is obtained by performing a modulo operation on the target physical address and the total number of controllers; The first storage controller is determined based on the controller number.

3. The deduplication space management method according to claim 1, characterized in that, Determining the logical address to be reclaimed corresponding to the space reclamation request includes: The space reclamation request is split into multiple sub-reclamation requests according to a preset byte length, and the logical address to be reclaimed corresponding to each sub-reclamation request is determined. Accordingly, deleting the logical address to be reclaimed corresponding to the target physical address from the second mapping table of the first storage controller includes: The sub-recycling request is stored in the sub-recycling chain list corresponding to the first storage controller; The total number of logical addresses corresponding to each sub-recycling request in the sub-recycling chain list is queried from the second mapping table of the first storage controller; the total number of logical addresses is the total number of different logical addresses corresponding to a single target physical address. The multiple sub-recycling requests in the sub-recycling chain list are reordered in descending order of the total number of logical addresses to obtain the sorted recycling requests; According to a preset priority, delete the logical addresses corresponding to each sub-recycling request in the sorted recycling requests from the second mapping table.

4. The deduplication space management method according to claim 3, characterized in that, The step of storing the sub-recycling request in the sub-recycling list corresponding to the first storage controller includes: Identify the request type of the currently received sub-recycling request to obtain the sub-request type; If the sub-request type is a recycling request type, then the sub-recycling request is stored in the sub-recycling list corresponding to the first storage controller.

5. The deduplication space management method according to claim 3, characterized in that, The step of querying the total number of logical addresses corresponding to each sub-recycling request in the sub-recycling chain list from the second mapping table of the first storage controller includes: According to a preset period, the total number of logical addresses corresponding to each sub-recycling request in the sub-recycling chain list is queried from the second mapping table of the first storage controller. Accordingly, deleting the logical addresses corresponding to each sub-recycling request in the sorted recycling requests from the second mapping table according to a preset priority includes: Logical addresses corresponding to each sub-recycling request in the sorted recycling requests are sequentially deleted from the second mapping table in ascending order of the total number of logical addresses.

6. The deduplication space management method according to claim 5, characterized in that, After sequentially deleting the logical addresses corresponding to each sub-recycling request in the sorted recycling requests from the second mapping table, the method further includes: Determine whether a logical address corresponding to the target physical address exists in the second mapping table; If a logical address corresponding to the target physical address exists in the second mapping table, a first deletion success response for the logical address to be reclaimed is sent to any of the storage controllers.

7. The deduplication space management method according to claim 6, characterized in that, Also includes: If the logical address corresponding to the target physical address does not exist in the second mapping table, the target hash value corresponding to the target physical address is queried from the third mapping table located in the first storage controller; the third mapping table is used to record the mapping relationship between different physical addresses and different hash values, and the hash value is the value obtained after hashing the target data corresponding to different physical addresses; Determine the second storage controller to which the target hash value belongs, and generate a derived sub-request of type recycling request; The derived sub-request, the target physical address, and the target hash value are sent to the second storage controller to identify the request type of the derived sub-request. The mapping relationship corresponding to the target hash value and the target physical address is deleted from the fourth mapping table of the second storage controller, and a second deletion success response for the target hash value and the target physical address is sent to the first storage controller. The fourth mapping table is used to record the mapping relationship between different hash values ​​and different physical addresses. When the first storage controller receives the second deletion success response sent by the second storage controller, it forwards the second deletion success response to either of the storage controllers.

8. The deduplication space management method according to claim 7, characterized in that, Also includes: When the second deletion success response forwarded by the first storage controller is received, the space of the sub-volume corresponding to the target physical address in the target de-deleted volume is reclaimed according to the preset byte length; Accordingly, after reclaiming space in the sub-volume corresponding to the target physical address within the target deduplication volume, the process further includes: The space occupied by the sub-volume corresponding to the first storage controller is updated based on the preset byte length through the first storage controller. A space change request for the preset byte length is generated and sent to the multi-controller storage system to update the free space size of the storage pool corresponding to the multi-controller storage system based on the preset byte length.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the deduplication volume space management method as described in any one of claims 1 to 8 when executing the computer program.

Images