Storage management method, device, program product and medium of multi-control storage system

Abstract

The application discloses a storage management method, device, program product and medium of a multi-controller storage system, relates to storage technology, and comprises K controllers in a memory system, K is a positive integer not less than 2, the storage management method of the multi-controller storage system is applied to each controller and comprises the following steps: receiving a write request and splitting the write request into each sub-request; for any sub-request, judging whether the to-be-written data block corresponding to the sub-request belongs to a repeated data block already stored in a storage device through a home controller of the sub-request; if not, writing the to-be-written data block into the storage device and updating metadata information based on the sub-request; if yes, updating metadata information based on the sub-request. According to the scheme, repeated data deletion can be realized in the multi-controller storage system, the system performance can be effectively exerted by using the K controllers, and load balancing can be guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, in particular to a storage management method, device, program product and medium of a multi-controller storage system. BACKGROUND

[0002] At present, as a technology for reducing storage cost, the deduplication has been applied more and more widely. The deduplication is essentially a lossless data compression technology. For data units (data blocks or files) with repeated contents, only one copy is stored, so it is also called single instance storage (SIS) or content addressable storage (CAS). The deduplication belongs to a coarse-grained compression technology. Due to lower computing overhead, the deduplication can find repeated data units in the entire storage system, thereby achieving higher compression efficiency.

[0003] According to the division of deduplicated data units, the deduplication technology can be divided into two categories: file-level deduplication and block-level deduplication. The file-level deduplication is to find and eliminate files with repeated contents in the storage system. The block-level deduplication is to divide files into data blocks, and then find and eliminate data blocks with complete repetition in the storage system. It can be understood that, compared with files, the granularity of data blocks is finer, so the block-level deduplication can usually find more redundant data, and thus is applied more widely.

[0004] At present, the storage system for the deduplication technology is mainly a single-controller storage system, and mainly aims at deleting local repeated data. However, due to the increasing requirements of storage performance, multi-controller storage systems are applied more and more widely.

[0005] In summary, how to effectively implement the deduplication in the multi-controller storage system is a technical problem that needs to be solved by the technical personnel in the field at present. SUMMARY

[0006] The present application provides a storage management method, device, program product and medium of a multi-controller storage system, to effectively implement the deduplication in the multi-controller storage system.

[0007] In a first aspect, the present application provides a storage management method of a multi-controller storage system. The storage system includes K controllers, K is a positive integer not less than 2, and the storage management method of the multi-controller storage system is applied to each controller, and includes the following steps.

[0008] receiving a write request and splitting the write request into sub-requests;

[0009] For any sub-request, whether the to-be-written data block corresponding to the sub-request belongs to the duplicate data block already stored in the storage device is determined by the home controller of the sub-request;

[0010] If no, the to-be-written data block is written into the storage device, and the metadata information is updated based on the sub-request;

[0011] If yes, the metadata information is updated based on the sub-request.

[0012] In a second aspect, the present application provides an electronic device, comprising:

[0013] a memory for storing a computer program;

[0014] a processor for implementing the steps of the storage management method of the multi-controller storage system when executing the computer program.

[0015] In a third aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the storage management method of the multi-controller storage system.

[0016] In a fourth aspect, the present application provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the steps of the storage management method of the multi-controller storage system.

[0017] In the scheme of the present application, the duplicate data deletion can be effectively implemented in the multi-controller storage system, and the storage cost is reduced. Specifically, the storage system comprises K controllers, K is a positive integer not less than 2, and the storage management method of the multi-controller storage system can be applied to each controller. For any controller, after receiving a write request, in order to effectively exert the system performance and guarantee the load balancing, the write request is split into sub-requests, so that each sub-request is processed by the corresponding home controller. For any sub-request, whether the to-be-written data block corresponding to the sub-request belongs to the duplicate data block already stored in the storage device is determined by the home controller of the sub-request. It can be understood that if the sub-request does not belong to the duplicate data block already stored in the storage device, the to-be-written data block corresponding to the sub-request needs to be written into the storage device, and the metadata information is updated based on the sub-request, otherwise, for the duplicate data block, only the metadata information is updated based on the sub-request, and the to-be-written data block does not need to be written repeatedly. In addition, it can be seen that for any sub-request, whether the to-be-written data block corresponding to the sub-request belongs to the duplicate data block is determined by the home controller of the sub-request, so the performance of the multi-controller storage system can be effectively exerted.

[0018] It can be seen that the scheme can realize the deduplication in the multi-controller storage system, and can utilize the K controllers to effectively exert the system performance and facilitate the load balancing. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0020] Figure 1 The implementation flowchart of the storage management method of the multi-controller storage system provided by the embodiment of the present application is shown in the figure.

[0021] Figure 2 The timing diagram of the storage management method of the multi-controller storage system of the embodiment of the present application is shown in the figure.

[0022] Figure 3 The structural diagram of the electronic device provided by the embodiment of the present application is shown in the figure.

[0023] Figure 4 The structural diagram of the computer readable storage medium of the present application is shown in the figure. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort are within the protection scope of the present application.

[0025] It should be noted that, in the description of the present application, the terms “include”, “contain” or any other variants thereof are intended to cover the non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0026] In order to make the skilled in the art better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments. Please refer to Figure 1 , Figure 1 The implementation flowchart of the storage management method of the multi-controller storage system provided by the embodiment of the present application is shown in the figure.

[0027] The storage system described in this application is a multi-controller storage system, meaning the memory system includes K controllers, where K is a positive integer not less than 2. The storage management method of this multi-controller storage system can be applied to each controller; see reference [link to relevant documentation]. Figure 1 The storage management method of this multi-controller storage system may include the following steps:

[0028] Step S101: Receive the write request and split the write request into sub-requests.

[0029] In practical applications, each controller can receive write requests to be processed and then perform subsequent processing of the write requests according to the principles of this application. This application uses the processing of a single write request as an example for illustration. The controller that receives the write request is referred to as the source controller of the write request in this application.

[0030] After receiving a write request, the controller does not process the write request itself, but splits the write request into sub-requests, so that multiple controllers can process it together. This effectively utilizes the controller resources of the multi-controller storage system. Furthermore, splitting the write request into sub-requests is also beneficial for subsequent deduplication.

[0031] In one specific embodiment of the present invention, receiving a write request and splitting the write request into sub-requests may specifically include:

[0032] Receive write requests and allocate resources for write requests;

[0033] According to the preset splitting rules, the write request is split into sub-requests, and resources are allocated to each sub-request.

[0034] This implementation takes into account the need to cache write requests after they are received. Therefore, resources can be allocated to write requests, specifically memory resources, and these resources can be initialized after allocation, thus enabling temporary storage of write requests. When splitting a write request into sub-requests, it can be done according to preset splitting rules, which can be set and adjusted according to actual needs. For each sub-request, resources can also be allocated and initialized.

[0035] In one specific embodiment of the present invention, the write request is split into sub-requests according to a preset splitting rule, which may specifically include:

[0036] Write requests are split into sub-requests according to a preset fixed granularity, so that the size of the data block to be written for each sub-request conforms to the fixed granularity.

[0037] The implementation considers that, in order to effectively implement the deduplication and improve the compression rate, the block-level deduplication can be performed, so when the write request is split into each sub-request, the write request can be split into each sub-request according to the preset fixed granularity, so that the size of the data block to be written corresponding to each sub-request conforms to the fixed granularity, that is, the data block size specified when the block-level deduplication is performed in the deduplication technology is used as the fixed granularity, so that the size of the data block to be written corresponding to each sub-request is equal to the fixed granularity.

[0038] For example, Figure 2 For example, Figure 2 Four controllers are shown in the example, that is, K=4, and are sequentially recorded as controller 0, controller 1, controller 2, and controller 3. Figure 2 In the example of the controller 0, the controller 0 receives the write request, so the controller 0 is the source controller of the write request. After the controller 0 receives the write request, the controller 0 allocates resources for the write request and initializes the write request, so that the write request can be cached in the allocated resources and wait for subsequent processing. Then, the controller 0 splits the write request, for example, the preset fixed granularity is 8k, the write request is split into four sub-requests, which are recorded as sub-request 0, sub-request 1, sub-request 2, and sub-request 3, and the controller 0 allocates resources for the four sub-requests and initializes the four sub-requests.

[0039] Step S102: For any sub-request, it is determined by the home controller of the sub-request whether the data block to be written corresponding to the sub-request belongs to the repeated data block already stored in the storage device. If not, step S103 is performed, and if yes, step S104 is performed.

[0040] In order to effectively exert the performance of the multi-control storage system, for each sub-request, the sub-request is not processed by the source controller, but is processed by the home controller of the sub-request. For any sub-request, it is determined by the home controller whether the data block to be written corresponding to the sub-request is a repeated data block, so as to determine whether the data block to be written corresponding to the sub-request needs to be data-dumped.

[0041] In one specific implementation of the present application, step S102 can specifically include:

[0042] For any sub-request, the home controller of the sub-request is determined;

[0043] For any sub-request, when the home controller of the sub-request is the source controller, it is determined whether the data block to be written corresponding to the sub-request belongs to the repeated data block;

[0044] When the home controller of the sub-request is a controller other than the source controller, the sub-request is sent to the home controller of the sub-request to allocate resources for the sub-request by the home controller of the sub-request, and to determine whether the to-be-written data block corresponding to the sub-request belongs to the duplicate data block already stored in the storage device by the home controller of the sub-request; and the determination result fed back by the home controller receiving the sub-request is received. The controller receiving the write request is the source controller of the write request.

[0045] This embodiment considers that for any sub-request, the home controller of the sub-request may be the source controller of the write request or a controller other than the source controller. Therefore, after determining the respective home controllers of the sub-requests, when the home controller of a certain sub-request happens to be the source controller, the source controller itself can directly determine whether the to-be-written data block corresponding to the sub-request belongs to the duplicate data block.

[0046] Correspondingly, when the home controller of a certain sub-request is a controller other than the source controller, the sub-request needs to be sent to the home controller of the sub-request through communication, and then the home controller of the sub-request is used to allocate resources and initialize the sub-request to realize the caching of the sub-request, and then the home controller of the sub-request is used to determine whether the to-be-written data block corresponding to the sub-request belongs to the duplicate data block, and the home controller of the sub-request feeds back the determination result to the source controller. It can be seen that this embodiment effectively realizes the determination of whether the to-be-written data block corresponding to each sub-request belongs to the duplicate data block.

[0047] In addition, it needs to be explained that for any sub-request, when the home controller of the sub-request is determined, there can be multiple specific implementation manners, for example, in one specific implementation manner, for any sub-request, the home controller of the sub-request can be determined according to the number of the sub-request. For example, the split sub-requests are numbered from 0, the home controller of the sub-request 0 is the controller 0, the home controller of the sub-request 1 is the controller 1, the home controller of the sub-request 2 is the controller 2, and so on. That is, in this example, each controller is sequentially used as the home controller of each sub-request, which is relatively simple.

[0048] In one specific implementation manner of the present application, for any sub-request, the home controller of the sub-request can be determined in the following manner:

[0049] Hash operation is performed on the to-be-written data block corresponding to the sub-request to obtain a hash operation value of the to-be-written data block;

[0050] The remainder value a obtained after the hash operation value is divided by K is determined;

[0051] The controller numbered a in the K controllers is used as the home controller of the sub-request.

[0052] Where a is an integer, and 0≤a≤K-1.

[0053] This implementation takes into account that, while assigning controllers based on the sub-request number in the previous implementation is simpler and more convenient, it increases the complexity of determining whether the data block to be written corresponding to the sub-request is a duplicate data block. In this implementation, however, the controller assignment is based on hash values, making the subsequent determination of whether the data block to be written corresponding to the sub-request is a duplicate data block much simpler and more convenient. Therefore, in practical applications, this implementation is usually chosen for assigning controllers.

[0054] Specifically, in this implementation, for any sub-request, a hash operation is performed on the corresponding data block to be written, thereby obtaining the hash value of the data block to be written. It is understood that different data blocks to be written will have different hash values; therefore, the hash value is essentially a reflection of the data content of the data block to be written.

[0055] After obtaining the hash value, divide it by K. The remainder will be between 0 and K-1. If the remainder is 0, controller 0 will be the controller of the sub-request. If the remainder is 1, controller 1 will be the controller of the sub-request. If the remainder is 2, controller 2 will be the controller of the sub-request, and so on.

[0056] exist Figure 2 In the example, after hashing the data blocks to be written corresponding to each sub-request, it is determined that the controller to which sub-request 0 belongs is controller 0 (i.e., Figure 2 In the example, the source controller determines that the controller to which sub-request 1 belongs is controller 1, the controller to which sub-request 2 belongs is controller 2, and the controller to which sub-request 3 belongs is controller 3. Therefore, sub-request 0 can be processed by controller 0. For example, if controller 0 determines that the data block to be written corresponding to sub-request 0 is not a duplicate data block, then controller 0 needs to write the data block to be written corresponding to sub-request 0 to the storage device and update the metadata information based on sub-request 0. After processing sub-request 0, Figure 2 In the example, controller 0 can release the resources occupied by sub-request 0.

[0057] In one specific embodiment of the present invention, when the owner controller of a sub-request is the source controller, determining whether the data block to be written corresponding to the sub-request belongs to a duplicate data block includes:

[0058] By checking the hash table managed by the source controller, determine whether the hash value of the data block to be written exists in the hash table managed by the source controller.

[0059] If it exists, then determine that the data block to be written corresponding to the sub-request belongs to the duplicate data block;

[0060] If it does not exist, then it is determined that the data block to be written corresponding to the sub-request does not belong to the duplicate data block;

[0061] Accordingly, when the home controller of a sub-request is a controller other than the source controller, the sub-request is sent to the home controller of the sub-request so that resources can be allocated to the sub-request through the home controller. The home controller of the sub-request also determines whether the data block to be written corresponding to the sub-request is a duplicate data block already stored in the storage device, including:

[0062] When the controller to which the sub-request belongs is a controller other than the source controller, the sub-request is sent to the controller to which the sub-request belongs, so that the controller to which the sub-request belongs can allocate resources for the sub-request, and the hash table managed by the controller to which the sub-request belongs can be used to determine whether there is a hash value of the data block to be written in the hash table managed by the controller.

[0063] If it exists, then determine that the data block to be written corresponding to the sub-request belongs to the duplicate data block;

[0064] If it does not exist, then it is determined that the data block to be written corresponding to the sub-request does not belong to the duplicate data block;

[0065] Specifically, for any data block already stored in the storage device, when the remainder obtained by dividing the hash value of the data block by K is a, the hash value of the data block is recorded in the hash table managed by the controller numbered a.

[0066] In the above implementation, the division of the sub-request to the attribution controller is based on the hash value of the data block to be written corresponding to the sub-request. Therefore, in this implementation, for any attribution controller, the attribution controller can easily determine whether the data block to be written corresponding to the sub-request is a duplicate data block by using the hash table it manages.

[0067] by Figure 2 Taking sub-request 1 as an example, for instance, the hash value of the data block to be written corresponding to sub-request 1 is calculated, and after dividing the hash value by 4, it can be determined that the remainder is 1. Therefore, it can be determined that the controller to which sub-request 1 belongs is controller 1.

[0068] The controller 1 manages a hash table, and for any one data block in the storage device, when the hash operation value of the data block is 1 after the hash operation value is divided by 4, the hash operation value of the data block is recorded in the hash table managed by the controller 1. Similarly, for any one data block in the storage device, when the hash operation value of the data block is a after the hash operation value is divided by 4, the hash operation value of the data block is recorded in the hash table managed by the controller a, a is an integer and can be a value from 0 to K-1.

[0069] As described above, different to-be-written data blocks have different hash operation values, and the hash operation value is a reflection of the data content of the to-be-written data block. In this example, since the controller to which the sub-request 1 belongs is the controller 1, if there is a hash operation value in the hash table managed by the controller 1 that is consistent with the hash operation value of the to-be-written data block corresponding to the sub-request 1, it can be directly determined that the to-be-written data block corresponding to the sub-request 1 is a duplicate data block. Otherwise, as long as there is no hash operation value in the hash table managed by the controller 1 that is the same as the hash operation value of the to-be-written data block corresponding to the sub-request 1, it can be directly determined that the to-be-written data block corresponding to the sub-request 1 is not a duplicate data block without traversing the hash tables managed by other controllers.

[0070] It can be seen that in this implementation, when determining whether the to-be-written data block is a duplicate data block, the controller only needs to use the hash table managed by itself to make the determination, which is simple and convenient, occupies less storage space, and takes less time to determine.

[0071] In addition, it can be understood that if the controller to which the sub-request belongs is determined according to the number of the sub-request as in the above implementation, each controller needs to maintain a global hash table, so that the determination of whether the to-be-written data block is a duplicate data block can be made based on the global hash table, which requires more storage resources, data synchronization of the global hash tables, and a larger amount of data in the global hash table when making the determination, resulting in a higher determination time.

[0072] Step S103: writing the to-be-written data block to the storage device and updating the metadata information based on the sub-request.

[0073] Step S104: updating the metadata information based on the sub-request.

[0074] For the case where the to-be-written data block is not a duplicate data block, it needs to be written to the storage device and the metadata information is updated based on the sub-request. The metadata information can include the mapping relationship between the logical address and the physical address of the to-be-written data block, the hash operation value, and the like.

[0075] If the data block to be written is a duplicate data block, it is not necessary to write it to the storage device. Instead, the metadata information can be updated directly based on the sub-request to save storage resources.

[0076] exist Figure 2 In the example, if the controller to which sub-request 1 belongs is controller 1, the controller to which sub-request 2 belongs is controller 2, and the controller to which sub-request 3 belongs is controller 3, then controller 0 needs to send sub-request 1, sub-request 2, and sub-request 3 to controller 1, controller 2, and controller 3 respectively.

[0077] After receiving sub-request 1, controller 1 can allocate resources and initialize them. Then, controller 1 can process sub-request 1, specifically determining whether the data block to be written corresponding to sub-request 1 is a duplicate data block. After processing, controller 1 can release the resources allocated to sub-request 1 and send the determination result back to controller 0. Upon receiving the determination result from controller 1, controller 0 can determine whether the data block to be written corresponding to sub-request 1 is a duplicate data block. For example, if the data block to be written corresponding to sub-request 1 is a duplicate data block, controller 0 will not perform the writing of that data block, but only needs to update the corresponding metadata information based on sub-request 1. If the data block to be written corresponding to sub-request 1 is not a duplicate data block, controller 0 needs to perform the writing of that data block and update the corresponding metadata information based on sub-request 1. After controller 0 executes the corresponding processing measures according to whether the data block to be written corresponding to sub-request 1 is a duplicate data block, sub-request 1 is processed, and controller 0 can release the resources allocated to sub-request 1.

[0078] Similarly, Figure 2 In the example, after receiving sub-request 2, controller 2 can allocate resources for it and initialize them. Then, controller 2 can process sub-request 2, that is, determine whether the data block to be written corresponding to sub-request 2 is a duplicate data block. After processing, controller 2 can release the resources allocated to sub-request 2 and feed back the judgment result to controller 0. After receiving the judgment result from controller 3, controller 0 can determine whether the data block to be written corresponding to sub-request 2 is a duplicate data block. For example, if the data block to be written corresponding to sub-request 2 is a duplicate data block, controller 0 will not write the data block to be written, but only needs to update the corresponding metadata information based on sub-request 2. If the data block to be written corresponding to sub-request 2 is not a duplicate data block, controller 0 needs to write the data block to be written and update the corresponding metadata information based on sub-request 2. After controller 0 performs the corresponding processing measures according to whether the data block to be written corresponding to sub-request 2 is a duplicate data block, sub-request 2 is processed and controller 0 can release the resources allocated to sub-request 2.

[0079] Similarly, Figure 2 In the example of FIG. 3, after receiving the sub-request 3, the controller 3 can allocate resources for the sub-request 3 and initialize the sub-request 3, and then the controller 3 can process the sub-request 3, i.e., determine whether the data block corresponding to the sub-request 3 is a duplicate data block, and after the processing is completed, the controller 3 can release the resources allocated for the sub-request 3 and feed back the determination result to the controller 0. After receiving the determination result from the controller 3, the controller 0 can determine whether the data block corresponding to the sub-request 3 is a duplicate data block. For example, if the data block corresponding to the sub-request 3 is a duplicate data block, the controller 0 does not perform the write operation on the data block, but only updates the metadata information based on the sub-request 3. If the data block corresponding to the sub-request 3 is not a duplicate data block, the controller 0 needs to perform the write operation on the data block and update the metadata information based on the sub-request 3. After the controller 0 performs the corresponding processing measure according to whether the data block corresponding to the sub-request 3 is a duplicate data block, the processing of the sub-request 3 is completed, and the controller 0 can release the resources allocated for the sub-request 3.

[0080] When all the sub-requests are processed, the controller 0 can determine that the write request is processed, and can release the resources allocated for the write request.

[0081] In one specific embodiment of the present application, the method can further include:

[0082] determining, by the controller to which the sub-request belongs, whether the sub-request generates a derivative sub-request;

[0083] If yes, generating the derivative sub-request of the sub-request, and sending the derivative sub-request to the controller corresponding to the derivative sub-request through the controller to which the sub-request belongs, so that the controller corresponding to the derivative sub-request processes the derivative sub-request to maintain the address mapping relationship for the physical address originally corresponding to the sub-request.

[0084] The condition under which the sub-request generates the derivative sub-request includes that the sub-request is a sub-request representing data update, and the physical address originally corresponding to the sub-request is not managed by the controller to which the sub-request belongs.

[0085] The physical address originally corresponding to the sub-request is managed by the controller corresponding to the derivative sub-request.

[0086] This embodiment takes into account that in some embodiments, data update may occur, and the data block that needs to be updated is not processed by the controller. In order to ensure the accuracy of the address mapping relationship, the derivative sub-request needs to be generated.

[0087] Figure 2For example, in the example, taking the sub-request 2 as an example, the sub-request 2 belongs to a sub-request representing data update, that is, according to the requirement of the sub-request 2, the data content of a certain logical address needs to be updated, and the physical address corresponding to the logical address is not managed by the controller (controller 2) to which the sub-request 2 belongs, then the controller 2 needs to generate a derived sub-request, and send the derived sub-request to the controller corresponding to the derived sub-request, that is, send to the controller managing the physical address originally corresponding to the sub-request 2, for example, the physical address is currently managed by the sub-controller 3, then the derived sub-request needs to be sent to the controller 3, so that the controller 3 processes the derived sub-request, thereby maintaining the address mapping relationship for the physical address originally corresponding to the sub-request, to ensure the accuracy of the address mapping relationship.

[0088] For example, the logical address A originally corresponds to the physical address A, and the sub-request 2 needs to update the data content of the logical address A, and for example, the original data content in the logical address A is “data A”, and due to the deduplication technology, the physical address corresponding to the logical address B and the logical address C is also the physical address A, and for example, the physical address A is managed by the controller 3, then the controller 3 can store the address mapping relationship of “physical address A = logical address A, logical address B, logical address C”, to represent that the physical address A corresponds to three different logical addresses.

[0089] And because the sub-request 2 needs to update the data content of the logical address A, for example, it needs to update the data content from “data A” to “data AA”, and for example, “data AA” is not a deduplication block, then “data AA” will be written to a new physical location (for example, written to the physical address F) by the controller 0, that is, after the execution of the write request, the physical address corresponding to the logical address A becomes the physical address F, it can be seen that the controller 3 needs to be notified to update the above-mentioned address mapping relationship of “physical address A = logical address A, logical address B, logical address C”, because the logical address corresponding to the physical address A no longer contains the logical address A. In this example, after the controller 3 processes the derived sub-request, the above-mentioned address mapping relationship is updated to “physical address A = logical address B, logical address C”.

[0090] In addition, it needs to be explained that which controller manages a certain physical address can have multiple setting methods, usually, it is decided according to the writing party of the data content in the physical address, that is, which controller writes the current data content of the physical address, then the physical address can be managed by the controller.

[0091] In one specific embodiment of the present application, after the write request is split into sub-requests, it can further include:

[0092] When the source controller of the write request fails, the write request and resources occupied by each sub-request are released, and the processing flow of the write request is re-executed.

[0093] This embodiment takes into account that the source controller of the write request may fail, at which time the write request cannot continue to be executed, and therefore the write request and resources occupied by each sub-request can be released, that is, the resources allocated to the write request and each sub-request by the source controller and the resources allocated to the corresponding sub-request by other controllers can be released. The write request can then be placed in a retry linked list to re-execute the processing flow of the write request.

[0094] In an embodiment, re-executing the processing flow of the write request can specifically include shielding the failed controller and re-executing the processing flow of the write request. This embodiment takes into account that after the failed controller is determined, the processing flow of the write request can be re-executed while shielding the failed controller to ensure smooth execution of the write request. For example, the write request is originally sent to controller 0 for processing, controller 0 fails and the write request is re-executed, and controller 0 can be shielded to send the write request to other normal controllers.

[0095] In a specific embodiment of the present application, the following can also be included:

[0096] When a controller other than the source controller of the write request fails, it is determined whether each sub-request is successfully executed;

[0097] If yes, it is determined that the write request processing is complete;

[0098] If no, the write request and resources occupied by each sub-request are released, and the processing flow of the write request is re-executed.

[0099] This embodiment further takes into account that when the write request is sent to a certain controller, the controller does not fail, but other controllers can fail, and it is necessary to determine whether each sub-request is successfully executed. For example, controller 0 divides the write request into sub-request 0, sub-request 1 and sub-request 2, which are processed by controller 0, controller 1 and controller 2, respectively, and for example, controller 3 fails, and since controller 3 does not need to process any sub-request, it does not affect the successful execution of each sub-request in this example. For another example, in this example, controller 2 fails, but controller 2 fails after processing sub-request 2, and each sub-request can still be successfully executed. For another example, controller 2 fails, and controller 2 fails before processing sub-request 2, and sub-request 2 cannot be successfully executed.

[0100] This implementation takes into account that even in the event of a controller failure, if the source controller can determine that all sub-requests have been successfully executed, then the write request processing is complete and does not need to be re-executed. Conversely, if any one sub-request fails to execute, the write request and the resources occupied by each sub-request can be released, and the write request processing flow can be re-executed. Furthermore, it can be understood that when re-executing the write request processing flow, the faulty controller can be shielded. For example, in the above example, after the write request is resent to controller 0, if controller 2 fails, controller 2 can be shielded, preventing sub-requests from being sent to controller 2 for processing; instead, other normal controllers will handle the sub-requests.

[0101] There are several ways to determine controller faults. For example, one implementation can perform heartbeat detection on each controller to determine its fault status. Another approach is to monitor the resources used by each controller, thus achieving more comprehensive fault detection. For instance, monitoring controller CPU utilization, I / O latency, bandwidth, and other metrics can be used. When resource metrics consistently exceed preset thresholds (e.g., CPU full load for more than 5 minutes) and cannot be recovered through load balancing or resource adjustment, a fault can be identified.

[0102] In one specific embodiment of the present invention, it may further include:

[0103] Receive read requests and allocate resources for them;

[0104] When the controller receiving the read request fails, the read request processing flow is re-executed after the failed controller is blocked.

[0105] When the controller receiving the read request is not faulty, it reads the data content specified in the read request based on the metadata information and provides feedback, and releases the resources allocated to the read request.

[0106] The above implementation described the processing of write requests. For read requests, since the data reading process does not require deleting duplicate data, the read request processing flow can be relatively simple, typically handled by a single controller. For example, after controller 0 receives a read request, it can allocate resources for that request. When controller 0 is functioning correctly, it can obtain the corresponding metadata information based on the read request, then read the data content specified in the read request from the hard drive and provide feedback. Finally, it can release the resources allocated to the read request. Of course, if controller 0 fails, it can be disabled, and the read request processing flow can be re-executed. For example, after disabling controller 0, the storage system can assign the read request to controller 1 for processing.

[0107] In the scheme of the present application, the repeated data deletion can be effectively implemented in the multi-controller storage system, and the storage cost is reduced. Specifically, the storage system includes K controllers, K is a positive integer not less than 2, and the storage management method of the multi-controller storage system can be applied to each controller. For any controller, after receiving a write request, in order to effectively exert the system performance and guarantee the load balancing, the write request is split into each sub-request, so that each sub-request is processed by the corresponding home controller. For any sub-request, whether the to-be-written data block corresponding to the sub-request belongs to the repeated data block already stored in the storage device can be determined by the home controller of the sub-request. It can be understood that if the sub-request does not belong to the repeated data block already stored in the storage device, the to-be-written data block corresponding to the sub-request needs to be written into the storage device, and the metadata information is updated based on the sub-request, otherwise, for the repeated data block, only the metadata information is updated based on the sub-request, and the repeated writing of the to-be-written data block is not needed. In addition, it can be seen that for any sub-request, whether the corresponding to-be-written data block belongs to the repeated data block is determined by the home controller of the sub-request, so the performance of the multi-controller storage system can be effectively exerted.

[0108] It can be seen that the scheme of the present application can implement the repeated data deletion in the multi-controller storage system, and the K controllers can be used to effectively exert the system performance and guarantee the load balancing.

[0109] Corresponding to the above method and system embodiments, the present application also provides an electronic device, a computer readable storage medium and a computer program product, which can be mutually corresponding and referred to above.

[0110] Referring to Figure 3 As shown in the figure, the device can include:

[0111] The memory 301 is configured to store a computer program.

[0112] The processor 302 is configured to execute the computer program to implement the steps of the storage management method in any of the above embodiments.

[0113] The computer program product includes computer programs / instructions, which are executed by the processor to implement the steps of the storage management method in any of the above embodiments.

[0114] Referring to Figure 4The computer readable storage medium 40 stores a computer program 41, which, when executed by the processor, implements the steps of the storage management method in any of the above embodiments. The computer readable storage medium 40 mentioned herein includes a RAM (Random Access Memory), a memory, a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a register, a hard disk, a removable disk, or any other form of storage medium known in the art.

[0115] Those skilled in the art will further appreciate that the individual steps of the example methods described in connection with the embodiments disclosed herein can be realized by electronic hardware, computer software, or any combination thereof. When the methods are realized in software, the software routine can be stored in one or more of the associated memories and executed by the appropriate processor unit so as to provide the described functionality. For clarity, the specific internal components of the processor unit and memories have not been described in detail since such components are well known. The specific components of the processor unit and memories that execute the routine steps described herein will be apparent to those skilled in the art from the description herein.

[0116] The above provides a storage management method, device, program product and medium of a multi-control storage system. The principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only applicable to help understand the method and core idea of the present application. It should be noted that those skilled in the art can make some improvements and modifications to the present application without departing from the principles of the present application. These improvements and modifications also fall within the protection scope of the present application.

Claims

1. A storage management method for a multi-controller storage system, characterized in that, The memory system includes K controllers, where K is a positive integer not less than 2. The memory management method for the multi-controller memory system is applied to each of the controllers, including: Receive a write request and break it down into sub-requests; For any sub-request, the controller that owns the sub-request determines whether the data block to be written corresponding to the sub-request belongs to a duplicate data block that has already been stored in the storage device; If not, the data block to be written is written to the storage device, and the metadata information is updated based on the sub-request; If so, then update the metadata information based on the sub-request; For any sub-request, the system uses the controller that owns the sub-request to determine whether the data block to be written corresponding to the sub-request belongs to a duplicate data block already stored in the storage device, including: For any sub-request, determine the controller to which the sub-request belongs; For any sub-request, when the controller to which the sub-request belongs is the source controller, determine whether the data block to be written corresponding to the sub-request is a duplicate data block; When the controller to which the sub-request belongs is a controller other than the source controller, the sub-request is sent to the controller to which the sub-request belongs, so that the controller to which the sub-request belongs can allocate resources for the sub-request, and determine whether the data block to be written corresponding to the sub-request belongs to a duplicate data block that has already been stored in the storage device; and receive the determination result fed back by the controller to which the sub-request belongs. The controller that receives the write request is the source controller of the write request; When the controller to which the sub-request belongs is the source controller, determining whether the data block to be written corresponding to the sub-request is a duplicate data block includes: By using the hash table managed by the source controller, it is determined whether the hash value of the data block to be written exists in the hash table managed by the source controller; If it exists, then the data block to be written corresponding to the sub-request is determined to be a duplicate data block; If it does not exist, then it is determined that the data block to be written corresponding to the sub-request does not belong to the duplicate data block; Accordingly, when the home controller of the sub-request is a controller other than the source controller, the sub-request is sent to the home controller of the sub-request so that the home controller of the sub-request can allocate resources for the sub-request, and determine whether the data block to be written corresponding to the sub-request belongs to a duplicate data block already stored in the storage device through the home controller of the sub-request, including: When the controller to which the sub-request belongs is a controller other than the source controller, the sub-request is sent to the controller to which the sub-request belongs, so that the controller to which the sub-request belongs can allocate resources for the sub-request, and the hash table managed by the controller to which the sub-request belongs can be used to determine whether the hash value of the data block to be written exists in the hash table managed by the controller to which the sub-request belongs. If it exists, then the data block to be written corresponding to the sub-request is determined to be a duplicate data block; If it does not exist, then it is determined that the data block to be written corresponding to the sub-request does not belong to the duplicate data block; Specifically, for any one data block already stored in the storage device, when the remainder obtained by dividing the hash value of the data block by K is a, the hash value of the data block is recorded in the hash table managed by the controller numbered a.

2. The storage management method for a multi-controller storage system according to claim 1, characterized in that, Receive a write request and break it down into sub-requests, including: Receive write requests and allocate resources for the write requests; According to the preset splitting rules, the write request is split into sub-requests, and resources are allocated to each sub-request.

3. The storage management method for a multi-controller storage system according to claim 2, characterized in that, According to the preset splitting rules, the write request is split into various sub-requests, including: The write request is divided into sub-requests according to a preset fixed granularity, so that the size of the data block to be written corresponding to each sub-request conforms to the fixed granularity.

4. The storage management method for a multi-controller storage system according to claim 1, characterized in that, For any sub-request, determining the attributing controller of the sub-request includes: For any sub-request, determine the host controller of the sub-request according to the sub-request number.

5. The storage management method for a multi-controller storage system according to claim 1, characterized in that, For any sub-request, determining the attributing controller of the sub-request includes: Perform a hash operation on the data block to be written corresponding to the sub-request to obtain the hash value of the data block to be written; Determine the remainder value 'a' obtained by dividing the hash value by K; The controller numbered 'a' out of the K controllers is designated as the owner controller of the sub-request. Where a is an integer, and 0≤a≤K-1.

6. The storage management method for a multi-controller storage system according to claim 1, characterized in that, Also includes: The controller that owns the sub-request determines whether the sub-request generates a derived sub-request. If so, a derived sub-request of the sub-request is generated, and the derived sub-request is sent to the controller corresponding to the derived sub-request through the controller to which the sub-request belongs, so that the derived sub-request can be processed by the controller corresponding to the derived sub-request, so as to maintain the address mapping relationship for the physical address originally corresponding to the sub-request. The conditions under which a sub-request has derived sub-requests include: the sub-request belongs to a sub-request representing a data update, and the physical address originally corresponding to the sub-request is not managed by the controller to which the sub-request belongs; The physical address originally corresponding to the sub-request is managed by the controller corresponding to the derived sub-request.

7. The storage management method for a multi-controller storage system according to claim 1, characterized in that, After splitting the write request into sub-requests, the process also includes: When the source controller of the write request fails, the resources occupied by the write request and each sub-request are released, and the processing flow of the write request is re-executed. The controller that receives the write request is the source controller of the write request.

8. The storage management method for a multi-controller storage system according to claim 7, characterized in that, Also includes: When a controller other than the source controller of the write request fails, determine whether each sub-request has been successfully executed. If so, confirm that the write request has been processed. If not, the resources occupied by the write request and each sub-request are released, and the processing flow of the write request is re-executed.

9. The storage management method for a multi-controller storage system according to claim 7, characterized in that, Re-execute the write request processing flow, including: The fault controller is disabled, and the write request processing flow is re-executed.

10. The storage management method for a multi-controller storage system according to any one of claims 1 to 8, characterized in that, Also includes: Receive read requests and allocate resources for the read requests; When the controller receiving the read request fails, the read request processing flow is re-executed after the failed controller is blocked. When the controller receiving the read request is not faulty, it reads the data content specified by the read request based on the metadata information and provides feedback, and releases the resources allocated to the read request.

11. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the storage management method of the multi-controller storage system as described in any one of claims 1 to 10 when executing the computer program.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the storage management method of the multi-controller storage system as described in any one of claims 1 to 10.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the storage management method of the multi-controller storage system as described in any one of claims 1 to 10.

Images