A storage space recovery method, product, device and storage medium

Abstract

The embodiments of the present application relate to the field of data storage technology, and more specifically, to a storage space recovery method, product, device, and storage medium, which are intended to achieve storage space recovery when a controller fails. The method comprises: in the event of a failed controller in a storage cluster, synchronizing the storage block information to be recovered corresponding to the failed controller to the remaining controllers in the storage cluster through a partner controller corresponding to the failed controller; adding the storage block information to be recovered to a linked list of storage blocks to be recovered; and recovering storage space for multiple storage blocks to be recovered in the linked list of storage blocks to be recovered.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of data storage technology, and more specifically, to a storage space recovery method, product, device, and storage medium. Background Art

[0002] During the operation of the all-flash storage system, it is necessary to allocate space for writing metadata to the controller of each storage device. In related technologies, space management is generally centralized by the main controller, and space is allocated to each controller based on the business situation of each controller. After the controller obtains the space, this part of the space is at the controller's disposal.

[0003] In the related art, when the space allocated to the controller is not fully used and the controller fails, metadata is not actually written and the space is not released. At this time, this part of the unused space is leaked, resulting in a waste of storage space. Summary of the invention

[0004] The embodiment of the present application provides a storage space recovery, aiming to realize storage space recovery when a controller fails.

[0005] A first aspect of an embodiment of the present application provides a storage space recovery method, the method comprising:

[0006] In the event of a faulty controller in the storage cluster, the to-be-reclaimed storage block information corresponding to the faulty controller is synchronized to the remaining controllers in the storage cluster through the partner controller corresponding to the faulty controller;

[0007] Adding the to-be-reclaimed storage block information to a to-be-reclaimed storage block linked list;

[0008] Storage space is reclaimed for multiple storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed.

[0009] Optionally, before synchronizing the to-be-reclaimed storage block information corresponding to the faulty controller to the remaining controllers in the storage cluster, the method further includes:

[0010] When the main controller allocates a storage block to each controller in the storage cluster, generating a record log corresponding to the controller;

[0011] The record log is stored in a partner controller corresponding to the controller.

[0012] Optionally, before storing the record log in a partner controller corresponding to the controller, the method further includes:

[0013] All the controllers in the storage cluster are paired into cache mirror pairs;

[0014] The two controllers in the cache mirroring pair serve as the partner controllers to each other.

[0015] Optionally, the method further comprises:

[0016] When the controller releases the storage block, sending a storage block release notification to the partner controller corresponding to the controller;

[0017] When the partner controller receives the storage block release notification, the storage block information corresponding to the storage block release notification is deleted from the record log.

[0018] Optionally, synchronizing the to-be-reclaimed storage block information corresponding to the faulty controller to other controllers in the storage cluster through a partner controller corresponding to the faulty controller includes:

[0019] Packing the record log including the information of the storage block to be reclaimed by the partner controller to obtain a log data packet;

[0020] The log data packet is sent to other controllers in the storage cluster through the partner controller.

[0021] Optionally, the step of packaging the record log including the information of the storage block to be reclaimed by the partner controller to obtain a log data packet includes:

[0022] Sorting the storage block information to be reclaimed according to the storage block numbers;

[0023] The record logs corresponding to the sorted storage block information to be reclaimed are packaged to obtain the log data packet.

[0024] Optionally, adding the information of the storage block to be reclaimed to a linked list of storage blocks to be reclaimed includes:

[0025] Parsing the information of the storage block to be recycled to obtain the storage block number corresponding to the storage block to be recycled;

[0026] According to the storage block number, the to-be-reclaimed storage block information is inserted into the to-be-reclaimed storage block linked list.

[0027] Optionally, inserting the to-be-reclaimed storage block information into the to-be-reclaimed storage block linked list according to the storage block number includes:

[0028] Traverse from left to right the first cursor corresponding to the to-be-reclaimed storage block information and the second cursor corresponding to the to-be-reclaimed storage block linked list;

[0029] When the storage block number corresponding to the first cursor is greater than the storage block number corresponding to the second cursor, the second cursor moves to the right;

[0030] When the storage block number corresponding to the first cursor is smaller than the storage block number corresponding to the second cursor, inserting the to-be-reclaimed storage block information corresponding to the storage block number corresponding to the first cursor into the to-be-reclaimed storage block linked list, and moving the first cursor to the next storage block number;

[0031] When the first cursor moves to the end, the insertion of the storage block information to be reclaimed is completed.

[0032] Optionally, reclaiming storage space of a plurality of storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed includes:

[0033] For each of the to-be-reclaimed storage blocks in the to-be-reclaimed storage block linked list, identifying valid storage space and invalid storage space in the to-be-reclaimed storage block;

[0034] The invalid storage space is reclaimed.

[0035] Optionally, the identifying valid storage space and invalid storage space in the storage block to be reclaimed includes:

[0036] Identify whether each metadata node in the storage block to be reclaimed is a valid metadata node;

[0037] In the case of determining that the metadata node is the valid metadata node, determining that the storage space corresponding to the metadata node is the valid storage space;

[0038] In the case where it is determined that the metadata node is an invalid metadata node, the storage space corresponding to the metadata node is determined to be an invalid storage space.

[0039] Optionally, the method for determining that the metadata node is a valid metadata node includes:

[0040] Performing format matching on the metadata in the metadata node;

[0041] When the format matching is successful, it is determined that the format corresponding to the metadata node is a valid format.

[0042] Optionally, the performing format matching on the metadata in the metadata node includes:

[0043] Comparing the format of the metadata in the metadata node with a preset metadata format;

[0044] In a case where the metadata format in the metadata node is the same as the preset metadata format, determining that the metadata match is successful;

[0045] In the case where the metadata format in the metadata node is different from the preset metadata format, it is determined that the metadata matching fails.

[0046] Optionally, the method further comprises:

[0047] When the metadata matches successfully, obtaining a key value from the metadata node;

[0048] A metadata query is performed according to the key value, and when it is found that the key value corresponds to the metadata node, the metadata node is determined to be a valid metadata node.

[0049] Optionally, before reclaiming storage space of a plurality of storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed, the method further includes:

[0050] Performing business pressure detection on the storage cluster;

[0051] When the pressure state of the storage cluster is detected, the pressure state is sent to the concurrent controller;

[0052] The concurrency of reclaiming the storage block to be reclaimed is determined by the concurrency controller.

[0053] Optionally, the performing business pressure detection on the storage cluster includes:

[0054] Determine the hard disk read and write times and bandwidth of each storage in the storage cluster;

[0055] The pressure state of the storage cluster is determined according to the hard disk read and write times and the bandwidth.

[0056] Optionally, determining the concurrency of reclaiming the storage block to be reclaimed by the concurrency controller includes:

[0057] When the pressure state of the storage cluster is low, the concurrency controller sets the concurrency to a first concurrency, where the first concurrency is the highest level of concurrency;

[0058] When the pressure state of the storage cluster is high pressure, the concurrency controller sets the concurrency to a second concurrency, where the second concurrency is the lowest level of concurrency.

[0059] Optionally, the method further comprises:

[0060] When the invalid storage space is completely reclaimed, the completion information of the reclaiming is synchronized to other controllers in the storage cluster.

[0061] Optionally, the method further comprises:

[0062] When it is detected that the faulty controller has returned to normal, the information of the storage block to be reclaimed is synchronized to the controller that has returned to normal.

[0063] A second aspect of an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps in the method described in the first aspect of the present application are implemented.

[0064] A third aspect of an embodiment of the present application provides a storage space recovery device, the device comprising:

[0065] A first information synchronization module is used to synchronize the to-be-reclaimed storage block information corresponding to the faulty controller to the remaining controllers in the storage cluster through the partner controller corresponding to the faulty controller when a faulty controller appears in the storage cluster;

[0066] An information insertion linked list module is used to add the to-be-reclaimed storage block information into the to-be-reclaimed storage block linked list;

[0067] The space recovery module is used to recover storage space of multiple storage blocks to be recovered in the storage block chain list to be recovered.

[0068] Optionally, the device further comprises:

[0069] A record log generation module, used for generating a record log corresponding to each controller in the storage cluster when the main controller allocates a storage block to the controller;

[0070] The record log backup module is used to store the record log in a partner controller corresponding to the controller.

[0071] Optionally, the device further comprises:

[0072] A mirror pair forming module, used for forming cache mirror pairs in pairs of all the controllers in the storage cluster;

[0073] The partner controller determination module is used to use the two controllers in the cache mirror pair as the partner controllers of each other.

[0074] Optionally, the device further comprises:

[0075] A release notification sending module, configured to send a storage block release notification to the partner controller corresponding to the controller when the controller releases the storage block;

[0076] The record log deletion module is used to delete the storage block information corresponding to the storage block release notification from the record log when the partner controller receives the storage block release notification.

[0077] Optionally, the information synchronization module includes:

[0078] A log packaging submodule, used for packaging the record log including the information of the storage block to be reclaimed through the partner controller to obtain a log data packet;

[0079] The data sending submodule is used to send the log data packet to the remaining controllers in the storage cluster through the partner controller.

[0080] Optionally, the log packaging submodule includes:

[0081] A storage block sorting submodule, used for sorting the storage block information to be recycled according to the storage block numbers;

[0082] The log data packet acquisition submodule is used to pack the record logs corresponding to the sorted storage block information to be recovered to obtain the log data packet.

[0083] Optionally, the information adding linked list module includes:

[0084] A storage block number determination submodule, used for parsing the information of the storage block to be recycled to obtain the storage block number corresponding to the storage block to be recycled;

[0085] The information insertion submodule is used to insert the to-be-reclaimed storage block information into the to-be-reclaimed storage block linked list according to the storage block number.

[0086] Optionally, the information insertion submodule includes:

[0087] A first traversal comparison submodule is used to traverse from left to right the first cursor corresponding to the storage block information to be reclaimed and the second cursor corresponding to the storage block linked list to be reclaimed;

[0088] A second traversal comparison submodule, configured to move the second cursor to the right when the storage block number corresponding to the first cursor is greater than the storage block number corresponding to the second cursor;

[0089] A storage block information insertion submodule, used for inserting the to-be-reclaimed storage block information corresponding to the storage block number corresponding to the first cursor into the to-be-reclaimed storage block linked list when the storage block number corresponding to the first cursor is smaller than the storage block number corresponding to the second cursor, and moving the first cursor to the next storage block number;

[0090] The insertion completion submodule is used to complete the insertion of the storage block information to be reclaimed when the first cursor moves to the end.

[0091] Optionally, the space recovery module includes:

[0092] A space identification submodule, for identifying, for each of the storage blocks to be reclaimed in the storage block chain list, valid storage space and invalid storage space in the storage block to be reclaimed;

[0093] The space recovery submodule is used to recover the invalid storage space.

[0094] Optionally, the space identification submodule includes:

[0095] A metadata node identification submodule, used to identify whether each metadata node in the storage block to be reclaimed is a valid metadata node;

[0096] An effective space determination submodule, used for determining that the storage space corresponding to the metadata node is an effective storage space when the metadata node is determined to be the effective metadata node;

[0097] The invalid storage space determination submodule is used to determine that the storage space corresponding to the metadata node is an invalid storage space when the metadata node is determined to be an invalid metadata node.

[0098] Optionally, the device further comprises:

[0099] A format matching submodule, used for performing format matching on the metadata in the metadata node;

[0100] The valid format determination submodule is used to determine that the format corresponding to the metadata node is a valid format when the format matching is successful.

[0101] Optionally, the format matching submodule includes:

[0102] A format comparison submodule, used to compare the format of the metadata in the metadata node with a preset metadata format;

[0103] A matching success submodule, used for determining that the metadata is successfully matched when the metadata format in the metadata node is the same as the preset metadata format;

[0104] The matching failure submodule is used to determine that the metadata matching fails when the metadata format in the metadata node is different from the preset metadata format.

[0105] Optionally, the device further comprises:

[0106] A key value acquisition submodule, used for acquiring a key value from the metadata node when the metadata is successfully matched;

[0107] The metadata query submodule is used to perform metadata query according to the key value, and determine that the metadata node is a valid metadata node when the key value is found to correspond to the metadata node.

[0108] Optionally, the device further comprises:

[0109] A pressure detection module, used to perform business pressure detection on the storage cluster;

[0110] A state data sending module, configured to send the pressure state to a concurrent controller when the pressure state of the storage cluster is detected;

[0111] The concurrency determination module is used to determine the concurrency of reclaiming the storage block to be reclaimed through the concurrency controller.

[0112] Optionally, the pressure detection module includes:

[0113] A data determination submodule, used to determine the hard disk read and write times and bandwidth of each storage in the storage cluster;

[0114] The pressure state determination submodule is used to determine the pressure state of the storage cluster according to the hard disk read and write times and the bandwidth.

[0115] Optionally, the concurrency determination module includes:

[0116] A first concurrency determination submodule, configured for the concurrency controller to set the concurrency to a first concurrency when the pressure state of the storage cluster is low pressure, wherein the first concurrency is the highest level of concurrency;

[0117] The second concurrency determination submodule is used for the concurrency controller to set the concurrency to a second concurrency when the pressure state of the storage cluster is high pressure, and the second concurrency is the lowest level of concurrency.

[0118] Optionally, the device further comprises:

[0119] The second information synchronization module is used to synchronize the completion information of the recovery to the remaining controllers in the storage cluster when the recovery of the invalid storage space is completed.

[0120] Optionally, the device further comprises:

[0121] The third information synchronization module synchronizes the information of the storage block to be recovered to the controller that has recovered when it is detected that the faulty controller has recovered.

[0122] A fourth aspect of an embodiment of the present application provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the method described in the first aspect of the present application are implemented.

[0123] A fifth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect of the present application are implemented.

[0124] By adopting the storage space recovery method proposed in the present application, when a faulty controller appears in a storage cluster, the to-be-reclaimed storage block information corresponding to the faulty controller is synchronized to the remaining controllers in the storage cluster through the partner controller corresponding to the faulty controller; the to-be-reclaimed storage block information is added to a to-be-reclaimed storage block linked list; and multiple to-be-reclaimed storage blocks in the to-be-reclaimed storage block linked list are recovered.

[0125] In this method, the storage block information of each controller is stored in the partner controller in advance. When a controller fails, the storage block information to be reclaimed is sent to the main controller and other controllers through the partner controller, so that the other controllers in the cluster can quickly synchronize the information of the storage blocks that need to be reclaimed, thereby quickly determining the scope of possible space leakage, setting a linked list of storage blocks to be reclaimed, inserting the storage block information to be reclaimed into the linked list of storage blocks to be reclaimed, waiting for subsequent recycling processing, and sequentially reclaiming the storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed by the main controller, thereby realizing rapid recovery of leaked empty storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0127] Figure 1 is a flow chart of a storage space recovery method proposed in an embodiment of the present application;

[0128] Figure 2 This is a schematic diagram of storage block allocation proposed in an embodiment of the present application;

[0129] Figure 3 This is a schematic diagram of data packaging proposed in an embodiment of the present application;

[0130] Figure 4 It is a schematic diagram of traversal recursion proposed in an embodiment of the present application;

[0131] Figure 5 This is a schematic diagram of storage space recovery proposed in an embodiment of the present application;

[0132] Figure 6 This is a schematic diagram of allocation information backup proposed in an embodiment of the present application;

[0133] Figure 7 is a schematic diagram of a storage space recovery device proposed in an embodiment of the present application;

[0134] Figure 8 It is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0135] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0136] refer to Figure 1 , Figure 1 1 is a flow chart of a storage space recovery method proposed in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0137] S11: When a faulty controller appears in the storage cluster, information of to-be-reclaimed storage blocks corresponding to the faulty controller is synchronized to other controllers in the storage cluster through a partner controller corresponding to the faulty controller.

[0138] In this embodiment, each storage device in the storage cluster corresponds to a storage controller for controlling the storage device to store and read data. A faulty controller is a storage controller that fails and cannot be used. A partner controller is a controller that synchronizes information with the storage controller. Each controller has one or more partner controllers, and the log information stored by the controller is synchronized to the partner controller. The storage block information to be recovered includes the storage block number, storage block location information, storage block capacity and other information. The main controller is the core controller in the storage cluster, which can unify the other controllers and allocate corresponding storage space to each controller. A storage block is a storage space of a certain size.

[0139] In this embodiment, reference Figure 2 , Figure 2 Schematic diagram of storage block allocation proposed in one embodiment of the present application. Figure 2 As shown in the figure, during the operation of the all-flash system, there will be multiple controllers working in parallel in the cluster, and each controller has metadata writing operations. In order to ensure the smooth completion of the controller writing operations, the main controller is usually required to pre-allocate space for each controller. The unit of space allocation is generally represented by a storage block. Each storage block represents a space of a fixed size. Each controller will prepare several storage blocks, and the storage blocks of each controller are different. When writing to the disk is required, each controller will write the metadata that needs to be written to the disk into its own storage block. Once a storage block is used up, the next storage block will be used. The main controller is not an independent controller, but a controller selected from all working controllers to temporarily undertake the work of storage block allocation. Once the controller fails, another controller will be selected from the remaining controllers as the main controller to allocate storage blocks.

[0140] In this embodiment, when a faulty controller is detected in a storage cluster in the background of the storage system, the log data corresponding to the faulty controller that was pre-stored in the partner controller corresponding to the faulty controller is packaged through the partner controller corresponding to the faulty controller, and the log data packet includes information about storage blocks to be recovered, and the log data packet is sent to the remaining controllers.

[0141] The specific steps of synchronizing the to-be-reclaimed storage block information corresponding to the faulty controller to the remaining controllers in the storage cluster through the partner controller corresponding to the faulty controller include:

[0142] S11 - 1 : Packing the record log including the information of the storage block to be reclaimed through the partner controller to obtain a log data packet.

[0143] In this embodiment, the log is a log of each controller during operation, and the log records information of a storage block corresponding to each controller. The log data packet is a data packet obtained by packaging the log information.

[0144] In this embodiment, after a faulty controller is detected in a cluster, the partner controller of the controller packages the log information corresponding to the faulty controller to obtain a log data packet. The specific steps include:

[0145] S11-1-1: Sort the information of the storage blocks to be reclaimed according to the storage block numbers.

[0146] In this embodiment, each storage block is assigned a corresponding storage block number when it is allocated, and the earlier the storage block is allocated, the smaller the storage block number.

[0147] In this embodiment, when the partner controller of the faulty controller learns that the controller has failed, it uses the storage block information in the controller as the storage block information to be recovered, and packages the log information corresponding to the faulty controller. When packaging, the storage block information is sorted according to the storage block number, and the storage block information can be sorted in order from small to large.

[0148] For example, the storage blocks to be reclaimed in the faulty controller are numbered 100, 101, and 102. When the partner controller packages the log information, it sorts them in the order of storage block 100, storage block 101, and storage block 102.

[0149] S11-1-2: Packing the record logs corresponding to the sorted storage block information to be reclaimed to obtain the log data packet.

[0150] In this embodiment, the sorted storage block information is packaged to obtain a log data packet.

[0151] refer to Figure 3 , Figure 3 is a schematic diagram of data packaging proposed in an embodiment of the present application, such as Figure 3 As shown, after the partner controller identifies the faulty controller, it packages the log information of the faulty controller, where the log information includes the storage block information of the faulty controller, thereby obtaining a log data packet.

[0152] For example, the log information is packaged into a MSG data packet, which is a text file usually used to store and transmit information.

[0153] S11 - 2: Send the log data packet to other controllers in the storage cluster through the partner controller.

[0154] In this embodiment, after obtaining the log data packet, the partner controller sends the log data packet to other controllers in the storage cluster.

[0155] In this embodiment, the log data packets are synchronized to each controller, so that no matter which controller is selected as the main controller in the future, it can be convenient to avoid those storage blocks waiting for recycling when allocating storage blocks, thereby preventing the problem of repeated allocation of storage blocks.

[0156] S12: Add the information of the storage block to be reclaimed into a linked list of storage blocks to be reclaimed.

[0157] In this embodiment, the linked list of storage blocks to be reclaimed is a linked list used to store information of storage blocks to be reclaimed, and the linked list stores information of storage blocks to be reclaimed in the entire cluster.

[0158] In this embodiment, after receiving the log data packet, the log data packet is parsed to obtain the storage block information of each storage block to be reclaimed, and the storage block information to be reclaimed is added to the storage block linked list to be reclaimed. When adding, the storage block numbers are sorted and inserted into the storage block linked list to be reclaimed in sequence.

[0159] In this embodiment, the specific steps of adding the to-be-reclaimed storage block information to the to-be-reclaimed storage block linked list include:

[0160] S12-1: Parse the information of the storage block to be reclaimed to obtain the storage block number corresponding to the storage block to be reclaimed.

[0161] In this embodiment, the main controller parses the storage block information to be recovered, obtains the storage block number corresponding to the storage block to be recovered from the storage block information to be recovered, the storage block information to be recovered is included in the log data packet, and the log data packet is parsed to obtain the storage block information to be recovered, the storage block information to be recovered includes the corresponding storage block number, because the storage block information to be recovered is sorted according to the storage block number in the log data, the multiple storage block numbers parsed out are also in sequence.

[0162] S12-2: inserting the to-be-reclaimed storage block information into the to-be-reclaimed storage block linked list according to the storage block number.

[0163] In this embodiment, after obtaining the storage block number of each storage block to be reclaimed, it is necessary to merge the storage block information to be reclaimed into the storage block linked list to be reclaimed. When merging, a traversal merge method is adopted to compare the storage block numbers corresponding to multiple storage block information to be reclaimed with the storage block numbers in the storage block linked list to be reclaimed, and insert the storage block information to be reclaimed into the storage block linked list to be reclaimed in the order of the storage block numbers.

[0164] In this embodiment, the specific steps of inserting the to-be-reclaimed storage block information into the to-be-reclaimed storage block linked list according to the storage block number include:

[0165] S12-2-1: Traverse from left to right the first cursor corresponding to the to-be-reclaimed storage block information and the second cursor corresponding to the to-be-reclaimed storage block linked list.

[0166] In this embodiment, the first cursor is a cursor in a queue composed of information of storage blocks to be reclaimed, and the second cursor is a cursor in a linked list of storage blocks to be reclaimed.

[0167] In this embodiment, reference Figure 4 , Figure 4 is a schematic diagram of a traversal recursion proposed in an embodiment of the present application, such as Figure 4 As shown, the queue corresponding to the storage block information to be recovered is arranged in order from small to large and from left to right according to the storage block numbers, and the storage block linked list is arranged in order from small to large and from left to right according to the storage block numbers. When the storage block information to be recovered is merged into the storage block linked list to be recovered, the first cursor corresponding to the storage block information to be recovered and the second cursor corresponding to the storage block linked list to be recovered are both traversed from left to right.

[0168] S12-2-2: When the storage block number corresponding to the first cursor is greater than the storage block number corresponding to the second cursor, the second cursor moves to the right.

[0169] In this embodiment, when the storage block information to be reclaimed is inserted into the storage block linked list to be reclaimed, when the storage block number corresponding to the first cursor is greater than the storage block number corresponding to the second cursor, the linked list in the storage block to be reclaimed moves rightward to point to the next storage block number.

[0170] S12-2-3: When the storage block number corresponding to the first cursor is smaller than the storage block number corresponding to the second cursor, the information of the storage block to be reclaimed corresponding to the storage block number corresponding to the first cursor is inserted into the linked list of storage blocks to be reclaimed, and the first cursor is moved to the next storage block number.

[0171] In this embodiment, when the storage block number corresponding to the first cursor is smaller than the storage block number corresponding to the second cursor, the corresponding insertion position is found, the to-be-reclaimed storage block information corresponding to the storage block number corresponding to the first cursor is inserted into the to-be-reclaimed storage block linked list, and the first cursor is moved to the next storage block number.

[0172] S12-2-4: When the first cursor moves to the end, the insertion of the storage block information to be reclaimed is completed.

[0173] In this embodiment, when the first cursor moves to the end of the queue of storage block information to be reclaimed, it means that all the storage block information to be reclaimed has been inserted into the corresponding position of the storage block linked list to be reclaimed, and the insertion of the storage block information to be reclaimed is completed. This process can actually be understood as a sorting process for two ordered series. In this way, it can be ensured that the merging process only needs to move in one direction, avoiding repeated searches for suitable positions and quickly completing the insertion of the storage block information to be reclaimed.

[0174] For example, the linked list of storage blocks to be reclaimed includes storage blocks 01, storage blocks 03, and storage blocks 10, and the queue of storage block information to be reclaimed includes storage blocks 02, storage blocks 08, and storage blocks 11. When inserting, first insert storage block 02 in front of storage block 03, insert storage block 08 in front of storage block 10, and insert storage block 11 after storage block 10. At this time, the linked list of storage blocks to be reclaimed is storage block 01, storage block 02, storage block 03, storage block 08, storage block 10, and storage block 11.

[0175] S13: reclaiming storage space of a plurality of storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed.

[0176] In this embodiment, after the information of the storage blocks to be reclaimed is inserted into the linked list of storage blocks to be reclaimed, space can be reclaimed for multiple storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed through the main controller, and space can also be reclaimed for multiple storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed through the partner controller or other controllers. It can be set according to actual conditions. When performing space recovery, recovery is performed in the order in the linked list of storage blocks to be reclaimed, and storage space recovery can be performed for multiple consecutive storage blocks to be reclaimed according to the previously set concurrency number. When performing storage space recovery, the data in each storage block to be reclaimed is traversed, and the space corresponding to the valid data is determined as the valid space, and the space corresponding to the invalid data is determined as the invalid storage space, and then the invalid storage space in the storage block to be reclaimed is reclaimed.

[0177] In this embodiment, reference Figure 5 , Figure 5 is a schematic diagram of storage space recovery proposed in an embodiment of the present application, such as Figure 5 As shown, firstly, the business pressure is detected by the business pressure detection unit, the concurrency is determined by the storage block recovery concurrency control unit, and then the invalid storage space in the storage block is recovered by the storage block recovery processing unit.

[0178] In this embodiment, the specific steps of reclaiming storage space of multiple storage blocks to be reclaimed in the storage block to be reclaimed linked list include:

[0179] S13-1: For each of the storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed, identifying valid storage space and invalid storage space in the storage block to be reclaimed.

[0180] In this embodiment, the valid storage space is a storage space in which the metadata stored in the storage block is valid data, and the invalid storage space is a storage space in which no data is stored or the stored metadata is erroneous.

[0181] In this embodiment, when reclaiming each storage block in the linked list of storage blocks to be reclaimed, for each storage block to be reclaimed, it is identified whether the metadata in each metadata node in the storage block to be reclaimed is valid metadata, and the storage space corresponding to the valid metadata is the valid storage space, and the storage space corresponding to the invalid metadata is the invalid storage space.

[0182] In this embodiment, the specific steps of identifying the valid storage space and the invalid storage space in the storage block to be reclaimed include:

[0183] S13-1-1: Identify whether each metadata node in the storage block to be reclaimed is a valid metadata node.

[0184] In this embodiment, a metadata node is the smallest unit for storing metadata, and a storage block includes a large number of metadata nodes.

[0185] In this embodiment, for space recovery of each storage block, the storage block to be recovered is traversed to identify whether the metadata node in the storage block is a valid node. The specific steps include:

[0186] S13-1-1-1: Perform format matching on the metadata in the metadata node.

[0187] In this embodiment, the metadata nodes are all stored in a formatted manner. The format of the metadata node can be queried in the corresponding storage space. According to the known metadata format, the metadata in the metadata node is format matched. The specific steps include:

[0188] S13-1-1-1-1: Compare the format of the metadata in the metadata node with the preset metadata format.

[0189] In this embodiment, the format of the metadata in the read metadata node is first compared with the preset metadata format.

[0190] S13-1-1-1-2: When the metadata format in the metadata node is the same as the preset metadata format, determine that the metadata match is successful.

[0191] In this embodiment, when the metadata format in the metadata node is the same as the preset metadata format, it means that the metadata format in the metadata node is the correct metadata format, and it is determined that the metadata match is successful.

[0192] S13-1-1-1-3: When the metadata format in the metadata node is different from the preset metadata format, determine that the metadata matching fails.

[0193] In this embodiment, when the metadata format in the metadata node is different from the preset metadata format, it is indicated that the metadata in the metadata node is erroneous data and its format is incorrect. At this time, please determine that the metadata is invalid metadata and determine that the metadata matching fails.

[0194] S13-1-1-2: When the format matching is successful, determine that the format corresponding to the metadata node is a valid format.

[0195] In this embodiment, when the format matching is successful, it means that the format of the metadata in the metadata node is a valid format, and the format corresponding to the metadata node is a valid format.

[0196] In this embodiment, the method further includes:

[0197] S13-1-1-3: When the metadata match is successful, obtain a key value from the metadata node.

[0198] In this embodiment, the metadata node contains multiple key-value pairs (KV). When the metadata is successfully matched, it is still necessary to determine whether the metadata is valid metadata. In this case, an arbitrary key value is obtained from the metadata node.

[0199] S13-1-1-4: Perform a metadata query based on the key value, and when it is found that the key value corresponds to the metadata node, determine that the metadata node is a valid metadata node.

[0200] In this embodiment, a metadata query is performed based on the key value. If the corresponding value queried based on the key in the key value is the same as the value in the metadata node, it means that the queried key value corresponds to the metadata node. In this case, the metadata node is determined to be a valid metadata node.

[0201] For example, find any key value in the metadata node where K=1, V=100, then search according to K=1 in the metadata information stored in the system, and find the corresponding V=100. The metadata node that stores the key value is the metadata node that was read. At this time, the node is determined to be a valid metadata node.

[0202] S13-1-2: When it is determined that the metadata node is the valid metadata node, determine that the storage space corresponding to the metadata node is the valid storage space.

[0203] In this embodiment, when the metadata node is a valid metadata node, it means that the metadata node has been stored in the storage block before the faulty controller fails, and thus this part of the metadata has been stored normally, and thus the storage space corresponding to the metadata node is determined to be a valid storage space.

[0204] S13-1-3: When it is determined that the metadata node is an invalid metadata node, determine that the storage space corresponding to the metadata node is an invalid storage space.

[0205] In this embodiment, when the metadata node is an invalid metadata node, it means that the metadata node is not stored normally when the faulty controller fails, and the stored metadata is incorrect. Therefore, it is determined that the storage space corresponding to the metadata node is an invalid storage space.

[0206] S13-2: reclaiming the invalid storage space.

[0207] In this embodiment, after the invalid storage space in the storage block to be reclaimed is determined, the invalid storage space is reclaimed.

[0208] In another embodiment of the present application, before synchronizing the to-be-reclaimed storage block information corresponding to the failed controller to the remaining controllers in the storage cluster, the method further includes:

[0209] S21: When the main controller allocates storage blocks to each controller in the storage cluster, a record log corresponding to the controller is generated.

[0210] In this embodiment, the recording log is a log for recording storage block information of all storage blocks allocated in the controller.

[0211] In this embodiment, when the main controller allocates storage blocks to each controller in the storage cluster, the storage blocks of each controller will be under the responsibility of the corresponding controller. At this time, other controllers will not be able to perceive the usage of the storage blocks and will assume that the storage blocks are already in use. Therefore, if the storage blocks pre-allocated to a controller fail before being used, the pre-allocated storage blocks will have space leakage. Therefore, in order to accurately and quickly identify storage blocks that may have space leakage, when pre-allocating storage blocks, a record log corresponding to the controller is synchronously generated and synchronously stored in other controllers, that is, the partner controller. When generating the record log, when a controller obtains the pre-allocated storage block, the log records the storage block information of the storage block, and synchronously sends information such as the storage block number to the partner controller for backup.

[0212] In this embodiment, reference Figure 6 , Figure 6 This is a schematic diagram of allocation information backup proposed in an embodiment of the present application, such as Figure 6 As shown, the master controller (master) allocates storage blocks to the controller (worker), and the storage block information of the controller is backed up to another controller. The two controllers are partner controllers to each other.

[0213] S22: Store the record log in a partner controller corresponding to the controller.

[0214] In this embodiment, after the record log is generated, the record log is stored in a partner controller corresponding to the controller.

[0215] In this embodiment, since multiple controllers in a cluster generally do not fail at the same time, similar risks still exist. Therefore, during the operation of the system, one or more log information can be generated and backed up to one or more controllers according to reliability requirements.

[0216] In this embodiment, in order to prevent the entire cluster from losing power, in addition to backing up the logs on the partner controller, the log information of each controller will be stored using a memory with power-off protection capability, thereby ensuring that even if a power outage occurs, the log information will not be lost after the power is restored. This ensures that storage blocks that may have space leaks will not lose records due to extreme failure conditions such as cluster power outages, resulting in the inability to subsequently recover them.

[0217] In another embodiment of the present application, before storing the record log in a partner controller corresponding to the controller, the method further includes:

[0218] S31: All the controllers in the storage cluster are grouped into cache mirror pairs.

[0219] In this embodiment, the mirror pair is two controllers that mirror each other. The data in the two controllers can back up each other, and when one controller fails, data will not be lost.

[0220] In this embodiment, all controllers in the storage cluster are firstly grouped into cache mirror pairs in pairs.

[0221] S32: The two controllers in the cache mirror pair serve as the partner controllers to each other.

[0222] In this embodiment, the two controllers in the cache mirror pair serve as partner controllers to each other. The two controllers, which serve as partner controllers, store each other's log data and record various data generated by each other in daily operation, including storage block information of the allocated storage blocks.

[0223] In another embodiment of the present application, the method further includes:

[0224] S41: When the controller releases the storage block, a storage block release notification is sent to the partner controller corresponding to the controller.

[0225] In this embodiment, when a controller releases a storage block, a storage block release notification is sent to a partner controller corresponding to the controller to ensure that the information stored in the partner controller is authentic information.

[0226] S42: When the partner controller receives the storage block release notification, the storage block information corresponding to the storage block release notification is deleted from the record log.

[0227] In this embodiment, when the partner controller receives the storage block release notification, the storage block information corresponding to the storage block release notification is deleted from the record log. At this time, the record log backed up in the partner controller is deleted.

[0228] In another embodiment of the present application, before reclaiming storage space of a plurality of storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed, the method further includes:

[0229] S51: Performing business pressure detection on the storage cluster.

[0230] In this embodiment, the recycling process must follow the principle of minimizing the impact on the host business. Therefore, the load of the current business in the storage cluster will be identified through business pressure detection. Generally, it will be identified based on the front-end IOPS (a measurement method used for performance testing of computer storage devices (such as hard disks (HDDs), solid-state drives (SSDs) or storage area networks (SANs)), which can be regarded as the number of reads and writes per second) and bandwidth conditions.

[0231] In this embodiment, the specific steps of performing service pressure detection on the storage cluster include:

[0232] S51 - 1 : Determine the hard disk read and write times and bandwidth of each storage in the storage cluster.

[0233] In this embodiment, the hard disk read and write times and bandwidth of each storage in the storage cluster are first determined.

[0234] S51 - 2 : Determine the pressure state of the storage cluster according to the hard disk read and write times and the bandwidth.

[0235] In this embodiment, the pressure state of the storage cluster is determined based on the hard disk read and write times and bandwidth. The pressure state can be divided into high pressure, medium pressure and low pressure states. When the hard disk read and write times and bandwidth exceed the preset high pressure threshold, the business pressure is determined to be high pressure. When the hard disk read and write times and bandwidth are higher than the preset medium pressure threshold and lower than the preset high pressure threshold, the business pressure is determined to be medium pressure. When the hard disk read and write times and bandwidth do not exceed the preset medium pressure threshold, the business pressure is determined to be low pressure.

[0236] S52: When the pressure state of the storage cluster is detected, the pressure state is sent to a concurrent controller.

[0237] In this embodiment, when the pressure state of the storage cluster is detected, the detection result, that is, the current pressure state, is sent to the concurrent controller, and the concurrent controller is used to determine how many storage blocks can be processed simultaneously.

[0238] S53: Determine the concurrency of reclaiming the storage block to be reclaimed through the concurrency controller.

[0239] In this embodiment, when receiving the business pressure status, the concurrency controller determines the concurrency of reclaiming the storage block according to the business pressure status. The specific steps include:

[0240] S53 - 1 : When the pressure state of the storage cluster is low, the concurrency controller sets the concurrency to a first concurrency, where the first concurrency is the highest level of concurrency.

[0241] In this embodiment, when the pressure state of the storage cluster is low, the concurrency controller sets the concurrency to the first concurrency, which is the highest level of concurrency, indicating that the number of storage blocks to be reclaimed that are processed in parallel is the largest.

[0242] S53 - 2 : When the pressure state of the storage cluster is high pressure, the concurrency controller sets the concurrency to a second concurrency, where the second concurrency is the lowest level of concurrency.

[0243] In this embodiment, when the pressure state of the storage cluster is high, the concurrency controller sets the concurrency to the second concurrency, which is the lowest level of concurrency. That is, in order not to affect the number of services at this time, the number of storage blocks to be reclaimed by the parallel processor is the minimum, which can be 1 or 0, depending on the situation.

[0244] In this embodiment, the selection of a specific threshold value can be set according to a specific platform, and is usually specifically defined according to the actual hardware configuration of the platform and some empirical values.

[0245] In another embodiment of the present application, the method further includes:

[0246] S61: When the invalid storage space is completely reclaimed, synchronization of the reclaim completion information to other controllers in the storage cluster.

[0247] In this embodiment, when the recovery of invalid storage space is completed, the information on the completion of the recovery is synchronized to the remaining controllers in the storage cluster through the main controller, ensuring that all controllers can know which space is available and that any controller can allocate all free space when acting as the main controller. During synchronization, the storage block number and the corresponding storage address are sent to each controller for synchronization.

[0248] In another embodiment of the present application, the method further includes:

[0249] S62: When it is detected that the faulty controller has returned to normal, the information of the storage block to be reclaimed is synchronized to the controller that has returned to normal.

[0250] In this embodiment, when it is detected that the failed controller has recovered, the information of the storage blocks to be reclaimed is synchronized to the controller. At this time, the controller that has recovered can also know which spaces have been recovered and can be allocated again.

[0251] The storage space recovery method provided in the above-mentioned embodiments of the present application performs metadata flushing to dynamically allocate space to solve the space leakage problem that may occur. By accurately locating and identifying the storage blocks that may leak, accurate inspection and recovery processing of the leaked space is achieved, thereby improving processing efficiency. Subsequently, information synchronization can be performed in each controller to ensure that when multiple controllers fail, the information of the storage blocks to be recovered will be synchronized in each controller, and the leaked storage space caused by the controller failure can be quickly recovered while ensuring that the cluster stability and host business are not affected.

[0252] Based on the same inventive concept, an embodiment of the present application provides a storage space recovery device. Figure 7 , Figure 7 FIG. 7 is a schematic diagram of a storage space recovery device 700 proposed in an embodiment of the present application. Figure 7 As shown, the device comprises:

[0253] The first information synchronization module 701 is used to synchronize the storage block information to be recovered corresponding to the faulty controller to the remaining controllers in the storage cluster through the partner controller corresponding to the faulty controller when a faulty controller appears in the storage cluster;

[0254] An information insertion linked list module 702 is used to add the to-be-reclaimed storage block information into a to-be-reclaimed storage block linked list;

[0255] The space recovery module 703 is used to recover storage space of multiple storage blocks to be recovered in the storage block to be recovered linked list.

[0256] Optionally, the device further comprises:

[0257] A record log generation module, used for generating a record log corresponding to each controller in the storage cluster when the main controller allocates a storage block to the controller;

[0258] The record log backup module is used to store the record log in a partner controller corresponding to the controller.

[0259] Optionally, the device further comprises:

[0260] A mirror pair forming module, used for forming cache mirror pairs in pairs of all the controllers in the storage cluster;

[0261] The partner controller determination module is used to use the two controllers in the cache mirror pair as the partner controllers of each other.

[0262] Optionally, the device further comprises:

[0263] A release notification sending module, configured to send a storage block release notification to the partner controller corresponding to the controller when the controller releases the storage block;

[0264] The record log deletion module is used to delete the storage block information corresponding to the storage block release notification from the record log when the partner controller receives the storage block release notification.

[0265] Optionally, the information synchronization module includes:

[0266] A log packaging submodule, used for packaging the record log including the information of the storage block to be reclaimed through the partner controller to obtain a log data packet;

[0267] The data sending submodule is used to send the log data packet to the remaining controllers in the storage cluster through the partner controller.

[0268] Optionally, the log packaging submodule includes:

[0269] A storage block sorting submodule, used for sorting the storage block information to be recycled according to the storage block numbers;

[0270] The log data packet acquisition submodule is used to pack the record logs corresponding to the sorted storage block information to be recovered to obtain the log data packet.

[0271] Optionally, the information adding linked list module includes:

[0272] A storage block number determination submodule, used for parsing the information of the storage block to be recycled to obtain the storage block number corresponding to the storage block to be recycled;

[0273] The information insertion submodule is used to insert the to-be-reclaimed storage block information into the to-be-reclaimed storage block linked list according to the storage block number.

[0274] Optionally, the information insertion submodule includes:

[0275] A first traversal comparison submodule is used to traverse from left to right the first cursor corresponding to the storage block information to be reclaimed and the second cursor corresponding to the storage block linked list to be reclaimed;

[0276] A second traversal comparison submodule, configured to move the second cursor to the right when the storage block number corresponding to the first cursor is greater than the storage block number corresponding to the second cursor;

[0277] A storage block information insertion submodule, used for inserting the to-be-reclaimed storage block information corresponding to the storage block number corresponding to the first cursor into the to-be-reclaimed storage block linked list when the storage block number corresponding to the first cursor is smaller than the storage block number corresponding to the second cursor, and moving the first cursor to the next storage block number;

[0278] The insertion completion submodule is used to complete the insertion of the storage block information to be reclaimed when the first cursor moves to the end.

[0279] Optionally, the space recovery module includes:

[0280] A space identification submodule, for identifying, for each of the storage blocks to be reclaimed in the storage block chain list, valid storage space and invalid storage space in the storage block to be reclaimed;

[0281] The space recovery submodule is used to recover the invalid storage space.

[0282] Optionally, the space identification submodule includes:

[0283] A metadata node identification submodule, used to identify whether each metadata node in the storage block to be reclaimed is a valid metadata node;

[0284] An effective space determination submodule, used for determining that the storage space corresponding to the metadata node is an effective storage space when the metadata node is determined to be the effective metadata node;

[0285] The invalid storage space determination submodule is used to determine that the storage space corresponding to the metadata node is an invalid storage space when the metadata node is determined to be an invalid metadata node.

[0286] Optionally, the device further comprises:

[0287] A format matching submodule, used for performing format matching on the metadata in the metadata node;

[0288] The valid format determination submodule is used to determine that the format corresponding to the metadata node is a valid format when the format matching is successful.

[0289] Optionally, the format matching submodule includes:

[0290] A format comparison submodule, used to compare the format of the metadata in the metadata node with a preset metadata format;

[0291] A matching success submodule, used for determining that the metadata is successfully matched when the metadata format in the metadata node is the same as the preset metadata format;

[0292] The matching failure submodule is used to determine that the metadata matching fails when the metadata format in the metadata node is different from the preset metadata format.

[0293] Optionally, the device further comprises:

[0294] A key value acquisition submodule, used for acquiring a key value from the metadata node when the metadata is successfully matched;

[0295] The metadata query submodule is used to perform metadata query according to the key value, and determine that the metadata node is a valid metadata node when the key value is found to correspond to the metadata node.

[0296] Optionally, the device further comprises:

[0297] A pressure detection module, used to perform business pressure detection on the storage cluster;

[0298] A state data sending module, configured to send the pressure state to a concurrent controller when the pressure state of the storage cluster is detected;

[0299] The concurrency determination module is used to determine the concurrency of reclaiming the storage block to be reclaimed through the concurrency controller.

[0300] Optionally, the pressure detection module includes:

[0301] A data determination submodule, used to determine the hard disk read and write times and bandwidth of each storage in the storage cluster;

[0302] The pressure state determination submodule is used to determine the pressure state of the storage cluster according to the hard disk read and write times and the bandwidth.

[0303] Optionally, the concurrency determination module includes:

[0304] A first concurrency determination submodule, configured for the concurrency controller to set the concurrency to a first concurrency when the pressure state of the storage cluster is low pressure, wherein the first concurrency is the highest level of concurrency;

[0305] The second concurrency determination submodule is used for the concurrency controller to set the concurrency to a second concurrency when the pressure state of the storage cluster is high pressure, and the second concurrency is the lowest level of concurrency.

[0306] Optionally, the device further comprises:

[0307] The second information synchronization module is used to synchronize the completion information of the recovery to the remaining controllers in the storage cluster when the recovery of the invalid storage space is completed.

[0308] Optionally, the device further comprises:

[0309] The third information synchronization module synchronizes the information of the storage block to be recovered to the controller that has recovered when it is detected that the faulty controller has recovered.

[0310] Based on the same inventive concept, another embodiment of the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method described in any of the above embodiments of the present application.

[0311] Based on the same inventive concept, another embodiment of the present application provides an electronic device, Figure 8 It is a schematic diagram of an electronic device 800 proposed in one embodiment of the present application, including a memory 802, a processor 801 and a computer program stored in the memory and executable on the processor, and when the processor executes, the steps in the dual-site data storage method described in any of the above embodiments of the present application are implemented.

[0312] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0313] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0314] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0315] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0316] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0317] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0318] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0319] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0320] The storage space recovery method, product, device and storage medium provided by the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A storage space recovery method, characterized in that: The method comprises: In the event of a faulty controller in the storage cluster, the to-be-reclaimed storage block information corresponding to the faulty controller is synchronized to the remaining controllers in the storage cluster through the partner controller corresponding to the faulty controller; Adding the to-be-reclaimed storage block information to a to-be-reclaimed storage block linked list; Reclaiming storage space of a plurality of the to-be-reclaimed storage blocks in the to-be-reclaimed storage block linked list; The step of adding the information of the storage block to be reclaimed to a linked list of storage blocks to be reclaimed includes: Parsing the information of the storage block to be recycled to obtain the storage block number corresponding to the storage block to be recycled; Inserting the to-be-reclaimed storage block information into the to-be-reclaimed storage block linked list according to the storage block number includes: Traverse from left to right the first cursor corresponding to the to-be-reclaimed storage block information and the second cursor corresponding to the to-be-reclaimed storage block linked list; When the storage block number corresponding to the first cursor is greater than the storage block number corresponding to the second cursor, the second cursor moves to the right; When the storage block number corresponding to the first cursor is smaller than the storage block number corresponding to the second cursor, inserting the to-be-reclaimed storage block information corresponding to the storage block number corresponding to the first cursor into the to-be-reclaimed storage block linked list, and moving the first cursor to the next storage block number; When the first cursor moves to the end, the insertion of the storage block information to be reclaimed is completed.

2. The storage space recovery method according to claim 1, characterized in that: Before synchronizing the to-be-reclaimed storage block information corresponding to the failed controller to the remaining controllers in the storage cluster, the method further includes: When the main controller allocates a storage block to each controller in the storage cluster, generating a record log corresponding to the controller; The record log is stored in a partner controller corresponding to the controller.

3. The storage space recovery method according to claim 2, characterized in that: Before storing the record log in a partner controller corresponding to the controller, the method further includes: All the controllers in the storage cluster are paired into cache mirror pairs; The two controllers in the cache mirroring pair serve as the partner controllers to each other.

4. The storage space recovery method according to claim 2, characterized in that: The method further comprises: When the controller releases the storage block, sending a storage block release notification to the partner controller corresponding to the controller; When the partner controller receives the storage block release notification, the storage block information corresponding to the storage block release notification is deleted from the record log.

5. The storage space recovery method according to claim 1, characterized in that: The step of synchronizing the to-be-reclaimed storage block information corresponding to the faulty controller to other controllers in the storage cluster through the partner controller corresponding to the faulty controller includes: Packing the record log including the information of the storage block to be reclaimed by the partner controller to obtain a log data packet; The log data packet is sent to other controllers in the storage cluster through the partner controller.

6. The storage space recovery method according to claim 5, characterized in that: The step of packaging the log including the information of the storage block to be reclaimed by the partner controller to obtain a log data packet includes: Sorting the storage block information to be reclaimed according to the storage block numbers; The record logs corresponding to the sorted storage block information to be reclaimed are packaged to obtain the log data packet.

7. The storage space recovery method according to claim 1, characterized in that: The reclaiming storage space of the plurality of storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed comprises: For each of the to-be-reclaimed storage blocks in the to-be-reclaimed storage block linked list, identifying valid storage space and invalid storage space in the to-be-reclaimed storage block; The invalid storage space is reclaimed.

8. The storage space recovery method according to claim 7, characterized in that: The identifying the valid storage space and the invalid storage space in the storage block to be reclaimed includes: Identify whether each metadata node in the storage block to be reclaimed is a valid metadata node; In the case of determining that the metadata node is the valid metadata node, determining that the storage space corresponding to the metadata node is the valid storage space; In the case where it is determined that the metadata node is an invalid metadata node, the storage space corresponding to the metadata node is determined to be an invalid storage space.

9. The storage space recovery method according to claim 8, characterized in that: The method for determining that the metadata node is a valid metadata node includes: Performing format matching on the metadata in the metadata node; When the format matching is successful, it is determined that the format corresponding to the metadata node is a valid format.

10. The storage space recovery method according to claim 9, characterized in that: The format matching of the metadata in the metadata node includes: Comparing the format of the metadata in the metadata node with a preset metadata format; In a case where the metadata format in the metadata node is the same as the preset metadata format, determining that the metadata match is successful; In the case where the metadata format in the metadata node is different from the preset metadata format, it is determined that the metadata matching fails.

11. The storage space recovery method according to claim 10, characterized in that: The method further comprises: When the metadata matches successfully, obtaining a key value from the metadata node; A metadata query is performed according to the key value, and when it is found that the key value corresponds to the metadata node, the metadata node is determined to be a valid metadata node.

12. The storage space recovery method according to claim 1, characterized in that: Before reclaiming storage space of a plurality of storage blocks to be reclaimed in the linked list of storage blocks to be reclaimed, the method further includes: Performing business pressure detection on the storage cluster; When the pressure state of the storage cluster is detected, the pressure state is sent to the concurrent controller; The concurrency of reclaiming the storage block to be reclaimed is determined by the concurrency controller.

13. The storage space recovery method according to claim 12, characterized in that: The performing business pressure detection on the storage cluster includes: Determine the hard disk read and write times and bandwidth of each storage in the storage cluster; The pressure state of the storage cluster is determined according to the hard disk read and write times and the bandwidth.

14. The storage space recovery method according to claim 12, characterized in that: Determining the concurrency of reclaiming the storage block to be reclaimed by the concurrency controller includes: When the pressure state of the storage cluster is low, the concurrency controller sets the concurrency to a first concurrency, where the first concurrency is the highest level of concurrency; When the pressure state of the storage cluster is high pressure, the concurrency controller sets the concurrency to a second concurrency, where the second concurrency is the lowest level of concurrency.

15. The storage space recovery method according to claim 7, characterized in that: The method further comprises: When the invalid storage space is completely reclaimed, the completion information of the reclaiming is synchronized to the remaining controllers in the storage cluster.

16. The storage space recovery method according to claim 1, characterized in that: The method further comprises: When it is detected that the faulty controller has returned to normal, the information of the storage block to be reclaimed is synchronized to the controller that has returned to normal.

17. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps in the method according to any one of claims 1 to 16 are implemented.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 16 are implemented.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 16 are implemented.

Images