Data reconstruction method, device, equipment, medium and product

Abstract

The invention relates to the technical field of data storage, in particular to a data reconstruction method, device, equipment, medium and product, and the method comprises the following steps: after monitoring that a disk is online, determining a redundant array group of independent disks of the disk according to the original information of the redundant array of independent disks of the disk; if the redundant array group of independent disks of the disk is the current redundant array group of independent disks, reading an input / output mapping table of the current redundant array group of independent disks; according to the input and output mapping table, determining a plurality of to-be-reconstructed stripe groups needing data reconstruction; according to the multiple to-be-reconstructed strip groups, data reconstruction is performed on the redundant array of independent disks, member disks are online after fault repair, the multiple to-be-reconstructed strip groups needing data reconstruction are determined only through the input and output information recorded in the input and output mapping table, reconstruction is performed based on the part of data, the reconstruction data size is reduced, and the reconstruction efficiency is improved. And the reconstruction efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a data reconstruction method, device, equipment, medium and product. Background Art

[0002] With the development of science and technology, we have entered the era of data explosion, and massive amounts of data need to be stored and read. Data has replaced computing and become the center of information computing. This has led to an increasing demand for storage technology, including capacity requirements, I / O (Input / Output) performance requirements, data security requirements, and scalability requirements. In order to solve related problems, RAID technology came into being.

[0003] RAID (redundant array of independent disks) technology has been widely used as a high-performance and high-reliability storage technology. RAID mainly uses data striping, mirroring and data verification technology to obtain high performance, high reliability, fault tolerance and scalability. According to the strategy and architecture of using or combining these three technologies, RAID can be divided into different levels to meet the needs of different scenarios. Among them, mirroring and data verification technology can improve data reliability.

[0004] When a member disk in the RAID system has a hardware link failure, it triggers the RAID system to enter a degraded state, and subsequent IO data will not be updated on the corresponding disk. The hardware link failure is subsequently restored, but there is no new IO data in the restored disk, so it cannot be directly added to the RAID group, which will cause stripe inconsistency problems. To add the restored disk to the RAID group, a full disk data reconstruction operation is required. This reconstruction process is very long and often takes several hours. During the reconstruction process, the RAID system is still in a degraded state, and its reliability and performance will be reduced.

[0005] Therefore, how to efficiently reconstruct data is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a data reconstruction method, device, equipment, medium and product, which do not require full disk data reconstruction, but only need to determine multiple stripe groups to be reconstructed that need data reconstruction through input and output information recorded in an input and output mapping table, so as to facilitate reconstruction based on this part of the data, thereby reducing the amount of reconstructed data, thereby improving the reconstruction efficiency, allowing independent disk redundant arrays to quickly enter a protected state, and also reducing the amount of data written to the disk during reconstruction to increase the service life of the disk.

[0007] In a first aspect, a data reconstruction method is provided, comprising: when a disk is detected to be online, determining an independent disk redundant array group of the disk according to original information of the independent disk redundant array of the disk; determining whether the independent disk redundant array group of the disk is a current independent disk redundant array group; if it is the current independent disk redundant array group, reading an input-output mapping table of the current independent disk redundant array group, wherein the input-output mapping table records input-output information of the disk after it goes offline, the input-output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that input-output information is written to a stripe group corresponding to the bit value; determining a plurality of stripe groups to be reconstructed that need to be reconstructed according to the input-output mapping table; and reconstructing data for the independent disk redundant array group after the disk is added according to the plurality of stripe groups to be reconstructed.

[0008] In a preferred example, the present invention can be further configured as: according to the input-output mapping table, a plurality of stripe groups to be reconstructed that need to perform data reconstruction are determined, including: determining a target mapping value identifier whose mapping value of the input-output mapping table is not 0; determining a target stripe group identifier according to a target offset bit of the target mapping value identifier, wherein the target offset bit is an offset bit with a bit value of 1; and determining a plurality of stripe groups to be reconstructed that need to perform data reconstruction according to the target stripe group identifier.

[0009] In a preferred example, the present invention can be further configured as follows: it also includes: when the abnormal disk goes offline, new input and output information is obtained; based on each position information of the new input and output information, the input and output position of each position information in the input and output mapping table is calculated, wherein the position information of the new input and output information is composed of the starting position information to the ending position information of the new input and output information, and the input and output position includes: a mapping value identifier and an offset bit identifier; the bit value of the input and output position of each position information of the new input and output information in the input and output mapping table is recorded as 1.

[0010] In a preferred example, the present invention can be further configured as: according to each position information of the new input and output information, calculating the input and output position of each position information in the input and output mapping table, including: using preset operations to calculate the mapping value identifier and the offset bit identifier of each position information of the new input and output information in turn; wherein the preset operations include: obtaining the stripe group identifier based on the position information of the new input and output information and the stripe group size; determining the mapping value identifier and the offset bit identifier based on the stripe group identifier and the second quantity.

[0011] In a preferred example, the present invention can be further configured as: according to each position information of the new input and output information, the input and output position of each position information in the input and output mapping table is calculated, including: according to the starting position information of the new input and output information, the starting input and output position of the starting position information in the input and output mapping table is calculated; according to the ending position information of the new input and output information, the ending input and output position of the ending position information in the input and output mapping table is calculated; all input and output positions between the starting input and output position and the ending input and output position are used as the input and output positions of the new input and output information.

[0012] In a preferred example, the present invention can be further configured as follows: it also includes: when the independent disk redundant array group is normal, marking the input-output mapping table as an invalid state; when the independent disk redundant array group is abnormal, marking the input-output mapping table as a valid state, and the valid state is used to indicate that the input-output information is recorded in the input-output mapping table.

[0013] In a preferred example, the present invention can be further configured as follows: further comprising: after reconstructing data on the independent redundant array of disks group after the disk is added according to the multiple stripe groups to be reconstructed, further comprising: clearing all bit values ​​in the input-output mapping table.

[0014] In a preferred example, the present invention can be further configured as follows: when it is detected that a disk is online, it also includes: determining whether the original information of the independent disk redundant array is stored in the disk; if the original information of the independent disk redundant array is stored in the disk, determining the independent disk redundant array group of the disk according to the original information of the independent disk redundant array of the disk; if the original information of the independent disk redundant array is not stored in the disk, performing full disk data reconstruction.

[0015] In a preferred example, the present invention may be further configured as follows: the data structure of the input-output mapping table is in the form of an array or a linked list.

[0016] In a preferred example, the present invention can be further configured as follows: according to the multiple stripe groups to be reconstructed, data is reconstructed for the independent redundant array of disks group after the disk is added, including: performing consistency check on the target stripe group to be reconstructed and the corresponding stripe group of the disk, wherein the target stripe group to be reconstructed is any stripe group among the multiple stripe groups to be reconstructed; if the verification passes, data reconstruction of the target stripe group to be reconstructed is not performed; if the verification fails, data reconstruction of the independent redundant array of disks group after the disk is added is performed according to the target stripe group to be reconstructed.

[0017] In a preferred example, the present invention can be further configured as follows: before reading the input-output mapping table of the current independent redundant array of disks group, it also includes: obtaining the capacity, input-output mapping granularity and memory capacity corresponding to the independent redundant array of disks group of the disk, the input-output mapping granularity representing the volume of each stripe group; determining the volume of the input-output mapping table according to the capacity and input-output mapping granularity corresponding to the independent redundant array of disks group; if the volume of the input-output mapping table is larger than the memory capacity, adjusting the capacity and input-output mapping granularity corresponding to the independent redundant array of disks group of the disk to obtain a new capacity and new input-output mapping granularity corresponding to the new redundant array of independent disks group, and confirming the new volume of the input-output mapping table until the volume of the input-output mapping table is no larger than the memory capacity; if the volume of the input-output mapping table is no larger than the memory capacity, setting the second number of input-output mapping tables according to the size of the input-output mapping table.

[0018] In a preferred example, the present invention can be further configured as: adjusting the capacity and input-output mapping granularity corresponding to the independent redundant array of disks group of the disk to obtain the capacity and new input-output mapping granularity corresponding to the new independent redundant array of disks group, including: determining capacity adjustment parameters and granularity adjustment parameters according to a mode selected by a user; adjusting the capacity corresponding to the independent redundant array of disks group of the disk according to the capacity adjustment parameters to obtain the capacity corresponding to the new independent redundant array of disks group; adjusting the input-output mapping granularity according to the granularity adjustment parameters to obtain the new input-output mapping granularity.

[0019] In a preferred example, the present invention can be further configured as follows: before determining the multiple stripe groups to be reconstructed that need to be reconstructed according to the input-output mapping table, it also includes: determining the total number of stripe groups of the multiple stripe groups to be reconstructed corresponding to the input-output mapping table; if the total number of stripe groups is greater than a preset total number threshold, performing full disk data reconstruction; if the total number of stripe groups is not greater than the preset total number threshold, executing the step of determining the multiple stripe groups to be reconstructed that need to be reconstructed according to the input-output mapping table.

[0020] In a preferred example, the present invention can be further configured as follows: after reconstructing the data of the independent disk redundant array group after the disk is added according to the multiple stripe groups to be reconstructed, it also includes: verifying the consistency of the reconstructed data of the disk and the data of the redundant space; if the consistency verification passes, confirming that the data reconstruction is successful; if the consistency verification fails, generating a prompt message.

[0021] In a preferred example, the present invention can be further configured as follows: it also includes: when the abnormal disk goes offline, determining a target disk with redundant space from the independent disk redundant array group where the abnormal disk is located; reconstructing the data of the abnormal disk according to the stripe group of the target disk, and storing it in the redundant space, and using the redundant space to process the received new input and output information.

[0022] In a preferred example, the present invention can be further configured as follows: it also includes: obtaining historical data reconstruction information, the data reconstruction information including: the total amount of historical stripe groups and the reconstruction time; constructing a relationship model of the total amount time based on the total amount of historical stripe groups and the reconstruction time; and determining a preset total amount threshold based on the relationship model of the total amount time.

[0023] In a second aspect, a data reconstruction device is provided, comprising: a first determination module, for determining the independent disk redundant array group of the disk according to the original information of the independent disk redundant array of the disk after monitoring that a disk is online; a judgment module, for determining whether the independent disk redundant array group of the disk is the current independent disk redundant array group; a reading module, for reading the input-output mapping table of the current independent disk redundant array group if it is the current independent disk redundant array group, wherein the input-output mapping table records the input-output information of the disk after it goes offline, the input-output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that input-output information is written to the stripe group corresponding to the bit value; a second determination module, for determining a plurality of to-be-reconstructed stripe groups that need to be reconstructed according to the input-output mapping table; a data reconstruction module, for reconstructing the data of the independent disk redundant array group after the disk is added according to the plurality of to-be-reconstructed stripe groups.

[0024] According to a third aspect, an electronic device is provided, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the method as described in any one of the first aspects.

[0025] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in any one of the first aspects is implemented.

[0026] In a fifth aspect, a computer program product is provided, comprising a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the method described in any one of the first aspects is implemented.

[0027] In summary, the data reconstruction method provided by the present invention includes the following beneficial technical effects: when a member disk fails and goes offline, subsequent input and output information can be recorded through the input-output mapping table; when the disk is monitored to be online, if the independent disk redundant array group determined by the original information of the independent disk redundant array of the disk is valid, it means that the member disk is online after the failure is repaired. At this time, there is no need to reconstruct the entire disk data, and it is only necessary to determine the multiple stripe groups to be reconstructed that need to be reconstructed through the input and output information recorded in the input-output mapping table, so as to facilitate reconstruction based on the partial data, thereby reducing the amount of reconstructed data, thereby improving the reconstruction efficiency, allowing the independent disk redundant array to quickly enter a protected state, and also reducing the amount of data written to the disk during reconstruction to increase the service life of the disk.

[0028] In addition, the present invention also provides a data reconstruction device, equipment, medium and product, all of which have the above-mentioned beneficial technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 A flowchart of a data reconstruction method provided by an embodiment of the present invention.

[0031] Figure 2 It is a schematic diagram of an input-output mapping table provided in an embodiment of the present invention.

[0032] Figure 3 It is a schematic diagram of a flow chart of input and output information recording provided by an embodiment of the present invention.

[0033] Figure 4 It is a schematic diagram of a data reconstruction process provided by an embodiment of the present invention.

[0034] Figure 5 It is a structural schematic diagram of a data reconstruction device provided by an embodiment of the present invention.

[0035] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0038] In order to facilitate understanding, some professional terms are now explained.

[0039] CPU cache: A high-speed cache device between the memory and the CPU, used to resolve the speed gap between the CPU and the memory.

[0040] SOC: The abbreviation of System on Chip, also known as system-level chip or system on chip, means that it is a product, an integrated circuit with a dedicated purpose, which contains all the contents of a complete system and embedded software.

[0041] RAID: Redundant Array of Independent Disks.

[0042] IO: abbreviation of Input Output, input and output.

[0043] DW: Abbreviation of Double Word, indicating that the data bit width is 32 bits.

[0044] SLBA: Start Logical Block Address, starting logical block address.

[0045] LBA: Logical Block Address, logical block address.

[0046] NLB: Number of Logical Blocks, the number of blocks for reading data.

[0047] Chunk: An equal number of blocks cut out of a disk in RAID.

[0048] stripe:RAID stripe.

[0049] In the related art, when a hardware link failure occurs in a member disk of a RAID system, the RAID system will be triggered to enter a degraded state, and subsequent IO data will not be updated on the corresponding disk. The hardware link failure is subsequently restored, but there is no new IO data in the restored disk, so it cannot be directly added to the RAID group, which will cause stripe inconsistency problems. To add the restored disk to the RAID group, a full disk data reconstruction operation is required. This reconstruction process is very long, often taking several hours, and during the reconstruction process, the RAID system is still in a degraded state, and its reliability and performance will be reduced.

[0050] In view of the above problems, the present invention proposes a method that can record subsequent IO information when a disk fails. When the disk is restored, it is not necessary to reconstruct the entire disk data, but only partial data. The reduction in the amount of reconstructed data can reduce the reconstruction time, allow the RAID to quickly enter a protected state, and reduce the amount of data written to the disk during reconstruction to increase the service life of the disk.

[0051] Specifically, an embodiment of the present invention provides a data reconstruction method, such as Figure 1 As shown, the method provided in the embodiment of the present invention can be executed by an electronic device, and the electronic device is a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this. The terminal device and the electronic device can be directly or indirectly connected through wired or wireless communication, and the embodiment of the present invention is not limited here. The method includes:

[0052] S101. When it is detected that a disk is online, a redundant array of independent disks group of the disk is determined according to original redundant array of independent disks information of the disk.

[0053] The original RAID information of the disk refers to the information related to the RAID configuration stored on the disk, including the RAID group to which the disk belongs, the RAID level, the stripe size, the role of the disk in the RAID group (such as data disk, check disk, etc.), etc. A RAID group is a logical storage unit composed of multiple disks, and the disks are configured according to a specific RAID level.

[0054] It is understandable that in the embodiment of the present invention, when a disk in the raid group is offline, the raid group level remains unchanged and can be used normally. For example, if 5 disks constitute raid5, when one disk is offline, it is still raid5 and can be used normally; if 2 disks are offline, raid5 fails and cannot be used, and the solution of the embodiment of the present invention is no longer applicable.

[0055] In one practicable manner, the disk interface is scanned periodically to detect whether a new disk is online; or, through an event trigger mechanism, an event notification is received when a new disk is online. When a new disk is detected to be added to the system and is in an available state, the original RAID information of the disk is obtained, and the RAID group of the disk is determined based on the original RAID information of the disk.

[0056] S102: Determine whether the redundant array of independent disks group of the disk is the current redundant array of independent disks group.

[0057] The current RAID group refers to the RAID group currently in use. In the RAID management system, when an operation is required on a disk, it is first necessary to determine whether the disk belongs to the current RAID group in order to determine the data processing mechanism (whole disk reconstruction or specific data stripe reconstruction based on the IO mapping table).

[0058] S103. If it is the current redundant array of independent disks group, read the input-output mapping table of the current redundant array of independent disks group, wherein the input-output mapping table records the input-output information after the disk is offline, the input-output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that there is input-output information written into the stripe group corresponding to the bit value.

[0059] If the RAID group of the disk is the current RAID group, it means that the link failure may be repaired and the disk is back online. At this time, read the IO mapping table corresponding to the current RAID group, and perform a quick reconstruction operation according to the IO mapping table corresponding to the RAID group. If the RAID group of the disk is not the current RAID group, perform a full disk reconstruction operation.

[0060] The IO mapping table is further described. The IO mapping table records the IO information processed by the RAID group after the disk goes offline. The mapping table is composed of a first number n of mapping values, each of which is composed of a second number of bit values. When the bit value is 1, it indicates that input and output information is written to the stripe group corresponding to the bit value. For example, see Figure 2 , Figure 2 It is a schematic diagram of an IO mapping table provided by an embodiment of the present invention. The embodiment of the present invention proposes an IO mapping table for recording historical IO information, the mapping table is composed of n mapping values, the mapping value is a 32-bit number, initialized to all 0, one bit corresponds to a stripe group on the RAID, a stripe group is composed of N stripes, the bit value is 1 indicates that there is IO falling on the corresponding stripe group, otherwise it indicates that there is no IO on this stripe group.

[0061] The IO mapping table is stored in the internal storage of the RAID chip. The size of the stripe group can be configured according to the size of the internal storage space of the chip. The size of the stripe group must be an integer multiple of the stripe size. The larger the stripe group, the smaller the IO mapping table, the coarser the corresponding marked IO record, and the more data that needs to be reconstructed after the link of the subsequent disk is restored.

[0062] S104: Determine, according to the input-output mapping table, a plurality of stripe groups to be reconstructed that require data reconstruction.

[0063] Based on all the data whose bit values ​​are 1 in the IO mapping table, a plurality of stripe groups to be reconstructed that need to be reconstructed are determined.

[0064] S105 , reconstructing data on the independent redundant array of disks group after the disks are added according to the multiple stripe groups to be reconstructed.

[0065] The specific data reconstruction method is not limited in the embodiment of the present invention, as long as it can achieve the purpose of the embodiment of the present invention.

[0066] It can be seen that in the embodiment of the present invention, when a member disk fails and goes offline, subsequent input and output information can be recorded through the input-output mapping table; when the disk is monitored to be online, if the independent disk redundant array group determined by the original information of the independent disk redundant array of the disk is valid, it means that the member disk is online after the failure is repaired. At this time, there is no need to reconstruct the entire disk data. It is only necessary to determine the multiple stripe groups to be reconstructed that need to be reconstructed through the input and output information recorded in the input-output mapping table, so as to facilitate reconstruction based on this part of the data, thereby reducing the amount of reconstructed data, thereby improving the reconstruction efficiency, allowing the independent disk redundant array to quickly enter a protected state, and also reducing the amount of data written to the disk during reconstruction to increase the service life of the disk.

[0067] A possible implementation method of an embodiment of the present invention, S104 determines multiple stripe groups to be reconstructed that need to perform data reconstruction based on the input-output mapping table, including: determining a target mapping value identifier whose mapping value of the input-output mapping table is not 0; determining a target stripe group identifier based on a target offset bit of the target mapping value identifier, the target offset bit being an offset bit with a bit value of 1; determining multiple stripe groups to be reconstructed that need to perform data reconstruction based on the target stripe group identifier.

[0068] In an embodiment of the present invention, the mapping table index is initialized, and the mapping value identifier item_id=0 is selected; the mapping value of the mapping value identifier item_id=0 in the mapping table is obtained, if the mapping value is not 0, then each bit in the mapping value is judged in turn whether it is 1, otherwise the item_id is increased by 1; each bit in the mapping value is judged in turn whether it is 1, if it is 1, the corresponding stripe group is calculated, and a reconstruction operation is performed, otherwise the item_id is increased by 1, the corresponding stripe group is calculated, and a reconstruction operation is performed; if all data in the mapping table are processed, it ends, otherwise repeat the above steps.

[0069] Among them, taking 32 bits as an example, A1, the mapping value identifier is set to item_id=0; A2, determine the mapping value table[item_id] ! =0? If yes, execute A3, if not, execute A10; A3, set the variable item_val == table[item_id]; A4, the offset bit identifier bit_offset=0; A5, determine (item_val >> bit_offset) & 0x1 == 1? If yes, execute A6, if not, execute A8; A6, the stripe group identifier group_id=the mapping value identifier item_id 32+bit_offset; A7, reconstruct the data of the corresponding stripe group to be reconstructed; A8, bit_offset++; A9, determine whether the offset bit flag bit_offset>=32? If yes, execute A10, if not, execute A5; A10, item_id++; A11, determine whether item_id>=table_capacity? If yes, end, if not, execute A2.

[0070] It can be seen that in the embodiment of the present invention, each mapping value is a binary array. Therefore, whether the corresponding mapping value identifier has a bit of IO information change can be determined based on whether the binary array is 0, and then the target stripe group identifier is determined based on the target offset bit identifier of the target mapping value identifier, that is, the number of bits; thereby, multiple stripe groups with reconstruction can be quickly read and located.

[0071] A possible implementation of an embodiment of the present invention is a data reconstruction method, which also includes: when an abnormal disk goes offline, obtaining new input and output information; calculating the input and output position of each position information in the input and output mapping table based on each position information of the new input and output information, wherein the position information of the new input and output information is composed of the starting position information to the ending position information of the new input and output information, and the input and output position includes: a mapping value identifier and an offset bit identifier; and recording the bit value of the input and output position of each position information of the new input and output information in the input and output mapping table as 1.

[0072] Among them, in an embodiment of the present invention, a new IO information writing mechanism is provided. Specifically, when a disk needs to be offline due to failure, aging or other reasons, the new IO (input and output) information generated after the disk is offline is obtained. The new IO information includes the read and write position, size, timestamp, etc. of the data. The read and write position is the IO position information, which is composed of multiple position information between the starting position information and the ending position information. The starting position information indicates the starting storage position of the data in the new IO information, and the ending position information indicates the ending storage position of the data in the new IO information.

[0073] For newly acquired IO information, the system needs to calculate its input and output positions in the mapping table according to each position information. An embodiment of the present invention provides an IO mapping table; according to each position information of the new input and output information, the input and output positions of each position information in the mapping table are calculated to obtain a mapping value identifier and an offset bit identifier, wherein the mapping value identifier points to the index of an entry in the mapping table, and the offset bit identifier indicates the specific offset position in the entry; the bit value of the input and output position of each position information of the new IO information in the mapping table is recorded as 1.

[0074] It can be seen that in the embodiment of the present invention, after the abnormal disk goes offline, the mapping value identifier and the offset bit identifier in the mapping table can be determined based on each position information of the new input and output information, and the bit value of the input and output position of each position information of the new IO information in the mapping table is recorded as 1, so as to accurately locate the stripe group updated by the position information.

[0075] A possible implementation method of an embodiment of the present invention is to calculate the input and output position of each position information in the input and output mapping table according to each position information of the new input and output information, including: using preset operations to calculate the mapping value identifier and the offset bit identifier of each position information of the new input and output information in turn; wherein the preset operations include: obtaining the stripe group identifier according to the position information of the new input and output information and the stripe group size; determining the mapping value identifier and the offset bit identifier according to the stripe group identifier and the second quantity.

[0076] The stripe group size indicates the capacity of the stripe group in the storage system. The stripe group identifier is used to indicate a unique identifier of a specific stripe group. The second quantity refers to the number of binary bits of each mapping value, which is used to subsequently determine the mapping value identifier and the offset bit identifier.

[0077] In the embodiment of the present invention, the starting position information (slba) of the IO information is recorded and stored in the variable lba; the stripe group on which the starting position information falls is calculated, that is, the stripe group identifier, and stored in the variable, where group_id=lba / group_size, and an integer is taken to obtain the stripe group identifier, wherein, when the stripe group identifier=1, it indicates that it is the second stripe group; the mapping value identifier corresponding to the stripe group is calculated, each bit in the mapping table represents a stripe group, and each mapping value is 32 bits, so it needs to be divided by 32 and stored in the variable mapping value identifier item_id=group_id / 32, and an integer is taken to obtain the mapping value identifier, wherein, when the mapping value identifier=1, it indicates that it is the second mapping value; the bit offset of the stripe group in the mapping value is calculated and stored in the variable bit_offset as_offset=group_id % 32; the IO position is marked by a bitwise OR method; lba is incremented by 1; if lba is greater than or equal to the IO end position information, it ends, otherwise repeat the above steps.

[0078] It can be seen that in the embodiment of the present invention, according to the target location information of the IO information and the stripe group size, it is accurately determined in which stripe group the data should be stored, and the specific location of the data in the mapping table is further determined by the stripe group identifier and the second quantity.

[0079] In order to improve the efficiency of determining the IO position, a possible implementation method of an embodiment of the present invention calculates the input-output position of each position information in the input-output mapping table according to each position information of the new input-output information, including: calculating the starting input-output position of the starting position information in the input-output mapping table according to the starting position information of the new input-output information; calculating the ending input-output position of the ending position information in the input-output mapping table according to the ending position information of the new input-output information; and taking all input-output positions between the starting input-output position and the ending input-output position as the input-output positions of the new input-output information.

[0080] Among them, the method of calculating the IO position includes: obtaining the stripe group identifier according to the position information of the IO information and the stripe group size; determining the mapping value identifier and the offset bit identifier according to the stripe group identifier and the second quantity. After obtaining all the IO positions of the input and output positions of the starting position information in the mapping table and the input and output positions of the ending position information in the mapping table, all the IO positions between the two are regarded as the IO positions to be marked as new IO information. In this way, all IO positions can be determined by calculating only the first and last two IO positions, which greatly improves the calculation efficiency.

[0081] A possible implementation of the embodiment of the present invention also includes: when the independent redundant array of disks is normal, marking the input-output mapping table as invalid; when the independent redundant array of disks is abnormal, marking the input-output mapping table as valid, and the valid state is used to indicate that the input-output information is recorded in the input-output mapping table; the IO mapping table is marked as invalid when the RAID group is normal, and the RAID group will not record IO information when processing IO. When a disk link abnormality event occurs in the system, the IO mapping table is set to be valid, and subsequent IO will be recorded in the mapping table. The process is as follows Figure 3 shown.

[0082] Specifically, the determination of the RAID group status may be achieved by setting a flag in the memory, modifying the mapping table header information, or updating the system configuration file. The specific means are not limited in the embodiment of the present invention, and the user may set it according to actual needs.

[0083] A possible implementation of the embodiment of the present invention further includes: clearing all bit values ​​in the input-output mapping table after reconstructing data on the independent disk redundant array group after the disk is added according to the multiple stripe groups to be reconstructed. Figure 4 , Figure 4 It is a flow chart of data reconstruction provided by an embodiment of the present invention; an independent disk redundant array card detects a newly inserted disk, obtains the original data of the independent disk redundant array group in the disk, and when the original data is valid and the corresponding input-output mapping table in the memory is valid, the disk data is quickly reconstructed through the input-output mapping table, and after completion, the input-output mapping table is set invalid, and all values ​​in the input-output mapping package are cleared.

[0084] Among them, the reconstruction progress is monitored in real time. When the RAID group data reconstruction is completed, all positions in the IO mapping table are cleared and set to 0, so that when the IO mapping table is valid again, the interference of the original data in the IO mapping table can further reduce the occurrence of reconstruction anomalies caused by erroneous mapping.

[0085] A possible implementation method of an embodiment of the present invention, when it is monitored that a disk is online, also includes: determining whether the disk stores the original information of the independent disk redundant array; if the disk stores the original information of the independent disk redundant array, determining the independent disk redundant array group of the disk according to the original information of the independent disk redundant array of the disk; if the disk does not store the original information of the independent disk redundant array, performing full disk data reconstruction.

[0086] In one possible situation, the newly online disk may not be the original failed disk, but a new disk, in which no RAID edge information is stored. In this case, full disk data reconstruction can be performed so that the RAID group can be used normally.

[0087] In a possible implementation manner of the embodiment of the present invention, the data structure of the input-output mapping table is in the form of an array or a linked list.

[0088] In the embodiment of the present invention, when a link failure occurs in a member disk, subsequent IO information can be recorded and recorded in the IO mapping table. The structure of the mapping table is not limited to the array form, and can also be other data structures such as a linked list; the mapping value can also be 16 bits, 32 bits, 64 bits, etc. according to actual conditions, not limited to 32 bits, and users can choose according to actual needs.

[0089] A possible implementation of an embodiment of the present invention is to reconstruct data of an independent redundant array of disks group after adding a disk according to multiple stripe groups to be reconstructed, including: performing consistency check on a target stripe group to be reconstructed and a corresponding stripe group of the disk, wherein the target stripe group to be reconstructed is any stripe group among the multiple stripe groups to be reconstructed; if the verification passes, data reconstruction of the target stripe group to be reconstructed is not performed; if the verification fails, data reconstruction of the independent redundant array of disks group after adding the disk is performed according to the target stripe group to be reconstructed.

[0090] In an embodiment of the present invention, before data reconstruction is performed on each stripe group to be reconstructed, a consistency check can be performed to determine whether the data of the stripe group before disk reconstruction is consistent with the data of the stripe group to be reconstructed. If they are consistent, it means that the stripe group has not performed IO operations; if they are inconsistent, it means that IO operations have been performed and data reconstruction is required. Through the above mechanism, only the stripe groups that really need to be reconstructed will be processed, thereby avoiding unnecessary reconstruction operations and improving the efficiency of data reconstruction.

[0091] The embodiment of the present invention further elaborates on the IO mapping table, and the IO mapping table structure positioning includes: is_valid, grain_size, table_size and map_table, where: is_valid is a bool type variable, true indicates that the IO mapping table is valid, and false indicates invalid; grain_size is a 32-bit unsigned integer variable, indicating the stripe group size, that is, the mapping granularity size, and the unit is stripe; table_size is an n-bit unsigned integer variable, indicating the size of the mapping table; map_table is a 32-bit unsigned integer array, one bit of the array element corresponds to a stripe group on the RAID, and a bit of 1 indicates that the IO falls on the corresponding stripe group, otherwise it indicates that there is no IO information on this stripe group.

[0092] The IO mapping table is stored in the RAID card chip. Since the mapping table is accessed on the IO path and needs to be accessed frequently and at high speed, the IO mapping table needs to have a faster read and write speed. It is recommended to store it in a cached memory. Each RAID group needs to have an IO mapping table corresponding to it. The size of the IO mapping table depends on the RAID size and the input and output mapping granularity setting. For example, if 5 1TB disks form a RAID5 with a Chunk size of 256KB, the size of each stripe is 1MB, and the size of the RAID group is 4TB. If the input and output mapping granularity is set to 8 stripes, the size of the IO mapping table is 64KB. You can select the appropriate mapping granularity based on actual needs and memory limitations.

[0093] Based on this, in a possible implementation method of an embodiment of the present invention, before reading the input-output mapping table of the current independent redundant array of disks group, it also includes: obtaining the capacity, input-output mapping granularity and memory capacity corresponding to the independent redundant array of disks group of the disk, the input-output mapping granularity represents the volume of each stripe group; determining the volume of the input-output mapping table according to the capacity and input-output mapping granularity corresponding to the independent redundant array of disks group; if the volume of the input-output mapping table is larger than the memory capacity, adjusting the capacity and input-output mapping granularity corresponding to the independent redundant array of disks group of the disk to obtain a new capacity and new input-output mapping granularity corresponding to the new redundant array of independent disks group, and confirming the new volume of the input-output mapping table until the volume of the input-output mapping table is no larger than the memory capacity; if the volume of the input-output mapping table is no larger than the memory capacity, setting the second number of input-output mapping tables according to the size of the input-output mapping table.

[0094] Among them, in an embodiment of the present invention, in one possible situation, the capacity corresponding to the redundant array of independent disks group of the disk, that is, the RAID group capacity, is the total storage capacity of all disks in the RAID group, minus the part used for redundancy and metadata storage, that is, if RAID5 includes 5 1Tb member disks, then the RAID group capacity is 4Tb; in another possible situation, the RAID group capacity can be specified by the user and is smaller than the theoretical RAID group capacity, such as 3Tb.

[0095] The input / output mapping granularity, or IO mapping granularity, indicates the size of each stripe group, that is, the minimum distribution unit of data in the RAID group. The memory capacity indicates the memory capacity used to store metadata such as the IO mapping table.

[0096] Determine the size of the input / output mapping table based on the capacity of the redundant array of independent disks and the input / output mapping granularity. For example, if five 1TB disks form a RAID5 with a chunk size of 256KB, then the size of each stripe is 1MB. If the RAID group capacity is 4TB and the input / output mapping granularity is set to 8 stripes, the size of the IO mapping table is 4TB / 8MB / 8bit=64KB.

[0097] The memory capacity is preset; the calculated input and output mapping table volume is compared with the memory capacity. If the IO mapping table volume is small, the number of bits of the mapping value can be determined directly according to the mapping table size. Specifically, any one of 16 bits, 32 bits, and 64 bits can be randomly selected as the mapping bit. If the IO mapping table volume is large, it may affect the IO performance. At this time, it is necessary to reduce the RAID group capacity and / or increase the IO mapping granularity to reduce the size of the IO mapping table until its volume is no larger than the memory capacity.

[0098] It can be seen that in the embodiment of the present invention, the three key parameters of the capacity of the redundant array of independent disks (RAID group) of the disk, the input-output mapping granularity and the memory capacity are obtained; the volume of the input-output mapping table is calculated and determined according to the capacity and input-output mapping granularity of the RAID group; when it is found that the volume of the input-output mapping table exceeds the memory capacity, the capacity or input-output mapping granularity of the RAID group is intelligently adjusted to reduce the volume of the input-output mapping table, and the new volume of the input-output mapping table is calculated again until the volume no longer exceeds the memory capacity; when the volume of the input-output mapping table meets the memory capacity limit, the mapping bit number of the mapping value is set according to the size of the input-output mapping table. Through the above technical solution, the capacity and input-output mapping granularity of the RAID group can be intelligently and dynamically adjusted according to the configuration of the RAID group and the memory capacity limit, thereby maximizing the data read and write performance while ensuring the stable operation of the system.

[0099] Specifically, in a possible implementation of an embodiment of the present invention, the capacity and input-output mapping granularity corresponding to the independent redundant array of disks group of the disk are adjusted to obtain the capacity and new input-output mapping granularity corresponding to the new independent redundant array of disks group, including: determining a capacity adjustment parameter and a granularity adjustment parameter according to a mode selected by a user; adjusting the capacity corresponding to the independent redundant array of disks group of the disk according to the capacity adjustment parameter to obtain the capacity corresponding to the new independent redundant array of disks group; adjusting the input-output mapping granularity according to the granularity adjustment parameter to obtain the new input-output mapping granularity.

[0100] Among them, the user can pre-select the RAID mode to determine a set of capacity adjustment parameters and granularity adjustment parameters. The RAID mode includes at least: refined fast reconstruction and conventional reconstruction. When refined fast reconstruction is selected, the input and output mapping granularity is not adjusted, and only the capacity corresponding to the independent disk redundant array group of the disk is adjusted. When conventional reconstruction is selected, both can be adjusted together.

[0101] A mapping relationship for adjusting the size corresponding to different RAID group levels is also set. After determining the mode and the current RAID group level, the capacity adjustment parameters and granularity adjustment parameters can be determined based on the mapping relationship; and the first adjustment is performed to obtain the capacity corresponding to the new independent disk redundant array group and the new input and output mapping granularity.

[0102] It can be seen that in the embodiment of the present invention, the capacity and IO mapping granularity of the RAID group are automatically adjusted according to the RAID mode selected by the user, so as to determine the capacity corresponding to the new independent disk redundant array group and the new input and output mapping granularity while meeting the actual needs of the user.

[0103] In a possible implementation of an embodiment of the present invention, before determining the multiple stripe groups to be reconstructed that require data reconstruction according to the input-output mapping table, it also includes: determining the total number of stripe groups of the multiple stripe groups to be reconstructed corresponding to the input-output mapping table; if the total number of stripe groups is greater than a preset total number threshold, performing full disk data reconstruction; if the total number of stripe groups is not greater than the preset total number threshold, executing the step of determining the multiple stripe groups to be reconstructed that require data reconstruction according to the input-output mapping table.

[0104] The total number of stripe groups of multiple stripe groups to be reconstructed is also the total number of bit values ​​"1" in the input-output mapping table. When the size of the RAID group corresponding to the input-output mapping table is the size of the standard RAID or is larger than the preset size, it means that the amount of data corresponding to the input-output table is large; if the total amount is increased and is larger than the preset total amount threshold, the amount of data that needs to be reconstructed is large. At this time, the full disk data reconstruction strategy can be executed, that is, all data in the entire RAID system is rebuilt and verified to ensure the integrity and reliability of the data. If the total number of stripe groups is not larger than the preset total amount threshold, it means that the current data reconstruction task is small in scale and the system resources are sufficient to cope with it. The step of determining the multiple stripe groups to be reconstructed that need data reconstruction according to the input-output mapping table can be executed.

[0105] It can be seen that in the embodiment of the present invention, the scale of the current reconstruction task can be intelligently evaluated through the total amount of stripe groups, and a suitable data reconstruction strategy can be selected according to the evaluation result. Specifically, when the scale of the data reconstruction task is large, the full disk data reconstruction strategy is selected to ensure the integrity and reliability of the data; when the scale of the data reconstruction task is small, the data reconstruction task is planned in detail according to the input-output mapping table to improve the efficiency and accuracy of data reconstruction.

[0106] In a possible implementation method of an embodiment of the present invention, after reconstructing the data of the independent disk redundant array group after the disk is added according to multiple stripe groups to be reconstructed, it also includes: verifying the consistency of the reconstructed data of the disk and the data of the redundant space; if the consistency verification passes, confirming that the data reconstruction is successful; if the consistency verification fails, generating a prompt message.

[0107] The consistency verification refers to the process of comparing and verifying the reconstructed data and the data in the redundant space to ensure the consistency of the two. The data in the redundant space refers to the data in the extra space reserved in the disk array to provide data redundancy and fault tolerance.

[0108] In the embodiment of the present invention, after the disk reconstruction operation is completed, the reconstructed data needs to be verified to ensure its correctness and integrity. Specifically, the reconstructed data of the disk is verified for consistency with the data in the redundant space to verify whether the reconstructed data is consistent with the original data. If the data content between the two is completely consistent, it means that the data reconstruction operation is successful and the data has been successfully recovered; if there is a difference in the data content between the two, it means that there is a problem with the data reconstruction operation and further processing is required.

[0109] In some embodiments, the data after disk reconstruction and the data in the redundant space are read; a specific verification algorithm is used for verification and a corresponding verification value is generated; the verification values ​​of the two parts of data are compared; if the verification values ​​are consistent, it means that the data reconstruction is successful and the data consistency is verified; if the verification values ​​are inconsistent, it means that there is a problem with the data reconstruction and a corresponding prompt message is generated for the user's reference and processing. In order to improve the verification efficiency, a number of data can be randomly selected as representatives for consistency verification. If all verifications are successful, the reconstruction is determined to be successful, otherwise the reconstruction fails.

[0110] It can be seen that in the embodiment of the present invention, the consistency of the reconstructed data of the disk and the data of the redundant space can be verified to ensure the reliability and accuracy of the data reconstruction.

[0111] A possible implementation method of the embodiment of the present invention also includes: when the abnormal disk goes offline, determining a target disk with redundant space from the independent disk redundant array group where the abnormal disk is located; reconstructing the data of the abnormal disk according to the stripe group of the target disk, and storing it in the redundant space, and using the redundant space to process the received new input and output information.

[0112] Redundant space refers to the extra storage space reserved in the RAID group to provide data redundancy and fault tolerance. When a disk in the RAID group fails, the redundant space can be used to store the reconstructed data to ensure data integrity and normal operation of the system. A healthy disk refers to a disk that is working normally and can be accessed in the RAID group.

[0113] Specifically, determine the RAID group where the abnormal disk is located, and identify the healthy disks and redundant space in the RAID group; use the stripe group on the healthy disk to reconstruct the data on the abnormal disk, and store the reconstructed data in the redundant space; even if the abnormal disk cannot work normally, the data in the redundant space can still be used to respond to subsequent input and output requests, thereby ensuring the normal operation of the system and the integrity of the data.

[0114] It can be seen that in the embodiment of the present invention, when an abnormal disk in the RAID system goes offline, it is possible to identify redundant space and use the stripe group on the healthy disk to reconstruct the data of the abnormal disk; use the redundant space to perform data reconstruction and input and output processing; and improve the system performance and resource utilization.

[0115] A possible implementation method of an embodiment of the present invention also includes: obtaining historical data reconstruction information, the data reconstruction information includes: the total amount of historical stripe groups and the reconstruction time; constructing a relationship model of the total amount time based on the total amount of historical stripe groups and the reconstruction time; and determining a preset total amount threshold based on the relationship model of the total amount time.

[0116] The historical data reconstruction information is obtained from the database, including the total amount of stripe groups and the corresponding reconstruction time at each data reconstruction. The historical data reconstruction information is not limited to the information of the RAID group of the current device, but can be the historical data reconstruction information under various working conditions of the same type of RAID group. The embodiment of the present invention is not limited thereto, as long as it can be diverse and have a sufficient amount of data.

[0117] Use the collected historical data to build a relationship model between the total amount of the stripe group and the reconstruction time. Specifically, draw a scatter plot of the total amount of the stripe group and the reconstruction time; select a suitable mathematical model for fitting according to the distribution trend of the scatter plot to build a relationship model of the total amount and time. Based on the constructed total amount and time relationship model, determine the preset stripe group total amount threshold. Specifically, the user can select the reconstruction requirement, such as the user-specified time, and determine the corresponding total amount from the model based on the user-specified time as the preset total amount threshold.

[0118] A data reconstruction device provided by an embodiment of the present invention is introduced below. The device described below and the method described above can be referred to each other. The device of this embodiment is set in an electronic device. Figure 5 , Figure 5 The present invention is a block diagram of a data reconstruction device 200 according to one embodiment of the present invention, comprising: a first determination module 210, which is used to determine the independent redundant array array group of the disk according to the original information of the independent redundant array array of the disk after a disk is detected to be online; a judgment module 220, which is used to determine whether the independent redundant array array group of the disk is the current independent redundant array array group; a reading module 230, which is used to read the input-output mapping table of the current independent redundant array array group if it is the current independent redundant array array group, wherein the input-output mapping table records the input-output information after the disk is offline, the input-output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that the input-output information is written to the stripe group corresponding to the bit value; a second determination module 240, which is used to determine a plurality of stripe groups to be reconstructed that need to be reconstructed according to the input-output mapping table; a data reconstruction module 250, which is used to reconstruct the data of the independent redundant array array group after the disk is added according to the plurality of stripe groups to be reconstructed.

[0119] In a preferred example, the present invention can be further configured as follows: the second determination module 240, when determining the multiple stripe groups to be reconstructed that need to be reconstructed according to the input-output mapping table, is used to: determine the target mapping value identifier of the input-output mapping table whose mapping value is not 0; determine the target stripe group identifier according to the target offset bit of the target mapping value identifier, the target offset bit is an offset bit with a bit value of 1; determine the multiple stripe groups to be reconstructed that need to be reconstructed according to the target stripe group identifier.

[0120] In a preferred example, the present invention can be further configured as follows: it also includes: a mapping table recording module, which is used to: obtain new input and output information when an abnormal disk goes offline; calculate the input and output position of each position information in the input and output mapping table based on each position information of the new input and output information, wherein the position information of the new input and output information is composed of the starting position information to the ending position information of the new input and output information, and the input and output position includes: a mapping value identifier and an offset bit identifier; the bit value of the input and output position of each position information of the new input and output information in the input and output mapping table is recorded as 1.

[0121] In a preferred example, the present invention can be further configured as follows: when the mapping table record module calculates the input and output position of each position information in the input-output mapping table according to each position information of the new input and output information, it is used to: use preset operations to calculate the mapping value identifier and the offset bit identifier of each position information of the new input and output information in sequence; wherein the preset operations include: obtaining the stripe group identifier according to the position information of the new input and output information and the stripe group size; determining the mapping value identifier and the offset bit identifier according to the stripe group identifier and the second quantity.

[0122] In a preferred example, the present invention can be further configured as follows: the mapping table recording module calculates the input and output position of each position information in the input-output mapping table based on each position information of the new input-output information, and is used to: calculate the starting input and output position of the starting position information in the input-output mapping table based on the starting position information of the new input-output information; calculate the ending input and output position of the ending position information in the input-output mapping table based on the ending position information of the new input-output information; and use all input and output positions between the starting input and output position and the ending input and output position as the input and output positions of the new input-output information.

[0123] In a preferred example, the present invention can be further configured as follows: it also includes: a marking module, which is used to: when the independent disk redundant array group is normal, mark the input-output mapping table as an invalid state; when the independent disk redundant array group is abnormal, mark the input-output mapping table as a valid state, and the valid state is used to indicate that the input and output information is recorded in the input-output mapping table.

[0124] In a preferred example, the present invention can be further configured to include: a clearing module for clearing all bit values ​​in the input-output mapping table.

[0125] In a preferred example, the present invention can be further configured as follows: when it is detected that a disk is online, it also includes: a storage judgment module, used to judge whether the original information of the independent disk redundant array is stored in the disk; a first determination module 210, used to determine the independent disk redundant array group of the disk according to the original information of the independent disk redundant array of the disk if the original information of the independent disk redundant array is stored in the disk; a data reconstruction module 250, also used to perform full disk data reconstruction if the original information of the independent disk redundant array is not stored in the disk.

[0126] In a preferred example, the present invention can be further configured as follows: the data structure of the input-output mapping table is in the form of an array or a linked list.

[0127] In a preferred example, the present invention can be further configured as follows: the data reconstruction module 250 reconstructs data on the independent redundant array of disks group after the disk is added according to multiple stripe groups to be reconstructed, and is used to: perform consistency check on the target stripe group to be reconstructed and the corresponding stripe group of the disk, wherein the target stripe group to be reconstructed is any stripe group among the multiple stripe groups to be reconstructed; if the verification passes, data reconstruction of the target stripe group to be reconstructed is not performed; if the verification fails, data reconstruction of the independent redundant array of disks group after the disk is added is performed according to the target stripe group to be reconstructed.

[0128] In a preferred example, the present invention can be further configured as follows: it also includes: a mapping table production module, which is used to: obtain the capacity, input-output mapping granularity and memory capacity corresponding to the independent disk redundant array group of the disk, the input-output mapping granularity represents the volume of each stripe group; determine the input-output mapping table volume according to the capacity and input-output mapping granularity corresponding to the independent disk redundant array group; if the input-output mapping table volume is larger than the memory capacity, adjust the capacity and input-output mapping granularity corresponding to the independent disk redundant array group of the disk to obtain a new capacity and new input-output mapping granularity corresponding to the new independent disk redundant array group, and confirm the new input-output mapping table volume until the input-output mapping table volume is not larger than the memory capacity; if the input-output mapping table volume is not larger than the memory capacity, set the second number of input-output mapping tables according to the input-output mapping table size.

[0129] In a preferred example, the present invention can be further configured as: a mapping table creation module, which adjusts the capacity corresponding to the independent disk redundant array group of the disk and the input-output mapping granularity to obtain the capacity corresponding to the new independent disk redundant array group and the new input-output mapping granularity, and is used to: determine the capacity adjustment parameter and the granularity adjustment parameter according to the mode selected by the user; adjust the capacity corresponding to the independent disk redundant array group of the disk according to the capacity adjustment parameter to obtain the capacity corresponding to the new independent disk redundant array group; adjust the input-output mapping granularity according to the granularity adjustment parameter to obtain the new input-output mapping granularity.

[0130] In a preferred example, the present invention can be further configured as follows: it also includes: a third determination module, used to determine the total number of stripe groups of multiple stripe groups to be reconstructed corresponding to the input-output mapping table; a data reconstruction module 250, which is also used to perform full disk data reconstruction if the total number of stripe groups is greater than a preset total threshold; if the total number of stripe groups is not greater than the preset total threshold, execute the step of determining the multiple stripe groups to be reconstructed that need to be reconstructed according to the input-output mapping table.

[0131] In a preferred example, the present invention can be further configured as follows: it also includes: a consistency verification module, which is used to verify the consistency of the reconstructed data of the disk and the data in the redundant space; if the consistency verification passes, it is confirmed that the data reconstruction is successful; if the consistency verification fails, a prompt message is generated.

[0132] In a preferred example, the present invention can be further configured as follows: it also includes: a fourth determination module, which is used to determine a target disk with redundant space from the independent disk redundant array group where the abnormal disk is located when the abnormal disk is offline; a redundant processing module, which is used to reconstruct the data of the abnormal disk according to the stripe group of the target disk, store it in the redundant space, and use the redundant space to process the received new input and output information.

[0133] In a preferred example, the present invention can be further configured as follows: it also includes: a threshold determination module, which is used to obtain historical data reconstruction information, and the data reconstruction information includes: the total amount of historical stripe groups and the reconstruction time; based on the total amount of historical stripe groups and the reconstruction time, a relationship model of the total amount time is constructed; based on the relationship model of the total amount time, a preset total amount threshold is determined.

[0134] Figure 6 A structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 6 As shown, the electronic device includes: a memory 60 for storing a computer program; and a processor 61 for implementing the steps of the data reconstruction method in the above embodiment when executing the computer program.

[0135] The electronic device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0136] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0137] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein the computer program, after being loaded and executed by the processor 61, can implement the relevant steps of the data reconstruction method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc.

[0138] In some embodiments, the electronic device may further include a display screen 62 , an input / output interface 63 , a communication interface 64 , a power supply 65 , and a communication bus 66 .

[0139] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure.

[0140] It is understandable that if the data reconstruction method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the current technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk, etc. Various media that can store program codes.

[0141] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned data reconstruction method are implemented.

[0142] An embodiment of the present invention provides a computer program product, including a computer program or instructions, which implements the corresponding contents of the aforementioned method embodiment when the computer program or instructions are executed by a processor.

[0143] The above is a detailed introduction to a data reconstruction method, device, equipment, medium and product provided by an embodiment of the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0144] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0145] The above is a detailed introduction to a data reconstruction method, device, equipment, medium and product provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A data reconstruction method, characterized in that: include: When it is detected that a disk is online, the independent disk redundant array group of the disk is determined according to the independent disk redundant array original information of the disk; Determining whether the redundant array of independent disks group of the disk is a current redundant array of independent disks group; If it is the current redundant array of independent disks group, read the input-output mapping table of the current redundant array of independent disks group, wherein the input-output mapping table records the input-output information after the disk is offline, the input-output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that the input-output information is written into the stripe group corresponding to the bit value; Determining, according to the input-output mapping table, a plurality of stripe groups to be reconstructed that require data reconstruction; According to the multiple stripe groups to be reconstructed, data is reconstructed for the redundant array of independent disks group after the disk is added.

2. The data reconstruction method according to claim 1, characterized in that: Determining a plurality of to-be-reconstructed stripe groups requiring data reconstruction according to the input-output mapping table includes: Determine a target mapping value identifier whose mapping value of the input-output mapping table is not 0; Determine a target stripe group identifier according to a target offset bit of the target mapping value identifier, wherein the target offset bit is an offset bit having a bit value of 1; According to the target stripe group identifier, a plurality of stripe groups to be reconstructed that require data reconstruction are determined.

3. The data reconstruction method according to claim 1, characterized in that: Also includes: When the abnormal disk goes offline, obtain new input and output information; According to each position information of the new input / output information, the input / output position of each position information in the input / output mapping table is calculated, wherein the position information of the new input / output information is composed of the starting position information to the ending position information of the new input / output information, and the input / output position includes: a mapping value identifier and an offset bit identifier; The bit value of each position information of the new input / output information at the input / output position in the input / output mapping table is recorded as 1.

4. The data reconstruction method according to claim 3, characterized in that: According to each position information of the new input and output information, calculating the input and output position of each position information in the input and output mapping table includes: Using the preset operation, the mapping value identifier and the offset bit identifier of each position information of the new input and output information are calculated in sequence; The preset operation includes: obtaining a stripe group identifier according to the location information of the new input / output information and the stripe group size; and determining a mapping value identifier and an offset bit identifier according to the stripe group identifier and the second quantity.

5. The data reconstruction method according to claim 3, characterized in that: According to each position information of the new input and output information, calculating the input and output position of each position information in the input and output mapping table includes: Calculate the starting input / output position of the starting position information in the input / output mapping table according to the starting position information of the new input / output information; Calculate the end input / output position of the end position information in the input / output mapping table according to the end position information of the new input / output information; All input-output positions between the start input-output position and the end input-output position are used as the input-output positions of the new input-output information.

6. The data reconstruction method according to claim 1, characterized in that: Also includes: When the redundant array of independent disks group is normal, marking the input-output mapping table as an invalid state; When the redundant array of independent disks group is abnormal, the input-output mapping table is marked as a valid state, and the valid state is used to indicate that the input-output information is recorded in the input-output mapping table.

7. The data reconstruction method according to claim 1, characterized in that: After reconstructing data on the independent redundant array of disks group after the disk is added according to the multiple stripe groups to be reconstructed, the method further includes: clearing all bit values ​​in the input-output mapping table.

8. The data reconstruction method according to claim 1, characterized in that: When a disk is detected to be online, the following is also included: Determining whether the disk stores original information of a redundant array of independent disks; If the disk stores original information of the redundant array of independent disks, determining the redundant array of independent disks group of the disk according to the original information of the redundant array of independent disks of the disk; If the original information of the redundant array of independent disks is not stored in the disk, full disk data reconstruction is performed.

9. The data reconstruction method according to claim 1, characterized in that: The data structure of the input-output mapping table is in the form of an array or a linked list.

10. The data reconstruction method according to any one of claims 1 to 9, characterized in that: According to the plurality of stripe groups to be reconstructed, data is reconstructed on the independent redundant array of disks group after the disk is added, including: Performing a consistency check between a target stripe group to be reconstructed and a corresponding stripe group of the disk, wherein the target stripe group to be reconstructed is any stripe group among the multiple stripe groups to be reconstructed; If the verification is successful, data reconstruction of the target stripe group to be reconstructed will not be performed; If the verification fails, data is reconstructed on the redundant array of independent disks group after the disk is added according to the target stripe group to be reconstructed.

11. The data reconstruction method according to any one of claims 1 to 9, characterized in that: Before reading the input and output mapping table of the current redundant array of independent disks group, the following steps are also included: Obtaining the capacity, input-output mapping granularity and memory capacity corresponding to the independent disk redundant array group of the disk, wherein the input-output mapping granularity represents the volume of each stripe group; determining the volume of the input-output mapping table according to the capacity and input-output mapping granularity corresponding to the independent disk redundant array group; If the volume of the input-output mapping table is larger than the memory capacity, the capacity corresponding to the redundant array of independent disks group of the disk and the input-output mapping granularity are adjusted to obtain the capacity corresponding to the new redundant array of independent disks group and the new input-output mapping granularity, and the new volume of the input-output mapping table is confirmed until the volume of the input-output mapping table is no larger than the memory capacity; If the volume of the input-output mapping table is not greater than the memory capacity, the second number of input-output mapping tables is set according to the size of the input-output mapping table.

12. The data reconstruction method according to claim 11, characterized in that: Adjusting the capacity and input / output mapping granularity corresponding to the independent redundant array of disks group of the disk to obtain the capacity and new input / output mapping granularity corresponding to the new independent redundant array of disks group includes: Determine capacity adjustment parameters and granularity adjustment parameters according to the mode selected by the user; Adjust the capacity corresponding to the redundant array of independent disks group of the disk according to the capacity adjustment parameter to obtain the capacity corresponding to the new redundant array of independent disks group; The input-output mapping granularity is adjusted according to the granularity adjustment parameter to obtain a new input-output mapping granularity.

13. The data reconstruction method according to any one of claims 1 to 9, characterized in that: Before determining a plurality of to-be-reconstructed stripe groups that need to be reconstructed according to the input-output mapping table, the method further includes: Determine the total number of stripe groups of the plurality of stripe groups to be reconstructed corresponding to the input-output mapping table; If the total amount of the stripe group is greater than a preset total amount threshold, performing full disk data reconstruction; If the total amount of the stripe groups is not greater than the preset total amount threshold, a step of determining a plurality of stripe groups to be reconstructed that require data reconstruction is performed according to the input-output mapping table.

14. The data reconstruction method according to claim 13, characterized in that: After reconstructing data on the independent disk redundant array group after adding the disk according to the multiple stripe groups to be reconstructed, the method further includes: Performing consistency verification on the reconstructed data of the disk and the data in the redundant space; If the consistency verification is passed, the data reconstruction is confirmed to be successful; If the consistency verification fails, a prompt message is generated.

15. The data reconstruction method according to claim 14, characterized in that: Also includes: When the abnormal disk goes offline, a target disk with redundant space is determined from the independent disk redundant array group where the abnormal disk is located; The data of the abnormal disk is reconstructed according to the stripe group of the target disk and stored in the redundant space, and the received new input and output information is processed by using the redundant space.

16. The data reconstruction method according to claim 13, characterized in that: Also includes: Obtaining historical data reconstruction information, the data reconstruction information including: the total amount of historical stripe groups and reconstruction time; Constructing a total amount time relationship model according to the total amount of the historical stripe group and the reconstruction time; Based on the relationship model of the total amount time, a preset total amount threshold is determined.

17. A data reconstruction device, characterized in that: include: A first determination module is used to determine the independent disk redundant array group of the disk according to the independent disk redundant array original information of the disk after monitoring that a disk is online; A judgment module, used for determining whether the independent disk redundant array group of the disk is a current independent disk redundant array group; A reading module is used to read an input-output mapping table of the current redundant array of independent disks group if it is a current redundant array of independent disks group, wherein the input-output mapping table records the input-output information of the disk after it goes offline, the input-output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that the input-output information is written into the stripe group corresponding to the bit value; A second determination module, configured to determine a plurality of to-be-reconstructed stripe groups for which data reconstruction is required according to the input-output mapping table; The data reconstruction module is used to reconstruct data of the independent disk redundant array group after the disk is added according to the multiple stripe groups to be reconstructed.

18. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the data reconstruction method according to any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data reconstruction method according to any one of claims 1 to 16 are implemented.

20. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the data reconstruction method according to any one of claims 1 to 16 are implemented.

Images