A data reconstruction method, apparatus, device, medium, and product

Abstract

The present invention relates to the technical field of data storage, and in particular to a data reconstruction method, apparatus, device, medium, and product. The method includes: after detecting that a disk goes online, determining the redundant array of independent disks (RAID) group of the disk according to the original RAID information of the disk; if the RAID group of the disk is the current RAID group, reading the input / output mapping table of the current RAID group; determining multiple strip groups to be reconstructed that require data reconstruction according to the input / output mapping table; reconstructing the data of the RAID group according to the multiple strip groups to be reconstructed. After the member disk failure is repaired and goes online, only the input / output information recorded in the input / output mapping table is needed to determine the multiple strip groups to be reconstructed that require data reconstruction, so as to facilitate reconstruction based on this part of the data, reduce the amount of reconstructed data, and thus improve the reconstruction efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular, to a data reconstruction method, apparatus, device, medium and product. Background Art

[0002] With the development of technology, the era of data explosion has arrived, and a large amount of data needs to be stored and read. Data has replaced computing as the center of information computing. This has led to an increasing demand for storage technology, including capacity requirements, I / O (Input / Output) performance requirements, data security requirements, scalability requirements, etc. To solve related problems, RAID technology has emerged.

[0003] RAID (redundant array of independent disks) technology, as a high-performance and highly reliable storage technology, has been widely used. RAID mainly uses data striping, mirroring and data verification technologies to obtain high performance, high reliability, fault tolerance and scalability. According to the strategies and architectures of applying or combining these three technologies, RAID can be divided into different levels to meet the needs of different scenarios. Among them, mirroring and data verification technologies can improve the reliability of data.

[0004] When a hardware link failure occurs in a member disk of a RAID system, the RAID system will be triggered to enter the degraded state, and subsequent IO data will not be updated to the corresponding disk. Subsequently, the hardware link failure is restored, but there is no new IO data in the restored disk, and it cannot be directly added to the RAID group, which will cause the problem of stripe inconsistency. To add the restored disk to the RAID group, a full-disk data reconstruction operation is required. This reconstruction process is very long, often taking several hours, and during the reconstruction process, the RAID system is still in the degraded state, and its reliability and performance will both decline.

[0005] Therefore, how to perform data reconstruction efficiently is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0006] The object of the present invention is to provide a data reconstruction method, apparatus, device, medium and product, which does not require full-disk data reconstruction. Only the input-output information recorded in the input-output mapping table needs to be used to determine multiple strip groups to be reconstructed that require data reconstruction, so as to facilitate reconstruction based on this part of the data, reduce the amount of reconstructed data, and thus improve the reconstruction efficiency. It can enable the redundant array of independent disks to quickly enter the protected state, and can also reduce the amount of data written to the disk during reconstruction and improve the service life of the disk.

[0007] In a first aspect, a data reconstruction method is provided, including: after detecting that a disk is online, determining the redundant array of independent disks (RAID) group of the disk according to the original RAID information of the disk; determining whether the RAID group of the disk is the current RAID group; if it is the current RAID group, reading the input / output mapping table of the current RAID group, where the input / output mapping table records the input / output information after the disk goes offline, the input / output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that there is input / output information written to the stripe group corresponding to the bit value; determining, according to the input / output mapping table, a plurality of stripe groups to be reconstructed that need data reconstruction; and performing data reconstruction on the RAID group after the disk is added according to the plurality of stripe groups to be reconstructed.

[0008] In a preferred example of the present invention, it can be further configured as: determining, according to the input / output mapping table, a plurality of stripe groups to be reconstructed that need data reconstruction, including: determining a target mapping value identifier whose mapping value in the input / output mapping table is not 0; determining a target stripe group identifier according to the target offset bit of the target mapping value identifier, where the target offset bit is the offset bit with a bit value of 1; and determining, according to the target stripe group identifier, a plurality of stripe groups to be reconstructed that need data reconstruction.

[0009] In a preferred example of the present invention, it can be further configured as: further including: after an abnormal disk goes offline, obtaining new input / output information; calculating the input / output position of each position information of the new input / output information in the input / output mapping table according to each position information of the new input / output information, where the position information of the new input / output information is composed of the starting position information to the ending position information of the new input / output information, and the input / output position includes: a mapping value identifier and an offset bit identifier; and setting the bit value of the input / output position of each position information of the new input / output information in the input / output mapping table to 1.

[0010] In a preferred example of the present invention, it can be further configured as: calculating the input / output position of each position information of the new input / output information in the input / output mapping table according to each position information of the new input / output information, including: using a preset operation to sequentially calculate the mapping value identifier and the offset bit identifier of each position information of the new input / output information; where the preset operation includes: obtaining a stripe group identifier according to the position information of the new input / output information and the stripe group size; and determining the mapping value identifier and the offset bit identifier according to the stripe group identifier and the second number.

[0011] In a preferred example, the present invention can be further configured as follows: According to each position information of the new input / output information, calculate the input / output positions of each position information in the input / output mapping table, including: According to the starting position information of the new input / output information, calculate the starting input / output position of the starting position information in the input / output mapping table; According to the ending position information of the new input / output information, calculate the ending input / output position of the ending position information in the input / output mapping table; Take all the input / output positions between the starting input / output position and the ending input / output position as the input / output positions of the new input / output information.

[0012] In a preferred example, the present invention can be further configured as follows: It further includes: When the redundant array of independent disks (RAID) group is normal, mark the input / output mapping table as an invalid state; When the RAID group is abnormal, mark the input / output mapping table as a valid state, and the valid state is used to indicate recording input / output information in the input / output mapping table.

[0013] In a preferred example, the present invention can be further configured as follows: It further includes: After performing data reconstruction on the RAID group after adding the disk according to the multiple strip groups to be reconstructed, it further includes: Clear all bit values in the input / output mapping table.

[0014] In a preferred example, the present invention can be further configured as follows: When it is detected that a disk goes online, it further includes: Determine whether the disk stores the original RAID information; If the disk stores the original RAID information, determine the RAID group of the disk according to the original RAID information of the disk; If the disk does not store the original RAID information, perform full disk data reconstruction.

[0015] In a preferred example, the present invention can be further configured as follows: The data structure of the input / output mapping table is in the form of an array or a linked list.

[0016] In a preferred example, the present invention can be further configured as follows: Perform data reconstruction on the RAID group after adding the disk according to the multiple strip groups to be reconstructed, including: Perform a consistency check on the target strip group to be reconstructed and the corresponding strip group of the disk, where the target strip group to be reconstructed is any one of the multiple strip groups to be reconstructed; If the verification passes, no data reconstruction is performed on the target strip group to be reconstructed; If the verification fails, perform data reconstruction on the RAID group after adding the disk according to the target strip group to be reconstructed.

[0017] In a preferred example, the present invention can be further configured as follows: Before reading the input / output mapping table of the current redundant array of independent disks (RAID) group, it further includes: obtaining the capacity, input / output mapping granularity, and memory capacity corresponding to the RAID group of the disk, where the input / output mapping granularity represents the volume of each stripe group; determining the volume of the input / output mapping table according to the capacity and input / output mapping granularity corresponding to the RAID group; if the volume of the input / output mapping table is greater than the memory capacity, adjusting the capacity and input / output mapping granularity corresponding to the RAID group of the disk to obtain the capacity corresponding to the new RAID group and the new input / output mapping granularity, and confirming the volume of the new input / output mapping table until the volume of the input / output mapping table is not greater than the memory capacity; if the volume of the input / output mapping table is not greater than the memory capacity, setting the second quantity of the input / output mapping table according to the size of the input / output mapping table.

[0018] In a preferred example, the present invention can be further configured as follows: Adjusting the capacity and input / output mapping granularity corresponding to the RAID group of the disk to obtain the capacity corresponding to the new RAID group and the new input / output mapping granularity includes: determining the capacity adjustment parameter and granularity adjustment parameter according to the mode selected by the user; adjusting the capacity corresponding to the RAID group of the disk according to the capacity adjustment parameter to obtain the capacity corresponding to the new RAID group; and adjusting the input / output mapping granularity according to the granularity adjustment parameter to obtain the new input / output mapping granularity.

[0019] In a preferred example, the present invention can be further configured as follows: Before determining multiple stripe groups to be reconstructed that require data reconstruction according to the input / output mapping table, it further includes: determining the total number of stripe groups of the multiple stripe groups to be reconstructed corresponding to the input / output mapping table; if the total number of stripe groups is greater than the preset total threshold, performing full-disk data reconstruction; if the total number of stripe groups is not greater than the preset total threshold, performing the step of determining multiple stripe groups to be reconstructed that require data reconstruction according to the input / output mapping table.

[0020] In a preferred example, the present invention can be further configured as follows: After performing data reconstruction on the RAID group after adding the disk according to the multiple stripe groups to be reconstructed, it further includes: verifying the consistency of the data after reconstruction of the disk and the data in the redundant space; if the consistency verification is passed, confirming that the data reconstruction is successful; if the consistency verification is not passed, generating a prompt message.

[0021] In a preferred example, the present invention can be further configured to further include: when an abnormal disk goes offline, determining a target disk with redundant space from the redundant array of independent disks (RAID) group where the abnormal disk is located; reconstructing the data of the abnormal disk according to the stripe group of the target disk and storing it in the redundant space, and using the redundant space to process the received new input / output information.

[0022] In a preferred example, the present invention can be further configured to further include: obtaining historical data reconstruction information, where the data reconstruction information includes: the total amount of historical stripe groups, reconstruction time; constructing a relationship model of total amount and time according to the total amount of historical stripe groups and the reconstruction time; determining a preset total amount threshold based on the relationship model of total amount and time.

[0023] In a second aspect, a data reconstruction device is provided, including: a first determination module, configured to, when it is monitored that a disk goes online, determine the redundant array of independent disks (RAID) group of the disk according to the original RAID information of the disk; a judgment module, configured to determine whether the RAID group of the disk is the current RAID group; a reading module, configured to, if it is the current RAID group, read the input / output mapping table of the current RAID group, where the input / output mapping table records the input / output information after the disk goes offline, the input / output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that input / output information is written to the stripe group corresponding to the bit value; a second determination module, configured to determine a plurality of stripe groups to be reconstructed that need data reconstruction according to the input / output mapping table; a data reconstruction module, configured to perform data reconstruction on the RAID group after adding the disk according to the plurality of stripe groups to be reconstructed.

[0024] In a third aspect, an electronic device is provided, including: a memory, configured to store a computer program; a processor, configured to execute the computer program to implement the method according to any one of the first aspect.

[0025] In a fourth aspect, a computer-readable storage medium is provided, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of the first aspect is implemented.

[0026] In a fifth aspect, a computer program product is provided, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the method according to any one of the first aspect is implemented.

[0027] In summary, the data reconstruction method provided by the present invention includes the following beneficial technical effects: when a certain member disk fails and goes offline, subsequent input and output information can be recorded through the input-output mapping table; when it is detected that the disk goes online, if the redundant array of independent disks (RAID) group determined by the original information of the disk is valid, it means that the member disk goes online after the failure is repaired. At this time, there is no need to perform full-disk data reconstruction. Only the multiple strip groups to be reconstructed that need data reconstruction need to be determined through the input and output information recorded in the input-output mapping table, so as to perform reconstruction based on this part of the data, reducing the amount of reconstructed data, thereby improving the reconstruction efficiency, enabling the RAID to quickly enter the protected state, and also reducing the amount of data written to the disk during reconstruction and increasing the service life of the disk.

[0028] In addition, the present invention also provides a data reconstruction device, equipment, medium and product, all of which have the above beneficial technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] To more clearly illustrate the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0030] Figure 1 It is a schematic flowchart of a data reconstruction method provided by an embodiment of the present invention.

[0031] Figure 2 It is a schematic diagram of an input-output mapping table provided by an embodiment of the present invention.

[0032] Figure 3 It is a schematic flowchart of an input-output information recording provided by an embodiment of the present invention.

[0033] Figure 4 It is a schematic flowchart of a data reconstruction provided by an embodiment of the present invention.

[0034] Figure 5 It is a schematic structural diagram of a data reconstruction device provided by an embodiment of the present invention.

[0035] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0037] The terms "including" and "having" in the specification of the present invention and the accompanying drawings above, and any variations related to "including" and "having", are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units not listed.

[0038] For ease of understanding, some professional terms are now explained.

[0039] CPU cache: Used to solve the speed gap between the CPU and memory, a high-speed cache device existing between the memory and the CPU.

[0040] SOC: Abbreviation for System on Chip, called a system-on-chip, also known as a system on a chip, meaning it is a product, an integrated circuit with a dedicated target, which contains a complete system and all the content of the embedded software.

[0041] RAID: Redundant Array of Independent Disks.

[0042] IO: Abbreviation for Input Output, input and output.

[0043] DW: Abbreviation for Double Word, double word, indicating that the bit width of the data is 32 bits.

[0044] SLBA: Start Logical Block Address, the starting logical block address.

[0045] LBA: Logical Block Address, the logical block address.

[0046] NLB: Number of Logical Blocks, the number of blocks for reading data.

[0047] chunk: Equal-sized blocks cut from a disk in RAID.

[0048] stripe: RAID stripe.

[0049] In the related art, when a hardware link failure occurs in a certain member disk in a RAID system, the RAID system will be triggered to enter the degraded state, and subsequent IO data will not be updated to the corresponding disk. Subsequently, the hardware link failure is restored, but there is no new IO data in the restored disk, and it cannot be directly added to the RAID group, which will cause the stripe inconsistency problem. To add the restored disk to the RAID group, a full-disk data reconstruction operation needs to be performed. This reconstruction process is very long, often taking several hours, and during the reconstruction process, the RAID system is still in the degraded state, and its reliability and performance will both decline.

[0050] In view of the above problems, the present invention proposes a method. When a certain disk fails, subsequent IO information can be recorded. When the disk is restored, there is no need to perform a full-disk data reconstruction, only a partial data reconstruction needs to be performed. If the amount of reconstructed data is reduced, the reconstruction time can be reduced, the RAID can quickly enter the protected state, and the amount of data written to the disk during the reconstruction can also be reduced, thereby improving the service life of the disk.

[0051] Specifically, an embodiment of the present invention provides a data reconstruction method, as Figure 1 shown. The method provided in the embodiment of the present invention can be executed by an electronic device. The electronic device is a server, which can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto. The terminal device and the electronic device can be directly or indirectly connected through wired or wireless communication methods. The embodiment of the present invention does not limit this here. The method includes:

[0052] S101. After detecting that a disk is online, determine the independent disk redundant array group of the disk according to the original information of the independent disk redundant array of the disk.

[0053] Among them, the original RAID information of the disk refers to the information related to the RAID configuration stored on the disk, including the RAID group to which the disk belongs, the RAID level, the stripe size, the role of the disk in the RAID group (such as data disk, parity disk, etc.). The RAID group refers to a logical storage unit composed of multiple disks, and the disks are configured according to a specific RAID level.

[0054] It can be understood that in the embodiment of the present invention, when a disk goes offline in the RAID group, the RAID group level remains unchanged and can still be used normally. Exemplarily, when a RAID5 is composed of 5 disks, when one disk goes offline, it is still a RAID5 and can be used normally; if 2 disks go offline, at this time the RAID5 fails and cannot be used, and the solution of the embodiment of the present invention is no longer applicable at this time.

[0055] In an implementable manner, the disk interface is scanned regularly to facilitate detecting whether a new disk is online; alternatively, through an event triggering mechanism, an event notification is received when a new disk is online. When it is monitored that a new disk is added to the system and is in an available state, the original RAID information of the disk is obtained at this time, and the RAID group of the disk is determined according to the original RAID information of the disk.

[0056] S102. Determine whether the redundant array of independent disks (RAID) group of the disk is the current RAID group.

[0057] Among them, the current RAID group refers to the RAID group currently in use. In the RAID management system, when an operation needs to be performed on a certain disk, it is first necessary to determine whether the disk belongs to the current RAID group to determine the data processing mechanism (full disk reconstruction or specific data stripe reconstruction based on the I / O mapping table).

[0058] S103. If it is the current RAID group, read the input / output mapping table of the current RAID group. Among them, the input / output mapping table records the input / output information after the disk goes offline. The input / output mapping table is composed of a first number of mapping values, and each mapping value is composed of a second number of bit values. When the bit value is 1, it means that there is input / output information written to the stripe group corresponding to the bit value.

[0059] If the RAID group of the disk is the current RAID group, it means that it may be that the link failure is repaired and the disk comes online again. At this time, read the I / O mapping table corresponding to the current RAID group, and perform a quick reconstruction operation according to the I / O mapping table corresponding to the RAID group. If the RAID group of the disk is not the current RAID group, perform a full disk reconstruction operation.

[0060] For further elaboration on the I / O mapping table, the I / O mapping table records the I / O information processed by the RAID group after the disk goes offline. The mapping table is composed of a first number n of mapping values, and each mapping value is composed of a second number of bit values. When the bit value is 1, it means that there is input / output information written to the stripe group corresponding to the bit value. Exemplarily, see Figure 2 , Figure 2 which is a schematic diagram of an I / O mapping table provided by an embodiment of the present invention. An embodiment of the present invention proposes an I / O mapping table for recording historical I / O information. The mapping table is composed of n mapping values. The mapping value is a 32-bit number and is initialized to all 0. One bit corresponds to a stripe group on the RAID. A stripe group is composed of N stripes. When the value of the bit is 1, it means that there is an I / O falling on the corresponding stripe group, otherwise it means that there is no I / O on this stripe group.

[0061] The I / O mapping table is stored in the internal storage of the RAID chip. The size of the stripe group can be configured according to the size of the internal storage space of the chip. The size of the stripe group must be an integer multiple of the stripe size. The larger the stripe group, the smaller the I / O mapping table, and the coarser the corresponding marked I / O records. After the link of the subsequent disk is restored, relatively more data needs to be reconstructed.

[0062] S104. Determine multiple stripe groups to be reconstructed that require data reconstruction according to the input / output mapping table.

[0063] Based on all the data with bit values of 1 in the I / O mapping table, determine multiple stripe groups to be reconstructed that require data reconstruction.

[0064] S105. Reconstruct the data of the redundant array of independent disks (RAID) group after adding the disk according to the multiple stripe groups to be reconstructed.

[0065] The specific data reconstruction method is not limited in the embodiments of the present invention as long as it can achieve the purpose of the embodiments of the present invention.

[0066] It can be seen that in the embodiments of the present invention, when a certain member disk fails and goes offline, the subsequent input / output information can be recorded through the input / output mapping table; when it is detected that the disk goes online, if the redundant array of independent disks group determined by the original information of the redundant array of independent disks of this disk is valid, it means that this member disk goes online after the fault is repaired. At this time, it is not necessary to perform full-disk data reconstruction. Only multiple stripe groups to be reconstructed that require data reconstruction need to be determined through the input / output information recorded in the input / output mapping table, so as to reconstruct based on this part of the data, reducing the amount of reconstructed data, thereby improving the reconstruction efficiency, enabling the redundant array of independent disks to quickly enter the protected state, and also reducing the amount of data written to the disk during reconstruction and increasing the service life of the disk.

[0067] In a possible implementation manner of the embodiments of the present invention, S104 determines multiple stripe groups to be reconstructed that require data reconstruction according to the input / output mapping table, including: determining a target mapping value identifier whose mapping value in the input / output mapping table is not 0; determining a target stripe group identifier according to the target offset bit of the target mapping value identifier, where the target offset bit is the offset bit with a bit value of 1; and determining multiple stripe groups to be reconstructed that require data reconstruction according to the target stripe group identifier.

[0068] In an embodiment of the present invention, initialize the mapping table index, and select the mapping value identifier item_id = 0; obtain the mapping value of the mapping value identifier item_id = 0 in the mapping table. If the mapping value is not 0, then sequentially determine whether each bit in the mapping value is 1. Otherwise, increment item_id by 1; sequentially determine whether each bit in the mapping value is 1. If it is 1, then calculate the corresponding stripe group and perform a reconstruction operation. Otherwise, increment item_id by 1, calculate the corresponding stripe group, and perform a reconstruction operation; if all the data in the mapping table has been processed, then end. Otherwise, repeat the above steps.

[0069] Among them, taking 32 bits as an example, A1. Set the mapping value identifier to item_id = 0; A2. Determine whether the mapping value table[item_id]!= 0? If so, execute A3. If not, execute A10; A3. Set the variable item_val == table[item_id]; A4. Set the offset bit identifier bit_offset = 0; A5. Determine whether (item_val >> bit_offset) & 0x1 == 1? If so, execute A6. If not, execute A8; A6. The stripe group identifier group_id = the mapping value identifier item_id 32 + bit_offset; A7. Reconstruct the data of the corresponding stripe group group to be reconstructed; A8. bit_offset++; A9. Determine whether the offset bit identifier bit_offset >= 32? If so, execute A10. If not, execute A5; A10. item_id++; A11. Determine whether item_id >= table_capacity? If so, end. If not, execute A2.

[0070] It can be seen that in an embodiment of the present invention, since each mapping value is a binary array, therefore, it is possible to determine whether there is a bit with a change in IO information for the corresponding mapping value identifier according to whether the binary array is 0. Furthermore, according to the target offset bit identifier of the target mapping value identifier, that is, which bit, determine the target stripe group identifier; thus, it is possible to quickly locate multiple stripe groups to be reconstructed.

[0071] A possible implementation manner of the embodiment of the present invention, the data reconstruction method, further includes: after an abnormal disk goes offline, obtain new input / output information; calculate the input / output position of each position information in the input / output mapping table according to each position information of the new input / output information, where the position information of the new input / output information is composed of the start position information to the end position information of the new input / output information, and the input / output position includes: the mapping value identifier and the offset bit identifier; record the bit value of the input / output position of each position information of the new input / output information in the input / output mapping table as 1.

[0072] Among them, in the embodiments of the present invention, a new IO information writing mechanism is provided. Specifically, when a disk needs to be taken offline due to disk failure, aging, or other reasons, new IO (input / output) information generated after the disk is taken offline is obtained. The new IO information includes the read / write position, size, timestamp, etc. of the data. The read / write position, that is, the IO position information, is composed of multiple position information between the start position information and the end position information. The start position information represents the starting storage position of the data in the new IO information, and the end position information represents the ending storage position of the data in the new IO information.

[0073] For the newly obtained IO information, the system needs to calculate its input / output position in the mapping table according to each position information. The embodiments of the present invention provide an IO mapping table; according to each position information of the new input / output information, calculate the input / output position of each position information in the mapping table to obtain a mapping value identifier and an offset bit identifier. Among them, the mapping value identifier points to the index of a certain entry in the mapping table, and the offset bit identifier represents the specific offset position in this entry; record the bit value of the input / output position of each position information of the new IO information in the mapping table as 1.

[0074] It can be seen that in the embodiments of the present invention, after an abnormal disk is taken offline, the mapping value identifier and the offset bit identifier in the mapping table can be determined according to each position information of the new input / output information, and the bit value of the input / output position of each position information of the new IO information in the mapping table is recorded as 1, so as to accurately locate the stripe group updated by this position information.

[0075] A possible implementation manner of the embodiments of the present invention, calculating the input / output position of each position information in the input / output mapping table according to each position information of the new input / output information includes: using a preset operation to calculate the mapping value identifier and the offset bit identifier of each position information of the new input / output information in sequence; among them, the preset operation includes: obtaining a stripe group identifier according to the position information of the new input / output information and the stripe group size; determining the mapping value identifier and the offset bit identifier according to the stripe group identifier and the second quantity.

[0076] Among them, the stripe group size represents the capacity of the stripe group in the storage system. The stripe group identifier is used to represent the unique identifier of a specific stripe group. The second quantity refers to the number of binary digits of each mapping value, which is used to determine the mapping value identifier and the offset bit identifier subsequently.

[0077] In an embodiment of the present invention, the starting position information (slba) of the recorded IO information is saved in the variable lba; calculate which stripe group the starting position information falls on, i.e., the stripe group identifier, and save it in the variable, where group_id = lba / group_size, and the integer obtained is the stripe group identifier. When the stripe group identifier = 1, it means it is the second stripe group; calculate the corresponding mapping value identifier of the stripe group. Each bit in the mapping table represents a stripe group, and each mapping value is 32 bits, so it needs to be divided by 32. Save it in the variable mapping value identifier item_id = group_id / 32, and take the integer to get the mapping value identifier. When the mapping value identifier = 1, it means it is the second mapping value; calculate the bit offset of the stripe group in the mapping value, and save it in the variable bit_offset as _offset = group_id % 32; mark the IO position by bitwise OR; increment lba by 1; if lba is greater than or equal to the IO end position information, end; otherwise, repeat the above steps.

[0078] It can be seen that in an embodiment of the present invention, according to the target position information of the IO information and the stripe group size, it is accurately determined which stripe group the data should be stored in, and further, the specific position of the data in the mapping table is determined by the stripe group identifier and the second quantity.

[0079] In order to improve the determination efficiency of the IO position, a possible implementation manner of an embodiment of the present invention is to calculate the input / output position of each position information in the input / output mapping table according to each position information of the new input / output information, including: calculating the starting input / output position of the starting position information in the input / output mapping table according to the starting position information of the new input / output information; calculating the ending input / output position of the ending position information in the input / output mapping table according to the ending position information of the new input / output information; and taking all the input / output positions between the starting input / output position and the ending input / output position as the input / output positions of the new input / output information.

[0080] Among them, the method for calculating the IO position includes: obtaining the stripe group identifier according to the position information of the IO information and the stripe group size; determining the mapping value identifier and the offset bit identifier according to the stripe group identifier and the second quantity. After obtaining all the IO positions of the starting position information in the input / output position in the mapping table and the ending position information in the input / output position in the mapping table, all the IO positions between the two are used as the IO positions to be marked by the new IO information. In this way, it is possible to determine all the IO positions by only calculating the first and last two IO positions, which greatly improves the calculation efficiency.

[0081] A possible implementation of an embodiment of the present invention further includes: when the redundant array of independent disks (RAID) group is normal, marking the input-output mapping table as an invalid state; when the RAID group is abnormal, marking the input-output mapping table as a valid state, and the valid state is used to indicate recording input-output information in the input-output mapping table; the IO mapping table is marked as an invalid state when the RAID group is normal. At this time, the RAID group will not record IO information when processing IO. When a disk link exception event occurs in the system, the IO mapping table is set to be valid, and subsequent IOs will be recorded in the mapping table. The process is as Figure 3 shown.

[0082] Specifically, the determination of the RAID group status can be achieved by setting a flag bit in the memory, modifying the mapping table header information, or updating the system configuration file. The specific means are no longer limited in the embodiments of the present invention, and users can set according to actual needs.

[0083] A possible implementation of an embodiment of the present invention, after data reconstruction of the redundant array of independent disks (RAID) group after adding a disk according to multiple strip groups to be reconstructed, further includes: clearing all bit values in the input-output mapping table. Refer to Figure 4 , Figure 4 FIG. is a schematic flow chart of data reconstruction provided by an embodiment of the present invention; when the RAID card detects a newly inserted disk, obtains the original data of the RAID group in the disk, the original data is valid, and when the corresponding input-output mapping table in the memory is valid, quickly reconstructs the disk data through the input-output mapping table. After completion, sets the input-output mapping table to be invalid and clears all values in the input-output mapping packet.

[0084] Among them, the reconstruction progress is monitored in real time. When it is monitored that the RAID group data reconstruction is completed, all positions in the IO mapping table are cleared and set to 0, so that when the IO mapping table is valid again, the interference of the original data in the IO mapping table can further reduce the occurrence of reconstruction abnormal situations caused by incorrect mapping.

[0085] A possible implementation of an embodiment of the present invention, when it is monitored that a disk goes online, further includes: determining whether the disk stores the original information of the redundant array of independent disks; if the disk stores the original information of the redundant array of independent disks, determining the RAID group of the disk according to the original information of the redundant array of independent disks of the disk; if the disk does not store the original information of the redundant array of independent disks, performing a full disk data reconstruction.

[0086] In a possible situation, the newly online disk may not be the original faulty disk, but a new disk, and at this time it does not store the RAID original information. At this time, a full disk data reconstruction can be performed so that the RAID group can be used normally.

[0087] In a possible implementation manner of an embodiment of the present invention, the data structure of the input-output mapping table is in the form of an array or a linked list.

[0088] In an embodiment of the present invention, when a link failure occurs in a certain member disk, subsequent IO information can be recorded and recorded in the IO mapping table. The structure of the mapping table is not limited to the form of an array, and can also be other data structures such as a linked list; the mapping value can also be 16 bits, 32 bits, 64 bits, etc. according to the actual situation, not limited to 32 bits, and the user can select according to actual needs.

[0089] In a possible implementation manner of an embodiment of the present invention, according to a plurality of strip groups to be reconstructed, data reconstruction is performed on the redundant array of independent disks (RAID) group after adding a disk, including: performing consistency verification on the target strip group to be reconstructed and the corresponding strip group of the disk, where the target strip group to be reconstructed is any one of the plurality of strip groups to be reconstructed; if the verification passes, data reconstruction of the target strip group to be reconstructed is not performed; if the verification fails, data reconstruction is performed on the redundant array of independent disks (RAID) group after adding a disk according to the target strip group to be reconstructed.

[0090] In an embodiment of the present invention, before performing data reconstruction on each strip group to be reconstructed, consistency verification can be performed to determine whether the data of the strip group before disk reconstruction is consistent with the data of the strip group to be reconstructed. If they are consistent, it means that no IO operation has been performed on this strip group; if they are inconsistent, it means that an IO operation has been performed and data reconstruction is required. Through the above mechanism, only the strip groups that really need to be reconstructed will be processed, thereby avoiding unnecessary reconstruction operations and improving the efficiency of data reconstruction.

[0091] An embodiment of the present invention further elaborates on the IO mapping table. The positioning of the IO mapping table structure includes: is_valid, grain_size, table_size, and map_table, where: is_valid is a bool type variable, true indicates that the IO mapping table is valid, and false indicates invalid; grain_size is a 32-bit unsigned integer variable, representing the strip group size, that is, the mapping granularity size, with the unit of strip; table_size is an n-bit unsigned integer variable, representing the size of the mapping table; map_table is a 32-bit unsigned integer array, and one of the bits in the array elements corresponds to a strip group on the RAID. A bit of 1 indicates that the IO falls on the corresponding strip group, otherwise it indicates that there is no IO information on this strip group.

[0092] The IO mapping table is stored in the RAID card chip. Since the access to the mapping table is on the IO path and needs to be accessed frequently and at high speed, the IO mapping table is required to have a fast read and write speed. It is recommended to store it in the memory with Cache. Each RAID group needs to have an IO mapping table corresponding to it. The size of the IO mapping table depends on the RAID size and the input / output mapping granularity setting. For example, if 5 disks of 1TB each form a RAID5 with a Chunk size of 256KB, then the size of each stripe is 1MB and the size of the RAID group is 4TB. If the input / output mapping granularity is set to 8 stripes, then the size of the IO mapping table is 64KB. The appropriate mapping granularity can be selected according to the actual needs and memory limitations.

[0093] Based on this, in a possible implementation manner of the embodiment of the present invention, before reading the input / output mapping table of the current redundant array of independent disks (RAID) group, it further includes: obtaining the capacity corresponding to the redundant array of independent disks group of the disk, the input / output mapping granularity, and the memory capacity, where the input / output mapping granularity represents the volume of each stripe group; determining the volume of the input / output mapping table according to the capacity corresponding to the redundant array of independent disks group and the input / output mapping granularity; if the volume of the input / output mapping table is greater than the memory capacity, then adjusting the capacity corresponding to the redundant array of independent disks group of the disk and the input / output mapping granularity to obtain the new capacity corresponding to the redundant array of independent disks group and the new input / output mapping granularity, and confirming the new volume of the input / output mapping table until the volume of the input / output mapping table is not greater than the memory capacity; if the volume of the input / output mapping table is not greater than the memory capacity, then setting the second quantity of the input / output mapping table according to the size of the input / output mapping table.

[0094] Among them, in the embodiment of the present invention, in a possible case, the capacity corresponding to the redundant array of independent disks group of the disk, that is, the RAID group capacity, is the total storage capacity of all disks in the RAID group minus the part used for redundancy and metadata storage. That is, if it is RAID5 and includes 5 member disks of 1Tb, then the RAID group capacity is 4Tb; in another possible case, the RAID group capacity can be specified by the user to be less than the theoretically RAID group capacity, such as 3Tb.

[0095] The input / output mapping granularity, that is, the IO mapping granularity, represents the size of each stripe group, that is, the minimum distribution unit of data in the RAID group. The memory capacity represents the memory capacity used to store metadata such as the IO mapping table.

[0096] Determine the input / output mapping table volume according to the capacity corresponding to the redundant array of independent disks (RAID) group and the input / output mapping granularity. Exemplarily, if 5 disks of 1TB form a RAID5 with a chunk size of 256KB, then the size of each stripe is 1MB. If the RAID group capacity is 4TB and the input / output mapping granularity is set to 8 stripes, then the size of the IO mapping table is 4TB / 8MB / 8bit = 64KB.

[0097] The memory capacity is preset; compare the calculated input / output mapping table volume with the memory capacity. If the IO mapping table volume is small, the number of bits of the mapping value can be directly determined according to the mapping table size. Specifically, any one of 16 bits, 32 bits, and 64 bits can be randomly selected as the mapping bit number. If the IO mapping table volume is large, it may affect the IO performance. At this time, it is necessary to reduce the RAID group capacity and / or increase the IO mapping granularity to reduce the size of the IO mapping table until its volume is not greater than the memory capacity.

[0098] It can be seen that in the embodiment of the present invention, three key parameters are obtained: the capacity of the redundant array of independent disks (RAID) group of the disk, the input / output mapping granularity, and the memory capacity; calculate and determine the input / output mapping table volume according to the capacity of the RAID group and the input / output mapping granularity; when it is found that the volume of the input / output mapping table exceeds the memory capacity, intelligently adjust the capacity of the RAID group or the input / output mapping granularity to reduce the volume of the input / output mapping table, and calculate the new input / output mapping table volume again until the volume no longer exceeds the memory capacity; when the volume of the input / output mapping table meets the memory capacity limit, set the mapping bit number of the mapping value according to the size of the input / output mapping table. Through the above technical solution, it is possible to intelligently adjust the capacity of the RAID group and the input / output mapping granularity according to the configuration of the RAID group and the memory capacity limit, thereby maximizing the data read / write performance while ensuring the stable operation of the system.

[0099] Specifically, in a possible implementation manner of the embodiment of the present invention, adjusting the capacity corresponding to the redundant array of independent disks (RAID) group of the disk and the input / output mapping granularity to obtain the new capacity corresponding to the redundant array of independent disks (RAID) group and the new input / output mapping granularity includes: determining the capacity adjustment parameter and the granularity adjustment parameter according to the mode selected by the user; adjusting the capacity corresponding to the redundant array of independent disks (RAID) group of the disk according to the capacity adjustment parameter to obtain the new capacity corresponding to the redundant array of independent disks (RAID) group; adjusting the input / output mapping granularity according to the granularity adjustment parameter to obtain the new input / output mapping granularity.

[0100] Among them, the user can pre-select a RAID mode to determine a set of capacity adjustment parameters and granularity adjustment parameters. The RAID mode at least includes: fine-grained rapid reconstruction and conventional reconstruction. When fine-grained rapid reconstruction is selected, the input / output mapping granularity is not adjusted, and only the capacity corresponding to the redundant array of independent disks (RAID) group of the disk is adjusted. When conventional reconstruction is selected, both can be adjusted together.

[0101] And the mapping relationship of resizing corresponding to different RAID group levels is set. After determining the mode and the current RAID group level, the capacity adjustment parameters and granularity adjustment parameters can be determined according to the mapping relationship; and the first adjustment is performed to obtain the capacity corresponding to the new redundant array of independent disks (RAID) group and the new input / output mapping granularity.

[0102] It can be seen that in the embodiment of the present invention, according to the RAID mode selected by the user, the capacity of the RAID group and the IO mapping granularity are automatically adjusted to determine the capacity corresponding to the new redundant array of independent disks (RAID) group and the new input / output mapping granularity while meeting the actual needs of the user.

[0103] In a possible implementation manner of the embodiment of the present invention, before determining multiple strip groups to be reconstructed that need to perform data reconstruction according to the input / output mapping table, it further includes: determining the total number of strip groups of the multiple strip groups to be reconstructed corresponding to the input / output mapping table; if the total number of strip groups is greater than a preset total threshold, then perform full-disk data reconstruction; if the total number of strip groups is not greater than the preset total threshold, then perform the step of determining multiple strip groups to be reconstructed that need to perform data reconstruction according to the input / output mapping table.

[0104] The total number of strip groups of the multiple strip groups to be reconstructed is also the total value of the bit value "1" in the input / output mapping table. When the size of the RAID group corresponding to the input / output mapping table is the size of a standard RAID or greater than a preset size, it means that the data volume corresponding to the input / output table is large; if the total adjustment amount is greater than the preset total threshold, the data volume to be reconstructed is large. At this time, a full-disk data reconstruction strategy can be executed, that is, all data in the entire RAID system is reconstructed and verified to ensure the integrity and reliability of the data. If the total number of strip groups is not greater than the preset total threshold, it means that the current data reconstruction task scale is small and the system resources are sufficient to handle it, and the step of determining multiple strip groups to be reconstructed that need to perform data reconstruction according to the input / output mapping table can be executed.

[0105] It can be seen that in the embodiments of the present invention, the scale of the current reconstruction task can be intelligently evaluated based on the total amount of stripe groups, and an appropriate data reconstruction strategy can be selected according to the evaluation results. Specifically, when the scale of the data reconstruction task is large, the full-disk data reconstruction strategy is selected to ensure the integrity and reliability of the data; when the scale of the data reconstruction task is small, the data reconstruction task is carefully planned according to the input-output mapping table to improve the efficiency and accuracy of data reconstruction.

[0106] In a possible implementation manner of the embodiments of the present invention, after data reconstruction is performed on the redundant array of independent disks (RAID) group after adding disks according to multiple stripe groups to be reconstructed, the method further includes: performing consistency verification on the data after disk reconstruction and the data in the redundant space; if the consistency verification is passed, it is confirmed that the data reconstruction is successful; if the consistency verification is not passed, a prompt message is generated.

[0107] Among them, consistency verification refers to the process of comparing and verifying the data after reconstruction and the data in the redundant space to ensure their consistency. The data in the redundant space refers to the data in the extra space reserved in the disk array to provide data redundancy and fault tolerance capabilities.

[0108] In the embodiments of the present invention, after the disk reconstruction operation is completed, it is necessary to verify the data after reconstruction to ensure its correctness and integrity. Specifically, consistency verification is performed on the data after disk reconstruction and the data in the redundant space to verify whether the data after reconstruction is consistent with the original data. If the data content between the two is exactly the same, it means that the data reconstruction operation is successful and the data has been successfully restored; if there are differences in the data content between the two, it means that there is a problem with the data reconstruction operation and further processing is required.

[0109] In some embodiments, the data after disk reconstruction and the data in the redundant space are read; a specific verification algorithm is used for verification and the corresponding verification value is generated; the verification values of these two parts of data are compared; if the verification values are consistent, it means that the data reconstruction is successful and the data consistency is verified; if the verification values are inconsistent, it means that there is a problem with the data reconstruction and a corresponding prompt message is generated for the user to refer to and process. To improve the verification efficiency, several data can be randomly selected as representatives for consistency verification. When all verifications are successful, it is determined that the reconstruction is successful; otherwise, the reconstruction fails.

[0110] It can be seen that in the embodiments of the present invention, consistency verification can be performed on the data after disk reconstruction and the data in the redundant space to ensure the reliability and accuracy of data reconstruction.

[0111] In a possible implementation manner of the embodiment of the present invention, it further includes: after the abnormal disk goes offline, determining a target disk with redundant space from the redundant array of independent disks (RAID) group where the abnormal disk is located; reconstructing the data of the abnormal disk according to the stripe group of the target disk and storing it in the redundant space, and using the redundant space to process the received new input / output information.

[0112] Among them, the redundant space refers to the additional storage space reserved in the RAID group to provide data redundancy and fault tolerance capabilities. When a certain disk in the RAID group fails, the redundant space can be used to store the reconstructed data to ensure data integrity and the normal operation of the system. A healthy disk refers to a disk that works normally and can be accessed in the RAID group.

[0113] Specifically, determining the RAID group where the abnormal disk is located, and identifying the healthy disks and redundant space in the RAID group; using the stripe groups on the healthy disks to reconstruct the data on the abnormal disk, and storing the reconstructed data in the redundant space; even if the abnormal disk cannot work properly, the data in the redundant space can still be used to respond to subsequent input / output requests, thereby ensuring the normal operation of the system and data integrity.

[0114] It can be seen that in the embodiment of the present invention, after the abnormal disk in the RAID system goes offline, the redundant space can be identified and the data of the abnormal disk can be reconstructed using the stripe groups on the healthy disks; the redundant space is used for data reconstruction and input / output processing; the performance and resource utilization rate of the system are improved.

[0115] In a possible implementation manner of the embodiment of the present invention, it further includes: obtaining historical data reconstruction information, where the data reconstruction information includes: the total amount of historical stripe groups, reconstruction time; constructing a relationship model of the total amount and time according to the total amount of historical stripe groups and the reconstruction time; determining a preset total amount threshold based on the relationship model of the total amount and time.

[0116] Among them, obtaining historical data reconstruction information from the database includes the total amount of stripe groups and the corresponding reconstruction time each time data is reconstructed. The historical data reconstruction information is not limited to the information of the RAID group of the current device, and can be the historical data reconstruction information under various working conditions of the same type of RAID group. The embodiment of the present invention no longer limits, as long as it can have diversity and sufficient data volume.

[0117] Using the collected historical data, a relationship model between the total amount of stripe groups and the reconstruction time is constructed. Specifically, a scatter plot of the total amount of stripe groups and the reconstruction time is drawn; according to the distribution trend of the scatter plot, a suitable mathematical model is selected for fitting to construct a relationship model of the total amount and time. Based on the constructed relationship model of the total amount and time, a preset threshold value of the total amount of stripe groups is determined. Specifically, the user can select the reconstruction requirement, such as the user-specified time, and determine the corresponding total amount from the model based on the user-specified time as the preset total amount threshold.

[0118] The following introduces a data reconstruction device provided by an embodiment of the present invention. The device described below can be correspondingly referred to the method described above. The device of this embodiment is set in an electronic device. Refer to Figure 5 , Figure 5 which is a structural block diagram of a data reconstruction device 200 according to an embodiment of the present invention, including: a first determination module 210, configured to determine an independent disk redundant array group of a disk according to the original information of the independent disk redundant array of the disk when it is detected that a disk is online; a judgment module 220, configured to determine whether the independent disk redundant array group of the disk is the current independent disk redundant array group; a reading module 230, configured to, if it is the current independent disk redundant array group, read the input-output mapping table of the current independent disk redundant array group, where the input-output mapping table records the input-output information after the disk is offline, the input-output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that there is input-output information written to the stripe group corresponding to the bit value; a second determination module 240, configured to determine a plurality of to-be-reconstructed stripe groups that need to be reconstructed according to the input-output mapping table; a data reconstruction module 250, configured to perform data reconstruction on the independent disk redundant array group after adding the disk according to the plurality of to-be-reconstructed stripe groups.

[0119] In a preferred example of the present invention, it can be further configured as follows: when the second determination module 240 determines a plurality of to-be-reconstructed stripe groups that need to be reconstructed according to the input-output mapping table, it is configured to: determine a target mapping value identifier whose mapping value in the input-output mapping table is not 0; determine a target stripe group identifier according to the target offset bit of the target mapping value identifier, where the target offset bit is the offset bit with a bit value of 1; and determine a plurality of to-be-reconstructed stripe groups that need to be reconstructed according to the target stripe group identifier.

[0120] In a preferred example, the present invention can be further configured to further include: a mapping table recording module, which is used for: after an abnormal disk goes offline, obtaining new input / output information; according to each location information of the new input / output information, calculating the input / output location of each location information in the input / output mapping table, wherein the location information of the new input / output information is composed of the starting location information to the ending location information of the new input / output information, and the input / output location includes: a mapping value identifier and an offset bit identifier; and recording the bit value of the input / output location of each location information of the new input / output information in the input / output mapping table as 1.

[0121] In a preferred example, the present invention can be further configured to: when the mapping table recording module calculates the input / output location of each location information in the input / output mapping table according to each location information of the new input / output information, it is used for: using a preset operation to sequentially calculate the mapping value identifier and the offset bit identifier of each location information of the new input / output information; wherein the preset operation includes: obtaining a stripe group identifier according to the location information of the new input / output information and the stripe group size; and determining the mapping value identifier and the offset bit identifier according to the stripe group identifier and the second quantity.

[0122] In a preferred example, the present invention can be further configured to: when the mapping table recording module calculates the input / output location of each location information in the input / output mapping table according to each location information of the new input / output information, it is used for: calculating the starting input / output location of the starting location information in the input / output mapping table according to the starting location information of the new input / output information; calculating the ending input / output location of the ending location information in the input / output mapping table according to the ending location information of the new input / output information; and using all the input / output locations between the starting input / output location and the ending input / output location as the input / output location of the new input / output information.

[0123] In a preferred example, the present invention can be further configured to further include: a marking module, which is used for: when the redundant array of independent disks group is normal, marking the input / output mapping table as an invalid state; when the redundant array of independent disks group is abnormal, marking the input / output mapping table as a valid state, and the valid state is used to indicate recording input / output information in the input / output mapping table.

[0124] In a preferred example, the present invention can be further configured to further include: a clearing module, which is used for: clearing all the bit values in the input / output mapping table.

[0125] In a preferred example, the present invention can be further configured as follows: after detecting that a disk is online, it further includes: a storage judgment module, configured to judge whether the original information of the redundant array of independent disks (RAID) is stored in the disk; a first determination module 210, configured to, if the original information of the RAID is stored in the disk, determine the RAID group of the disk according to the original information of the RAID of the disk; a data reconstruction module 250, further configured to, if the original information of the RAID is not stored in the disk, perform a full-disk data reconstruction.

[0126] In a preferred example, the present invention can be further configured as follows: the data structure of the input-output mapping table is in the form of an array or a linked list.

[0127] In a preferred example, the present invention can be further configured as follows: when the data reconstruction module 250 performs data reconstruction on the RAID group after adding a disk according to a plurality of strip groups to be reconstructed, it is used for: performing a consistency check on the target strip group to be reconstructed and the corresponding strip group of the disk, where the target strip group to be reconstructed is any one of the plurality of strip groups to be reconstructed; if the verification passes, no data reconstruction is performed on the target strip group to be reconstructed; if the verification fails, data reconstruction is performed on the RAID group after adding the disk according to the target strip group to be reconstructed.

[0128] In a preferred example, the present invention can be further configured as follows: it further includes: a mapping table making module, configured to: obtain the capacity, input-output mapping granularity, and memory capacity corresponding to the RAID group of the disk, where the input-output mapping granularity represents the volume of each strip group; determine the volume of the input-output mapping table according to the capacity and input-output mapping granularity corresponding to the RAID group; if the volume of the input-output mapping table is greater than the memory capacity, adjust the capacity and input-output mapping granularity corresponding to the RAID group of the disk to obtain the new capacity corresponding to the RAID group and the new input-output mapping granularity, and confirm the new volume of the input-output mapping table until the volume of the input-output mapping table is not greater than the memory capacity; if the volume of the input-output mapping table is not greater than the memory capacity, set the second quantity of the input-output mapping table according to the size of the input-output mapping table.

[0129] In a preferred example, the present invention can be further configured as follows: when the mapping table making module adjusts the capacity and input-output mapping granularity corresponding to the RAID group of the disk to obtain the new capacity corresponding to the RAID group and the new input-output mapping granularity, it is configured to: determine the capacity adjustment parameter and the granularity adjustment parameter according to the mode selected by the user; adjust the capacity corresponding to the RAID group of the disk according to the capacity adjustment parameter to obtain the new capacity corresponding to the RAID group; adjust the input-output mapping granularity according to the granularity adjustment parameter to obtain the new input-output mapping granularity.

[0130] In a preferred example, the present invention can be further configured as follows: It further includes: a third determination module, configured to determine the total number of stripe groups of a plurality of stripe groups to be reconstructed corresponding to the input-output mapping table; a data reconstruction module 250, further configured to, if the total number of stripe groups is greater than a preset total threshold, perform full-disk data reconstruction; if the total number of stripe groups is not greater than the preset total threshold, perform the step of determining a plurality of stripe groups to be reconstructed that require data reconstruction according to the input-output mapping table.

[0131] In a preferred example, the present invention can be further configured as follows: It further includes: a consistency verification module, configured to perform consistency verification on the data after reconstructing the disk and the data in the redundant space; if the consistency verification is passed, confirm that the data reconstruction is successful; if the consistency verification is not passed, generate a prompt message.

[0132] In a preferred example, the present invention can be further configured as follows: It further includes: a fourth determination module, configured to, after an abnormal disk goes offline, determine a target disk with redundant space from the redundant array of independent disks (RAID) group where the abnormal disk is located; a redundancy processing module, configured to reconstruct the data of the abnormal disk according to the stripe group of the target disk and store it in the redundant space, and use the redundant space to process the received new input-output information.

[0133] In a preferred example, the present invention can be further configured as follows: It further includes: a threshold determination module, configured to obtain historical data reconstruction information, where the data reconstruction information includes: historical total number of stripe groups, reconstruction time; construct a relationship model of total amount and time according to the historical total number of stripe groups and the reconstruction time; based on the relationship model of total amount and time, determine the preset total threshold.

[0134] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention, as Figure 6 shown, the electronic device includes: a memory 60, configured to store a computer program; a processor 61, configured to implement the steps of the data reconstruction method in the above embodiment when executing the computer program.

[0135] The electronic device provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a laptop computer, or a desktop computer, etc.

[0136] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake state, also known as the central processing unit (CPU); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may further include an artificial intelligence (AI) processor, and the AI processor is used to process computational operations related to machine learning.

[0137] The memory 60 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 60 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601. After the computer program is loaded and executed by the processor 61, it can implement the relevant steps of the data reconstruction method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 60 may further include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc.

[0138] In some embodiments, the electronic device may further include a display screen 62, an input / output interface 63, a communication interface 64, a power supply 65, and a communication bus 66.

[0139] Those skilled in the art can understand that Figure 6 the structure shown in

[0140] It can be understood that if the data reconstruction method in the above embodiments is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), electrically erasable programmable ROMs, registers, hard disks, removable disks, CD-ROMs, magnetic disks, or optical discs.

[0141] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data reconstruction method as described above are implemented.

[0142] An embodiment of the present invention provides a computer program product, including a computer program or instruction. When the computer program or instruction is executed by a processor, the corresponding content in the foregoing method embodiment is implemented.

[0143] The above has introduced in detail a data reconstruction method, device, equipment, medium, and product provided by the embodiments of the present invention. The various embodiments in the specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0144] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0145] The above has introduced in detail a data reconstruction method, apparatus, device, medium and product provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A data reconstruction method, characterized in that: include: When it is detected that a disk is online, the independent disk redundant array group of the disk is determined according to the independent disk redundant array original information of the disk; Determining whether the redundant array of independent disks group of the disk is a current redundant array of independent disks group; If it is the current redundant array of independent disks group, read the input-output mapping table of the current redundant array of independent disks group, wherein the input-output mapping table records the input-output information after the disk is offline, the input-output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that the input-output information is written into the stripe group corresponding to the bit value; Determining, according to the input-output mapping table, a plurality of stripe groups to be reconstructed that require data reconstruction; Reconstructing data on the independent redundant array of disks group after the disk is added according to the multiple stripe groups to be reconstructed; Determining a plurality of to-be-reconstructed stripe groups requiring data reconstruction according to the input-output mapping table includes: Determine a target mapping value identifier whose mapping value of the input-output mapping table is not 0; Determine a target stripe group identifier according to a target offset bit of the target mapping value identifier, wherein the target offset bit is an offset bit having a bit value of 1; According to the target stripe group identifier, a plurality of stripe groups to be reconstructed that require data reconstruction are determined.

2. The data reconstruction method according to claim 1, characterized in that: Also includes: When the abnormal disk goes offline, obtain new input and output information; According to each position information of the new input / output information, the input / output position of each position information in the input / output mapping table is calculated, wherein the position information of the new input / output information is composed of the starting position information to the ending position information of the new input / output information, and the input / output position includes: a mapping value identifier and an offset bit identifier; The bit value of each position information of the new input / output information at the input / output position in the input / output mapping table is recorded as 1.

3. The data reconstruction method according to claim 2, characterized in that: According to each position information of the new input and output information, calculating the input and output position of each position information in the input and output mapping table includes: Using the preset operation, the mapping value identifier and the offset bit identifier of each position information of the new input and output information are calculated in sequence; The preset operation includes: obtaining a stripe group identifier according to the location information of the new input / output information and the stripe group size; and determining a mapping value identifier and an offset bit identifier according to the stripe group identifier and the second quantity.

4. The data reconstruction method according to claim 2, characterized in that: According to each position information of the new input and output information, calculating the input and output position of each position information in the input and output mapping table includes: Calculate the starting input / output position of the starting position information in the input / output mapping table according to the starting position information of the new input / output information; Calculate the end input / output position of the end position information in the input / output mapping table according to the end position information of the new input / output information; All input-output positions between the start input-output position and the end input-output position are used as the input-output positions of the new input-output information.

5. The data reconstruction method according to claim 1, characterized in that: Also includes: When the redundant array of independent disks group is normal, marking the input-output mapping table as an invalid state; When the redundant array of independent disks group is abnormal, the input-output mapping table is marked as a valid state, and the valid state is used to indicate that the input-output information is recorded in the input-output mapping table.

6. The data reconstruction method according to claim 1, characterized in that: After reconstructing data on the independent redundant array of disks group after the disk is added according to the multiple stripe groups to be reconstructed, the method further includes: clearing all bit values ​​in the input-output mapping table.

7. The data reconstruction method according to claim 1, characterized in that: When a disk is detected to be online, the following is also included: Determining whether the disk stores original information of a redundant array of independent disks; If the disk stores original information of the redundant array of independent disks, determining the redundant array of independent disks group of the disk according to the original information of the redundant array of independent disks of the disk; If the original information of the redundant array of independent disks is not stored in the disk, full disk data reconstruction is performed.

8. The data reconstruction method according to claim 1, characterized in that: The data structure of the input-output mapping table is in the form of an array or a linked list.

9. The data reconstruction method according to any one of claims 1 to 8, characterized in that: According to the plurality of stripe groups to be reconstructed, data is reconstructed on the independent redundant array of disks group after the disk is added, including: Performing a consistency check between a target stripe group to be reconstructed and a corresponding stripe group of the disk, wherein the target stripe group to be reconstructed is any stripe group among the multiple stripe groups to be reconstructed; If the verification is successful, data reconstruction of the target stripe group to be reconstructed will not be performed; If the verification fails, data is reconstructed on the redundant array of independent disks group after the disk is added according to the target stripe group to be reconstructed.

10. The data reconstruction method according to any one of claims 1 to 8, characterized in that: Before reading the input and output mapping table of the current redundant array of independent disks group, the following steps are also included: Obtaining the capacity, input-output mapping granularity and memory capacity corresponding to the independent disk redundant array group of the disk, wherein the input-output mapping granularity represents the volume of each stripe group; determining the volume of the input-output mapping table according to the capacity and input-output mapping granularity corresponding to the independent disk redundant array group; If the volume of the input-output mapping table is larger than the memory capacity, the capacity corresponding to the redundant array of independent disks group of the disk and the input-output mapping granularity are adjusted to obtain the capacity corresponding to the new redundant array of independent disks group and the new input-output mapping granularity, and the new volume of the input-output mapping table is confirmed until the volume of the input-output mapping table is no larger than the memory capacity; If the volume of the input-output mapping table is not greater than the memory capacity, the second number of input-output mapping tables is set according to the size of the input-output mapping table.

11. The data reconstruction method according to claim 10, characterized in that: Adjusting the capacity and input / output mapping granularity corresponding to the independent redundant array of disks group of the disk to obtain the capacity and new input / output mapping granularity corresponding to the new independent redundant array of disks group includes: Determine capacity adjustment parameters and granularity adjustment parameters according to the mode selected by the user; Adjust the capacity corresponding to the redundant array of independent disks group of the disk according to the capacity adjustment parameter to obtain the capacity corresponding to the new redundant array of independent disks group; The input-output mapping granularity is adjusted according to the granularity adjustment parameter to obtain a new input-output mapping granularity.

12. The data reconstruction method according to any one of claims 1 to 8, characterized in that: Before determining a plurality of to-be-reconstructed stripe groups that need to be reconstructed according to the input-output mapping table, the method further includes: Determine the total number of stripe groups of the plurality of stripe groups to be reconstructed corresponding to the input-output mapping table; If the total amount of the stripe group is greater than a preset total amount threshold, performing full disk data reconstruction; If the total amount of the stripe groups is not greater than the preset total amount threshold, a step of determining a plurality of stripe groups to be reconstructed that require data reconstruction is performed according to the input-output mapping table.

13. The data reconstruction method according to claim 12, characterized in that: After reconstructing data on the independent disk redundant array group after adding the disk according to the multiple stripe groups to be reconstructed, the method further includes: Performing consistency verification on the reconstructed data of the disk and the data in the redundant space; If the consistency verification is passed, the data reconstruction is confirmed to be successful; If the consistency verification fails, a prompt message is generated.

14. The data reconstruction method according to claim 13, characterized in that: Also includes: When the abnormal disk goes offline, a target disk with redundant space is determined from the independent disk redundant array group where the abnormal disk is located; The data of the abnormal disk is reconstructed according to the stripe group of the target disk and stored in the redundant space, and the received new input and output information is processed by using the redundant space.

15. The data reconstruction method according to claim 12, characterized in that: Also includes: Obtaining historical data reconstruction information, the data reconstruction information including: the total amount of historical stripe groups and reconstruction time; Constructing a total amount time relationship model according to the total amount of the historical stripe group and the reconstruction time; Based on the relationship model of the total amount time, a preset total amount threshold is determined.

16. A data reconstruction device, characterized in that: include: A first determination module is used to determine the independent disk redundant array group of the disk according to the independent disk redundant array original information of the disk after monitoring that a disk is online; A judgment module, used for determining whether the independent disk redundant array group of the disk is a current independent disk redundant array group; A reading module is used to read an input-output mapping table of the current redundant array of independent disks group if it is a current redundant array of independent disks group, wherein the input-output mapping table records the input-output information of the disk after it goes offline, the input-output mapping table is composed of a first number of mapping values, each mapping value is composed of a second number of bit values, and when the bit value is 1, it indicates that the input-output information is written into the stripe group corresponding to the bit value; A second determination module, configured to determine a plurality of to-be-reconstructed stripe groups for which data reconstruction is required according to the input-output mapping table; A data reconstruction module, used for reconstructing data of the independent disk redundant array group after the disk is added according to the multiple stripe groups to be reconstructed; A second determination module, configured to determine a target mapping value identifier whose mapping value in the input-output mapping table is not 0; Determine a target stripe group identifier according to a target offset bit of the target mapping value identifier, wherein the target offset bit is an offset bit having a bit value of 1; According to the target stripe group identifier, a plurality of stripe groups to be reconstructed that require data reconstruction are determined.

17. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the data reconstruction method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data reconstruction method according to any one of claims 1 to 15 are implemented.

19. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the data reconstruction method according to any one of claims 1 to 15 are implemented.

Images