Method for off-line fast transfer of large amounts of data using disk array

Abstract

The application provides a method, device and equipment for quickly transferring a large amount of data offline by using a disk array, and relates to the technical field of communication.The method comprises the following steps: in response to the insertion of a mobile hard disk, reconstructing a disk array according to an array slot and hard disk correspondence table, wherein the mobile hard disk comprises data to be transferred, the disk array and / or the reconstructed disk array is a RAID51 mode disk array, and the array slot and hard disk correspondence table are pre-stored in the disk array; and in response to a user input instruction, synchronizing the data to be transferred to the reconstructed disk array.The application designs a RAID51 disk array and a correspondence table for transferring data, and realizes reliable and fast data transfer by combining the hot plug characteristics of the disk array, and has good stability.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a method for rapidly transferring large amounts of data offline using a disk array. Background Technology

[0002] Currently, data transfer is primarily achieved through copying data to external hard drives. External hard drives mainly refer to small, portable hard drive storage devices that use USB or IEEE 1394 interfaces, allowing for easy plugging and unplugging and enabling high-speed data transfer with the system. Among these, the USB 3.0 interface offers a higher transfer rate. Therefore, when transferring large amounts of data, the USB 3.0 external hard drive interface is superior to USB 1.1, USB 2.0, and IEEE 1394 interfaces.

[0003] However, any type of external hard drive requires data to be copied twice when transferring data: from the existing storage device to the external hard drive, and then from the external hard drive to the storage device, and the data needs to be offline. Furthermore, while the theoretical transfer rate of a USB 3.0 interface is 625MB / s, the actual current rate is still less than 200MB / s.

[0004] Therefore, existing data transfer methods using external hard drives still suffer from problems when transferring large amounts of data (terabytes in size). These problems include limited capacity of the external hard drive, relatively small data transfer bandwidth compared to the amount of data, long opening and closing time of data files, long data copying time, and low data transfer efficiency. Summary of the Invention

[0005] This invention provides a method, device, and medium for rapidly transferring large amounts of data offline using a disk array.

[0006] In a first aspect, the present invention provides a method for rapidly transferring large amounts of data offline using a disk array, comprising: in response to the insertion of a portable hard drive, reconstructing a disk array according to an array slot and hard drive lookup table, wherein the portable hard drive contains data to be transferred, and the disk array and / or the reconstructed disk array is a RAID51 disk array, and the array slot and hard drive lookup table is pre-stored in the disk array; in response to a user input command, synchronizing the data to be transferred to the reconstructed disk array; wherein the disk array and / or the reconstructed disk array includes at least one RAID1 subgroup, the subgroup being used to store parity information of data from other subgroups, each subgroup consisting of at least one hard drive, and at least one hard drive in the subgroup being a mirror image of each other.

[0007] Furthermore, prior to the method, the method further includes: constructing a lookup table of array slots and hard disk numbers, and storing the lookup table in the disk array; wherein, the array slot represents the number of the mounting slot corresponding to the installation of the portable hard disk in the disk array cabinet, and the disk array determines the hard disk to be accessed by using the array slot when reading and writing hard disks; the hard disk number is a number generated based on the position of the hard disk in the subgroup when constructing the subgroup.

[0008] Furthermore, the step of reconstructing the disk array based on the array slot and hard drive number lookup table includes: after the external hard drive is inserted into the disk array, the disk array generates a new array slot and hard drive number lookup table based on the array slot and hard drive number lookup table; and reconstructs the disk array based on the new array slot and hard drive number lookup table.

[0009] Furthermore, the step of reconstructing the disk array according to the new array slot and hard drive number lookup table includes: storing the new array slot and hard drive number lookup table in the portable hard drive and the disk array to obtain the reconstructed disk array.

[0010] Furthermore, synchronizing the data to be transferred to the reconstructed disk array includes: when the data to be transferred is written to the reconstructed disk array, striping and parity calculation are performed on the data to be transferred, and then the data is written to the subgroups respectively; in the subgroups, at least one hard disk of the mirror image is written simultaneously.

[0011] Furthermore, the method also includes: during data reading, reading the corresponding data from any hard drive within the subgroup.

[0012] Secondly, the present invention provides a data transfer device based on a disk array, comprising: a first processing module, configured to reconstruct a disk array according to an array slot and hard drive lookup table in response to the insertion of a portable hard drive, wherein the portable hard drive contains data to be transferred, and the disk array and / or the reconstructed disk array is a RAID51 disk array, and the array slot and hard drive lookup table is pre-stored in the disk array; and a second processing module, configured to synchronize the data to be transferred to the reconstructed disk array in response to a user input command; wherein the disk array and / or the reconstructed disk array includes at least one RAID1 subgroup, the subgroup being used to store parity information of data from other subgroups, each subgroup consisting of at least one hard drive, and at least one hard drive in the subgroup being a mirror image of each other.

[0013] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for rapidly transferring large amounts of data offline using a disk array as described above.

[0014] Fourthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described methods for rapidly transferring large amounts of data offline using a disk array.

[0015] The present invention provides a method for offline rapid transfer of large amounts of data using a disk array, which has the following advantages: The present invention designs a RAID51 disk array for data transfer, and achieves more stable and reliable data transfer based on the characteristics of the RAID51 disk array; in addition, by combining the array slot and hard drive mapping table with the hot-swappable characteristics of the disk array, reliable and rapid data transfer is achieved. When transferring large amounts of data (TB-level data volume), the present invention can stably, reliably, and quickly complete the data transfer while ensuring normal data operation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a RAID 0 application scenario;

[0018] Figure 2 This is a schematic diagram of a RAID1 application scenario;

[0019] Figure 3 This is a schematic diagram of a RAID 5 application scenario;

[0020] Figure 4 This is a schematic diagram of a RAID 10 application scenario;

[0021] Figure 5 This is a schematic diagram of a RAID 51 application scenario;

[0022] Figure 6 A flowchart illustrating some embodiments of the method for rapidly transferring large amounts of data offline using a disk array provided by the present invention;

[0023] Figure 7 Schematic diagrams of some embodiments of the data transfer device based on a disk array provided by the present invention;

[0024] Figure 8 This is a schematic diagram of the structure of an electronic device provided according to the present invention. Detailed Implementation

[0025] For storing large amounts of data, RAID (Redundant Array of Independent Disks) is the primary method.

[0026] Simply put, RAID is a technology that combines multiple independent hard drives (physical hard drives) in different ways to form a hard drive group (logical hard drive), thereby providing higher storage performance and data backup technology than a single hard drive.

[0027] A disk array consists of multiple hard drives, therefore the probability of a disk array failing is N times greater than that of a single hard drive, where N is the number of hard drives in the array. To improve the reliability of a disk array and ensure no data loss, redundant hard drives are added. Different ways of creating a disk array with redundancy are called RAID levels, and there are currently seven basic RAID levels, from RAID 0 to RAID 6. In addition, there are combinations of basic RAID levels, such as RAID 10 (a combination of RAID 0 and RAID 1), etc.

[0028] RAID technology has the following three basic functions:

[0029] (1) By striping the data on the disk, data can be accessed in blocks, reducing the mechanical seek time of the disk and improving the data access speed.

[0030] (2) By reading several disks in an array at the same time, the mechanical seek time of the disks is reduced and the data access speed is improved.

[0031] (3) By mirroring or storing parity information, data redundancy protection is achieved, which improves reliability.

[0032] The most commonly used RAID configurations are: RAID 0, RAID 1, RAID 5, and RAID 10.

[0033] 1) RAID 0 (also known as striped volume, offering the best performance)

[0034] RAID 0 technology connects multiple physical hard drives (at least two) together via hardware or software to form a large volume group, and writes data to each physical hard drive at once. Figure 1 As shown. This will improve the read and write performance of the hard drive several times over, but if any one of the hard drives fails, the data in the entire system will be corrupted.

[0035] Advantages: Using n hard drives, you can have nearly n times the read and write performance, which is the highest among all RAID levels.

[0036] Disadvantages: It does not provide data redundancy, so once user data is damaged, the damaged data cannot be recovered. When RAID0 is running, if any one of the hard drives fails, the entire data will be lost.

[0037] 2) RAID 1 (also known as mirrored volume, full backup)

[0038] RAID 1 is a RAID disk array consisting of two hard drives, such as... Figure 2 As shown, its capacity is only equal to that of one hard drive, because the other is only used as a data "mirror".

[0039] Advantages: Security increases exponentially with the number of physical hard drives in the array.

[0040] Disadvantage: It has the least efficient space utilization of all RAID configurations. Disk space utilization: 50%.

[0041] 3) RAID 5 (balancing performance and data backup considerations)

[0042] RAID 5 technology stores parity information from multiple hard drives (at least three) across the other hard drives. In a RAID 5 array, parity information isn't stored on a single drive, but rather on every single drive except the RAID 1 itself. This design prevents a fatal failure if any one drive fails. Figure 3 As shown.

[0043] RAID 5 does not back up the stored data. Instead, it stores the data and corresponding parity information across the individual disks that make up the RAID 5 array, with the parity information and corresponding data stored on separate disks. When one disk in a RAID 5 array fails, the remaining data and the corresponding parity information are used to recover the damaged data.

[0044] Advantages: It balances space utilization and safety.

[0045] Disadvantages: Requires additional computing resources and can only tolerate the failure of one hard drive.

[0046] 4) RAID 10

[0047] RAID 10 technology is a combination of RAID 1 and RAID 0 technologies. For example... Figure 4RAID 10 technology requires at least four hard drives to build. First, two drives are paired up to form a RAID 1 array to ensure data security. Then, RAID 0 technology is applied to the two RAID 1 arrays to further improve the read and write speeds of the hard drives. Theoretically, as long as the failure doesn't occur in all drives within the same group, up to 50% of the hard drives can be damaged without data loss. RAID 10 technology inherits the high read and write speeds of RAID 0 and the data security of RAID 1.

[0048] Current disk arrays only consider data storage. Although most disk arrays have hot-swapping capabilities (hot-swapping means quickly removing or installing a hard drive while the disk array is working), due to system design reasons, if a hard drive removed from one disk array is inserted into another disk array, it cannot be read because the disk array information is missing.

[0049] Using RAID 10 is one solution for data movement. During data transfer, a set of disks from a mirrored RAID 0 subgroup is extracted and used for data transfer. However, due to the lack of redundancy in this array, overall reliability is low. As mentioned earlier, RAID 0 suffers from the drawback of any single disk failure, leading to data transfer failure. The problem with RAID 15 is that after mirroring, the two RAID 5 subgroups perform parity checks twice (which can be optimized to once), reducing write efficiency.

[0050] This invention provides a mapping table for RAID51 disk arrays and array slots (array slots are used to insert hard drives; multiple hard drives inserted into slots can form a disk array) for data transfer. Combined with the hot-swappable feature of disk arrays, it achieves reliable and fast data transfer with good stability.

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0052] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0053] It should be noted that the concepts of "first" and "second" mentioned in this invention are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0054] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0055] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0056] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0057] Please see Figure 6 , Figure 6 This is a flowchart illustrating some embodiments of the method for rapidly transferring large amounts of data offline using a disk array provided by the present invention. For example... Figure 6 As shown, the method includes the following steps:

[0058] Step 601: In response to the insertion of a portable hard drive, the disk array is reconstructed according to the array slot and hard drive lookup table. The portable hard drive contains data to be transferred. The disk array and / or the reconstructed disk array is a RAID51 disk array. The array slot and hard drive lookup table is pre-stored in the disk array.

[0059] As an example, a disk array in RAID51 mode is as follows: Figure 5 As shown:

[0060] RAID 51 is composed of RAID 5 and RAID 1. That is, a RAID 5 group consists of multiple RAID 1 sub-RAID groups. Each RAID 1 sub-group consists of two hard drives, such as disk1 and disk2 forming a RAID 1 sub-group. When writing data, the data is striped and parity-calculated before being written to the respective RAID 1 sub-groups. Within each sub-group, the data is simultaneously written to two mirrored hard drives. When reading data, the controller (or processor) treats each RAID 1 sub-group as one hard drive and reads the data from any one of the hard drives within the RAID 1 sub-group. Thus, the write and read efficiency of RAID 51 is the same as RAID 5. Even if any hard drive in a RAID 1 sub-group is removed, the system can still function normally. Therefore, one hard drive from each RAID 1 sub-group in a RAID 51 configuration can be removed for data transfer without affecting the operation of the entire RAID group.

[0061] RAID 51 adds array information (i.e., a table of array slots and hard drives) to each hard drive for data transfer. After a hard drive containing data is inserted into the RAID array, the array can reconstruct its information based on the inserted disk, enabling the retrieval of the moved data. Simultaneously, the data redundancy in RAID 51 ensures the reliability of the data used for transfer. This allows for the fast and reliable transfer of large amounts of data. Furthermore, RAID 51 allows for the simultaneous replication of two RAID 5 data arrays without increasing computational load (i.e., only requiring a single parity check calculation on the RAID group's data).

[0062] In this context, an array slot is a reserved location on a storage device to accommodate hard drives (or disks). Each hard drive corresponds to one slot, and each slot has its own slot number. Multiple slots form an array slot (or disk array cabinet). Multiple hard drives (i.e., disks) inserted into an array slot form an array (or disk array). Each hard drive may also have its own hard drive number.

[0063] The array slot and hard drive mapping table records each hard drive in the current disk array and its corresponding slot. In some embodiments, before the method, it may further include: constructing an array slot and hard drive number mapping table, and storing the array slot and hard drive number mapping table in the disk array; wherein, the array slot represents the number of the mounting slot corresponding to the disk array enclosure where the external hard drive is installed, and the disk array determines the hard drive to be accessed by the array slot when reading and writing hard drives; the hard drive number is a number generated according to the position of the hard drive in the subgroup when constructing the subgroup. As an example, the storage device can automatically generate the array slot and hard drive number mapping table (for example, based on information from sensors and / or manual input in the storage device, combined with software to automatically generate the array slot and hard drive number mapping table; of course, the array slot and hard drive number mapping table can also be manually modified). The automatic generation of the array slot and hard drive number mapping table based on the hard drive number and array information enables data transfer hard drives to be inserted into the array at will without affecting the function of the disk array. Each hard drive stores array information and hard drive number, enabling the receiving array to reconstruct the array based on the hard drive information.

[0064] As an example, an array slot is the number of the mounting slot corresponding to the hard drive in the disk array enclosure, used for accessing the hard drive; a hard drive number is a number generated when building a RAID group based on the hard drive's position in the RAID group, such as... Figure 5 In the disk array enclosure, disk1 is used to identify hard drives by their slot numbers when reading and writing to them. After a hard drive used for data movement is inserted into the enclosure, the enclosure regenerates a hard drive number mapping table for each slot based on the hard drive number and slot. Figure 5Disk1 and Disk2 are automatically grouped into a RAID1 subgroup. This means that the order in which hard drives used for data movement are inserted into the array enclosure is not strictly required. For example, taking a 12-slot disk array as an example, we can set up one RAID51 group, group number 0, with 6 RAID1 subgroups within the group, numbered from 0 to 5. The array's hard drive slots are numbered from 1 to 12. The array hard drive slot and hard drive number mapping table is shown in Table 1:

[0065] Table 1. Array Hard Drive Slot and Hard Drive Number Correspondence Table

[0066]

[0067]

[0068] The first row in Table 1 indicates that the RAID 51 disk array is numbered 0. This array contains a subgroup numbered 0, which has two slots numbered 1 and 2. The hard drives inserted into these slots are numbered 0001101 and 0001202, respectively. The remaining rows have similar explanations. Except for the slot number, the data in each row of the table is written to the corresponding hard drive for data transfer.

[0069] In some embodiments, the process of reconstructing the disk array according to the array slot and hard disk number lookup table further includes: after a portable hard drive is inserted into the disk array, the disk array generates a new array slot and hard disk number lookup table according to the array slot and hard disk number lookup table; and reconstructs the disk array according to the new array slot and hard disk number lookup table.

[0070] When a new hard drive is inserted into another empty slot, the storage device reads the number of the newly inserted hard drive and the slot number of the newly inserted hard drive, and adds (or integrates) the number of the newly inserted hard drive and the slot number of the newly inserted hard drive into the original array slot and hard drive mapping table to obtain an updated array slot and hard drive mapping table. The updated array slot and hard drive mapping table is then stored in each hard drive, thus obtaining a reconstructed disk array.

[0071] As an example, if Table 1 is a lookup table containing the data to be transferred and Table 2 is a lookup table for receiving the data to be transferred, then the array slot lookup table for the disk array used to receive the data is shown in Table 2: half of the slots in the RAID group are used for the hard drives that receive the transferred data.

[0072] Table 2. Reference Table of Hard Disk Slots and Hard Disk Numbers for Receiver Array

[0073]

[0074]

[0075] In Table 1, hard drives in even-numbered slots are removed from the disk array for data transfer. After random insertion into the receiving array, hard drive numbered 0004208, belonging to RAID subgroup 3, is inserted into slot 2. The array reads the array information stored in the hard drive inserted into slot 2, obtaining the hard drive number 0004208, which belongs to RAID 0 group and RAID 3 subgroup. This information is then inserted into the array hard drive slot and hard drive number lookup table. The operation for the remaining inserted hard drives is the same as above.

[0076] The updated Table 2 is shown in Table 3. Table 3 is the updated table of array hard disk slots and hard disk numbers for receiving and transferring data.

[0077] Table 3

[0078]

[0079]

[0080] In this new array, compared to Table 1, subgroup 0 consists of hard drives from slots 1 and 6, subgroup 3 consists of hard drives from slots 2 and 7, and subgroup 2 consists of hard drives from slots 5 and 8. This allows for a less strict requirement on the order in which hard drives are inserted.

[0081] In some embodiments, reconstructing the disk array according to the new array slot and hard drive number lookup table may include: storing the new array slot and hard drive number lookup table in the portable hard drive and the disk array to obtain the reconstructed disk array.

[0082] The array configuration information includes the RAID mode, the RAID group to which the current hard drive belongs, the RAID subgroup, the array slot and hard drive number lookup table, and the hard drive number. This information is written to each disk so that when the disk array is reconfigured, i.e., when a hard drive used for moving data is inserted into the disk array cabinet, the disk array cabinet (or other electronic equipment) can identify which RAID5 group the hard drive belongs to, and which RAID1 subgroup under the RAID5 group.

[0083] Step 602: In response to the user input command, the data to be transferred is synchronized to the reconstructed disk array.

[0084] The disk array and / or the reconstructed disk array includes at least one RAID1 subgroup, which is used to store parity information of data from other subgroups. Each subgroup consists of at least one hard drive, and at least one hard drive in the subgroup is a mirror image of the others.

[0085] In some embodiments, synchronizing the data to be transferred to the reconstructed disk array may include: when the data to be transferred is written to the reconstructed disk array, striping and parity calculation are performed on the data to be transferred, and then the data is written to the subgroups respectively; in the subgroups, at least one hard disk of the mirror image is written simultaneously.

[0086] In some embodiments, the method may further include: during data reading, reading the corresponding data from any hard disk within the subgroup.

[0087] When one or more hard drives used for data movement are inserted into the RAID array, the RAID array detects the new hard drives and reconstructs the RAID 51 array based on the information of the inserted hard drives. Based on the RAID 51 array reconstruction, the user is prompted to choose between rebuilding or transferring the data (based on the user's input). Rebuilding involves reading all data from the hard drives already on the RAID array and copying it entirely to the inserted hard drives; transferring involves reading the data from the inserted hard drives and copying it entirely to the hard drives already on the RAID array.

[0088] Specifically, data transfer (or reconstruction) leverages the RAID 1 mode of the reconstructed disk array. During data generation, data is simultaneously written to two mirrored hard drives, creating a copy of the data at the same time as writing it. Furthermore, RAID 1's data recovery function is used to quickly replicate the data during the transfer. Secondly, the reconstructed disk array uses RAID 5 mode to generate redundant parity data. If one hard drive in the RAID group fails during data transfer, it will not affect the normal data transfer, improving data reliability and preventing data corruption caused by errors discovered during the transfer process.

[0089] In one application scenario, data transfer can be achieved using two disk arrays: one installed on an aircraft to store the generated data (source array), and another installed at the data processing site to receive and process the data. The source array, depending on its capacity, consists of one to n RAID51 arrays (or other types such as RAID50, RAID10, etc.). Each RAID51 array in the source array comprises 10-16 hard drives, paired into RAID1 groups, with one hard drive (or portable hard drive) used for data transfer. Thus, each RAID51 array in the source array has 5-8 RAID1 subgroups, typically offering optimal RAID5 efficiency. The number of array groups in the receiving array is similar to that of the source data disk array.

[0090] When transferring data from a hard drive to the source array, it can be written simultaneously to two hard drives mirrored in each subgroup of the source array. When data transfer is needed, one of the two hard drives from each RAID1 subgroup in the source array is removed, transported to the receiving array, and inserted into it to complete the data transfer. The receiving array reconstructs the RAID51 array based on the array information stored in the inserted hard drive and synchronously copies the data from the transferred hard drive to the hard drives in the receiving array. Because multiple hard drives can be copied simultaneously, the copying time is comparable to copying a single hard drive. Furthermore, since the hard drives in the RAID1 subgroup are mirrored, data reading and copying can occur simultaneously, improving data transfer speed without affecting the normal operation of the array. Moreover, RAID5 redundancy ensures that if any hard drive in the RAID group fails during the transfer process, data loss will not occur.

[0091] It should be noted that when a hard drive in the first disk array is removed and inserted into the second disk array, the reconstruction method for both the first and second disk arrays is similar. The data in the second disk array and on the hard drive can be used and copied using the methods described above.

[0092] Please see Figure 7 , Figure 7 These are schematic diagrams illustrating some embodiments of a data transfer device based on a disk array according to the present invention. As an implementation of the methods shown in the above figures, the present invention also provides some embodiments of a data transfer device based on a disk array, which are similar to... Figure 6 The embodiments of some of the methods shown correspond to this, and the device can be applied to a variety of electronic devices.

[0093] like Figure 7 As shown, a data transfer device 700 based on a disk array in some embodiments includes a first processing module 701 and a second processing module 702: the first processing module is used to reconstruct the disk array according to the array slot and hard disk lookup table in response to the insertion of a portable hard drive, wherein the portable hard drive contains data to be transferred, and the disk array and / or the reconstructed disk array is a RAID51 disk array, and the array slot and hard disk lookup table is pre-stored in the disk array; the second processing module is used to synchronize the data to be transferred to the reconstructed disk array in response to a user input command; wherein the disk array and / or the reconstructed disk array includes at least one RAID1 subgroup, the subgroup is used to store parity information of data from other subgroups, each subgroup consists of at least one hard drive, and at least one hard drive in the subgroup is a mirror image of each other.

[0094] In some optional implementations of the embodiments, the method further includes, before the above, constructing a lookup table of array slots and hard disk numbers, and storing the lookup table in the disk array; wherein, the array slot represents the number of the mounting slot corresponding to the installation of the portable hard disk in the disk array cabinet, and the disk array determines the hard disk to be accessed by means of the array slot when reading and writing the hard disk; the hard disk number is a number generated according to the position of the hard disk in the subgroup when the subgroup is constructed.

[0095] In some optional implementations of the embodiments, the step of reconstructing the disk array according to the array slot and hard disk number lookup table includes: after the external hard disk is inserted into the disk array, the disk array generates a new array slot and hard disk number lookup table according to the array slot and hard disk number lookup table; and reconstructs the disk array according to the new array slot and hard disk number lookup table.

[0096] In some optional implementations of the embodiments, the step of reconstructing the disk array according to the new array slot and hard disk number lookup table includes: storing the new array slot and hard disk number lookup table in the portable hard disk and the disk array to obtain the reconstructed disk array.

[0097] In some optional implementations of the embodiments, synchronizing the data to be transferred to the reconstructed disk array includes: when the data to be transferred is written to the reconstructed disk array, striping and parity calculation are performed on the data to be transferred, and then the data is written to the subgroups respectively; in the subgroups, at least one hard disk of the mirror image is written simultaneously.

[0098] In some optional implementations of the embodiments, the method further includes: when reading data, reading the corresponding data from any hard disk within the subgroup.

[0099] It is understandable that the modules described in the device 700 are similar to those in the reference. Figure 6 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 700 and the modules and units contained therein, and will not be repeated here.

[0100] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a method for quickly transferring large amounts of data offline using a disk array. This method includes: in response to the insertion of a portable hard drive, reconstructing the disk array according to an array slot and hard drive lookup table, wherein the portable hard drive contains data to be transferred, and the disk array and / or the reconstructed disk array is a RAID51 disk array, and the array slot and hard drive lookup table is pre-stored in the disk array; in response to a user input command, synchronizing the data to be transferred to the reconstructed disk array; wherein the disk array and / or the reconstructed disk array includes at least one RAID1 subgroup, the subgroup is used to store parity information of data from other subgroups, each subgroup consists of at least one hard drive, and at least one hard drive in the subgroup is a mirror image of the others.

[0101] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, enable the computer to execute the method provided by the above methods for quickly transferring large amounts of data offline using a disk array. The method includes: in response to the insertion of a portable hard drive, reconstructing a disk array according to an array slot and hard drive lookup table, wherein the portable hard drive contains data to be transferred, and the disk array and / or the reconstructed disk array is a RAID51 disk array, and the array slot and hard drive lookup table is pre-stored in the disk array; in response to a user input command, synchronizing the data to be transferred to the reconstructed disk array; wherein the disk array and / or the reconstructed disk array includes at least one RAID1 subgroup, the subgroup being used to store parity information of data from other subgroups, each subgroup consisting of at least one hard drive, and at least one hard drive in the subgroup being a mirror image of each other.

[0103] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program is implemented to perform the aforementioned methods for rapidly transferring large amounts of data offline using a disk array. The method includes: in response to the insertion of a portable hard drive, reconstructing a disk array according to an array slot and hard drive lookup table, wherein the portable hard drive contains data to be transferred, and the disk array and / or the reconstructed disk array is a RAID51 disk array, and the array slot and hard drive lookup table is pre-stored in the disk array; in response to a user input command, synchronizing the data to be transferred to the reconstructed disk array; wherein the disk array and / or the reconstructed disk array includes at least one RAID1 subgroup, the subgroup being used to store parity information of data from other subgroups, each subgroup consisting of at least one hard drive, and at least one hard drive in the subgroup being a mirror image of each other.

[0104] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for rapidly transferring large amounts of data offline using a disk array, characterized in that, Applied to disk arrays, including: In response to the insertion of a portable hard drive, the disk array is reconstructed according to the array slot and hard drive lookup table. The portable hard drive contains data to be transferred. The disk array and the reconstructed disk array are RAID51 disk arrays. The array slot and hard drive lookup table is pre-stored in the disk array. RAID51 is composed of RAID 5 and RAID 1, where a RAID 5 group consists of multiple RAID 1 sub-RAID groups. In response to user input commands, the data to be transferred is synchronized to the reconstructed disk array; The disk array and the reconstructed disk array include at least one RAID1 subgroup, which is used to store parity information of data from other subgroups. Each subgroup consists of two hard drives, and the two hard drives in the subgroup are mirror images of each other. The process of reconstructing the disk array based on the array slot and hard disk lookup table includes: After the external hard drive is inserted into the disk array, the disk array generates a new array slot and hard drive number lookup table based on the array slot and hard drive number lookup table. Reconstruct the disk array based on the new array slot and hard drive number lookup table. The process of reconstructing the disk array based on the new array slot and hard drive number lookup table includes: The new array slot and hard drive number lookup table is stored in the external hard drive and the disk array to obtain the reconstructed disk array; The step of synchronizing the data to be transferred to the reconstructed disk array includes: When the data to be transferred is written to the reconstructed disk array, the data to be transferred is striped and parity calculated before being written to the subgroups respectively; in the subgroups, at least one hard disk of the mirror is written at the same time.

2. The method for rapidly transferring large amounts of data offline using a disk array according to claim 1, characterized in that, Prior to the method, it also includes: Construct a mapping table between array slots and hard drive numbers, and store the mapping table between array slots and hard drive numbers in the disk array; The array slot number represents the number of the mounting slot corresponding to the external hard drive installed in the disk array enclosure. When the disk array reads or writes a hard drive, it uses the array slot number to determine the hard drive to be accessed. The hard drive number is a number generated when a subgroup is built, based on the hard drive's position in the subgroup.

3. The method for rapidly transferring large amounts of data offline using a disk array according to claim 1, characterized in that, The method further includes: When reading data, the corresponding data is read from any hard drive within the subgroup.

4. A data transfer device based on a disk array, characterized in that, include: The first processing module is used to reconstruct the disk array according to the array slot and hard drive lookup table in response to the insertion of the external hard drive. The external hard drive contains data to be transferred. The disk array and / or the reconstructed disk array is a RAID51 disk array. The array slot and hard drive lookup table is pre-stored in the disk array. RAID51 is composed of RAID5+RAID1. A RAID5 group is composed of multiple RAID1 sub-RAID groups. The second processing module is used to respond to user input commands and synchronize the data to be transferred to the reconstructed disk array; The disk array and / or the reconstructed disk array includes at least one RAID1 subgroup, which stores parity information of data from other subgroups. Each subgroup consists of two hard drives, and the two hard drives in the subgroup are mirror images of each other. The process of reconstructing the disk array based on the array slot and hard disk lookup table includes: After the external hard drive is inserted into the disk array, the disk array generates a new array slot and hard drive number lookup table based on the array slot and hard drive number lookup table. Reconstruct the disk array based on the new array slot and hard drive number lookup table. The process of reconstructing the disk array based on the new array slot and hard drive number lookup table includes: The new array slot and hard drive number lookup table is stored in the external hard drive and the disk array to obtain the reconstructed disk array; The step of synchronizing the data to be transferred to the reconstructed disk array includes: When the data to be transferred is written to the reconstructed disk array, the data to be transferred is striped and parity calculated before being written to the subgroups respectively; in the subgroups, at least one hard disk of the mirror is written at the same time.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for rapidly transferring large amounts of data offline using a disk array as described in any one of claims 1 to 3.

Images