Data storage method and device, electronic equipment and nonvolatile storage medium

Abstract

The invention discloses a data storage method and device, electronic equipment and a nonvolatile storage medium. The method comprises the following steps: acquiring a target data volume which needs to be transferred into a magnetic tape from a disk; redundant distribution arrangement planning is carried out on the target data volume to obtain a distribution matrix corresponding to the target data volume, and the redundant distribution arrangement planning is used for determining the storage position of the target data volume in the magnetic tape; and storing the target data volume into the magnetic tape according to the distribution matrix. The technical problem that rapid conversion of data between disk storage and tape storage cannot be achieved in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a data storage method, device, electronic device and non-volatile storage medium. Background Art

[0002] Ice storage equipment has the characteristics of large storage capacity, low energy consumption and low cost of ownership, and is an important facility for building a green and low-carbon data center. In order to further reduce the carbon emissions of ice storage equipment, it can also be equipped with a photovoltaic energy storage system to achieve carbon neutrality.

[0003] However, the cold data that is "frozen" in the ice storage device is usually stored in a file system with low performance and low price. Magnetic tapes are usually used. Magnetic tapes are an important option in the field of cold storage. Their biggest feature is low cost. With the improvement of digital transformation and digital service level requirements, although some data is not frequently accessed, it needs to be accessible at any time and responded as quickly as possible when needed. In many cases, the boundary between such cold data and warm data has become less absolute and clear. For example, some historical data with customer service response time requirements, although the access probability is relatively small, once it is needed, it needs to be found in a relatively short time; in addition, for the disaster recovery of some core real-time production systems, although the chance of disasters occurring is relatively small, once a disaster occurs, the time limit for data recovery needs to be as short as possible. For the above-mentioned similar scenarios, if the storage method of warm data (usually stored on disks with relatively high cost performance) is used, although the access response problem can be solved, the cost is relatively high (including energy consumption, operation and maintenance costs, etc.) relative to the probability of its being accessed. If the traditional cold data (tape-based) storage and recovery method is used, although the cost is low, the "thawing" time of cold data is often unbearable. Cold data in this type of scenario is called sub-cold data. There is no related technology that can not only take advantage of the low cost and low power consumption of ice storage, but also achieve cost-effectiveness and balance under limited cost increase, and shorten the gap with warm data response effect as much as possible, and cannot achieve the rapid conversion of data between disk storage and tape storage according to the business usage characteristics or needs of specific data.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method, device, electronic device and non-volatile storage medium for data storage, so as to at least solve the technical problem in the related art that it is impossible to quickly convert data between disk storage and tape storage.

[0006] According to one aspect of an embodiment of the present application, a method for data storage is provided, including: obtaining a target amount of data to be transferred to a magnetic tape from a disk; performing redundant distribution arrangement planning on the target amount of data to obtain a distribution matrix corresponding to the target amount of data, wherein the redundant distribution arrangement planning is used to determine the location at which the target amount of data is stored in the magnetic tape; and storing the target amount of data in the magnetic tape according to the distribution matrix.

[0007] In some embodiments of the present application, after obtaining the target amount of data to be transferred to the tape from the disk, the method also includes: determining the storage capacity of a single tape for storing the target amount of data; determining the number of target tapes for storing the target amount of data based on the target data amount and the storage capacity of the single tape, wherein each target tape carries a logical label.

[0008] In some embodiments of the present application, a target data volume is subjected to redundant distribution arrangement planning to obtain a distribution matrix corresponding to the target data volume, including: dividing each target tape into multiple sub-tapes composed of a plurality of data block arrangements and combinations, wherein the number of data blocks in each sub-tape is the same as the number of sub-tapes, the order of the data blocks in the target sub-tape is obtained by placing the head data block in the previous sub-tape of the target sub-tape to the tail of the previous sub-tape of the target sub-tape, and the target sub-tape is any sub-tape of the multiple sub-tapes except the first sub-tape; obtaining the distribution of the data blocks corresponding to each sub-tape in the target tape; determining the sub-distribution matrix corresponding to the target tape according to the order of each sub-tape in the target tape and the distribution of the data blocks corresponding to each sub-tape; and determining the distribution matrix corresponding to the target data volume according to the sub-distribution matrices corresponding to all the target tapes.

[0009] In some embodiments of the present application, after obtaining the distribution matrix corresponding to the target data volume, the method also includes: obtaining the position information of the target data blocks in each target tape in the distribution matrix in the sub-tape corresponding to the target tape, and obtaining the logical label corresponding to each target tape, wherein the target data block is any data block in any target tape; and determining the logical redundant distribution coordinates corresponding to the target data blocks in each target tape based on the position information and the logical label corresponding to each target tape.

[0010] In some embodiments of the present application, the target data volume is stored in a tape according to a distribution matrix, including: obtaining the logical redundant distribution coordinates corresponding to the target data blocks in each target tape; and transferring the target data volume from the disk to the corresponding position of the target tape according to the logical redundant distribution coordinates.

[0011] In some embodiments of the present application, the method also includes: obtaining first logical redundant distribution coordinates corresponding to the first data, wherein the first data is any one of the target data volumes transferred from the disk to the tape; determining the target logical redundant distribution coordinates closest to the first data from the first logical redundant distribution coordinates; determining the first tape storing the first data based on the target logical redundant distribution coordinates; obtaining the first data from the first tape, and copying the first data to the disk.

[0012] In some embodiments of the present application, the method also includes: when the first tape corresponding to the target logical redundant distribution coordinates cannot be accessed, determining the second logical redundant distribution coordinates from the first logical redundant distribution coordinates, wherein the second logical redundant distribution coordinates are the coordinates in the first logical redundant distribution coordinates that are closest to the first data except the target logical redundant distribution coordinates; determining the second tape storing the first data based on the second logical redundant distribution coordinates; accessing the second tape to obtain the first data, and copying the first data to the disk.

[0013] According to another aspect of an embodiment of the present application, a data storage device is also provided, including: an acquisition module, used to acquire a target amount of data to be transferred to a magnetic tape from a disk; a redundant distribution orchestration planning module, used to perform redundant distribution orchestration planning on the target amount of data to obtain a distribution matrix corresponding to the target amount of data, wherein the redundant distribution orchestration planning is used to determine the location where the target amount of data is stored in the magnetic tape; and a storage module, used to store the target amount of data in the magnetic tape according to the distribution matrix.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, in which a program is stored, wherein when the program is running, a device where the non-volatile storage medium is located is controlled to execute the above-mentioned data storage method.

[0015] According to another aspect of an embodiment of the present application, there is further provided an electronic device, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the above-mentioned data storage method is executed when the program is run.

[0016] According to another aspect of an embodiment of the present application, a computer program product is also provided, including computer instructions, which implement the above-mentioned data storage method when executed by a processor.

[0017] In an embodiment of the present application, a target amount of data to be transferred to a magnetic tape is obtained from a disk; a redundant distribution arrangement planning is performed on the target amount of data to obtain a distribution matrix corresponding to the target amount of data, wherein the redundant distribution arrangement planning is used to determine the location where the target amount of data is stored in the magnetic tape; the target amount of data is stored in the magnetic tape in accordance with the distribution matrix, by performing redundant distribution arrangement planning on the target amount of data, thereby obtaining a distribution matrix corresponding to the target amount of data, and storing the target amount of data in the magnetic tape based on the distribution matrix, thereby solving the problem of the inability to achieve rapid conversion of data between magnetic disk storage and magnetic tape storage in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing a method for data storage provided in an embodiment of the present application;

[0020] Figure 2 is a flow chart of a data storage method according to an embodiment of the present application;

[0021] Figure 3 This is a schematic diagram of a magnetic tape logical label for data according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a redundant distribution solution according to an embodiment of the present application;

[0023] Figure 5 It is a structural schematic diagram of a data storage device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0027] Hot data: Hot data refers to online data that is frequently accessed and critical to business and applications. Hot data usually needs to be accessed and processed quickly and efficiently, so it needs to be stored on high-performance, low-latency storage devices.

[0028] Cold data: Cold data refers to data that is not frequently accessed and can usually tolerate a longer response time when it needs to be accessed.

[0029] Warm data: refers to data that is moderately accessed and of some importance to business and applications. This data does not need to be accessed and processed as quickly as hot data, but it also needs to be reliably stored and accessed within a certain period of time. Warm data is between hot data and cold data. It is neither critical data that is frequently accessed, nor data that is stored for a long time and accessed infrequently.

[0030] Ice storage: A low-cost, low-energy storage solution and device for cold data.

[0031] Secondary cold data: It is cold data, but when it is accessed, it requires a faster response time than general cold data.

[0032] In the related art, if the storage method of warm data is used (usually stored on a disk with a relatively high cost performance), although the access response problem can be solved, the cost is relatively high relative to the probability of it being accessed (including energy consumption, operation and maintenance costs, etc.). If the traditional cold data (tape-based) storage and recovery method is adopted, although the cost is low, the "thawing" time of cold data is often unbearable. How to take advantage of the low cost and low power consumption of ice storage while achieving cost performance improvement and balance with limited cost increase, and shortening the gap with warm data response effect as much as possible, has become an urgent problem to be solved. Therefore, there is a problem in the related art that it is impossible to achieve rapid conversion of data between disk storage and tape storage. In order to solve this problem, a relevant solution is provided in the embodiment of the present application, which is described in detail below.

[0033] According to an embodiment of the present application, a method embodiment of data storage is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a method for data storage. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0035] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0036] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data storage method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned data storage method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0038] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0039] In the above operating environment, the embodiment of the present application provides a flow chart of a method for storing data, such as Figure 2 As shown, the method comprises the following steps:

[0040] Step S202, obtaining the target data volume to be transferred to the tape from the disk.

[0041] In the technical solution provided in step S202, the target data amount obtained from the disk to be transferred to the tape can be the data amount of sub-cold data. The data amount of sub-cold data obtained from the disk to be transferred to the tape is the data amount required to transfer the sub-cold data from warm data to cold storage.

[0042] Before performing redundant distribution planning for the target data volume, it is also necessary to determine the storage capacity of a single tape for storing the target data volume (for example, the volume of sub-cold data); based on the target data volume and the storage capacity of a single tape, determine the number of target tapes for storing the target data volume, wherein each target tape carries a logical label.

[0043] The following is a specific embodiment:

[0044] For example, the amount of sub-cold data that needs to be transferred from warm data to cold storage (i.e., the target data amount) is represented by Z. The capacity of a single tape disk used to store this amount of data (i.e., the target data amount) is calculated, and the data storage capacity of the single tape disk is Y. The value of Z / Y is rounded (hereinafter represented by [Z / Y]). If there is no remainder, it means that the Z amount of data can be stored on [Z / Y] single tape disks. If there is a remainder, it means that the Z amount of data needs to be stored on [Z / Y]+1 single tape disks. The number of tapes (i.e., the target tapes) required for Z is represented by X (i.e., the number of target tapes used to store the target data amount), and the tape logical labels corresponding to each segment of the Z amount of data divided in units of Y are marked as Tape. i (i.e. the above logical label), where the i interval is [1,…X].

[0045] Step S204, performing redundancy distribution arrangement planning on the target data volume to obtain a distribution matrix corresponding to the target data volume, wherein the redundancy distribution arrangement planning is used to determine the storage position of the target data volume in the magnetic tape.

[0046] In the technical solution provided in step S204, there are many ways to perform redundant distribution planning on the target data volume and obtain a distribution matrix corresponding to the target data volume. For example, the distribution matrix corresponding to the target data volume can be obtained by the following method: each target tape is divided into a plurality of data blocks (for example, divided equally into 2 HThe invention relates to a method for obtaining a plurality of sub-tapes composed of a plurality of data blocks arranged and combined, wherein the number of data blocks in each sub-tape is the same as the number of sub-tapes, the order of data blocks in a target sub-tape is obtained by placing a head data block in a previous sub-tape of the target sub-tape to the tail of the previous sub-tape of the target sub-tape, and the target sub-tape is any sub-tape of the plurality of sub-tapes except the first sub-tape; obtaining the distribution of data blocks corresponding to each sub-tape in the target tape; determining a sub-distribution matrix corresponding to the target tape according to the order of each sub-tape in the target tape and the distribution of data blocks corresponding to each sub-tape; and determining a distribution matrix corresponding to a target data amount according to the sub-distribution matrices corresponding to all the target tapes.

[0047] The following is a specific embodiment:

[0048] Tape i For example, the corresponding logical label Tape i The data (hereinafter referred to as E) is divided into 2 H data blocks (the value of H is optional), based on this, prepare the same number of tapes for E (i.e., the sub-tapes of the above target tape) for redundancy distribution planning, such as 4 or 8 tapes (by analogy with powers of 2), and H Redundancy distribution planning is performed on the tapes (where the logical sequence of the sub-tapes is marked as K f , the interval of f is [2 0 …2 H ]), for the target sub-tape (i.e., any sub-tape except the first sub-tape among the multiple sub-tapes), the cyclic bubbling method is used for redundant distribution arrangement planning: the head data block (i.e., the head data block in the previous sub-tape of the above target sub-tape) is placed at the end in turn, and the remaining blocks are promoted one bit in turn, that is: the second data block becomes the first, the third data block becomes the second, and so on.

[0049] For example: Taking H as 2, divide data E into 4 (2 2 ) data blocks, denoted as A, B, C, and D; and 4 sub-tapes are also required, denoted as K1, K2, K3, and K4. The loop bubbling arrangement method is as follows:

[0050] (1) K1 distribution: ABCD;

[0051] (2) K2 distribution: Based on the previous round of arrangement results (K1 distribution), A is placed at the end, and the distribution is BCDA;

[0052] (3) K3 distribution: Based on the previous round of arrangement results (K2 distribution), put B at the end and the distribution is CDAB;

[0053] (4) K4 distribution: According to the previous round of arrangement results (K3 distribution), C is placed at the end, and the distribution is DABC;

[0054] Logical label is Tape i The sub-distribution matrix of the target tape is as follows:

[0055] ABCD,

[0056] BCDA,

[0057] CDAB,

[0058] DABC.

[0059] Based on the cyclic bubbling method, the position of each data block ABCD in the four tapes is equally distributed, that is, each data block is in 4 (2 2 ) target tapes have an equal chance of distribution for each sequential position. i (where i is in the interval [1,…X]) all perform redundant distribution planning according to the method to obtain the sub-distribution matrix corresponding to all target tapes.

[0060] After determining the distribution matrix corresponding to the target data volume based on the sub-distribution matrix corresponding to all target tapes, the logical redundant distribution coordinates corresponding to the target data blocks in each target tape can also be determined based on the distribution matrix. The logical redundant distribution coordinates can be used to quickly transfer the target data volume from cold storage to warm data. There are many ways to determine the logical redundant distribution coordinates corresponding to the target data blocks in each target tape, for example: obtaining the position information of the target data blocks in each target tape in the distribution matrix in the sub-tape corresponding to the target tape, and obtaining the logical label corresponding to each target tape, wherein the target data block is any data block in any target tape; determining the logical redundant distribution coordinates corresponding to the target data block in each target tape based on the position information and the logical label corresponding to each target tape.

[0061] The following is a specific embodiment:

[0062] The logical redundancy distribution coordinates of each data unit that will be transferred from warm data (such as stored on disk) to cold storage (such as tape) are marked (i.e., the logical redundancy distribution coordinates corresponding to the target data block in each target tape are determined as above): Tape logical label Tape i And the redundant distribution coordinates (K f , a logical sequence of the target tape to which the data block belongs) group, where K f The f interval is [2 0 …2 H]). For example, in a distribution scheme where H is 2 (i.e., 4 tapes are used for redundant distribution), a data block B has redundant distribution coordinates on the logical label Tape1, which are (K1, 2) (K2, 1) (K3, 4) (K4, 3). The logical redundant distribution coordinates of B are recorded in the format of (Tape1).(K1, 2), (Tape1).(K2, 1), (Tape1).(K3, 4), (Tape1).(K4, 3).

[0063] Step S206, storing the target data volume in the magnetic tape according to the distribution matrix.

[0064] In the technical solution provided in step S206, there are multiple ways to implement storing the target data volume in the tape according to the distribution matrix. For example, the target data volume can be stored in the tape according to the distribution matrix in the following ways: obtaining the logical redundant distribution coordinates corresponding to the target data blocks in each target tape: based on the logical redundant distribution coordinates, transferring the target data volume from the disk to the corresponding position of the target tape.

[0065] After completing step S206, the target data volume can be transferred from the disk to the corresponding position of the target tape, that is, the sub-cold data can be dumped from the disk storing warm data to the tape storing cold data, thereby realizing the conversion and storage of data from warm data to cold storage.

[0066] After completing step S206, the sub-cold data can also be dumped from the tape storing the cold data to the disk storing the warm data, for example, in the following manner: obtain the first logical redundant distribution coordinates corresponding to the first data, wherein the first data is any one of the target data volume transferred from the disk to the tape; determine the target logical redundant distribution coordinates closest to the first data from the first logical redundant distribution coordinates; determine the first magnetic tape storing the first data based on the target logical redundant distribution coordinates; obtain the first data from the first magnetic tape, and copy the first data to the disk; when the first magnetic tape corresponding to the target logical redundant distribution coordinates cannot be accessed, determine the second logical redundant distribution coordinates from the first logical redundant distribution coordinates, wherein the second logical redundant distribution coordinates are the coordinates in the first logical redundant distribution coordinates that are closest to the first data except the target logical redundant distribution coordinates; determine the second magnetic tape storing the first data based on the second logical redundant distribution coordinates; access the second magnetic tape to obtain the first data, and copy the first data to the disk.

[0067] The following is a specific embodiment:

[0068] Dumping from warm data state to ice storage (i.e. storing the target amount of data on tape according to the distribution matrix):

[0069] The second cold data with a data volume of Z (i.e. the above target data volume) is arranged according to the redundancy distribution plan, and the data blocks in the disk are arranged according to (Tape i ).(K1) version of the redundant distribution coordinates are dumped to each corresponding tape in turn (i interval is [1,…X]). i The data distribution scheme corresponding to K1 (for example, in the order of ABCD in the previous embodiment) is dumped from the disk to the tape. Thus, the first complete secondary cold data with a data volume of Z is dumped from the disk to the tape in the arrangement sequence version of the redundant distribution coordinates of K1. Hereinafter referred to as the original tape dump T1, based on the original tape dump T1, the K f The f interval is [2 0 …2 H ]). Perform each Tape i Based on K f The solution version of the disk to tape dump is to dump Tape1, Tape2, and Tape3 according to the arrangement sequence version of the redundant distribution coordinates of K2, K3, and K4. So far, the dump of all sub-cold data from warm state to cold storage based on redundant distribution arrangement is completed.

[0070] Dump from cold storage to warm data:

[0071] When it is necessary to access a certain cold data in ice storage, first obtain the first logical redundant distribution coordinates corresponding to the data block (i.e. the first data mentioned above), and select the coordinate (i.e. the target logical redundant distribution coordinate) that is closest to the access distance of the data block (i.e. the first data mentioned above) from multiple coordinates in the first logical redundant distribution coordinates, referred to as coordinate M for short. For example, when the record format of the logical redundant distribution coordinates of data block B (i.e. the first logical redundant distribution coordinate) is: (Tape1).(K1, 2), (Tape1).(K2, 1), (Tape1).(K3, 4), (Tape1).(K4, 3), the coordinate (i.e. the target logical redundant distribution coordinate) that is closest to the access distance of data block B is M(Tape1).(K2, 1).

[0072] Based on coordinate M, the tape corresponding to coordinate M (i.e., the first tape mentioned above) is selected from the tape cabinet through the tape driver to access the corresponding data, and copy it to the disk to become warm data for use by the application. When an abnormal situation occurs, such as the tape corresponding to coordinate M is damaged and inaccessible, the coordinate with the next closest access distance to the data unit (i.e., the second logical redundant distribution coordinate mentioned above) can be selected from the logical redundant distribution coordinates. For example: the coordinate with the next closest access distance to B is (Tape1).(K1,2).

[0073] Through the above steps, data storage can be realized, especially for the fast conversion storage between warm data and cold data (ice storage), the method of logical redundancy distribution arrangement planning is adopted so that each data block is (2 H Each sequence position in the tape group (composed of 100 tapes) has redundant distribution, which can greatly speed up the access time when it needs to be accessed. Under normal circumstances, at most (1 / 2) of the worst case of the original (non-redundant distribution) tape solution is required. H ) time, and in the event of damage to any tape, the maximum damage will not exceed ((2 / 2 H ) time, the storage of sub-cold data on ice storage adopts the idea of ​​space for time, and only needs to achieve redundant distribution through the addition of low-cost tapes, which is low-cost, has good scalability, and can fully utilize the existing tape resources. And when the H value is larger, the data thawing speed is faster, which is conducive to users to achieve cost-effectiveness and balance according to application requirements and cost conditions. The sub-cold data is dumped from the warm data state to the ice storage, and only the disk-to-tape method is used for the first dump, and the remaining redundant distributed data is dumped by tape-to-tape. The green electricity method of photovoltaic energy storage is fully utilized, which can greatly reduce energy consumption and cost, and is conducive to achieving carbon neutrality. The logical position of the data block on the ice storage is marked and addressed by the method of logical redundant distribution coordinates, which is conducive to the low-cost and rapid implementation of the software solution based on the existing tape control equipment. In order to achieve a balance between the access speed and energy consumption cost of sub-cold data, the warm data (disk) solution is adopted, and the power is turned off when not in use and re-powered when in use. This mode has high requirements for operation and maintenance and personnel, and the cost is relatively high. The method provided in this application has low requirements on the user's operation and maintenance capabilities, is green and carbon neutral, does not require human intervention, and does not require changes to the user's traditional usage habits.

[0074] The present application embodiment also provides a data tape logical label schematic diagram, such as Figure 3 As shown in the figure, a data volume of Z (i.e., the target data volume) is shown. The capacity of a single tape disk used to store the data volume (i.e., the target data volume) is Y. The value of Z / Y is rounded to the integer (hereinafter referred to as [Z / Y]). If there is no remainder, it means that the data volume of Z can be stored on [Z / Y] single tape disks. If there is a remainder, it means that the data volume of Z needs to be stored on [Z / Y]+1 single tape disks. The number of tapes (i.e., the target tapes) required for Z is represented by X (i.e., the number of target tapes used to store the target data volume), and the tape logical labels corresponding to each segment of the data after the Z volume of data is divided into units of Y are marked as Tape. i (i.e. the above logical label), where i is in the range [1,…X], such as Figure 3As shown, the data needs three single-reel magnetic tapes to be stored, and the magnetic tape logical labels (ie, the above-mentioned logical labels) are Tape1, Tape2, and Tape3 respectively.

[0075] The present application embodiment also provides a schematic diagram of a redundant distribution solution, such as Figure 4 As shown, the corresponding Figure 3 The redundant distribution plan for the redundant distribution arrangement planning of the tape with the logical label of the tape (i.e. the above logical label) as Tape1 is to divide the data (hereinafter referred to as E) belonging to a logical label marked as Tape1 into 4 data blocks. Based on this, 4 tapes (i.e. the sub-tapes of the above target tape) are prepared for E to perform redundant distribution arrangement planning. H Redundancy distribution planning is performed on the tapes (where the logical sequence of the target sub-tape is marked as K f , the interval of f is [2 0 …2 H ]), Figure 4 The logical sequence of the sub-tapes of the target tape is marked as K1-K4. The first sub-tape is distributed in order, i.e., ABCD. The target sub-tape (i.e., any sub-tape except the first sub-tape among the multiple sub-tapes) is used, and the cyclic bubbling method is used for redundant distribution arrangement planning: the head data block (i.e., the head data block in the previous sub-tape of the above target sub-tape) is placed at the end in a cyclic manner, and the remaining blocks are promoted one position in turn, i.e., the second data block becomes the first, the third data block becomes the second, and so on.

[0076] like Figure 4 As shown: Divide data E into 4 (2 2 ) data blocks, denoted as A, B, C, and D; and 4 sub-tapes are also required, denoted as K1, K2, K3, and K4. The loop bubbling arrangement method is as follows:

[0077] (1) K1 distribution: ABCD;

[0078] (2) K2 distribution: Based on the previous round of arrangement results (K1 distribution), A is placed at the end, and the distribution is BCDA;

[0079] (3) K3 distribution: Based on the previous round of arrangement results (K2 distribution), put B at the end and the distribution is CDAB;

[0080] (4) K4 distribution: According to the previous round of arrangement results (K3 distribution), C is placed at the end, and the distribution is DABC;

[0081] The sub-distribution matrix of the target tape with logical label Tape1 is as follows:

[0082] ABCD,

[0083] BCDA,

[0084] CDAB,

[0085] DABC.

[0086] The embodiment of the present application also provides a redundant distribution arrangement plan for storing secondary cold data from warm data to cold data, and the steps are as follows:

[0087] Step 11, calculating the amount of sub-cold data that needs to be archived to ice storage, and the amount of data is represented by Z (ie, the above-mentioned target data amount).

[0088] Step 12, calculating the capacity of a single magnetic tape for storage, where the magnetic tape data storage capacity is represented by Y.

[0089] Step 13, round the value of Z / Y (hereinafter referred to as [Z / Y]). If there is no remainder, it means that the data of Z can be stored on [Z / Y] tapes. If there is a remainder, it means that the data of Z needs to be stored on [Z / Y]+1 tapes. The number of tapes required for Z (i.e., the target tapes) is represented by X (i.e., the number of target tapes used to store the target data).

[0090] Step 14, divide the Z quantity data in units of Y in sequence until the remainder is 0 or the remainder is less than Y.

[0091] Step 15: Mark the tape logical labels corresponding to each segment of the Z data divided into Y units as Tape i , where i is in the interval [1,…X], when i is 1, such as Figure 3 shown.

[0092] Step 16: Attribute the corresponding i The data (hereinafter referred to as D) is divided into 2 H power data blocks (the value of H is optional). Based on this, prepare 2 H tapes (for redundant distribution), such as 4 or 8 tapes (and so on in powers of 2).

[0093] Step 17: Based on step 16, D is redundancy distributed on 2H tapes (where the tape logic sequence is marked as K). f , f interval is [20,…2H]), the first sub-tape is distributed in order, that is, ABCD, and the target sub-tape (that is, any sub-tape except the first sub-tape among the multiple sub-tapes) is used. The cyclic bubbling method is adopted: the head data block (that is, the head data block in the previous sub-tape of the above target sub-tape) is placed at the end in turn, and the remaining blocks are promoted one by one, that is: the second data block becomes the first, the third data block becomes the second, and so on.

[0094] Step 18: For each Tape i (where i is in the interval [1,…X]) all perform redundant distribution scheduling and planning according to the method of steps 16-17.

[0095] Step 19, mark the logical redundancy distribution coordinates of each data unit that is transferred from warm data to cold storage, that is, the tape logical label Tape. i And the redundant distribution coordinates (K f , a group of sequence positions of data blocks to which the data unit belongs (i.e., a logical sequence of sub-tapes of the target tape to which the above data blocks belong)), where K f The f interval is [2 0 …2 H ]).

[0096] The embodiment of the present application also provides a method for dumping sub-cold data from a warm data state to cold storage, the steps are as follows: Step 21, the sub-cold data with a data volume of Z (i.e., the above-mentioned target data volume) is arranged according to the redundant distribution plan, and the data blocks in the disk are dumped to each corresponding tape (i interval is [1, ...X]) in sequence according to the redundant distribution scheme of version (Tapei).(K1), that is, each Tape i The data distribution scheme corresponding to K1 (in the order of ABCD) is dumped from the disk to the tape, thereby completing the dump of the first complete K1 arrangement sequence version of the sub-cold data with a data volume of Z from the disk to the tape. Hereinafter referred to as the original tape dump T1.

[0097] Step 22, based on step 21, dump T1 based on the original tape, and then f (f value range is from 2 1 To 2 H ) for each Tape i Based on K f The solution version of the tape-to-tape dump is to dump Tape1, Tape2, and Tape3 according to the arrangement sequence version of K2, K3, and K4. So far, the dump of all secondary cold data from warm state to cold storage based on redundant distribution arrangement is completed.

[0098] The embodiment of the present application also provides a method for dumping sub-cold data from cold storage to warm data, the steps of which are as follows:

[0099] Step 31, when it is necessary to access a certain cold data in the cold storage, firstly obtain a plurality of logical redundant distribution coordinates corresponding to the data block (i.e., the first data) from the warm-cold data conversion control device, and select the coordinate with the closest access distance to the data unit from the plurality of logical redundant distribution coordinates (i.e., the target logical redundant distribution coordinate), hereinafter referred to as coordinate M, Figure 4In the example, the coordinate that B visits closest is (Tape1).(K2,1).

[0100] Step 32, based on the coordinate M, the tape driver selects the tape corresponding to the coordinate M (ie, the first tape mentioned above) from the tape cabinet, accesses the corresponding data, and copies it to the disk to become warm data for use by the application.

[0101] Step 3201, based on steps 31 and 32, when an abnormal situation occurs, such as the tape corresponding to the M coordinate is damaged and inaccessible, the coordinate with the second closest access distance to the data unit can be selected from the logical redundant distribution coordinates, that is, the above-mentioned second logical redundant distribution coordinates), for example, the coordinate with the second closest access distance to B is (Tape1).(K1, 2).

[0102] The present application also provides a schematic diagram of a data storage device. Figure 5 As shown, including:

[0103] The acquisition module 502 is used to acquire the target data volume to be transferred to the tape from the disk.

[0104] The acquisition module 502 is also used to determine the storage capacity of a single tape for storing a target amount of data (for example, the amount of sub-cold data); based on the target amount of data and the storage capacity of a single tape, determine the number of target tapes for storing the target amount of data, wherein each target tape carries a logical label.

[0105] The redundant distribution arrangement planning module 504 is used to perform redundant distribution arrangement planning on the target data volume to obtain a distribution matrix corresponding to the target data volume, wherein the redundant distribution arrangement planning is used to determine the location where the target data volume is stored in the tape.

[0106] The redundancy distribution arrangement planning module 504 is further used to divide each target tape into a plurality of data blocks (for example, equally divided into 2 HThe invention relates to a method for obtaining a plurality of sub-tapes composed of a plurality of data blocks arranged and combined, wherein the number of data blocks in each sub-tape is the same as the number of sub-tapes, the order of data blocks in a target sub-tape is obtained by placing a head data block in a previous sub-tape of the target sub-tape to the tail of the previous sub-tape of the target sub-tape, and the target sub-tape is any sub-tape of the plurality of sub-tapes except the first sub-tape; obtaining the distribution of data blocks corresponding to each sub-tape in the target tape; determining a sub-distribution matrix corresponding to the target tape according to the order of each sub-tape in the target tape and the distribution of data blocks corresponding to each sub-tape; determining a distribution matrix corresponding to a target data amount according to the sub-distribution matrices corresponding to all the target tapes; obtaining position information of a target data block in each target tape in the distribution matrix in the sub-tape corresponding to the target tape, and obtaining a logical label corresponding to each target tape, wherein the target data block is any data block in any target tape; determining a logical redundancy distribution coordinate corresponding to the target data block in each target tape according to the position information and the logical label corresponding to each target tape.

[0107] The storage module 506 is used to store the target data volume into the magnetic tape according to the distribution matrix.

[0108] The storage module 506 is further configured to read the target data volume from the magnetic tape and transfer the data to the magnetic disk according to the logical redundancy distribution coordinates.

[0109] The storage module 506 is further used to obtain the logical redundancy distribution coordinates corresponding to the target data blocks in each target tape: based on the logical redundancy distribution coordinates, the target data volume is transferred from the disk to the corresponding position of the target tape.

[0110] The storage module 506 is also used to obtain the first logical redundant distribution coordinates corresponding to the first data, wherein the first data is any one of the target data volume transferred from the disk to the tape; determine the target logical redundant distribution coordinates closest to the first data from the first logical redundant distribution coordinates; determine the first tape storing the first data based on the target logical redundant distribution coordinates; obtain the first data from the first tape, and copy the first data to the disk; when the first tape corresponding to the target logical redundant distribution coordinates cannot be accessed, determine the second logical redundant distribution coordinates from the first logical redundant distribution coordinates, wherein the second logical redundant distribution coordinates are the coordinates in the first logical redundant distribution coordinates that are closest to the first data except the target logical redundant distribution coordinates; determine the second tape storing the first data based on the second logical redundant distribution coordinates; access the second tape to obtain the first data, and copy the first data to the disk.

[0111] It should be noted that the various modules in the above-mentioned data storage device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0112] The embodiment of the present application also provides a non-volatile storage medium, the non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above data storage method. For example, the target amount of data to be transferred to the tape is obtained from the disk; the target amount of data is subjected to redundant distribution and arrangement planning to obtain a distribution matrix corresponding to the target amount of data, wherein the redundant distribution and arrangement planning is used to determine the location where the target amount of data is stored in the tape; and the target amount of data is stored in the tape according to the distribution matrix.

[0113] The embodiment of the present application also provides an electronic device, the electronic device includes a processor, the processor is used to run a program, wherein the above data storage method is executed when the program is running. For example, the target amount of data to be transferred to the tape is obtained from the disk; the target amount of data is subjected to redundant distribution arrangement planning to obtain a distribution matrix corresponding to the target amount of data, wherein the redundant distribution arrangement planning is used to determine the location where the target amount of data is stored in the tape; and the target amount of data is stored in the tape according to the distribution matrix.

[0114] According to another aspect of the embodiment of the present application, a computer program product is also provided, including a computer program, which implements the above data storage method when executed by a processor. For example, the target amount of data to be transferred to the tape is obtained from the disk; the target amount of data is subjected to redundant distribution and arrangement planning to obtain a distribution matrix corresponding to the target amount of data, wherein the redundant distribution and arrangement planning is used to determine the location where the target amount of data is stored in the tape; and the target amount of data is stored in the tape according to the distribution matrix.

[0115] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0118] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0120] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for data storage, characterized in that: include: Get the target amount of data to be transferred from disk to tape; Performing a redundancy distribution arrangement plan on the target data volume to obtain a distribution matrix corresponding to the target data volume, wherein the redundancy distribution arrangement plan is used to determine a location where the target data volume is stored in the magnetic tape; The target data volume is stored in the magnetic tape according to the distribution matrix.

2. The method according to claim 1, characterized in that After obtaining the target data volume to be transferred to the tape from the disk, the method further includes: Determining the storage capacity of a single magnetic tape for storing the target amount of data; The number of target magnetic tapes used to store the target data volume is determined according to the target data volume and the storage capacity of the single magnetic tape, wherein each target magnetic tape carries a logical label.

3. The method according to claim 2, characterized in that Performing redundant distribution planning on the target data volume to obtain a distribution matrix corresponding to the target data volume includes: Divide each target tape into a plurality of sub-tapes composed of a plurality of data blocks arranged and combined, wherein the number of data blocks in each sub-tape is the same as the number of the sub-tapes, the order of the data blocks in the target sub-tape is obtained by placing the head data block in the previous sub-tape of the target sub-tape to the tail of the previous sub-tape of the target sub-tape, and the target sub-tape is any sub-tape of the plurality of sub-tapes except the first sub-tape; Obtaining the distribution of data blocks corresponding to each sub-tape in the target tape; Determine a sub-distribution matrix corresponding to the target tape according to the order of each sub-tape in the target tape and the distribution of data blocks corresponding to each sub-tape; A distribution matrix corresponding to the target data volume is determined according to the sub-distribution matrices corresponding to all target magnetic tapes.

4. The method according to claim 3, characterized in that After obtaining the distribution matrix corresponding to the target data volume, the method further includes: Obtaining the position information of the target data block in each target tape in the distribution matrix in the sub-tape corresponding to the target tape, and obtaining the logical label corresponding to each target tape, wherein the target data block is any data block in any target tape; The logical redundancy distribution coordinates corresponding to the target data blocks in each target tape are determined according to the position information and the logical label corresponding to each target tape.

5. The method according to claim 4, characterized in that Storing the target data volume in the magnetic tape according to the distribution matrix comprises: Get the logical redundancy distribution coordinates corresponding to the target data blocks in each target tape: According to the logical redundancy distribution coordinates, the target data volume is transferred from the disk to a corresponding position of the target magnetic tape.

6. The method according to claim 4, characterized in that The method further comprises: Acquire first logical redundancy distribution coordinates corresponding to first data, wherein the first data is any data in the target data volume transferred from the disk to the tape; Determine, from the first logical redundancy distribution coordinates, a target logical redundancy distribution coordinate that is closest to the first data; Determine a first magnetic tape storing the first data according to the target logical redundancy distribution coordinates; The first data is acquired from the first magnetic tape, and the first data is copied to the magnetic disk.

7. The method according to claim 6, characterized in that The method further comprises: When the first magnetic tape corresponding to the target logical redundant distribution coordinates is inaccessible, determining a second logical redundant distribution coordinate from the first logical redundant distribution coordinates, wherein the second logical redundant distribution coordinates are coordinates of the first logical redundant distribution coordinates that are closest to the first data except the target logical redundant distribution coordinates; Determine a second magnetic tape storing the first data according to the second logical redundancy distribution coordinates; The second magnetic tape is accessed to obtain the first data, and the first data is copied to the magnetic disk.

8. A data storage device, characterized in that: include: The acquisition module is used to obtain the target data volume that needs to be transferred to the tape from the disk; A redundant distribution arrangement planning module, used for performing redundant distribution arrangement planning on the target data volume to obtain a distribution matrix corresponding to the target data volume, wherein the redundant distribution arrangement planning is used to determine the location where the target data volume is stored in the magnetic tape; A storage module is used to store the target data volume in the magnetic tape according to the distribution matrix.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the data storage method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein when the program is run, the method for storing data according to any one of claims 1 to 7 is executed.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the data storage method according to any one of claims 1 to 7 is implemented.

Images