Erasure block copy to three-copy bidirectional mapping method and device and electronic equipment

Abstract

The application discloses a kind of erasure block copy to three-copy bidirectional mapping method, device and electronic equipment.The method comprises the following steps: obtaining the multiple target data blocks of three-copy data of three-copy storage, wherein the multiple target data blocks are the multiple data blocks obtained by blocking one data in three-copy data;Each data block in the multiple target data blocks is regarded as the current data block, and the current data block is subjected to erasure coding operation, to obtain M block sub data block and N block sub check block;M block sub data block and N block sub check block are written into the corresponding erasure copy of three-copy storage;After writing successfully, the forward mapping relationship and the reverse mapping relationship of the current data block in three-copy storage and erasure copy are written into disk.The application solves the technical problem that there is no reverse mapping relationship in the erasure coding process.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, in particular to a method and device for bi-directional mapping of erasure block copies to three copies and electronic equipment. BACKGROUND

[0002] In the prior art, in the EC (erasure coding) process, only the erasure copy data block corresponding to the three copies is persisted in the Manifest file (a kind of Android software, a persistent file recording the main metadata information of the data block, on the disk) of the data block, so there is only a forward mapping relationship, and there is no reverse mapping relationship of the erasure copy to the three copies. In the cluster read-write process, if there is no reverse mapping relationship, it is not possible to locate which three copies the erasure copy corresponds to. SUMMARY

[0003] The embodiments of the present application provide a method and device for bi-directional mapping of erasure block copies to three copies and electronic equipment, to at least solve the technical problem of no reverse mapping relationship in the erasure coding process.

[0004] According to an aspect of the embodiments of the present application, a method for bi-directional mapping of erasure block copies to three copies is provided, comprising: obtaining a plurality of target data blocks of three-copy data stored in a three-copy storage, wherein the plurality of target data blocks are a plurality of data blocks obtained by block splitting of one piece of data in the three-copy data; taking each data block in the plurality of target data blocks as a current data block, performing an erasure coding operation on the current data block to obtain M sub-data blocks and N sub-check blocks; writing the M sub-data blocks and the N sub-check blocks into the corresponding erasure copy of the three-copy storage; and after successful writing, writing the forward mapping relationship and the reverse mapping relationship of the current data block in the three-copy storage and the erasure copy to a disk.

[0005] According to another aspect of the embodiments of the present application, a device for bi-directional mapping of erasure block copies to three copies is provided, comprising: an obtaining module configured to obtain a plurality of target data blocks of three-copy data stored in a three-copy storage, wherein the plurality of target data blocks are a plurality of data blocks obtained by block splitting of one piece of data in the three-copy data; a processing module configured to take each data block in the plurality of target data blocks as a current data block, and perform an erasure coding operation on the current data block to obtain M sub-data blocks and N sub-check blocks; a first writing module configured to write the M sub-data blocks and the N sub-check blocks into the corresponding erasure copy of the three-copy storage; and a second writing module configured to, after successful writing, write the forward mapping relationship and the reverse mapping relationship of the current data block in the three-copy storage and the erasure copy to a disk.

[0006] As an optional example, the processing module comprises: a first processing unit configured to divide the current data block into M sub-data blocks; and a second processing unit configured to perform erasure coding on the M sub-data blocks to obtain N sub-check blocks.

[0007] As an optional example, the first processing unit comprises: a first processing sub-unit configured to take each object in the current data block as a current object; a second processing sub-unit configured to divide the current object into M sub-data blocks; and a third processing sub-unit configured to take the i-th sub-data block as an object in the i-th sub-data block.

[0008] As an optional example, the second processing unit comprises: a fourth processing sub-unit configured to select N+1 continuous sub-data blocks from the M sub-data blocks; and a fifth processing sub-unit configured to take each N+1 sub-data block as a sub-check block to obtain the N sub-check blocks.

[0009] As an optional example, the obtaining module comprises: a third processing unit configured to determine the three copies of data as three identical first data; a fourth processing unit configured to take any one of the first data as target data; and a fifth processing unit configured to take each data block of a predetermined size of the target data as a target data block.

[0010] As an optional example, the device further comprises: a first querying module configured to send, to a domain name server, a first file identifier of a query file stored in three copies; and a first receiving module configured to receive a second data block identifier of the query file in an erasure copy returned by the server.

[0011] As an optional example, the device further comprises: a second querying module configured to send, to a domain name server, a second data block identifier of a query data block stored in an erasure copy; and a second receiving module configured to receive a first file identifier of the query data block in three copies returned by the server.

[0012] According to another aspect of the embodiments of the present application, a storage medium is provided, and the storage medium stores a computer program. When the computer program is run by a processor, the erasure block copy to three-copy bidirectional mapping method is executed.

[0013] According to another aspect of the embodiments of the present application, an electronic device is provided, and the electronic device comprises a memory and a processor. The memory stores a computer program, and the processor is configured to execute the erasure block copy to three-copy bidirectional mapping method by using the computer program.

[0014] In the embodiment of the present application, a plurality of target data blocks of three-copy data stored in a three-copy storage are obtained, wherein the plurality of target data blocks are a plurality of data blocks obtained by dividing one of the three-copy data; each of the plurality of target data blocks is taken as a current data block, and erasure coding operation is performed on the current data block to obtain M sub-data blocks and N sub-check blocks; the M sub-data blocks and the N sub-check blocks are written into corresponding erasure copies of the three-copy storage; and the forward mapping relationship and the reverse mapping relationship between the current data block and the three-copy storage and the erasure copies are written into a disk after the writing is successful. In the method, the M sub-data blocks and the N sub-check blocks are obtained by performing erasure coding on the three-copy data stored in the three-copy storage, and the M sub-data blocks and the N sub-check blocks are written into corresponding erasure copies of the three-copy storage, so that when the domain name server is queried for target data block information, the corresponding forward mapping relationship and reverse mapping relationship can be quickly displayed, and the technical problem that there is no reverse mapping relationship in the erasure coding process is solved. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0016] Figure 1 is a flow chart of an optional erasure block copy to three-copy bidirectional mapping method according to an embodiment of the present application;

[0017] Figure 2 is a system schematic diagram of an optional erasure block copy to three-copy bidirectional mapping device according to an embodiment of the present application;

[0018] Figure 3 is a structural schematic diagram of an optional erasure block copy to three-copy bidirectional mapping device according to an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the protection scope of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] According to a first aspect of an embodiment of the present application, a method for mapping erasure block copies to three copies in both directions is provided, which is optional, as shown in Figure 1 The above method comprises:

[0023] S102, obtaining a plurality of target data blocks of three-copy data stored in a three-copy storage, wherein the plurality of target data blocks are a plurality of data blocks obtained by blocking one piece of data in the three-copy data;

[0024] S104, taking each data block in the plurality of target data blocks as a current data block, performing erasure coding operation on the current data block to obtain M sub-data blocks and N sub-check blocks;

[0025] S106, writing the M sub-data blocks and the N sub-check blocks into corresponding erasure copies of the three-copy storage;

[0026] S108, after writing is successful, writing the forward mapping relationship and the reverse mapping relationship of the current data block in the three-copy storage and the erasure copies into a disk.

[0027] Optionally, in this embodiment, the erasure coding is a data protection method that divides data into fragments, expands and encodes redundant data blocks, and stores them in different locations, such as disks, storage nodes, or other geographic locations. It can increase m parts of data from n parts of original data, and restore the original data from any n parts of n+m parts of data. That is, if any less than or equal to m parts of data fail, the remaining data can still be restored. The target data block is a large file that is a target data block combined from small files, and each target data block has a unique target data block identifier. The object is a target data block composed of multiple objects. The three copies are stored as three copies in the cluster to improve data reliability and security. The erasure copy is stored on the erasure copy by erasure coding, thereby saving data storage space, and each erasure copy has a unique identifier. The target metadata file is the metadata file of the target data block, which mainly records the key metadata information of the target data block.

[0028] Optionally, in this embodiment, one of the three copies of the three-copy storage is obtained, and the data is divided into multiple data blocks according to a certain rule to obtain multiple target data blocks. The erasure coding operation is performed on each target data block in the multiple target data blocks to obtain M block sub-data blocks and N block sub-check blocks for each target data block. For example, the data in the three-copy storage includes data 1 and two copies of data 1. Data 1 is divided into three target data blocks, and eight sub-data blocks and four sub-check blocks are obtained for each target data block. A total of 24 sub-data blocks and 12 sub-check blocks are obtained. Finally, the 24 sub-data blocks and 12 sub-check blocks are written into the corresponding 12 nodes of the corresponding erasure copy of the three-copy storage. When writing, 8 sub-data blocks and 4 sub-check blocks corresponding to 1 target data block are stored in 12 blocks. Each block is stored in a node in the erasure copy. The 12 blocks are stored in different nodes. After successful writing, the bidirectional mapping relationship between each block target data block and the corresponding 8 sub-data blocks and 4 sub-check blocks is recorded to obtain the forward mapping relationship and the reverse mapping relationship between the three-copy storage and the erasure copy, and the mapping relationship is written into the disk. When searching and positioning, the mapping relationship of the target data block can be quickly located.

[0029] Optionally, in this embodiment, the M block sub-data blocks and the N block sub-check blocks are obtained by erasure coding the three-copy data of the three-copy storage, and the M block sub-data blocks and the N block sub-check blocks are written into the corresponding erasure copy of the three-copy storage, thereby achieving the purpose of quickly displaying the corresponding forward mapping relationship and reverse mapping relationship when querying the target data block information from the domain name server, and further solving the technical problem that there is no reverse mapping relationship in the erasure coding process.

[0030] As an optional example, each of the plurality of target data blocks is taken as a current data block, and the current data block is subjected to erasure coding operation to obtain M sub-data blocks and N sub-check blocks, including:

[0031] The current data block is evenly divided into M blocks to obtain M sub-data blocks;

[0032] The M sub-data blocks are subjected to erasure coding operation to obtain N sub-check blocks.

[0033] Optionally, in the embodiment, M and N are positive integers, M is an even number, and N is half of M. Each of the plurality of target data blocks is evenly divided into M blocks to obtain M sub-data blocks, and the M sub-data blocks are subjected to erasure coding operation to obtain N sub-check blocks.

[0034] As an optional example, each of the plurality of target data blocks is taken as a current data block, and the current data block is subjected to erasure coding operation to obtain M sub-data blocks and N sub-check blocks, including:

[0035] Each object in the current data block is taken as a current object;

[0036] The current object is evenly divided into M blocks;

[0037] The i-th current object is taken as an object in the i-th sub-data block.

[0038] Optionally, in the embodiment, each data block is composed of a plurality of object objects, each of the plurality of target data blocks is taken as a current data block, for example, there are three target data blocks, the first target data block is taken as a current data block, the first target data block is composed of four object objects, each object object is evenly divided into eight blocks, the first object is evenly divided to obtain eight sub-objects, the first sub-object, the second sub-object, …, and the eighth sub-object. By analogy, eight sub-objects of each object object are obtained, the first sub-object in each object object is taken as the first sub-data block, the second sub-object in each object object is taken as the second sub-data block, and by analogy, eight sub-data blocks are obtained.

[0039] As an optional example, the M sub-data blocks are subjected to erasure coding operation to obtain N sub-check blocks, including:

[0040] Selecting N+1 consecutive sub-data blocks from the M sub-data blocks;

[0041] Taking each N+1 sub-data block as a sub-check block to obtain N sub-check blocks.

[0042] Optionally, in the embodiment, from the M block sub-data blocks, N+1 continuous block sub-data blocks are selected as a sub-check block, for example, from 8 block sub-data blocks, 4+1 continuous block sub-data blocks are selected as a sub-check block, to obtain a first sub-check block (including a first block sub-data block, a second block sub-data block, a fifth block sub-data block), a second block sub-check block (including a second block sub-data block, a third block sub-data block, a sixth block sub-data block), a third block sub-check block (including a third block sub-data block, a fourth block sub-data block, a seventh block sub-data block), and a fourth block sub-check block (including a fourth block sub-data block, a fifth block sub-data block, an eighth block sub-data block).

[0043] As an optional example, the plurality of target data blocks of the three-copy data stored in the three-copy storage are obtained by:

[0044] The three-copy data is determined as 3 identical first data;

[0045] Any one of the first data is taken as the target data;

[0046] Each data of the target data of a predetermined size is taken as a block of target data.

[0047] Optionally, in the embodiment, in the three-copy storage, the three-copy data includes one original data and two copy data. Any one of the original data or the copy data is taken as the first data. The first data is divided into target data blocks according to a predetermined size. For example, the predetermined size can be 1G, 2G, 8G, etc. If it is processed according to 8G, the data from the beginning to the end of the first data is divided into a target data block every 8G. The last data less than 8G is also taken as a target data block.

[0048] As an optional example, the above method further includes:

[0049] Sending the first file identifier of the query file in the three-copy storage to the domain name server;

[0050] Receiving the second data block identifier of the query file in the erasure copy returned by the domain name server.

[0051] In the embodiment, one original file and two copy files of a file in the three-copy storage share one file ID. If the file is divided into blocks, a plurality of target data blocks are obtained. In the erasure copy, the identifier of 8 block sub-data blocks and 4 block sub-check blocks corresponding to one target data block is the same. It is mapped with the file ID.

[0052] Optionally, in the embodiment, the domain name server is a server for converting domain names and corresponding IP addresses. The domain name server performs a location search, inputs a first file identifier of a query file under three-copy storage, sends a query request to the data server, and carries the first file identifier in the query request. The data server queries a corresponding second data block identifier in the local storage according to the first file identifier, and returns the second data block identifier to the domain name server. The domain name server returns the second data block identifier to the client. Through the above query method, the second data block identifier in the corresponding erasure-corrected copy can be located and positioned, that is, the forward mapping relationship from the three-copy to the erasure-corrected copy can be displayed.

[0053] As an optional example, the above method further includes:

[0054] sending, to the domain name server, a second data block identifier of a query data block under erasure-corrected copy;

[0055] receiving a first file identifier of the query data block in the three-copy storage returned by the domain name server.

[0056] Optionally, in the embodiment, the domain name server performs a location search, inputs a second data block identifier of a query data block under erasure-corrected copy, can locate and position a first file identifier in the corresponding three-copy, sends a query request to the data server, carries the second data block identifier in the query request, the data server queries a corresponding first file identifier in the local storage according to the second data block identifier, and returns the first file identifier to the domain name server. The domain name server returns the first file identifier to the client. Through the above query method, the reverse mapping relationship from the erasure-corrected copy to the three-copy can be displayed.

[0057] Optionally, in the embodiment, a bidirectional mapping method from erasure-corrected block copy to three-copy is mainly provided, Figure 2 is a mapping relationship diagram. File 1 in the three-copy storage is divided into two target data blocks, each target data block generates 4 sub-data blocks and 2 sub-check blocks, and each of the 6 blocks is stored in 1 node of the erasure-corrected copy. Another target data block also generates 4 sub-data blocks and 2 sub-check blocks, and each block is stored in a node of the erasure-corrected copy. The whole scheme includes:

[0058] 1. The three-copy data contains one original data and two copy data. Any one of the original data or the copy data is taken as the first data;

[0059] 2. The first data is divided into target data blocks according to a predetermined size. For example, the predetermined size can be 1G, 2G, 8G, etc. For example, the predetermined size is 2G, and 15G of the first data is divided into 8 target data blocks, each of which is 2G in size. The last data block that is less than 2G is also a target data block. Thus, 8 target data blocks are obtained.

[0060] 3. Each object of each target data block is evenly divided into M blocks, and the i-th block of each object is used as an object in the i-th sub-data block. Thus, one target data block obtains M sub-data blocks, and 8 target data blocks obtain 8*M sub-data blocks.

[0061] 4. The M sub-data blocks of one target data block are processed, and N+1 continuous sub-data blocks are selected from the M sub-data blocks as a sub-check block, and N sub-check blocks are obtained. The M sub-data blocks of other target data blocks are processed in the same way.

[0062] 5. The M sub-data blocks and the N sub-check blocks are written into the corresponding erasure replica of the three-copy storage. When writing, M+N blocks corresponding to one target data block are stored in M+N nodes in the erasure replica. The M+N blocks are stored in different nodes, and any N or fewer sub-data blocks are lost, and the lost data can be restored according to the remaining sub-data blocks and all sub-check blocks.

[0063] 6. After writing is successful, the forward mapping relationship and the reverse mapping relationship between the current data block and the three-copy storage and the erasure replica are written into the disk.

[0064] 7. When searching and positioning, the first file identifier of the query file under the three-copy storage is input into the domain name server, and the second data block identifier is returned to the client by the domain name server. The second data block identifier of the query data block under the erasure replica is input into the domain name server, and the first file identifier is returned to the client by the domain name server.

[0065] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0066] According to another aspect of the embodiments of the present application, a three-copy storage erasure block replica bidirectional mapping device is also provided, as shown in Figure 3As shown, comprising:

[0067] The acquisition module 302 is configured to acquire a plurality of target data blocks of the three-copy stored three-copy data, wherein the plurality of target data blocks are a plurality of data blocks obtained by dividing one data in the three-copy data.

[0068] The processing module 304 is configured to take each data block in the plurality of target data blocks as a current data block, perform erasure coding operation on the current data block, and obtain M block sub-data blocks and N block sub-check blocks.

[0069] The first writing module 306 is configured to write the M block sub-data blocks and the N block sub-check blocks into corresponding erasure copies of the three-copy storage.

[0070] The second writing module 308 is configured to write, after successful writing, the forward mapping relationship and the reverse mapping relationship of the current data block between the three-copy storage and the erasure copies into the disk.

[0071] Optionally, in the embodiment, the erasure coding is a data protection method, which divides data into fragments, expands and encodes redundant data blocks, and stores them in different locations such as disks, storage nodes or other geographical locations. It can increase m pieces of data from n pieces of original data, and can restore the original data from any n pieces of data in n+m pieces of data. That is, if any less than or equal to m pieces of data are invalid, the remaining data can still be restored. The target data block is a target data block of a large file, which is combined by small files, and each target data block has a unique target data block identifier. The object is a target data block composed of multiple objects. The three copies are three copies of data stored in the cluster to improve the reliability and security of the data. The erasure copy is an erasure copy of data stored on the erasure copy by erasure coding, so as to save data storage space, and each erasure copy has a unique identifier. The target metadata file is a metadata file of the target data block, which mainly records the key metadata information of the target data block.

[0072] Optionally, in the embodiment, one of the three copies of the data in the three-copy storage is obtained, the data is divided into a plurality of data blocks according to a certain rule to obtain a plurality of target data blocks, each target data block in the plurality of target data blocks is subjected to an erasure coding operation to obtain M sub-data blocks and N sub-check blocks of each target data block, for example, two copies of data 1 and data 1 in the three-copy storage are obtained, data 1 is divided into three target data blocks, each target data block corresponds to 8 sub-data blocks and 4 sub-check blocks, a total of 24 sub-data blocks and 12 sub-check blocks are obtained, and finally the 24 sub-data blocks and 12 sub-check blocks are written into corresponding 12 nodes of the corresponding erasure copy of the three-copy storage. When writing, 8 sub-data blocks and 4 sub-check blocks corresponding to 1 target data block, i.e., a total of 12 blocks, are stored in one node in the erasure copy. The 12 blocks are stored in different nodes. After successful writing, a bidirectional mapping relationship of each target data block and the corresponding 8 sub-data blocks and 4 sub-check blocks is recorded to obtain a forward mapping relationship and a reverse mapping relationship of the three-copy storage and the erasure copy, and the mapping relationship is written into a disk. When searching and positioning, the mapping relationship of the target data block can be quickly located.

[0073] Optionally, in the embodiment, M sub-data blocks and N sub-check blocks are obtained by performing erasure coding on the three-copy data of the three-copy storage, and the M sub-data blocks and the N sub-check blocks are written into the corresponding erasure copy of the three-copy storage, so that when the domain name server is queried for target data block information, the corresponding forward mapping relationship and reverse mapping relationship can be quickly displayed, and the technical problem that there is no reverse mapping relationship in the erasure coding process is solved.

[0074] As an optional example, the above processing module includes:

[0075] The first processing unit is configured to divide the current data block into M blocks evenly to obtain M sub-data blocks.

[0076] The second processing unit is configured to perform an erasure coding operation on the M sub-data blocks to obtain N sub-check blocks.

[0077] Optionally, in the embodiment, M and N are positive integers, M is an even number, and N is half of M. Each target data block in the plurality of target data blocks is divided into M blocks evenly to obtain M sub-data blocks, and the M sub-data blocks are subjected to an erasure coding operation to obtain N sub-check blocks.

[0078] As an optional example, the above first processing unit includes:

[0079] The first processing sub-unit is configured to take each object in the current data block as a current object.

[0080] The second processing subunit is configured to divide the current object into M blocks averagely.

[0081] The third processing subunit is configured to take the i-th block of the current object as an object in the i-th block of the sub-data block.

[0082] Optionally, in the embodiment, each data block is composed of a plurality of object objects, each block of the plurality of target data blocks is taken as a current data block, for example, there are three target data blocks, the first target data block is taken as a current data block, the first target data block is composed of four object objects, each object object is divided into eight blocks averagely, the first object is divided into eight sub-objects, i.e., a first sub-object, a second sub-object,..., and an eighth sub-object, and the same operation is performed on each object object to obtain eight sub-objects of each object object, the first sub-object in each object object is taken as a first block of the sub-data block, the second sub-object in each object object is taken as a second block of the sub-data block, and the same operation is performed to obtain eight blocks of the sub-data block.

[0083] As an optional example, the second processing unit includes:

[0084] The fourth processing subunit is configured to select continuous N+1 blocks of the sub-data block from the M blocks of the sub-data block.

[0085] The fifth processing subunit is configured to take each N+1 block of the sub-data block as a sub-check block to obtain N blocks of the sub-check block.

[0086] Optionally, in the embodiment, continuous N+1 blocks of the sub-data block are selected from M blocks of the sub-data block as a sub-check block, for example, continuous 4+1 blocks of the sub-data block are selected from eight blocks of the sub-data block as a sub-check block to obtain a first block of the sub-check block (including a first block of the sub-data block, a second block of the sub-data block,..., and a fifth block of the sub-data block), a second block of the sub-check block (including a second block of the sub-data block, a third block of the sub-data block,..., and a sixth block of the sub-data block), a third block of the sub-check block (including a third block of the sub-data block, a fourth block of the sub-data block,..., and a seventh block of the sub-data block), and a fourth block of the sub-check block (including a fourth block of the sub-data block, a fifth block of the sub-data block,..., and an eighth block of the sub-data block).

[0087] As an optional example, the acquisition module includes:

[0088] The third processing unit is configured to determine the three copies of data as three identical first data.

[0089] The fourth processing unit is configured to take any one of the first data as target data.

[0090] Each predetermined size of data of the target data is taken as a block of the target data block.

[0091] Optionally, in the embodiment, in the three-copy storage, the three-copy data includes one original data and two copy data. Any one of the original data or the copy data is taken as the first data. The first data is divided into target data blocks according to a predetermined size. For example, the predetermined size can be 1G, 2G, 8G, etc. If processing according to 8G, the data from the beginning to the end of the first data is divided into a target data block every 8G. The last data less than 8G is also taken as a target data block.

[0092] As an optional example, the apparatus further includes:

[0093] The first query module is configured to send, to the domain name server, a first file identifier of a query file in the three-copy storage.

[0094] The first receiving module is configured to receive, from the server, a second data block identifier of the query file in the erasure copy.

[0095] In the embodiment, one original file and two copy files of a file in the three-copy storage share one file ID. If the file is divided into blocks, a plurality of target data blocks are obtained. In the erasure copy, the identifier of 8 sub-data blocks and 4 sub-check blocks corresponding to one target data block is the same as the file ID.

[0096] Optionally, in the embodiment, the domain name server is a server for converting a domain name and a corresponding IP address. The positioning and searching are performed in the domain name server. The first file identifier of a query data block in the three-copy storage is input. The domain name server sends a query request to the data server. The query request carries the first file identifier. The data server queries the corresponding second data block identifier in the local storage according to the first file identifier and returns the second data block identifier to the domain name server. The domain name server returns the second data block identifier to the client. Through the above query method, the second data block identifier in the corresponding erasure copy can be located and searched, i.e., the positive mapping relationship from the three-copy to the erasure copy can be displayed.

[0097] As an optional example, the apparatus further includes:

[0098] The second query module is configured to send, to the domain name server, a second data block identifier of a query data block in the erasure copy.

[0099] The second receiving module is configured to receive, from the server, a first file block identifier of the query data block in the three-copy storage.

[0100] Optionally, in the embodiment, the locating lookup is performed in the domain name server, the second data block identifier of the query data block under the erasure coding replica is input, the first file identifier corresponding to the three replicas can be located, the domain name server sends a query request to the data server, the query request carries the second data block identifier, the data server queries the corresponding first file identifier in the local storage according to the second data block identifier, and returns the first file identifier to the domain name server, and the domain name server returns the first file identifier to the client. Through the above query method, the reverse mapping relationship from the erasure coding replica to the three replicas can be displayed.

[0101] Other examples of the embodiment can refer to the above examples, which will not be described here.

[0102] Figure 4 is a structural block diagram of an optional electronic device according to an embodiment of the application, as shown in Figure 4 The processor 402, the communication interface 404 and the memory 406 can complete mutual communication through the communication bus 408, wherein,

[0103] The memory 406 is configured to store a computer program.

[0104] The processor 402 is configured to execute the computer program stored in the memory 406, and implement the following steps:

[0105] Obtain a plurality of target data blocks of the three-replica data stored in the three-replica storage, wherein the plurality of target data blocks are a plurality of data blocks obtained by dividing one data in the three-replica data;

[0106] Take each data block in the plurality of target data blocks as a current data block, perform an erasure coding operation on the current data block, and obtain M sub-data blocks and N sub-check blocks;

[0107] Write the M sub-data blocks and the N sub-check blocks into the corresponding erasure coding replica of the three-replica storage;

[0108] After the writing is successful, write the forward mapping relationship and the reverse mapping relationship between the current data block and the three-replica storage and the erasure coding replica into the disk.

[0109] Optionally, in the embodiment, the communication bus can be a PCI (Peripheral Component Interconnect, peripheral component interconnect) bus, an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, or the like. The communication bus can be divided into an address bus, a data bus, a control bus, and the like. For the convenience of representation,Figure 4 Only one bus is shown among the possible number of buses, but it is understood that there can be many buses for passing data / information, commands, instructions, or the like, between any two or more of the various components or devices shown or described in the specification. Communication interface is used for communication between the above-mentioned electronic device and other devices.

[0110] The memory can include a RAM, and can also include a non-volatile memory, for example, at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0111] As an example, the above-mentioned memory 406 can include, but is not limited to, the above-mentioned obtaining module 302, processing module 304, first writing module 306 and second writing module 308 in the erasure block copy to three-copy bidirectional mapping device. In addition, other module units in the above-mentioned processing device of the request can also be included, but not limited to, which will not be described in detail in this example.

[0112] The above-mentioned processor can be a general-purpose processor, which can include, but is not limited to: CPU (Central Processing Unit, Central Processing Unit), NP (Network Processor, Network Processor) and the like; it can also be a DSP (Digital Signal Processing, Digital Signal Processor), ASIC (Application Specific Integrated Circuit, Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array, Field-Programmable Gate Array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.

[0113] Optionally, the specific examples in the present embodiment can refer to the examples described in the above-mentioned embodiments, which will not be described in detail in this embodiment.

[0114] Those of ordinary skill in the art can understand that, Figure 4 The structure shown is only schematic, and the device for implementing the above-mentioned erasure block copy to three-copy bidirectional mapping method can be a terminal device, which can be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD terminal device, etc. Figure 4 It does not limit the structure of the above-mentioned electronic device. For example, the electronic device can further include more or less components (such as network interfaces, display devices, etc.) than Figure 4 The structure shown. Figure 4

[0115] ​Those skilled in the art can understand that all or part of the steps of various methods in the above embodiments can be completed by instructing the terminal device related hardware through a program, and the program can be stored in a computer readable storage medium, and the storage medium can include a flash disk, a ROM, a RAM, a magnetic disk, an optical disk, etc.

[0116] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program. When the computer program is run by a processor, the steps of the above-mentioned erasure block copy to three-copy bidirectional mapping method are executed.

[0117] Optionally, in the present embodiment, those skilled in the art can understand that all or part of the steps of various methods in the above embodiments can be completed by instructing the terminal device related hardware through a program, and the program can be stored in a computer readable storage medium, and the storage medium can include a flash disk, a ROM, a RAM, a magnetic disk, an optical disk, etc.

[0118] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0119] The integrated units in the above-mentioned embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the above-mentioned computer readable storage medium. Based on such understanding, the technical solutions of the present application or all or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions for causing one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application.

[0120] In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0121] In several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other manners. Of course, the described apparatus embodiments are merely schematic, and the division of units is merely logical function division, and there can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, units or modules, and can be in electrical, mechanical or other forms.

[0122] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0123] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0124] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled persons in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A bidirectional mapping method from erasure block replicas to three replicas, characterized in that, The method comprises the following steps: obtaining a plurality of target data blocks of three-copy data stored in a three-copy storage, wherein the plurality of target data blocks are a plurality of data blocks obtained by block division on one of the three-copy data; taking each of the plurality of target data blocks as a current data block, performing erasure coding operation on the current data block to obtain M sub-data blocks and N sub-check blocks; writing the M sub-data blocks and the N sub-check blocks into corresponding erasure copies of the three-copy storage; after successful writing, writing a forward mapping relationship and a reverse mapping relationship between the current data block and the erasure copies into a disk; the method further comprises: sending, to a domain name server, a second data block identifier of a query data block to be queried under an erasure copy; and receiving a first file identifier of the query data block in the three-copy storage returned by a server, wherein the server also returns the first file identifier to the domain name server.

2. The method of claim 1, wherein, the step of taking each of the plurality of target data blocks as a current data block, performing erasure coding operation on the current data block to obtain M sub-data blocks and N sub-check blocks comprises: dividing the current data block into M blocks equally to obtain the M sub-data blocks; performing erasure coding operation on the M sub-data blocks to obtain the N sub-check blocks.

3. The method of claim 2, wherein, the step of dividing the current data block into M blocks equally to obtain the M sub-data blocks comprises: taking each object in the current data block as a current object; dividing the current object into M blocks equally; taking an i-th current object as an object in an i-th sub-data block.

4. The method of claim 2, wherein, the step of performing erasure coding operation on the M sub-data blocks to obtain the N sub-check blocks comprises: selecting N+1 continuous sub-data blocks from the M sub-data blocks; taking each N+1 sub-data blocks as a sub-check block to obtain the N sub-check blocks.

5. The method of claim 1, wherein, the step of obtaining a plurality of target data blocks of three-copy data stored in a three-copy storage comprises: determining the three-copy data as three identical first data; taking any one of the first data as target data; taking each data of a predetermined size of the target data as a target data block.

6. The method of claim 1, wherein, the method further comprises: sending, to a domain name server, a first file identifier of a query file to be queried under a three-copy storage; receiving a second data block identifier of the query file in an erasure copy returned by a server.

7. A bidirectional mapping device for erasure block replicas to three replicas, characterized in that, The method comprises the following steps: an obtaining module is configured to obtain a plurality of target data blocks of three-copy data stored in a three-copy storage, wherein the plurality of target data blocks are a plurality of data blocks obtained by block division on one of the three-copy data; a processing module is configured to take each of the plurality of target data blocks as a current data block, perform erasure coding operation on the current data block to obtain M sub-data blocks and N sub-check blocks; a first writing module is configured to write the M sub-data blocks and the N sub-check blocks into corresponding erasure copies of the three-copy storage; a second writing module, configured to write the current data block in the three-copy storage and the positive and negative mapping relations of the erasure-corrected copy into the disk after successful writing; a second querying module, configured to send, to a domain name server, a second data block identifier of a query data block under the erasure-corrected copy; a second receiving module, configured to receive a first file identifier of the query data block in the three-copy storage returned by a server, and the server also returns the first file identifier to the domain name server.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium being characterized by, The computer program is run by the processor to execute the method in any one of claims 1 to 6.

9. An electronic device comprising a memory and a processor, characterized in that The computer program is run by the processor to execute the method in any one of claims 1 to 6.

Images