Data copy processing method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a data copy processing method and device, a storage medium and electronic equipment, and relates to the technical field of storage, the method is applied to a distributed storage system, the distributed storage system comprises a plurality of storage nodes, and storage data of the distributed storage system has a plurality of data copies. A plurality of data copies are respectively stored in different storage nodes, and the method comprises the following steps: analyzing a received data read-write request to obtain a data identifier of target storage data requested by the data read-write request; selecting a target data copy from the plurality of data copies according to the request type of the data read-write request and copy states of the plurality of data copies corresponding to the data identifier; and processing the target data copy according to the data read-write request. The problem of how to improve the data processing efficiency in the distributed storage system in related technologies is solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of storage technology, and specifically, to a method, device, storage medium and electronic device for processing data copies. Background Art

[0002] Distributed storage systems usually use a multi-copy strategy to ensure the security of stored data, that is, the same data object is stored on multiple storage nodes to avoid the problem of being unable to access data after a storage node fails. At the same time, in order to ensure the consistency between multiple data copies of the data object, the business request is usually forwarded to all data copies when making a business request, and the business success message is returned only after confirming that the business request has been completed on all data copies.

[0003] In the prior art, when a failure occurs in a distributed storage system and some data copies become inaccessible, the system will recover based on the existing data copies to maintain the multiple-copy strategy of the business data object, that is, to reconstruct the data copies. During the process of reconstructing the copies, in order to ensure the consistency of multiple copies, the distributed storage system needs to wait for the reconstruction to be completed before responding to business requests, which will obviously increase the delay in responding to business requests during the failure process, causing a serious degradation in the performance of the distributed storage system.

[0004] Therefore, in the related art, there is a problem of how to improve the efficiency of data processing in a distributed storage system.

[0005] Regarding the problem of how to improve the data processing efficiency in a distributed storage system in related technologies, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present application provide a method, device, storage medium and electronic device for processing data copies, so as to at least solve the problem of how to improve the efficiency of data processing in a distributed storage system in the related art.

[0007] According to one embodiment of the present application, a method for processing data copies is provided, which is applied to a distributed storage system, wherein the distributed storage system includes multiple storage nodes, and the storage data of the distributed storage system has multiple data copies, and the multiple data copies are respectively stored in different storage nodes. The method includes: parsing a received data read / write request to obtain a data identifier of the target storage data requested by the data read / write request; selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy status of the multiple data copies corresponding to the data identifier; and processing the target data copy according to the data read / write request.

[0008] In an exemplary embodiment, the request types of the data read and write requests include read requests and write requests, the copy status includes a complete status and an alternative status, and a target data copy is selected from the multiple data copies according to the request type of the data read and write requests and the copy status of the multiple data copies corresponding to the data identifier, including: when it is determined that the request type is a read request, a first target data copy is determined from the multiple data copies, wherein the first target data copy represents a data copy whose copy status is the complete status among the multiple data copies; when it is determined that the request type is a write request, a second target data copy is determined from the multiple data copies, wherein the second target data copy represents a data copy whose copy status is the complete status or the alternative status among the multiple data copies.

[0009] In an exemplary embodiment, when it is determined that the request type is a write request, the target data copy is processed according to the data read and write request, including: determining a target data block from the second target data copy, wherein the second target data copy includes multiple data blocks, and the target data block represents a data block among the multiple data blocks to which data writing has occurred; and updating a data bitmap of the second target data copy according to the target data block, wherein the data bitmap includes multiple bits, and one bit corresponds to a data block in the second target copy.

[0010] In an exemplary embodiment, when it is determined that the request type is a write request, the target data copy is processed according to the data read and write request, and it also includes: when it is determined that the copy status of the second target data copy is an alternative status, the second target data copy is synchronized according to the content of the third target data copy, wherein the third target data copy represents a data copy with any one copy status as a complete status.

[0011] In an exemplary embodiment, data synchronization is performed on the second target data copy according to the content of the third target data copy, including: obtaining a data bitmap of the second target data copy from the storage node where the second target data copy is located; sending the data bitmap of the second target data copy to the storage node where the third target data copy is located, and obtaining the data bitmap of the third target data copy from the storage node where the third target data copy is located; determining a difference bitmap and difference data according to the data bitmap of the second target data copy and the data bitmap of the third target data copy, wherein the difference bitmap is used to represent data blocks with inconsistent contents in the second target data copy and the third target data copy, and the difference data represents data corresponding to the inconsistent data blocks in the third target data copy; sending the difference bitmap and the difference data to the storage node where the second target data copy is located, and updating the second target data copy according to the difference bitmap and the difference data to complete the data synchronization.

[0012] In an exemplary embodiment, the copy status includes a complete status, an alternative status, and an eliminated status, and the method further includes: in the case where it is determined that the storage node where the first data copy is located has failed, updating the copy status of the first data copy to an eliminated status, wherein the first data copy represents any copy whose copy status is in a complete status; and / or, after determining that the first storage node has repaired the failure and is back online, comparing the contents of the second data copy and the third data copy, and in the case where it is determined that the contents of the second data copy and the third data copy are consistent, updating the copy status of the second data copy to a complete status, and in the case where it is determined that the contents of the second data copy and the third data copy are inconsistent, updating the copy status of the second data copy to an alternative status, wherein the first storage node indicates that it has entered the state after sending the failure. a storage node for offline processing, the second data copy represents the copy in the first storage node, and the third data copy represents any one of the copies in a complete state; and / or, comparing the contents of the fourth data copy and the fifth data copy, and when it is determined that the contents of the fourth data copy and the fifth data copy are consistent, updating the copy status of the fourth data copy to a complete state, and when it is determined that the contents of the fourth data copy and the fifth data copy are inconsistent, performing data synchronization on the fourth data copy according to the content of the fifth data copy, and after determining that the data synchronization is completed, updating the copy status of the fourth data copy to a complete state, wherein the fourth data copy represents any one of the copies in an alternative state, and the fifth data copy represents any one of the copies in a complete state.

[0013] In an exemplary embodiment, when it is determined that the storage node where the first data copy is located fails, after updating the status of the first data copy to an obsolete state, the method further includes: selecting any one storage node from the second storage node to build a new data copy of the stored data, wherein the second storage node represents a storage node among the multiple storage nodes that does not store a data copy of the target storage data; during the construction of the new data copy, setting the copy status of the new data copy to an alternative state; and when it is determined that the new data copy has been completed, setting the copy status of the new data copy to a complete state.

[0014] According to another embodiment of the present application, a data copy processing device is provided, including: a parsing module, used to parse a received data read / write request to obtain a data identifier of a target storage data requested by the data read / write request; a selection module, used to select a target data copy from the multiple data copies according to the request type of the data read / write request and the copy status of the multiple data copies corresponding to the data identifier; and a processing module, used to process the target data copy according to the data read / write request.

[0015] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0016] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0017] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0018] Through the present application, different data copy states can be defined, and a target data copy can be selected from multiple data copies to execute the data read or write request based on the request type of the received data read or write request and the copy state of the data copy corresponding to the storage data requested by the data read or write request; thereby solving the problem in the related art of how to improve the data processing efficiency in a distributed storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a hardware structure block diagram of a server device of a data copy processing method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of a method for processing a data copy according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a method for processing a data copy according to an embodiment of the present application (I);

[0022] Figure 4 is a schematic diagram of a method for processing a data copy according to an embodiment of the present application (II);

[0023] Figure 5 is a schematic diagram of a method for processing a data copy according to an embodiment of the present application (III);

[0024] Figure 6 It is a structural block diagram of a data copy processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0027] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a data copy processing method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 The server device may include a processor 102 (only one is shown) (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components than shown, or with different Figure 1 Different configurations shown.

[0028] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for processing data copies in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the method for processing the above-mentioned data copies is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory set relative to the processor 102 process, and these process memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] In this embodiment, a method for processing data copies is provided, which is applied to a distributed storage system. The distributed storage system includes multiple storage nodes. The storage data of the distributed storage system has multiple data copies, and the multiple data copies are respectively stored in different storage nodes. Figure 2 is a flowchart of a method for processing a data copy according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0031] Step S202, parsing the received data read / write request to obtain the data identifier of the target storage data requested by the data read / write request;

[0032] Optionally, in the above step S202, the target storage data has been stored in the distributed storage system and has multiple data copies. All data copies corresponding to the target storage data can be found through the above data identifier.

[0033] Step S204, selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy states of the multiple data copies corresponding to the data identifier;

[0034] Step S206: Process the target data copy according to the data read and write request.

[0035] Optionally, in the above step S206, processing the target data copy includes reading data in the data copy, writing data to the data copy, etc.

[0036] Through the above steps, different data copy states can be defined, and a target data copy can be selected from multiple data copies to execute the data read or write request based on the request type of the received data read or write request and the copy state of the data copy corresponding to the storage data requested by the data read or write request; thereby solving the problem in the related technology of how to improve the data processing efficiency in the distributed storage system.

[0037] In an exemplary embodiment, the request types of the data read and write requests include read requests and write requests, the copy status includes a complete status and an alternative status, and a target data copy is selected from the multiple data copies according to the request type of the data read and write requests and the copy status of the multiple data copies corresponding to the data identifier, including: when it is determined that the request type is a read request, a first target data copy is determined from the multiple data copies, wherein the first target data copy represents a data copy whose copy status is the complete status among the multiple data copies; when it is determined that the request type is a write request, a second target data copy is determined from the multiple data copies, wherein the second target data copy represents a data copy whose copy status is the complete status or the alternative status among the multiple data copies.

[0038] Optionally, in the above embodiment, the data copy in the complete state has the complete content of the target storage data and can perform business IO (Input / Output) read and write requests. The data copy in the alternative state is the state when the data copy storage node is changed due to a system exception, and the data copy on the changed node has not completed data recovery. Business IO write operations can be performed, but read operations cannot be performed.

[0039] In an exemplary embodiment, when it is determined that the request type is a write request, the target data copy is processed according to the data read and write request, including: determining a target data block from the second target data copy, wherein the second target data copy includes multiple data blocks, and the target data block represents a data block among the multiple data blocks to which data writing has occurred; and updating a data bitmap of the second target data copy according to the target data block, wherein the data bitmap includes multiple bits, and one bit corresponds to a data block in the second target copy.

[0040] In an alternative embodiment, Figure 3 The data bitmap showing the data copies, such as Figure 3 As shown, the data copy is divided into multiple data blocks of the same size, and each bit ( Figure 3 The block in the figure corresponds to a data block, and the data block can also be marked as used state and idle state, which are used to mark the validity of the business data in the data block. That is, when the business data is written, the business data is written to the associated data block and marked as used state. As the data copy processes more write business, the bitmap will gradually change from idle state to used state.

[0041] In an exemplary embodiment, when it is determined that the request type is a write request, the target data copy is processed according to the data read and write request, and it also includes: when it is determined that the copy status of the second target data copy is an alternative status, the second target data copy is synchronized according to the content of the third target data copy, wherein the third target data copy represents a data copy with any one copy status as a complete status.

[0042] Optionally, in the above embodiment, for the data copy in the alternative state, after the data is written, the data copy needs to be synchronized, and the data source of the synchronization comes from the data copy in the complete state. After the synchronization is completed, the state of the data copy can be converted to the complete state.

[0043] In an exemplary embodiment, data synchronization is performed on the second target data copy according to the content of the third target data copy, including: obtaining a data bitmap of the second target data copy from the storage node where the second target data copy is located; sending the data bitmap of the second target data copy to the storage node where the third target data copy is located, and obtaining the data bitmap of the third target data copy from the storage node where the third target data copy is located; determining a difference bitmap and difference data according to the data bitmap of the second target data copy and the data bitmap of the third target data copy, wherein the difference bitmap is used to represent data blocks with inconsistent contents in the second target data copy and the third target data copy, and the difference data represents data corresponding to the inconsistent data blocks in the third target data copy; sending the difference bitmap and the difference data to the storage node where the second target data copy is located, and updating the second target data copy according to the difference bitmap and the difference data to complete the data synchronization.

[0044] Optionally, in the above embodiment, when the data copy in the alternative state performs data synchronization, the data bitmap of the alternative copy is first synchronized to the storage node where the complete copy is located, and the data bitmap of the incomplete copy is compared and the difference bitmap and difference data are output. The storage node where the complete copy is located synchronizes the difference bitmap and difference data to the storage node where the alternative copy is located. The alternative copy applies the difference data to the storage node where the alternative copy is located according to the difference bitmap information. After successful application, the alternative copy is converted into a data copy in the complete state.

[0045] In an exemplary embodiment, the copy status includes a complete status, an alternative status, and an eliminated status, and the method further includes: in the case where it is determined that the storage node where the first data copy is located has failed, updating the copy status of the first data copy to an eliminated status, wherein the first data copy represents any copy whose copy status is in a complete status; and / or, after determining that the first storage node has repaired the failure and is back online, comparing the contents of the second data copy and the third data copy, and in the case where it is determined that the contents of the second data copy and the third data copy are consistent, updating the copy status of the second data copy to a complete status, and in the case where it is determined that the contents of the second data copy and the third data copy are inconsistent, updating the copy status of the second data copy to an alternative status, wherein the first storage node indicates that it has entered the state after sending the failure. a storage node for offline processing, the second data copy represents the copy in the first storage node, and the third data copy represents any one of the copies in a complete state; and / or, comparing the contents of the fourth data copy and the fifth data copy, and when it is determined that the contents of the fourth data copy and the fifth data copy are consistent, updating the copy status of the fourth data copy to a complete state, and when it is determined that the contents of the fourth data copy and the fifth data copy are inconsistent, performing data synchronization on the fourth data copy according to the content of the fifth data copy, and after determining that the data synchronization is completed, updating the copy status of the fourth data copy to a complete state, wherein the fourth data copy represents any one of the copies in an alternative state, and the fifth data copy represents any one of the copies in a complete state.

[0046] In an optional embodiment, if Figure 4 As shown in the figure, the copy states of data copies can be converted to each other. For example, when an abnormality occurs in the storage node where the complete copy is located, such as network disconnection, power outage, or service restart, the complete copy is converted to a retired copy; after the abnormal node is online, the retired copy on the abnormal node can be compared with the complete copies of other nodes. If the two are inconsistent, the retired copy is converted to an alternative copy, and if the two are consistent, the retired copy is converted to a complete copy; the alternative copy can synchronize data with the incomplete copy. After the synchronization is completed, the alternative copy is converted to a complete copy.

[0047] Through the above embodiments, the replica state can be converted according to the data integrity of the replica and the situation of the storage node, the replica data can be quickly restored, and the impact of the faulty node on the data read and write request is minimized.

[0048] In an exemplary embodiment, when it is determined that the storage node where the first data copy is located fails, after updating the status of the first data copy to an obsolete state, the method further includes: selecting any one storage node from the second storage node to build a new data copy of the stored data, wherein the second storage node represents a storage node among the multiple storage nodes that does not store a data copy of the target storage data; during the construction of the new data copy, setting the copy status of the new data copy to an alternative state; and when it is determined that the new data copy has been completed, setting the copy status of the new data copy to a complete state.

[0049] Optionally, in the above embodiment, for example, a piece of data has three data copies, which are stored on different storage nodes. After a node where one of the data copies is located fails, it is necessary to build a new data copy on a new storage node. This new storage node cannot be the same node as the other two data copies. The new data copy is in an alternative state during the construction process and is in a complete state after the construction is completed.

[0050] Through the above embodiments, it can be ensured that the number of data copies is maintained at a preset number, and the security and unreliability of the data can be fully guaranteed by distributing the data copies in different storage nodes.

[0051] Obviously, the embodiments described above are only some embodiments of the present application, not all embodiments. In order to better understand the verification method of the above test case, the above process is described below in conjunction with the embodiments, but it is not intended to limit the technical solutions of the embodiments of the present application. Specifically:

[0052] In an optional embodiment, if Figure 5As shown, there are three storage nodes in the distributed storage system, namely, node A, node B, and node C. The storage strategy of the system for the target storage data is to keep storing two data copies, namely, data copy 1 and data copy 2. In the initial state, the target storage data has complete data copies saved on both storage node A and storage node B. When there is no abnormality in the storage system, the IO request for the target storage data is forwarded to node A and node B for processing. When node B fails, data copy 2 on node B becomes an obsolete copy. In order to ensure that the target storage data maintains two copies, a data copy 2 is rebuilt on node C based on data copy 1. During the construction process, data copy 2 is an alternative copy. The copies of the above three states are defined as follows:

[0053] Complete copy: It has complete data of the target storage data and can perform business IO reading and writing;

[0054] Alternative copy: The data copy storage node is changed due to a system exception. The data copy on the changed node has not yet completed data recovery. Business IO write operations can be performed but reading is unavailable.

[0055] Eliminate copies: If a data copy is unavailable due to a storage node failure, the data validity needs to be further confirmed after the node failure is repaired before business IO reading and writing can be performed.

[0056] The copies of the above three states can be converted, and the conversion relationship is as follows: Figure 3 As shown:

[0057] 1. When an exception occurs in the storage node where the full copy is located, such as network disconnection, power outage, or service restart, the full copy is converted to an obsolete copy.

[0058] 2. After the abnormal node comes online, the eliminated copies on the abnormal node are compared with the complete copies of other nodes. If both are consistent, the eliminated copies are converted to complete copies; the standby copies can synchronize data with incomplete copies. After the synchronization is completed, the standby copies are converted to complete copies.

[0059] 3. After the abnormal node comes online, the eliminated copy on the abnormal node is compared with the complete copies of other nodes. If the two are inconsistent, the eliminated copy is converted into a backup copy.

[0060] 4. The candidate replica is converted into a complete replica after data synchronization with the complete replicas of other nodes.

[0061] Through the above embodiments, the problem of maintaining consistency in the write service of data copies when a failure occurs is solved. By changing the state of the data copy after the old node is online, the problem of blocking the write service after failure recovery is solved. By defining different data copy states and state machine flows as well as read-write strategies, the impact of failures on the performance of the distributed storage system is greatly reduced.

[0062] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of inferior software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0063] In this embodiment, a data copy processing device is also provided. The system is used to implement the above-mentioned embodiments and preferred implementation modes. The descriptions that have been made are not repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0064] Figure 6 is a structural block diagram of a data copy processing device according to an embodiment of the present application, such as Figure 6 As shown, the device comprises:

[0065] The parsing module 62 is used to parse the received data read / write request to obtain the data identifier of the target storage data requested by the data read / write request;

[0066] A selection module 64, configured to select a target data copy from the multiple data copies according to the request type of the data read / write request and the copy states of the multiple data copies corresponding to the data identifier;

[0067] The processing module 66 is used to process the target data copy according to the data read and write request.

[0068] Through the above-mentioned device, different data copy states can be defined, and according to the request type of the received data read and write request and the copy state of the data copy corresponding to the storage data requested by the data read and write request, a target data copy can be selected from multiple data copies to execute the data read and write request; thereby, the problem in the related technology of how to improve the data processing efficiency in the distributed storage system can be solved.

[0069] In an exemplary embodiment, the selection module 64 is further used to: when it is determined that the request type is a read request, determine a first target data copy from the multiple data copies, wherein the first target data copy represents a data copy whose copy status is the complete status among the multiple data copies; when it is determined that the request type is a write request, determine a second target data copy from the multiple data copies, wherein the second target data copy represents a data copy whose copy status is the complete status or the alternative status among the multiple data copies.

[0070] In an exemplary embodiment, the above-mentioned processing module 66 is also used to: determine a target data block from the second target data copy, wherein the second target data copy includes multiple data blocks, and the target data block represents a data block among the multiple data blocks where data writing has occurred; and update a data bitmap of the second target data copy according to the target data block, wherein the data bitmap includes multiple bits, and one bit corresponds to a data block in the second target copy.

[0071] In an exemplary embodiment, the above-mentioned processing module 66 is also used to: when it is determined that the copy state of the second target data copy is an alternative state, synchronize the data of the second target data copy according to the content of the third target data copy, wherein the third target data copy represents a data copy whose copy state is a complete state.

[0072] In an exemplary embodiment, the above-mentioned processing module 66 is also used to: obtain the data bitmap of the second target data copy from the storage node where the second target data copy is located; send the data bitmap of the second target data copy to the storage node where the third target data copy is located, and obtain the data bitmap of the third target data copy from the storage node where the third target data copy is located; determine the difference bitmap and difference data according to the data bitmap of the second target data copy and the data bitmap of the third target data copy, wherein the difference bitmap is used to represent data blocks with inconsistent contents in the second target data copy and the third target data copy, and the difference data represents the data corresponding to the inconsistent data blocks in the third target data copy; send the difference bitmap and the difference data to the storage node where the second target data copy is located, and update the second target data copy according to the difference bitmap and the difference data to complete the data synchronization.

[0073] In an exemplary embodiment, the above-mentioned device is also used to: when it is determined that the storage node where the first data copy is located has failed, update the copy status of the first data copy to an eliminated state, wherein the first data copy represents a copy whose copy status is in a complete state; and / or, after determining that the first storage node has repaired the failure and is back online, compare the contents of the second data copy and the third data copy, and when it is determined that the contents of the second data copy and the third data copy are consistent, update the copy status of the second data copy to a complete state, and when it is determined that the contents of the second data copy and the third data copy are inconsistent, update the copy status of the second data copy to an alternative state, wherein the first storage node represents a storage node that has been offline after sending the failure, and the The second data copy represents the copy in the first storage node, and the third data copy represents a copy whose status is a complete state; and / or, comparing the contents of the fourth data copy and the fifth data copy, and when it is determined that the contents of the fourth data copy and the fifth data copy are consistent, updating the copy status of the fourth data copy to a complete state, and when it is determined that the contents of the fourth data copy and the fifth data copy are inconsistent, performing data synchronization on the fourth data copy according to the content of the fifth data copy, and after determining that the data synchronization is completed, updating the copy status of the fourth data copy to a complete state, wherein the fourth data copy represents a copy whose status is an alternative state, and the fifth data copy represents a copy whose status is a complete state.

[0074] In an exemplary embodiment, the above-mentioned device is also used to: select any one storage node from the second storage node to build a new data copy of the stored data, wherein the second storage node represents a storage node among the multiple storage nodes that does not store a data copy of the target storage data; during the construction of the new data copy, set the copy status of the new data copy to an alternative status; when it is determined that the new data copy has been built, set the copy status of the new data copy to a complete status.

[0075] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0076] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0077] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0078] S1, selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy status of the multiple data copies corresponding to the data identifier;

[0079] S2, selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy states of the multiple data copies corresponding to the data identifier;

[0080] S3: Process the target data copy according to the data read and write request.

[0081] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0082] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0083] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0084] S1, selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy status of the multiple data copies corresponding to the data identifier;

[0085] S2, selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy states of the multiple data copies corresponding to the data identifier;

[0086] S3: Process the target data copy according to the data read and write request.

[0087] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0088] Optionally, in this embodiment, the computer program may be configured to perform the following steps by means of the computer program:

[0089] S1, selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy status of the multiple data copies corresponding to the data identifier;

[0090] S2, selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy states of the multiple data copies corresponding to the data identifier;

[0091] S3: Process the target data copy according to the data read and write request.

[0092] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0093] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0094] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing a data copy, characterized in that: Applied to a distributed storage system, the distributed storage system includes multiple storage nodes, the storage data of the distributed storage system has multiple data copies, and the multiple data copies are respectively stored in different storage nodes. The method includes: parsing a received data read and write request to obtain a data identifier of the target storage data requested by the data read and write request; Selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy states of the multiple data copies corresponding to the data identifier; The target data copy is processed according to the data read and write request.

2. The method according to claim 1, characterized in that The request type of the data read / write request includes a read request and a write request, the copy status includes a complete status and an alternative status, and selecting a target data copy from the multiple data copies according to the request type of the data read / write request and the copy status of the multiple data copies corresponding to the data identifier includes: In the case where it is determined that the request type is a read request, determining a first target data copy from the multiple data copies, wherein the first target data copy represents a data copy of any one of the multiple data copies whose state is the complete state; When it is determined that the request type is a write request, a second target data copy is determined from the multiple data copies, wherein the second target data copy represents a data copy among the multiple data copies whose copy status is the complete status or the alternative status.

3. The method according to claim 2, characterized in that When it is determined that the request type is a write request, processing the target data copy according to the data read and write request includes: Determine a target data block from the second target data copy, wherein the second target data copy includes a plurality of data blocks, and the target data block represents a data block in which data writing occurs among the plurality of data blocks; The data bitmap of the second target data copy is updated according to the target data block, wherein the data bitmap includes a plurality of bits, and one bit corresponds to one data block in the second target copy.

4. The method according to claim 2, characterized in that: In the case where it is determined that the request type is a write request, processing the target data copy according to the data read and write request further includes: When it is determined that the copy state of the second target data copy is an alternative state, data synchronization is performed on the second target data copy according to the content of the third target data copy, wherein the third target data copy represents any data copy whose copy state is a complete state.

5. The method according to claim 4, characterized in that Synchronizing data on the second target data copy according to the content of the third target data copy includes: Acquire a data bitmap of the second target data copy from the storage node where the second target data copy is located; Sending the data bitmap of the second target data copy to the storage node where the third target data copy is located, and obtaining the data bitmap of the third target data copy from the storage node where the third target data copy is located; Determine a difference bitmap and difference data according to the data bitmap of the second target data copy and the data bitmap of the third target data copy, wherein the difference bitmap is used to represent data blocks with inconsistent contents between the second target data copy and the third target data copy, and the difference data represents data corresponding to the inconsistent data blocks in the third target data copy; The difference bitmap and the difference data are sent to the storage node where the second target data copy is located, and the second target data copy is updated according to the difference bitmap and the difference data to complete the data synchronization.

6. The method according to claim 1, characterized in that The replica status includes a complete status, an alternative status, and an eliminated status. The method further includes: When it is determined that the storage node where the first data copy is located fails, updating the copy state of the first data copy to an eliminated state, wherein the first data copy represents any copy whose copy state is a complete state; And / or, after determining that the first storage node has repaired the fault and is back online, compare the contents of the second data copy and the third data copy, and when it is determined that the contents of the second data copy are consistent with the third data copy, update the copy status of the second data copy to a complete status; when it is determined that the contents of the second data copy are inconsistent with the third data copy, update the copy status of the second data copy to an alternative status, wherein the first storage node represents a storage node that has been offline after sending the fault, the second data copy represents a copy in the first storage node, and the third data copy represents a copy with any copy status of a complete status; And / or, compare the contents of the fourth data copy with the fifth data copy, and when it is determined that the contents of the fourth data copy are consistent with the fifth data copy, update the copy status of the fourth data copy to a complete status; when it is determined that the contents of the fourth data copy are inconsistent with the fifth data copy, synchronize the fourth data copy according to the contents of the fifth data copy; after determining that the data synchronization is completed, update the copy status of the fourth data copy to a complete status, wherein the fourth data copy represents a copy with any one copy status as an alternative status, and the fifth data copy represents a copy with any one copy status as a complete status.

7. The method according to claim 6, characterized in that In the case where it is determined that the storage node where the first data copy is located fails, after updating the state of the first data copy to an obsolete state, the method further includes: Select any one storage node from the second storage nodes to build a new data copy of the stored data, wherein the second storage node represents a storage node among the multiple storage nodes that does not store a data copy of the target storage data; During the construction of the new data copy, setting the copy state of the new data copy to a candidate state; When it is determined that the new data copy has been built, the copy state of the new data copy is set to a complete state.

8. A data copy processing device, characterized in that: The device comprises: A parsing module, used to parse the received data read / write request to obtain the data identifier of the target storage data requested by the data read / write request; A selection module, configured to select a target data copy from the multiple data copies according to a request type of the data read / write request and copy states of the multiple data copies corresponding to the data identifier; A processing module is used to process the target data copy according to the data read and write request.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Images