An erasure band writing method, system, electronic device and readable storage medium

Abstract

The application discloses a kind of erasure band write method, system, electronic equipment and readable storage medium, it is related to storage field, it is applied to the main node in the distributed storage system of numerical control separation architecture, the method comprises the following steps: obtaining the single side message structured according to the data to be written by client;Metadata is distributed to target node, so that target node reads aligned data directly from client according to metadata and falls disk;After the alignment processing of non-aligned data, erasure calculation is carried out and segment data is generated;Fragment data is forwarded to target node, so that target node reads segment data and falls disk.The application splits the data to be written into aligned data and non-aligned data in client, aligned data is quickly read to target node, and the data amount of the remaining non-aligned data forwarded from the main node is significantly reduced, so that the time consumption of data processing is small, the data bandwidth amount occupied by forwarding is reduced, and the write processing efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of storage, in particular to a parity stripe writing method and system, electronic equipment and readable storage medium. BACKGROUND

[0002] At present, there is a number separation architecture in a distributed storage system. By separating data and control information, the number separation architecture realizes the purpose of quickly pulling block data of a client to a local disk through one-way operation of RDMA (Remote Direct Memory Access) and then writing the data to the disk, so that the cluster can maximize the provision of business bandwidth to the outside.

[0003] In the data separation architecture, there is a common large non-aligned writing scenario. For the conventional processing of the parity non-aligned scenario, the large non-aligned data is directly read and then written on the master node, and then the master node transmits the data to other slave nodes after performing parity calculation on the large non-aligned data. This method takes a long time for data transmission, occupies a large amount of data bandwidth, and has low writing processing efficiency.

[0004] Therefore, how to provide a solution to the above technical problems is a problem that needs to be solved by those skilled in the art at present. SUMMARY

[0005] Therefore, the purpose of the present application is to provide an efficient and fast parity stripe writing method, system, electronic equipment and readable storage medium. The specific scheme is as follows:

[0006] A parity stripe writing method applied to a master node of a distributed storage system with a number separation architecture, wherein the master node is any node in the distributed storage system, and the method comprises the following steps:

[0007] obtaining a one-way message constructed by a client according to to-be-written data; the to-be-written data comprises aligned data and non-aligned data, and the one-way message comprises metadata obtained by performing parity calculation on the aligned data and the non-aligned data;

[0008] distributing the metadata to a target node, so that the target node reads the aligned data from the client directly according to the metadata and writes the data to a disk; the target node is the node corresponding to the one-way message in the distributed storage system;

[0009] performing parity calculation on the non-aligned data after performing alignment processing on the non-aligned data and generating segment data;

[0010] transmitting the segment data to the target node, so that the target node reads the segment data and writes the data to the disk.

[0011] Preferably, the process of performing alignment processing on the misaligned data, performing erasure coding calculation and generating fragment data, comprises:

[0012] determining a write processing mode of the misaligned data;

[0013] if the write processing mode is a modification mode, reading the minimum erasure coding stripe data in the target node first, performing erasure coding calculation on the minimum erasure coding stripe data and the misaligned data after merging, and generating fragment data;

[0014] if the write processing mode is a write mode, directly performing alignment processing on the misaligned data, performing erasure coding calculation and generating fragment data.

[0015] Preferably, the process of determining the write processing mode of the misaligned data comprises:

[0016] judging whether there is a storage object corresponding to the misaligned data in the target node;

[0017] if yes, determining that the write processing mode of the misaligned data is a modification mode;

[0018] if no, determining that the write processing mode is a write mode.

[0019] Preferably, the process of distributing the metadata to the target node so that the target node reads the aligned data directly from the client according to the metadata and falls the disk comprises:

[0020] constructing a single-side request of the metadata and distributing it to the target node so that the target node reads the aligned data directly from the client according to the metadata and falls the disk.

[0021] Preferably, the metadata comprises original data memory addresses of the aligned data and erasure coding data memory addresses after erasure coding calculation of the aligned data.

[0022] Preferably, the erasure coding stripe write method further comprises:

[0023] when receiving an acknowledgement message that the target node has read the aligned data and the fragment data and fallen the disk, returning a write completion message to the client.

[0024] Correspondingly, the application also discloses an erasure coding stripe write method applied to a target node of a distributed storage system in a number control separation architecture, comprising:

[0025] receiving metadata of aligned data sent by a master node, reading the aligned data directly from a client according to the metadata and falling the disk;

[0026] Receiving the segment data sent by the master node, reading the segment data and writing to disk.

[0027] Correspondingly, the application further discloses a parity stripe writing system applied to a master node of a distributed storage system in a numerical control separation architecture.

[0028] The acquisition module is configured to acquire a unilaterally message constructed by the client according to to-be-written data; the to-be-written data includes aligned data and unaligned data, and the unilaterally message includes metadata obtained by performing parity calculation on the aligned data and the unaligned data;

[0029] The first sending unit is configured to distribute the metadata to a target node, so that the target node reads the aligned data directly from the client and writes to disk according to the metadata; the target node is the node corresponding to the unilaterally message in the distributed storage system;

[0030] The parity calculation unit is configured to perform parity calculation on the unaligned data after alignment processing and generate segment data.

[0031] The second sending unit is configured to forward the segment data to the target node, so that the target node reads the segment data and writes to disk.

[0032] Correspondingly, the application further discloses an electronic device, which comprises:

[0033] The memory is configured to store a computer program.

[0034] The processor is configured to implement the steps of the parity stripe writing method according to any one of the above.

[0035] Correspondingly, the application further discloses a readable storage medium, wherein the readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the parity stripe writing method according to any one of the above.

[0036] The application discloses a parity stripe writing method applied to a master node of a distributed storage system in a numerical control separation architecture, the master node being any node in the distributed storage system, and the method comprises the following steps: acquiring a single-side message constructed by a client according to to-be-written data; the to-be-written data comprises aligned data and misaligned data, and the single-side message comprises metadata obtained by performing parity calculation on the aligned data and the misaligned data; distributing the metadata to a target node, so that the target node reads the aligned data from the client directly according to the metadata and writes the aligned data to a disk; the target node is the node corresponding to the single-side message in the distributed storage system; performing alignment processing on the misaligned data, performing parity calculation on the misaligned data after the alignment processing, and generating fragment data; and forwarding the fragment data to the target node, so that the target node reads the fragment data and writes the fragment data to the disk. In the application, the to-be-written data is split into the aligned data and the misaligned data at the client, the aligned data is directly read from the client to the target node, and the misaligned data is forwarded to the target node after being processed at the master node. Since the aligned data in the to-be-written data is quickly read to the target node, the data amount of the misaligned data forwarded from the master node is obviously reduced, so that the data processing time is short, the data bandwidth occupied by forwarding is reduced, and the writing processing efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0038] Figure 1 A step flow chart of a parity stripe writing method in an embodiment of the present application;

[0039] Figure 2 A flow chart of a parity stripe writing method in an embodiment of the present application;

[0040] Figure 3 A structure distribution diagram of a parity stripe writing system in an embodiment of the present application;

[0041] Figure 4 A structure distribution diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0042] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0043] In the data separation architecture, there is a common large non-aligned write scenario. For the conventional processing of the non-aligned scenario, the large non-aligned data is directly read and then written in the master node, and then the large non-aligned data is calculated by the master node, and the data is forwarded from the master node to other slave nodes. This method takes a long time for data transmission, occupies a large amount of data bandwidth, and has low write processing efficiency.

[0044] In the present application, the to-be-written data is split into aligned data and non-aligned data in the client, the aligned data is directly read from the client to the target node, and the non-aligned data is forwarded to the target node after being processed in the master node. Since the aligned data in the to-be-written data is quickly read to the target node, the amount of non-aligned data forwarded from the master node is significantly reduced, so that the data processing time is short, the data bandwidth occupied by the forwarding is reduced, and the write processing efficiency is improved.

[0045] The embodiment of the present application discloses a method for writing an erasure stripe, which is applied to a master node in a distributed storage system of a number control separation architecture. The master node is any node in the distributed storage system, and the distributed storage system includes a plurality of nodes. Any node is a master node, and the remaining nodes are managed by the master node. The number control separation architecture relies on the single-sided operation of RDMA.

[0046] Referring to Figure 1 The method for writing an erasure stripe includes the following steps:

[0047] S1: obtaining a single-sided message constructed by a client according to to-be-written data;

[0048] The to-be-written data includes aligned data and non-aligned data, and the single-sided message includes metadata obtained by performing erasure correction calculation on the aligned data and the non-aligned data.

[0049] It can be understood that the unilateral message here is a message constructed by the client according to the to-be-written data. After analyzing the to-be-written data, the client divides the to-be-written data into aligned data and non-aligned data. It can be considered that the aligned data is the data part that has been aligned in the client, and the non-aligned data is the data part that needs to be processed again in the master node. After the client divides the to-be-written data into aligned data and non-aligned data, the aligned data is subjected to erasure calculation to generate erasure data, and the address information of the erasure data and the address information of the original aligned data are jointly used as metadata. The metadata and the non-aligned data are the main part of the unilateral message. The metadata can also include keys, other information of the to-be-written data, and the like.

[0050] It can be understood that the master node receives the unilateral message through the transmission path of the control information in the original numerical control separation architecture, and then separates the components of the unilateral message to determine the metadata corresponding to the aligned data and the non-aligned data from the unilateral message.

[0051] S2: Distribute the metadata to the target node, so that the target node reads the aligned data directly from the client according to the metadata and writes it to the disk;

[0052] The target node is a node corresponding to the unilateral message in the distributed storage system.

[0053] It can be understood that the aligned data in the erasure pool IO no longer needs to be processed and can be directly pulled from the client by the target node. Specifically, step S2 includes:

[0054] Construct a unilateral request for the metadata and distribute it to the target node, so that the target node reads the aligned data directly from the client according to the metadata and writes it to the disk.

[0055] Further, the metadata includes the original data memory address of the aligned data and the erasure data memory address after the erasure calculation of the aligned data.

[0056] It can be understood that the unilateral request for the metadata is actually converted into a format corresponding to the unilateral request, so that the target node can recognize and execute the unilateral request.

[0057] S3: Perform alignment processing on the non-aligned data, perform erasure calculation, and generate fragment data;

[0058] The main means of aligning the non-aligned data is zero padding alignment.

[0059] S4: Forward the fragment data to the target node, so that the target node reads the fragment data and writes it to the disk.

[0060] It can be understood that, unlike the aligned data is directly pulled by the client to the target node, the processing and forwarding of the misaligned data is performed through the master node, and the different landing ways of the misaligned data and the aligned data do not interfere with each other, that is, the execution of step S2 does not interfere with the execution of steps S3-S4, as long as the landing of all to-be-written data can be finally realized.

[0061] Specifically, step S3 performs alignment processing on the misaligned data, and then performs erasure calculation and generates segment data, which has different processing according to different write processing modes. Specifically, step S3 includes:

[0062] determining the write processing mode of the misaligned data;

[0063] If the write processing mode is the modification mode, the minimum erasure strip data in the target node is read first, the minimum erasure strip data is combined with the misaligned data, and then erasure calculation is performed to generate segment data;

[0064] If the write processing mode is the write mode, the misaligned data is directly aligned and then erasure calculation is performed to generate segment data.

[0065] Further, the process of determining the write processing mode of the misaligned data includes:

[0066] determining whether the storage object corresponding to the misaligned data exists in the target node;

[0067] If yes, it is determined that the write processing mode of the misaligned data is the modification mode;

[0068] If no, it is determined that the write processing mode is the write mode.

[0069] It can be understood that when the target node already has the storage object corresponding to the misaligned data, the storage object must have stored data, which is the minimum erasure strip data that has been erasure calculated. The current new misaligned data is to be written into the storage object again, and the write processing mode at this time is the modification mode. The minimum erasure strip data is directly combined with the current misaligned data. Since the minimum erasure strip data is already aligned data, the combined misaligned data does not need to be aligned, and the combined erasure calculation is directly performed to generate segment data;

[0070] When there is no storage object corresponding to the misaligned data in the target node, there is also no minimum erasure strip data that needs to be combined, and the segment data is directly written, and the write processing mode at this time is the write mode. The misaligned data is aligned, and then erasure calculation is performed to generate segment data.

[0071] Further, the erasure strip writing method can further include:

[0072] S5: When receiving the confirmation message that the target node has read the alignment data and the segment data and landed the disk, return the write completion message to the client.

[0073] It can be understood that before the target node lands the disk on the alignment data and the segment data, a step of merging the alignment data and the segment data can be further included.

[0074] It can be understood that after the target node completes the landing action on the alignment data and the segment data, it can be considered that all the to-be-written data has been landed, and therefore the target node sends a confirmation message to the master node, and the master node returns a write completion message to the client after receiving the confirmation message, to notify the client that the to-be-written data completes the write operation.

[0075] Further, if there is unlanded data in the alignment data and the segment data, the confirmation message will not be sent, thereby ensuring the reliable judgment of the client on whether the to-be-written data completes the write operation.

[0076] Specifically, taking the target node including the master node, the slave node 1 and the slave node 2 as an example, a flowchart of the erasure coding stripe writing method is shown in Figure 2 Further, when the distributed storage system is of other structures or the target node is of other nodes, the flow of executing the erasure coding stripe writing method is similar to Figure 2 .

[0077] Embodiments of the present application disclose an erasure coding stripe writing method applied to a master node in a distributed storage system of a number control separation architecture, the master node being any node in the distributed storage system, comprising: acquiring a unilaterally message constructed by a client according to to-be-written data; the to-be-written data comprising alignment data and non-alignment data, the unilaterally message comprising: metadata obtained by performing erasure coding calculation on the alignment data and the non-alignment data; distributing the metadata to a target node, so that the target node reads the alignment data directly from the client and lands the disk according to the metadata; the target node being the node corresponding to the unilaterally message in the distributed storage system; performing alignment processing on the non-alignment data, performing erasure coding calculation and generating segment data; and forwarding the segment data to the target node, so that the target node reads the segment data and lands the disk. Embodiments of the present application split the to-be-written data into alignment data and non-alignment data at the client, the alignment data is directly read from the client to the target node, and the non-alignment data is forwarded to the target node after processing at the master node. Since the alignment data in the to-be-written data is quickly read to the target node, the data amount of the remaining non-alignment data forwarded from the master node is significantly reduced, thereby reducing the data processing time and the data bandwidth occupied by forwarding, and improving the write processing efficiency.

[0078] Correspondingly, embodiments of the present application also disclose an erasure coding stripe writing method applied to a target node in a distributed storage system of a number control separation architecture, comprising:

[0079] receiving metadata of the alignment data sent by the master node, reading the alignment data directly from the client according to the metadata and writing to disk;

[0080] receiving the segment data sent by the master node, reading the segment data and writing to disk.

[0081] Further, after reading the alignment data and the segment data, the alignment data and the segment data can be combined and then written to disk.

[0082] Further, after the alignment data and the segment data are both written to disk, an acknowledgement message can be sent to the master node, so that the master node returns a write completion message to the client.

[0083] It can be understood that in the embodiment, the target node can write the alignment data and the segment data to disk respectively, combine the alignment data and the segment data, and send the acknowledgement message after the alignment data and the segment data are both written to disk, all of which are for the client and provide more accurate and reliable judgment for the client that the write operation is completed.

[0084] In the embodiment, the client splits the to-be-written data into alignment data and non-alignment data, reads the alignment data directly from the client to the target node, and forwards the non-alignment data to the target node after processing by the master node. Since the alignment data in the to-be-written data is quickly read to the target node, the amount of data of the non-alignment data forwarded from the master node is significantly reduced, so that the data processing time is short, the data bandwidth occupied by forwarding is reduced, and the write processing efficiency is improved.

[0085] Correspondingly, the embodiment of the application also discloses a parity stripe writing system applied to a master node of a distributed storage system in a number control separation architecture, the master node being any node in the distributed storage system and participating in Figure 3 As shown in the figure, the parity stripe writing system comprises:

[0086] The acquisition module 1 is configured to acquire a unilateral message constructed by a client according to to-be-written data; the to-be-written data comprises alignment data and non-alignment data, and the unilateral message comprises metadata obtained by performing parity calculation on the alignment data and the non-alignment data;

[0087] The first sending unit 2 is configured to distribute the metadata to a target node, so that the target node reads the alignment data directly from the client according to the metadata and writes to disk; the target node is the node corresponding to the unilateral message in the distributed storage system;

[0088] The parity calculation unit 3 is configured to perform parity calculation on the non-alignment data after alignment processing and generate segment data;

[0089] The second sending unit 4 is configured to forward the segment data to the target node, so that the target node reads the segment data and stores the segment data.

[0090] It can be understood that the unilateral message here is a message constructed by the client according to the to-be-written data. After analyzing the to-be-written data, the client divides the to-be-written data into aligned data and unaligned data. It can be considered that the aligned data is a data part that has been aligned in the client, and the unaligned data is a data part that needs to be processed again in the master node. After dividing the to-be-written data into aligned data and unaligned data, the client generates erasure coding data by performing erasure coding calculation on the aligned data. The address information of the erasure coding data and the address information of the original aligned data are used as metadata together. The metadata and the unaligned data are the main part of the unilateral message. The metadata can also include keys, other information of the to-be-written data, and the like.

[0091] It can be understood that the master node receives the unilateral message through the transmission path of the control information in the original numerical control separation architecture, and then separates the unilateral message into components to determine the metadata corresponding to the aligned data and the unaligned data from the unilateral message.

[0092] It can be understood that, unlike the aligned data being directly pulled to the target node by the client, the processing and forwarding of the unaligned data are performed by the master node. The different storage paths of the unaligned data and the aligned data do not interfere with each other, that is, the execution of the first sending unit 2 does not interfere with the execution of the erasure coding calculation unit 3 and the second sending unit 4, as long as the final storage of all to-be-written data can be realized.

[0093] In the client, the to-be-written data is split into aligned data and unaligned data. The aligned data is directly read from the client to the target node, and the unaligned data is forwarded to the target node after being processed in the master node. Since the aligned data in the to-be-written data is quickly read to the target node, the amount of data of the remaining unaligned data forwarded from the master node is significantly reduced, so that the data processing time is short, the data bandwidth occupied by forwarding is reduced, and the write processing efficiency is improved.

[0094] In some specific embodiments, the process of performing alignment processing on the unaligned data, performing erasure coding calculation on the unaligned data after the alignment processing, and generating segment data includes:

[0095] determining a write processing mode of the unaligned data;

[0096] If the write processing mode is a modification mode, the minimum erasure coding stripe data in the target node is read first, and the minimum erasure coding stripe data and the unaligned data are combined to perform erasure coding calculation and generate segment data;

[0097] If the write processing mode is a write-in mode, the non-aligned data is directly aligned, and then erasure calculation is performed to generate fragment data.

[0098] In some specific embodiments, the process of determining the write processing mode of the non-aligned data comprises:

[0099] determining whether the storage object corresponding to the non-aligned data exists in the target node;

[0100] If yes, it is determined that the write processing mode of the non-aligned data is a modification mode.

[0101] If no, it is determined that the write processing mode is a write-in mode.

[0102] It can be understood that when the target node already has the storage object corresponding to the non-aligned data, the storage object must have stored data, which is the minimum erasure stripe data that has been subjected to erasure calculation. The current new non-aligned data is to be written into the storage object again, and at this time, the write processing mode is a modification mode. The minimum erasure stripe data and the current non-aligned data are directly merged. Since the minimum erasure stripe data is already aligned data, the merged minimum erasure stripe data and non-aligned data do not need to be aligned, and the merged data is directly subjected to erasure calculation to generate fragment data.

[0103] When the target node does not have the storage object corresponding to the non-aligned data, there is also no minimum erasure stripe data that needs to be merged. The fragment data is directly written in, and at this time, the write processing mode is a write-in mode. The non-aligned data is aligned, and then erasure calculation is performed to generate fragment data.

[0104] In some specific embodiments, the process of distributing the metadata to the target node so that the target node reads the aligned data directly from the client according to the metadata and writes it to disk comprises:

[0105] Constructing a single-side request of the metadata and distributing it to the target node so that the target node reads the aligned data directly from the client according to the metadata and writes it to disk.

[0106] In some specific embodiments, the metadata comprises an original data memory address of the aligned data and an erasure data memory address of the aligned data after erasure calculation.

[0107] In some specific embodiments, the erasure stripe writing system further comprises:

[0108] The feedback module 5 is configured to return a write completion message to the client when receiving an acknowledgement message that the target node has read the aligned data and the fragment data and written them to disk.

[0109] It can be understood that after the target node completes the landing operation of the alignment data and the segment data, it can be considered that all the to-be-written data has been landed, and thus the target node sends a confirmation message to the master node, and the master node returns a write completion message to the client after receiving the confirmation message, to notify the client that the to-be-written data completes the write operation.

[0110] Further, if there is unlanded data in the alignment data and the segment data, the confirmation message will not be sent, thereby ensuring the reliable judgment of the client on whether the to-be-written data completes the write operation.

[0111] Correspondingly, the embodiment of the application further discloses an electronic device, referring to Figure 4 as shown, comprising a processor 11 and a memory 12; wherein the processor 11 implements the following steps when executing the computer program saved in the memory 12:

[0112] obtain a unilateral message constructed by the client according to to-be-written data; the to-be-written data includes alignment data and non-alignment data, and the unilateral message includes metadata obtained by performing erasure correction calculation on the alignment data and the non-alignment data;

[0113] distribute the metadata to a target node, so that the target node reads the alignment data directly from the client and lands according to the metadata; the target node is the node corresponding to the unilateral message in the distributed storage system;

[0114] perform alignment processing on the non-alignment data, perform erasure correction calculation, and generate segment data;

[0115] forward the segment data to the target node, so that the target node reads the segment data and lands.

[0116] The embodiment of the application splits the to-be-written data into alignment data and non-alignment data at the client, the alignment data is directly read from the client to the target node, and the non-alignment data is forwarded to the target node after being processed at the master node. Since the alignment data in the to-be-written data is quickly read to the target node, the data amount of the remaining non-alignment data forwarded from the master node is significantly reduced, thereby reducing the data processing time and the data bandwidth occupied by forwarding, and improving the write processing efficiency.

[0117] In some specific embodiments, the processor 11 can specifically implement the following steps when executing the computer sub-program saved in the memory 12:

[0118] determine the write processing mode of the non-alignment data;

[0119] If the write processing mode is the modification mode, the minimum erasure stripe data in the target node is read first, the minimum erasure stripe data is combined with the misaligned data, erasure calculation is performed on the combined data, and segment data is generated;

[0120] If the write processing mode is the write mode, the misaligned data is directly aligned, erasure calculation is performed on the aligned data, and segment data is generated.

[0121] In some specific embodiments, when the processor 11 executes the computer program stored in the memory 12, the following steps can be specifically implemented:

[0122] It is determined whether the storage object corresponding to the misaligned data exists in the target node;

[0123] If yes, it is determined that the write processing mode of the misaligned data is the modification mode;

[0124] If no, it is determined that the write processing mode is the write mode.

[0125] In some specific embodiments, when the processor 11 executes the computer program stored in the memory 12, the following steps can be specifically implemented:

[0126] The single-side request of the metadata is constructed and distributed to the target node, so that the target node reads the aligned data directly from the client according to the metadata and writes the data to the disk.

[0127] In some specific embodiments, the metadata includes the original data memory address of the aligned data and the erasure data memory address after erasure calculation of the aligned data.

[0128] In some specific embodiments, when the processor 11 executes the computer program stored in the memory 12, the following steps can be specifically implemented:

[0129] When the confirmation message that the target node has read the aligned data and the segment data and written the data to the disk is received, a write completion message is returned to the client.

[0130] Further, the electronic device in the embodiment can further include:

[0131] The input interface 13 is configured to acquire the computer program imported from the outside and save the acquired computer program to the memory 12, and can be configured to acquire various instructions and parameters transmitted by the terminal device from the outside and transmit the instructions and parameters to the processor 11, so that the processor 11 performs corresponding processing by using the instructions and parameters. In the embodiment, the input interface 13 can include, but is not limited to, a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, and the like.

[0132] An output interface 14 is configured to output various data generated by the processor 11 to a terminal device connected thereto, so that other terminal devices connected to the output interface 14 can obtain the various data generated by the processor 11. In this embodiment, the output interface 14 can include, but is not limited to, a USB interface, a serial interface, and the like.

[0133] A communication unit 15 is configured to establish a remote communication connection between the electronic device and an external server, so that the electronic device can mount the mirror file into the external server. In this embodiment, the communication unit 15 can include, but is not limited to, a remote communication unit based on wireless communication technology or wired communication technology.

[0134] A keyboard 16 is configured to obtain various parameter data or instructions input by a user by real-time tapping of keycaps.

[0135] A display 17 is configured to display relevant information of the erasure correction strip writing process in real time, so that the user can timely understand the writing situation of the current erasure correction strip.

[0136] A mouse 18 can be used to assist the user in inputting data and simplifying the operation of the user.

[0137] Further, the embodiment of the present application further discloses a readable storage medium, which includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable hard disk, a CD-ROM, or any other form of storage medium known in the technical field. The readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0138] obtaining a unilateral message constructed by a client according to to-be-written data; the to-be-written data includes aligned data and unaligned data, and the unilateral message includes metadata obtained by performing erasure correction calculation on the aligned data and the unaligned data;

[0139] distributing the metadata to a target node, so that the target node reads the aligned data from the client directly according to the metadata and writes the aligned data to a disk; the target node is a node corresponding to the unilateral message in the distributed storage system;

[0140] performing alignment processing on the unaligned data, performing erasure correction calculation on the unaligned data after the alignment processing, and generating segment data;

[0141] forwarding the segment data to the target node, so that the target node reads the segment data and writes the segment data to the disk.

[0142] The embodiment of the application splits the to-be-written data into aligned data and non-aligned data at the client, the aligned data is directly read from the client to the target node, and the non-aligned data is forwarded to the target node after being processed at the master node. Since the aligned data in the to-be-written data is quickly read to the target node, the amount of non-aligned data forwarded from the master node is significantly reduced, so that the data processing time is short, the data bandwidth occupied by forwarding is reduced, and the write processing efficiency is improved.

[0143] In some specific embodiments, the computer program stored in the readable storage medium can specifically implement the following steps when executed by the processor:

[0144] determining the write processing mode of the non-aligned data;

[0145] if the write processing mode is the modification mode, the minimum erasure stripe data in the target node is read first, the minimum erasure stripe data and the non-aligned data are merged to perform erasure calculation and generate segment data;

[0146] if the write processing mode is the write-in mode, the non-aligned data is directly aligned and processed to perform erasure calculation and generate segment data.

[0147] In some specific embodiments, the computer program stored in the readable storage medium can specifically implement the following steps when executed by the processor:

[0148] determining whether the storage object corresponding to the non-aligned data exists in the target node;

[0149] if yes, it is determined that the write processing mode of the non-aligned data is the modification mode;

[0150] if no, it is determined that the write processing mode is the write-in mode.

[0151] In some specific embodiments, the computer program stored in the readable storage medium can specifically implement the following steps when executed by the processor:

[0152] constructing a single-side request of the metadata and distributing it to the target node, so that the target node directly reads the aligned data from the client according to the metadata and falls the disk.

[0153] In some specific embodiments, the metadata includes the original data memory address of the aligned data and the erasure data memory address after erasure calculation of the aligned data.

[0154] In some specific embodiments, the computer program stored in the readable storage medium can specifically implement the following steps when executed by the processor:

[0155] When receiving the confirmation message that the target node has read the alignment data and the segment data and written to disk, a write completion message is returned to the client.

[0156] Finally, it is also necessary to point out that, in this document, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0157] The above describes in detail the erasure band writing method, system, electronic device and readable storage medium provided by the present application. The principles and implementation manners of the present application are described by applying specific examples in this document. The above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method of erasure-banded writing, the method comprising: A master node applied to a distributed storage system of a numerical control separation architecture, the master node being any node in the distributed storage system, comprising: obtaining a unilaterally message constructed by a client according to to-be-written data, the to-be-written data including aligned data and unaligned data, the unilaterally message including metadata and the unaligned data, the metadata including address information of erasure data generated after erasure calculation on the aligned data and address information of original aligned data; distributing the metadata to a target node, so that the target node reads the aligned data directly from the client and writes to disk according to the metadata, the target node being the node corresponding to the unilaterally message in the distributed storage system; performing erasure calculation on the unaligned data after alignment processing and generating segment data; forwarding the segment data to the target node, so that the target node reads the segment data and writes to disk.

2. The method of claim 1, wherein, The process of performing erasure calculation on the unaligned data after alignment processing and generating segment data, comprising: determining a write processing mode of the unaligned data; if the write processing mode is a modification mode, reading minimum erasure strip data in the target node first, performing erasure calculation on the minimum erasure strip data and the unaligned data after merging, and generating segment data; if the write processing mode is a write mode, directly performing erasure calculation on the unaligned data after alignment processing and generating segment data.

3. The method of claim 2, wherein, The process of determining the write processing mode of the unaligned data, comprising: judging whether there is a storage object corresponding to the unaligned data in the target node; if yes, determining that the write processing mode of the unaligned data is a modification mode; if no, determining that the write processing mode is a write mode.

4. The method of claim 1, wherein, The process of distributing the metadata to a target node, so that the target node reads the aligned data directly from the client and writes to disk according to the metadata, comprising: constructing a unilaterally request of the metadata and distributing it to a target node, so that the target node reads the aligned data directly from the client and writes to disk according to the metadata.

5. The method of claim 1, wherein, The metadata includes original data memory address of the aligned data and erasure data memory address after erasure calculation on the aligned data.

6. The method of claim 1 to 5, wherein, Further comprising: when receiving an acknowledgement message that the target node has read the aligned data and the segment data and written to disk, returning a write completion message to the client.

7. A method of erasure-banded writing, the method comprising: A target node applied to a distributed storage system of a numerical control separation architecture, comprising: receiving metadata of aligned data sent by a master node, and reading the aligned data directly from a client and writing to disk according to the metadata, the metadata including address information of erasure data generated after erasure calculation on the aligned data and address information of original aligned data; receiving segment data sent by the master node, and reading the segment data and writing to disk, the segment data being data generated after erasure calculation on unaligned data after alignment processing.

8. A striped write system for erasure correction, characterized by, A master node applied to a distributed storage system of a numerical control separation architecture, the master node being any node in the distributed storage system, comprising: An obtaining module is configured to obtain a unilaterally written message constructed by a client according to to-be-written data; the to-be-written data includes aligned data and non-aligned data, and the unilaterally written message includes metadata and the non-aligned data, and the metadata includes address information of erasure data generated by performing erasure calculation on the aligned data and address information of original aligned data; A first sending unit is configured to distribute the metadata to a target node, so that the target node reads the aligned data directly from the client and writes the aligned data to a disk according to the metadata; the target node is the node corresponding to the unilaterally written message in the distributed storage system; An erasure calculation unit is configured to perform erasure calculation on the non-aligned data after performing alignment processing on the non-aligned data and generate segment data; A second sending unit is configured to forward the segment data to the target node, so that the target node reads the segment data and writes the segment data to a disk.

9. An electronic device, comprising: The computer program is stored on the readable storage medium and is executed by the processor to implement the steps of the erasure stripe writing method according to any one of claims 1 to 6 or the erasure stripe writing method according to claim 7. The computer program is stored on the readable storage medium and is executed by the processor to implement the steps of the erasure stripe writing method according to any one of claims 1 to 6 or the erasure stripe writing method according to claim 7. ​ 10. A readable storage medium, characterized by, ​

Images