Data writing method, cache information updating method and related device

Abstract

This invention provides a data writing method, a cache information updating method, and related apparatus, relating to the field of distributed storage. The data writing method involves: splitting the data to be stored into multiple data fragments according to a preset erasure ratio, and determining a storage block corresponding to each data fragment; the storage block being a data block used to store the data fragment; determining the target storage node where each storage block is located at the current time based on multiple cached information entries locally, wherein the multiple cached information entries are updated based on the recovery progress of the recovered data blocks reported by multiple storage nodes, and each recovered data block reported by each storage node meets preset reporting conditions, which are periodically sent by the client to multiple storage nodes; and sending each data fragment to the target storage node where the corresponding storage block is located for data writing, thereby avoiding the data block recovery process affecting the data fragment writing process and reducing write latency.

Description

Technical Field

[0001] This invention relates to the field of distributed storage, and more specifically, to a data writing method, a cache information updating method, and related apparatus. Background Technology

[0002] In a distributed storage system, a metadata server locates stored information, multiple data servers act as storage nodes, and clients provide access interfaces for data read / write operations. For data write operations, the client divides the data to be stored into N+M data fragments according to the erasure ratio, and then writes each data fragment into the data blocks of different storage nodes. When the number of successfully written data fragments is greater than N, the entire data to be stored is considered to have been successfully written. For data fragments that failed to be written, the storage nodes recover them through erasure calculations.

[0003] The collaborative operation of multiple devices presents significant challenges to data consistency. For example, while a storage node is recovering a data block, a client may be writing data fragments to that same block. Currently, the main approach to handling simultaneous data writing and recovery is for the client to suspend the write operation on the current data fragment, waiting for the data block recovery to complete before executing the write operation, which greatly increases write latency. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, embodiments of the present invention provide a data writing method, a cache information updating method, and related apparatus, which enable the data block recovery process to not affect the data fragment writing process and reduce writing latency.

[0005] The technical solution of this invention can be implemented as follows:

[0006] In a first aspect, embodiments of the present invention provide a data writing method applied to a client in a distributed storage system, the distributed storage system further comprising multiple storage nodes, the client being communicatively connected to the multiple storage nodes, and at least one data block being created on each of the storage nodes, the method comprising:

[0007] The data to be stored is split into multiple data fragments according to a preset erasure ratio, and a storage block corresponding to each data fragment is determined. The storage block is a data block used to store the data fragment.

[0008] Based on multiple cached information entries in the local cache, determine the target storage node where each of the storage blocks is located at the current moment;

[0009] The plurality of pieces of cache information are updated based on recovery progress of the recovered data blocks reported by the plurality of storage nodes, and each of the recovered data blocks reported by the storage nodes satisfies a preset reporting condition which is sent by the client to the plurality of storage nodes at a regular time.

[0010] Each of the data shards is sent to a target storage node where the storage block corresponding to the data shard is located, so as to perform data writing.

[0011] Optionally, the plurality of pieces of cache information include a plurality of first pieces of information cached in a local cache area, and each of the first pieces of information records an identifier of a data block, a state of the data block, and a storage node where the data block is located. The distributed storage system further includes a management node in communication connection with the client.

[0012] The step of determining, according to the plurality of pieces of cache information cached locally, a target storage node where each of the storage blocks is located at a current time includes:

[0013] For each of the storage blocks, if there is target first information recording the identifier of the storage block in the plurality of first pieces of information, and the state of the data block recorded in the target first information is a recovery state or a normal state, the storage node where the data block recorded in the target first information is located is taken as the target storage node where the storage block is located at the current time.

[0014] If there is target first information recording the identifier of the storage block in the plurality of first pieces of information, and the state of the data block recorded in the target first information is an abnormal state, a query request is sent to the management node to obtain the target storage node where the storage block is located at the current time.

[0015] Optionally, the plurality of pieces of cache information further include a plurality of second pieces of information cached in a local cache linked list, and each of the second pieces of information includes an identifier of a data block and a storage node where the data block is located. The plurality of second pieces of information are updated based on recovery progress of the recovered data blocks reported by the plurality of storage nodes, and the plurality of first pieces of information are updated based on update of the plurality of second pieces of information.

[0016] The step of determining, according to the plurality of pieces of cache information cached locally, a target storage node where each of the storage blocks is located at a current time further includes:

[0017] If there is no target first information recording the identifier of the storage block in the plurality of first pieces of information, it is determined whether there is target second information recording the identifier of the storage block in the plurality of second pieces of information.

[0018] If the target second information exists, a storage node where the data block recording the target second information is located is taken as a target storage node where the storage block is located at the current time;

[0019] If the target second information does not exist, a query request is sent to the management node to obtain a target storage node where the storage block is located at the current time.

[0020] Optionally, the method further comprises:

[0021] The response message returned by each target storage node is received, and the response message represents whether the target storage node successfully writes the data shard into the storage block corresponding to the data shard;

[0022] For each response message representing successful writing, if a state of the storage block writing the data shard on the target storage node returning the response message is a recovery state, the response message is not included in erasure count;

[0023] If the state of the storage block writing the data shard on the target storage node returning the response message is a normal state, the response message is included in erasure count;

[0024] After the judgment of whether each response message representing successful writing is included in erasure count is completed, whether the to-be-stored data is successfully stored is determined according to a value of the erasure count;

[0025] If the value of the erasure count meets a preset condition, it is determined that the to-be-stored data is successfully stored;

[0026] If the value of the erasure count does not meet the preset condition, the step of determining the target storage node where each storage block is located at the current time according to the plurality of pieces of cache information locally cached is returned until the to-be-stored data is successfully stored.

[0027] In a second aspect, an embodiment of the present application provides a cache information updating method applied to a client in a distributed storage system, the distributed storage system further comprising a plurality of storage nodes, at least one data block being created on each storage node, the client being in communication connection with the plurality of storage nodes, and the method comprising:

[0028] A preset reporting condition is sent to the plurality of storage nodes at a time, so that the plurality of storage nodes report recovery progress of a recovered data block meeting the preset reporting condition to the client;

[0029] According to the recovery progress of the recovered data block received each time, a plurality of pieces of cache information locally cached are updated;

[0030] The plurality of pieces of cache information are used to determine a target storage node where a storage block corresponding to each data shard of the data to be stored at a current time is located when the client executes the data writing method according to any one of the preceding embodiments, and the storage block is a data block used to store the data shard.

[0031] Optionally, the plurality of pieces of cache information include a plurality of first information cached in a local cache area and a plurality of second information cached in a local cache linked list, and each piece of the second information includes an identifier of a data block and a storage node where the data block is located.

[0032] The step of updating the plurality of pieces of cache information in the local cache according to a recovery progress of each received recovered data block includes:

[0033] If the recovery progress of each received recovered data block is in recovery, a second information recording an identifier of the recovered data block and a storage node where the recovered data block is located is added to the local cache linked list.

[0034] If the recovery progress of each received recovered data block is recovery success or recovery failure, a second information recording an identifier of the recovered data block in the local cache linked list is deleted.

[0035] The plurality of pieces of first information are updated according to an update condition of the plurality of pieces of second information.

[0036] Optionally, each piece of the first information records an identifier, a state, and a storage node of a data block.

[0037] The step of updating the plurality of pieces of first information according to the update condition of the plurality of pieces of second information includes:

[0038] For the newly added second information in the local cache linked list, if there is reference first information recording an identifier of a data block that is the same as an identifier of a data block recorded by the second information in the local cache area, a storage node where the data block recorded by the reference first information is located is updated to a storage node where the data block recorded by the second information is located, and a state of the data block recorded by the reference first information is set to a recovery state.

[0039] For the newly deleted second information in the local cache linked list, if a recovery progress of a data block recorded by the second information is recovery success and there is reference first information in the local cache area, a state of a data block recorded by the reference first information is set to a normal state.

[0040] If the recovery progress of the data block recorded by the second information is recovery failure, and the reference first information exists in the local cache area, the state of the data block recorded by the reference first information is set as an abnormal state.

[0041] In a third aspect, an embodiment of the present application provides a data writing device, applied to a client in a distributed storage system, the distributed storage system further comprising a plurality of storage nodes, the client being in communication connection with the plurality of storage nodes, at least one data block being created on each of the storage nodes, and the device comprising:

[0042] a splitting module, configured to split to-be-stored data into a plurality of data shards according to a preset erasure coding ratio, and determine a storage block corresponding to each of the data shards, the storage block being a data block for storing the data shard;

[0043] a determining module, configured to determine, according to a plurality of pieces of cache information locally cached, a target storage node where each of the storage blocks is located at a current time;

[0044] wherein the plurality of pieces of cache information are updated based on recovery progress of a recovered data block reported by the plurality of storage nodes, each of the recovered data blocks reported by the plurality of storage nodes satisfying a preset reporting condition, and the preset reporting condition being sent to the plurality of storage nodes by the client at a regular time;

[0045] a first sending module, configured to send each of the data shards to a target storage node where the storage block corresponding to each of the data shards is located, so as to perform data writing.

[0046] In a fourth aspect, an embodiment of the present application provides a cache information updating device, applied to a client in a distributed storage system, the distributed storage system further comprising a plurality of storage nodes, at least one data block being created on each of the storage nodes, and the client being in communication connection with the plurality of storage nodes, and the device comprising:

[0047] a second sending module, configured to send a preset reporting condition to the plurality of storage nodes at a regular time, so as to make the plurality of storage nodes report recovery progress of a recovered data block satisfying the preset reporting condition to the client;

[0048] an updating module, configured to update a plurality of pieces of cache information locally cached according to recovery progress of a recovered data block received each time;

[0049] wherein the plurality of pieces of cache information are used to determine, when the client executes the data writing method according to any one of the preceding embodiments, a target storage node where a storage block corresponding to each of the data shards of the to-be-stored data is located at a current time, the storage block being a data block for storing the data shard.

[0050] In a fifth aspect, an embodiment of the present application provides a client, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the data writing method according to the first aspect, and / or the cache information updating method according to the second aspect.

[0051] In a sixth aspect, an embodiment of the present application provides a distributed storage system, comprising a management node, a plurality of storage nodes, and the client according to the fifth aspect.

[0052] In a seventh aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the data writing method according to the first aspect, and / or the cache information updating method according to the second aspect.

[0053] Compared with the prior art, the data writing method, the cache information updating method and the related apparatus provided by the embodiment of the present application firstly split the to-be-stored data into a plurality of data shards according to a preset erasure coding ratio, and determine a storage block corresponding to each data shard, wherein the storage block is a data block used for storing the data shard; then, according to a plurality of pieces of cache information locally cached, a target storage node where each storage block is located at a current time is determined, wherein the plurality of pieces of cache information are updated based on a recovery progress of a recovered data block reported by a plurality of storage nodes, and each recovered data block reported by each storage node satisfies a preset reporting condition, and the preset reporting condition is sent to the plurality of storage nodes by a client at a regular time; finally, each data shard is sent to the target storage node where the storage block corresponding to the data shard is located, so as to perform data writing. Since the client in the embodiment of the present application determines the storage node where each data block used for storing the data shard is located at the current time through the plurality of pieces of cache information locally cached and updated at a regular time, and then sends the data shard to the storage node where the data block is located at the current time, so as to perform data writing, the data block recovery process does not affect the data shard writing process, and the writing time delay is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0055] Figure 1 A structure diagram of a distributed storage system provided by an embodiment of the present application;

[0056] Figure 2The existing data distributed storage principle schematic diagram provided for the embodiment of the present application is shown in the figure.

[0057] Figure 3 The flowchart of the data writing method provided for the embodiment of the present application is shown in the figure. Figure 1

[0058] Figure 4 The flowchart of the data writing method provided for the embodiment of the present application is shown in the figure. Figure 2

[0059] Figure 5 The flowchart of the data writing method provided for the embodiment of the present application is shown in the figure. Figure 3

[0060] Figure 6 The flowchart of the cache information updating method provided for the embodiment of the present application is shown in the figure. Figure 1

[0061] Figure 7 The flowchart of the cache information updating method provided for the embodiment of the present application is shown in the figure. Figure 2

[0062] Figure 8 The specific example of the data writing and cache information updating process provided for the embodiment of the present application is shown in the figure.

[0063] Figure 9 The functional unit block diagram of the data writing device provided for the embodiment of the present application is shown in the figure.

[0064] Figure 10 The functional unit block diagram of the cache information updating device provided for the embodiment of the present application is shown in the figure.

[0065] Figure 11 The structural schematic block diagram of the client provided for the embodiment of the present application is shown in the figure.

[0066] Icon: 100-data writing device; 101-splitting module; 102-determining module; 103-first sending module; 104-judging module; 200-cache information updating device; 201-second sending module; 202-updating module; 300-client; 310-memory; 320-processor. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. ​​​​​

[0068] The following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the application claimed, but merely represents selected embodiments of the application. Based upon the embodiments of the application, all other embodiments obtained by persons of ordinary skill in the art without creative work are within the scope of the application.

[0069] It should be noted that similar reference numbers and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings.

[0070] In addition, if the terms "first", "second" and the like are used, they are merely used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0071] It should be noted that the features in the embodiments of the application can be combined with each other without conflict.

[0072] Please refer to Figure 1 , the distributed storage system includes a client, a plurality of storage nodes and a management node. The client is in communication connection with the management node, the client is in communication connection with the plurality of storage nodes, the management node is in communication connection with the plurality of storage nodes, and at least one data block for storing data shards is created on each storage node.

[0073] As shown in Figure 2 , the client can interact with an upper-layer application or an external device, can receive data to be stored from the upper-layer application or the external device, split the received data to be stored into a plurality of data shards according to a preset erasure coding ratio, and distribute each data shard to a data block on a different storage node for storage by sending a write data request to the storage node. The client can also send information about the data block that fails to store the data shard to the management node for recording.

[0074] The client can be a server, a personal computer (PC), a notebook computer, etc., the client can also be one or more program modules on a device, or a virtual machine or a container running on a device, and the client can also be a cluster composed of multiple devices, for example, it can be a collective name of multiple program modules distributed on multiple devices.

[0075] The storage node can receive a write data request to store a data shard from the client to a corresponding data block. The storage node can also receive a data block recovery request issued by the management node to recover the data block that fails to store the data shard. The storage node can be a server, a PC, a notebook computer, etc. The storage node can be a physical storage node, or a logical storage node divided from a physical storage node.

[0076] The management node can receive the information about the failed storage data blocks sent by the client, and recover each failed storage data block according to the information. The management node can be a server, a PC, a notebook computer, etc. The management node can also be one or more program modules on a device, or a virtual machine or a container running on a device. The management node can also be a cluster composed of multiple devices, for example, a collective name of multiple program modules distributed on multiple devices.

[0077] For the case where data shard writing and data block recovery are triggered at the same time, there are mainly two existing processing methods: one is that the client suspends the writing operation of the current data shard, waits for the completion of the data block recovery, and then executes the writing operation of the data shard, which greatly increases the writing delay; the other is that the client does not perform the writing operation on the current data shard, and directly waits for the next data recovery process to recover the data shard, which requires an additional recovery operation and increases the system load.

[0078] In order to avoid the influence of the data block recovery process on the writing process of the data shard, an embodiment of the present application provides a data writing method, the execution subject of which is a client in Figure 1 which will be described in detail below.

[0079] Please refer to Figure 3 The data writing method provided by the embodiment of the present application includes steps S101-S103.

[0080] S101, according to a preset erasure ratio, splitting the to-be-stored data into a plurality of data shards, and determining a storage block corresponding to each data shard.

[0081] Among them, the storage block is a data block used for storing the data shard.

[0082] The distributed storage system usually uses the erasure code (Erasure Code, hereinafter referred to as EC) technology to realize data storage, that is, the to-be-stored data is divided into n data shards, and then the n data shards are erasure coded to obtain m redundant shards for data recovery processing.

[0083] It can be understood that the ratio of the number n of data shards to the number m of redundant shards is the erasure ratio, and the plurality of data shards obtained by splitting the to-be-stored data according to the preset erasure ratio n:m includes n data shards and m redundant shards.

[0084] S102, according to a plurality of cache information cached locally, determining a target storage node where each storage block is located at the current time.

[0085] The plurality of pieces of cache information are updated based on recovery progress of the recovered data blocks reported by the plurality of storage nodes, and each of the recovered data blocks reported by the storage nodes satisfies a preset reporting condition, which is sent to the plurality of storage nodes by the client in a timely manner.

[0086] When the data block is recovered, the storage node where the data block is located changes. For example, the data block 1 is initially created on the storage node 1, and the storage node where the data block 1 is located recorded on the management node is the storage node 1. If the data block 1 fails to store the data shard, the management node selects the storage node 4 to recover the data block 1, that is, the storage node 4 creates a new data block 1, and obtains the stored data shard from other storage nodes to recover the data block 1. When the storage node 4 successfully recovers the data block 1, the storage node 4 sends the recovery status of the data block 1 to the management node. At this time, the management node modifies the storage node where the data block 1 is located to the storage node 4, and sends a deletion instruction to the storage node 1 to delete the original data block 1.

[0087] If the client needs to write a data shard into the data block 1 during the process of recovering the data block 1 on the storage node 4, at this time, the storage node where the data block 1 is located obtained by the client from the management node is the storage node 1, not the storage node 4.

[0088] The client caches a plurality of pieces of cache information locally, each piece of cache information records a storage node where a data block is located, and the plurality of pieces of cache information are updated in a timely manner according to the recovery progress of the recovered data blocks reported by all the storage nodes, that is, at any time, the storage node where the data block recorded by each piece of cache information is located is the storage node where the data block is located at the current time.

[0089] It can be understood that the target storage node refers to the storage node where the data block needs to store the data shard at the current time. For example, the data block 1 is initially created on the storage node 1, and is currently recovered on the storage node 4. Therefore, the storage node where the data block 1 is located recorded in the cache information about the data block 1 cached locally is the storage node 4. The data block 2 is initially created on the storage node 2, and until the current time, the management node does not specify other storage nodes to recover the data block 2. Therefore, the storage node where the data block 1 is located recorded in the cache information about the data block 1 cached locally is the storage node 2. The data shard 1 corresponds to the data block 1, and the data shard 2 corresponds to the data block 2. Therefore, according to the cache information cached locally, the target storage node where the data block corresponding to the data shard 1 is located is the storage node 4, and the target storage node where the data block corresponding to the data shard 2 is located is the storage node 2.

[0090] S103, each data shard is sent to the target storage node where the data block corresponding to each data shard is located to perform data writing.

[0091] After determining the target storage node where each data shard corresponding storage block locates, a write data request containing the corresponding data shard is sent to each target storage node, so that each target storage node writes the data shard in the respective write data request into the data shard corresponding storage block.

[0092] The step S102 is described in detail as follows.

[0093] On the basis of Figure 3 , refer to Figure 4 , the step S102 includes sub-steps S102-1-S102-2.

[0094] In the embodiment of the application, the process of steps S102-1-S102-2 is performed once for each storage block to determine the target storage node where each storage block locates at the current time.

[0095] S102-1, if the target first information recording the identifier of the storage block exists in the plurality of first information, and the state of the data block recorded in the target first information is the recovery state or the normal state, the storage node where the data block recorded in the target first information locates is taken as the target storage node where the storage block locates at the current time.

[0096] The plurality of cache information includes the plurality of first information cached in the local cache area, and each first information records the identifier, the state and the storage node where the data block locates of one data block.

[0097] Since the space of the local cache area is limited, each first information in the local cache area records the identifier, the state and the storage node where the data block locates of the data block which has performed the data shard write operation.

[0098] For any one storage block, if the target first information recording the identifier of the storage block exists in the local cache area, it means that the storage block has performed the data shard write operation before the current time, and if the state of the storage block is determined to be the normal state or the recovery state according to the target first information, it means that the data shard can be continuously written into the data block, that is, the storage node where the data block recorded in the target first information locates is the target storage node where the storage block locates.

[0099] S102-2, if the target first information recording the identifier of the storage block exists in the plurality of first information, and the state of the data block recorded in the target first information is the abnormal state, a query request is sent to the management node to obtain the target storage node where the storage block locates at the current time.

[0100] For any one storage block, although the target first information recording the identity of the storage block exists in the local cache area, that is, the storage block has performed data slice write operation before the current time, but according to the target first information, it is determined that the current state of the storage block is an abnormal state, which means that the storage node where the data block recorded in the target first information is located may not be the target storage node where the storage block is located. At this time, the target storage node where the storage block is located at the current time needs to be obtained from the management node.

[0101] Since there may also be a situation that the target first information recording the identity of the storage block does not exist in the plurality of first information, for this, please refer to Figure 4 Each storage block also needs to perform the process of steps S102-3 to S102-5 once.

[0102] S102-3, if the target first information recording the identity of the storage block does not exist in the plurality of first information, it is determined whether the target second information recording the identity of the storage block exists in the plurality of second information.

[0103] The plurality of cache information further includes a plurality of second information cached in the local cache chain table, each second information includes the identity of a data block and the storage node where the data block is located, and the plurality of second information is updated based on the recovery progress of the recovered data block reported by the plurality of storage nodes. The plurality of first information is updated based on the update of the plurality of second information.

[0104] In the embodiment of the application, the local cache chain table is used to record the identity of the data block being recovered and the storage node where the data block is located, including the data block which has performed data slice write operation and the data block which has not performed data slice write operation.

[0105] It can be understood that for the recovery progress of the recovered data block reported by the plurality of storage nodes, if the recovered data block has performed data slice write operation, after updating the related second information in the local cache chain table according to the recovery progress thereof, the related first information in the local cache area also needs to be updated. If the recovered data block has not performed data slice write operation, only the related second information in the local cache chain table needs to be updated according to the recovery progress thereof.

[0106] For any one storage block, if the target first information recording the identity of the storage block does not exist in the local cache area, it means that the storage block has not performed data slice write operation before the current time. At this time, it is necessary to determine whether the storage block is being recovered by judging whether the target second information recording the identity of the storage block exists in the plurality of second information.

[0107] S102-4, if the target second information exists, the storage node where the data block recorded by the target second information is located is taken as the target storage node where the storage block is located at the current time.

[0108] Wherein, when the target second information recording the identification of the storage block exists, it means that the storage block is being recovered, and the storage node where the data block recorded in the target second information is located is the target storage node where the storage block is located.

[0109] S102-5, if the target second information does not exist, a query request is sent to the management node to obtain the target storage node where the storage block is located at the current time.

[0110] Wherein, when the target second information recording the identification of the storage block does not exist, it means that the storage block is not in the situation of being recovered. At this time, the storage node where the storage block is located recorded in the management node is correct, and the target storage node where the storage block is located is obtained by sending a query request for the storage block to the management node.

[0111] For each storage block, the processes of steps S102-1 to S102-5 are executed once to determine the storage node where each storage block is located at the current time.

[0112] According to the erasure strategy n+m, when the number of successfully written data shards is greater than n, that is, the value of the erasure count is greater than n, the to-be-stored data can be regarded as successfully stored. However, due to the fact that the states of some data shards corresponding to the storage blocks are in the state of being recovered, and the recovery result is unknown, it will affect the judgment of whether the to-be-stored data is successfully written as a whole, and further affect the consistency of the data.

[0113] Therefore, on the basis of Figure 3 , please refer to Figure 5 After step S103, the data writing method further includes steps S104 to S108.

[0114] S104, receiving the response message returned by each target storage node.

[0115] Wherein, the response message represents whether the target storage node successfully writes the data shard into the storage block corresponding to the data shard.

[0116] It can be understood that each response message representing a writing failure is not included in the erasure count. For each response message representing a writing success, the processes of steps S105 to S106 are executed to determine whether the response message representing the writing success is included in the erasure count.

[0117] S105, if the state of the storage block where the data shard is written on the target storage node returning the response message is in the state of being recovered, the response message is not included in the erasure count.

[0118] For each response message representing a successful write, whether to include the response message in the erasure count is determined according to the state of the storage block in which the data shard is written on the target storage node that returns the response message. If the state of the storage block in which the data shard is successfully written is a normal state, the corresponding response message is included in the erasure count.

[0119] S106, if the state of the storage block in which the data shard is written on the target storage node that returns the response message is a normal state, the response message is included in the erasure count.

[0120] For the case that the state of the storage block in which the data shard is successfully written is a recovery state, since the recovery result is unknown, if the subsequent recovery of the storage block fails, the data shard successfully written in the storage block will be deleted together with the storage block. In order to ensure data consistency, the corresponding response message cannot be included in the erasure count.

[0121] S107, after the judgment of whether to include each response message representing a successful write in the erasure count is completed, whether the to-be-stored data is successfully stored is determined according to the value of the erasure count.

[0122] After the above operation is performed on each response message representing a successful write, the value of the final erasure count is obtained, which can be used to determine whether the to-be-stored data is successfully stored.

[0123] S108, if the value of the erasure count meets a preset condition, it is determined that the to-be-stored data is successfully stored.

[0124] The preset condition refers to whether the value of the erasure count is greater than n (the preset erasure ratio is n:m). If the value of the erasure count meets the preset condition, it is determined that the to-be-stored data is successfully stored.

[0125] If the value of the erasure count does not meet the preset condition, the step of determining the target storage node in which each storage block is located at the current time according to the plurality of cache information stored in the local cache (i.e., step S102) is returned, until the to-be-stored data is successfully stored.

[0126] Since the plurality of cache information is used to determine the target storage node in which each data shard of the to-be-stored data corresponds to a storage block at the current time when the client executes the above data write method, it needs to be constantly updated. For this purpose, the embodiment of the application further provides a cache information updating method, the execution subject of which is the client in Figure 1 which will be described in detail below.

[0127] Please refer to Figure 6 , the cache information updating method comprises steps S201-S202.

[0128] S201, send preset reporting conditions to the plurality of storage nodes in a timing manner, so that the plurality of storage nodes report the recovery progress of the recovered data block meeting the preset reporting conditions to the client.

[0129] The preset reporting condition refers to that the state change time of the data block is greater than the incremental ID, and the incremental ID is the time when the client last received the recovery progress of the recovered data block. The client sends a command to all storage nodes in a timing manner, and the command contains the incremental ID.

[0130] As a possible implementation manner, the client can send a message to all storage nodes every second.

[0131] It can be understood that when the state change time of the data block is greater than the incremental ID, it means that the data block can be a newly created recovered data block, or a data block that has just been successfully recovered, and also can be a data block that has just been recovered unsuccessfully. When the state change time of the data block is less than the incremental ID, it means that the data block has been reported, and its state has not changed again until the current time, so it does not need to be reported repeatedly. For the case that the incremental ID is 0, it means that the recovery progress of all recovered data blocks needs to be reported.

[0132] S202, update the plurality of pieces of cached information in the local cache according to the recovery progress of the recovered data block received each time.

[0133] The step S202 will be described in detail below.

[0134] Since the plurality of pieces of cached information in the local cache includes the plurality of pieces of first information cached in the local cache area and the plurality of pieces of second information cached in the local cache linked list, and the plurality of pieces of second information are updated based on the recovery progress of the recovered data block reported by the plurality of storage nodes, the plurality of pieces of first information are updated based on the update of the plurality of pieces of second information.

[0135] For this purpose, as shown in Figure 7 , for the recovery progress of the recovered data block received each time, the steps S202-1-S202-2 are first executed to update the plurality of pieces of second information, and then the step S202-3 is executed to update the plurality of pieces of first information.

[0136] S202-1, if the recovery progress is in recovery, add the second information recording the identifier of the recovered data block and the storage node in the local cache linked list.

[0137] Each piece of second information in the local cache linked list is used to record the identifier of a recovered data block and the storage node.

[0138] Understandably, when the recovery progress of the recovered data block received by the client is in recovery, it means that the recovery process of the recovered data block has not ended, and thus the second information recorded with the identifier of the data block and the storage node where the data block is located needs to be added in the local cache chain table.

[0139] S202-2, if the recovery progress is recovery success or recovery failure, the second information recorded with the identifier of the recovered data block in the local cache chain table is deleted.

[0140] Wherein, the recovery progress of the recovered data block received by the client is recovery success or recovery failure, which means that the recovery process of the recovered data block has ended, and the second information recorded with the identifier of the data block and the storage node where the data block is located needs to be deleted in the local cache chain table.

[0141] S202-3, according to the updating situation of the plurality of second information, the plurality of first information is updated.

[0142] Optionally, the implementation process of step S202-3 can be as follows:

[0143] For the newly added second information in the local cache chain table, if the reference first information recorded with the identifier of the data block in the local cache area is the same as the identifier of the data block recorded in the second information, the storage node where the data block recorded in the reference first information is located is updated to the storage node where the data block recorded in the second information is located, and the state of the data block recorded in the reference first information is set to the recovery state;

[0144] For the second information newly deleted in the local cache chain table, if the recovery progress of the data block recorded in the second information is recovery success, and the reference first information exists in the local cache area, the state of the data block recorded in the reference first information is set to the normal state;

[0145] If the recovery progress of the data block recorded in the second information is recovery failure, and the reference first information exists in the local cache area, the state of the data block recorded in the reference first information is set to the abnormal state.

[0146] It should be noted that the storage node may be powered off or restarted due to various conditions, and all data blocks on the restarted storage node may be abnormal, and the data fragments cannot be directly written into these data blocks. For this purpose, each storage node reports the process ID of the node when reporting the recovery progress of the recovered data block, and judges whether the storage node is powered off or restarted according to whether the process ID changes.

[0147] For any storage node, if the process ID reported at the current moment is the same as the process ID reported in the previous time, then the relevant operations of step S202 above are executed directly according to the recovery progress of the reported recovered data block, so as to update multiple second information entries in the local cache linked list and multiple first information entries in the local cache area.

[0148] If the process ID reported at the current moment is different from the process ID reported in the previous time, it can be determined that a power outage or restart has occurred. In this case, all the second information related to the storage node in the local cache list will be deleted, and the status of the data blocks in the first information related to the storage node in the local cache area will be set to an abnormal state.

[0149] Regarding the recovery progress of the currently reported recovered data blocks, the relevant operations in step S202 are also executed to update multiple second pieces of information in the local cache linked list and multiple first pieces of information in the local cache area.

[0150] To explain the technical solutions provided by the above method embodiments in more detail, the embodiments of the present invention will be further described using the following specific examples.

[0151] like Figure 8 As shown, a 4-node environment with an erasure ratio of n:m=2:1 is created, and LUNs are mapped via ISCSI.

[0152] Data block BLK-1 is created on storage node DN-1, data block BLK-2 is created on storage node DN-2, and data block BLK-3 is created on storage node DN-3.

[0153] When the first data write is performed, since BLK-1, BLK-2, and BLK-3 have not been written to any data fragments, there is no record of the identifiers, statuses, and first information of the storage nodes of BLK-1, BLK-2, and BLK-3 in the local cache area. Furthermore, BLK-1, BLK-2, and BLK-3 are not in the process of being restored. Therefore, there is no record of the identifiers and second information of the storage nodes of BLK-1, BLK-2, and BLK-3 in the local cache list.

[0154] The client obtains information from the management node that BLK-1 is on DN-1, BLK-2 is on DN-2, and BLK-3 is on DN-3. It then generates three pieces of information in the local cache, which record the identifier, status, and storage node of BLK-1, BLK-2, and BLK-3, respectively. Among them, the status of BLK-1, BLK-2, and BLK-3 are all normal.

[0155] The client splits the data to be stored into data shards data-1 and data-2, generates a redundant shard data-3, and sends data-1 to DN-1 for writing to BLK-1, data-2 to DN-2 for writing to BLK-2, and data-3 to DN-3 for writing to BLK-3.

[0156] The client learns from the response message returned by DN-1 that data-1 fails to be written to BLK-1, learns from the response message returned by DN-2 that data-2 succeeds to be written to BLK-2, and learns from the response message returned by DN-3 that data-3 succeeds to be written to BLK-3.

[0157] The client determines that the data to be stored succeeds to be written based on the value of the erasure count obtained from the response messages returned by DN-2 and DN-3 indicating that the writing succeeds, and reports the case that data-1 fails to be written to BLK-1 to the management node, which then selects a storage node DN-4 to perform recovery of BLK-1.

[0158] The DN-4 reports the recovery progress of BLK-1 to the client, which adds a record with the identifier of BLK-1 and the second information of the storage node where BLK-1 is located in the local cache chain table, and modifies the storage node where BLK-1 is located recorded in the first information related to BLK-1 in the local cache area from DN-1 to DN-4, and modifies the state of BLK-1 from a normal state to a recovery state.

[0159] When performing the second data writing, the client obtains from the local cache area that BLK-1 is on DN-4, BLK-2 is on DN-2, and BLK-3 is on DN-3.

[0160] The client splits the data to be stored into data shards data-1 and data-2, generates a redundant shard data-3, and sends data-1 to DN-4 for writing to BLK-1, data-2 to DN-2 for writing to BLK-2, and data-3 to DN-3 for writing to BLK-3.

[0161] The client learns from the response message returned by DN-4 that data-1 succeeds to be written to BLK-1, learns from the response message returned by DN-2 that data-2 fails to be written to BLK-2, and learns from the response message returned by DN-3 that data-3 succeeds to be written to BLK-3.

[0162] The client determines that the data to be stored fails to be written based on the value of the erasure count obtained from the response messages returned by DN-4 and DN-3 indicating that the writing succeeds, and then re-performs the data writing.

[0163] The BLK-1 fails to be restored on the DN-4, the DN-4 reports the restoration progress of the BLK-1 to the client, the client deletes the second information related to the BLK-1 in the local cache link table, and then modifies the state of the BLK-1 from the restoration state to the abnormal state in the first information related to the BLK-1 in the local cache area.

[0164] The management node selects the DN-4 again to restore the BLK-1, the DN-4 reports the restoration progress of the BLK-1 to the client, the client adds the second information related to the BLK-1 in the local cache link table, and then modifies the state of the BLK-1 from the abnormal state to the restoration state in the first information related to the BLK-1 in the local cache area.

[0165] The BLK-1 is successfully restored on the DN-4, the DN-4 reports the restoration progress of the BLK-1 to the client, the client deletes the second information related to the BLK-1 in the local cache link table, and then modifies the state of the BLK-1 from the restoration state to the normal state in the first information related to the BLK-1 in the local cache area.

[0166] When the third data writing is performed, the client obtains, from the local cache area, that the BLK-1 is on the DN-4, the BLK-2 is on the DN-2, and the BLK-3 is on the DN-3, and the states are all normal states.

[0167] The client splits the to-be-stored data into data fragments data-1 and data-2, generates a redundant fragment data-3, sends the data-1 to the DN-4 to write the BLK-1, sends the data-2 to the DN-2 to write the BLK-2, and sends the data-3 to the DN-3 to write the BLK-3.

[0168] The client knows, according to the response message returned by the DN-4, that the data-1 is successfully written into the BLK-1, knows, according to the response message returned by the DN-2, that the data-2 is successfully written into the BLK-2, and knows, according to the response message returned by the DN-3, that the data-3 is successfully written into the BLK-3.

[0169] The client obtains the value of the erasure correction count as 3 > n based on the response messages returned by the DN-1, the DN-2 and the DN-3, and determines that the to-be-stored data is successfully written.

[0170] Compared with the prior art, the embodiment of the application has the following effects:

[0171] (1) Through the local cache information updated by the timing, the current storage node of each data block for storing the data fragment is determined, and then the data fragment is sent to the storage node to write data, thereby avoiding the influence of the data block recovery process on the data fragment writing process and reducing the writing delay;

[0172] (2) The data block recovery process and the data fragment writing process can be performed simultaneously without triggering an additional recovery process;

[0173] (3) The update mechanism of the local cache information makes the data block recovery process or the data fragment writing process not affect the correctness and consistency of the data even if the network is abnormal, the storage node is powered off or restarted, etc.

[0174] In order to perform the above-mentioned method embodiments and the corresponding steps in each possible implementation, an implementation of a data writing device 100 and an implementation of a cache information updating device 200 are respectively given below.

[0175] Please refer to Figure 9 , the data writing device 100 is applied to a client in Figure 1 may include a splitting module 101, a determining module 102, a first sending module 103 and a judging module 104.

[0176] The splitting module 101 is configured to split the to-be-stored data into a plurality of data fragments according to a preset erasure coding ratio, and determine a storage block corresponding to each data fragment, wherein the storage block is a data block for storing the data fragment.

[0177] The determining module 102 is configured to determine a target storage node of each storage block at a current time according to a plurality of cache information locally cached, wherein the plurality of cache information is updated based on a recovery progress of a recovered data block reported by a plurality of storage nodes, and each recovered data block reported by each storage node satisfies a preset reporting condition, and the preset reporting condition is sent to the plurality of storage nodes by the client at a timing.

[0178] The first sending module 103 is configured to send each data fragment to the target storage node where the storage block corresponding to each data fragment is located, so as to write data.

[0179] Optionally, the plurality of cache information includes a plurality of first information cached in the local cache area, each of the first information records an identifier of a data block, a state of the data block, and a storage node where the data block is located, and the distributed storage system further includes a management node in communication connection with the client; the determining module 102 is specifically configured to, for each storage block, if there is target first information recording the identifier of the storage block in the plurality of first information, and the state of the data block recorded in the target first information is the recovery state or the normal state, taking the storage node where the data block recorded in the target first information is located as the target storage node where the storage block is located at the current time; if there is target first information recording the identifier of the storage block in the plurality of first information, and the state of the data block recorded in the target first information is the abnormal state, sending a query request to the management node to obtain the target storage node where the storage block is located at the current time.

[0180] Optionally, the plurality of cache information further includes a plurality of second information cached in the local cache linked list, each of the second information includes an identifier of a data block and a storage node where the data block is located, the plurality of second information is updated based on the recovery progress of the recovered data block reported by the plurality of storage nodes, and the plurality of first information is updated based on the update of the plurality of second information; the determining module 102 is further specifically configured to, if there is no target first information recording the identifier of the storage block in the plurality of first information, judging whether there is target second information recording the identifier of the storage block in the plurality of second information; if there is the target second information, taking the storage node where the data block recorded in the target second information is located as the target storage node where the storage block is located at the current time; if there is no target second information, sending a query request to the management node to obtain the target storage node where the storage block is located at the current time.

[0181] The determining module 104 is configured to receive the response message returned by each target storage node, the response message representing whether the target storage node successfully writes the data shard into the storage block corresponding to the data shard; for each response message representing the writing success, if the state of the storage block where the data shard is written on the target storage node returning the response message is the recovery state, the response message is not included in the erasure count; if the state of the storage block where the data shard is written on the target storage node returning the response message is the normal state, the response message is included in the erasure count; after the judgment of whether each response message representing the writing success is included in the erasure count is completed, it is judged whether the to-be-stored data is stored successfully according to the value of the erasure count; if the value of the erasure count meets the preset condition, it is determined that the to-be-stored data is stored successfully; if the value of the erasure count does not meet the preset condition, the step of determining the target storage node where each storage block is located at the current time according to the plurality of cache information cached locally is returned until the to-be-stored data is stored successfully.

[0182] Please refer to Figure 10 , the cache information updating device 200 is appliedFigure 1 The client can include a second sending module 201 and an updating module 202.

[0183] The second sending module 201 is configured to send the preset reporting condition to the plurality of storage nodes at a timing, so that the plurality of storage nodes report the recovery progress of the recovered data block satisfying the preset reporting condition to the client.

[0184] The updating module 202 is configured to update a plurality of pieces of cache information locally cached according to the recovery progress of the recovered data block received each time. The plurality of pieces of cache information are used to determine a target storage node where a storage block corresponding to each data shard of the to-be-stored data at a current time is located when the data writing method is executed at the client. The storage block is a data block used to store the data shard.

[0185] Optionally, the plurality of pieces of cache information include a plurality of first information cached in a local cache area and a plurality of second information cached in a local cache linked list. Each piece of second information includes an identifier of a data block and a storage node where the data block is located. The updating module 202 is specifically configured to, for the recovery progress of the recovered data block received each time, if the recovery progress is in recovery, add, in the local cache linked list, the second information recording the identifier of the recovered data block and the storage node where the recovered data block is located; if the recovery progress is recovery success or recovery failure, delete the second information recording the identifier of the recovered data block in the local cache linked list; and update the plurality of first information according to the update of the plurality of second information.

[0186] Optionally, when the updating module 202 is used to update the plurality of first information according to the update of the plurality of second information, the updating module 202 is specifically configured to, for the newly added second information in the local cache linked list, if there is reference first information recording the same identifier of the data block in the local cache area as the identifier of the data block recorded in the second information, update the storage node where the data block recorded in the reference first information is located to the storage node where the data block recorded in the second information is located, and set the state of the data block recorded in the reference first information to a recovery state; for the newly deleted second information in the local cache linked list, if the recovery progress of the data block recorded in the second information is recovery success and there is reference first information in the local cache area, set the state of the data block recorded in the reference first information to a normal state; if the recovery progress of the data block recorded in the second information is recovery failure and there is reference first information in the local cache area, set the state of the data block recorded in the reference first information to an abnormal state.

[0187] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the data writing apparatus 100 and the cache information updating apparatus 200 described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein again.

[0188] Further, the embodiment of the present application further provides a client 300, which can be a client in the system 100. Please refer to Figure 1 , Figure 11 , Figure 11 Fig. 1 shows a schematic block diagram of a structure of the client 300 provided by the embodiment of the present application. The client 300 can include a memory 310 and a processor 320.

[0189] The processor 320 can be a general central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs for the data writing method and / or the cache information updating method provided by the above-mentioned method embodiments.

[0190] The memory 310 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. The memory 310 can exist independently and be connected to the processor 320 through a communication bus. The memory 310 can also be integrated with the processor 320. The memory 310 is used to store machine-executable instructions for implementing the scheme of the present application. The processor 320 is used to execute the machine-executable instructions stored in the memory 310 to implement the above-mentioned method embodiments.

[0191] The embodiment of the present application further provides a computer-readable storage medium containing a computer program, which can be used to execute the related operations in the data writing method and / or the cache information updating method provided by the above-mentioned method embodiments when executed.

[0192] To sum up, the data writing method, the cache information updating method and the related device provided by the embodiments of the present application first split the to-be-stored data into a plurality of data shards according to a preset erasure coding ratio, and determine a storage block corresponding to each data shard, the storage block being a data block for storing the data shard; then, according to a plurality of pieces of cached information, determine a target storage node where each storage block is located at a current time, wherein the plurality of pieces of cached information are updated based on a recovery progress of a recovered data block reported by a plurality of storage nodes, and each recovered data block reported by each storage node satisfies a preset reporting condition, and the preset reporting condition is sent to the plurality of storage nodes by a client at a regular time; finally, each data shard is sent to the target storage node where the storage block corresponding to each data shard is located, so as to perform data writing. Since in the embodiments of the present application, the client determines the storage node where each data block for storing the data shard is located at the current time through the plurality of pieces of cached information which are updated at a regular time, and then sends the data shard to the storage node where the data block is located at the current time, so as to perform data writing, the data block recovery process is avoided from affecting the data shard writing process, and the writing delay is reduced.

[0193] The above merely describes specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data write method, characterized by, The application relates to a client applied to a distributed storage system, wherein the distributed storage system further comprises a plurality of storage nodes, the client is in communication connection with the plurality of storage nodes, at least one data block is created on each of the storage nodes, and the method comprises the following steps: According to a preset erasure correction ratio, to-be-stored data is split into a plurality of data fragments, and a storage block corresponding to each data fragment is determined, wherein the storage block is a data block used for storing the data fragment; According to a plurality of pieces of cached information cached locally, a target storage node where each storage block is located at a current time is determined; The plurality of pieces of cached information are updated based on recovery progress of a recovered data block reported by the plurality of storage nodes, and the recovered data block reported by each storage node satisfies a preset reporting condition, wherein the preset reporting condition is sent to the plurality of storage nodes by the client at a regular time; Each data fragment is sent to a target storage node where a storage block corresponding to each data fragment is located, so as to perform data writing; The plurality of pieces of cached information comprise a plurality of first information cached in a local cache area, each piece of first information records an identifier, a state and a storage node where a data block is located, and the distributed storage system further comprises a management node, wherein the management node is in communication connection with the client; The step of determining, according to the plurality of pieces of cached information cached locally, a target storage node where each storage block is located at a current time comprises the following steps: For each storage block, if there is target first information recording the identifier of the storage block in the plurality of pieces of first information, and the state of the data block recorded in the target first information is a recovery state or a normal state, the storage node where the data block recorded in the target first information is located is taken as a target storage node where the storage block is located at the current time; If there is target first information recording the identifier of the storage block in the plurality of pieces of first information, and the state of the data block recorded in the target first information is an abnormal state, a query request is sent to the management node, so as to obtain a target storage node where the storage block is located at the current time.

2. The method of claim 1, wherein, The plurality of pieces of cached information further comprise a plurality of second information cached in a local cache linked list, each piece of second information comprises an identifier of a data block and a storage node where the data block is located, the plurality of pieces of second information are updated based on recovery progress of a recovered data block reported by the plurality of storage nodes, and the plurality of pieces of first information are updated based on updating conditions of the plurality of pieces of second information; The step of determining, according to the plurality of pieces of cached information cached locally, a target storage node where each storage block is located at a current time further comprises the following steps: If there is no target first information recording the identifier of the storage block in the plurality of pieces of first information, it is judged whether there is target second information recording the identifier of the storage block in the plurality of pieces of second information; If the target second information exists, the storage node where the data block recorded in the target second information is located is taken as a target storage node where the storage block is located at the current time. If the target second information does not exist, a query request is sent to the management node to obtain the target storage node where the storage block is located at the current time.

3. The method of claim 1, wherein, The method further comprises: receiving a response message returned by each target storage node, the response message representing whether the target storage node successfully writes the data shard into the storage block corresponding to the data shard; for each response message representing a successful write, if the state of the storage block where the data shard is written on the target storage node returning the response message is a recovery state, the response message is not included in the erasure count; if the state of the storage block where the data shard is written on the target storage node returning the response message is a normal state, the response message is included in the erasure count; after the judgment of whether each response message representing a successful write is included in the erasure count is completed, it is judged whether the data to be stored is successfully stored according to the value of the erasure count; if the value of the erasure count meets a preset condition, it is determined that the data to be stored is successfully stored; if the value of the erasure count does not meet the preset condition, the step of determining the target storage node where each storage block is located at the current time according to the plurality of cached information cached locally is returned until the data to be stored is successfully stored.

4. A cache information update method characterized by comprising: A client applied to a distributed storage system, the distributed storage system further comprising a plurality of storage nodes, at least one data block is created on each of the storage nodes, the client is in communication connection with the plurality of storage nodes, and the method comprises: sending a preset reporting condition to the plurality of storage nodes at a time, so that the plurality of storage nodes report the recovery progress of the recovered data block meeting the preset reporting condition to the client; updating a plurality of cached information cached locally according to the recovery progress of the recovered data block received each time; wherein the plurality of cached information is used to determine the target storage node where each data shard corresponding to the data to be stored is located at the current time when the client executes the data write method of any one of claims 1-3, and the storage block is a data block used to store the data shard.

5. The method of claim 4, wherein, The plurality of cached information comprises a plurality of first information cached in a local cache area and a plurality of second information cached in a local cache linked list, and each second information comprises an identifier of a data block and a storage node where the data block is located. The step of updating the plurality of cached information cached locally according to the recovery progress of the recovered data block received each time comprises: for the recovery progress of the recovered data block received each time, if the recovery progress is in recovery, a second information recording the identifier of the recovered data block and the storage node where the recovered data block is located is added in the local cache linked list; if the recovery progress is recovery success or recovery failure, the second information recording the identifier of the recovered data block in the local cache linked list is deleted; the plurality of first information is updated according to the updating condition of the plurality of second information. The plurality of cached information comprises a plurality of first information cached in a local cache area and a plurality of second information cached in a local cache linked list, and each second information comprises an identifier of a data block and a storage node where the data block is located. The step of updating the plurality of cached information cached locally according to the recovery progress of the recovered data block received each time comprises: for the recovery progress of the recovered data block received each time, if the recovery progress is in recovery, a second information recording the identifier of the recovered data block and the storage node where the recovered data block is located is added in the local cache linked list; if the recovery progress is recovery success or recovery failure, the second information recording the identifier of the recovered data block in the local cache linked list is deleted; the plurality of first information is updated according to the updating condition of the plurality of second information.

6. The method of claim 5, wherein, Each of the first information records an identification of the data block, a state and a storage node where the data block is located; The updating of the plurality of first information according to the updating of the plurality of second information comprises: For the second information newly added in the local cache chain table, if there is reference first information recorded in the local cache area and the identification of the data block recorded in the reference first information is same as the identification of the data block recorded in the second information, the storage node where the data block recorded in the reference first information is located is updated to the storage node where the data block recorded in the second information is located, and the state of the data block recorded in the reference first information is set to the recovery state; For the second information newly deleted in the local cache chain table, if the recovery progress of the data block recorded in the second information is recovery success, and there is the reference first information in the local cache area, the state of the data block recorded in the reference first information is set to the normal state; If the recovery progress of the data block recorded in the second information is recovery failure, and there is the reference first information in the local cache area, the state of the data block recorded in the reference first information is set to the abnormal state.

7. A data writing apparatus, characterized by comprising: The application is applied to a client in a distributed storage system, the distributed storage system further comprises a plurality of storage nodes, the client is in communication connection with the plurality of storage nodes, at least one data block is created on each of the storage nodes, and the device comprises: A splitting module is configured to split to-be-stored data into a plurality of data shards according to a preset erasure coding ratio, and determine a storage block corresponding to each of the data shards, wherein the storage block is a data block used for storing the data shard. A determining module is configured to determine a target storage node where each of the storage blocks is located at a current time according to a plurality of cache information cached locally. The plurality of cache information is updated based on a recovery progress of a recovered data block reported by the plurality of storage nodes, each of the recovered data blocks reported by each of the storage nodes satisfies a preset reporting condition, and the preset reporting condition is sent to the plurality of storage nodes by the client at a regular time. A first sending module is configured to send each of the data shards to a target storage node where a storage block corresponding to each of the data shards is located, so as to perform data writing. The plurality of cache information includes a plurality of first information cached in a local cache area, each of the first information records an identifier of the data block, a state and a storage node where the data block is located, the distributed storage system further includes a management node, and the management node is in communication connection with the client; the determination module is specifically configured to, for each of the storage blocks, if there is target first information recording the identifier of the storage block in the plurality of first information, and the state of the data block recorded in the target first information is a recovery state or a normal state, the storage node where the data block recorded in the target first information is located is taken as a target storage node where the storage block is located at the current time; if there is target first information recording the identifier of the storage block in the plurality of first information, and the state of the data block recorded in the target first information is an abnormal state, a query request is sent to the management node to obtain the target storage node where the storage block is located at the current time.

8. A cache information updating apparatus characterized by comprising: The client applied to the distributed storage system, the distributed storage system further includes a plurality of storage nodes, at least one data block is created on each of the storage nodes, the client is in communication connection with the plurality of storage nodes, and the device includes: The second sending module is configured to send a preset reporting condition to the plurality of storage nodes at a time, so that the plurality of storage nodes report the recovery progress of the recovered data block meeting the preset reporting condition to the client; The updating module is configured to update a plurality of cache information locally cached according to the recovery progress of the recovered data block received each time; The plurality of cache information is used to determine the target storage node where the storage block corresponding to each data shard of the to-be-stored data at the current time when the data writing method of any one of claims 1-3 is executed on the client, and the storage block is a data block used to store the data shard.

9. A client, characterized in that, The memory stores a computer program, and the computer program is executed by the processor to implement the following operations: The data writing method of any one of claims 1-3 is executed; or The cache information updating method of any one of claims 4-6 is executed; or The data writing method of any one of claims 1-3 and the cache information updating method of any one of claims 4-6 are executed simultaneously.

10. A distributed storage system, characterized by, The client of claim 9 is included.

11. A computer readable storage medium, characterized in that, The memory stores a computer program, and the computer program is executed by the processor to implement the following operations: The data writing method of any one of claims 1-3 is executed; or The cache information updating method of any one of claims 4-6 is executed; or The data writing method of any one of claims 1-3 and the cache information updating method of any one of claims 4-6 are executed simultaneously.

Images