An implementation method of sector reading based on distributed storage erasure code

Abstract

The present invention discloses an implementation method for sector reading based on distributed storage erasure code, which includes the following steps: The client sends a read request to the INFINITY storage pool with an EC data protection policy. INFINITY determines the PG and OSD to be read through the CRUSHMAP core mechanism; after receiving the message, the OSDC sends a message to the primary OSD to obtain the object lock. After the primary OSD replies, the OSDC sends a message to the OSDs storing K copies of data to read the data. After successfully reading the data, the OSDC sends a message to the primary OSD to unlock; after the primary OSD releases the object lock, it replies to the OSDC with a message, and the OSDC does not process this message. It can improve the read performance in the INFINITY EC scenario while retaining the original EC read logic. Through sector reading, the read bandwidth in the INFINITY EC scenario can be significantly improved and storage resources such as CPU and memory can be saved.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage performance, and particularly to an implementation method of sector reading based on distributed storage erasure code. Background Art

[0002] Detuo INFINITY Storage provides a software-defined and unified storage solution, which is a distributed storage system that can be scaled up massively, has high performance and no single point of failure. The INFINITY storage system provides block, file and object storage on the same underlying architecture, enabling users to independently select the storage method they need. Objects are the basis of the INFINITY storage system, that is, its basic storage unit. Any format of data, whether it is a block, an object or a file, is stored in the placement group (PG) of the INFINITY cluster in the form of an object. Using object storage, we can decouple the platform from the hardware. In INFINITY, since objects have no physical storage path binding, objects are very flexible and location-independent. This also enables the scale of INFINITY to expand nearly linearly from the PB level to the EB level.

[0003] Erasure Code (EC) is a method of redundant data storage. Its principle is similar to RAID, but it is more flexible than the RAID method. It divides the written data into N pieces of original data, calculates M pieces of parity data from these N pieces of original data, stores the N + M pieces of data in different devices or nodes respectively, and restores all data through any N pieces of data blocks among the N + M pieces.

[0004] The INFINITY Object Storage Device (OSD) processes data replication of the INFNITY client. When data is replicated multiple times between devices, network load easily occurs among INFINITY OSDs. Network load between the client and the INFINITY storage cluster. This can introduce latency and cause performance problems. Recovery and rebalancing can also bring problems of significant latency and bandwidth contention on the public network. Summary of the Invention

[0005] The purpose of the present invention is to provide an implementation method of sector reading based on distributed storage erasure code, providing a better read I / O path design, which not only has a shorter I / O path, but also saves network bandwidth and storage resources such as CPU and memory required for data processing and copying on the original I / O path.

[0006] An implementation method of sector reading based on distributed storage erasure code directly sends a read request from the OSDC (OSD Client) to the member OSD, and directly integrates the data after the OSDC receives the data from the member OSD, without going through the original read I / O path of the INFINITY erasure code (the OSDC sends a read request to the primary OSD, the primary OSD then sends a read request to other member OSDs, and the primary OSD returns the data received from the member OSDs to the OSDC after receiving it).

[0007] To achieve the above object, the present invention provides the following technical solutions: An implementation method of sector reading based on distributed storage erasure code includes the following steps:

[0008] Step 1: The client sends a read request to the INFINITY storage pool with an EC data protection policy, and the INFINITY distributed storage system determines the PG and OSD to be read through core mechanisms such as CRUSHMAP;

[0009] Step 2: Send an INFINITY_MSG_OSD_OBJLOCK message to the primary OSD of the PG through the OSDC. After the primary OSD obtains this message, it mainly performs various parameter checks, condition checks for sector reading OP, and obtains the object lock. After the condition checks pass and the object lock is successfully obtained, the primary OSD replies to the OSDC with an INFINITY_MSG_OSD_OBJLOCK_REPLY message;

[0010] Step 3: After the OSDC receives the INFINITY_MSG_OSD_OBJLOCK_REPLY message and the exception check passes, it directly sends a MSG_OSD_EC_READ message to the primary OSD and member OSDs to read the data. Under normal circumstances, only read requests need to be sent to the OSDs storing K copies of data, and no read requests will be sent to the OSDs storing M copies of erasure code. After the member OSDs complete reading the data, they return a MSG_OSD_EC_READ_REPLY message to the OSDC;

[0011] Step 4: After the OSDC receives the MSG_OSD_EC_READ_REPLY message, it sends an INFINITY_MSG_OSD_OBJLOCK message to the primary OSD to unlock, and at the same time assembles the data and returns it to the client;

[0012] Step 5: After the primary OSD releases the object lock, it replies to the OSDC with an INFINITY_MSG_OSD_OBJLOCK_REPLY message, and the OSDC does not perform any processing on this message.

[0013] Preferably, in the above step 1, the client includes block storage, object storage, and file system.

[0014] Preferably, in the INFINITY EC scenario of step 1, sector reading can be enabled by configuring parameters. During the read operation in the INFINITY EC scenario, the sector read I / O process is automatically followed, which can improve the INFINITY EC read performance.

[0015] Preferably, in the INFINITY EC scenario of step 1, the client sends a read request through a standard interface. Before sector reading, the storage system obtains an object lock through the primary OSD. The lock granularity is low, which does not affect the normal read process and increases the probability of sector read concurrency.

[0016] Compared with the prior art, the beneficial effects of the present invention are:

[0017] In order to read data more quickly and accurately during the read operation in the INFINITY EC scenario, under the same network bandwidth, the read performance in the INFINITY EC scenario can be improved while retaining the original EC read logic. Sector reading can significantly increase the read bandwidth in the INFINITY EC scenario and save storage resources such as CPU and memory. At the same time, in case of abnormal situations where sector reading is not suitable, it can quickly switch to the original I / O process, ensuring both performance and the stability of the business system in abnormal scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is the timing diagram of sector reading in the embodiment of the present invention;

[0019] Figure 2 It is the flow chart of the OSDC obtaining the object lock for sector reading in the embodiment of the present invention;

[0020] Figure 3 It is the flow chart of data processing and lock release for sector reading in the embodiment of the present invention;

[0021] Figure 4 It is the bandwidth comparison diagram of sector reading in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0023] This embodiment provides a technical solution: Step 1: The client (block storage, object storage, file system, etc.) sends a read request to the INFINITY storage pool with an EC data protection policy. The INFINITY storage system determines the PG and OSD to be read through core mechanisms such as CRUSHMAP;

[0024] In the INFINITY EC scenario, sector reading can be enabled through configuration parameters. During the read operation in the INFINITY EC scenario, the sector read I / O process is automatically followed, which can improve the INFINITY EC read performance.

[0025] As Figure 4 shown, in the case of k pieces of data plus M pieces of parity codes (normally only k pieces of data are read), the number of data pieces read by the original I / O process is k+(k - 1), and the number of data pieces read by the sector read I / O process is k, saving bandwidth (k - 1) / (2k - 1).

[0026] In the INFINITY EC scenario, the client sends a read request through a standard interface. Before sector reading, the storage system obtains an object lock through the primary OSD. The lock granularity is low, which does not affect the normal read process and increases the probability of sector read concurrency.

[0027] Step 2: The request enters the OSDC. See Figure 2 , first check the legality of the request and parameters, set a timeout mechanism for the original OP, obtain the OSD list of the PG, and send an INFINITY_MSG_OSD_OBJLOCK message to the primary OSD. The message lock status is of the MOSDObjLock:OBJ_LOCK_READ type, and this message also sets a timeout. If the object lock is not obtained within the timeout, the original process is followed.

[0028] Step 3: After the primary OSD obtains the INFINITY_MSG_OSD_OBJLOCK message, first check the legality of the parameters and the execution conditions of the sector read op, etc. After passing the check, obtain the object lock. After successfully obtaining the object lock, set the object lock holding timeout, and at the same time reply to the client with an INFINITY_MSG_OSD_OBJLOCK_REPLY message. If the object lock times out and is not released, the primary OSD automatically releases it.

[0029] Step 4: After the OSDC receives the INFINITY_MSG_OSD_OBJLOCK_REPLY message, check the parameters and execution conditions and cancel the timeout setting of the OBJLOCK message. After passing the check, it will enter the sector read logic. If the check fails, the original normal read process is followed.

[0030] Step 5: Enter the sector read logic layer. See Figure 3, convert the original OP to EC_OP. Obtain the list of member OSDs that need to be read, and send MSG_OSD_EC_READ messages to these OSDs respectively. The member OSDs (primary / secondary) check the legality of the requests and parameters. After successful checking, they read data from the object store (the essence of INFINITY is object storage). After successfully reading the data, they return a MSG_OSD_EC_READ_REPLY message to the OSDC.

[0031] Step 6: The OSDC waits for all OSDs to reply with MSG_OSD_EC_READ_REPLY messages. Then the data collection is completed. Subsequently, the data is assembled through the EC algorithm logic. If the data collection is not completed, it continues to wait. Eventually, the original OP times out and follows the original IO path. If it fails to complete within the timeout, it follows the original IO path. Next, it enters the lock release process.

[0032] Step 7: After the OSDC processes all the data, it returns to the client and simultaneously sends an INFINITY_MSG_OSD_OBJLOCK message to the primary OSD. The message lock status is of the MOSDObjLock: OBJ_UNLOCK type.

[0033] Step 8: After receiving the INFINITY_MSG_OSD_OBJLOCK message, the primary OSD releases the object lock and replies with an INFINITY_MSG_OSD_OBJLOCK_REPLY to the OSDC. The message lock status is also of the MOSDObjLock: OBJ_UNLOCK type. When the OSDC receives this message and the lock status, it doesn't need to do anything.

[0034] The content not described in detail in this specification belongs to the prior art well-known to those skilled in the art.

[0035] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An implementation method of sector reading based on distributed storage erasure code, characterized in that: The specific implementation method includes the following steps: Step 1: The client sends a read request to the INFINITY storage pool with the data protection policy of EC. The INFINITY distributed storage system determines the PG and OSD to be read through core mechanisms such as CRUSHMAP. In the INFINITY EC scenario of Step 1, fan-out read is enabled through configuration parameters. During the read operation in the INFINITY EC scenario, the fan-out read IO process is automatically followed; Step 2: An INFINITY_MSG_OSD_OBJLOCK message is sent to the primary OSD of the PG through OSDC. After the primary OSD obtains this message, it performs various parameter checks, condition checks for fan-out read OP, and obtains the object lock. After the condition checks pass and the object lock is successfully obtained, the primary OSD replies to OSDC with an INFINITY_MSG_OSD_OBJLOCK_REPLY message; Step 3: After OSDC receives the INFINITY_MSG_OSD_OBJLOCK_REPLY message and passes the exception check, it directly sends a MSG_OSD_EC_READ message to the primary OSD and member OSDs to read data, sends a read request to the OSD storing K copies of data, and does not send a read request to the OSD storing M copies of erasure code. After the member OSDs complete reading the data, they return a MSG_OSD_EC_READ_REPLY message to OSDC; Step 4: After OSDC receives the MSG_OSD_EC_READ_REPLY message, it sends an INFINITY_MSG_OSD_OBJLOCK message to the primary OSD to unlock, and at the same time assembles the data and returns it to the client; Step 5: After the primary OSD releases the object lock, it replies to OSDC with an INFINITY_MSG_OSD_OBJLOCK_REPLY message, and OSDC does not perform any processing on this message.

2. The implementation method of sector reading based on distributed storage erasure code according to claim 1, characterized in that: In Step 1, the client includes block storage, object storage, and file system.

3. The implementation method of sector reading based on distributed storage erasure code according to claim 1, characterized in that: In the INFINITY EC scenario of Step 1, the client sends a read request through a standard interface. Before fan-out read, the storage system obtains the object lock through the primary OSD, and the lock granularity is relatively low, which increases the probability of fan-out read concurrency.

Images