Erasure reading request processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of servers, and discloses an erasure reading request processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a target reading request; the target reading request is an erasure reading request; according to storage information in the to-be-read data and a data structure of an erasure pool in the server side, obtaining each target fragment corresponding to the to-be-read data in the server side; and respectively generating reading operation for each target fragment, and sending each reading operation to the server, so that the server reads data in each target fragment according to the reading operation and pushes the data to a memory specified by the client. According to the scheme, the reading operation on the whole strip in the erasure pool is split into the reading operation on part of the target fragments, so that the operation efficiency of the reading operation is improved, an unnecessary disk reading process is avoided, and resource consumption is reduced.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a method, device, equipment and storage medium for processing erasure read requests. Background Art

[0002] Distributed storage system is to store data in multiple independent devices. Traditional network storage system uses centralized storage server to store all data, and storage server becomes the bottleneck of system performance. Distributed network storage system adopts scalable system structure and uses multiple storage servers to share storage load.

[0003] Erasure coding (EC) is a data protection method that divides data into fragments, expands and encodes redundant data blocks, and stores them in different locations, such as disks, storage nodes, or other geographical locations. In an erasure coding system, the original data is divided into multiple fragments and encoded by adding redundant data blocks. These encoded data blocks are stored in multiple locations, such as different disks, storage nodes, or geographical locations. This technology describes the data by creating a mathematical function so that even if one or more data blocks are lost, the original data can be restored from the remaining data blocks.

[0004] In a storage system, erasure read requests usually account for the vast majority of the total number, so there is an urgent need for an optimization method for erasure read requests to improve the overall performance of the storage system. Summary of the invention

[0005] In view of this, the present application provides a method, device, equipment and storage medium for processing erasure read requests, which improves the overall performance of the storage system. The technical solution is as follows.

[0006] In a first aspect, a method for processing an erasure read request is provided, the method being executed by a client in a distributed storage system, the distributed storage system also including a server, the method comprising:

[0007] Obtaining a target read request; the target read request is an erasure read request; the target read request includes storage information of the data to be read;

[0008] According to the storage information in the data to be read and the data structure of the erasure pool in the server, obtaining each target shard corresponding to the data to be read in the server;

[0009] A read operation is generated for each target shard, and each read operation is sent to the server, so that the server reads the data in each target shard according to the read operation and pushes it to the memory specified by the client.

[0010] The method for processing an erasure read request provided by the present invention has the following advantages:

[0011] When the client in the distributed request obtains each read request sent by the business end, it can detect each read request. If the target read request is an erasure read request and contains storage information with read data, the client can determine which shards in the erasure pool the data to be read is related to based on the data structure of the erasure pool in its corresponding server, thereby obtaining each target shard related to the data to be read; the client then generates read operations for the target shards respectively, and then sends the read operation of each target shard to the server, so that the server can read the data of the target shard according to the read operation of each target shard, and push it to the memory specified by the client. After the client receives the data pushed by the server, it can return the data to the business end, thereby completing the erasure read operation. The above scheme splits the read operation of the entire stripe in the erasure pool into read operations on some target shards, which improves the operational efficiency of the read operation, avoids unnecessary disk reading processes, and reduces resource consumption.

[0012] In a possible implementation, sending each read operation to the server so that the server reads the data in each target shard according to the read operation and pushes the data to the memory specified by the client includes:

[0013] Combine individual read operations into an operation request;

[0014] The operation request is serialized and sent to the server through the message module of the client, so that the server reads the data in each target shard according to the read operation and directly pushes it to the memory specified by the client.

[0015] In a possible implementation, the storage information includes the offset and data length of the data to be read; the erasure pool stores an erasure stripe; the erasure stripe is composed of a plurality of storage blocks; the storage block is used to store erasure coded data fragments of a specified size;

[0016] The acquiring, according to the storage information in the data to be read and the data structure of the erasure pool in the server, each target shard corresponding to the data to be read in the server includes:

[0017] Determine whether the logic group corresponding to the data to be read is degraded; the logic group is used to indicate the fragmentation status in the server;

[0018] When the logical group corresponding to the data to be read is not degraded, reading the offset and data length of the data to be read;

[0019] Obtaining the size of storage blocks and the size of erasure stripes in the erasure pool in the server;

[0020] Determine, in each shard of the server, each target shard corresponding to the data to be read according to the offset and data length of the data to be read, the size of the storage block, and the size of the erasure stripe;

[0021] Generating a read operation for each target shard respectively includes:

[0022] Calculate the offset and data length of each target shard corresponding to the data to be read according to the offset and data length of the data to be read, the size of the storage block and the size of the erasure stripe;

[0023] For each target slice corresponding to the to-be-read data, the read operations corresponding to each target slice are respectively obtained according to the offset and data length corresponding to each target slice.

[0024] In a second aspect, a method for processing an erasure read request is provided, the method being executed by a server in a distributed storage system, the distributed storage system also including a client, the method comprising:

[0025] Receive each read operation sent by the client; the read operation is that the client obtains each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server and generates for each target shard;

[0026] According to each of the read operations, data is read from each of the target shards and pushed to the memory specified by the client.

[0027] The method for processing an erasure read request provided by the present invention has the following advantages:

[0028] When the client in the distributed request obtains each read request sent by the business end, it can detect each read request. If the target read request is an erasure read request and contains storage information with read data, the client can determine which shards in the erasure pool the data to be read is related to based on the data structure of the erasure pool in its corresponding server, thereby obtaining each target shard related to the data to be read; the client then generates read operations for the target shards respectively, and then sends the read operation of each target shard to the server, so that the server can read the data of the target shard according to the read operation of each target shard, and push it to the memory specified by the client. After the client receives the data pushed by the server, it can return the data to the business end, thereby completing the erasure read operation. The above scheme splits the read operation of the entire stripe in the erasure pool into read operations on some target shards, which improves the operational efficiency of the read operation, avoids unnecessary disk reading processes, and reduces resource consumption.

[0029] In a possible implementation, the storage information includes the offset and data length of the data to be read; the erasure pool stores an erasure stripe; the erasure stripe is composed of multiple storage blocks; and the storage block is used to store erasure coded data fragments of a specified size.

[0030] In a possible implementation, reading data from each target shard respectively according to each read operation includes:

[0031] According to each of the read operations, respectively obtain the offset and data length in each target shard;

[0032] Data is read from each target slice according to the offset and data length in each target slice.

[0033] In a third aspect, a device for processing erasure read requests is provided, the device being arranged at a client in a distributed storage system, the distributed storage system further comprising a server, the device comprising:

[0034] A read request acquisition module, used to acquire a target read request; the target read request is an erasure read request; the target read request includes storage information of the data to be read;

[0035] A shard acquisition module, configured to acquire each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server;

[0036] The read operation generation module is used to generate a read operation for each target shard, and send each read operation to the server, so that the server reads the data in each target shard according to the read operation and pushes it to the memory specified by the client.

[0037] The present invention provides an erasure read request processing device, which has the following advantages:

[0038] When the client in the distributed request obtains each read request sent by the business end, it can detect each read request. If the target read request is an erasure read request and contains storage information with read data, the client can determine which shards in the erasure pool the data to be read is related to based on the data structure of the erasure pool in its corresponding server, thereby obtaining each target shard related to the data to be read; the client then generates read operations for the target shards respectively, and then sends the read operation of each target shard to the server, so that the server can read the data of the target shard according to the read operation of each target shard, and push it to the memory specified by the client. After the client receives the data pushed by the server, it can return the data to the business end, thereby completing the erasure read operation. The above scheme splits the read operation of the entire stripe in the erasure pool into read operations on some target shards, which improves the operational efficiency of the read operation, avoids unnecessary disk reading processes, and reduces resource consumption.

[0039] In a possible implementation, the read operation generation module is further used to:

[0040] Combine individual read operations into an operation request;

[0041] The operation request is serialized and sent to the server through the message module of the client, so that the server reads the data in each target shard according to the read operation and directly pushes it to the memory specified by the client.

[0042] In a possible implementation, the storage information includes the offset and data length of the data to be read; the erasure pool stores an erasure stripe; the erasure stripe is composed of a plurality of storage blocks; the storage block is used to store erasure coded data fragments of a specified size;

[0043] The fragment acquisition module is further used for:

[0044] Determine whether the logic group corresponding to the data to be read is degraded; the logic group is used to indicate the fragmentation status in the server;

[0045] When the logical group corresponding to the data to be read is not degraded, reading the offset and data length of the data to be read;

[0046] Obtaining the size of storage blocks and the size of erasure stripes in the erasure pool in the server;

[0047] Determine, in each shard of the server, each target shard corresponding to the data to be read according to the offset and data length of the data to be read, the size of the storage block, and the size of the erasure stripe;

[0048] The read operation generating module is further used to: calculate the offset and data length of each target shard corresponding to the data to be read according to the offset and data length of the data to be read, the size of the storage block and the size of the erasure stripe;

[0049] For each target slice corresponding to the to-be-read data, the read operations corresponding to each target slice are respectively obtained according to the offset and data length corresponding to each target slice.

[0050] In a fourth aspect, a device for processing erasure read requests is provided, the device being arranged at a server end of a distributed storage system, the distributed storage system further comprising a client, the device comprising:

[0051] A read operation receiving module, used to receive each read operation sent by the client; the read operation is generated by the client acquiring each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server;

[0052] The data reading module is used to read data from each target shard and push the data to the memory specified by the client according to each reading operation.

[0053] The present invention provides an erasure read request processing device, which has the following advantages:

[0054] When the client in the distributed request obtains each read request sent by the business end, it can detect each read request. If the target read request is an erasure read request and contains storage information with read data, the client can determine which shards in the erasure pool the data to be read is related to based on the data structure of the erasure pool in its corresponding server, thereby obtaining each target shard related to the data to be read; the client then generates read operations for the target shards respectively, and then sends the read operation of each target shard to the server, so that the server can read the data of the target shard according to the read operation of each target shard, and push it to the memory specified by the client. After the client receives the data pushed by the server, it can return the data to the business end, thereby completing the erasure read operation. The above scheme splits the read operation of the entire stripe in the erasure pool into read operations on some target shards, which improves the operational efficiency of the read operation, avoids unnecessary disk reading processes, and reduces resource consumption.

[0055] In a possible implementation, the storage information includes the offset and data length of the data to be read; the erasure pool stores an erasure stripe; the erasure stripe is composed of multiple storage blocks; and the storage block is used to store erasure coded data fragments of a specified size.

[0056] In a possible implementation, the data reading module is further used to:

[0057] According to each of the read operations, respectively obtain the offset and data length in each target shard;

[0058] Data is read from each target slice according to the offset and data length in each target slice.

[0059] In a fifth aspect, a computer device is provided, comprising a processor and a storage medium, wherein the storage medium stores program instructions executable by the processor, and the processor executes the program instructions to implement the above-mentioned correction and erasure read request processing method.

[0060] In a sixth aspect, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned correction and erasure read request processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 is a system architecture diagram of a distributed storage system according to an exemplary embodiment;

[0063] Figure 2 is a method flow chart of a method for processing an erasure read request according to an exemplary embodiment;

[0064] Figure 3 is a method flow chart of a method for processing an erasure read request according to an exemplary embodiment;

[0065] Figure 4 is a method flow chart of a method for processing an erasure read request according to an exemplary embodiment;

[0066] Figure 5 A schematic diagram of a data structure of an erasure read request involved in an embodiment of the present application is shown;

[0067] Figure 6 A flowchart showing the steps of executing an erasure read request by a client in a numerical control separation scenario according to an embodiment of the present application is shown;

[0068] Figure 7 A flowchart showing the steps executed by a sub-object storage device on a server side for an erasure read request in a numerical control separation scenario according to an embodiment of the present application is shown.

[0069] Figure 8 A flowchart of steps executed by a main object storage device in a server of an erasure read request in a numerical control separation scenario involved in an embodiment of the present application is shown.

[0070] Fig. 9 This is a correction and erasure read request processing device provided by this embodiment.

[0071] Fig.10 This is a correction and erasure read request processing device provided by this embodiment.

[0072] Fig.11 It is a structural schematic diagram of a computer device provided by an optional embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0074] In modern storage systems, whether centralized or distributed, it is increasingly important to maximize the use of various physical resources. Therefore, in actual use, it is necessary to fully tap the extreme performance of physical resources while reducing resource waste within the system. Since read requests account for the vast majority of storage systems, the efficiency of the entire storage system can be significantly improved by reducing the time of read IO operations and improving their performance. In order to meet this demand, the present application proposes an optimization method for erasure code reading in CNC separation scenarios, thereby improving the read performance of the system.

[0075] Figure 1 FIG. 1 is a system architecture diagram of a distributed storage system according to an exemplary embodiment. Figure 1 As shown, the system includes a business end, a client end and a server end.

[0076] The client is used to accept requests from the business end and send the requests to the server. After receiving the results returned by the server, the client forwards the results to the business end, that is, the client can serve as a bridge between the server and the business end.

[0077] The server is responsible for the specific execution of the business end request, that is, it processes the request forwarded by the client after receiving it, returns the result to the client after processing, and forwards it to the business end after processing by the client.

[0078] It should be noted that the steps shown in the present application can be performed in a distributed storage system including, for example, a set of computer executable instructions, and that, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that shown here.

[0079] Based on this, Figure 2 is a flowchart of a method for processing an erasure read request according to an exemplary embodiment. The method comprises: Figure 1 The client in the distributed storage system shown in the figure is executed, and the distributed storage system also includes a server. The method includes:

[0080] Step S201, obtaining a target read request.

[0081] The target read request is an erasure read request; the target read request includes storage information of data to be read.

[0082] First, the system needs to obtain a target read request. This target read request refers to an erasure read request initiated by a user or application to the storage system, with the purpose of obtaining stored data fragments from the distributed storage system. The request contains the storage information of the data to be read, such as the location of the data in the system, the erasure pool to which it belongs, and other key information.

[0083] Step S202: acquiring each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server.

[0084] Next, the system will locate the target shards corresponding to the data to be read based on the storage information provided in the target read request and the erasure pool data structure on the server. The erasure pool data structure contains the storage locations of the shards after the original data is split. These shards may be distributed across multiple storage nodes, disks, or geographical locations. Through this process, the system can accurately identify the specific shards that need to be read.

[0085] Step S203, generating a read operation for each target shard respectively, and sending each read operation to the server, so that the server reads the data in each target shard according to the read operation and pushes it to the memory specified by the client.

[0086] After identifying each target shard, the system generates corresponding read operations for each target shard. These read operations are then sent to the server of the storage system. After receiving the read operation, the server executes the corresponding read operation to obtain the required data from each target shard. To further improve performance, the server pushes the read data directly to the memory specified by the client, and can store the read data directly in the memory specified by the client without passing through the CPU. In this way, the data can reach the client faster, reducing the delay in the intermediate process and reducing the CPU load.

[0087] Specifically, RDMA technology can be used to achieve direct push of read data. RDMA (Remote Direct Memory Access) is a mature technology that reduces the latency of server-side data processing by directly transferring data to the computer storage area without CPU involvement and without affecting the operating system. The advantages of this technology make modern mass storage systems widely support RDMA. The effective use of RDMA through CNC separation technology has promoted the innovation of hardware technology and the advancement of software system design.

[0088] In summary, after the client in the distributed request obtains each read request sent by the business end, each read request can be detected. If the target read request is an erasure read request and contains storage information with read data, the client can determine which shards in the erasure pool the data to be read is related to based on the data structure of the erasure pool in its corresponding server, thereby obtaining each target shard related to the data to be read; the client then generates read operations for the target shards respectively, and then sends the read operation of each target shard to the server, so that the server can read the data of the target shard according to the read operation of each target shard, and push it to the memory specified by the client. After the client receives the data pushed by the server, it can return the data to the business end, thereby completing the erasure read operation. The above scheme splits the read operation of the entire stripe in the erasure pool into read operations on some target shards, which improves the operational efficiency of the read operation, avoids unnecessary disk reading processes, and reduces resource consumption.

[0089] and Figure 2 correspond, Figure 3 The present invention is a method flow chart of a method for processing an erasure read request according to an exemplary embodiment. The method is executed by a server in a distributed storage system, and the distributed storage system also includes a client. The method includes:

[0090] Step S301, receiving each read operation sent by the client.

[0091] The read operation is that the client obtains each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server and generates for each target shard.

[0092] In the embodiment of the present application, the client locates each relevant target shard in the server according to the storage information of the data to be read (including the storage location of the data shard, the erasure pool structure, etc.). Each shard corresponds to a read operation, which includes the specific shard location and the data range to be read.

[0093] The client sends these generated read operations to the server of the storage system through the network. Due to the characteristics of the distributed storage system, these requests may be sent to different physical storage nodes respectively to ensure that the read operation of each shard can reach the server storing the shard.

[0094] After receiving these read operations, the server parses each operation request to determine which storage location to read the data from, ensuring that the server can clearly understand the client's needs and prepare for the next operation.

[0095] Step S302: According to each read operation, data is read from each target shard and pushed to the memory specified by the client.

[0096] According to the read operation received in step S301, the server starts to read data from each target shard. These shards may be distributed on different storage nodes, so the server may read data from multiple nodes in parallel to improve reading efficiency.

[0097] Once the server reads the data, it will not temporarily store the data in its own cache or intermediate storage layer, but will directly push the data to the memory address specified by the client.

[0098] When the data of all target shards are successfully pushed to the memory specified by the client, the entire data reading process is completed. The client can directly access the data in its memory for further processing or recovery.

[0099] In summary, after the client in the distributed request obtains each read request sent by the business end, each read request can be detected. If the target read request is an erasure read request and contains storage information with read data, the client can determine which shards in the erasure pool the data to be read is related to based on the data structure of the erasure pool in its corresponding server, thereby obtaining each target shard related to the data to be read; the client then generates read operations for the target shards respectively, and then sends the read operation of each target shard to the server, so that the server can read the data of the target shard according to the read operation of each target shard, and push it to the memory specified by the client. After the client receives the data pushed by the server, it can return the data to the business end, thereby completing the erasure read operation. The above scheme splits the read operation of the entire stripe in the erasure pool into read operations on some target shards, which improves the operational efficiency of the read operation, avoids unnecessary disk reading processes, and reduces resource consumption.

[0100] Combined with Figure 2 and Figure 3 , Figure 4 The present invention is a method flow chart of a method for processing an erasure read request according to an exemplary embodiment. The method is jointly executed by a server and a client in a distributed storage system, and the method includes:

[0101] Step S401: The client obtains a target read request.

[0102] The target read request is an erasure read request; the target read request includes storage information of the data to be read.

[0103] That is, the client receives a target read request sent by the service end. The target read request is an erasure read request, which means that the target read request involves reading data from a distributed storage system, and the data is protected by erasure coding technology. The target read request includes the storage information of the data to be read, such as the logical location, offset, and data length of the data. The storage information of the data to be read is the key to determining which data fragments the client needs to read from the server.

[0104] Step S402: The client obtains each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server.

[0105] In the embodiment of the present application, the storage information includes the offset and data length of the data to be read; the erasure pool stores an erasure stripe; the erasure stripe is composed of multiple storage blocks; and the storage block is used to store erasure coded data fragments of a specified size.

[0106] Specifically, the client determines whether the logical group corresponding to the data to be read is degraded; the logical group is used to indicate the sharding status in the server; when the logical group corresponding to the data to be read is not degraded, read the offset and data length of the data to be read; obtain the size of the storage block and the size of the erasure stripe in the erasure pool in the server; according to the offset and data length of the data to be read, the size of the storage block and the size of the erasure stripe, determine each target shard corresponding to the data to be read in each shard of the server.

[0107] The client first reads the offset and data length of the data to be read. The offset and data length of the data to be read are used to indicate the logical starting position and number of bytes required for the data that the client wants to obtain from the storage system. The client obtains the relevant information of the erasure stripe from the erasure pool of the server. An erasure stripe consists of multiple storage blocks, each of which is used to store erasure coded data fragments of a certain size. The data structure of the erasure pool contains the size information of the stripe and storage block.

[0108] Before executing the above steps, the client also needs to check whether the logical group corresponding to the data to be read is degraded.

[0109] In a distributed storage system, the degradation of a logical group (PG, Placement Group) means that the logical group has lost some redundant data, resulting in it being unable to fully provide the originally set redundant protection level. PG is a logical unit used to organize and manage data in a distributed storage system. It usually maps a group of data blocks or objects to a specific set of storage nodes according to a certain rule (such as hash). PG degradation means that in the PG, one or more data blocks or redundant blocks (such as erasure coded data blocks or replicas) are unavailable or lost. After the PG is downgraded, although the system may still be able to provide data access (for example, by reading the remaining available blocks), the data redundancy is reduced and the system loses the ability to cope with further failures.

[0110] When the logical group is not degraded, the client will calculate the specific target shards corresponding to the data to be read in each shard on the server based on the offset and data length of the data to be read, the size of the storage block, the size of the erasure stripe, etc. Through these calculations, the client can clearly know which shards contain the data fragments to be read.

[0111] Optionally, when a logical group is degraded, the server uses the remaining available data blocks and redundant blocks to recalculate and reconstruct the lost data blocks through an erasure coding algorithm.

[0112] Step S403: The client generates a read operation for each target shard.

[0113] Specifically, the client calculates the offset and data length of each target shard corresponding to the data to be read according to the offset and data length of the data to be read, the size of the storage block and the size of the erasure stripe; for each target shard corresponding to the data to be read, the client obtains the read operation corresponding to each target shard according to the offset and data length corresponding to each target shard.

[0114] Specifically, the client calculates a start stripe index and an end stripe index according to the offset of the data to be read; the start stripe index is used to indicate the start stripe of the data to be read; the end stripe index is used to indicate the end stripe of the data to be read;

[0115] The client then calculates the offset in the starting stripe and the offset in the ending stripe based on the offset of the data to be read;

[0116] For each strip from the start strip to the end strip, perform the following steps:

[0117] The physical storage location of the data of each stripe is calculated; the target shards involved in the physical storage location are determined, so as to determine the data to be read in each target shard.

[0118] Since the data in each stripe is scattered on each device of the distributed storage system, the above solution can select the target shards corresponding to the scattered data in each stripe, so as to generate corresponding read operations according to the target shards. Each read operation contains the location of the target shard and the data range to be read, and these operations will be sent to the server to obtain the actual data.

[0119] Please refer to Figure 5 , which shows a data structure diagram of an erasure read request involved in an embodiment of the present application. Figure 5 As shown in the figure, the erasure read operation in the erasure coding (EC) system involves reading data blocks from multiple shards. The figure describes a 4+2 erasure code configuration, which means there are 4 data shards (Shard0, Shard1, Shard2, Shard3) and 2 redundant shards (erasure blocks for erasure code verification). Figure 5 The "4+2" in the figure means there are 4 data blocks (Shard0 to Shard3) and 2 erasure blocks (used for redundancy and erasure code verification). The size of each data block and erasure block is 8k. Figure 5 There are five different erasure read requests listed in the figure. Each request reads different data ranges and involves different shards.

[0120] Type 1 reading (erasure reading 0k~8k):

[0121] Only read data from Shard0, and the reading range is 0k to 8k, that is, read the complete data block of Shard0.

[0122] The second type of reading (erasure reading 0k~16k):

[0123] The data read this time ranges from 0k to 16k, involving Shard0 and Shard1. Data blocks of 0k to 8k are read from Shard0, and data blocks of 0k to 8k are read from Shard1.

[0124] The third type of reading (erasure reading 16k~8k):

[0125] Only read data from Shard2, the reading range is 16k~24k, that is, read data from 0k~8k blocks of Shard2.

[0126] The fourth type of reading (erasure reading 8k~8k, 24k~8k):

[0127] This read is divided into two parts:

[0128] The first part reads data blocks 8k to 16k from Shard1. The second part reads data from blocks 24k to 32k from Shard3.

[0129] The fifth type of reading (erasure reading 0k~56k):

[0130] This time, the data read ranges from 0k to 56k, involving all data shards (Shard0, Shard1, Shard2, Shard3). Each shard reads a complete 8k data block.

[0131] Therefore, if the received read requests are concentrated on one or some specific shards (i.e., target shards), you only need to build read requests for these shards. In other words, the above steps split the read operation of the entire stripe in the erasure pool into read operations on some target shards, which improves the efficiency of the read operation, avoids unnecessary disk reading, and reduces resource consumption.

[0132] Step S404: The client combines various read operations into an operation request.

[0133] The client merges multiple read operations generated for each target shard into one operation request. The purpose of this step is to reduce the communication overhead with the server and optimize network transmission. By merging, the client can convey all read operations in one request, avoiding the delay and overhead caused by multiple requests.

[0134] Step S405: the client serializes the operation request and sends it to the server through the message module of the client.

[0135] The client serializes the merged operation request. Serialization is the process of converting the operation request into a standard format for transmission over the network. The client then sends the serialized operation request to the server through its message module. This step ensures that the server can correctly receive and understand the client's read request.

[0136] Step S406: the server receives each read operation sent by the client.

[0137] After receiving the serialization operation request sent by the client, the server parses the request to obtain the specific read operation. These read operations indicate which shards the server needs to read data from, as well as the offset and length of the data. By parsing this information, the server can prepare to perform data read operations.

[0138] Step S407: The server obtains the offset and data length in each target shard according to each read operation.

[0139] The server reads data from each target shard according to the offset and data length in each target shard, and directly pushes it to the memory specified by the client.

[0140] Specifically, the server locates the specific location of the target shard according to the instructions of each read operation and reads the required data. The data will be extracted according to the offset and length specified in the read operation. After the read operation is completed, the server will be ready to push the read data directly to the specified memory address of the client. For example, this can be done through efficient data transmission technologies such as RDMA to reduce latency and improve performance.

[0141] Please refer to Figure 6 , which shows a flowchart of the steps of executing an erasure read request by a client in a CNC separation scenario according to an embodiment of the present application. Please refer to Figure 7 , which shows a flowchart of the steps executed by the sub-object storage device in the server side of an erasure read request in a numerical control separation scenario according to an embodiment of the present application. Please refer to Figure 8 , which shows a flowchart of the steps executed by the main object storage device in the server of an erasure read request in a numerical control separation scenario involved in an embodiment of the present application. Figure 6 , Figure 7 as well as Figure 8 , the optimization processing flow of erasure read involved in the embodiment of the present application is described:

[0142] 1. Startup and initialization of the client and server:

[0143] Startup and initialization: When the client and server are started, they will load the configuration files related to CNC separation (separation of data and control planes) and initialize the CNC separation engine. This is to ensure that the system can perform data and control separation according to the configured strategy immediately after startup.

[0144] 2. The business end initiates a read request to the client:

[0145] The business end sends a read request to the client. This request contains the offset (offset) and data length (length) of the data to be read, which are used to specify the data range to be read.

[0146] 3. Processing after the client receives the request:

[0147] 3.1 Convert to Op (Operation) request:

[0148] The client converts the business end's requests into Op request format to facilitate subsequent unified processing. This standardized request format allows different types of requests to be processed by the same set of processing logic.

[0149] 3.2 Injection CNC separation engine:

[0150] If the NC separation function is enabled, the Op request will be injected into the NC separation engine for processing. The NC separation engine is responsible for separating the data plane operations from the control plane operations to improve the performance and scalability of the system.

[0151] 3.3 Detect whether the Op request is an erasure read request and process it:

[0152] When the CNC separation engine detects that the Op request is an erasure read request, it first determines whether the current PG (PlacementGroup) is downgraded. A downgraded PG may affect data reading, so the erasure read optimization process will only be entered when the PG is not downgraded.

[0153] Take the offset and length in the read request, calculate the shard involved in this read based on the chunk size and erasure stripe size in the erasure pool, and then calculate the offset and length of each shard.

[0154] Generate separate Read operations for each calculated shard. These operations are read instructions specific to each shard.

[0155] 3.4 Reorganization Read Op:

[0156] Reorganize the Read operations generated for each shard into a new Read Op. This is to be sent to the cluster end through the message module for unified processing.

[0157] 4. Serialization and sending of read Op:

[0158] The reorganized Op is serialized and sent to the cluster through the client's message module. This process ensures that the Op can be transmitted in the network and can be correctly understood and processed by the cluster.

[0159] 5. Receiving and processing the cluster-side master OSD (Object Storage Device):

[0160] After receiving the message, the master OSD on the cluster side deserializes it into Op. Op is processed by the PG queue, and the required shard information is parsed, and an ECSubRead operation is generated for each shard.

[0161] 6. Send Op message to child OSD:

[0162] The master OSD sends ECSubRead operations to the child OSDs corresponding to each shard. These operations instruct the child OSDs to read the corresponding data shards.

[0163] 7. Receiving and processing of sub-OSD:

[0164] 7.1 Parsing read operations:

[0165] After receiving the ECSubRead operation, the child OSD parses the Offset and Length of the data to be read this time.

[0166] 7.2 Reading data from disk:

[0167] The child OSD reads data from the disk based on the parsed Offset and Length.

[0168] 7.3 Push data to the client:

[0169] The read data is pushed directly to the memory specified by the client through RDMA. RDMA (Remote Direct Memory Access) allows data to be directly transmitted between the server and the client without the need for CPU processing, greatly improving the efficiency of data transmission.

[0170] 7.4 Send ECSubReadReply:

[0171] After the sub-OSD completes data reading, it sends an ECSubReadReply message to the main OSD to inform the completion status of the data reading operation.

[0172] 8. The primary OSD receives the reply from the child OSD:

[0173] After receiving the replies from all slave OSDs, the primary OSD confirms that the read operations on all data shards have been completed.

[0174] 9. The primary OSD returns a reply to the client:

[0175] The primary OSD sends a final Reply to the client, informing it that the data read request has been successfully completed.

[0176] 10. The client completes the operation:

[0177] After receiving the Reply message, the client returns it to the business end, indicating that the read operation has been successfully completed. In summary, in a distributed storage system, a large number of small IO read requests will become a key bottleneck affecting cluster performance, especially when the read request needs to be distributed to multiple storage nodes. This situation is particularly obvious. The optimization method proposed in this application is to optimize the erasure code read request by numerical control separation (separate processing of data plane operations and control plane operations). When the client detects that the read request is an erasure code read, the corresponding optimization processing logic will be triggered. Specifically, the client splits the original read request into Chunk blocks according to the offset and length of the request. If the split read requests are concentrated on certain specific shards, the client only builds read requests for these specific Shards, and no longer sends redundant requests to other Shards. After receiving a read request for a specific Shard, the server only needs to read data from the Shard specified by the request, and no redundant operations are required. After the reading is completed, the server feeds back the result to the client through a Reply message. After receiving the Reply message from the server, the client returns the read result to the business end. Through this optimization method, read operations can be concentrated on specific Shards, reducing the number of data transmissions in the cluster network, thereby reducing the amplification effect of read operations and reducing the read latency of the storage system. At the same time, this optimized splitting method can effectively save cluster network bandwidth and improve the performance of the entire cluster system. In addition, in the embodiment of the present application, the splitting of control flow and data flow is achieved by introducing RDMA (remote direct memory access) technology. RDMA allows data to be transmitted directly between memories, avoiding CPU intervention, thereby reducing resource consumption and improving system performance.

[0178] In summary, after the client in the distributed request obtains each read request sent by the business end, each read request can be detected. If the target read request is an erasure read request and contains storage information with read data, the client can determine which shards in the erasure pool the data to be read is related to based on the data structure of the erasure pool in its corresponding server, thereby obtaining each target shard related to the data to be read; the client then generates read operations for the target shards respectively, and then sends the read operation of each target shard to the server, so that the server can read the data of the target shard according to the read operation of each target shard, and push it to the memory specified by the client. After the client receives the data pushed by the server, it can return the data to the business end, thereby completing the erasure read operation. The above scheme splits the read operation of the entire stripe in the erasure pool into read operations on some target shards, which improves the operational efficiency of the read operation, avoids unnecessary disk reading processes, and reduces resource consumption.

[0179] In this embodiment, a correction and erasure read request processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0180] This embodiment provides a correction and erasure read request processing device, such as Fig. 9 As shown, the device is arranged at a client in a distributed storage system, and the distributed storage system also includes a server, and the device includes:

[0181] The read request acquisition module 901 is used to acquire a target read request; the target read request is an erasure read request; the target read request includes storage information of the data to be read;

[0182] A shard acquisition module 902, configured to acquire each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server;

[0183] The read operation generation module 903 is used to generate a read operation for each target shard, and send each read operation to the server, so that the server reads the data in each target shard according to the read operation and pushes it to the memory specified by the client.

[0184] In a possible implementation, the read operation generation module is further used to:

[0185] Combine individual read operations into an operation request;

[0186] The operation request is serialized and sent to the server through the message module of the client, so that the server reads the data in each target shard according to the read operation and directly pushes it to the memory specified by the client.

[0187] In a possible implementation, the storage information includes the offset and data length of the data to be read; the erasure pool stores an erasure stripe; the erasure stripe is composed of a plurality of storage blocks; the storage block is used to store erasure coded data fragments of a specified size;

[0188] The fragment acquisition module is further used for:

[0189] Determine whether the logic group corresponding to the data to be read is degraded; the logic group is used to indicate the fragmentation status in the server;

[0190] When the logical group corresponding to the data to be read is not degraded, reading the offset and data length of the data to be read;

[0191] Obtaining the size of storage blocks and the size of erasure stripes in the erasure pool in the server;

[0192] Determine, in each shard of the server, each target shard corresponding to the data to be read according to the offset and data length of the data to be read, the size of the storage block, and the size of the erasure stripe;

[0193] The read operation generating module is further used to: calculate the offset and data length of each target shard corresponding to the data to be read according to the offset and data length of the data to be read, the size of the storage block and the size of the erasure stripe;

[0194] For each target slice corresponding to the to-be-read data, the read operations corresponding to each target slice are respectively obtained according to the offset and data length corresponding to each target slice.

[0195] This embodiment also provides another erasure read request processing device, such as Fig.10 As shown, the device is arranged at a server end of a distributed storage system, and the device comprises:

[0196] The read operation receiving module 1001 is used to receive each read operation sent by the client; the read operation is that the client obtains each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server and generates for each target shard;

[0197] The data reading module 1002 is used to read data from each target shard according to each reading operation and push the data to the memory specified by the client.

[0198] In a possible implementation, the storage information includes the offset and data length of the data to be read; the erasure pool stores an erasure stripe; the erasure stripe is composed of multiple storage blocks; and the storage block is used to store erasure coded data fragments of a specified size.

[0199] In a possible implementation, the data reading module is further used to:

[0200] According to each of the read operations, respectively obtain the offset and data length in each target shard;

[0201] Data is read from each target slice according to the offset and data length in each target slice.

[0202] The erasure read request processing device in this embodiment is presented in the form of a functional module, where the module refers to an application specific integrated circuit (ASIC), a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0203] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0204] The present invention provides an erasure read request processing device, which has the following advantages:

[0205] When the client in the distributed request obtains each read request sent by the business end, it can detect each read request. If the target read request is an erasure read request and contains storage information with read data, the client can determine which shards in the erasure pool the data to be read is related to based on the data structure of the erasure pool in its corresponding server, thereby obtaining each target shard related to the data to be read; the client then generates read operations for the target shards respectively, and then sends the read operation of each target shard to the server, so that the server can read the data of the target shard according to the read operation of each target shard, and push it to the memory specified by the client. After the client receives the data pushed by the server, it can return the data to the business end, thereby completing the erasure read operation. The above scheme splits the read operation of the entire stripe in the erasure pool into read operations on some target shards, which improves the operational efficiency of the read operation, avoids unnecessary disk reading processes, and reduces resource consumption.

[0206] An embodiment of the present invention further provides a computer device having the above-mentioned erasure read request processing apparatus.

[0207] See also Fig.11 , Fig.11 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.11As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.11 A processor 10 is taken as an example.

[0208] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0209] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0210] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0211] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0212] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0213] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0214] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0215] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0216] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for processing an erasure read request, characterized in that: The method is executed by a client in a distributed storage system, wherein the distributed storage system also includes a server, and the method includes: Obtaining a target read request; the target read request is an erasure read request; the target read request includes storage information of the data to be read; According to the storage information in the data to be read and the data structure of the erasure pool in the server, obtaining each target shard corresponding to the data to be read in the server; A read operation is generated for each target shard, and each read operation is sent to the server, so that the server reads the data in each target shard according to the read operation and pushes it to the memory specified by the client.

2. The method according to claim 1, characterized in that The sending of each read operation to the server, so that the server reads the data in each target shard according to the read operation and pushes the data to the memory specified by the client, includes: Combine individual read operations into an operation request; The operation request is serialized and sent to the server through the message module of the client, so that the server reads the data in each target shard according to the read operation and directly pushes it to the memory specified by the client.

3. The method according to claim 1 or 2, characterized in that: The storage information includes the offset and data length of the data to be read; the erasure pool stores an erasure stripe; the erasure stripe is composed of a plurality of storage blocks; the storage block is used to store erasure coded data fragments of a specified size; The acquiring, according to the storage information in the data to be read and the data structure of the erasure pool in the server, each target shard corresponding to the data to be read in the server includes: Determine whether the logic group corresponding to the data to be read is degraded; the logic group is used to indicate the fragmentation status in the server; When the logical group corresponding to the data to be read is not degraded, reading the offset and data length of the data to be read; Obtaining the size of storage blocks and the size of erasure stripes in the erasure pool in the server; Determine, in each shard of the server, each target shard corresponding to the data to be read according to the offset and data length of the data to be read, the size of the storage block, and the size of the erasure stripe; Generating a read operation for each target shard respectively includes: Calculate the offset and data length of each target shard corresponding to the data to be read according to the offset and data length of the data to be read, the size of the storage block and the size of the erasure stripe; For each target slice corresponding to the to-be-read data, the read operations corresponding to each target slice are respectively obtained according to the offset and data length corresponding to each target slice.

4. A method for processing an erasure read request, characterized in that: The method is executed by a server in a distributed storage system, wherein the distributed storage system also includes a client, and the method includes: Receive each read operation sent by the client; the read operation is that the client obtains each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server and generates for each target shard; According to each of the read operations, data is read from each of the target shards and pushed to the memory specified by the client.

5. The method according to claim 4, characterized in that The storage information includes the offset and data length of the data to be read; the erasure pool stores an erasure stripe; the erasure stripe is composed of multiple storage blocks; and the storage block is used to store erasure coded data fragments of a specified size.

6. The method according to claim 5, characterized in that The step of reading data from each target shard respectively according to each read operation includes: According to each of the read operations, respectively obtain the offset and data length in each target shard; Data is read from each target slice according to the offset and data length in each target slice.

7. A device for processing erasure read requests, characterized in that: The device is arranged at a client in a distributed storage system, the distributed storage system also includes a server, and the device includes: A read request acquisition module, used to acquire a target read request; the target read request is an erasure read request; the target read request includes storage information of the data to be read; A shard acquisition module, configured to acquire each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server; The read operation generation module is used to generate a read operation for each target shard, and send each read operation to the server, so that the server reads the data in each target shard according to the read operation and pushes it to the memory specified by the client.

8. A device for processing erasure read requests, characterized in that: The device is arranged at a server end of a distributed storage system, the distributed storage system further comprising a client, and the device comprises: A read operation receiving module, used to receive each read operation sent by the client; the read operation is generated by the client acquiring each target shard corresponding to the data to be read in the server according to the storage information in the data to be read and the data structure of the erasure pool in the server; The data reading module is used to read data from each target shard and push the data to the memory specified by the client according to each reading operation.

9. A computer device, characterized in that: The computer device includes a processor and a storage medium, the storage medium stores program instructions executable by the processor, and the processor executes the program instructions to implement the erasure read request processing method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the erasure read request processing method according to any one of claims 1 to 6.

Images