A striping data packet processing method, a server, a device and a medium

Abstract

The application relates to the technical field of striping data processing, and discloses a striping data packet processing method, a server, equipment and a medium, which can use a first request packet to screen a matched target request packet, combine the first request packet and the target request packet to produce a first combined packet, and when the current combined packet is a non-striping alignment request packet, continue to screen and combine the matched non-striping alignment request packet in a second queue on the basis of the current combined packet, and when the latest obtained combined packet is a striping alignment request packet, perform a corresponding write operation on the striping data according to the latest obtained combined packet, or after the second queue is traversed, perform a corresponding write operation on the striping data according to the combined packet after the second queue is traversed, so that the number of read operations in the write operation caused by multiple non-striping alignment request packets is further reduced, the read operation in the write operation caused by the non-striping alignment request packet is even avoided, the disk operation efficiency is further improved, and the disk performance is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of striped data processing technology, specifically to a method, server, device, and medium for processing striped data packets. Background Technology

[0002] With the development of science and technology, distributed storage technology continues to improve.

[0003] Currently, in distributed storage erasure scenarios, the server can divide the continuous data that the client wants to store into multiple data blocks of the same size, and generate one or more check blocks based on these data blocks. These data blocks and check blocks are then stored together. The multiple data blocks and check blocks constitute striped data.

[0004] Specifically, the server can receive request packets from the client to perform write operations on striped data, and perform corresponding write operations on the striped data according to the request packets.

[0005] However, during the write operation on striped data, there will be reads while writing the striped data, which will result in low hard drive efficiency and affect write performance. Summary of the Invention

[0006] In view of this, the present invention provides a method, server, device and medium for processing striped data packets, in order to solve the problem that during the write operation of striped data, there is a read-while-write operation of striped data, which leads to low hard disk operating efficiency and affects write performance.

[0007] In a first aspect, the present invention provides a packet processing method for striped data, applied to a server; the server includes a first queue and a second queue, each of the first queue and the second queue including at least one request packet from a client, each request packet being used to perform a write operation on the striped data stored in the server; the method includes:

[0008] Retrieve the first request packet from the first queue;

[0009] If the first request packet is determined to be a non-striped aligned request packet, one or more target request packets are selected by traversing each non-striped aligned request packet in the second queue based on the first request packet; wherein the first selected target request packet matches the first request packet.

[0010] The first request packet is merged with the first target request packet to generate a first combined packet;

[0011] Determine whether the first combined packet is a stripe alignment request packet. If so, end the filtering operation and perform the corresponding write operation on the striped data according to the first combined packet.

[0012] In an optional implementation, after determining whether the first combined packet is a stripe alignment request packet, the method further includes:

[0013] If the first combined packet is not a stripe-aligned request packet, then continue to filter the target request packets that match the first combined packet, and merge them with the first combined packet to generate a second combined packet. Continue to determine whether the second combined packet is a stripe-aligned request packet until the latest combined packet is a stripe-aligned request packet, or until every non-stripe-aligned request packet in the second queue has been traversed.

[0014] Based on the latest obtained combination packet or the combination packet after traversing the second queue, perform the corresponding write operation on the striped data.

[0015] In one optional implementation, the first request packet includes first data to be written, a first offset, and a first data length; the step of traversing each non-striped aligned request packet in the second queue based on the first request packet and filtering out one or more target request packets includes:

[0016] The non-striped aligned request packets traversed in the second queue are identified as the second request packets; the second request packets include the second data to be written, the second offset, and the second data length;

[0017] Based on the first offset and the second offset, and based on at least one of the first data length and the second data length, determine whether the second request packet matches the first request packet;

[0018] If so, the second request packet is determined to be the target request packet that matches the first request packet;

[0019] If not, continue traversing the next non-striped aligned write request packet in the second queue until the target request packet that matches the first request packet is identified.

[0020] In one optional implementation, determining whether the second request packet matches the first request packet based on the first offset and the second offset, and based on at least one of the first data length and the second data length, includes:

[0021] If it is determined that the first estimated offset is equal to the second offset, or the second estimated offset is equal to the first offset, then the second request packet is determined to match the first request packet; wherein, the first estimated offset is the sum of the first offset and the first data length, and the second estimated offset is the sum of the second offset and the second data length;

[0022] If it is determined that the first estimated offset is not equal to the second offset, and the second estimated offset is not equal to the first offset, then it is determined that the second request packet does not match the first request packet.

[0023] In an optional implementation, when the second request packet is the first target request packet, the step of merging the first request packet and the first target request packet to generate a first combined packet includes:

[0024] The first data to be written is merged into the second data to be written in the second request packet to generate a third request packet;

[0025] Based on the first data length and the second data length, and based on one of the first offset and the second offset, generate a merge offset and a merge data length;

[0026] The offset in the third request packet is adjusted from the second offset to the merged offset, and the data length in the third request packet is adjusted from the second data length to the merged data length to generate the first combined packet.

[0027] In one optional implementation, determining whether the first combined packet is a stripe alignment request packet includes:

[0028] If both the merge offset and the merged data length are divisible by the preset minimum storage unit value, then the first combined packet is determined to be a stripe alignment request packet.

[0029] If neither the merge offset nor the merged data length can be divided by the minimum storage unit value, then the first combined packet is determined to be a non-strip alignment request packet.

[0030] In one optional implementation, each request packet in the second queue originates from the first queue; before retrieving the first request packet from the first queue, the method further includes:

[0031] Retrieve the fourth request packet from the first queue;

[0032] If the fourth request packet is a stripe alignment request packet, then the fourth request packet is pushed to the second queue;

[0033] If the fourth request packet is a non-strip-aligned request packet and there is no matching request packet in the second queue, then the fourth request packet is pushed to the second queue.

[0034] In an alternative implementation, after retrieving the first request packet from the first queue, the method further includes:

[0035] The first moment is determined as the moment when the first request packet arrives at the first queue.

[0036] After the first combined packet is generated, a second time is determined; the second time is the time when the first target request packet arrives at the first queue.

[0037] Determine the earlier time from the first time and the second time;

[0038] The step of performing a corresponding write operation on the striped data according to the first combined packet includes:

[0039] When the time elapsed since the earlier time exceeds a preset time threshold, a corresponding write operation is performed on the striped data according to the first combined packet.

[0040] In an alternative implementation, after performing a corresponding write operation on the striped data according to the first combined package, the method further includes:

[0041] Based on the first request packet, generate a corresponding first reply message;

[0042] Based on the first target request packet, generate a corresponding second reply message;

[0043] The first reply message and the second reply message are returned to the client.

[0044] Secondly, the present invention provides a server, comprising: a first queue and a second queue; both the first queue and the queues include at least one request packet from a client, each request packet being used to perform a write operation on striped data stored in the server; the server further includes: a first acquisition unit, a first filtering unit, a first merging unit, a first judgment unit, a termination unit, and a first execution unit; wherein:

[0045] The first acquisition unit is used to acquire a first request packet from the first queue;

[0046] The first filtering unit is configured to, when determining that the first request packet is a non-strip-aligned request packet, traverse each non-strip-aligned request packet in the second queue according to the first request packet and filter out one or more target request packets; wherein the first target request packet filtered out matches the first request packet;

[0047] The first merging unit is used to merge the first request packet with the first target request packet to generate a first combined packet;

[0048] The first determining unit is used to determine whether the first combined packet is a stripe alignment request packet; if so, the ending unit is executed.

[0049] The termination unit is used to terminate the filtering operation;

[0050] The first execution unit is configured to perform a corresponding write operation on the striped data according to the first combined package.

[0051] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the packet processing method for striped data described in the first aspect or any corresponding embodiment thereof.

[0052] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the packet processing method for striped data described in the first aspect or any corresponding embodiment thereof.

[0053] The packet processing method, server, device, and medium for striped data proposed in this invention can use a first request packet to filter out matching target request packets, merge the first request packet and the target request packet to produce a first combined packet, and when the current combined packet is a non-striped aligned request packet, it can continue to try to filter and merge matching non-striped aligned request packets in a second queue based on the current combined packet, and when the latest combined packet is a striped aligned request packet, it can perform corresponding write operations on the striped data according to the latest combined packet, or after traversing the second queue, it can perform corresponding write operations on the striped data according to the combined packets after traversing the second queue, further reducing the number of write reads caused by multiple non-striped aligned request packets, or even avoiding write reads caused by non-striped aligned request packets, further improving disk operating efficiency and ensuring disk performance. Attached Figure Description

[0054] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0055] Figure 1 This is one of the flowcharts illustrating a packet processing method for striped data according to an embodiment of the present invention;

[0056] Figure 2 This is a second schematic flowchart of a packet processing method for striped data according to an embodiment of the present invention;

[0057] Figure 3 This is a third flowchart illustrating the packet processing method for striped data according to an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the server structure according to an embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] like Figure 1 As shown, this invention proposes a first method for processing striped data packets, applied to a server. The server includes a first queue and a second queue, each containing at least one request packet from a client. Each request packet is used to perform a write operation on the striped data stored on the server. The method may include the following steps:

[0062] S101. Obtain the first request packet from the first queue.

[0063] It should be noted that the first and second queues can be used to store different request packets from clients within a certain time period. Specifically, all request packets in the first and second queues can be different from each other.

[0064] Specifically, both the first and second queues can include stripe alignment request packets and non-stripe alignment request packets.

[0065] Among them, the striped alignment request packet is a request packet used to perform striped aligned write operations on striped data (there is no read-in-write), and the non-striped alignment request packet is a request packet used to perform non-striped aligned write operations on striped data (there is read-in-write).

[0066] Optionally, the server can receive consecutive request packets from the client within a certain time period for the same striped data, and place the request packets with earlier times into a second queue for storage, and the request packets with later times into a first queue for storage.

[0067] Optionally, after receiving the aforementioned continuous request packets, the server may randomly place some of the request packets into the second queue for storage, and place the remaining request packets into the first queue for storage.

[0068] Optionally, the server can, upon receiving any request packet from the client for a specific striped data, place it into a first queue, and then move it to a second queue later when certain conditions are met. In this case, all request packets in the first queue can originate from the client, and all request packets in the second queue can originate from the first queue.

[0069] Optionally, the request packets in the first and second queues can be arranged according to the time of receipt or randomly.

[0070] Specifically, after receiving a request packet from a client, the server can parse the packet to determine if it's for a write operation. If so, it can place the packet in either the first or second queue and mark it as parsed. In essence, the server can use the presence of a parsed flag to determine whether to add a request packet to a queue, avoiding duplicate entries.

[0071] Specifically, the server can include multiple disk arrays and has corresponding multiple object storage device (OSD) services. After receiving a request packet from a client, the server can determine the corresponding OSD service for that request packet, send the request packet to that OSD service, and then the OSD service will execute subsequent processes based on the request packet.

[0072] The first request packet can be a request packet in the first queue.

[0073] Specifically, the present invention can randomly obtain a request packet from the first queue and use it as the first request packet.

[0074] Specifically, the present invention can iterate through each request packet in the first queue and use the iterated request packet as the first request packet.

[0075] S102. If it is determined that the first request packet is a non-striped aligned request packet, traverse each non-striped aligned request packet in the second queue according to the first request packet, and filter out one or more target request packets. Among them, the first selected target request packet matches the first request packet.

[0076] Specifically, the present invention can parse the request packet, obtain the packet data in the request packet, and determine whether the request packet is a non-strip-aligned request packet based on the packet data.

[0077] Specifically, when the first request packet is a non-strip-aligned request packet, the present invention can start traversing each non-strip-aligned request packet in the second queue based on the first request packet.

[0078] Specifically, this invention can sequentially filter out one or more target request packets during the traversal of each non-striped aligned request packet in the second queue. It is understood that the filtered target request packets belong to the non-striped aligned request packet category.

[0079] Specifically, when traversing the second queue, the present invention can first filter out the target request packet that matches the first request packet.

[0080] It should be noted that the present invention can determine whether the data to be written into the striped data by the two request packets are continuous data at their respective data positions in the striped data. If they are, the two request packets can be merged, i.e., they match. If not, they cannot be merged, i.e., they do not match.

[0081] S103. Merge the first request packet with the first target request packet to generate the first combined packet.

[0082] Specifically, after traversing to a target request packet that matches the first request packet, the present invention can merge the first request packet with the target request packet to generate a first combined packet, determine whether the first combined packet is a stripe alignment request packet, and if so, end the filtering and traversal operations.

[0083] Specifically, the present invention can generate a first combined packet by merging the packet data of the first request packet and the first target request packet, and store it in a second queue.

[0084] S104. Determine whether the first combined packet is a stripe alignment request packet. If so, proceed to step S106.

[0085] Specifically, the present invention can perform subsequent steps if it is determined that the first combined packet is a stripe alignment request packet.

[0086] S105, End the filtering operation.

[0087] Specifically, the present invention can terminate the filtering operation and the traversal operation in the second queue when it is determined that the first combined packet is a stripe alignment request packet.

[0088] It should be noted that, in the case that the first request packet is determined to be a stripe alignment request packet, the present invention can directly perform the corresponding write operation on the striped data based on the first request packet.

[0089] S106. Based on the first combination packet, perform the corresponding write operation on the striped data.

[0090] Specifically, the present invention can terminate the filtering operation and perform a corresponding write operation on the striped data according to the first combined packet, thereby updating the striped data accordingly. Since the first combined packet is a stripe alignment request packet, the corresponding write operation performed is a stripe alignment write operation.

[0091] The packet processing method for striped data proposed in this invention can obtain a first request packet from a first queue. If the first request packet is determined to be a non-striped aligned request packet, a second queue is traversed to find a matching non-striped aligned request packet, i.e., a target request packet. Both the first request packet and the target request packet can be used to perform non-striped aligned write operations on the same striped data. This invention can combine the first request packet and the target request packet to generate a first combined packet. If the first combined packet is determined to be a striped aligned request packet, the corresponding striped aligned write operation is performed on the striped data based on the first combined packet. This invention, through operations such as obtaining, filtering, and merging request packets, can effectively avoid read-in-write operations that occur during write operations using the first request packet and the target request packet, thereby improving disk operating efficiency and ensuring disk performance.

[0092] based on Figure 1 ,like Figure 2 As shown, this invention proposes a second method for packet processing of striped data. After executing step S104, if the determination result is negative, this method may further include:

[0093] S201. Continue filtering for target request packets that match the first combination packet.

[0094] Specifically, if the first combined packet is not a stripe-aligned request packet, then continue filtering for target request packets that match the first combined packet.

[0095] S202. Merge the first combined packet with the matching target request packet to generate the second combined packet.

[0096] Specifically, when the first combined packet is determined to be a non-strip aligned request packet, the present invention can continue to traverse the second queue from the currently traversed position, filter out target request packets that match the first combined packet, and merge them with the first combined packet to generate a second combined packet.

[0097] S203. Continue to determine whether the second combined packet is a stripe alignment request packet until the latest obtained combined packet is a stripe alignment request packet, or until every non-stripe alignment request packet in the second queue has been traversed.

[0098] Specifically, this invention can determine whether the second combined packet is a stripe alignment request packet. If so, the filtering operation can end. If not, it can continue to traverse the second queue from the currently traversed position, filtering target request packets that match the second combined packet, merging them with the second combined packet to generate the third combined packet, and so on, until the latest obtained combined packet is a stripe alignment request packet, or until every request packet in the second queue has been traversed.

[0099] If the currently generated merged packet is a stripe alignment request packet, this invention can immediately terminate the traversal and filtering operations, prohibiting further filtering of target request packets matching the current merged packet in the second queue. For example, if the second merged packet is a stripe alignment request packet, the traversal and filtering operations can be terminated immediately, prohibiting further filtering of target request packets matching the second merged packet in the second queue.

[0100] In this invention, when the last request packet is encountered during traversal of the second queue, regardless of whether the request packet matches the current combined packet, the traversal operation is considered complete, and the traversal and filtering operations are terminated. Of course, if the last request packet matches the current combined packet, the invention will still merge the last request packet with the current combined packet to generate the corresponding combined packet, which will then be used as the final combined packet.

[0101] S204. Based on the latest obtained combination packet or the combination packet after traversing the second queue, perform the corresponding write operation on the striped data.

[0102] Specifically, when the latest obtained combined packet is a stripe alignment request packet, the present invention can perform corresponding write operations on the striped data based on the latest obtained combined packet. Alternatively, the present invention can perform corresponding write operations on the striped data based on the combined packets after traversing each request packet in the second queue.

[0103] The packet processing method for striped data proposed in this invention can use a first request packet to filter out matching target request packets, merge the first request packet and the target request packet to produce a first combined packet, and continue to try to filter and merge matching non-striped aligned request packets in a second queue based on the current combined packet when the current combined packet is a non-striped aligned request packet, and perform corresponding write operations on the striped data according to the latest combined packet when the latest combined packet is a striped aligned request packet, or perform corresponding write operations on the striped data according to the combined packets after traversing the second queue, further reducing the number of write reads caused by multiple non-striped aligned request packets, and even avoiding write reads caused by non-striped aligned request packets, further improving disk operating efficiency and ensuring disk performance.

[0104] based on Figure 1 As shown, this invention proposes a third method for processing striped data packets. In this method, the first request packet includes first data to be written, a first offset, and a first data length. Then, the above-mentioned method of traversing each non-striped aligned request packet in the second queue based on the first request packet to filter out one or more target request packets includes:

[0105] The non-striped aligned request packet encountered in the second queue is identified as the second request packet. The second request packet includes the second data to be written, the second offset, and the second data length.

[0106] Based on the first offset and the second offset, and based on at least one of the first data length and the second data length, it is determined whether the second request packet matches the first request packet.

[0107] If so, the second request packet is determined as the target request packet that matches the first request packet.

[0108] If not, continue traversing the next non-striped aligned write request packet in the second queue until a target request packet matching the first request packet is identified.

[0109] Specifically, the present invention can obtain the first data to be written, the first offset, and the first data length from the first request packet, and obtain the second data to be written, the second offset, and the second data length from the second request packet. Then, based on the first offset and the second offset, and based on at least one of the first data length and the second data length, it can determine whether the second request packet matches the first request packet.

[0110] In this invention, when determining that the second request packet matches the first request packet, the second request packet can be identified as the target request packet that matches the first request packet.

[0111] In this invention, when it is determined that the second request packet does not match the first request packet, it can start from the position currently traversed in the second queue and continue to traverse the subsequent non-striped aligned request packets, and continue to determine whether the traversed non-striped aligned request packets match the first request packet, until a target request packet that matches the first request packet is determined.

[0112] Optionally, in other methods proposed in this invention, determining whether the second request packet matches the first request packet based on at least one of the first offset and the second offset, and the first data length and the second data length, includes:

[0113] If the first estimated offset is equal to the second offset, or the second estimated offset is equal to the first offset, then the second request packet is determined to match the first request packet. The first estimated offset is the sum of the first offset and the first data length, and the second estimated offset is the sum of the second offset and the second data length.

[0114] If it is determined that the first estimated offset is not equal to the second offset, and the second estimated offset is not equal to the first offset, then it is determined that the second request packet does not match the first request packet.

[0115] Specifically, when the first calculated offset equals the second offset, or the second calculated offset equals the first offset, the present invention can determine that the positions of the first data to be written and the second data to be written in the striped data are continuous. At this time, the present invention can determine that the second request packet and the first request packet can be merged, thereby determining that the second request packet matches the first request packet.

[0116] Specifically, when the first calculated offset is not equal to the second offset, and the second calculated offset is not equal to the first offset, the present invention can determine that the positions of the first data to be written and the second data to be written in the striped data are not continuous. At this time, the present invention can determine that the second request packet and the first request packet cannot be merged, thereby determining that the second request packet and the first request packet do not match.

[0117] Optionally, the request packets in the first and second queues can be sorted in ascending order of offset.

[0118] Optionally, in other methods proposed in this invention, when the second request packet is the first target request packet, the first request packet and the first target request packet are merged to generate a first combined packet, including:

[0119] The first data to be written is merged into the second data to be written in the second request packet to generate the third request packet.

[0120] The merge offset and merge data length are generated based on the first data length and the second data length, and based on one of the first offset and the second offset.

[0121] The offset in the third request packet is adjusted from the second offset to the merged offset, and the data length in the third request packet is adjusted from the second data length to the merged data length, thus generating the first combined packet.

[0122] Optionally, step S104 above may include:

[0123] If both the merge offset and the merged data length are divisible by the preset minimum storage unit value, then the first combined packet is determined to be a stripe alignment request packet.

[0124] If neither the merge offset nor the merged data length is divisible by the minimum storage unit value, then the first combined packet is determined to be a non-striped alignment request packet.

[0125] The smallest storage unit value is the data size of the data block in the striped data mentioned above.

[0126] The packet processing method for striped data proposed in this invention can effectively determine whether two request packets match, thereby effectively filtering the target request packets.

[0127] based on Figure 1 This invention proposes a fourth method for processing striped data packets. In this method, each request packet in the second queue originates from the first queue. Furthermore, before step S101, the method may further include:

[0128] Retrieve the fourth request packet from the first queue.

[0129] If the fourth request packet is a striped alignment request packet, then the fourth request packet is pushed to the second queue.

[0130] If the fourth request packet is a non-strip-aligned request packet and there is no matching request packet in the second queue, then the fourth request packet is pushed to the second queue.

[0131] It should be noted that when the present invention receives any of the above-mentioned request packets from the client, it can first place them in the first queue. Then, when a preset condition is met (such as the number of request packets in the first queue reaching a certain threshold, or the reception time exceeding a certain threshold), a request packet is obtained from the first queue, and it is determined whether the request packet is a stripe alignment request packet. If so, the request packet can be placed in the second queue.

[0132] If the request packet is not a stripe alignment request packet, the present invention can traverse the second queue to search for a matching request packet. If no matching request packet is found, it can be placed in the second queue. If a matching request packet is found, it can be merged with the found request packet to generate a combined packet. It is then determined whether the combined packet is a stripe alignment request packet. If so, the corresponding write operation can be performed based on the combined packet. If not, the traversal can continue from the position already traversed in the second queue until the latest merged combined packet is a stripe alignment request packet. The corresponding write operation can then be performed based on the latest merged combined packet. Alternatively, after traversing all request packets in the second queue, the corresponding write operation can be performed based on the combined packet after traversal.

[0133] Optionally, when the latest obtained combined packet is a stripe alignment request packet, the present invention may also store it in a second queue.

[0134] Optionally, when the combined package after traversal is obtained, the present invention can also save it in a second queue.

[0135] Optionally, in other methods proposed in this invention, after step S101 above, the method further includes:

[0136] Lock the first queue.

[0137] After performing the corresponding write operation on the unstriped data, the first queue is unlocked.

[0138] It should be noted that after locking the first queue, the present invention can no longer obtain request packets from the first queue. This avoids traversing and filtering request packets in the second queue based on multiple request packets in the first queue at the same time, thereby effectively avoiding program errors caused by traversing, filtering or merging multiple request packets at the same time.

[0139] Specifically, after the corresponding traversal, filtering, merging, and writing operations based on the first request packet are completed, the present invention can unlock the first queue, continue to obtain request packets from the first queue and use them as new first request packets for execution. Figure 1 The process is shown below.

[0140] The packet processing method for striped data proposed in this invention can effectively avoid errors in operations such as traversal, filtering, or merging by performing a latch on the first queue.

[0141] based on Figure 1 This invention proposes a fifth method for packet processing of striped data, which, after step S101 above, may further include:

[0142] The first moment is determined as the moment when the first request packet arrives in the first queue.

[0143] After generating the first combined packet, determine the second time point. The second time point is the moment when the first target request packet arrives in the first queue.

[0144] The earlier time is determined from the first time and the second time.

[0145] At this point, step S107 may include:

[0146] When the time elapsed since the earlier time exceeds a preset time elapsed threshold, the corresponding write operation is performed on the striped data according to the first combined packet.

[0147] Specifically, in the present invention, when the first combined packet is obtained and it is determined that the first combined packet is a stripe alignment request packet, the first combined packet can be stored in the second queue first. When the time elapsed since the first time exceeds a preset time threshold, the corresponding write operation is performed according to the first combined packet.

[0148] Specifically, this invention records the reception time of any request packet when it is placed in the first queue. Then, the invention retrieves the first request packet from the first queue, performs traversal, filtering, and merging operations on the first request packet in the second queue, and obtains the latest merged combined packet or the combined packet after traversal. This combined packet can then be stored in the second queue, and the reception time of the earliest received request packet in the combined packet is determined as the earlier time. When the elapsed duration of this online time exceeds a preset duration threshold, the invention then performs the corresponding write operation based on the combined packet.

[0149] If a first request packet obtained from the first queue does not have a matching request packet in the second queue, the first request packet can be placed in the second queue, awaiting merging attempts with other request packets in the first queue. If the first request packet has not been merged and the time elapsed since its reception exceeds a preset time threshold, the present invention can perform a corresponding write operation based on the first request packet. Alternatively, after a certain time has elapsed since the earlier time of the combined packet containing the first request packet, the present invention can perform a corresponding write operation based on that combined packet.

[0150] Optionally, the present invention may also first traverse each request packet in the first queue, and treat each traversed request packet as the first request packet for execution. Figure 2 The process shown, and in Figure 2Before executing the write operation, the latest obtained combined packet or the combined packet after all iterations are stored in the second queue. If the first request packet is a stripe alignment request packet, it can also be stored in the second queue. This invention can, after iterating through each request packet in the first queue, begin iterating through each request packet in the second queue, determining whether the waiting time (the time obtained by subtracting the receiving time from the time of iteration) of each iterated request packet exceeds a preset time threshold. If so, the corresponding write operation can be performed based on the iterated request packet.

[0151] In this invention, after traversing all request packets in the second queue, a certain time interval, such as 1 millisecond, can be waited before starting to traverse all request packets in the first queue and execute the operation. Figure 1 The process is shown below.

[0152] The packet processing method for striped data proposed in this invention can store request packets or merged combined packets from the first queue in the second queue, waiting for request packets from the first queue to attempt to merge, thereby improving the merging rate of non-striped aligned request packets in the second queue and further reducing the number of write reads caused by non-striped aligned request packets.

[0153] based on Figure 1 This invention proposes a sixth method for processing striped data packets, which, after step S107, may further include:

[0154] Based on the first request packet, generate the corresponding first reply message.

[0155] Based on the first target request packet mentioned above, a corresponding second reply message is generated.

[0156] Return the first reply message and the second reply message to the client.

[0157] Specifically, this invention can set a merge identifier in both the first request packet and the target request packet before merging them, and retain the merge identifiers in the generated first combined packet during the merging process. Subsequently, when performing a corresponding write operation based on the first combined packet, this invention can extract the two merge identifiers, generate two response messages based on these identifiers, and return these two response messages to the client.

[0158] In this invention, when setting the merge identifier, the offset and data length can be extracted from the first request packet and the target request packet, respectively. The offset, data length, and merge identifier of the first request packet are associated and stored in a pre-set third queue, and the offset, length, and merge identifier of the target request packet are associated and stored in the same third queue. Subsequently, when performing a write operation based on the first combined packet, this invention can obtain the merge identifier of the first request packet and the target request packet from the first combined packet, find the corresponding offset and data length from the third queue, generate a corresponding reply message based on the offset and data length, and return it to the client.

[0159] The packet processing method for striped data proposed in this invention can return a corresponding number of messages containing relevant information to the client in response to the client's request packets.

[0160] like Figure 3 As shown, to better illustrate the above-mentioned method proposed in the embodiments of the present invention, the present invention proposes a seventh method for packet processing of striped data, combined with application scenarios. This method may include the following steps:

[0161] S301, Receive a request packet op from the client and use it as the first op.

[0162] S302. Determine the primary OSD to which the first op belongs and push it to the primary OSD.

[0163] S303. When the master OSD determines that the message type of the first op is a write request and the first op does not include the parsed flag, it pushes the first op into the first queue, records the enqueue time of the first op in the first queue, and sets the parsed flag in the first op.

[0164] S304. In the merge thread, an op is retrieved from the first queue and used as the second op. If the second op is a stripe alignment request packet, the second op is pushed to the second queue for storage.

[0165] S305. If the second op is a non-strip-aligned request packet, then traverse each request packet in the second queue and try to filter out non-strip-aligned request packets that match the second op. If no matching packets are found, then push the second op into the second queue for storage.

[0166] S306. If a match is found, merge the second op with the matching op to generate the first combined op. Determine if the first combined op is a stripe alignment request packet. If so, end the filtering operation and save the first combined op in the second queue.

[0167] S307. If the first combination of ops is a non-strip-aligned request packet, then continue traversing the second queue from the position already traversed to find an op that matches the first combination of ops.

[0168] S308. If found, merge the first combination op with the matching op to generate the second combination op. Determine whether the second combination op is a stripe alignment request packet. Continue until the latest obtained combination op is a stripe alignment request packet, or until every request packet in the second queue has been traversed. Store the latest obtained combination packet or the combination op after traversing the second queue in the second queue.

[0169] S309. Traverse each op in the first queue. Any op can be used as the second op to perform corresponding filtering, judgment, merging and saving operations.

[0170] S310. Traverse each op in the second queue, determine the ops whose enqueue and dequeue durations exceed the preset duration threshold and the ops that belong to the stripe alignment request packet, and perform the corresponding write operation based on the ops whose enqueue durations exceed the preset duration threshold and the ops that belong to the stripe alignment request packet.

[0171] Optionally, the method may also include:

[0172] S311. When a matching op is selected, set a merge identifier in the two ops to be merged, and obtain the offset and data length of the two ops respectively. Establish an association between the merge identifier, offset and data length and send them to the preset third queue for storage.

[0173] S312. When performing a write operation based on a certain op, if the op includes a merge identifier, then extract all the merge identifiers from the op, determine the corresponding offset and data length from the third queue according to each merge identifier, and generate the corresponding reply message according to the determined offset and data length.

[0174] S313. After performing a write operation based on the op including the merge identifier, return the corresponding reply message to the client.

[0175] The packet processing method for striped data proposed in this invention can effectively reduce the number of read operations during write operations for striped data, or even avoid read operations during write operations altogether, thereby improving disk operating efficiency and ensuring disk performance.

[0176] and Figure 1 The method shown corresponds to, for example Figure 4As shown, this invention proposes a server, including: a first queue and a second queue. Both the first queue and the second queue include at least one request packet from a client, each request packet being used to perform a write operation on striped data stored in the server. The server also includes: a first acquisition unit 401, a first filtering unit 402, a first merging unit 403, a first judgment unit 404, a termination unit 405, and a first execution unit 406. Wherein:

[0177] The first acquisition unit 401 is used to acquire the first request packet from the first queue.

[0178] The first filtering unit 402 is used to, when it is determined that the first request packet is a non-striped aligned request packet, traverse each non-striped aligned request packet in the second queue according to the first request packet and filter out one or more target request packets. The first filtered target request packet matches the first request packet.

[0179] The first merging unit 403 is used to merge the first request packet with the first target request packet to generate a first combined packet.

[0180] The first judgment unit 404 is used to determine whether the first combined packet is a stripe alignment request packet. If so, the termination unit 405 is executed.

[0181] End unit 405 is used to end the filtering operation.

[0182] The first execution unit 406 is used to perform corresponding write operations on the striped data according to the first combined package.

[0183] Optionally, the server may also include: a second filtering unit, a second merging unit, a second judgment unit, and a second execution unit. Wherein:

[0184] The second filtering unit is used to continue filtering target request packets that match the first combined packet if the first combined packet is not a stripe-aligned request packet.

[0185] The second merging unit is used to merge the target request packets that will continue to be filtered with the first combined packet to generate the second combined packet.

[0186] The second judgment unit is used to continue to judge whether the second combined packet is a stripe alignment request packet, until the latest obtained combined packet is a stripe alignment request packet, or until every non-stripe alignment request packet in the second queue has been traversed.

[0187] The second execution unit is used to perform corresponding write operations on striped data based on the latest obtained combo packet or the combo packet after traversing the second queue.

[0188] Optionally, the first request packet includes first data to be written, a first offset, and a first data length. The first filtering unit 402 includes: a first determining unit, a third judging unit, a second determining unit, and a third determining unit.

[0189] The first determining unit is used to identify the non-striped aligned request packet traversed in the second queue as the second request packet. The second request packet includes second data to be written, a second offset, and a second data length.

[0190] The third determination unit is used to determine whether the second request packet matches the first request packet based on at least one of the first offset and the second offset, and the first data length and the second data length. If yes, the second determination unit is triggered. If no, the third determination unit is triggered.

[0191] The second determining unit is used to determine the second request packet as the target request packet that matches the first request packet.

[0192] The third determining unit is used to continue traversing the next non-striped aligned write request packet in the second queue until a target request packet matching the first request packet is determined.

[0193] Optionally, the third judgment unit includes: a fourth determination unit and a fifth determination unit.

[0194] The fourth determining unit is used to determine that the second request packet matches the first request packet when the first estimated offset is equal to the second offset, or the second estimated offset is equal to the first offset. The first estimated offset is the sum of the first offset and the first data length, and the second estimated offset is the sum of the second offset and the second data length.

[0195] The fifth determining unit is used to determine that the second request packet does not match the first request packet when the first calculated offset is not equal to the second offset and the second calculated offset is not equal to the first offset.

[0196] Optionally, when the second request packet is the first target request packet, the first merging unit 403 includes: a third merging unit, a first generating unit, and an adjusting unit.

[0197] The third merging unit is used to merge the first data to be written into the second data to be written in the second request packet to generate the third request packet.

[0198] The first generation unit is configured to generate a merge offset and a merge data length based on a first data length and a second data length, and based on one of a first offset and a second offset.

[0199] The adjustment unit is used to adjust the offset in the third request packet from the second offset to the merged offset, and to adjust the data length in the third request packet from the second data length to the merged data length, thereby generating the first combined packet.

[0200] Optionally, the first determination unit 404 includes a sixth determination unit and a seventh determination unit.

[0201] The sixth determining unit is used to determine the first combined packet as a stripe alignment request packet if both the merge offset and the merged data length are divisible by a preset minimum storage unit value.

[0202] The seventh determining unit is used to determine the first combined packet as a non-strip alignment request packet if neither the merge offset nor the merged data length can be divided by the minimum storage unit value.

[0203] Optionally, each request packet in the second queue originates from the first queue. The server also includes: a second acquisition unit, a first push unit, and a second push unit.

[0204] The second acquisition unit is used to acquire the fourth request packet from the first queue before acquiring the first request packet from the first queue.

[0205] The first push unit is used to push the fourth request packet to the second queue if the fourth request packet is a stripe-aligned request packet.

[0206] The second push unit is used to push the fourth request packet to the second queue if the fourth request packet is a non-strip-aligned request packet and there is no matching request packet in the second queue.

[0207] Optionally, the server may also include a locking unit and an unlocking unit.

[0208] The locking unit is used to lock the first queue after obtaining the first request packet from the first queue.

[0209] The unlocking unit is used to unlock the first queue after performing the corresponding write operation on the non-striped data.

[0210] Optionally, the server may also include: a first-time determination unit, a second-time determination unit, and a third-time determination unit, wherein:

[0211] The first moment determination unit is used to determine the first moment after obtaining the first request packet from the first queue. The first moment is the moment when the first request packet arrives at the first queue.

[0212] The second time-determination unit is used to determine the second time after the first combined packet is generated. The second time is the moment when the first target request packet arrives at the first queue.

[0213] The third time-determining unit is used to determine the earlier time from the first time and the second time.

[0214] The first execution unit 406 is used to perform a corresponding write operation on the striped data according to the first combination packet when the duration of the distance from the earlier time exceeds a preset duration threshold.

[0215] Optionally, the server may also include: a second generation unit, a third generation unit, and a return unit.

[0216] The second generation unit is used to generate a corresponding first reply message based on the first request packet.

[0217] The third generation unit is used to generate a corresponding second reply message based on the first target request packet mentioned above.

[0218] The return unit is used to return the first reply message and the second reply message to the client.

[0219] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0220] The server proposed in this invention can use a first request packet to filter out matching target request packets, merge the first request packet and the target request packet to produce a first combined packet, and when the current combined packet is a non-striped alignment request packet, it can continue to try to filter and merge matching non-striped alignment request packets in a second queue based on the current combined packet. When the latest combined packet is a striped alignment request packet, it performs corresponding write operations on the striped data according to the latest combined packet, or after traversing the second queue, it performs corresponding write operations on the striped data according to the combined packets after traversing the second queue. This further reduces the number of read operations during writes caused by multiple non-striped alignment request packets, and even avoids read operations during writes caused by non-striped alignment request packets, thereby further improving disk operating efficiency and ensuring disk performance.

[0221] In this embodiment, the server is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0222] This invention also provides a computer device having the above-described features. Figure 4 The server shown.

[0223] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 5 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 Take a processor 10 as an example.

[0224] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0225] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0226] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0227] Memory 20 may include volatile memory, such as random access memory. Memory may also include non-volatile memory, such as flash memory, hard disk, or solid-state drive. Memory 20 may also include combinations of the above types of memory.

[0228] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0229] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc. Further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0230] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A packet processing method for striped data, characterized in that, The method is applied to a server; the server includes a first queue and a second queue, each of which includes at least one request packet from a client, and each request packet is used to perform a write operation on striped data stored in the server; the method includes: Retrieve the first request packet from the first queue; If the first request packet is determined to be a non-striped aligned request packet, one or more target request packets are selected by traversing each non-striped aligned request packet in the second queue based on the first request packet; wherein the first selected target request packet matches the first request packet. The first request packet is merged with the first target request packet to generate a first combined packet; Determine whether the first combined packet is a stripe alignment request packet. If so, end the filtering operation and perform the corresponding write operation on the striped data according to the first combined packet. The first request packet includes a first data to be written, a first offset, and a first data length; the step of traversing each non-striped aligned request packet in the second queue based on the first request packet and filtering out one or more target request packets includes: The non-striped aligned request packets traversed in the second queue are identified as the second request packets; the second request packets include the second data to be written, the second offset, and the second data length; Based on the first offset and the second offset, and based on at least one of the first data length and the second data length, determine whether the second request packet matches the first request packet; If so, the second request packet is determined to be the target request packet that matches the first request packet; If not, continue traversing the next non-striped aligned write request packet in the second queue until the target request packet that matches the first request packet is determined; Each request packet in the second queue originates from the first queue; prior to retrieving the first request packet from the first queue, the method further includes: Retrieve the fourth request packet from the first queue; If the fourth request packet is a stripe alignment request packet, then the fourth request packet is pushed to the second queue; If the fourth request packet is a non-strip-aligned request packet and there is no matching request packet in the second queue, then the fourth request packet is pushed to the second queue.

2. The method according to claim 1, characterized in that, After determining whether the first combined packet is a stripe alignment request packet, the method further includes: If the first combined packet is not a stripe-aligned request packet, then continue to filter the target request packets that match the first combined packet, and merge them with the first combined packet to generate a second combined packet. Continue to determine whether the second combined packet is a stripe-aligned request packet until the latest combined packet is a stripe-aligned request packet, or until every non-stripe-aligned request packet in the second queue has been traversed. Based on the latest obtained combination packet or the combination packet after traversing the second queue, perform the corresponding write operation on the striped data.

3. The method according to claim 1, characterized in that, The step of determining whether the second request packet matches the first request packet based on the first offset and the second offset, and based on at least one of the first data length and the second data length, includes: If it is determined that the first estimated offset is equal to the second offset, or the second estimated offset is equal to the first offset, then the second request packet is determined to match the first request packet; wherein, the first estimated offset is the sum of the first offset and the first data length, and the second estimated offset is the sum of the second offset and the second data length; If it is determined that the first estimated offset is not equal to the second offset, and the second estimated offset is not equal to the first offset, then it is determined that the second request packet does not match the first request packet.

4. The method according to claim 1, characterized in that, When the second request packet is the first target request packet, the step of merging the first request packet and the first target request packet to generate a first combined packet includes: The first data to be written is merged into the second data to be written in the second request packet to generate a third request packet; Based on the first data length and the second data length, and based on one of the first offset and the second offset, generate a merge offset and a merge data length; The offset in the third request packet is adjusted from the second offset to the merged offset, and the data length in the third request packet is adjusted from the second data length to the merged data length to generate the first combined packet.

5. The method according to claim 4, characterized in that, The step of determining whether the first combined packet is a stripe alignment request packet includes: If both the merge offset and the merged data length are divisible by the preset minimum storage unit value, then the first combined packet is determined to be a stripe alignment request packet. If neither the merge offset nor the merged data length can be divided by the minimum storage unit value, then the first combined packet is determined to be a non-strip alignment request packet.

6. The method according to claim 1, characterized in that, After retrieving the first request packet from the first queue, the method further includes: The first moment is determined as the moment when the first request packet arrives at the first queue. After the first combined packet is generated, a second time is determined; the second time is the time when the first target request packet arrives at the first queue. Determine the earlier time from the first time and the second time; The step of performing a corresponding write operation on the striped data according to the first combined packet includes: When the time elapsed since the earlier time exceeds a preset time threshold, a corresponding write operation is performed on the striped data according to the first combined packet.

7. The method according to any one of claims 1-6, characterized in that, After performing the corresponding write operation on the striped data according to the first combined packet, the method further includes: Based on the first request packet, generate the corresponding first reply message; Based on the first target request packet, generate a corresponding second reply message; The first reply message and the second reply message are returned to the client.

8. A server-side component, characterized in that, include: A first queue and a second queue; both the first queue and the queues include at least one request packet from the client, each request packet being used to write striped data stored in the server; the server further includes: a first acquisition unit, a first filtering unit, a first merging unit, a first judgment unit, a termination unit, and a first execution unit; wherein: The first acquisition unit is used to acquire a first request packet from the first queue; The first filtering unit is configured to, when determining that the first request packet is a non-strip-aligned request packet, traverse each non-strip-aligned request packet in the second queue according to the first request packet and filter out one or more target request packets; wherein the first target request packet filtered out matches the first request packet; The first merging unit is used to merge the first request packet with the first target request packet to generate a first combined packet; The first determining unit is used to determine whether the first combined packet is a stripe alignment request packet; if so, the ending unit is executed. The termination unit is used to terminate the filtering operation; The first execution unit is configured to perform a corresponding write operation on the striped data according to the first combined package; The first request packet includes a first data to be written, a first offset, and a first data length; the first filtering unit includes a first determining unit, a third judging unit, a second determining unit, and a third determining unit. The first determining unit is configured to determine the non-striped aligned request packet traversed in the second queue as the second request packet; the second request packet includes second data to be written, a second offset, and a second data length; The third determination unit is used to determine whether the second request packet matches the first request packet based on the first offset and the second offset, and based on at least one of the first data length and the second data length; if yes, the second determination unit is triggered; if no, the third determination unit is triggered. The second determining unit is configured to determine the second request packet as the target request packet that matches the first request packet; The third determining unit is used to continue traversing the next non-striped aligned write request packet in the second queue until the target request packet that matches the first request packet is determined. Each request packet in the second queue originates from the first queue; the server further includes: a second acquisition unit, a first push unit, and a second push unit; The second acquisition unit is configured to acquire a fourth request packet from the first queue before acquiring the first request packet from the first queue; The first push unit is configured to push the fourth request packet to the second queue if the fourth request packet is a stripe alignment request packet; The second push unit is configured to push the fourth request packet to the second queue if the fourth request packet is a non-strip-aligned request packet and there is no matching request packet in the second queue.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the packet processing method for striped data as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the packet processing method for striped data as described in any one of claims 1 to 7.

Images