Metadata request retry method, apparatus, computer equipment, medium and product

Abstract

The present application relates to a metadata request retry method, apparatus, computer equipment, medium, and product. The method comprises: upon receiving a request response message sent by a server in response to a metadata request, obtaining the number of request retries for the metadata request within a preset sending period; if it is determined based on the number of request retries that the metadata request meets the penalty retry condition, then determining the initial retry time interval and penalty time interval of the metadata request based on the number of request retries; the initial retry time interval is positively correlated with the number of request retries; determining the target retry time interval for the metadata request based on the initial retry time interval and the penalty time interval; and resending the metadata request to the server according to the target retry time interval. The use of this method can reduce the pressure exerted on the server by metadata during the retry process.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a metadata request retry method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] With the rapid advancement of information technology, big data and cloud computing have gradually permeated every corner of society. The explosive growth of data has brought unprecedented challenges to distributed storage systems. In distributed storage systems, metadata management and operation are critical to ensuring stable system operation, and its processing capacity directly determines the overall system performance. However, in actual operation, servers inevitably encounter various anomalies when processing metadata requests, such as communication delays, server busyness, resource contention, concurrent lock conflicts, software design flaws, network failures, and storage device failures. These anomalies often lead to request failures, which in turn affect user experience and system stability.

[0003] Existing metadata request retry mechanisms are often led by the server and implemented in conjunction with methods such as queues. That is, when the server detects a request failure, it places the failed metadata request in a retry queue and attempts to process it again after a certain period of time. Although this retry mechanism can achieve order preservation and priority control of requests to a certain extent, it also requires the server to consume additional computing resources and memory space to maintain the retry mechanism, which increases the pressure on the server and thus affects the performance of metadata operations. Summary of the Invention

[0004] Based on this, it is necessary to provide a metadata request retry method, apparatus, computer device, computer-readable storage medium and computer program product that can reduce the pressure on the server caused by metadata during the retry process to address the above technical problems.

[0005] In a first aspect, the present application provides a metadata request retry method, the method comprising:

[0006] Upon receiving a request response message sent by the server in response to the metadata request, obtaining a request retry count of the metadata request within a preset sending period;

[0007] If it is determined that the metadata request meets the penalty retry condition based on the request retry count, then determining an initial retry time interval and a penalty time interval for the metadata request based on the request retry count; the initial retry time interval is positively correlated with the request retry count;

[0008] determining a target retry time interval for the metadata request based on the initial retry time interval and the penalty time interval;

[0009] Resend the metadata request to the server according to the target retry time interval.

[0010] In one embodiment, the method further comprises:

[0011] Obtain the total number of metadata requests sent to the server within the preset sending period;

[0012] Determine the ratio of the number of request retries to the total number of requests as the request abnormality ratio;

[0013] If the request exception ratio is greater than or equal to the penalty ratio threshold, it is determined that the metadata request meets the penalty retry condition.

[0014] In one embodiment, determining the initial retry time interval and the penalty time interval of the metadata request based on the number of request retries includes:

[0015] Determining a random jitter value and a basic unit time for calculating a time interval according to the number of request retries;

[0016] Determining an initial retry time interval for the metadata request using an improved exponential backoff algorithm based on the request retry count, the random jitter value, the basic unit time, and a preset interval growth amplitude value;

[0017] Determine a penalty time interval for the metadata request based on the request anomaly ratio.

[0018] In one embodiment, determining the random jitter value and the basic unit time for calculating the time interval according to the number of request retries includes:

[0019] Randomly select an initial jitter value from a preset random jitter value range;

[0020] Assigning a positive or negative direction to the initial jitter value according to the parity of the request retry number to obtain a random jitter value for time interval calculation;

[0021] Based on the number of request retries, a preset interval increase value, and a maximum retry time, a basic unit time for calculating the time interval is determined.

[0022] In one embodiment, determining the penalty time interval for the metadata request based on the request anomaly ratio includes:

[0023] Get the preset penalty factor of the retry penalty mechanism;

[0024] The product of the preset penalty factor and the request abnormality ratio is determined as the penalty time interval of the metadata request.

[0025] In one embodiment, the method further comprises:

[0026] If the request exception ratio is less than the penalty ratio threshold, determining that the metadata request does not meet the penalty-retry condition;

[0027] An initial retry time interval of the metadata request determined based on the number of request retries is determined as a target retry time interval of the metadata request.

[0028] In a second aspect, the present application provides a metadata request retry device, the device comprising:

[0029] A request retry count acquisition module, configured to, upon receiving a request response message sent by the server in response to a metadata request, acquire the request retry count of the metadata request within a preset sending period;

[0030] a condition judgment module, configured to, if it is determined based on the request retry count that the metadata request satisfies a retry condition with a penalty, determine an initial retry time interval and a penalty time interval for the metadata request based on the request retry count; wherein the initial retry time interval is positively correlated with the request retry count;

[0031] a target retry time interval determining module, configured to determine a target retry time interval for the metadata request based on the initial retry time interval and the penalty time interval;

[0032] The request retry module is used to resend the metadata request to the server according to the target retry time interval.

[0033] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0035] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0036] In the metadata request retry method, apparatus, computer device, storage medium, and computer program product, upon receiving a response message from a server regarding a metadata request, the client obtains the number of metadata request retries within a preset transmission period. If the metadata request satisfies the penalty-based retry condition based on the number of request retries, the client determines an initial retry interval and a penalty interval for the metadata request based on the number of request retries. The initial retry interval is positively correlated with the number of metadata request retries. Based on the initial retry interval and the penalty interval, a target retry interval for the metadata request is determined, and the metadata request is resent to the server at the target retry interval. By shifting the execution of the request retry mechanism from the server to the client, the server's computing resources and memory usage can be effectively reduced. Furthermore, by matching the metadata request's retry count with the retry interval that meets the metadata request's requirements and applying a retry penalty, the client can further reduce the pressure of invalid requests placed on the server during metadata request retries. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A diagram illustrating an application environment of a metadata request retry method in some embodiments;

[0038] Figure 2 A flowchart of a metadata request retry method in some embodiments;

[0039] Figure 3 A flowchart of a metadata request retry method in some other embodiments;

[0040] Figure 4 Schematic diagram of a process for determining an initial retry time interval and a penalty time interval for a metadata request based on the number of request retries in some embodiments;

[0041] Figure 5 A schematic diagram of a flow chart for determining a random jitter value and a basic unit time for time interval calculation according to the number of request retries in some embodiments;

[0042] Figure 6 A flowchart of a metadata request retry method in some other embodiments;

[0043] Figure 7 A schematic diagram of a range of values ​​for a retry time interval of 10 retries in some embodiments;

[0044] Figure 8 A structural block diagram of a metadata request retry apparatus in some embodiments;

[0045] Figure 9 1 is a diagram of the internal structure of a computer device in some embodiments. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0047] The metadata request retry method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the client 102 communicates with the server 104 through the network. The data storage system can store the data that the client 102 needs to process. The data storage system can be integrated on the client 102, or it can be placed on the cloud or other network servers. When the client 102 receives the request return information sent by the server 104 for the metadata request, it obtains the number of request retries of the metadata request within the preset sending cycle. If it is determined that the metadata request meets the retry condition with penalty based on the number of request retries, the initial retry time interval and the penalty time interval of the metadata request are determined based on the number of request retries. The initial retry time interval is positively correlated with the number of request retries of the metadata request. The client 102 determines the target retry time interval of the metadata request based on the initial retry time interval and the penalty time interval, and resends the metadata request to the server 104 according to the target retry time interval.

[0048] The client 102 is the interface through which users interact with the distributed storage system. It is responsible for initiating metadata operation requests, accepting request return information, and calculating and processing request retry mechanisms. The client 102 can communicate with the server via a dedicated API or protocol, such as the TCP protocol. As will be appreciated, the client 102 includes a metadata request processing module and a retry mechanism calculation module. The metadata request processing module is used to initiate metadata operation requests and accept request return information, while the retry mechanism calculation module is used to calculate and process request retry mechanisms.

[0049] In a distributed storage system, server 104 is primarily responsible for data storage, management, and processing client requests. It can include two main components: a metadata server and a data node. As will be appreciated, a single server 104 can connect to multiple clients 102. Server 104 can also include a metadata request processing module responsible for processing metadata operation requests.

[0050] In some embodiments, as Figure 2 As shown, a metadata request retry method is provided, which is applied to Figure 1 Taking the client 102 in the example as an example, the following steps are included:

[0051] S202: Upon receiving the request return information sent by the server in response to the metadata request, obtain the number of request retries of the metadata request within a preset sending period.

[0052] Among them, metadata refers to data that describes data, and metadata request is the request information generated when the client requests the server to access or query metadata.

[0053] The request return information is notification information generated and sent by the server when it determines that the metadata request has failed. When the client receives the request return information sent by the server in response to the metadata request, it can determine that the metadata request has failed.

[0054] The preset sending period is a preset time period used to count request retries. To reduce statistical complexity and improve statistical consistency, designers can pre-set the preset sending period in the client. When determining the retry interval for the request retry mechanism, the client's metadata request sending status can be determined based on the number of request retries counted within the preset sending period, thereby matching the metadata request with a retry interval that matches the sending request. It is understood that the specific length of the preset sending period can be determined by the designer based on actual statistical requirements. For example, the preset sending period can be one hour or one day.

[0055] The request retry count refers to the number of times a client sends metadata requests to the server using the request retry mechanism within a preset sending period. The value of the request retry count can indicate the current status of the client's metadata request. For example, it can provide feedback on whether there are any abnormalities on the client, such as communication delays, server busyness, resource contention, concurrent lock conflicts, software design flaws, network failures, storage device failures, etc., which may affect the success rate of metadata requests.

[0056] Specifically, when the server determines that the metadata request sent by the client has failed, it can generate request return information for the metadata request and send the request return information to the client corresponding to the metadata request. When the client receives the request return information sent by the server for the metadata request, it obtains the number of request retries for the metadata request within a preset sending period.

[0057] In some embodiments, a request retry count module is provided in the client, which can count the number of retries of the metadata request to determine the number of request retries of the metadata request within a preset sending period.

[0058] S204: If it is determined that the metadata request meets the retry condition with penalty according to the number of request retries, an initial retry time interval and a penalty time interval of the metadata request are determined based on the number of request retries.

[0059] Among them, the penalty retry condition is a preset judgment condition used to determine whether it is necessary to add a penalty time interval on the basis of the initial retry time interval when the client performs a request retry operation on the metadata request. The penalty retry condition can be set in advance by the designer based on the abnormal cause that causes the metadata request failure. For example, the designer can determine abnormal causes such as software design defects, storage device failure, and network failure as penalty retry causes. When the abnormal cause that causes the failure of the client metadata request is at least one of the above-mentioned penalty retry causes, it can be determined that the metadata request meets the penalty retry condition.

[0060] In some embodiments, the client may determine the abnormal reason that caused the failure of the client metadata request based on the number of request retries, and determine that the metadata request meets the penalty retry condition if the abnormal reason matches at least one of the penalty retry reasons.

[0061] The initial retry interval refers to the retry interval that the client needs to wait for when executing the request retry mechanism if the metadata request does not meet the penalty retry conditions. Existing methods for clients to execute request retry mechanisms usually set the maximum number of retries or the longest retry time, and then initiate retry requests at equal time intervals. Under this request retry mechanism, if the retry interval is too short, it will not only lead to frequent returns of previous abnormal problems, such as concurrent resource contention, network failures, storage hardware failures, etc., but will also cause frequent request pressure on the server and even affect other metadata operation requests that could have been executed normally; if the retry interval is too long, it will cause the client to respond too slowly, affecting the performance of the application layer; at the same time, retrying at equal time intervals will also cause multiple retry tasks from multiple clients to be sent to the server at the same time in certain scenarios, which may lead to an avalanche effect due to excessive load pressure, such as scheduled tasks set by multiple clients, server failures causing all requests within a period of time to fail, network congestion causing a large number of requests to arrive at the server at the same time, etc.

[0062] To reduce the pressure of invalid requests from the client on the server during the retry process, the client can determine an initial retry time interval that matches the number of request retries when executing the request retry mechanism. The initial retry time interval is positively correlated with the number of request retries, that is, the more request retries, the longer the initial retry time interval. In this way, even if the metadata request does not meet the penalty retry conditions, the client needs to resend the metadata request to the server according to the initial retry time interval that matches the number of request retries. This effectively reduces the number of invalid retries of metadata requests that are prone to failure on the server, thereby reducing the pressure of invalid requests from the client on the server during the retry process.

[0063] In some of these embodiments, the initial retry time interval can be determined by the client by looking up the preset correspondence between the number of retry times and the retry time interval based on the number of requested retries. Specifically, the client can look up the preset correspondence between the number of retry times and the retry time interval based on the number of requested retries, and determine the retry time interval that matches the number of requested retries as the initial retry time interval.

[0064] In some embodiments, the initial retry time interval may be calculated by the client according to the number of request retries and a preset time interval retry algorithm. For example, the client may calculate the initial retry time interval based on an exponential backoff algorithm.

[0065] The penalty time interval is the additional waiting time that the client needs to wait before resending the metadata request based on the initial retry time interval.

[0066] In some embodiments, the penalty time interval can be determined by the client by searching for a preset correspondence between the number of retry attempts and the penalty time interval based on the number of request retries. Specifically, the client can search for a preset correspondence between the number of retry attempts and the penalty time interval based on the number of request retries, and determine a penalty time interval that matches the number of request retries.

[0067] In some embodiments, a calculation function for a penalty time interval is pre-set in the client, and the client may determine the penalty time interval according to the number of request retries and the preset calculation function for the penalty time interval.

[0068] Specifically, after obtaining the number of request retries for a metadata request within a preset sending period, the client can determine whether the metadata request meets the retry-with-penalty condition based on the number of request retries. If it is determined that the metadata request meets the retry-with-penalty condition based on the number of request retries, the initial retry time interval and penalty time interval of the metadata request are determined based on the number of request retries.

[0069] S206: Determine a target retry time interval for the metadata request according to the initial retry time interval and the penalty time interval.

[0070] Specifically, the client may determine the sum of the initial retry time interval and the penalty time interval as the target retry time interval for the metadata request.

[0071] S208: Resend the metadata request to the server according to the target retry time interval.

[0072] Specifically, after determining the target retry time interval for the metadata request, the client may resend the metadata request to the server according to the target retry time interval to perform the metadata request retry operation.

[0073] In the metadata request retry method described above, upon receiving a response from the server regarding a metadata request, the client obtains the number of metadata request retries within a preset retry cycle. If the metadata request satisfies the penalty-based retry condition based on the retry count, the client determines an initial retry interval and a penalty interval for the metadata request based on the retry count. The initial retry interval is positively correlated with the number of metadata request retries. Based on the initial retry interval and the penalty interval, a target retry interval is determined for the metadata request, and the metadata request is resent to the server at the target retry interval. By shifting the execution of the request retry mechanism from the server to the client, the server's computing resources and memory usage can be effectively reduced. Furthermore, by matching the metadata request's retry count with the retry interval that meets the requirements and applying a retry penalty, the client can further reduce the pressure on the server from invalid requests during metadata request retries.

[0074] In some embodiments, as Figure 3 As shown, the metadata request retry method further includes the following steps:

[0075] S302: Obtain the total number of metadata request requests sent to the server within a preset sending period.

[0076] The total number of requests refers to the total number of metadata requests sent by the client to the server within the preset sending period, including successful requests and retries due to failed requests. The total number of requests can be calculated using the client's built-in request count module.

[0077] Specifically, after obtaining the number of request retries for the metadata request, the client may obtain the total number of metadata request requests sent by the client to the server within a preset sending period.

[0078] S304: Determine the ratio of the number of request retries to the total number of requests as the request abnormality ratio.

[0079] Specifically, the client may calculate the ratio of the number of request retries to the total number of requests, and determine the ratio as the request anomaly ratio. The request anomaly ratio may represent the overall anomaly situation in which anomalies occur in metadata requests sent by the client.

[0080] S306: If the request exception ratio is greater than or equal to the penalty ratio threshold, determine that the metadata request meets the penalty retry condition.

[0081] Among them, the penalty ratio threshold is a preset threshold parameter used to determine whether the metadata request meets the penalty retry condition. The penalty ratio threshold can be determined by the designer based on experimental data or empirical data. If the request abnormality ratio is greater than or equal to the penalty ratio threshold, it means that the proportion of client request abnormalities is relatively high. The current distributed storage system has certain faults in metadata request processing, which may be caused by client, server or network factors. That is, the abnormal reason for the failure of the metadata request may be at least one of software design defects, storage device failure, and network failure. In this case, if the request is still sent at the regular frequency and abnormal retries are performed, a large number of metadata operation requests will be retried, resulting in a large amount of resource occupation and waste. Therefore, it is necessary to impose a certain penalty time on the client's request retry until the overall system returns to normal and the proportion of abnormal requests is significantly reduced. At this time, the client can determine that the metadata request meets the penalty retry condition.

[0082] Specifically, after calculating the request exception ratio, the client can compare the request exception ratio with the penalty ratio threshold. If the request exception ratio is greater than or equal to the penalty ratio threshold, it can be determined that the metadata request meets the penalty retry condition.

[0083] In the above embodiment, by calculating the request abnormality ratio of the number of request retries to the total number of requests, the proportion of request abnormalities in the current client can be accurately reflected. The preset penalty ratio threshold can quickly determine whether the proportion of request abnormalities is too high. When it is determined that the proportion of request abnormalities is too high, it is determined that the metadata request meets the penalty retry condition, which effectively improves the accuracy and efficiency of the judgment of the penalty retry condition.

[0084] In some embodiments, when the request exception ratio is less than a penalty ratio threshold, it is determined that the metadata request does not meet the penalty-retry condition.

[0085] In some embodiments, as Figure 4 As shown, determining the initial retry time interval and penalty time interval of the metadata request based on the number of request retries in S204 includes:

[0086] S402: Determine a random jitter value and a basic unit time for time interval calculation according to the number of request retries.

[0087] Random jitter introduces a random factor into the wait time, causing the retry interval to vary. The random jitter value is the factor value of this random factor. Introducing random jitter into the calculation of the initial retry interval can, to a certain extent, prevent the avalanche effect caused by request backlogs. In other words, random jitter can disperse peak retry request rates to a relatively constant rate, reducing network traffic bursts and improving overall system performance.

[0088] The basic unit time is the basic waiting time interval in the exponential backoff algorithm, and represents the basic unit time of the client request retry mechanism.

[0089] Specifically, after obtaining the number of request retries, the client can determine the random jitter value and basic unit time for time interval calculation based on the number of request retries, providing a data basis for subsequent execution of the improved exponential backoff algorithm.

[0090] S404 : Determine an initial retry time interval of the metadata request using an improved exponential backoff algorithm based on the number of request retries, the random jitter value, the basic unit time, and the preset interval growth value.

[0091] The exponential backoff algorithm is an error recovery and retry strategy used in network requests or data transmission. When network transmission errors or congestion occur, the algorithm gradually increases the interval between retry requests to reduce conflicts and retry load, thereby improving system stability and efficiency. The improved exponential backoff algorithm combines the advantages of the equal time interval and exponential backoff algorithms while also introducing random jitter.

[0092] The interval growth rate is the base of the exponential backoff algorithm and determines the interval growth rate. It represents the increase in the waiting time before the next request after each request failure relative to the previous one, typically based on an exponential growth rate based on a fixed base. The interval growth rate can be pre-set by the designer. For example, the designer can select the interval growth rate from a range of values, such as (1, 2).

[0093] Specifically, the client may use an improved exponential backoff algorithm to determine an initial retry time interval for metadata requests based on the obtained request retry count, random jitter value, basic unit time, and a preset interval growth amplitude value.

[0094] In some embodiments, the formula for the improved exponential backoff algorithm is as follows:

[0095]

[0096] Where T represents the initial retry interval, and N represents the number of retries for the current metadata request. As metadata requests accumulate due to abnormal failures, the retry interval increases exponentially, effectively reducing the request pressure on the server. d is the interval growth rate, the base of the exponential backoff algorithm, which determines the interval growth rate. A value typically selected in the range of (1, 2] is sufficient. r is the random jitter value. Adding random jitter prevents avalanche effects while ensuring that each retry interval increases. τ represents the basic unit time of the request retry mechanism.

[0097] S406: Determine a penalty time interval for metadata requests based on the request anomaly ratio.

[0098] Specifically, the client may determine the penalty time interval for metadata requests based on the request anomaly ratio.

[0099] In some embodiments, the client may determine the penalty time interval corresponding to the request abnormality ratio by searching a preset mapping relationship.

[0100] In the above embodiment, the client determines the initial retry time interval of the metadata request by using an improved exponential backoff algorithm. This can prevent the avalanche effect while ensuring that the retry time interval increases each time, effectively reducing the number of times invalid requests impact the service and reducing the pressure on the server.

[0101] In some embodiments, as Figure 5 As shown, S402, determining the random jitter value and basic unit time for time interval calculation according to the number of request retries, includes:

[0102] S502: Randomly select an initial jitter value from a preset random jitter value range.

[0103] The initial jitter value is a value randomly selected from a preset random jitter value range. This random selection ensures the randomness of the random jitter value. The preset random jitter value range can be set by the designer based on experimental data or empirical data. For example, the preset random jitter value range can be (0, 0.2).

[0104] Specifically, the client may randomly select a value from a preset random jitter value range as the initial jitter value.

[0105] S504 , assigning a positive or negative direction to the initial jitter value according to the parity of the request retry times, to obtain a random jitter value for time interval calculation.

[0106] The parity of the request retry count indicates whether the request retry count is an odd or even number. The parity of the request retry count assigns a positive or negative direction to the initial jitter value. This means that if the request retry count is an odd number, the initial jitter value is determined to be positive, and if the request retry count is an even number, the initial jitter value is determined to be negative.

[0107] Specifically, the client may assign a positive or negative direction to the initial jitter value according to the parity of the request retry times, and obtain a random jitter value for time interval calculation.

[0108] Taking the preset random jitter value range of (0, 0.2] as an example, when the request is retried for the first time, the corresponding request retry count is an odd number. The client can select a random value between [80%, 100%) as the random jitter value. When the request is retried for the second time, the corresponding request retry count is an even number. The client can select a random value between (100%, 120%) as the random jitter value. When the request is retried for the third time, the corresponding request retry count is an odd number again. The client can select a random value between [80%, 100%) as the random jitter value. And so on, alternating in sequence.

[0109] S506: Determine a basic unit time for calculating the time interval based on the number of request retries, a preset interval increase value, and a maximum retry time.

[0110] The maximum retry time is a preset waiting time limit for metadata request retries. Setting the maximum retry time can effectively prevent excessive waiting times, avoid slow client responses, and prevent the performance of the application layer from being affected.

[0111] Specifically, the client may determine a basic unit time for calculating the time interval based on the number of request retries, a preset interval growth value, and a maximum retry time.

[0112] In some embodiments, the calculation formula for the basic unit time is as follows:

[0113]

[0114] Where τ represents the basic unit time of the request retry mechanism, d represents the interval growth value, and t represents the maximum retry time.

[0115] In the above embodiment, the initial jitter value is assigned a positive or negative direction according to the parity of the number of request retries, which can prevent the hidden dangers caused by the same basic waiting time for adjacent retries, further avoid the avalanche effect, and ensure that the waiting time interval of each retry request is incremental. The basic waiting time of the odd-numbered and next even-numbered retry requests is the same, which is similar to the retry mechanism with equal time intervals. It can improve the possibility of successful client retry requests to a certain extent. For scenarios where occasional failures will not occur continuously, the client's request response speed can be guaranteed. At the same time, the basic unit time is calculated in combination with the geometric series summation formula. When the retry time interval is calculated based on the improved exponential backoff result, the average expectation is that the maximum number of retries can be completed just within the longest retry time.

[0116] In some embodiments, S406, determining the penalty time interval for metadata requests based on the request anomaly ratio, includes: obtaining a preset penalty factor of the retry penalty mechanism, and multiplying the preset penalty factor by the request anomaly ratio to determine the penalty time interval for metadata requests.

[0117] The preset penalty factor is a preset parameter used to quantify and determine the degree of penalty, which can be pre-determined by the designer according to specific client requirements and configured in the client.

[0118] Specifically, the client may obtain a preset penalty factor of the retry penalty mechanism, and determine the product of the preset penalty factor and the request abnormality ratio as the penalty time interval for the metadata request.

[0119] In the above embodiment, the preset penalty factor specifies the penalty baseline value, and then determines the penalty time interval that matches the current client's request anomaly based on the request anomaly ratio, which effectively improves the matching degree between the penalty time interval and the current client's request anomaly, thereby improving the accuracy of determining the penalty time interval.

[0120] In some embodiments, the metadata request retry method further includes: when the request exception ratio is less than a penalty ratio threshold, determining that the metadata request does not meet the penalty retry condition, and determining the initial retry time interval of the metadata request determined based on the number of request retries as the target retry time interval of the metadata request.

[0121] Specifically, if the request anomaly ratio is less than the penalty ratio threshold, it indicates that the proportion of client request anomalies is not high. The abnormal reasons for the metadata request failure may be due to server busyness, resource contention, concurrent lock conflicts, etc. The abnormality caused by this situation is usually recoverable in a short period of time without human intervention, but the specific recovery time is uncertain. Therefore, there is no need to impose a specific penalty time on the client's request retry. The client can determine the initial retry interval for the metadata request based on the number of request retries and set this initial retry interval as the target retry interval for the metadata request. It is understandable that the steps for determining the initial retry interval are basically the same as those in the above embodiment and will not be repeated here.

[0122] In the above embodiment, when the request anomaly ratio is less than the penalty ratio threshold, the initial retry time interval is directly determined as the target retry time interval for the metadata request, which can reduce the invalid request pressure on the server by the client during the metadata request retry process.

[0123] In the existing technology, usually, a server will process metadata requests from multiple clients. If a server-led retry mechanism is adopted, it is usually necessary to use structures such as queues. The server needs to consume additional computing resources and memory space to maintain the retry mechanism, which increases the pressure on the server. The client-led retry method is not only simple and easy to implement, but can also effectively disperse the operational pressure and resource consumption caused by request retries. Especially in scenarios such as one-write-multiple-read and asymmetric distributed storage systems, it can effectively reduce the load pressure on the server, thereby ensuring that the server provides efficient and reliable metadata processing performance, and has strong robustness in abnormal scenarios.

[0124] The metadata request processing module in the client is responsible for both generating and sending requests, as well as receiving and parsing request response messages. Generally speaking, metadata requests require retrying in two situations: first, if the request response information is parsed and indicates a request processing failure; second, if the request processing timeout occurs after the expected waiting time has passed and no response message is received. In distributed storage systems, communication delays, busy servers, resource contention, concurrent lock conflicts, software design flaws, network failures, and storage device failures are unavoidable. These issues can all lead to metadata operation request retries in either of these situations. Metadata operation request retries are a common and frequent occurrence in large-scale distributed storage systems.

[0125] However, in a client-driven retry mechanism, calculating the timing for request retries (or the interval between retries) in the event of a request exception is challenging. From the client's perspective, it lacks a comprehensive understanding of the server's status, making it difficult to determine whether the server can correctly handle retried requests.

[0126] Currently, the client-led retry mechanism commonly uses a method to set a maximum number of retries or a maximum retry time, and then initiate retry requests at equal time intervals. In this retry mechanism, if the retry interval is too short, it will not only lead to frequent returns of previous exceptions, such as concurrent resource contention, network failures, storage hardware failures, etc., but will also lead to frequent request pressure on the server, and even affect other metadata operation requests that could have been executed normally; if the retry interval is too long, the client will respond too slowly, affecting the performance of the application layer; and retrying at equal time intervals will also cause multiple retry tasks from multiple clients to be sent to the server at the same time in certain scenarios. This may lead to an avalanche effect due to excessive load pressure, such as scheduled tasks set by multiple clients, server failures causing all requests within a period of time to fail, network congestion causing a large number of requests to arrive at the server at the same time, etc.

[0127] Another common approach is to use an exponential backoff algorithm, where the wait time for the nth request is randomly selected from a base wait time (τ) between 0 and 2^(n-1). The exponential backoff algorithm aims to increase the retry interval with the number of retries, reducing the number of ineffective server retries for requests prone to failure. The introduction of randomness can also, to some extent, prevent the avalanche effect caused by request backlogs. However, the algorithm's calculation results contain a large and uncontrollable random factor, potentially resulting in a later request waiting time shorter than the previous one or excessively long retry wait times, leading to poor client performance. Furthermore, the exponential backoff algorithm uses a critical parameter—the base wait time. If a large number of requests arrive in a short period of time and fail to retry due to network congestion or other factors, these requests are likely to be retried simultaneously at the τth, 2τth, or 3τth time points, posing a potential risk.

[0128] In order to solve the above problems, in some embodiments, as Figure 6 As shown, a metadata request retry method is provided, which specifically includes the following steps:

[0129] First, if a metadata request fails, the client should determine the reason for the failure. If the failure is due to a request timeout, it means the client has been waiting for a long time and the cause of the failure is unclear. This is likely due to a network issue, and the client should immediately retry the request without calculating the retry wait interval.

[0130] If the failure reason is not request timeout, the client can obtain the number of request retries N within the preset sending cycle for the metadata request, as well as the total number of metadata requests n sent by the client to the server. t , the total number of requests n t The ratio of the request anomaly ratio to the number of request retries N is determined as the request anomaly ratio. The request anomaly ratio is compared with the penalty ratio threshold. When the request anomaly ratio is greater than or equal to the penalty ratio threshold, it means that the proportion of client request anomalies is relatively high. The client can determine that the cause of the metadata request failure is a software design defect, storage device failure, network failure, etc. In this case, the overall system has already had certain faults and it is difficult to recover in a short time without human intervention. If requests are still sent at the regular frequency and abnormal retries are performed, a large number of metadata operation requests will be retried, resulting in a large amount of resource occupation and waste. Therefore, a certain penalty time needs to be imposed on the client's request retries until the overall system returns to normal and the proportion of abnormal requests is significantly reduced.

[0131] Specifically, the client can obtain a preset interval increment value d and a maximum retry time t, randomly select an initial jitter value from a preset random jitter value range, assign a positive or negative direction to the initial jitter value based on the parity of the request retry count N, and obtain a random jitter value r. The basic unit time τ is calculated based on the request retry count N, the maximum retry time t, and the interval increment value d. The client can use a modified exponential backoff algorithm to calculate the initial retry interval T for metadata requests based on the request retry count N, the interval increment value d, the random jitter value r, and the basic unit time τ.

[0132] The improved exponential backoff algorithm proposed in this embodiment has the following characteristics:

[0133] First, the basic waiting time for the odd-numbered request retry and the next even-numbered request retry is the same, which is similar to the retry mechanism with equal time intervals. This can increase the possibility of successful client retry requests to a certain extent. For scenarios where occasional failures will not occur continuously, the client's request response speed can be guaranteed.

[0134] Second, the random jitter introduced is negative for odd-numbered retries and positive for even-numbered retries, which not only prevents the hidden dangers caused by the same basic waiting time of adjacent retries, further avoids the avalanche effect, but also ensures that the waiting time interval of each retry request is incremental.

[0135] Third, overall, the retry time of a request increases exponentially with the number of retries. This can effectively reduce the impact of invalid requests on the service when requests are unlikely to succeed in certain failure scenarios.

[0136] For example, if the maximum retry time t is 10 seconds, the number of request retries N is 10, and the base d of the exponential backoff algorithm is 1.6, the calculated basic unit time τ of the retry mechanism is approximately 0.316 seconds. The initial retry interval is shown in the following table:

[0137]

[0138] As shown in the table above, the retry time interval is calculated according to the improved exponential backoff algorithm proposed in this embodiment, and the cumulative expected average time is about 10 seconds. In order to more intuitively reflect the value range of each retry calculated, the retry time interval value range of these 10 retries is as follows: Figure 7 shown.

[0139] After obtaining the initial retry time interval T, the client can obtain the preset penalty factor P and determine the penalty time interval T' by multiplying the request abnormality ratio and the penalty factor P, that is:

[0140] ;

[0141] After obtaining the initial retry time interval T and the penalty time interval T', the client can determine the sum of the initial retry time interval T and the penalty time interval T' as the target retry time interval T p .

[0142] If the request anomaly ratio is less than the penalty ratio threshold, it means that the reason for the failure of the metadata request is that the server is busy, resource contention, concurrent lock conflict, etc. The anomalies caused by such situations can usually be recovered in a short period of time and do not require human intervention, but the specific recovery time is uncertain. Therefore, the client can obtain the preset interval growth amplitude value d and the maximum retry time t, randomly select the initial jitter value from the preset random jitter value range, and assign positive and negative directions to the initial jitter value according to the parity of the request retry number N to obtain the random jitter value r. The basic unit time τ is calculated based on the request retry number N, the maximum retry time t and the interval growth amplitude value d. Using the improved exponential backoff algorithm, the initial retry time interval T of the metadata request is calculated based on the request retry number N, the interval growth amplitude value d, the random jitter value r and the basic unit time τ, and the initial retry time interval T is determined as the target retry time interval T p .

[0143] The above embodiment proposes a metadata request retry method led by the client, which is simple and easy to implement. Compared with the existing common server-side retry method relying on queues, it does not require complex data structures and software logic, reduces the consumption of computing resources and memory space on the server, and enables the server to provide more efficient and reliable metadata management and operation performance. When performing the initial retry time interval calculation, an improved exponential backoff algorithm is used, and random jitter is introduced. This can not only ensure that the retry time interval increases exponentially with the number of request retries, but also avoid the avalanche effect caused by a large number of requests at the same time. At the same time, the proportion of request anomalies is counted on the client, and a retry penalty time is applied accordingly, thereby reducing resource occupation and waste in the event of a system failure.

[0144] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0145] Based on the same inventive concept, embodiments of the present application also provide a metadata request retry device for implementing the metadata request retry method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more metadata request retry device embodiments provided below can be found in the limitations of the metadata request retry method described above and will not be further elaborated here.

[0146] In some embodiments, as Figure 8 As shown, a metadata request retry device 800 is provided, comprising: a request retry count acquisition module 801, a condition judgment module 802, a target retry time interval determination module 803, and a request retry module 804, wherein:

[0147] The request retry count acquisition module 801 is configured to acquire the request retry count of the metadata request within a preset sending period upon receiving the request return information sent by the server in response to the metadata request.

[0148] Condition judgment module 802 is used to determine an initial retry time interval and a penalty time interval for the metadata request based on the number of request retries if the metadata request satisfies the penalty retry condition based on the number of request retries; the initial retry time interval is positively correlated with the number of request retries.

[0149] The target retry time interval determination module 803 is configured to determine a target retry time interval for the metadata request according to the initial retry time interval and the penalty time interval.

[0150] The request retry module 804 is configured to resend the metadata request to the server according to a target retry time interval.

[0151] In some embodiments, the metadata request retrying means further comprises:

[0152] The total request times acquisition module is used to obtain the total number of requests for metadata requests sent to the server within a preset sending period.

[0153] The request abnormality ratio determination module is used to determine the request abnormality ratio by taking the ratio of the number of request retries to the total number of requests.

[0154] The threshold comparison module is used to determine that the metadata request meets the penalty retry condition when the request abnormality ratio is greater than or equal to the penalty ratio threshold.

[0155] In some embodiments, the conditional judgment module 802 is used to: determine the random jitter value and basic unit time for time interval calculation based on the number of request retries; use an improved exponential backoff algorithm to determine the initial retry time interval of the metadata request based on the number of request retries, the random jitter value, the basic unit time, and a preset interval growth amplitude value; and determine the penalty time interval of the metadata request based on the request abnormality ratio.

[0156] In some embodiments, the conditional judgment module 802 is used to: randomly select an initial jitter value from a preset random jitter value range; assign a positive or negative direction to the initial jitter value according to the parity of the number of requested retries to obtain a random jitter value for time interval calculation; and determine the basic unit time for time interval calculation based on the number of requested retries, the preset interval growth amplitude value, and the longest retry time.

[0157] In some embodiments, the condition determination module 802 is configured to: obtain a preset penalty factor of the retry penalty mechanism; and determine the product of the preset penalty factor and the request abnormality ratio as the penalty time interval for the metadata request.

[0158] In some embodiments, the target retry time interval determination module is further used to: determine that the metadata request does not meet the penalty retry condition when the request exception ratio is less than the penalty ratio threshold; and determine the initial retry time interval of the metadata request determined based on the number of request retries as the target retry time interval of the metadata request.

[0159] Each module in the metadata request retry apparatus described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in hardware form, or may be stored in a memory in the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0160] In some embodiments, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as request return information, number of request retries, retry conditions with penalties, initial retry time interval, penalty time interval, target retry time interval, etc. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a metadata request retry method is implemented.

[0161] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0162] In some embodiments, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the specific steps of the above-mentioned metadata request retry method embodiment when executing the computer program.

[0163] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the specific steps of the above-mentioned metadata request retry method embodiment are implemented.

[0164] In some embodiments, a computer program product is provided, including a computer program, which, when executed by a processor, implements the specific steps of the above-mentioned metadata request retry method embodiment.

[0165] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage, and display, etc.) involved in this application are all authorized by the user or have been fully authorized by all parties. Furthermore, the acquisition, storage, processing, and transmission of this data comply with relevant laws and regulations.

[0166] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0167] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0168] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A metadata request retry method, characterized in that: The method comprises: Upon receiving a request response message sent by the server in response to the metadata request, obtaining a request retry count of the metadata request within a preset sending period; If it is determined that the metadata request meets the penalty retry condition based on the request retry count, then determining an initial retry time interval and a penalty time interval for the metadata request based on the request retry count; the initial retry time interval is positively correlated with the request retry count; determining a target retry time interval for the metadata request based on the initial retry time interval and the penalty time interval; Resending the metadata request to the server according to the target retry time interval; Obtain the total number of metadata requests sent to the server within the preset sending period; Determine the ratio of the number of request retries to the total number of requests as the request abnormality ratio; If the request exception ratio is greater than or equal to the penalty ratio threshold, determining that the metadata request meets the penalty retry condition; The determining, based on the number of request retries, an initial retry time interval and a penalty time interval for the metadata request, includes: Determining a random jitter value and a basic unit time for calculating a time interval according to the number of request retries; Determining an initial retry time interval for the metadata request using an improved exponential backoff algorithm based on the request retry count, the random jitter value, the basic unit time, and a preset interval growth amplitude value; Determine a penalty time interval for the metadata request based on the request anomaly ratio.

2. The method according to claim 1, characterized in that The determining, according to the number of request retries, a random jitter value and a basic unit time for calculating a time interval includes: Randomly select an initial jitter value from a preset random jitter value range; Assigning a positive or negative direction to the initial jitter value according to the parity of the request retry number to obtain a random jitter value for time interval calculation; Based on the number of request retries, a preset interval increase value, and a maximum retry time, a basic unit time for calculating the time interval is determined.

3. The method according to claim 1, characterized in that The determining, based on the request anomaly ratio, a penalty time interval for the metadata request includes: Get the preset penalty factor of the retry penalty mechanism; The product of the preset penalty factor and the request abnormality ratio is determined as the penalty time interval of the metadata request.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: If the request exception ratio is less than the penalty ratio threshold, determining that the metadata request does not meet the penalty-retry condition; An initial retry time interval of the metadata request determined based on the number of request retries is determined as a target retry time interval of the metadata request.

5. A metadata request retry device, characterized in that: The device comprises: A request retry count acquisition module, configured to, upon receiving a request response message sent by the server in response to a metadata request, acquire the request retry count of the metadata request within a preset sending period; a condition judgment module, configured to, if it is determined based on the request retry count that the metadata request satisfies a retry condition with a penalty, determine an initial retry time interval and a penalty time interval for the metadata request based on the request retry count; wherein the initial retry time interval is positively correlated with the request retry count; a target retry time interval determining module, configured to determine a target retry time interval for the metadata request based on the initial retry time interval and the penalty time interval; A request retry module, configured to resend the metadata request to the server according to the target retry time interval; A total request count acquisition module is used to acquire the total number of metadata request times sent to the server within the preset sending period; a request abnormality ratio determining module, configured to determine a ratio of the number of request retries to the total number of requests as a request abnormality ratio; A threshold comparison module is configured to determine that the metadata request satisfies a retry condition with penalty if the request exception ratio is greater than or equal to a penalty ratio threshold; The conditional judgment module is used to determine the random jitter value and basic unit time for time interval calculation based on the number of request retries; determine the initial retry time interval of the metadata request using an improved exponential backoff algorithm based on the number of request retries, the random jitter value, the basic unit time, and a preset interval growth amplitude value; and determine the penalty time interval of the metadata request based on the request abnormality ratio.

6. The device according to claim 5, characterized in that The conditional judgment module is used to: randomly select an initial jitter value from a preset random jitter value range; assign a positive or negative direction to the initial jitter value based on the parity of the number of requested retries to obtain a random jitter value for time interval calculation; and determine the basic unit time for time interval calculation based on the number of requested retries, the preset interval growth amplitude value, and the maximum retry time.

7. The device according to claim 5, characterized in that The condition judgment module is used to: obtain a preset penalty factor of the retry penalty mechanism; and determine the product of the preset penalty factor and the request abnormality ratio as the penalty time interval of the metadata request.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images