Error reporting request scheduling method and system for distributed application

Abstract

This specification discloses an error request scheduling method and system for distributed applications. The method includes a load balancing layer responding to a client's request by distributing the request to a first gateway node in the gateway layer. If an error matching a first set of fault types occurs during request distribution at the first gateway node, the request is sequentially distributed to a second gateway node in the gateway layer until a successful request distribution occurs at a target second gateway node. The target second gateway node in the gateway layer sends the request to a first microservice instance in the microservice layer. If an error matching a second set of fault types occurs during a call to the first microservice instance, the request is sequentially sent to a second microservice instance in the microservice layer until a successful call occurs at a target second microservice instance. In this embodiment, a multi-level collaborative fault-tolerance system is constructed, enabling proactive fault-tolerant scheduling of error requests, minimizing the unavailability of the business system and improving the overall user experience.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of cloud computing technology, and specifically relate to a method and system for scheduling error requests in distributed applications. Background Technology

[0002] In current software architectures, distributed systems and microservices are widely used in business scenarios with high traffic and high concurrency. These architectures, by introducing redundant nodes and load balancing mechanisms (such as Nginx and microservice gateways), theoretically greatly improve the system's fault tolerance and overall availability. Specifically, the load balancer can dynamically distribute external requests to multiple backend service instances, effectively distributing system pressure. Furthermore, when an abnormal node's health status is detected, it can be temporarily removed from the service cluster, thus avoiding single points of failure and ensuring the stability of the overall architecture.

[0003] However, most existing load balancing solutions rely on passive node removal, such as using heartbeat detection to determine whether a node is available. However, they lack proactive and rapid error correction capabilities and are unable to quickly avoid errors in the current request when a failure occurs, resulting in a poor user experience in some situations. Summary of the Invention

[0004] The embodiments of this disclosure provide a method and system for scheduling error requests in distributed applications, which aims to solve one or more of the above-mentioned problems and other potential problems.

[0005] According to a first aspect of this disclosure, a method for scheduling error requests in a distributed application is provided. The method includes a load balancing layer responding to a client's request by distributing the request to a first gateway node in a gateway layer. When an error matching a first set of fault types occurs during request distribution at the first gateway node, the request is sequentially distributed to a second gateway node in the gateway layer based on a preset first retry waiting time, until a request is successfully distributed to a target second gateway node or the load balancing layer's stop-retry condition is met. The second gateway node is a healthy gateway node other than the first gateway node, and the first fault type is a specified fault type among the fault types corresponding to the gateway node. The target second gateway node in the gateway layer... The node sends the request to the first microservice instance in the microservice layer. When an error matching the second fault type set occurs in the call of the first microservice instance, the request is sent sequentially to the second microservice instance in the microservice layer based on the second retry wait time, until a target second microservice instance is successfully called or the stop retry condition of the gateway layer is met. The second microservice instance is a healthy microservice instance other than the first microservice instance. The second retry wait time is set based on the exponential backoff algorithm. The total number of retries between the gateway node and the microservice instance does not exceed the total number threshold and the total retry time does not exceed the total duration threshold. The second fault type is the specified fault type among the fault types corresponding to the microservice instance.

[0006] According to a second aspect of this disclosure, a distributed application error request scheduling system is provided. The system includes a client for generating requests; one or more load balancers configured to perform load balancing layer operations in the method provided in the first aspect; one or more API gateways configured to perform gateway layer operations in the method provided in the first aspect; a microservice system configured with multiple microservice instance nodes; and a continuous integration / continuous deployment system configured to perform release process layer operations in the method provided in the first aspect.

[0007] According to a third aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to the first aspect.

[0008] The solution provided in the embodiments of this specification can construct a multi-level collaborative fault-tolerant system by introducing proactive retry mechanisms based on different fault types and retry waiting times at the load balancing layer and gateway layer, respectively, combined with constraints on the total number of retries and the total retry duration. This enables proactive fault-tolerant scheduling of erroneous requests, ensuring that the entire call chain still has elastic recovery capabilities when facing intermittent faults. At the same time, it effectively prevents resource exhaustion caused by retries, making the unavailability of the business system approach zero, and improving the overall user experience. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 This invention illustrates the architecture of an error request scheduling system for a distributed application according to some embodiments of the present disclosure.

[0011] Figure 2 A flowchart illustrating an error request scheduling method for a distributed application according to some embodiments of this disclosure is shown.

[0012] Figure 3 A flowchart illustrating an error request scheduling method for a distributed application according to further embodiments of this disclosure is shown.

[0013] Figure 4 A flowchart illustrating an error request scheduling method for a distributed application according to further embodiments of this disclosure is shown.

[0014] Figure 5 A schematic block diagram of an electronic device according to some embodiments of the present disclosure is shown. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0016] The terms “comprising” and “having”, and any variations thereof, in this specification, claims, and the foregoing drawings are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. Depending on the context, the word “if” as it applies herein may be interpreted as “when”, “in response to determination”, or “in response to detection”.

[0017] As mentioned earlier, although the existing architecture has good fault tolerance, in actual production environments, there are still some situations that can cause platform problems and affect user experience. Two typical scenarios that can lead to brief business request failures are as follows: 1. Transient single point of failure: When a backend service node suddenly fails due to unpredictable reasons such as high CPU load, host resource contention, or network jitter, but the load balancer has not yet marked it as "offline" through health checks, all requests sent to the failed node will fail within a very short period. 2. Service restart: In application update and release scenarios, multiple nodes of the same service will restart sequentially during the rolling update process. During this period, if the load balancer is not configured to retry requests, or if the retry strategy is inappropriate, requests sent to these restarting nodes will return errors, causing the service to be unavailable for a short time. It can be seen that the reason for the above problems is that the current solution relies on passive node removal. Therefore, a solution with proactive and rapid error correction capabilities is needed to address such issues.

[0018] Figure 1 A schematic diagram of the architecture of an error request scheduling system 100 for a distributed application, representing some embodiments of this disclosure, is shown. For example... Figure 1As shown, system 100 includes a client 110, one or more load balancers 120, one or more API gateways 130, a microservice system 140, and a continuous integration / continuous deployment system 150. The client 110 may include, but is not limited to, mobile phones, tablets, desktop computers, servers, etc., and is used to generate corresponding requests based on user operations on the client. The load balancer 120 can constitute a load balancing layer, used to receive requests sent by the client, distribute the requests to available gateway nodes in the gateway layer, and trigger an active retry mechanism when a gateway node encounters an error of the first set of fault types, autonomously selecting other available gateway nodes for distribution until distribution is successful or its corresponding stop retry condition is met. The API gateway 130 can constitute a gateway layer, used to receive requests distributed by the load balancing layer and distribute the requests to microservice instances in the microservice system 140. If a microservice instance encounters an error of the second set of fault types during the call process, an active retry mechanism is triggered, autonomously selecting other available microservice instances for call until the call is successful or its corresponding stop retry condition is met. The continuous integration / continuous deployment system 150 can form a release process layer. When rolling updates to microservice instances are needed, it determines the node update batches based on release time limits and node restart times, thus updating microservice instance nodes in batches to avoid service unavailability caused by simultaneous updates to all nodes. Through this four-layer architecture design, the load balancing layer can act as the first line of defense, specifically handling the first type of failure—that is, it doesn't concern itself with business semantics and is only responsible for rapid failover at the gateway node. The gateway layer acts as the second line of defense, specifically handling the second type of failure—that is, triggering fine-grained retry strategies for specific error codes based on business awareness. Since the microservice system does not need to implement any retry logic, it can maintain the purity of its business logic. In contrast to the conventional approach of uniformly implementing retry mechanisms on the client or a single gateway, which leads to chaotic fault handling, this application vertically divides retry responsibilities according to fault type. This allows each layer to only perceive and handle typical faults within its own layer, significantly reducing the risk of numerous retries. Furthermore, each layer's retry strategy can be independently optimized without interference, and the client still only perceives that it sent the original request once, without adding any extra burden to the client. In addition, the continuous integration / continuous deployment system can architecturally couple the release batch calculation logic with the runtime retry capabilities of the other three layers, enabling the update and release process to naturally adapt to the underlying fault self-healing mechanism, ensuring high availability of the release process. Through this architecture, the coupling challenge of handling instantaneous faults and smooth releases in distributed systems can be effectively solved.

[0019] Figure 2 A flowchart illustrating an error request scheduling method 200 for a distributed application according to some embodiments of this disclosure is shown. Figure 2As shown, in method 200, step 202 can be that the load balancing layer responds to the client's request, distributes the request to the first gateway node of the gateway layer, and when the request distribution of the first gateway node encounters an error that matches the first fault type set, the request is distributed sequentially to the second gateway node of the gateway layer based on a preset first retry waiting time, until the request distribution of the target second gateway node is successful or the stop retry condition of the load balancing layer is met. The second gateway node is a healthy gateway node other than the first gateway node, and the first fault type is a specified fault type among the fault types corresponding to the gateway node.

[0020] In this embodiment, after receiving a request from a client, the load balancing layer can respond to the request by selecting a first gateway node from the current healthy gateway nodes and distributing the request to that first gateway node. The selection of the first gateway node can be random or based on the current load of each healthy gateway node, choosing the one with the lowest load. If the first gateway node experiences a brief unavailability due to a momentary single point of failure, service restart, or other reasons, causing request distribution failure, the type of error will be determined. If the error type matches a preset first set of error types (e.g., connection error, response timeout, invalid response header, server returning a 500 status code, etc.), it is considered that the current failure is indeed likely a temporary unavailability, and a request retry strategy will be automatically activated. A second gateway node will be selected from the remaining healthy gateway nodes in the same manner for request distribution. If the second gateway node still reports an error, the remaining second gateway nodes will continue to be selected until a successful request distribution occurs at a target second gateway node, or the entire retry process has met the load balancing layer's stop-retry condition. Conditions for stopping retrying may include, for example, the maximum number of retries and the maximum cumulative retry duration for all retry operations, to prevent system avalanche caused by infinite retries. If the conditions for stopping retrying are met, a formal error message indicating that the service is unavailable can be sent to the client. The retry process can be based on a preset first retry waiting time; that is, if no response is received from the gateway node within the first retry waiting time, the response is considered timed out, and the next gateway node is selected for retrying.

[0021] As an example, taking Nginx as the load balancing component in the load balancing layer, the `proxy_next_upstream` directive can be configured, defining a first set of failure types, such as error (connection error), timeout (response timeout), invalid header (invalid response header), and HTTP 500 (server returns a 500 status code). When these specific failures are encountered, requests will be automatically redistributed to the next healthy gateway node in the gateway layer. The `proxy_next_upstream_timeout` parameter can be used to define the retry wait time, and the `proxy_next_upstream_tries` parameter can be used to limit the maximum number of retries. For example, if the retry wait time is 5ms and the maximum number of retries is 3, then when node A encounters a specific failure, it will be redistributed to node B after 5ms. If node B also fails, it will be redistributed to node C, and so on, with a maximum of 3 retries. If all 3 retries fail, a failure error will be returned. In this way, when a request is routed to a faulty node, the load balancer can complete a secondary route for the same request within milliseconds. As long as there is any available node in the cluster, the user request can be successfully processed. The impact on the user experience is that the access time is slightly longer, but it is generally only a matter of seconds and will not significantly affect the user experience.

[0022] In method 200, step 204 allows the target second gateway node of the gateway layer to send a request to the first microservice instance of the microservice layer. When an error matching the second fault type set occurs in the call of the first microservice instance, the request is sequentially sent to the second microservice instance of the microservice layer based on the second retry waiting time until a target second microservice instance successfully calls or the stop retry condition of the gateway layer is met. The second microservice instance is a healthy microservice instance other than the first microservice instance. The second retry waiting time is set based on the exponential backoff algorithm. The total number of retries of the gateway node and the microservice instance does not exceed the total number threshold and the total retry time does not exceed the total duration threshold. The second fault type is a specified fault type among the fault types corresponding to the microservice instance.

[0023] In this embodiment, after a request is distributed from the load balancing layer to the target second gateway node or the first gateway node in the gateway layer, the gateway node will select a first microservice instance in the microservice layer for invocation to perform business processing according to the request. The selection method of the first microservice instance can be consistent with the selection method of the first gateway node. In addition to the retry fault tolerance mechanism of the load balancing layer, the call chain of the microservice layer may also experience temporary unavailability, so a corresponding retry mechanism is also needed for fault tolerance enhancement. When the call of the first microservice instance encounters an error matching the second fault type set, such as an abnormal HTTP status code or a specified IO exception, the second microservice instance will be selected for retry based on the second retry waiting time until a target second microservice instance can be selected for invocation, or the stop retry condition of the gateway layer is met. The stop retry condition of the gateway layer can be similar to the stop retry condition of the load balancing layer, that is, setting the corresponding maximum number of retries and the maximum cumulative retry time of all retry operations, but the specific values ​​of the two can be set according to the actual configuration of the system and are not necessarily the same as the stop retry condition of the load balancing layer.

[0024] The second retry wait time for a single request can be set based on an exponential backoff algorithm. This means the retry wait time increases exponentially with the number of retries, significantly increasing the wait time with each retry. This is because gateway layer connection failures are typically caused by network jitter, node restarts, or ports not listening. For the load balancer, it doesn't need to concern itself with the internal business state of nodes; it only needs to determine if the gateway can establish a connection to quickly complete the initial request distribution and avoid rapidly exhausting the connection pool, causing ingress congestion. Therefore, retrying can be performed directly based on the set first retry wait time. However, call failures at microservice instances are generally business-related. Compared to the load balancer, the reasons for failures here are less certain. For example, the microservice instance might be temporarily busy processing other requests or temporarily stuck, and it might recover after a short wait. Furthermore, microservice instances may also call each other, meaning that even with rapid retries, other microservice instances might still fail due to mutual calls or receiving too many requests from other gateway nodes simultaneously. Therefore, the exponential backoff algorithm will be used here to apply negative feedback to handle instantaneous high loads, which can complete the call more efficiently.

[0025] As an example, a "Retry" filter can be configured in the Spring Cloud Gateway API gateway. Similar to load balancing retries, when microservice A fails, it is resent to microservice B based on the second retry wait time determined by the exponential backoff algorithm. Specifically, the "Retry" filter can add dedicated parameters for exponential backoff: `backoff.firstBackoff` (the initial delay time for the first retry), `backoff.maxBackoff` (the maximum backoff time), `backoff.factor` (the exponential factor, usually 2), and `backoff.basedOnPreviousValue` (standard exponential backoff / considering the previous delay value).

[0026] Furthermore, when the load balancing layer and the gateway layer coordinate retries, the overlapping of their retries needs to be considered. For example, assuming the load balancer retryes 3 times and the gateway retryes 3 times, the maximum possible retries are 9. This could lead to an excessively long final response time for the client due to the cumulative number of retries. Therefore, it is necessary to set appropriate thresholds to constrain the total number of retries and the total retry duration (i.e., the total duration after summing the waiting time for each retry) for the gateway node and microservice instances. This allows for two options: either ensuring that the total number of retries does not exceed the total threshold when initially setting the retries for the load balancer and gateway, or setting the maximum cumulative number of retries to exceed the total threshold when initially setting the retries for the load balancer and gateway, but in practice, using the total threshold as the constraint—that is, stopping retries once the actual total number of retries reaches the threshold—to prevent excessively long client response times due to the cumulative number of retries by strictly limiting the total number of retries and the total retry duration.

[0027] By implementing a retry mechanism at the gateway node, the fault tolerance gap between the load balancer and the service mesh is filled, ensuring that the entire call chain from the gateway entry point to the internal microservices has elastic recovery capabilities. Through this setup, the load balancing layer and the gateway layer form a multi-level collaborative fault tolerance system, constructing a seamless collaborative protection mechanism against business interruptions. Furthermore, by limiting the total duration threshold and the total number of retries, the maximum number of collaborative attempts and the maximum duration are restricted, significantly improving system robustness and user experience. The collaborative retry mechanism is autonomously completed with almost no user awareness, minimizing the actual failure of user requests.

[0028] In one possible implementation, the stop retry conditions of the load balancing layer include a first maximum number of retries and a first maximum cumulative retry duration, and the stop retry conditions of the gateway layer include a second maximum number of retries and a second maximum cumulative retry duration. The sum of the first maximum cumulative retry duration and the second maximum cumulative retry duration is a total duration threshold. The product of the first maximum number of retries and the second maximum number of retries is not greater than the total number of retries threshold. The first maximum number of retries is determined based on a first fault type, and the first maximum cumulative retry duration is determined based on the single retry waiting time strategy corresponding to the first fault type.

[0029] In this embodiment, when setting the corresponding maximum number of retries and maximum cumulative retry time for the load balancing layer and the gateway layer, the sum of the first maximum cumulative retry time and the second maximum cumulative retry time can be set as the total time threshold. This ensures that even if the cumulative retry time of each layer reaches the preset maximum value during the collaboration process between the two layers, it will not exceed the total time threshold, i.e., it will not lead to excessively long response times. Otherwise, the first maximum cumulative retry time and / or the second maximum cumulative retry time will be shortened during the initial setup. In addition, after the initial conditions are set, the retry stop condition can be dynamically adjusted adaptively according to changes in the actual situation. For example, when the request distribution of the load balancing layer encounters an error of the first fault type set, it is first determined which specific first fault type it belongs to. Different first fault types can be pre-assigned different first maximum number of retries (e.g., 3 retries for network errors, 2 retries for server errors, etc.). In this way, the maximum number of retries can be dynamically adjusted according to the severity of the specific first fault type, so that the retry process will not perform too many retries on nodes that are difficult to distribute successfully, nor will it give up retries too quickly on nodes that are easy to redistribute successfully. Meanwhile, different retry wait duration strategies can be pre-set for different first fault types. These strategies include an initial value for the retry wait duration and rules for each retry. For example, an initial value of 5ms, where the time is doubled for each new retry; or an initial value of 6ms, where the retry wait duration remains unchanged for each new retry. This allows for the allocation of different strategies based on the severity of different first fault types, thereby dynamically adjusting the first maximum cumulative retry duration (generally calculated based on the determined first maximum number of retries and the single retry wait duration strategy). This, in turn, adjusts the second maximum cumulative retry duration accordingly. This prevents excessively long client response times while dynamically adjusting specific retry parameters based on different fault types. It avoids overly rigid retry conditions that could prevent retry failures from being successfully distributed / invoked, thus improving the success rate of collaborative retries.

[0030] In one possible implementation, when an error matching a second set of fault types occurs during a call to the first microservice instance, the request is sequentially sent to the second microservice instance in the microservice layer based on a second retry wait duration, including:

[0031] When an error matching the second set of fault types occurs during a call to the first microservice instance, and the error operation satisfies the idempotency condition, the request is sequentially sent to the second microservice instance in the microservice layer based on the second retry wait time.

[0032] In this embodiment, idempotency means that an operation produces the same result whether it is executed once or multiple times. Taking the second fault type as an HTTP status code exception as an example, common HTTP methods include GET (reading data, does not affect status), POST (creating / modifying operations, multiple operations may create the same data repeatedly), and DELETE (deleting data, repeated deletion may fail). Generally, only GET operations are idempotent, and idempotent operations are usually retried. Therefore, after an error matching the second fault type set occurs, it will be further determined whether the error operation corresponding to the error of the second fault type satisfies the idempotency condition, that is, whether the operation is idempotent. Only if the idempotency condition is met will a retry be performed.

[0033] In one possible implementation, the method further includes:

[0034] The circuit breaker triggers the circuit breaker when the number of consecutive error reports for a target object within a preset time period exceeds the error count threshold.

[0035] In this embodiment, in addition to setting the total retry waiting time to ensure that the final response time of a single round of requests is within an acceptable range, a circuit breaker can also be set up to work in conjunction with the retry mechanism. When the target object (usually a microservice instance) experiences consecutive errors within a preset time period, and the number of consecutive errors exceeds a preset error count threshold, the target object will be quickly circuit-broken to avoid resource exhaustion.

[0036] Figure 3 A flowchart illustrating an error request scheduling method 300 for a distributed application according to further embodiments of this disclosure is shown. Figure 3As shown, in method 300, step 302 can be that the load balancing layer responds to the client's request, distributes the request to the first gateway node of the gateway layer, and when the request distribution of the first gateway node encounters an error that matches the first fault type set, the request is distributed sequentially to the second gateway node of the gateway layer based on a preset first retry waiting time, until the request distribution of the target second gateway node is successful or the stop retry condition of the load balancing layer is met. The second gateway node is a healthy gateway node other than the first gateway node, and the first fault type is a specified fault type among the fault types corresponding to the gateway node.

[0037] In this embodiment, step 302 can be referred to step 202, and will not be repeated here.

[0038] In method 300, step 304 allows the target second gateway node of the gateway layer to send a request to the first microservice instance of the microservice layer. When an error matching the second fault type set occurs in the call of the first microservice instance, the request is sequentially sent to the second microservice instance of the microservice layer based on the second retry waiting time until a target second microservice instance successfully calls or the stop retry condition of the gateway layer is met. The second microservice instance is a healthy microservice instance other than the first microservice instance. The second retry waiting time is set based on the exponential backoff algorithm. The total number of retries between the gateway node and the microservice instance does not exceed the total number threshold and the total retry time does not exceed the total duration threshold. The second fault type is a specified fault type among the fault types corresponding to the microservice instance.

[0039] In this embodiment, step 304 can refer to step 204, and will not be repeated here.

[0040] In method 300, step 306 can publish the process layer response to the rolling update instruction, determine the release limit duration corresponding to the rolling update instruction, determine the node update batch based on the release limit duration and the preset node restart duration, and perform batch updates on the microservice instances of the microservice layer based on the node update batches. The node update batches are not less than two batches, and the node restart duration is not less than the complete cycle duration of the node restart.

[0041] In this embodiment, a continuous integration / continuous deployment system can be used as the release process layer, and this layer can be further coordinated with the load balancing layer and gateway layer to build a collaborative scheduling scheme. Specifically, to address service interruptions caused by planned application releases, intelligent scheduling strategies need to be introduced into the continuous integration / continuous deployment process. The existing continuous integration / continuous deployment process follows a chain of code packaging - distribution to the server - stopping the original service program - starting the new service. During the release process, all nodes of a certain service may be updating, causing the service to be unavailable for a short period of time. Therefore, in this application, if the release process layer detects a rolling update instruction, it first determines the release limit duration, that is, determines the release time limit for this release and how long the release needs to be completed. In addition, a node restart duration is pre-set. This duration is the time delay interval for forcibly setting node restarts in the release pipeline. It needs to be strictly evaluated and set in advance to ensure that it is greater than the complete cycle time of a single node from stopping, updating, restarting, completing health checks, to being re-identified as the current state by the load balancer. For example, it can be set by adding a certain amount of redundancy time to the complete cycle time. The node update batch can be calculated based on the release time limit and node restart time. For example, if the release time limit is 1 minute and the node restart time is 30 seconds, dividing the two gives a node update batch of 2. Next, each microservice instance in the microservice layer is assigned to a specific node update batch, and updates are performed in batches according to the batches. The assignment method can be average or random, and each batch must contain at least one microservice instance. Furthermore, in special cases, such as a release time limit of 1 minute and a node restart time of 60 seconds, the determined node update batch is 1. However, to ensure that at least one node is available during updates, it is necessary to force at least two update batches to ensure that at least one node is online.

[0042] If a release failure is detected during the release process, it can be automatically rolled back to the previous version to prevent service crashes.

[0043] By using the above methods, it can be ensured from the source of operation and maintenance that at any time during rolling release, the microservice layer retains at least a sufficient number of online nodes to normally handle access traffic. By combining this with the aforementioned retry mechanism, smooth release and rolling updates can be achieved without the user's awareness.

[0044] In one possible implementation, before determining the node update batch based on the release restriction duration and the preset node restart duration, the method further includes:

[0045] Query the number of healthy microservice instances in the microservice layer at the current moment. If the number of instances is less than the threshold, restart the abnormal microservice instances until the number of instances is not less than the threshold.

[0046] In this embodiment, in order to perform updates in batches, it is necessary to ensure that there are enough healthy microservice instances (i.e., nodes with normal service status). Therefore, the number of healthy microservice instances at the current moment will be queried first. If the number of instances is small, abnormal microservice instances will be restarted first. After ensuring that there are enough healthy microservice instances, the process of updating in batches will be carried out.

[0047] In one possible implementation, the method further includes:

[0048] The client calls the third microservice instance stored locally based on the request. When the call to the third microservice instance encounters an error that matches the second set of fault types, the client sends the request to each fourth microservice instance in sequence based on the second retry waiting time until a target fourth microservice instance is successfully called. The fourth microservice instance is a healthy microservice instance stored locally other than the third microservice instance.

[0049] In this embodiment, besides the client sending requests to the load balancing layer, a client-side load balancing approach can also be adopted, independent of a centralized load balancer. Specifically, the client can use service discovery mechanisms (such as Nacos, Eureka, etc.) to locally cache a list of available microservice instances and integrate gateways and retry components (such as Spring Retry). This way, when a call fails, the client can directly and quickly select another instance for retry. This approach reduces the number of service proxy layers, lowers the single-point pressure and bandwidth bottleneck of the central load balancer, avoids additional latency through the central node, and provides faster fault-tolerant responses. However, this solution requires integrating a corresponding SDK into each client, increasing application complexity and making it less of a preferred choice.

[0050] Figure 4 A flowchart illustrating an error request scheduling method 400 for a distributed application according to further embodiments of this disclosure is shown. Figure 4 As shown, in method 400, step 402 can be that the load balancing layer responds to the client's request and distributes the request to the first gateway node of the gateway layer. When the request distribution of the first gateway node encounters an error that matches the first fault type set, the request is distributed sequentially to the second gateway node of the gateway layer based on a preset first retry waiting time, until the request distribution to the target second gateway node is successful or the stop retry condition of the load balancing layer is met. The second gateway node is a healthy gateway node other than the first gateway node, and the first fault type is a specified fault type among the fault types corresponding to the gateway node.

[0051] In this embodiment, step 402 can be referred to step 202, and will not be repeated here.

[0052] In method 400, step 404 allows the target second gateway node of the gateway layer to submit the request to a message queue, where the request is used to be pulled and processed by the healthy microservice instance.

[0053] In this embodiment, for non-real-time business scenarios that allow asynchronous processing, the synchronous call mode can be changed to be processed through a message queue. After receiving a request, the gateway layer submits it as a persistent message to a message queue (such as RabbitMQ, Apache Kafka, RocketMQ, etc.), allowing healthy backend microservice instances to act as consumers, pulling tasks from the queue and processing them. In this way, the failure or restart of a single instance node only affects its own consumption capacity; new messages will be consumed by other healthy microservice instances. This solution can ignore request failures caused by node restarts, achieve complete decoupling between the service provider and the consumer, and has the ability to smooth traffic peaks and valleys, improving the system's throughput and resilience. However, because this also increases the complexity of the system architecture, the increase in request response latency, and the need to maintain the high availability of the message queue, it is only suitable for specific business scenarios and is not the first choice in general scenarios.

[0054] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this specification is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0055] Figure 5 A block diagram of an electronic device 500 that can implement various embodiments of the present disclosure is shown. For example... Figure 5As shown, the electronic device 500 includes a processor 510, a disk drive 520, an input / output interface 530, a network interface 540, and a memory 550. The processor 510, disk drive 520, input / output interface 530, network interface 540, and memory 550 can communicate with each other via a communication bus 560.

[0056] The processor 510 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0057] The memory 550 can be implemented in the form of ROM (Read Only Memory), RAM (Read Access Memory), static memory, dynamic storage devices, etc. The memory 550 can store the operating system 551 used to control the operation of the electronic device 500, and the basic input / output system (BIOS) 552 used to control the low-level operations of the electronic device 500. Additionally, it can store a web browser 553, a data storage management system 554, etc. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 550 and is called and executed by the processor 510.

[0058] Input / output interface 530 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, vibrators, indicator lights, etc.

[0059] Network interface 540 is used to connect a communication module (not shown in the figure) to enable communication and interaction between the device and other devices. The communication module can communicate via wired means (e.g., USB, Ethernet cable) or wireless means (e.g., mobile network, Wi-Fi, Bluetooth).

[0060] Bus 560 includes a pathway for transmitting information between various components of the device, such as processor 510, disk drive 520, input / output interface 530, network interface 540, and memory 550.

[0061] It should be noted that although the above-described device only shows the processor 510, disk drive 520, input / output interface 530, network interface 540, memory 550, bus 560, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the method of this application, and does not necessarily include all the components shown in the figures.

[0062] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0063] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0064] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for scheduling error reporting requests of a distributed application, characterized in that, The method includes: In response to a client's request, the load balancing layer distributes the request to the first gateway node of the gateway layer. When an error matching the first fault type set occurs during the request distribution at the first gateway node, the request is sequentially distributed to the second gateway node of the gateway layer based on a preset first retry waiting time, until the request distribution to the target second gateway node is successful or the stop retry condition of the load balancing layer is met. The second gateway node is a healthy gateway node other than the first gateway node, and the first fault type is a specified fault type among the fault types corresponding to the gateway node. The target second gateway node in the gateway layer sends the request to the first microservice instance in the microservice layer. When an error matching the second fault type set occurs in the call of the first microservice instance, the request is sequentially sent to the second microservice instance in the microservice layer based on the second retry waiting time, until a target second microservice instance successfully calls or the stop retry condition of the gateway layer is met. The second microservice instance is a healthy microservice instance other than the first microservice instance. The second retry waiting time is set based on the exponential backoff algorithm. The total number of retries of the gateway node and the microservice instance does not exceed the total number threshold and the total retry time does not exceed the total duration threshold. The second fault type is a specified fault type among the fault types corresponding to the microservice instance.

2. The error request scheduling method of distributed applications according to claim 1, characterized in that, The method further includes: The release process layer responds to the rolling update command, determines the release limit duration corresponding to the rolling update command, determines the node update batch based on the release limit duration and the preset node restart duration, and performs batch updates on the microservice instances of the microservice layer according to the node update batches. The node update batches are not less than two batches, and the node restart duration is not less than the complete cycle duration of the node restart.

3. The error reporting request scheduling method of distributed applications according to claim 2, characterized in that, Before determining the node update batch based on the release restriction duration and the preset node restart duration, the method further includes: Query the number of healthy microservice instances in the microservice layer at the current moment. If the number of instances is less than the threshold, restart the abnormal microservice instances until the number of instances is not less than the threshold.

4. The error request scheduling method for a distributed application according to claim 1, characterized in that, The retry stopping conditions of the load balancing layer include a first maximum number of retries and a first maximum cumulative retry duration. The retry stopping conditions of the gateway layer include a second maximum number of retries and a second maximum cumulative retry duration. The sum of the first maximum cumulative retry duration and the second maximum cumulative retry duration is the total duration threshold. The product of the first maximum number of retries and the second maximum number of retries is not greater than the total number of retries threshold. The first maximum number of retries is determined based on a first fault type. The first maximum cumulative retry duration is determined based on the single retry waiting time strategy corresponding to the first fault type.

5. The method for scheduling error requests in a distributed application according to claim 1, characterized in that, When an error matching the second set of fault types occurs during a call to the first microservice instance, the request is sequentially sent to the second microservice instance in the microservice layer based on the second retry wait time, including: When an error matching the second set of fault types occurs during a call to the first microservice instance, and the error operation satisfies the idempotency condition, the request is sequentially sent to the second microservice instance in the microservice layer based on the second retry wait time.

6. The error request scheduling method for a distributed application according to claim 1, characterized in that, The method further includes: The fuse triggers the circuit breaker when the number of consecutive error reports for a target object within a preset time period exceeds an error count threshold.

7. The error request scheduling method for a distributed application according to claim 1, characterized in that, The method further includes: The client invokes a third microservice instance stored locally based on a request. When an error matching the second set of fault types occurs during the invocation of the third microservice instance, the client sequentially sends the request to each fourth microservice instance based on the second retry waiting time until a target fourth microservice instance is successfully invoked. The fourth microservice instance is a healthy microservice instance stored locally other than the third microservice instance.

8. The error request scheduling method for a distributed application according to claim 1, characterized in that, The method further includes: The target second gateway node of the gateway layer submits the request to a message queue, where the request is retrieved and processed by the healthy microservice instance.

9. An error request scheduling system for a distributed application, characterized in that, The system includes: The client is used to generate requests; One or more load balancers are configured to perform the load balancing layer operations as described in any one of claims 1-8; One or more API gateways are configured to perform gateway layer operations as described in any one of claims 1-8; A microservice system is configured with multiple microservice instance nodes; The continuous integration / continuous deployment system is configured to perform the release process layer operations as described in any one of claims 1-8.

10. A computer program product, characterized in that, The system includes a computer program that, when executed by a processor, implements an error request scheduling method for a distributed application according to any one of claims 1-8.

Images