Cross-domain PaaS service fault tolerance system and method under unreliable network condition

Abstract

The invention discloses a cross-domain PaaS service fault-tolerant system and method under an unreliable network condition, and belongs to the technical field of cloud computing, and the method comprises the steps: deploying PaaS service nodes, forming a cross-domain service network, and collecting link state data of a user request in a cross-domain calling process; different feedback based on the link state data reflects different faults; and adjusting a routing strategy according to the link state data and the fault. The invention has beneficial effects. According to the invention, aiming at the operation characteristics of the cross-domain PaaS service in an unreliable network environment, by constructing a multi-dimensional fault detection system, different types of network problems such as link success rate reduction, error code abnormity and high network delay can be identified at the same time. According to the detected fault features, corresponding processing mechanisms including request fusing, service instance dynamic adjustment, intelligent route switching and the like are automatically matched, and a complete fault handling closed loop is formed.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and more specifically to a fault-tolerant system and method for cross-domain PaaS services under unreliable network conditions. Background Technology

[0002] With the rapid development of cloud computing technology, cross-domain PaaS services are becoming increasingly important in enterprise applications. However, unreliable network conditions pose a serious challenge to the stability and reliability of cross-domain PaaS services. In practical applications, network environments often contain many unstable factors, such as network latency, packet loss, and service call failures, which severely affect the performance and user experience of cross-domain PaaS services.

[0003] While existing technologies offer methods for network fault detection and repair, most are geared towards specific scenarios or single fault types, lacking a comprehensive consideration of the overall fault tolerance capabilities of cross-domain PaaS services. For example, some methods focus solely on link status monitoring without addressing the integration of fault detection and repair; others lack adaptability and flexibility during fault repair, failing to effectively adjust routes based on real-time link status. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is that existing fault detection mechanisms are relatively simple and cannot simultaneously identify and classify diverse fault types such as decreased link success rate, abnormal error codes, and high network latency. Furthermore, existing solutions lack effective coordination between fault detection, repair strategies, and dynamic routing adjustments, resulting in fragmented processes and significant delays in system fault-tolerant response. In addition, regarding cross-domain collaboration, because state synchronization between distributed nodes relies solely on single link monitoring, lacking global link tracing capabilities and consistency protocol support, real-time sharing and collaborative decision-making regarding fault states are difficult, ultimately affecting overall service reliability.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a cross-domain PaaS service fault-tolerant system under unreliable network conditions, comprising,

[0007] The cross-domain service node deployment module is used to deploy PaaS service nodes to form a cross-domain service network.

[0008] The link status monitoring module is used to integrate a distributed link tracing agent into the PaaS service nodes of the cross-domain service network to collect link status data of user requests during cross-domain calls.

[0009] The fault detection and repair module is used to detect whether a fault has occurred in the cross-domain link based on the link status data.

[0010] The routing policy adjustment module is used to adjust the routing policy based on the link status data.

[0011] As a preferred embodiment of the cross-domain PaaS service fault-tolerant system under unreliable network conditions described in this invention, wherein: the formation of the cross-domain service network includes,

[0012] Deploy a local service registry on the PaaS service node to register and manage service instances on this node.

[0013] Data synchronization between various local service registries is achieved through a distributed consistency protocol.

[0014] As a preferred embodiment of the cross-domain PaaS service fault-tolerant system under unreliable network conditions described in this invention, the integrated distributed link tracing agent includes,

[0015] Generate user tracking tags for users, and connect Pass service nodes through user tracking tags to form a complete call path.

[0016] The link status data is stored in the database.

[0017] As a preferred embodiment of the cross-domain PaaS service fault-tolerant system under unreliable network conditions described in this invention, the step of detecting whether a cross-domain link has failed based on link status data includes:

[0018] Retrieve link status data from the database, and based on different feedback from the link status data, identify different faults, distinguishing between the first fault, the second fault, and the third fault.

[0019] The link status data includes the link success rate, error codes, and network latency of service calls.

[0020] As a preferred embodiment of the cross-domain PaaS service fault-tolerant system under unreliable network conditions described in this invention, the different feedbacks reflect different faults, including:

[0021] If the success rate of the detected service call link is lower than the preset service call link success rate threshold, then the link is determined to have experienced the first failure.

[0022] If the service call returns a predefined error code, it is determined that there is a second fault in the link.

[0023] If the network latency exceeds a preset threshold, the link is determined to have experienced a third fault.

[0024] As a preferred embodiment of the cross-domain PaaS service fault-tolerant system under unreliable network conditions described in this invention, the link success rate threshold is calculated based on historical service call data and by time window.

[0025] When the first failure occurs, the circuit breaker mechanism is triggered, new requests are stopped from being distributed to the failed link, the link status is updated to circuit breaker through the service registry, and the current request is switched to the backup link.

[0026] As a preferred embodiment of the cross-domain PaaS service fault tolerance system under unreliable network conditions described in this invention, when the third fault occurs, an automatic scaling mechanism is triggered to increase the number of service instances; when the number of occurrences of the third fault exceeds the limit threshold, the call to the corresponding link service is suspended.

[0027] As a preferred embodiment of the cross-domain PaaS service fault-tolerant system under unreliable network conditions described in this invention, the method for adjusting the routing strategy based on the link state data includes:

[0028] The collected link status data is analyzed. If a link is faulty, the health score of the faulty link is not calculated, and the health score of the link without fault is calculated.

[0029] Links are sorted from highest to lowest based on their health score.

[0030] If the health score of a link is lower than the preset threshold, the corresponding link will be marked as a backup link; if the health score of a link is higher than the preset threshold, the corresponding link will be marked as a priority link.

[0031] The links marked as preferred for use are selected as the routes for service calls.

[0032] This invention provides a fault-tolerant method for cross-domain PaaS services under unreliable network conditions.

[0033] To address the aforementioned technical problems, this invention provides the following technical solution: a fault-tolerant method for cross-domain PaaS services under unreliable network conditions, comprising:

[0034] Deploy PaaS service nodes to form a cross-domain service network and collect link status data of user requests during cross-domain calls.

[0035] Different feedback based on link status data indicates different faults.

[0036] The routing strategy is adjusted based on the link status data and faults.

[0037] As a preferred embodiment of the cross-domain PaaS service fault tolerance method under unreliable network conditions described in this invention, the link status data includes the link success rate of service calls, error codes, and network latency.

[0038] If the success rate of the detected service call link is lower than the preset service call link success rate threshold, then the link is determined to have experienced the first failure.

[0039] If the service call returns a predefined error code, it is determined that there is a second fault in the link.

[0040] If the network latency exceeds a preset threshold, the link is determined to have experienced a third fault.

[0041] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the cross-domain PaaS service fault tolerance method under unreliable network conditions.

[0042] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the cross-domain PaaS service fault tolerance method under unreliable network conditions.

[0043] The beneficial effects of this invention are as follows: This invention addresses the operational characteristics of cross-domain PaaS services in unreliable network environments by constructing a multi-dimensional fault detection system. This system can simultaneously identify different types of network problems, such as decreased link success rate, abnormal error codes, and high network latency. Based on the detected fault characteristics, this invention automatically matches corresponding processing mechanisms, including request circuit breaking, dynamic adjustment of service instances, and intelligent route switching, forming a complete fault response closed loop. Leveraging distributed service registration and global link tracing technologies, it achieves state synchronization and collaborative management between cross-domain nodes, enabling timely detection and adjustment of network state changes. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a general framework diagram of a cross-domain PaaS service fault-tolerant system under unreliable network conditions provided in one embodiment of the present invention.

[0046] Figure 2This is a flowchart illustrating a cross-domain PaaS service fault tolerance method under unreliable network conditions, as provided in an embodiment of the present invention. Detailed Implementation

[0047] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0048] Example 1, referring to Figure 1 This is one embodiment of the present invention, which provides a cross-domain PaaS service fault-tolerant system under unreliable network conditions, including:

[0049] The cross-domain service node deployment module is used to deploy PaaS service nodes to form a cross-domain service network.

[0050] The link status monitoring module is used to integrate a distributed link tracing agent into the PaaS service nodes of the cross-domain service network to collect link status data of user requests during cross-domain calls.

[0051] The fault detection and repair module is used to detect whether a fault has occurred in the cross-domain link based on the link status data.

[0052] The routing policy adjustment module is used to adjust the routing policy based on link status data.

[0053] The cross-domain service node deployment module includes a service node deployment unit, a local service registry deployment unit, a data synchronization unit, and a cross-domain network configuration unit.

[0054] The service node deployment unit is used to select multiple geographical locations to deploy PaaS service nodes to form a cross-domain service network, where each node has independent computing, storage and network resources.

[0055] The local service registry deployment unit is used to deploy a local service registry in each geographical location's PaaS service node. This registry is used to register and manage service instances on the node, including computing services, storage services, and network services.

[0056] The data synchronization unit is used to synchronize data between service registration centers on various nodes through distributed consensus protocols such as Raft and Paxos.

[0057] The cross-domain network configuration unit is used to configure cross-domain network connections, adopt high-speed, high-bandwidth, and low-latency network links, optimize the quality of network communication between nodes, and establish redundant network paths.

[0058] The link status monitoring module includes a link tracing unit, a complete call path formation unit, a data acquisition unit, and a data storage unit.

[0059] The link tracing unit is used to integrate a distributed link tracing agent in each PaaS service node of the cross-domain service network to collect link status data of user requests during cross-domain calls.

[0060] The complete call path forming unit is used to generate a unique trace identifier for each user request, and to connect the various service nodes of the cross-domain call chain through the unique trace identifier to form a complete call path.

[0061] The data acquisition unit is used to collect real-time link status data of user requests, including service call response time, success rate, error codes, and network latency.

[0062] The data storage unit is used to store the collected link status data into the database.

[0063] The fault detection and repair module includes a first fault repair submodule, a second fault repair submodule, a third fault repair submodule, a fault recording submodule, and a fault inspection and maintenance submodule.

[0064] The first fault repair submodule is used to determine whether the link has a first fault and to repair the first fault.

[0065] The second fault repair submodule is used to analyze the error codes returned by the service call, determine whether there is a second fault in the link, and repair the second fault if it occurs.

[0066] The third fault repair submodule is used to determine the delay time. If the network delay time is greater than the preset threshold, it is determined that the link has a third fault, and the third fault is repaired.

[0067] The fault logging submodule is used to record detailed fault information and processing procedures during fault handling.

[0068] The fault detection and maintenance submodule is used to perform regular health checks and maintenance on all links.

[0069] The first fault repair submodule includes a success rate threshold calculation unit, a first fault determination unit, and a first fault repair unit.

[0070] The success rate threshold calculation unit calculates the success rate threshold based on historical service call data and time windows. The calculation formula is as follows.

[0071] S = μ s -2σ s

[0072] Where, μ s σ represents the average success rate within the time window. s The standard deviation of the average success rate.

[0073] The first fault determination unit is used to determine if the link success rate is lower than the success rate threshold for three consecutive detection cycles, and then determine that the link has experienced a first fault.

[0074] The first fault repair unit is used to trigger the circuit breaker mechanism, stop distributing new requests to the faulty link, update the link status to "circuit breaker" through the service registry center, and switch the current request to the backup link.

[0075] The second fault repair submodule includes an error code analysis unit, a second fault determination unit, and a second fault repair unit.

[0076] The error code analysis unit analyzes the error codes returned by service calls and matches them against a predefined list of error codes, which includes, but is not limited to:

[0077] 502 Bad Gateway: This usually indicates a gateway error, caused by unstable cross-domain links leading to reverse proxy errors, etc.

[0078] 503 Service Unavailable: This indicates that the target service node is overloaded, under maintenance, or temporarily unavailable.

[0079] 504 GatewayTimeout: Indicates that the request timed out while being transmitted across a cross-domain link or while waiting for a response.

[0080] The second fault determination unit determines that if, within a preset time window, the cumulative number of error codes returned by the service call reaches a preset number and matches an error code in a predefined error code list, then the link has a second fault.

[0081] Including but not limited to, if any of the above predefined error codes are detected on the same link within a preset time window (e.g., 30 seconds) and the cumulative number of times reaches a threshold (e.g., 10 times), it is determined that a specific type of second-class fault has occurred on the link, such as being marked as "503 fault", "504 fault", or "502 fault".

[0082] The second fault repair unit is used to repair the second fault operation.

[0083] This includes, but is not limited to, notifying the network operations and maintenance system if a 502 Bad Gateway failure is detected, checking and repairing the configuration or connection status of network devices (such as routers and firewalls) along the cross-domain link, and resolving gateway proxy errors.

[0084] If a 503 Service Unavailable fault is detected, the resource management module of the target node is notified to trigger service instance expansion or resource quota adjustment, such as increasing CPU / memory limits, to alleviate service overload.

[0085] If a 504 GatewayTimeout error is detected, the network configuration module is notified to optimize the cross-domain network path configuration, such as switching routes, enabling better paths, negotiating to increase link bandwidth, or adjusting the timeout settings for service calls, in order to resolve the transmission timeout issue.

[0086] The third fault repair submodule includes a delay threshold calculation unit, a third fault determination unit, and a third fault repair order.

[0087] The latency threshold calculation unit is used to calculate the latency threshold based on historical network latency data and within a time window. The calculation formula is as follows.

[0088] L t =μ d +3σ d

[0089] Where, μ d σ represents the average network latency within the time window. d The standard deviation of average network latency.

[0090] The third fault determination unit is used to determine if a link delay exceeds the delay threshold for two consecutive detection cycles, and then it is determined to be a third fault.

[0091] The third fault repair unit is used to repair the third fault.

[0092] The third fault repair unit includes a service instance addition subunit and a service fault tolerance subunit.

[0093] The service instance adds a sub-unit to trigger the auto-scaling mechanism and increase the number of service instances when the service node is overloaded and causes response delays.

[0094] The service fault tolerance subunit is used to introduce a service fault tolerance mechanism. When a service call times out or fails, it will automatically retry the operation to increase the probability of successful call. It also sets a circuit breaker to temporarily cut off the service call when the number of third failures exceeds a preset threshold, and then re-enable the service after it recovers.

[0095] The routing policy adjustment module includes a health score calculation unit for each link, a link sorting unit, a link classification unit, a route invocation unit, and a link health score update unit.

[0096] The health score calculation unit for each link is used to comprehensively analyze the collected link status data. If a link has a fault, the health score of the faulty link is not calculated, and the health score of the link without a fault is calculated.

[0097] The health score is calculated based on a weighted average of various indicators, with the weights preset according to specific circumstances.

[0098] The link sorting unit sorts the links from highest to lowest based on their health scores.

[0099] The link classification unit is used to determine whether a link is marked as a backup link if its health score is lower than a preset threshold, or as a priority link if its health score is higher than the preset threshold.

[0100] The routing invocation unit is responsible for prioritizing links marked as preferred links as routes for service invocation.

[0101] The Link Health Score Update Unit is responsible for periodically reassessing link status data and updating link health scores and rankings.

[0102] Example 2, an embodiment of the present invention, provides a fault-tolerant method for cross-domain PaaS services under unreliable network conditions based on the previous embodiment, including:

[0103] S1. Deploy PaaS service nodes to form a cross-domain service network and collect link status data of user requests during cross-domain calls.

[0104] S2. Different feedback based on link status data reflects different faults.

[0105] S3. Adjust the routing policy based on link status data and faults.

[0106] The link status data in S1 includes the link success rate of service calls, error codes, and network latency.

[0107] In S2, different feedbacks reflect different faults. Specifically, if the success rate of the detected service call link is lower than the preset service call link success rate threshold, then the link is determined to have the first fault.

[0108] If the service call returns a predefined error code, it is determined that there is a second fault in the link.

[0109] If the network latency exceeds a preset threshold, the link is determined to have experienced a third fault.

[0110] The routing strategy in S3 includes analyzing the collected link status data. If a link is faulty, the health score of the faulty link is not calculated, and the health score of the link without fault is calculated.

[0111] Links are sorted from highest to lowest based on their health score.

[0112] If the health score of a link is lower than the preset threshold, the corresponding link will be marked as a backup link; if the health score of a link is higher than the preset threshold, the corresponding link will be marked as a priority link.

[0113] The links marked as preferred for use are selected as the routes for service calls.

[0114] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A cross-domain PaaS service fault-tolerant system under unreliable network conditions, characterized in that: include, The cross-domain service node deployment module is used to deploy PaaS service nodes to form a cross-domain service network; The link status monitoring module is used to integrate a distributed link tracing agent into the PaaS service nodes of the cross-domain service network to collect link status data of user requests during cross-domain calls. The fault detection and repair module is used to detect whether a fault has occurred in the cross-domain link based on the link status data. The routing policy adjustment module is used to adjust the routing policy based on the link status data.

2. The cross-domain PaaS service fault-tolerant system under unreliable network conditions as described in claim 1, characterized in that: The formation of a cross-domain service network includes Deploy a local service registry in the PaaS service node to register and manage service instances of this node; Data synchronization between various local service registries is achieved through a distributed consistency protocol.

3. The cross-domain PaaS service fault-tolerant system under unreliable network conditions as described in claim 2, characterized in that: The integrated distributed tracing agent includes, Generate user tracking tags for users, and connect Pass service nodes through user tracking tags to form a complete call path; The link status data is stored in the database.

4. The cross-domain PaaS service fault-tolerant system under unreliable network conditions as described in claim 3, characterized in that: The method of detecting whether a cross-domain link is faulty based on link status data includes... Retrieve link status data from the database, and based on different feedback from the link status data, distinguish between the first fault, the second fault, and the third fault; The link status data includes the link success rate, error codes, and network latency of service calls.

5. The cross-domain PaaS service fault-tolerant system under unreliable network conditions as described in claim 4, characterized in that: The different feedbacks reflect different faults, including If the success rate of the detected service call link is lower than the preset service call link success rate threshold, the link is determined to have experienced the first failure. If the service call returns a predefined error code, it is determined that there is a second failure in the link; If the network latency exceeds a preset threshold, the link is determined to have experienced a third fault.

6. The cross-domain PaaS service fault-tolerant system under unreliable network conditions as described in claim 5, characterized in that: The link success rate threshold is calculated based on historical service call data and by time window. When the first failure occurs, the circuit breaker mechanism is triggered, new requests are stopped from being distributed to the failed link, the link status is updated to circuit breaker through the service registry, and the current request is switched to the backup link.

7. The cross-domain PaaS service fault-tolerant system under unreliable network conditions as described in claim 6, characterized in that: When the third failure occurs, the automatic scaling mechanism is triggered to increase the number of service instances. When the number of occurrences of the third failure exceeds the limit threshold, the calls to the corresponding link service are suspended.

8. The cross-domain PaaS service fault-tolerant system under unreliable network conditions as described in claim 7, characterized in that: Adjusting the routing strategy based on the link state data includes: The collected link status data is analyzed. If a link is faulty, the health score of the faulty link is not calculated, and the health score of the link without fault is calculated. Based on the health score of the link, sort the links from high to low; If the health score of a link is lower than the preset threshold, the corresponding link will be marked as a backup link; if the health score of a link is higher than the preset threshold, the corresponding link will be marked as a priority link. The links marked as preferred for use are selected as the routes for service calls.

9. A fault-tolerant method for cross-domain PaaS services under unreliable network conditions, employing the cross-domain PaaS service fault-tolerant system under unreliable network conditions as described in any one of claims 1 to 8, characterized in that, include: Deploy PaaS service nodes to form a cross-domain service network and collect link status data of user requests during cross-domain calls; Different feedback based on link status data reflects different faults; The routing strategy is adjusted based on the link status data and faults.

10. The cross-domain PaaS service fault tolerance method under unreliable network conditions as described in claim 9, characterized in that: The link status data includes the link success rate of service calls, error codes, and network latency. If the success rate of the detected service call link is lower than the preset service call link success rate threshold, the link is determined to have experienced the first failure. If the service call returns a predefined error code, it is determined that there is a second failure in the link; If the network latency exceeds a preset threshold, the link is determined to have experienced a third fault.

Images