A communication system external dependency protection method and system
By employing two-way health awareness and a full-stack circuit breaker degradation mechanism, the shortcomings of external dependency protection in communication systems have been addressed, enabling flexible governance of external dependencies and fault liability location, thus ensuring the continuity of core business and data integrity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIAMEN XINGZONG DIGITAL TECH CO LTD
- Filing Date
- 2026-04-01
- Publication Date
- 2026-06-16
AI Technical Summary
Existing communication systems lack two-way health awareness in protecting against external third-party dependencies, cannot achieve full-stack linkage in circuit breaker states, lack differentiated degradation in read and write links, and are difficult to quickly locate fault responsibilities, resulting in impact on core business continuity and high costs for locating responsibilities.
Through two-way health perception and a full-stack circuit breaker and degradation mechanism, the health status of third-party systems is assessed in real time, the circuit breaker status is synchronized to the front end, the read business is switched to local cache, the write business is transferred to the compensation queue, and the system is automatically switched back after recovery, forming visual evidence of responsibility.
It enables flexible governance of external dependencies, ensures the continuity of core businesses, reduces the risk of critical data loss, and provides a clear basis for fault liability identification.
Smart Images

Figure CN122226792A_ABST
Abstract
Description
Technical Field
[0006] , ,
[0005] , , ,
[0001] The present invention relates to the technical fields of high availability of distributed communication systems, integration protection of third-party systems, front-end fault tolerance degradation, and operation status visualization, and particularly relates to a method and system for protecting external dependencies of a communication system. Background Art
[0002] In communication systems such as PBX, contact center workbenches, and enterprise communication middle platforms, key capabilities such as incoming call pop-up screens, customer information queries, call record backwriting, recording archiving, and work order linkage often rely on CRM, work order platforms, or other third-party SaaS interfaces. Existing integration methods usually regard third-party interfaces as synchronously available resources and lack a sufficient protection mechanism for external dependencies.
[0003] When a third-party interface experiences high latency, continuous timeouts, or intermittent errors, the threads, connection pools, or asynchronous task queues used for interface calls and backwriting within the communication system are easily occupied for a long time, thereby affecting call control, agent workbenches, and basic service links that should be prioritized for guarantee; some systems only perform timeout and retry controls on third-party calls on the server side, but the fuse status is not synchronized to the front-end workbench, resulting in the front-end still continuously initiating invalid requests, or directly exposing blank pages, timeout pop-up windows, and unclear errors to users; existing solutions often uniformly process read-type services and write-type services, either all fail or all block, and cannot achieve differential degradation; when customers perceive "system lag", platform providers usually can only separately view application logs, third-party interface monitoring, and user screenshots, and it is difficult to quickly explain whether the exception is due to the internal of the communication system or external dependencies, resulting in high cost for responsibility positioning.
[0004] In summary, existing communication systems have technical defects in protecting external third-party dependencies, including lack of two-way health perception, inability to fully stack-link the fuse status, lack of differential degradation for read and write links, and difficulty in quickly positioning fault responsibilities. Summary of the Invention
[0005] Aiming at the problems of existing communication systems lacking two-way health perception, full-stack linkage degradation, differential processing of reading and writing, and fast positioning ability for external dependencies, the present invention provides a protection mechanism based on two-way health perception and full-stack fuse degradation. This mechanism continuously evaluates the health of third parties through a gateway, synchronizes the status to the front end when a fuse is triggered, drives read services to switch to local caches, write services to transfer to compensation queues, and automatically switches back after recovery, thereby forming visual responsibility evidence while ensuring the continuity of core services.
[0006] In a first aspect, an embodiment of the present invention provides a method for protecting external dependencies of a communication system, including: Collect synchronous API request metrics sent to third-party systems, and simultaneously collect metrics from asynchronous Webhook, callback confirmation, or equivalent receipt links. The collected synchronous API request metrics and asynchronous Webhook receipt metrics are uniformly aggregated based on the same external dependency identifier to form bidirectional health time-series data. Based on a sliding time window or a fixed sample window, the API call timeout rate, error rate, Webhook receipt failure rate, and average confirmation delay in the bidirectional health time series data are statistically analyzed. The overall health score is calculated according to a preset weight, and the circuit breaker state machine is driven to switch between closed, open, and half-open states based on the overall health score. When the circuit breaker state machine switches to the disconnected state, the state change event is synchronized to the front-end workbench via WebSocket, SSE, polling configuration interface or equivalent method, so that the front-end workbench switches the data source of read-type business to local cache, and encapsulates write-type business into a compensation task with at least associated target dependency identifier, business object identifier and idempotent key, and stores it in persistent queue. When the circuit breaker state machine returns to the closed state, the tasks to be compensated in the persistent queue are gradually released, idempotent replay is performed according to the idempotent key, and the front-end workbench is notified to exit the cache mode through WebSocket, SSE, polling configuration interface or equivalent method.
[0007] Secondly, embodiments of the present invention also provide a communication system external dependency protection system, comprising: The two-way health perception module is used to collect synchronous API request metrics sent to third-party systems, as well as metrics from asynchronous Webhook, callback confirmation, or equivalent receipt links. The collected synchronous API request metrics and asynchronous Webhook receipt metrics are uniformly aggregated based on the same external dependency identifier to form two-way health time series data. The health assessment and threshold determination module is used to statistically analyze the API call timeout rate, error rate, Webhook receipt failure rate and average confirmation delay in the bidirectional health time series data based on a sliding time window or a fixed sample window, and calculate the comprehensive health score according to preset weights. The circuit breaker state machine and the hierarchical interception module are used to switch between closed state, open state and half-open state according to the comprehensive health status, and to execute the interception strategy for requests sent to third-party systems in the open state. The front-end degradation routing and cache reading module is used to synchronize the state change event to the front-end workbench through WebSocket, SSE, polling configuration interface or equivalent method when the circuit breaker state machine switches to the disconnected state, so that the front-end workbench switches the data source of the read-type business to the local cache. The write compensation and idempotent replay module is used to encapsulate the write-type business into a compensation task that is at least associated with the target dependency identifier, the business object identifier and the idempotent key when the circuit breaker state machine switches to the open state, and store it in the persistent queue. The recovery detection and convergence module is used to gradually release the tasks to be compensated in the persistent queue when the circuit breaker state machine recovers to the closed state, perform idempotent replay according to the idempotent key, and notify the front-end workbench to exit the cache mode through WebSocket, SSE, polling configuration interface or equivalent methods.
[0008] The technical solution provided by this invention, through bidirectional health awareness of synchronous APIs and asynchronous Webhooks, enables the system to comprehensively and in real-time assess the true health status of third-party dependencies, avoiding misjudgments caused by monitoring only one-way links. Based on a circuit breaker state machine driven by comprehensive health, external calls can be promptly cut off or restricted when a third party is abnormal, preventing the failure from dragging down core communication services. By synchronizing the circuit breaker status to the front-end workbench, read-type services are automatically switched to local cache, avoiding continuous invalid requests from the front-end and ensuring the basic continuity of agent operations. Simultaneously, write-type services are encapsulated as tasks to be compensated and stored in a persistent queue. Combined with idempotent replay after recovery, this effectively reduces the risk of loss of critical data such as call records and work order write-back during abnormal periods. Throughout the process, the system forms a closed-loop protection mechanism from back-end circuit breaking to front-end degradation, from read cache to write compensation, and automatically switches back after recovery, achieving flexible governance of external dependencies and providing clear and visual evidence for fault liability localization. Attached Figure Description
[0009] Figure 1 A flowchart illustrating a method for protecting external dependencies in a communication system, as provided in an embodiment of the present invention; Figure 2 This is a diagram illustrating the bidirectional health awareness and full-stack degradation architecture for third-party dependencies provided in this embodiment of the invention. Figure 3 This is a flowchart of the circuit breaker state switching and recovery detection provided in an embodiment of the present invention; Figure 4 The timing diagram for read degradation, write compensation, and status dashboard during anomalies provided in this embodiment of the invention. Detailed Implementation
[0010] In systems such as PBX, contact center workbench, and enterprise communication platforms, key capabilities such as caller ID display, customer information query, call detail record (CDR) writing, call recording archiving, and work order linkage often rely on CRM, work order platforms, or other third-party SaaS interfaces. Existing integration methods typically treat third-party interfaces as synchronously available resources, which mainly presents the following problems: 1. External dependencies can easily drag down the core system: When third-party interfaces experience high latency, continuous timeouts, or intermittent errors, the threads, connection pools, or asynchronous task queues used for interface calls and write-back within the communication system can easily be occupied for a long time, thereby affecting call control, agent workbench, and basic business links that should be prioritized.
[0011] 2. Existing circuit breakers are usually limited to the backend: Some systems only control timeouts and retryes for third-party calls on the server side, but the circuit breaker status is not synchronized to the frontend, causing the frontend to continue to send invalid requests, or directly expose blank pages, timeout pop-ups, and ambiguous errors to the user.
[0012] 3. Lack of differentiated degradation in the read-write link: Read-type capabilities such as caller ID pop-up can usually accept short-term use of local snapshots or cached data, while write-type capabilities such as call detail record writing and recording index are more suitable for temporary storage and compensation; existing solutions often treat read and write uniformly, which either all fail or all are blocked.
[0013] 4. Lack of a unified evidence view for after-sales troubleshooting: When customers perceive "system lag", the platform can usually only check application logs, third-party interface monitoring and user screenshots separately, which makes it difficult to quickly determine whether the problem is an abnormal time consumption within the communication system or a problem with the external third-party dependency, resulting in high cost of locating responsibility.
[0014] To address the aforementioned technical problems in existing technologies, this invention aims to overcome the shortcomings of current communication systems, such as lack of bidirectional health awareness of external third-party dependencies, inability to achieve full-stack linkage during circuit breaker states, lack of differentiated degradation in read / write links, and difficulty in quickly locating fault responsibility. It provides an external dependency protection mechanism that uses synchronous API calls and asynchronous Webhook receipts as common inputs. This mechanism continuously evaluates the health of third parties on the gateway side, automatically triggers tiered circuit breakers when thresholds are reached, and synchronizes the circuit breaker status to the front-end workbench, cache read path, compensation write queue, and status dashboard. This protects core communication services while maintaining business continuity and preserving evidence for fault location.
[0015] The following detailed description of a communication system external dependency protection method provided by the present invention, with reference to specific embodiments, provides a detailed explanation.
[0016] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for protecting external dependencies of a communication system, which may include the following steps: S110 collects synchronous API request metrics sent to third-party systems, and simultaneously collects metrics from asynchronous Webhook, callback confirmation, or equivalent receipt links. The collected synchronous API request metrics and asynchronous Webhook receipt metrics are uniformly aggregated based on the same external dependency identifier to form bidirectional health time-series data.
[0017] Specifically, the communication system monitors the interaction process with third-party systems in real time at the external gateway or integration layer. For synchronous call chains, the system collects relevant metrics for each API request sent to the third-party system; for asynchronous callback chains, the system collects retrieval metrics returned by the third-party system through Webhook, callback confirmation, or equivalent methods. All collected metrics are associated with the same external dependency identifier, forming health time-series data reflecting the bidirectional communication status of the third-party system.
[0018] In one implementation, metrics for synchronous API requests sent to third-party systems are collected, including at least the target system identifier, interface identifier, request initiation time, response time, status code, timeout flag, and failure reason; simultaneously, metrics for asynchronous Webhook, callback confirmation, or equivalent receipt links are collected, including at least the delivery time, confirmation time, confirmation status, number of retries, and final result.
[0019] Specifically, the communication system generates a record for each synchronous API request sent to a third-party system at the external gateway or integration layer. The target system identifier distinguishes different third-party dependencies (such as CRM systems, ticketing platforms, etc.), the interface identifier distinguishes different interfaces within the same third-party system (such as customer query interfaces, call detail record (CDR) write-back interfaces), the request initiation time and response time are used to calculate interface response latency, and the status code, timeout flag, and failure reason are used to determine whether the call was successful or encountered an error. By collecting these metrics, the system can monitor the health of the active call chain in real time.
[0020] Meanwhile, the system monitors receipts returned asynchronously by third-party systems. For Webhook callbacks, confirmation receipts, or equivalent links, the system records the delivery time (i.e., the time it takes for the receipt to arrive in the system), confirmation time (i.e., the time difference between sending the business request and receiving the receipt), confirmation status (e.g., success, failure, timeout), number of retries (used to identify unstable links), and final result (indicating the final success or failure of the asynchronous interaction). By collecting these metrics, the system can perceive the stability of asynchronous receipt links, compensating for the blind spots caused by only monitoring synchronous calls.
[0021] By unifying the synchronous API request metrics and asynchronous Webhook receipt metrics based on the same external dependency identifier (such as the target system identifier), the system generates health time-series data reflecting the bidirectional communication status of this third-party dependency. This data includes both the smoothness of actively initiated calls and the stability of passively received receipts, providing a complete data foundation for subsequent comprehensive health assessments.
[0022] S120, based on a sliding time window or a fixed sample window, statistically analyze the API call timeout rate, error rate, Webhook receipt failure rate, and average confirmation delay in the bidirectional health time series data, calculate the comprehensive health score according to preset weights, and drive the circuit breaker state machine to switch between closed, open, and half-open states based on the comprehensive health score.
[0023] Specifically, the system uses a sliding time window or a fixed sample window to statistically analyze the bidirectional health time-series data within each window, calculating the API call timeout rate and error rate, as well as the Webhook receipt failure rate and average confirmation latency. These metrics are then combined into a comprehensive health score using preset weights, reflecting the overall health level of the third-party system. Based on changes in the comprehensive health score, the system drives the circuit breaker state machine to switch between closed, open, and half-open states to achieve dynamic protection against external dependencies.
[0024] To ensure a complete and clear description of the solution, the specific implementation of S120 will be described in detail in subsequent embodiments.
[0025] S130, when the circuit breaker state machine switches to the disconnected state, the state change event is synchronized to the front-end workbench through WebSocket, SSE, polling configuration interface or equivalent method, so that the front-end workbench switches the data source of read-type business to local cache, and encapsulates write-type business into a compensation task with at least associated target dependency identifier, business object identifier and idempotent key, and stores it in the persistent queue.
[0026] Specifically, when the circuit breaker state machine determines that a third-party system is abnormal and switches to the disconnected state, the system immediately synchronizes the state change event to the front-end console via real-time communication. Upon receiving the event, the front-end console automatically switches the data reading path for read-type services from real-time requests to the third-party interface to local caching, avoiding invalid requests. Simultaneously, the system intercepts write-type services that would otherwise be sent to the third-party system, encapsulates them as compensation tasks, and stores them in a persistent queue for subsequent processing.
[0027] S140, when the circuit breaker state machine returns to the closed state, gradually release the tasks to be compensated in the persistent queue, perform idempotent replay according to the idempotent key, and notify the front-end workbench to exit the cache mode through WebSocket, SSE, polling configuration interface or equivalent method.
[0028] Specifically, when the circuit breaker state machine determines that the third-party system has recovered and switches back to the closed state, the system begins to gradually release the backlog of tasks awaiting compensation in the persistent queue. Each task is replayed based on its idempotent key to ensure that write operations are not executed repeatedly. Simultaneously, the system sends a recovery notification to the front-end console via real-time communication. Upon receiving the notification, the front-end console exits cache mode and resumes the normal data reading path.
[0029] The technical solution provided by this invention, through bidirectional health awareness of synchronous APIs and asynchronous Webhooks, enables the system to comprehensively and in real-time assess the true health status of third-party dependencies, avoiding misjudgments caused by monitoring only one-way links. Based on a circuit breaker state machine driven by comprehensive health, external calls can be promptly cut off or restricted when a third party is abnormal, preventing the failure from dragging down core communication services. By synchronizing the circuit breaker status to the front-end workbench, read-type services are automatically switched to local cache, avoiding continuous invalid requests from the front-end and ensuring the basic continuity of agent operations. Simultaneously, write-type services are encapsulated as tasks to be compensated and stored in a persistent queue. Combined with idempotent replay after recovery, this effectively reduces the risk of loss of critical data such as call records and work order write-back during abnormal periods. Throughout the process, the system forms a closed-loop protection mechanism from back-end circuit breaking to front-end degradation, from read cache to write compensation, and automatically switches back after recovery, achieving flexible governance of external dependencies and providing clear and visual evidence for fault liability localization.
[0030] exist Figure 1 Based on the illustrated embodiment, as an implementation of this invention, the step of calculating the comprehensive health score by statistically analyzing the API call timeout rate, error rate, Webhook receipt failure rate, and average confirmation delay in the bidirectional health time series data based on a sliding time window or a fixed sample window, and according to preset weights, may include the following steps: Step a1: Using a sliding time window or a fixed sample window as the statistical unit, count the total number of API calls made by the third-party system corresponding to the external dependency identifier within the window. Timeout and number of errors and the total number of Webhook deliveries and number of failures The average confirmation delay is calculated by dividing the sum of confirmation delays for successful Webhook receipts by the number of successful receipts. .
[0031] Specifically, the system uses a sliding time window (e.g., the last 30 seconds) or a fixed sample window (e.g., the last 100 requests) as the statistical unit to count the total number of API calls within the window corresponding to each external dependency identifier and the third-party system. Timeout and number of errors and the total number of Webhook deliveries and number of failures Simultaneously, the system collects the confirmation delays for each successful receipt, calculates the ratio of the sum of these delays to the number of successful receipts, and obtains the average confirmation delay. This average acknowledgment latency reflects the response stability of the asynchronous callback chain.
[0032] Step a2, based on the timeout number With total number of calls The ratio of API call timeout rate is used to calculate the API call timeout rate. T According to the number of errors With total number of calls Error rate in ratio calculation E According to the number of failures Total number of deliveries The ratio is used to calculate the Webhook receipt failure rate. .
[0033] Specifically, based on the basic data collected in step a1, the system further calculates three core ratios: API call timeout rate. T = / This reflects the timeliness of the synchronous call chain response; API call error rate. E = / This reflects the success rate of the synchronous call chain; Webhook receipt failure rate. F = / This reflects the reliability of the asynchronous callback chain. The above three ratios are related to the average acknowledgment latency. Together, these constitute the key dimensions for assessing the health status of third-party dependencies.
[0034] Step a3, according to the preset weights Through formula Calculate the overall health score ,in, .
[0035] Specifically, the system uses preset weights. The above four indicators are weighted and combined using the formula. Calculate the overall health score H. The sum of all weights is 1, and this can be flexibly adjusted according to the business scenario; for example, the timeout rate can be increased for scenarios with high real-time requirements. T The weighting of data can improve the failure rate of receipts in scenarios with high data consistency requirements. F Weighting of overall health. HThe value ranges from 0 to 100. H A higher value indicates a healthier third-party dependency, while a lower value indicates a risk.
[0036] Step a4: When the number of valid samples used for statistics is lower than a preset threshold, the third-party system corresponding to the external dependency identifier is marked as an observation state to avoid cold start misjudgment. The number of valid samples is the sum of the total number of API calls used for statistics and the total number of Webhook deliveries.
[0037] Specifically, during the initial system startup or when third-party dependencies are first integrated, the number of samples in the statistics window may be insufficient. At this time, the calculated ratio indicators are prone to misjudgment due to occasional fluctuations. Therefore, before calculating the overall health score, the system first checks whether the number of valid samples used for statistics (i.e., the sum of the total number of API calls and the total number of Webhook deliveries) is below a preset threshold. If it is below the threshold, the system does not execute a circuit breaker; instead, it marks the corresponding third-party system as "to be observed" and continues to collect data until the number of samples reaches the threshold, thus avoiding false circuit breakers during the cold start phase.
[0038] Through the above steps, the system can dynamically and accurately assess the health status of third-party dependencies based on complete data from synchronous and asynchronous bidirectional links, providing a reliable basis for subsequent circuit breaker decisions.
[0039] exist Figure 1 Based on the illustrated embodiment, as one implementation of this invention, the step of switching the fuse state machine between a closed state, an open state, and a half-open state according to the comprehensive health status may include the following steps: Step b1: Maintain a state machine for each external dependency identifier corresponding to a third-party system, including at least closed, open, and half-open states.
[0040] Specifically, the system maintains an independent circuit breaker state machine for each third-party system corresponding to each external dependency identifier. This state machine includes at least three working states: closed state (normal state, allowing requests to be sent to the third-party system), open state (circuit breaker state, blocking requests to the third-party system), and half-open state (probing state, allowing requests to be sent in a limited way to verify the recovery status). Each third-party dependency's state machine runs independently and does not affect each other, avoiding the possibility that an anomaly in one place will cause other dependencies to be mistakenly circuit-broken.
[0041] Step b2: When the overall health score is lower than the preset disconnection threshold, or when consecutive timeouts or consecutive Webhook receipt failures are detected within the preset time window, the state machine is switched from the closed state to the disconnected state.
[0042] Specifically, the system monitors the overall health score calculated in step S120 in real time and continuously tracks anomalies in synchronous calls and asynchronous receipts. The system will switch the state machine from a closed state to an open state when any of the following conditions are met: First, the overall health score is lower than a preset disconnection threshold (e.g., ...). H A value below 30 indicates a severe deterioration in the overall health of third-party dependencies. Secondly, if consecutive timeouts (e.g., 5 consecutive timeouts for synchronous APIs) or consecutive Webhook callback failures (e.g., 3 consecutive failures for asynchronous callbacks) are detected within a preset time window, it indicates that while the overall health of the third-party dependencies has not yet fallen below the threshold, persistent anomalies have occurred. Through these dual triggering conditions, the system can respond to both gradual health declines and rapid, sudden, continuous anomalies.
[0043] Step b3: In the disconnected state, execute at least one of the following blocking strategies for requests sent to the external dependency: fast failure, rate limiting, or whitelisting.
[0044] Specifically, after the state machine switches to the disconnected state, the system executes a unified interception strategy on all requests destined for that external dependency to protect the core communication link from being disrupted. The specific strategies adopted can be flexibly configured according to business importance: a fast-fail strategy directly returns an error response without consuming thread and connection resources; a rate-limiting strategy allows a small number of requests to pass through to maintain basic business connectivity; a whitelisting strategy only allows requests to specific critical interfaces or specific tenants, blocking all other requests. Through these interception strategies, the system minimizes the impact during external dependency anomalies.
[0045] Step b4: After a preset cooling time in the disconnected state, the state machine is switched from the disconnected state to the half-open state to assess whether the third-party system has recovered through a small number of probe requests or dedicated probe requests.
[0046] Specifically, after a preset cooldown period (e.g., 30 seconds) in the disconnected state, the system no longer unconditionally remains disconnected but switches its state machine from the disconnected state to a half-open state. In the half-open state, the system no longer blocks all requests but allows a controlled number of probe requests or dedicated health check requests to assess whether the third-party system has recovered. The proportion of probe requests allowed can be configured according to the business scenario (e.g., allowing one request every 10 seconds) to avoid a sudden surge in traffic that could cause the third-party system to crash again before it has fully recovered. Through the buffering mechanism of the half-open state, the system can smoothly switch back after the third-party dependency recovers, avoiding secondary failures caused by the instantaneous traffic surge during recovery.
[0047] exist Figure 1Based on the illustrated embodiment, as one implementation of this invention, synchronizing state change events to the front-end workbench via WebSocket, SSE, polling configuration interface, or equivalent methods may include the following steps: Step c1: When a state change occurs in the circuit breaker state machine, the event containing the dependency identifier and the current circuit breaker state is synchronized to the front-end workbench via WebSocket, SSE, polling configuration interface, or equivalent method.
[0048] Specifically, when the circuit breaker state machine undergoes a state change (e.g., from closed to open, from open to half-open, or from half-open back to closed), the system immediately triggers a state synchronization mechanism. Specifically, the system can select one or more of the following methods to push the state change event to the front end, depending on the deployment architecture and network environment of the front-end workbench: WebSocket method: A persistent WebSocket connection is established between the system and the front-end workbench. When a state change occurs, the system actively pushes the event to the front-end through this connection, achieving millisecond-level real-time synchronization. This is suitable for agent workbench scenarios that require immediate response.
[0049] SSE (Server-Sent Events) method: The system pushes event streams unidirectionally through a long HTTP connection, and the front end subscribes to a specified event channel. It is suitable for scenarios that need to receive status updates in real time but do not require bidirectional communication.
[0050] Polling configuration interface method: The front-end workbench actively calls the configuration interface provided by the system at fixed time intervals (e.g., every 2 seconds) to obtain the latest circuit breaker status. This method is simple to implement, has good compatibility, and is suitable for scenarios with low real-time requirements or for compatibility with older browsers.
[0051] Equivalent methods: The system can also use other equivalent communication mechanisms, such as message queue broadcast, UDP multicast, long polling, etc., as long as the state change event can be reliably transmitted from the server to the front-end workbench.
[0052] Synchronized event content includes at least two key fields: a dependency identifier (used by the frontend to identify which specific third-party dependency has undergone a state change) and the current circuit breaker status (closed, open, or half-open). Upon receiving the event, the frontend can then execute corresponding degradation or recovery operations based on this information. Through this synchronization mechanism, the circuit breaker status is no longer limited to the server side but achieves full-stack linkage, enabling the frontend to maintain state consistency with the backend and jointly respond to abnormal situations involving external dependencies.
[0053] exist Figure 1Based on the illustrated embodiment, as one implementation of this invention, switching the data source for read-type business operations to a local cache in the front-end workbench may include the following steps: Step d1: After receiving an event indicating that the circuit breaker status is disconnected, the front-end workbench switches the data reading path from real-time request to third-party interface to reading local cache for read-type business related to the dependency. The local cache preferably uses IndexedDB, local database or equivalent persistent storage.
[0054] Specifically, upon receiving an event indicating that the circuit breaker status is "disconnected," the front-end console first parses the dependency identifier in the event to identify the third-party dependency that triggered the circuit breaker. For all read-based services related to this dependency (such as customer information queries during incoming call pop-ups, customer information displays on the agent's workbench, etc.), the front-end console automatically switches the data reading path from the original "real-time request to third-party interfaces" to "reading from the local cache." The local cache preferably uses IndexedDB, a local database, or an equivalent persistent storage solution. These storage methods can persistently save the most recently synchronized data snapshot locally on the browser or client. Even if the network is disconnected or the third-party interface is unavailable, the front-end can still quickly read data from the local cache, ensuring the operability of basic business operations. The switching process is transparent to the agent user and requires no manual intervention; the front-end console automatically completes the data source switch in the background.
[0055] Step d2: The front-end workbench displays a status message indicating that it is currently in degraded mode. The message includes at least dependency status information, an identifier that the data source is cached, and a message indicating that the data may not be up-to-date.
[0056] Specifically, to ensure users understand that the system is currently in degraded mode, the front-end workbench displays clear status prompts in real time. These prompts include at least three types of information: first, dependency status information, clearly informing users which third-party dependencies are experiencing issues and are currently in a circuit breaker-broken state; second, data source identification, indicating that the currently displayed data comes from local cache rather than being retrieved in real-time from third-party interfaces; and third, a message indicating that the data may not be up-to-date, reminding users that the currently displayed data may be a historical snapshot, and the latest data will be updated synchronously after the third-party dependencies are restored. Through these prompts, users can understand the system's current operating mode, avoiding misunderstandings due to outdated data, and also providing intuitive evidence for after-sales troubleshooting.
[0057] Through the above steps, the front-end workbench can automatically complete the degradation switch of read-type business during the circuit breaker period, which not only avoids continuously sending invalid requests to unavailable third-party interfaces, but also ensures the basic continuity of agent operations, while maintaining the transparency of user experience through clear interface prompts.
[0058] exist Figure 1 Based on the illustrated embodiment, as one implementation of this invention, the write-type business is encapsulated into a compensation task that is at least associated with a target dependency identifier, a business object identifier, and an idempotent key, and stored in a persistent queue. This may include the following steps: Step e1: During the circuit breaker period, intercept write requests that should have been sent to external dependencies that are in a disconnected state.
[0059] During the circuit breaker period, when the circuit breaker state machine is in the open state, the system uniformly intercepts all write-type business requests that should have been sent to the third-party dependency (such as call detail record (CDR) write-back after a call ends, recording index synchronization, work order push, etc.). The interception operation is completed at the gateway or integration layer, ensuring that write requests do not reach third-party interfaces that are already abnormal or unavailable, thereby avoiding problems such as thread blocking and connection pool exhaustion, and protecting the stability of the core communication system.
[0060] Step e2 encapsulates the write-type business request into a task to be compensated. The task is associated with at least the target dependency identifier, business object identifier, payload summary, retry status, and idempotent key to prevent duplicate writes.
[0061] The system encapsulates each intercepted write request into a structured compensation task. This task contains at least the following key information: Target dependency identifier: Indicates which third-party system the task was originally intended to be sent to, for correct routing after recovery; Business object identifier: Identifies specific business records (such as call ID, work order number, etc.) to facilitate subsequent tracking and troubleshooting; Payload Summary: Records the core data content of the original request, used for rebuilding the request after recovery; Retry status: Records the number of times the current task has been retried, the maximum number of retry limits, and the time window for the next retry, which is used to control the replay pace; Idempotent key: An identifier used to uniquely identify a single business write operation (can be generated from the business object identifier combined with the operation type). During replay recovery, third-party systems can use the idempotent key to determine whether the request has been processed, thereby avoiding data duplication caused by repeated replays.
[0062] Through the above encapsulation, the task to be compensated carries all the information needed to restore the replay, while also having idempotency guarantees.
[0063] Step e3: Write the task to be compensated into a persistent queue. The persistent queue is implemented as a server-side dead-letter queue, a local persistent queue, or a combination of the two.
[0064] Specifically, the encapsulated tasks to be compensated are written to a persistent queue. The persistent queue can be implemented using one of the following methods depending on the system deployment architecture: Server-side dead-letter queue: Tasks are stored in a server-side message queue (such as Kafka, RabbitMQ) or a dedicated dead-letter queue. It is suitable for distributed deployment scenarios, and tasks are not easily lost and support high-concurrency processing.
[0065] Local persistent queue: Stores tasks in persistent storage (such as local files or embedded databases) of the client or local agent process, suitable for desktop workbench or lightweight deployment scenarios.
[0066] The combination of the two is to use both server-side queues and local queues, with the local queue serving as a primary buffer and the server-side queue serving as reliable storage, thus balancing performance and reliability.
[0067] By using a persistent queue, the tasks awaiting compensation will not be lost even in the event of a system restart or a prolonged network interruption, and replay can be reliably executed once the third-party dependencies are restored.
[0068] Through the above steps, the system switches write-type services from synchronous blocking mode to asynchronous compensation mode during the circuit breaker period. This avoids impacting the core link and ensures that critical data is not lost due to external dependency anomalies, thus guaranteeing data consistency after recovery.
[0069] exist Figure 1 Based on the illustrated embodiment, this embodiment of the invention further includes the steps of state visualization and output of evidence of responsibility: Step f1 displays the third-party dependency topology diagram, the trend curve of overall health, the current circuit breaker status, the number of requests blocked due to circuit breaker, the backlog of persistent queues, and the comparison information between internal processing time and external interface processing time on the management backend interface.
[0070] Specifically, the system provides a comprehensive and visual display of the running status of external dependencies in the management backend interface (operations and maintenance console or operations support system). The displayed content includes: Third-party dependency topology diagram: This graphically displays all third-party dependencies integrated into the current communication system and their calling relationships, making it easy for administrators to quickly locate dependency nodes with abnormalities.
[0071] Overall Health Trend Curve: With time as the horizontal axis, this curve displays the overall health of each third-party dependency. H The historical change curve helps administrators observe the evolution trend of health status and predict potential risks.
[0072] Current circuit breaker status: Real-time display of the status (closed, open, or half-open) of each third-party dependent circuit breaker state machine, distinguished by color (e.g., green for closed, red for open, and yellow for half-open).
[0073] Number of blocked requests: This counts the total number of requests blocked due to the disconnected state since the circuit breaker was triggered, reflecting the actual effectiveness of the circuit breaker protection.
[0074] Persistent queue backlog: Displays the current backlog of tasks awaiting compensation, used to assess the processing pressure and recovery progress of the compensation queue.
[0075] Internal and external processing time comparison: The internal processing time of the communication system (including business logic processing, queue waiting, etc.) and the external processing time of calling third-party interfaces are displayed side by side. Administrators can directly compare them and quickly determine whether the anomaly originates from the internal system or external dependencies.
[0076] Through the above visualized information, administrators can monitor the health status and circuit breaker status of all third-party dependencies in real time, providing intuitive data support for daily operation and maintenance and troubleshooting.
[0077] Step f2: When an anomaly occurs, in response to an operation, the health curve, state transition records, and internal and external time consumption comparison results within the time window related to the current fault are generated as screenshots, reports, or structured event records as a basis for troubleshooting.
[0078] Specifically, when the system detects a circuit breaker trigger or an administrator actively queries for anomalies, the system automatically packages and outputs key evidence related to the current fault. Specifically, the system extracts data within a configurable time window before and after the fault occurrence (e.g., 5 minutes before the fault to 5 minutes after the fault), including: health trend curves, circuit breaker status switching records (the time of each status change and the status before and after the change), and comparison curves of internal processing time and external interface time. The system generates evidence materials in various formats from this data. Screenshot: Automatically captures high-resolution images of relevant charts and status information in the management backend for quick sharing or insertion into fault reports; Reports: Generate structured reports (such as PDF or Excel format) containing detailed information such as failure time, dependency identifiers, key indicator values, and status transition details; Structured event logging: Outputs event logs in JSON or XML format, which can be accessed by automated operation and maintenance platforms or log analysis systems to achieve automated fault identification and analysis.
[0079] The aforementioned evidence can serve as a basis for internal troubleshooting and can also be used for service level communication and liability definition with third-party service providers, significantly reducing the cost and time of liability determination.
[0080] Through the above steps, the system not only protects against external dependencies, but also forms a complete closed loop of visual status display and fault evidence, making after-sales troubleshooting based on evidence and ensuring clear and efficient determination of responsibility.
[0081] exist Figure 1 Based on the illustrated embodiment, as one implementation of this invention, when the circuit breaker state machine returns to the closed state, the tasks to be compensated in the persistent queue are gradually released, idempotent replay is performed according to the idempotent key, and the front-end workbench is notified to exit the cache mode via WebSocket, SSE, polling configuration interface, or equivalent methods. This may include the following steps: Step g1: When the fuse state machine is in a half-open state, only detection requests that meet the proportional limit or special health check requests are allowed.
[0082] When the circuit breaker state machine switches from the open state to the half-open state after a preset cooling time, the system does not immediately restore all traffic. Instead, it performs strict probe control on requests sent to the third-party dependency. Specifically, the system only allows probe requests that meet a certain percentage limit (e.g., one request every 10 seconds, or a percentage not exceeding 5% of normal traffic), or only allows dedicated health check probe requests (e.g., a lightweight interface that does not involve business data). Through this low-volume probe mechanism, the system can avoid sudden traffic surges when the third-party dependency is not yet fully stable, while verifying whether it has truly recovered at minimal cost.
[0083] Step g2: When several consecutive detections are successful and the overall health level is again above the closing threshold, the fuse state machine is restored from the half-open state to the closed state.
[0084] In the half-open state, the system continuously monitors the execution results of probe requests. When several consecutive probe requests are successful (e.g., three consecutive probes return normal responses), and the overall health score is again above the preset closure threshold (e.g., H value greater than 70), the system determines that the third-party dependency has returned to normal. At this point, the system restores the circuit breaker state machine from the half-open state to the closed state, marking the end of the circuit breaker protection cycle and the resumption of normal call mode.
[0085] Step g3: After restoring to the closed state, gradually release the tasks to be compensated in the persistent queue and perform idempotent replay based on the idempotent keys they carry.
[0086] After the state machine returns to a closed state, the system begins processing the backlog of tasks awaiting compensation during the circuit breaker period. To avoid overwhelming the newly restored third-party dependencies with a sudden surge of requests, the system employs a phased release strategy, retrieving tasks awaiting compensation from the persistent queue at a preset rate (e.g., 10 tasks per second) or by priority (e.g., urgent tasks are prioritized). During replay of each task, the system sends its idempotency key to the third-party interface. The third-party interface uses the idempotency key to determine whether the request has already been processed: if not, the write operation is performed normally; if processed, success is returned directly, thus ensuring data consistency and idempotency of operations, and avoiding business issues such as duplicate writes and duplicate billing caused by repeated replays.
[0087] In step g4, a notification to exit degrade mode is sent to the front-end workbench via WebSocket, SSE, or polling configuration interface, so that the front-end workbench can resume the real-time data reading path for requesting third-party interfaces.
[0088] As the state machine returns to its closed state, the system sends a "Exit Degradation Mode" notification to the front-end console via WebSocket, SSE, polling configuration interface, or equivalent methods. Upon receiving this notification, the front-end console switches the data retrieval path for read-based services back from local caching to real-time requests to third-party interfaces, restoring normal data acquisition mode. Simultaneously, the degradation notification on the front-end interface is removed or updated to "Restored to Normal," allowing agents to continue receiving real-time and accurate data services.
[0089] Through the above steps, the system achieves a smooth transition from circuit breaker recovery to normal operation, ensuring that third-party dependencies are not overloaded in the early stages of recovery, and ensuring reliable replay of compensation data and seamless front-end switching, thus forming a complete flexible governance closed loop.
[0090] To ensure clarity, the following will be combined with the appendix. Figure 2 The diagram shown illustrates the architecture of bidirectional health awareness and full-stack degradation for third-party dependencies, providing a detailed explanation of each component and its interaction process.
[0091] As attached Figure 2 As shown, this architecture consists of four main components: a front-end workbench, a communication system integration layer, third-party systems, and a status visualization dashboard.
[0092] I. Front-end workbench The front-end workbench consists of three sub-modules: the agent interface, the fallback router, and the local cache (IndexedDB). The agent interface displays the agent's operation interface and business data; the fallback router is responsible for dynamically selecting the data read path based on the circuit breaker status; and the local cache is used to store and retrieve business snapshots such as customer information in fallback mode.
[0093] II. Communication System Integration Layer The communication system integration layer, as the core part of this invention, integrates an integrated gateway, a two-way health sensing module, a health assessment module, a circuit breaker state machine, a compensation queue, and a state synchronization module.
[0094] Integration Gateway: Interacts with third-party systems via synchronous API for request and response, and receives asynchronous confirmation receipts from third-party systems via Webhook / callback.
[0095] Two-way health perception module: Collects synchronous API links (request metrics) and asynchronous Webhook / callback links (receipt metrics) respectively, and collects the data in a unified manner.
[0096] Health assessment module: Receives time-series data from the two-way health perception module and calculates the overall health score.
[0097] Fuse state machine: Based on the health status output by the health assessment module, it switches between closed, open, and half-open states, and outputs the fuse status to the state synchronization module.
[0098] Compensation queue: Receives write compensation tasks during the circuit breaker disconnection period and performs replay after recovery.
[0099] Status synchronization module: Sends the circuit breaker status to the status visualization dashboard and the degradation router of the front-end workbench through two paths: "circuit breaker status" and "status event".
[0100] III. Third-party systems Third-party systems, including CRM / work order systems and other external SaaS services, are external dependencies of the communication system. These systems interact with the integration gateway through synchronous API responses and asynchronous confirmation receipts.
[0101] IV. Status Visualization Dashboard The status visualization dashboard receives circuit breaker status information from the status synchronization module and displays operational data such as health trends, circuit breaker status, and queue backlog.
[0102] V. Interaction Flow Appendix Figure 2 The document illustrates two main paths: normal invocation and circuit breaker degradation. First, the normal call path: The fallback router in the front-end workbench selects the "normal read" path, and through the integrated gateway of the communication system integration layer, initiates a request to the third-party system via a synchronous API. The third-party system returns a response. The third-party system then sends an acknowledgment receipt to the integrated gateway via a Webhook / callback, completing the asynchronous interaction.
[0103] Second, the circuit breaker downgrade path: When the circuit breaker state machine switches to the open state, the state synchronization module sends the circuit breaker state event to the downgraded router. The downgraded router switches to the "read on circuit breaker" path and reads data from the local cache IndexedDB.
[0104] Meanwhile, write-type business that originally needed to be sent to a third-party system was encapsulated as a task to be compensated and written into the compensation queue, i.e., the "write compensation" path.
[0105] Once the third-party system recovers, the circuit breaker state machine closes again, and the state synchronization module sends a recovery event. The compensation queue resends the backlogged tasks to the third-party system according to the "replay after recovery" path, completing idempotent replay.
[0106] VI. Status Synchronization and Visualization The state changes (fuse-out states) generated by the circuit breaker state machine are pushed in real time to the degraded routers on the front-end workbench via the state synchronization module, and also to the state visualization dashboard. The state visualization dashboard displays information such as health status, circuit breaker status, and queue backlog for operations and maintenance personnel to monitor and troubleshoot.
[0107] Through append Figure 2 The architecture shown in this invention implements a complete external dependency protection mechanism, from bidirectional health awareness to full-stack circuit breaking and degradation, and from read / write differential processing to recovery loop, effectively ensuring the business continuity and maintainability of the communication system when third-party dependencies are abnormal.
[0108] To ensure clarity, the following description, in conjunction with the appendix, will be provided. Figure 3 The three states of a state machine and their switching conditions are explained in detail.
[0109] As attached Figure 3 As shown, the fuse state machine includes a closed state, an open state, and a half-open state. The system defaults to the closed state after initialization.
[0110] I. Closed State In the closed state, the system forwards requests to third-party dependencies normally. Simultaneously, the system continuously collects synchronous API request metrics and asynchronous Webhook receipt metrics to assess the health of third-party dependencies in real time. The state machine transitions from the closed state to the open state when any of the following conditions are detected: Health level is below the preset threshold; A series of anomalies occurred (such as consecutive timeouts or consecutive Webhook receipt failures).
[0111] II. Open Status In the disconnected state, the system no longer forwards requests normally, but instead executes a circuit breaker protection strategy: for requests destined for third-party dependencies, it performs fast failure (returning an error directly), rate limiting (allowing only a small number of requests to pass through), or task pausing (encapsulating the request and storing it in a compensation queue). The disconnected state lasts for a preset cooldown time. After the cooldown time is reached, the state machine automatically switches from the disconnected state to the half-open state.
[0112] III. Half-Open State In the half-open state, the system only allows a small amount of probe traffic (such as proportionally limited probe requests or dedicated health check requests) to verify whether third-party dependencies have recovered. Based on the probe results, the state machine may switch in one of the following two ways: Restoring to a closed state: When several consecutive probes are successful and the health status returns to normal, the system determines that the third-party dependency has been restored, and the state machine switches from a half-open state back to a closed state. After restoring closure, the system gradually releases the backlog of tasks in the compensation queue and notifies the front end to exit the degradation mode.
[0113] Rollback to disconnected state: When the number of failed probes reaches the threshold or the health status drops again, the system determines that the third-party dependency has not been restored or a secondary anomaly has occurred. The state machine rolls back from the half-open state to the disconnected state and re-enters the cooldown waiting period.
[0114] IV. Summary of State Transition The entire state transition process can be summarized as follows: Close → Open: Health level is below the threshold or continuous abnormal triggers; Disconnect → Partially Open: Triggered when cooldown time is reached; Half-open → Closed: Triggered by successful continuous detection and recovery of health; Half-open → Disconnected: Triggered by detection failure or health dropping again.
[0115] Through append Figure 3 The state switching and recovery detection process shown in this invention enables dynamic protection against external dependencies: rapid circuit breaking in case of anomalies, careful detection after cooling, and smooth reconnection after recovery, avoiding traffic surges during recovery and ensuring system stability and business continuity.
[0116] To ensure the plan is clearly described, the following will combine... Figure 4 The timing diagrams for read degradation, write compensation, and status dashboard during abnormal periods are presented to illustrate the invention in detail.
[0117] As attached Figure 4 As shown, this sequence diagram involves six interactive objects: agent workbench, front-end degradation route, integration gateway, compensation queue, third-party CRM, and status dashboard. The entire process is divided into a normal phase, an abnormal phase, and a recovery phase.
[0118] I. Normal Stage The agent's workstation initiates the "Open Customer Information" operation, triggering a "Query Customer Information" request.
[0119] The request, determined to be in a normal state by the front-end fallback routing, selects the "normal read" path and forwards it to the integration gateway. The integration gateway then calls the third-party CRM's query interface via a synchronous API. The third-party CRM processes the request normally and returns real-time data. The integration gateway then returns the real-time data to the front-end fallback routing, ultimately displaying it on the agent's workbench interface, completing the normal call.
[0120] During this process, the integration gateway continuously collects synchronous API call metrics (time consumption, status codes, etc.) and subsequent asynchronous receipt metrics, and reports the data to the status dashboard for health assessment.
[0121] II. Third-Party Dependency Anomaly Phase When a third-party CRM experiences high latency, continuous timeouts, or intermittent errors, the integration gateway detects the timeout or error response (such as "timeout or error") and reports the abnormal indicators to the health assessment module. If the overall health drops below the disconnect threshold, or if continuous anomalies are detected, the circuit breaker state machine switches from closed to open.
[0122] The integrated gateway pushes status change and time comparison information to the status dashboard through the status synchronization module, displaying health decline, circuit breaker status switching, and internal and external time comparisons. At the same time, the integrated gateway sends a dependency disconnection event to the front-end degraded route, notifying the third-party dependency that the circuit breaker has been triggered.
[0123] The agent's workbench executes the "Open Customer Profile" operation again. After receiving the disconnection event, the front-end degradation route no longer forwards the request to the integration gateway, but instead "switches to local cache mode" and reads historical snapshots from the local cache (such as IndexedDB). The front-end degradation route returns the cached snapshot, and the agent's workbench "displays the cached snapshot and degradation prompts" (such as "Cache mode, data may not be up-to-date").
[0124] If a write-type service initiated by the agent's workbench (such as writing back the call detail record after the call ends) arrives during the circuit breaker period, the integrated gateway will no longer forward it to the third-party CRM. Instead, it will encapsulate the request as a task to be compensated (carrying an idempotent key), execute "push request / write back data" and store it in the compensation queue.
[0125] The status dashboard displays the health decline curve of the third-party CRM, the circuit breaker status (disconnected), the number of blocked requests, the backlog of the compensation queue, and the comparison of internal and external time consumption in real time, providing operation and maintenance personnel with a visual basis for troubleshooting.
[0126] After the circuit breaker state machine enters the half-open state after the cooling-off period, the integrated gateway sends a probe request to a third-party CRM (such as a lightweight health check interface). The third-party CRM responds normally, and the probes succeed several times in a row. The health assessment module determines that the health status has recovered to above the closing threshold, and the circuit breaker state machine switches from half-open back to closed.
[0127] The integrated gateway notifies the front-end degraded route to "exit degraded mode" and pushes a status recovery event to the status dashboard.
[0128] The integrated gateway retrieves the backlog of tasks to be compensated from the compensation queue, "gradually replays" the compensation tasks at a preset rate, and resends them to the third-party CRM with idempotent keys. The third-party CRM avoids duplicate writes based on the idempotent keys and returns a success response.
[0129] After receiving the notification to exit fallback mode, the front-end degraded route resumes the "normal read" path. When the agent's workbench queries customer information again, it will retrieve real-time data from the integration gateway again, restoring normal business operations.
[0130] Through append Figure 3 As shown in the sequence flow, this invention achieves the following effects during third-party dependency exceptions: Read degradation: The agent's workbench automatically switches to the local cache to avoid blank pages or timeout errors; Write compensation: Write tasks are temporarily stored in a queue to prevent data loss; Status visualization: The status dashboard displays health status, circuit breaker status, and time consumption comparison in real time to assist in quick troubleshooting; Recovery loop: After successful detection, the compensation task is replayed step by step and the front end is notified to exit the degradation, so as to achieve a smooth switchback.
[0131] This timing diagram fully illustrates the end-to-end protection mechanism of this invention, from normal call, abnormal circuit breaking, degradation processing to recovery and replay.
[0132] The following is a preferred scenario embodiment of the present invention: Scenario Description: A PBX workbench integrates a third-party CRM for incoming call pop-up, customer information query, and call detail record (CDR) write-back after a call. Normally, agents initiate a customer information query upon receiving a call, and the system pushes a CDR summary and call metadata to the third-party CRM after the call ends. When the third-party CRM experiences persistent high latency or intermittent unavailability, this invention operates as follows: 1. Two-way anomaly detection: The gateway continuously records the timeout rate of the CRM query interface, and simultaneously records the confirmation time and failure rate of the corresponding Webhook or confirmation interface for call detail record (CDR) write-back. When the overall health status drops below the threshold, the third-party dependency is determined to be in a high-risk state.
[0133] 2. Trigger circuit breaker and protect the main link: The circuit breaker state machine switches to the disconnected state, the gateway stops sending unnecessary requests to the abnormal CRM, and synchronizes the disconnected state to the front-end workbench.
[0134] 3. Automatic degradation of read-based business: After receiving a status event, the agent workbench no longer requests CRM in real time, but reads the most recently synchronized customer data snapshot from the local cache and displays a cache mode prompt on the interface.
[0135] 4. Compensation for write-type business: Data that is not written back to CRM in time after the call ends is encapsulated as a task to be compensated, written to a persistent queue and carrying an idempotent key to avoid repeated writing after recovery.
[0136] 5. Dashboard display of status and responsibility information: The management backend synchronously displays the CRM health status decline, circuit breaker status, number of blocked requests, backlog compensation amount, and the comparison results of the processing time of the communication system and the processing time of the external interface, to help locate the source of the anomaly.
[0137] 6. Smooth rollback after recovery: After the cooldown period ends, the system enters a half-open state and verifies whether the CRM has recovered by probing traffic; if the continuous probing is successful, the closed state is restored, the compensation queue is released, and the front end is notified to exit the cache mode.
[0138] Based on the above embodiments, the present invention may also include the following embodiments, which can be implemented individually or combined with each other without technical conflict.
[0139] 1. Desktop client implementation based on local proxy.
[0140] For desktop applications that cannot directly access local caching capabilities (such as IndexedDB) within a browser, this invention employs a local lightweight proxy process as an alternative. This proxy process runs independently in the operating system background, responsible for maintaining cache snapshots, synchronizing state with the integration gateway, and managing compensation tasks. The desktop application's main process and the local proxy process interact via inter-process communication (such as named pipes, local sockets, or HTTP local services). When a circuit breaker occurs, the proxy process switches the data source for read-based applications to the local cache and encapsulates write-based applications as compensation tasks, storing them in a local persistent queue. Once the third-party dependencies are restored, the proxy process performs a replay. This implementation allows desktop applications in non-browser environments to also enjoy the full-stack degradation protection of this invention.
[0141] 2. Multi-dependency group circuit breaker implementation method When a single third-party platform (such as a large CRM system) contains multiple interface domains (such as customer interface, order interface, and work order interface) or multiple tenant instances, to avoid the impact of an anomaly in one interface or tenant on all dependencies, this invention can maintain independent health and circuit breaker states according to interface group, tenant identifier, or region. Specifically, the system assigns an independent external dependency identifier to each interface group or each tenant instance, collects corresponding synchronous API metrics and asynchronous Webhook receipt metrics, independently calculates health, and runs its own circuit breaker state machine. In this way, when an anomaly occurs in a certain interface group or a certain tenant, only that specific sub-dependency is circuit-broken, while calls from other interface groups or other tenants can still proceed normally, thereby isolating the fault within a minimal scope and improving the system's granular management capabilities.
[0142] 3. Implementation methods for predictive early warning.
[0143] With historical time-series data available, this invention can further introduce a predictive early warning mechanism. The system performs trend analysis on the historical health curves of each external dependency (e.g., using linear regression, moving average, or lightweight time-series prediction models) to predict the health trend over a future period. When the prediction results indicate that the health may reach the disconnection threshold in the short term, the system enters an early warning state in advance, proactively taking preventative measures before the circuit breaker is triggered. These measures include increasing the refresh frequency of the front-end local cache, reducing unnecessary calls to third-party dependencies (e.g., reducing heartbeat frequency, merging query requests), or notifying operations and maintenance personnel to intervene and check in advance. Specific parameters in the early warning state (e.g., prediction model type, early warning threshold, cache refresh rate) can be configured according to the actual business scenario. This implementation allows for proactive intervention before a failure actually occurs, further reducing the risk of business interruption.
[0144] 4. Implementation method for automatically generating evidence packages.
[0145] To improve the efficiency of fault diagnosis and responsibility determination, this invention can automatically generate a structured fault summary after an alarm is triggered (i.e., when the circuit breaker status switches from closed to open). This fault summary includes at least the following information: fault time window (start and end times), target external dependency identifiers, circuit breaker status switching records (the time point of each status change and the status before and after the change), key health indicators within the fault window (such as average timeout rate, error rate, and receipt failure rate), statistical information on compensation tasks (such as the number of backlogged tasks, the number of successful retries, and the number of final failures), and internal and external time consumption comparison curves. The fault summary can be exported in JSON, XML, PDF, or HTML format and supports automatic uploading to the maintenance work order system, sending it to the relevant responsible person's email address, or displaying it directly on the status dashboard. This implementation provides objective, complete, and standardized evidence for internal troubleshooting and communication with third-party service providers, significantly reducing communication costs and time required for responsibility determination.
[0146] This invention also provides a communication system external dependency protection system, comprising: The two-way health perception module is used to collect synchronous API request metrics sent to third-party systems, as well as metrics from asynchronous Webhook, callback confirmation, or equivalent receipt links. The collected synchronous API request metrics and asynchronous Webhook receipt metrics are uniformly aggregated based on the same external dependency identifier to form two-way health time series data. The health assessment and threshold determination module is used to statistically analyze the API call timeout rate, error rate, Webhook receipt failure rate and average confirmation delay in the bidirectional health time series data based on a sliding time window or a fixed sample window, and calculate the comprehensive health score according to preset weights. The circuit breaker state machine and the hierarchical interception module are used to switch between closed state, open state and half-open state according to the comprehensive health status, and to execute the interception strategy for requests sent to third-party systems in the open state. The front-end degradation routing and cache reading module is used to synchronize the state change event to the front-end workbench through WebSocket, SSE, polling configuration interface or equivalent method when the circuit breaker state machine switches to the disconnected state, so that the front-end workbench switches the data source of the read-type business to the local cache. The write compensation and idempotent replay module is used to encapsulate the write-type business into a compensation task that is at least associated with the target dependency identifier, the business object identifier and the idempotent key when the circuit breaker state machine switches to the open state, and store it in the persistent queue. The recovery detection and convergence module is used to gradually release the tasks to be compensated in the persistent queue when the circuit breaker state machine recovers to the closed state, perform idempotent replay according to the idempotent key, and notify the front-end workbench to exit the cache mode through WebSocket, SSE, polling configuration interface or equivalent methods.
[0147] Since the system implementation and the method implementation are similar in principle, please refer to the method implementation for details, and will not be repeated here.
Claims
1. A method for protecting external dependencies in a communication system, characterized in that, include: Collect synchronous API request metrics sent to third-party systems, and simultaneously collect metrics from asynchronous Webhook, callback confirmation, or equivalent receipt links. The collected synchronous API request metrics and asynchronous Webhook receipt metrics are uniformly aggregated based on the same external dependency identifier to form bidirectional health time-series data. Based on a sliding time window or a fixed sample window, the API call timeout rate, error rate, Webhook receipt failure rate, and average confirmation delay in the bidirectional health time series data are statistically analyzed. The overall health score is calculated according to a preset weight, and the circuit breaker state machine is driven to switch between closed, open, and half-open states based on the overall health score. When the circuit breaker state machine switches to the disconnected state, the state change event is synchronized to the front-end workbench via WebSocket, SSE, polling configuration interface or equivalent method, so that the front-end workbench switches the data source of read-type business to local cache, and encapsulates write-type business into a compensation task with at least associated target dependency identifier, business object identifier and idempotent key, and stores it in persistent queue. When the circuit breaker state machine returns to the closed state, the tasks to be compensated in the persistent queue are gradually released, idempotent replay is performed according to the idempotent key, and the front-end workbench is notified to exit the cache mode through WebSocket, SSE, polling configuration interface or equivalent method.
2. The method according to claim 1, characterized in that, The metrics collected include synchronous API request metrics sent to third-party systems, as well as metrics collected from asynchronous Webhooks, callback confirmations, or equivalent receipt links, including: Collect synchronous API request metrics sent to third-party systems. The synchronous API request metrics include at least the target system identifier, interface identifier, request initiation time, response time, status code, timeout flag, and failure reason. Simultaneously, metrics are collected from asynchronous Webhook, callback confirmation, or equivalent receipt links. These metrics include at least delivery time, confirmation time, confirmation status, number of retries, and final result.
3. The method according to claim 1, characterized in that, The method involves statistically analyzing the API call timeout rate, error rate, Webhook receipt failure rate, and average confirmation latency in the bidirectional health time-series data based on a sliding time window or a fixed sample window, and calculating the overall health score according to preset weights, including: Using a sliding time window or a fixed sample window as the statistical unit, count the total number of API calls made by the third-party system corresponding to the external dependency identifier within the window. Timeout and number of errors and the total number of Webhook deliveries and number of failures The average confirmation delay is calculated by dividing the sum of confirmation delays for successful Webhook receipts by the number of successful receipts. ; Based on the timeout number With total number of calls The ratio is used to calculate the API call timeout rate T, based on the number of errors. With total number of calls The error rate E is calculated based on the ratio of the number of failures. Total number of deliveries The ratio is used to calculate the Webhook receipt failure rate. ; According to the preset weight Through formula Calculate the overall health score ,in, ; When the number of valid samples used for statistics is lower than a preset threshold, the third-party system corresponding to the external dependency identifier is marked as an observation state to avoid false judgments during cold start. The number of valid samples is the sum of the total number of API calls used for statistics and the total number of Webhook deliveries.
4. The method according to claim 1, characterized in that, The step of driving the fuse state machine to switch between closed, open, and partially open states based on the comprehensive health status includes: Maintain a state machine for each external dependency identifier corresponding to the third-party system, which includes at least closed, open and half-open states. When the overall health score is lower than the preset disconnection threshold, or when consecutive timeouts or consecutive Webhook receipt failures are detected within the preset time window, the state machine will be switched from the closed state to the disconnected state. In the disconnected state, at least one of the following blocking strategies is executed for requests sent to the external dependency: fast failure, rate limiting, or whitelisting. After a preset cooling time in the disconnected state, the state machine is switched from the disconnected state to the half-open state to assess whether the third-party system has recovered through a small number of probe requests or dedicated probe requests.
5. The method according to claim 1, characterized in that, The process of synchronizing state change events to the front-end workbench via WebSocket, SSE, polling configuration interface, or equivalent methods includes: When the circuit breaker state machine undergoes a state change, the event containing the dependency identifier and the current circuit breaker state is synchronized to the front-end workbench via WebSocket, SSE, polling configuration interface, or equivalent methods.
6. The method according to claim 1, characterized in that, The step of switching the data source for read-based business operations to a local cache in the front-end workbench includes: After receiving an event that the circuit breaker status is disconnected, the front-end workbench switches the data reading path from real-time request to third-party interface to reading local cache for read-type business related to the dependency. The local cache adopts IndexedDB, local database or equivalent persistent storage. The front-end workbench displays a status message indicating that it is currently in degraded mode. The message includes at least dependency status information, an identifier that the data source is cached, and a message indicating that the data may not be up-to-date.
7. The method according to claim 1, characterized in that, The process of encapsulating write-type business operations into a compensation task that is at least associated with a target dependency identifier, a business object identifier, and an idempotent key, and storing it in a persistent queue includes: During the circuit breaker period, intercept write requests that should have been sent to external dependencies that are in a disconnected state; The write-type business request is encapsulated into a task to be compensated, and the task is associated with at least a target dependency identifier, a business object identifier, a payload summary, a retry status, and an idempotent key to prevent duplicate writes; The task to be compensated is written into a persistent queue, which is implemented as a server-side dead-letter queue, a local persistent queue, or a combination of the two.
8. The method according to claim 1, characterized in that, It also includes steps for status visualization and output of evidence of responsibility: The management backend interface displays a third-party dependency topology diagram, the trend curve of the overall health, the current circuit breaker status, the number of requests blocked due to the circuit breaker, the backlog of the persistent queue, and a comparison of internal processing time and external interface processing time. When an anomaly occurs, the system responds by either operating or automatically generating screenshots, reports, or structured event records from the health curve, status transition records, and internal and external time consumption comparison results within the time window related to the current fault, which can then be used as a basis for troubleshooting.
9. The method according to claim 1, characterized in that, When the circuit breaker state machine returns to the closed state, the tasks to be compensated in the persistent queue are gradually released, idempotent replay is performed according to the idempotent key, and the front-end workbench is notified to exit the cache mode via WebSocket, SSE, polling configuration interface, or equivalent method, including: When the fuse state machine is in a half-open state, only detection requests that meet the proportional limit or special health check requests are allowed. When several consecutive detections are successful and the overall health level is again above the closing threshold, the fuse state machine is restored from the half-open state to the closed state. After restoring to the closed state, the tasks to be compensated in the persistent queue are gradually released and idempotently replayed according to the idempotent keys they carry. Simultaneously, a notification to exit the degraded mode is sent to the front-end workbench via WebSocket, SSE, or polling configuration interface, so that the front-end workbench can resume the data reading path for real-time requests to third-party interfaces.
10. A communication system external dependency protection system, characterized in that, include: The two-way health perception module is used to collect synchronous API request metrics sent to third-party systems, as well as metrics from asynchronous Webhook, callback confirmation, or equivalent receipt links. The collected synchronous API request metrics and asynchronous Webhook receipt metrics are uniformly aggregated based on the same external dependency identifier to form two-way health time series data. The health assessment and threshold determination module is used to statistically analyze the API call timeout rate, error rate, Webhook receipt failure rate and average confirmation delay in the bidirectional health time series data based on a sliding time window or a fixed sample window, and calculate the comprehensive health score according to preset weights. The circuit breaker state machine and the hierarchical interception module are used to switch between closed state, open state and half-open state according to the comprehensive health status, and to execute the interception strategy for requests sent to third-party systems in the open state. The front-end degradation routing and cache reading module is used to synchronize the state change event to the front-end workbench through WebSocket, SSE, polling configuration interface or equivalent method when the circuit breaker state machine switches to the disconnected state, so that the front-end workbench switches the data source of the read-type business to the local cache. The write compensation and idempotent replay module is used to encapsulate the write-type business into a compensation task that is at least associated with the target dependency identifier, the business object identifier and the idempotent key when the circuit breaker state machine switches to the open state, and store it in the persistent queue. The recovery detection and convergence module is used to gradually release the tasks to be compensated in the persistent queue when the circuit breaker state machine recovers to the closed state, perform idempotent replay according to the idempotent key, and notify the front-end workbench to exit the cache mode through WebSocket, SSE, polling configuration interface or equivalent methods.