Data service fault self-recovery method and device based on event-driven architecture, electronic equipment and storage medium

Abstract

The application discloses a data service fault self-recovery method and device based on an event-driven architecture, electronic equipment and a storage medium, which are applied to a data service cluster server. The method comprises the following steps: a scheduling task is issued to all service instances in the data service cluster in a sharding broadcast mode, and cluster instances are determined as a Leader instance for executing a business and a Candidate instance for standby according to a lock competition result; an external data source access process is completed by the Leader instance in an event-driven mode, the Leader instance enters a continuous data receiving running state after successful access, and link exceptions and data exceptions are monitored in real time during the running process and abnormal events are triggered; and the abnormal events generated during the running process are accumulated and counted, and a self-recovery strategy is executed according to the abnormal accumulated number of times. Through the application, invalid cluster switching is reduced, automatic fault processing and non-sensing primary and backup takeover are realized, and the continuity and high availability of financial market data service are improved.

Description

Technical Field

[0001] This application relates to the fields of financial data services, distributed cluster high availability and fault self-healing technology, and in particular to a data service fault self-healing method, device and electronic device and storage medium based on event-driven architecture. Background Technology

[0002] In the financial industry, banks and other financial institutions routinely connect to authoritative external market data sources such as Bloomberg B-PIPE and Reuters to obtain real-time market data for financial products such as foreign exchange, securities, and bonds. This provides data support for downstream core businesses such as transaction pricing, risk management, and market analysis. These external market data sources typically use TCP, WebSocket, or proprietary long-lived connections to continuously push incremental market data to subscribing clients. Clients must sequentially complete long-lived connection establishment, identity authorization authentication, and subscription to specific business products before receiving real-time data pushes, placing extremely high demands on service continuity, real-time performance, and data accuracy.

[0003] To meet high availability requirements, existing financial market data acquisition systems generally adopt a microservice multi-instance cluster deployment model. However, external market data sources have limitations on the number of connections a single institutional account can make. Multiple instances establishing connections simultaneously can lead to issues such as duplicate subscriptions, duplicate data pushes, data source rate limiting, and resource contention. Therefore, in cluster deployment scenarios, it is necessary to ensure that only a single valid instance executes business data acquisition tasks at any given time. Currently, the mainstream high availability solutions in the industry mainly include master-slave failover schemes and leader election schemes based on distributed locks. However, both of these existing technologies have many inherent technical defects in practical production applications and cannot fully adapt to the high-precision, high-continuity, and high-self-healing operational requirements of financial market data.

[0004] The existing technology is a traditional primary / standby failover solution, which deploys one primary and one standby instance. The primary instance's status is monitored via a heartbeat mechanism, and the standby instance takes over the service in case of primary instance failure. This solution has significant drawbacks: First, heartbeat detection has a fixed time interval, resulting in a delay of seconds or even minutes between fault detection and primary / standby failover, creating a window of data disconnection and failing to meet the millisecond-level continuous data collection requirements of the financial sector. Second, it relies on independent external monitoring or scheduling components for status management, introducing additional middleware single-point-of-failure risks and increasing system architecture complexity. Third, failover is passively triggered, making it prone to frequent primary / standby failovers due to intermittent, transient faults such as network jitter, leading to poor system stability and a lack of automated self-healing capabilities. Fault recovery largely depends on manual intervention, resulting in low recovery efficiency.

[0005] The second existing technology is a cluster leader election scheme based on distributed locks. It uses middleware such as ZooKeeper, etcd, and Redis to achieve distributed lock contention, dynamically elect a unique leader instance to be responsible for market data collection, and triggers cluster re-election after the lock is released or expires. Compared to the primary / standby mode, this solution improves cluster scheduling flexibility, but it still has several key technical shortcomings: First, the fault self-healing strategy is simplistic and crude. For faults that can be recovered instantly, such as link anomalies and network fluctuations, it directly releases the distributed lock and triggers node re-election without setting a retry self-healing mechanism, which can easily cause unnecessary business interruptions and service jitter. Second, it lacks fault classification and handling capabilities, and cannot distinguish between instantaneous and persistent faults. The fault handling logic is one-size-fits-all, resulting in poor service fault tolerance. Third, there is a risk of lock starvation. When the external market data source is unavailable for a long time, instances repeatedly compete for locks, fail to establish connections, and release locks, causing other standby instances to be unable to obtain execution permissions for a long time, resulting in unbalanced cluster load scheduling. Fourth, it lacks a standardized full lifecycle management mechanism. The logic of each stage, such as data source connection, data reception, exception handling, and connection disconnection, is scattered, and there is no unified standard for fault handling, resulting in poor maintainability and scalability. Fifth, it does not optimize time-series data for high-concurrency market push scenarios. When multiple threads concurrently write to storage, data overwriting and time-series disorder problems are prone to occur, which seriously affect the accuracy of subsequent time-series market statistics and fluctuation analysis.

[0006] In summary, existing market data service clusters generally suffer from single-point-of-failure risks, weak self-healing capabilities, inaccurate master-slave failover, unbalanced cluster lock scheduling, fragmented business process control, and poor consistency of concurrent time-series data. These issues make it difficult to meet the 24 / 7 requirements for high continuity, high reliability, and high self-healing in the collection and storage of financial market data. Therefore, this invention proposes a data service fault self-healing method based on an event-driven architecture. Summary of the Invention

[0007] This application provides a data service fault self-healing method, device, electronic device, and storage medium based on an event-driven architecture. It achieves unified modeling of the entire lifecycle of market data access through an event-driven architecture, decoupling and managing business processes and fault self-healing logic. By setting a tiered self-healing strategy with instantaneous fault retry and continuous fault silent degradation, it avoids unnecessary data interruptions caused by the one-size-fits-all lock release and reselection in traditional solutions. It avoids cluster lock starvation through an instance silent cooling mechanism and balances cluster instance scheduling permissions. Simultaneously, it leverages the atomic concurrent write capability of Redis Lua scripts to solve the timing disorder problem in multi-threaded market data push scenarios. This application can also dynamically allocate cluster instance roles using sharded broadcast scheduling and distributed lock contention, monitor link and data anomalies using an event-driven mechanism, implement tiered fault self-healing, and allow backup instances to compete for promotion to primary instances.

[0008] The embodiments of this application adopt the following technical solutions:

[0009] In a first aspect, embodiments of this application provide a data service fault self-healing method based on an event-driven architecture, wherein the method is applied to a data service cluster server and includes:

[0010] The scheduling task is distributed to all service instances in the data service cluster through a segmented broadcast method. Based on the lock contention result, the cluster instances are respectively determined as Leader instances to execute business and Candidate instances to standby.

[0011] The Leader instance completes the external data source access process via an event-driven approach. Upon successful access, it enters a continuous data receiving and running state, and during this process, it monitors for link and data anomalies in real time and triggers exception events accordingly.

[0012] An abnormal event generated during operation is cumulatively counted, and a self-healing strategy is executed in stages according to the cumulative number of abnormal events. Each Candidate instance continuously monitors the distributed lock status, and can be upgraded to a Leader instance after successfully competing for and acquiring the distributed lock.

[0013] In some embodiments, the step of accumulating and counting abnormal events generated during operation, and executing a self-healing strategy in stages based on the cumulative number of abnormal events, includes:

[0014] When the number of exceptions does not reach the preset threshold, the data source connection will be re-initiated after a fixed retry interval.

[0015] When the number of anomalies reaches a preset threshold, it is determined to be a persistent fault. The distributed lock is actively released and the current instance is downgraded to a Candidate instance, entering a silent cooling state. After the silent cooling ends, the anomaly count is reset, allowing the current instance to re-participate in the cluster's distributed lock contention.

[0016] In some embodiments, each Candidate instance continuously monitors the distributed lock status and, upon successfully acquiring the distributed lock, can be upgraded to a Leader instance, including:

[0017] Each Candidate instance continuously performs polling hot standby failover operations, polling in real time to detect the holding and expiration status of the distributed lock; when a lock is detected to be idle or expired, it participates in the distributed lock contention. After successfully competing, it is promoted to Leader instance and automatically takes over data service tasks, realizing automatic master-slave failover of the cluster.

[0018] In some embodiments, the method further includes:

[0019] For multi-threaded concurrent data writing scenarios, Redis ordered sets are used to store market data. The Redis server-side Lua scripts atomically perform duplicate data verification, auto-incrementing sequence number completion, ordered writing of time-series data, and set capacity pruning operations to avoid multi-threaded concurrency race conditions and ensure the time-series ordering and data consistency of market data in concurrent writing scenarios.

[0020] In some embodiments, before distributing the scheduling task to all service instances within the data service cluster via fragmented broadcast, the method further includes:

[0021] After startup, preset configuration parameters are loaded, including external market data source connection parameters, foreign exchange data subscription list, distributed lock database configuration, fault self-healing parameters, and time scheduling plan;

[0022] Initialize the event-driven lifecycle manager and establish the mapping relationship between events and business processing functions;

[0023] Initialize the fault self-healing engine and load the hierarchical self-healing strategy.

[0024] In some embodiments, the Leader instance manages the entire service lifecycle in an event-driven manner, abstracting the entire process of external data source access, data reception, exception handling, and connection disconnection into a standardized event model, thereby decoupling the normal business process from the fault self-healing logic.

[0025] In some embodiments, during the silent cooling state, the downgraded Candidate instance suspends participation in distributed lock contention and rejoins the cluster lock contention queue after the cooling period ends and the exception count is reset.

[0026] Secondly, embodiments of this application also provide a data service fault self-healing device based on an event-driven architecture, wherein the device is applied to a data service cluster server and includes:

[0027] The distributed lock contention module is used to distribute scheduling tasks to all service instances in the data service cluster through a sharded broadcast method. Based on the lock contention result, the cluster instances are respectively determined as Leader instances to execute business and Candidate instances to standby.

[0028] The Leader lifecycle management module is used by the Leader instance to complete the external data source access process via an event-driven approach. After successful access, it enters a continuous data receiving and running state, and during the running process, it monitors link anomalies and data anomalies in real time and triggers anomaly events; and

[0029] The self-healing strategy module is used to accumulate and count abnormal events generated during operation, and execute self-healing strategies in stages according to the number of accumulated abnormal events. Each Candidate instance continuously monitors the distributed lock status, and can be upgraded to a Leader instance after successfully competing for and acquiring the distributed lock.

[0030] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the above-described method.

[0031] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the above-described method.

[0032] The above-mentioned technical solutions adopted in this application embodiment can achieve the following beneficial effects: Scheduling tasks are distributed to all service instances within the data service cluster via sharded broadcast, and cluster instances are determined as Leader instances executing business and Candidate instances waiting in standby mode based on lock contention results. Based on sharded broadcast scheduling combined with lock contention for leader selection, decentralized automatic task allocation is achieved, avoiding the risks of multi-instance concurrency conflicts and single-point failures. This application embodiment distributes scheduling tasks uniformly to all service instances in the cluster via sharded broadcast, automatically distinguishing Leader instances executing business and Candidate instances waiting in standby mode based on a distributed lock contention mechanism. Compared to traditional fixed master / slave and manually configured role solutions, it does not rely on external monitoring and scheduling components, enabling dynamic adaptive allocation of cluster instance roles. It effectively solves the problems of duplicate connections, duplicate data pushes, data source rate limiting, and resource waste caused by multiple instances simultaneously connecting to external market data sources in distributed deployment scenarios. Simultaneously, it abandons the fixed single-point operation mode, avoiding the overall data link interruption caused by single-instance downtime or failure at the architectural level, and improving the security and standardization of market data collection service cluster deployment.

[0033] The Leader instance completes the external data source access process via an event-driven approach. After successful access, it enters a continuous data receiving and running state, and monitors for link and data anomalies in real time during operation, triggering abnormal events accordingly. By relying on an event-driven architecture to manage the entire data source process, accurate anomaly detection is achieved, improving service controllability and stability. This embodiment innovatively adopts an event-driven approach to manage the entire external data source access process of the Leader instance, uniformly completing the end-to-end control of data source access, continuous data reception, and real-time anomaly monitoring. It can accurately capture link and data anomalies during operation and trigger corresponding abnormal events. Unlike existing technologies that suffer from scattered fault logic, delayed anomaly detection, and difficulty in problem localization, this solution uses an event-driven mechanism to standardize business processes, make abnormal events perceptible, and ensure traceable fault triggering. This highly streamlines the market data access, operation, and anomaly handling processes, significantly reducing the probability of service anomalies and ensuring the continuous and stable operation of the data receiving link.

[0034] Anomalies generated during operation are cumulatively counted, and a tiered self-healing strategy is implemented based on the cumulative number of anomalies. Each Candidate instance continuously monitors the distributed lock status, and can be upgraded to a Leader instance after successfully acquiring the distributed lock. This tiered self-healing strategy based on the cumulative number of anomalies solves the problem of a one-size-fits-all approach in traditional fault handling, maximizing the continuity of data services. This embodiment of the application cumulatively counts operational anomalies and implements a tiered self-healing strategy based on the cumulative number of anomalies, which differs from the coarse-grained handling mode of releasing locks and forcibly switching nodes upon the occurrence of anomalies in existing technologies. It can accurately distinguish between instantaneous fluctuation anomalies and persistent faults, and carry out targeted self-healing processing, avoiding invalid node degradation and cluster re-election problems caused by temporary recoverable faults such as network jitter. This significantly reduces unnecessary service interruptions and cluster jitter, and maintains the continuity of market data collection to the greatest extent possible while ensuring that faults are repairable, adapting to the high-stability operation requirements of financial scenarios.

[0035] Furthermore, the Candidate instances continuously monitor the lock status and dynamically upgrade, enabling seamless hot standby switching across the cluster and significantly improving service high availability. In this embodiment, all Candidate instances continuously monitor the distributed lock status. After the lock becomes idle, expires, or is released due to a primary instance failure, the instance can compete to become the Leader instance and take over the business. Compared to the traditional primary-standby switching methods, which suffer from detection delays and switching lags, this solution achieves 24 / 7 hot standby for the cluster, automatic fault takeover, and seamless switching without manual intervention or delay. It completely solves the problems of untimely fault recovery and long data disconnection windows in traditional solutions, comprehensively improving the fault tolerance and high availability performance of financial market data services. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0037] Figure 1 This is a flowchart illustrating the data service fault self-healing method based on an event-driven architecture in an embodiment of this application.

[0038] Figure 2 This is a schematic diagram of the internal structure of the data service fault self-healing device based on event-driven architecture in the embodiments of this application;

[0039] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] In the following embodiments, the external data source is an external market data source (such as Bloomberg B-PIPE, Reuters, and other real-time market data platforms), and the data service cluster is a foreign exchange market data service cluster deployed in the bank's intranet environment. It should be understood that the embodiments of this application are also applicable to other real-time data service scenarios that connect to external data sources through long connections, such as securities market data access, interest rate data access, and third-party information data access, etc., and this invention does not limit these.

[0042] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0043] This application provides a data service fault self-healing method based on an event-driven architecture, such as... Figure 1 The diagram illustrates a data service fault self-healing method based on an event-driven architecture, applicable to a data service cluster server. The method includes at least the following steps S110 to S130:

[0044] Step S110: Distribute scheduling tasks to all service instances in the data service cluster via sharded broadcast. Based on the lock contention results, determine the cluster instances as Leader instances to execute business operations and Candidate instances to standby.

[0045] Specifically, the data service cluster consists of multiple service instances deployed in the intranet environment of financial institutions. Each service instance runs the exact same program and loads the exact same configuration. Each service instance has the complete ability to independently connect to external market data sources.

[0046] In one embodiment, the time scheduler is implemented based on XXL-JOB and its routing strategy is configured as a sharding broadcast (SHARDING_BROADCAST) policy. When the trigger time specified by the preset time scheduling plan arrives (e.g., triggering the start of the scheduling task at 8:50 AM on a weekday and triggering the stop of the scheduling task at 6:10 PM on a weekday), the time scheduler broadcasts the same scheduling task to all service instances in the data service cluster, thereby ensuring that every service instance in the cluster receives the task instruction without relying on an additional scheduling center to manage the status of each instance, thus avoiding the scheduling center itself becoming a single point of failure.

[0047] After receiving the scheduling task, each service instance triggers its own distributed lock contention process. In one embodiment, each service instance first uses a local reentrant lock (ReentrantLock) to achieve thread-level mutual exclusion within the instance: within the same instance, there may be multiple concurrent threads such as threads from the time scheduler network calls and watchdog polling threads. The local reentrant lock ensures that only one thread can enter the subsequent distributed lock contention; the thread that wins the local lock contention then initiates a distributed lock acquisition request to the lock store on behalf of this instance.

[0048] In one embodiment, the distributed lock is implemented based on a database lock table. The lock table includes at least the following fields: lock key name (lock_key, primary key), holder identifier (records the unique identifier of the service instance currently holding the lock, such as the combination of "service name:IP:port"), expiration time (expire_time), and heartbeat time (heartbeat_time).

[0049] Based on the lock contention results, each service instance within the cluster is assigned one of two roles: the service instance that successfully acquires the distributed lock is designated as the Leader instance to execute business logic; the service instance that fails to acquire the distributed lock is designated as a Candidate instance to wait in standby mode. Therefore, at most one Leader instance in the cluster can establish a connection with the external market data source at any given time, avoiding the problems of duplicate connections, duplicate data pushes, and resource waste caused by multiple concurrent connections.

[0050] In step S120, the Leader instance completes the external data source access process through an event-driven approach. After successful access, it enters a continuous data receiving and running state, and monitors link anomalies and data anomalies in real time during the running process and triggers abnormal events.

[0051] Specifically, the event-driven lifecycle manager within the Leader instance abstracts the external data source access process into a series of standardized events, and drives the flow of the access process through these events. In one embodiment, the access process includes:

[0052] (1) The lifecycle manager triggers the connection establishment event and starts to establish a long connection with the external market data source, including: creating a Socket connection to the address and port of the external market data source (the connection type can be a TCP long connection, a WebSocket long connection or a private protocol long connection specified by the external market data source), setting the connection timeout time and read timeout time, and registering event callback handlers such as connection success callback, connection failure callback, data reception callback and connection closure callback.

[0053] (2) If the long connection is successfully established, the lifecycle manager triggers a connection success event and starts the distributed lock renewal task: the watchdog thread periodically updates the heartbeat time and expiration time of the lock record in the database lock table to prevent the lock from being preempted by other instances due to expiration during normal service; and resets the exception counter to 0 to avoid the counting of historical faults affecting the subsequent judgment of new exceptions. If the long connection fails to be established, a connection failure event is triggered, and the fault self-healing process described in the embodiment is entered.

[0054] (3) After a successful connection, the lifecycle manager triggers an authorization event and sends an identity authorization request message containing pre-configured authorization credential information to the external market data source. If an authorization success response is received, an authorization success event is triggered; if an authorization failure response or authorization timeout is received, an authorization failure event is triggered. Since authorization failure is an unrecoverable configuration error, the distributed lock is released directly and the current lifecycle is terminated, and the current instance is converted into a Candidate instance.

[0055] (4) After successful authorization, the lifecycle manager triggers a subscription event and sends a data subscription request message to the external market data source. The message contains a pre-configured list of subscription data types (e.g., a list of foreign exchange pairs, such as EUR / USD, USD / JPY, etc.). If a subscription success response is received, a subscription success event is triggered, and the Leader instance completes the external data source access process and enters a continuous data receiving and running state. If a subscription failure response or subscription timeout is received, a subscription failure event is triggered, the distributed lock is released, and the current lifecycle is terminated. The current instance is converted into a Candidate instance.

[0056] (5) When the data receiving operation is in progress, the lifecycle manager will trigger a data push event for each piece of market data pushed by the external market data source, which will then be handled by the data processing and writing process.

[0057] Meanwhile, during operation, the Leader instance monitors for link and data anomalies in real time. In one embodiment, the triggering of an anomaly event includes at least one of the following: unexpected disconnection of a long-lived connection (such as network interruption or the peer closing the connection); data reception timeout (no data push is received within a preset time); data transmission verification failure (such as incorrect data packet format or checksum mismatch). Once any of the above anomalies is detected, the lifecycle manager immediately triggers the anomaly event and hands it over to the fault self-healing engine for processing.

[0058] Step S130: Accumulate and count the abnormal events generated during the operation, and execute the self-healing strategy in stages according to the number of accumulated abnormal events. Each Candidate instance continuously monitors the distributed lock status, and can be upgraded to a Leader instance after successfully competing for and acquiring the distributed lock.

[0059] Specifically, the fault self-healing engine receives abnormal events triggered by the lifecycle manager (including connection failure events and abnormal events during data reception). Each received abnormal event increments the current Leader instance's exception counter by 1, accumulating the count of abnormal events. Then, a tiered self-healing strategy is executed based on the accumulated exception count: when the accumulated exception count has not reached a preset threshold, a retry self-healing strategy is implemented, attempting to rebuild the connection while retaining the distributed lock; when the accumulated exception count reaches the preset threshold, a lock release and silent cooling self-healing strategy is implemented, proactively relinquishing the Leader role. The detailed implementation of the tiered self-healing strategy is described later.

[0060] In parallel with the Leader instance, each Candidate instance continuously monitors the distributed lock status through its internal watchdog thread. When a lock is detected to be idle or expired, the Candidate instance participates in the distributed lock contention. Upon successfully acquiring the distributed lock, it is promoted to Leader instance and automatically takes over data service tasks, initiating the external data source access process described in step S120, thereby achieving automatic primary / standby failover of the cluster. The polling hot standby failover mechanism of the Candidate instance is detailed below.

[0061] It should be noted that when the time scheduler triggers the stop scheduling task at the preset stop time (e.g., 18:10 on a weekday), the lifecycle manager of the Leader instance triggers a disconnection event, which executes in sequence: stop the distributed lock renewal task, send a disconnection request to the external market data source, close the Socket connection, release the distributed lock in the database, reset the exception counter to 0, and cancel the event callback handler, so that the lifecycle of the data service ends normally according to the predetermined plan.

[0062] In one embodiment of this application, the step of accumulating and counting abnormal events generated during operation and executing a self-healing strategy in stages according to the number of accumulated abnormal events includes: when the number of abnormal events does not reach a preset threshold, re-initiating the reconstruction of the data source connection after a fixed retry interval; when the number of abnormal events reaches the preset threshold, determining it as a persistent fault, actively releasing the distributed lock and downgrading the current instance to a Candidate instance, entering a silent cooling state, and resetting the abnormal count after the silent cooling ends, so that the current instance can re-participate in the cluster distributed lock contention.

[0063] If the number of exceptions does not reach the preset threshold, the data source connection will be re-initiated after a fixed retry interval.

[0064] Specifically, each time the fault self-healing engine receives an abnormal event, it increments the abnormality counter by 1 and checks whether the current value of the abnormality counter has reached the preset maximum number of retries N (e.g., N=3). If it has not reached N, it determines that the current fault is likely a recoverable transient fault (such as a momentary network jitter) and executes the following retry self-healing strategy:

[0065] (1) The fault self-healing engine waits for a preset fixed retry time T1 (e.g., T1=5 seconds) to avoid frequent retries in a short period of time, which would increase the burden on external market data sources and the network. During the waiting period, the distributed lock renewal task continues to run, that is, the watchdog thread continues to periodically update the heartbeat time and expiration time of the lock record in the database, ensuring that the current Leader instance always holds the distributed lock during the retry process, preventing the Candidate instance from preemptively taking over due to lock expiration, thereby ensuring the stability of the cluster role during the retry self-healing period.

[0066] (2) After waiting for T1 time, the fault self-healing engine triggers a retry connection event. In response to this retry connection event, the lifecycle manager first releases any remaining connection resources (closes the socket and clears the buffer), and then re-initiates the data source connection reconstruction with the external market data source, that is, re-executes the access process of connection establishment, authorization, and subscription. If the connection reconstruction is successful, it means that the previous momentary fault has been eliminated. The lifecycle manager resets the exception counter to 0 to avoid the historical count affecting the subsequent new exception judgment, and the Leader instance resumes continuous data receiving operation.

[0067] When the number of anomalies reaches a preset threshold, it is determined to be a persistent fault. The distributed lock is actively released and the current instance is downgraded to a Candidate instance, entering a silent cooling state. After the silent cooling ends, the anomaly count is reset, allowing the current instance to re-participate in the cluster's distributed lock contention.

[0068] Specifically, if the current value of the anomaly counter reaches the preset maximum number of retries N, meaning that the problem has not been resolved after multiple retries, the fault self-healing engine determines that the current fault is a persistent fault (such as unavailability of external market data source services, long-term network interruption, etc.), triggers multiple retry failure events, and executes the following silent self-healing strategy:

[0069] (1) The fault self-healing engine instructs the distributed lock manager to actively release the distributed locks in the database. The release operation is implemented through an atomic database update statement with holder verification to ensure that only the lock holder can release the lock, preventing the accidental release of locks that have been acquired by other instances.

[0070] (2) The current service instance is downgraded from a Leader instance to a Candidate instance and enters a silent cooling state, waiting for a preset silent cooling time T2 (e.g., T2=30 seconds). The purpose of setting the silent cooling state is: firstly, to avoid the current instance repeatedly trying to acquire the lock and failing to connect when the external market data source is continuously unavailable, thus wasting system resources; secondly, to provide other Candidate instances with the opportunity to acquire the lock, preventing lock starvation; and thirdly, to give the external market data source sufficient time to recover from the failure.

[0071] (3) After the silent cooling period T2 expires, the fault self-healing engine triggers the silent period expiration event, resets the exception counter to 0, and the current instance recovers from the silent cooling state and re-participates in the cluster distributed lock competition.

[0072] Furthermore, in one embodiment, boundary case handling is also included: if the time scheduler triggers a preset stop scheduling task during the retry self-healing or silent cooling process, the current service instance immediately interrupts the self-healing process and directly performs normal disconnection operations (stop renewal, disconnect connection, release lock, reset exception counter) to ensure that the data service stops normally according to the predetermined plan.

[0073] Through the hierarchical self-healing strategy of this application embodiment, for recoverable transient faults, the present invention re-establishes the connection while retaining the distributed lock, avoiding unnecessary leader switching and data interruption caused by releasing the lock and re-electing as soon as an anomaly occurs in the prior art; for persistent faults, the lock is released and silent cooling is performed, thereby ensuring the continuity of data services to the greatest extent while maintaining the high availability of the system.

[0074] In one embodiment of this application, each Candidate instance continuously monitors the distributed lock status. After successfully acquiring the distributed lock, it can be upgraded to a Leader instance. This includes: each Candidate instance continuously performs a polling hot standby switchover operation, and polls in real time to detect the holding and expiration status of the distributed lock; when a lock is detected to be idle or expired, it participates in the distributed lock competition, and after successfully competing, it is upgraded to a Leader instance and automatically takes over data service tasks, thereby realizing automatic master-slave switchover of the cluster.

[0075] Specifically, each Candidate instance maintains a watchdog thread, which queries the database for the status of the distributed lock at a fixed polling interval (e.g., once every 2 seconds): it queries the lock record in the lock table using a SELECT statement, and checks in real time whether the holder identifier of the lock record is empty and whether the expiration time has expired (i.e., whether the current time has exceeded expiration time), thereby keeping track of the holding and expiration status of the distributed lock in real time.

[0076] Specifically, when the watchdog thread detects that the lock record's holder is empty or the expire_time has expired, it indicates that the original Leader instance has released the lock (possibly due to a persistent failure, passive expiration due to instance crash causing lock renewal to stop, or release due to normal scheduling termination). The Candidate instance immediately initiates distributed lock contention, that is, it re-executes the local reentrant lock mutual exclusion and database atomic update lock acquisition process described in the embodiment. If the contention is successful, the Candidate instance's role changes from Candidate to Leader, and it automatically takes over the data service tasks, begins executing the external data source access process (connection establishment, authorization, subscription), until it enters a continuous data receiving and running state, providing market data services to the outside world.

[0077] When the watchdog thread detects that the lock record holder is not null and the expire_time has not expired, it means that the current Leader instance is running normally. The Candidate instance continues to remain in standby mode and will continue to poll and detect after the next polling interval.

[0078] Through the polling hot standby switching mechanism in this embodiment, when the Leader instance fails or releases its lock, the Candidate instance can detect and automatically take over within at most one polling interval (e.g., 2 seconds). The entire switching process does not require manual intervention and does not rely on external monitoring systems or scheduling centers, thus realizing rapid automatic switching between cluster primary and backup, and significantly shortening the interruption window of market data links.

[0079] In one embodiment of this application, the method further includes: for multi-threaded concurrent data writing scenarios, using a Redis ordered set to store market time-series data, and using a Redis server-side Lua script to atomically perform duplicate data verification, auto-incrementing sequence number completion, ordered writing of time-series data, and set capacity pruning operations to avoid multi-threaded concurrent race conditions and ensure the time-series orderliness and data consistency of market data in concurrent writing scenarios.

[0080] When the Leader instance is in the data receiving running state and receives market data pushed from an external market data source, the time-series data writer writes the market time-series data to Redis. Since the callback handling function for the data push event executes concurrently in multiple independent worker threads, if each thread directly performs multiple steps such as "query-write-prune" on Redis, a concurrency race condition will occur. This embodiment uses a Redis ordered set to store the market time-series data and atomically executes all write-related operations through a Redis server-side Lua script, specifically including:

[0081] (1) Redis Sorted Set (ZSET) is used to store market time series data. ZSET is a data structure that associates members with their scores. It is implemented internally based on a skip list and a hash table, and supports automatic sorting by score. The insertion time complexity is O(log N). In this embodiment, the timestamp of the market data is used as the score, and the serialized string of the market data (such as JSON format) is used as the member. Each market data record includes at least: data type identifier (such as foreign exchange currency pair name), price value, and timestamp provided by an external market data source; market data of different data types are stored separately using different ZSET key names (such as "fx:price:EUR / USD").

[0082] (2) Perform duplicate data verification, auto-incrementing sequence number completion, time-series data ordered writing, and set capacity pruning operations atomically through Redis server Lua script.

[0083] (3) Downstream query. After the time series data is written, the downstream business system can query all market data in ascending order of timestamp using the ZRANGE key 0-1WITHSCORES command, or obtain the most recent N market data records using the ZREVRANGE key 0 N-1WITHSCORES command, which are used to calculate analysis indicators such as the historical volatility of foreign exchange prices.

[0084] In one embodiment of this application, before the scheduling task is distributed to all service instances in the data service cluster via sharded broadcast, the method further includes: loading preset configuration parameters after startup, the configuration parameters including external market data source connection parameters, foreign exchange data subscription list, distributed lock database configuration, fault self-healing parameters, and time scheduling plan; initializing the event-driven lifecycle manager and establishing the mapping relationship between events and business processing functions; initializing the fault self-healing engine and loading the hierarchical self-healing strategy.

[0085] In one embodiment, the configuration parameters include:

[0086] (1) External market data source connection parameters: connection address, port, authorization certificate information, etc. of the external market data source;

[0087] (2) Foreign exchange data subscription list: a list of foreign exchange data types to be subscribed to, i.e. a list of foreign exchange currency pairs (such as EUR / USD, USD / JPY, etc.).

[0088] (3) Distributed lock database configuration: database connection information and lock table name for the distributed lock;

[0089] (4) Fault self-healing parameters: maximum number of retries N (e.g., N=3), fixed retry time T1 (e.g., T1=5 seconds), silent cooling time T2 (e.g., T2=30 seconds);

[0090] (5) Time scheduling plan: the time to start the data service and the time to stop the data service (e.g., start at 8:50 and stop at 18:10 on weekdays).

[0091] Specifically, register event handlers (processing functions) corresponding to various events, and establish an "event-processing function" mapping table so that when any subsequent event is triggered, the lifecycle manager can call the corresponding business processing function according to the mapping relationship to realize event subscription and distribution.

[0092] Specifically, the fault self-healing engine loads a hierarchical self-healing strategy configuration, including a retry self-healing strategy and a silent self-healing strategy (including parameters such as the maximum number of retries N, fixed retry time T1, and silent cooling time T2), to prepare for subsequent hierarchical handling of abnormal events.

[0093] After the above initialization is completed, each service instance enters a standby state, waiting for the time scheduler to issue scheduling tasks via segmented broadcast.

[0094] In one embodiment of this application, the Leader instance manages the entire service lifecycle based on an event-driven approach, abstracting the entire process of external data source access, data reception, exception handling, and connection disconnection into a standardized event model, thereby decoupling the normal business process from the fault self-healing logic.

[0095] The Leader instance abstracts the entire process of external data source access, data reception, exception handling, and connection termination into a standardized event model.

[0096] In one embodiment, the standardized event model includes the following events:

[0097] Connection establishment event Lifecycle events Time scheduler triggered or retry triggered Start establishing long-term connections with external market data sources Connection success event normal events Long connection established successfully Connection established successfully, proceeding to the next stage. Connection failure event Abnormal events Long connection establishment failed Triggering the self-healing engine Authorization event Lifecycle events Connection established successfully Send an authentication request to an external market data source Authorization success event normal events Authorization verification passed Entering the subscription stage Authorization failure event Abnormal events Authorization verification failed Release the distributed lock and switch to Candidate polling. Subscription Events Lifecycle events Authorization successful Declare the data type to subscribe to (foreign currency pairs) Subscription success event normal events Subscription announcement successful Enter data receiving state Subscription failure event Abnormal events Subscription announcement failed Release the distributed lock and switch to Candidate polling. Data push event normal events External market data source pushes price data Triggering data processing and writing Abnormal events Abnormal events Connection interruption, data transmission error, etc. Trigger the self-healing engine Disconnection event Lifecycle events Time scheduler triggers stop Actively disconnect long-lived connections Retry connection event Self-healing event Fault self-healing engine decision to retry Try to establish a connection again Multiple retry failures Self-healing event The number of retries exceeded the threshold. Enter silent waiting state Silent waiting for the event Self-healing event Enter silent waiting Temporarily abandon to release distributed lock Quiet period expires Self-healing event Silence time has arrived Re-enter Candidate polling and participate in lock contention.

[0098] The above events can be classified into four categories: lifecycle events, normal events, abnormal events, and self-healing events. Lifecycle events drive the phase transitions of the service lifecycle; normal events indicate the normal completion of each phase; abnormal events are uniformly aggregated into the fault self-healing engine for hierarchical processing; and self-healing events drive the state transitions of the self-healing process itself.

[0099] By abstracting the entire process into a standardized event model, the normal business process (connection establishment, authorization, subscription, data reception) is driven by the lifecycle manager according to the event sequence, while the fault self-healing logic (retry, lock release, silent cooling) is triggered by the fault self-healing engine in response to exception events. The two collaborate through events, achieving decoupling between the normal business process and the fault self-healing logic: business developers only need to focus on the implementation of various event handling functions, without needing to embed fault handling logic in the business code, thereby reducing development and maintenance costs, and making the state transition and fault handling logic of the entire data service clear and traceable.

[0100] In one embodiment of this application, during the silent cooling state, the downgraded Candidate instance suspends participation in distributed lock contention and rejoins the cluster lock contention queue after the cooling period ends and the exception count is reset.

[0101] Specifically, when a Leader instance is determined to be a persistent failure due to the cumulative number of anomalies reaching a preset threshold, actively releases the distributed lock, and is downgraded to a Candidate instance, the downgraded Candidate instance enters a silent cooling state. In this silent cooling state:

[0102] (1) The downgraded Candidate instance is suspended from participating in distributed lock contention. That is, during the silent cooling period T2 (e.g., 30 seconds), even if the watchdog thread of the instance detects that the distributed lock is idle or expired, it will not initiate a lock contention request. This avoids the instance immediately re-acquiring the lock and failing to connect again when the external market data source is still unavailable, thus avoiding the ineffective consumption of system resources. At the same time, it allows other Candidate instances that are not in the silent cooling state to have priority to obtain the opportunity to compete for the lock, preventing lock starvation.

[0103] (2) After the cooldown period ends and the exception counter is reset, the instance is rejoined to the cluster lock contention queue. That is, after the silent cooldown period T2 expires, the fault self-healing engine triggers the silent period expiration event, resets the exception counter of the instance to 0, and the instance exits the silent cooldown state, recovers as a normal Candidate instance, and rejoins the cluster lock contention queue: its watchdog thread resumes polling the distributed lock status, and participates in the distributed lock contention normally when it detects that the lock is idle or expired. After successfully competing, it can be upgraded to a Leader instance to take over the data service task again.

[0104] The silent cooling mechanism in this embodiment not only prevents faulty instances from repeatedly and ineffectively contending for the distributed lock, but also reserves time for the recovery of external data sources, while ensuring the fairness of the opportunity for each instance in the cluster to acquire the lock.

[0105] This application also provides a data service fault self-healing device 200 based on an event-driven architecture, such as... Figure 2 As shown, a schematic diagram of a data service fault self-healing device based on an event-driven architecture is provided in an embodiment of this application. The data service fault self-healing device 200 based on an event-driven architecture includes at least: a distributed lock contention module 210, a Leader lifecycle management module 220, and a self-healing strategy module 230, wherein:

[0106] In one embodiment of this application, the distributed lock contention module 210 is specifically used to: issue scheduling tasks to all service instances in the data service cluster through a sharded broadcast method, and determine the cluster instances as Leader instances that execute business and Candidate instances that are waiting in standby according to the lock contention results.

[0107] Specifically, the data service cluster consists of multiple service instances deployed in the intranet environment of financial institutions. Each service instance runs the exact same program and loads the exact same configuration. Each service instance has the complete ability to independently connect to external market data sources.

[0108] In one embodiment, the time scheduler is implemented based on XXL-JOB and its routing strategy is configured as a sharding broadcast (SHARDING_BROADCAST) policy. When the trigger time specified by the preset time scheduling plan arrives (e.g., triggering the start of the scheduling task at 8:50 AM on a weekday and triggering the stop of the scheduling task at 6:10 PM on a weekday), the time scheduler broadcasts the same scheduling task to all service instances in the data service cluster, thereby ensuring that every service instance in the cluster receives the task instruction without relying on an additional scheduling center to manage the status of each instance, thus avoiding the scheduling center itself becoming a single point of failure.

[0109] After receiving the scheduling task, each service instance triggers its own distributed lock contention process. In one embodiment, each service instance first uses a local reentrant lock (ReentrantLock) to achieve thread-level mutual exclusion within the instance: within the same instance, there may be multiple concurrent threads such as threads from the time scheduler network calls and watchdog polling threads. The local reentrant lock ensures that only one thread can enter the subsequent distributed lock contention; the thread that wins the local lock contention then initiates a distributed lock acquisition request to the lock store on behalf of this instance.

[0110] In one embodiment, the distributed lock is implemented based on a database lock table. The lock table includes at least the following fields: lock key name (lock_key, primary key), holder identifier (records the unique identifier of the service instance currently holding the lock, such as the combination of "service name:IP:port"), expiration time (expire_time), and heartbeat time (heartbeat_time).

[0111] Based on the lock contention results, each service instance within the cluster is assigned one of two roles: the service instance that successfully acquires the distributed lock is designated as the Leader instance to execute business logic; the service instance that fails to acquire the distributed lock is designated as a Candidate instance to wait in standby mode. Therefore, at most one Leader instance in the cluster can establish a connection with the external market data source at any given time, avoiding the problems of duplicate connections, duplicate data pushes, and resource waste caused by multiple concurrent connections.

[0112] In one embodiment of this application, the Leader lifecycle management module 220 is specifically used to: complete the external data source access process by the Leader instance through an event-driven approach, enter a continuous data receiving and running state after successful access, and monitor link anomalies and data anomalies in real time during the running process and trigger abnormal events.

[0113] Specifically, the event-driven lifecycle manager within the Leader instance abstracts the external data source access process into a series of standardized events, and drives the flow of the access process through these events. In one embodiment, the access process includes:

[0114] (1) The lifecycle manager triggers the connection establishment event and starts to establish a long connection with the external market data source, including: creating a Socket connection to the address and port of the external market data source (the connection type can be a TCP long connection, a WebSocket long connection or a private protocol long connection specified by the external market data source), setting the connection timeout time and read timeout time, and registering event callback handlers such as connection success callback, connection failure callback, data reception callback and connection closure callback.

[0115] (2) If the long connection is successfully established, the lifecycle manager triggers a connection success event and starts the distributed lock renewal task: the watchdog thread periodically updates the heartbeat time and expiration time of the lock record in the database lock table to prevent the lock from being preempted by other instances due to expiration during normal service; and resets the exception counter to 0 to avoid the counting of historical faults affecting the subsequent judgment of new exceptions. If the long connection fails to be established, a connection failure event is triggered, and the fault self-healing process described in the embodiment is entered.

[0116] (3) After a successful connection, the lifecycle manager triggers an authorization event and sends an identity authorization request message containing pre-configured authorization credential information to the external market data source. If an authorization success response is received, an authorization success event is triggered; if an authorization failure response or authorization timeout is received, an authorization failure event is triggered. Since authorization failure is an unrecoverable configuration error, the distributed lock is released directly and the current lifecycle is terminated, and the current instance is converted into a Candidate instance.

[0117] (4) After successful authorization, the lifecycle manager triggers a subscription event and sends a data subscription request message to the external market data source. The message contains a pre-configured list of subscription data types (e.g., a list of foreign exchange pairs, such as EUR / USD, USD / JPY, etc.). If a subscription success response is received, a subscription success event is triggered, and the Leader instance completes the external data source access process and enters a continuous data receiving and running state. If a subscription failure response or subscription timeout is received, a subscription failure event is triggered, the distributed lock is released, and the current lifecycle is terminated. The current instance is converted into a Candidate instance.

[0118] (5) When the data receiving operation is in progress, the lifecycle manager will trigger a data push event for each piece of market data pushed by the external market data source, which will then be handled by the data processing and writing process.

[0119] Meanwhile, during operation, the Leader instance monitors for link and data anomalies in real time. In one embodiment, the triggering of an anomaly event includes at least one of the following: unexpected disconnection of a long-lived connection (such as network interruption or the peer closing the connection); data reception timeout (no data push is received within a preset time); data transmission verification failure (such as incorrect data packet format or checksum mismatch). Once any of the above anomalies is detected, the lifecycle manager immediately triggers the anomaly event and hands it over to the fault self-healing engine for processing.

[0120] In one embodiment of this application, the self-healing strategy module 230 is specifically used to: accumulate and count abnormal events generated during operation, execute self-healing strategies in stages according to the number of accumulated abnormal events, and each Candidate instance continuously monitors the distributed lock status and can be upgraded to a Leader instance after successfully competing for and acquiring the distributed lock.

[0121] In one embodiment of this application, the self-healing strategy module 230 is further configured to:

[0122] When the number of exceptions does not reach the preset threshold, the data source connection will be re-initiated after a fixed retry interval.

[0123] When the number of anomalies reaches a preset threshold, it is determined to be a persistent fault. The distributed lock is actively released and the current instance is downgraded to a Candidate instance, entering a silent cooling state. After the silent cooling ends, the anomaly count is reset, allowing the current instance to re-participate in the cluster's distributed lock contention.

[0124] Specifically, the fault self-healing engine receives abnormal events triggered by the lifecycle manager (including connection failure events and abnormal events during data reception). Each time an abnormal event is received, the abnormal counter value of the current Leader instance is incremented by 1, thus accumulating the count of abnormal events. Then, a tiered self-healing strategy is executed based on the accumulated number of abnormal events: when the accumulated number of abnormal events has not reached a preset threshold, a retry self-healing strategy is executed, attempting to rebuild the connection while retaining the distributed lock; when the accumulated number of abnormal events reaches the preset threshold, a lock release and silent cooling self-healing strategy is executed, proactively relinquishing the Leader role. The specific implementation of the tiered self-healing strategy is detailed in the following embodiments.

[0125] In parallel with the Leader instance, each Candidate instance continuously monitors the distributed lock status through its internal watchdog thread. When a lock is detected to be idle or expired, the Candidate instance participates in the distributed lock contention. Upon successfully acquiring the distributed lock, it is promoted to Leader instance and automatically takes over data service tasks, initiating the external data source access process described in step S120, thereby achieving automatic primary / standby failover of the cluster. The polling hot standby failover mechanism of the Candidate instance is detailed below.

[0126] It should be noted that when the time scheduler triggers the stop scheduling task at the preset stop time (e.g., 18:10 on a weekday), the lifecycle manager of the Leader instance triggers a disconnection event, which executes in sequence: stop the distributed lock renewal task, send a disconnection request to the external market data source, close the Socket connection, release the distributed lock in the database, reset the exception counter to 0, and cancel the event callback handler, so that the lifecycle of the data service ends normally according to the predetermined plan.

[0127] In one embodiment of this application, the self-healing strategy module 230 is further configured to:

[0128] Each Candidate instance continuously performs polling hot standby failover operations, polling in real time to detect the holding and expiration status of the distributed lock; when a lock is detected to be idle or expired, it participates in the distributed lock contention. After successfully competing, it is promoted to Leader instance and automatically takes over data service tasks, realizing automatic master-slave failover of the cluster.

[0129] In one embodiment of this application, a caching module is also included, for:

[0130] For multi-threaded concurrent data writing scenarios, Redis ordered sets are used to store market data. The Redis server-side Lua scripts atomically perform duplicate data verification, auto-incrementing sequence number completion, ordered writing of time-series data, and set capacity pruning operations to avoid multi-threaded concurrency race conditions and ensure the time-series ordering and data consistency of market data in concurrent writing scenarios.

[0131] In one embodiment of this application, it further includes: an initialization configuration module, used for

[0132] After startup, preset configuration parameters are loaded, including external market data source connection parameters, foreign exchange data subscription list, distributed lock database configuration, fault self-healing parameters, and time scheduling plan;

[0133] Initialize the event-driven lifecycle manager and establish the mapping relationship between events and business processing functions;

[0134] Initialize the fault self-healing engine and load the hierarchical self-healing strategy.

[0135] In one embodiment of this application, the Leader instance manages the entire service lifecycle based on an event-driven approach, abstracting the entire process of external data source access, data reception, exception handling, and connection disconnection into a standardized event model, thereby decoupling the normal business process from the fault self-healing logic.

[0136] In one embodiment of this application, during the silent cooling state, the downgraded Candidate instance suspends participation in distributed lock contention and rejoins the cluster lock contention queue after the cooling period ends and the exception count is reset.

[0137] It is understood that the above-mentioned data service fault self-healing device based on event-driven architecture can implement each step of the data service fault self-healing method based on event-driven architecture provided in the foregoing embodiments. The relevant explanations of the data service fault self-healing method based on event-driven architecture are applicable to the data service fault self-healing device based on event-driven architecture, and will not be repeated here.

[0138] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0139] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0140] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0141] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a data service fault self-healing device based on an event-driven architecture at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0142] The scheduling task is distributed to all service instances in the data service cluster through a segmented broadcast method. Based on the lock contention result, the cluster instances are respectively determined as Leader instances to execute business and Candidate instances to standby.

[0143] The Leader instance completes the external data source access process via an event-driven approach. Upon successful access, it enters a continuous data receiving and running state, and during this process, it monitors for link and data anomalies in real time and triggers exception events accordingly.

[0144] An abnormal event generated during operation is cumulatively counted, and a self-healing strategy is executed in stages according to the cumulative number of abnormal events. Each Candidate instance continuously monitors the distributed lock status, and can be upgraded to a Leader instance after successfully competing for and acquiring the distributed lock.

[0145] The above is as stated in this application. Figure 1 The method executed by the data service fault self-healing device based on an event-driven architecture disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0146] The electronic device can also perform Figure 1 A method for executing a data service fault self-healing device based on an event-driven architecture is presented, and the implementation of the event-driven architecture-based data service fault self-healing device in... Figure 1 The functions of the embodiments shown are not described again in this application.

[0147] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1The method executed by the data service fault self-healing device based on an event-driven architecture in the illustrated embodiment is specifically used to execute:

[0148] The scheduling task is distributed to all service instances in the data service cluster through a segmented broadcast method. Based on the lock contention result, the cluster instances are respectively determined as Leader instances to execute business and Candidate instances to standby.

[0149] The Leader instance completes the external data source access process via an event-driven approach. Upon successful access, it enters a continuous data receiving and running state, and during this process, it monitors for link and data anomalies in real time and triggers exception events accordingly.

[0150] Anomalies generated during operation are cumulatively counted, and a self-healing strategy is implemented in stages based on the cumulative number of anomalies. Each Candidate instance continuously monitors the distributed lock status, and can be upgraded to a Leader instance after successfully acquiring the distributed lock. Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0152] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0154] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0155] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0156] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0157] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0158] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data service fault self-healing method based on event-driven architecture, wherein, Applied to a data service cluster server, the method includes: The scheduling task is distributed to all service instances in the data service cluster through a segmented broadcast method. Based on the lock contention result, the cluster instances are respectively determined as Leader instances to execute business and Candidate instances to standby. The Leader instance completes the external data source access process via an event-driven approach. Upon successful access, it enters a continuous data receiving and running state, and during this process, it monitors for link and data anomalies in real time and triggers exception events accordingly. An abnormal event generated during operation is cumulatively counted, and a self-healing strategy is executed in stages according to the cumulative number of abnormal events. Each Candidate instance continuously monitors the distributed lock status, and can be upgraded to a Leader instance after successfully competing for and acquiring the distributed lock.

2. The method as described in claim 1, wherein, The step of accumulating and counting abnormal events generated during operation, and implementing a self-healing strategy in a tiered manner based on the cumulative number of abnormal events, includes: When the number of exceptions does not reach the preset threshold, the data source connection will be re-initiated after a fixed retry interval. When the number of anomalies reaches a preset threshold, it is determined to be a persistent fault. The distributed lock is actively released and the current instance is downgraded to a Candidate instance, entering a silent cooling state. After the silent cooling ends, the anomaly count is reset, allowing the current instance to re-participate in the cluster's distributed lock contention.

3. The method as described in claim 1, wherein, Each Candidate instance continuously monitors the distributed lock status. Upon successfully acquiring the distributed lock, it can be promoted to a Leader instance, including: Each Candidate instance continuously performs polling hot standby failover operations, polling in real time to detect the holding and expiration status of the distributed lock; when a lock is detected to be idle or expired, it participates in the distributed lock contention. After successfully competing, it is promoted to Leader instance and automatically takes over data service tasks, realizing automatic master-slave failover of the cluster.

4. The method of claim 1, wherein, The method further includes: For multi-threaded concurrent data writing scenarios, Redis ordered sets are used to store market data. The Redis server-side Lua scripts atomically perform duplicate data verification, auto-incrementing sequence number completion, ordered writing of time-series data, and set capacity pruning operations to avoid multi-threaded concurrency race conditions and ensure the time-series ordering and data consistency of market data in concurrent writing scenarios.

5. The method of claim 1, wherein, Before distributing the scheduling task to all service instances in the data service cluster via fragmented broadcast, the following steps are also included: After startup, preset configuration parameters are loaded, including external market data source connection parameters, foreign exchange data subscription list, distributed lock database configuration, fault self-healing parameters, and time scheduling plan; Initialize the event-driven lifecycle manager and establish the mapping relationship between events and business processing functions; Initialize the fault self-healing engine and load the hierarchical self-healing strategy.

6. The method of claim 1, wherein, The Leader instance manages the entire service lifecycle in an event-driven manner, abstracting the entire process of external data source access, data reception, exception handling, and connection disconnection into a standardized event model, thereby decoupling the normal business process from the fault self-healing logic.

7. The method of claim 1, wherein, During the silent cooling state, the downgraded Candidate instance is suspended from participating in distributed lock contention. After the cooling period ends and the exception count is reset, it rejoins the cluster lock contention queue.

8. A data service fault self-healing device based on an event-driven architecture, wherein, The device is applied to a data service cluster server and includes: The distributed lock contention module is used to distribute scheduling tasks to all service instances in the data service cluster through a sharded broadcast method. Based on the lock contention result, the cluster instances are respectively determined as Leader instances to execute business and Candidate instances to standby. The Leader lifecycle management module is used by the Leader instance to complete the external data source access process via an event-driven approach. After successful access, it enters a continuous data receiving and running state, and during the running process, it monitors link anomalies and data anomalies in real time and triggers anomaly events; and The self-healing strategy module is used to accumulate and count abnormal events generated during operation, and execute self-healing strategies in stages according to the number of accumulated abnormal events. Each Candidate instance continuously monitors the distributed lock status, and can be upgraded to a Leader instance after successfully competing for and acquiring the distributed lock.

9. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 7.

Images