Cross-cluster data processing method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses a cross-cluster data processing method and device, computer equipment and a storage medium in the fields of financial science and technology and medical health. The method comprises the following steps: determining a target processing cluster from an initial master cluster and an initial slave cluster through an obtained data processing request; the initial master cluster is simultaneously associated with the read-write domain name identifier and the read-only domain name identifier, and the initial slave cluster is associated with the read-only domain name identifier; when the target processing cluster is the initial main cluster and the initial main cluster is in an abnormal state, sending abnormal report information to a preset notification party; after a cluster switching instruction is received, executing an index following canceling operation on a service index in the initial slave cluster according to the cluster switching instruction, and determining the initial slave cluster after completing the index following canceling operation as an updated master cluster; and responding to the data processing request by updating the main cluster to obtain a data processing result. According to the invention, the data processing reasonability is improved, and the service continuity is ensured.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a cross-cluster data processing method, device, computer equipment and storage medium in the fields of financial technology and medical health. Background Art

[0002] In practical application scenarios, the full-text search (Elastic Search, ES) database needs to support active-active mode. Active-active mode runs ES clusters simultaneously in two or more data centers. When one data center fails, the other data center can continue to provide services. In the financial and insurance industries, online transaction systems need to process a large number of transaction requests in real time. If a database failure occurs, transaction records may not be queried and updated in a timely manner, thereby interrupting the transaction process. In this case, the other data center in the active-active mode can immediately take over the business, ensuring the smooth progress of the online transaction process. In the healthcare industry, if the electronic medical record system fails and becomes inaccessible due to a database failure, doctors may not be able to obtain critical data such as the patient's medical history and allergy information in a timely manner, which may affect the diagnosis and treatment plan. In this case, the other data center in the active-active mode can immediately take over the business, ensuring the normal operation of medical services.

[0003] In the existing technology, commonly used active-active mode solutions include application-side self-duplex reading and writing, gateway queue dual-reading and writing, and snapshot dual-reading and writing. These dual-reading and writing solutions have defects to varying degrees. Among them, when adopting the application-side self-duplex reading and writing solution, a set of ES clusters are deployed in two data centers respectively, and the client performs dual-reading and dual-writing to these two ES clusters, which increases the burden on the client and makes read and write processing inconvenient. When adopting the gateway queue dual-reading and dual-writing solution, a set of ES clusters are deployed in each of the two data centers, and a gateway and queue are deployed at the same time. The additional gateway and queue consume read and write processing resources, and once conditional deletion occurs, data inconsistency is likely to occur. When adopting the snapshot dual-reading and dual-writing solution, there is a time interval delay in the snapshots between clusters, and data inconsistency is also likely to occur.

[0004] Therefore, a new active-active mode is urgently needed to implement cross-cluster data processing solutions. Summary of the Invention

[0005] Based on this, it is necessary to provide a cross-cluster data processing method, device, computer equipment and storage medium to address the above technical problems, so as to solve the problems of unreasonable read and write processing and poor data consistency in the existing dual-read and dual-write scheme during cross-cluster data processing.

[0006] A cross-cluster data processing method, comprising: Obtaining a data processing request, and determining a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request; the initial master cluster is associated with both a read-write domain name identifier and a read-only domain name identifier, and the initial slave cluster is associated with the read-only domain name identifier; When the target processing cluster is the initial main cluster and the initial main cluster is in an abnormal state, sending abnormality reporting information to a preset notification party; If a cluster switching instruction corresponding to the abnormal reporting information is received from the preset notification party within the preset waiting time, a cancel index following operation is performed on the business index in the initial slave cluster according to the cluster switching instruction, and the initial slave cluster after completing the cancel index following operation is determined as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier; The updating main cluster responds to the data processing request to obtain a data processing result of the data processing request.

[0007] A cross-cluster data processing device, comprising: a target cluster determination module, configured to obtain a data processing request and determine a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request; the initial master cluster is associated with both a read-write domain name identifier and a read-only domain name identifier, and the initial slave cluster is associated with the read-only domain name identifier; an abnormality reporting module, configured to send abnormality reporting information to a preset notification party when the target processing cluster is the initial main cluster and the initial main cluster is in an abnormal state; A cluster switching module is configured to, if a cluster switching instruction corresponding to the abnormal reporting information is received from the preset notification party within a preset waiting time, perform a cancel index following operation on the business index in the initial slave cluster according to the cluster switching instruction, and determine the initial slave cluster after completing the cancel index following operation as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier; The data processing module is configured to respond to the data processing request through the update main cluster to obtain a data processing result of the data processing request.

[0008] A computer device includes a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the cross-cluster data processing method is implemented.

[0009] A computer-readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the cross-cluster data processing method as described above.

[0010] In the above-mentioned cross-cluster data processing method, device, computer equipment and storage medium, the cross-cluster data processing method obtains a data processing request and determines a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request; the initial master cluster is associated with a read-write domain name identifier and a read-only domain name identifier at the same time, and the initial slave cluster is associated with a read-only domain name identifier; when the target processing cluster is the initial master cluster and the initial master cluster is in an abnormal state, an abnormality reporting information is sent to a preset notification party; if a cluster switching instruction corresponding to the abnormality reporting information is received from a preset notification party within a preset waiting time, an index cancellation follow-up operation is performed on the business index in the initial slave cluster according to the cluster switching instruction, and the initial slave cluster after completing the index cancellation follow-up operation is determined as an updated master cluster associated with both a read-write domain name identifier and a read-only domain name identifier; the data processing result of the data processing request is obtained by responding to the data processing request by updating the master cluster. In the active-active mode solution of the present invention, the read-write separation of the master-slave cluster is realized by using the read-write domain name identifier and the read-only domain name identifier, thereby avoiding the irrationality of dual reading and dual writing and improving data processing efficiency. In addition, the present invention ensures data consistency of the master-slave cluster by following the business index, thereby realizing a cross-cluster data processing flow when a cluster fails, efficiently responding to data processing requests, and ensuring business continuity. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0012] Figure 1 This is a schematic diagram of an application environment of a cross-cluster data processing method according to an embodiment of the present invention; Figure 2 This is a flow chart of a cross-cluster data processing method according to an embodiment of the present invention; Figure 3 is a schematic structural diagram of a cross-cluster data processing device in one embodiment of the present invention; Figure 4 FIG. 1 is a schematic diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0014] The cross-cluster data processing method provided in this embodiment can be applied in Figure 1 In an application environment, clients communicate with servers. Clients include, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented as a standalone server or a server cluster consisting of multiple servers. Users send data processing requests to the server through the client. After receiving the data processing requests, the server uses the active-active cluster or the active-active cluster to read and write data.

[0015] The cross-cluster data processing method of this embodiment can be applied to the financial and insurance industries. On the one hand, in business scenarios such as financial transactions and insurance claims, millisecond-level responses are required, and data processing delays may lead to transaction failures or business interruptions. For example, the stock trading system needs to update stock prices, trading volumes and other information in real time so that investors can make accurate decisions. In the active-active mode, cross-cluster data processing can be achieved between the clusters of the two data centers to ensure business continuity. On the other hand, financial technology businesses are usually faced with high-concurrency access requests. For example, during holidays or promotional activities, the transaction volume of online financial platforms may increase significantly. In the active-active mode, user requests can be evenly distributed to different data centers to achieve cross-cluster data processing, achieve load balancing, and avoid performance bottlenecks in a single data center due to excessive load.

[0016] The cross-cluster data processing method of this embodiment can also be applied to the medical and health industry. On the one hand, with the development of telemedicine, medical institutions in different regions need to realize real-time sharing and interaction of data. For example, patients in grassroots hospitals can consult with experts in higher-level hospitals through telemedicine systems, and experts need to view the patient's medical records, imaging materials and other information in real time. In the active-active mode, cross-regional data processing between clusters of two data centers can be ensured, ensuring that doctors can obtain the latest patient information and avoid abnormal interruption of telemedicine services. On the other hand, during the peak influenza season, the hospital's registration system, online consultation system, etc. may face high-concurrency user access requests. In the active-active mode, user requests can be evenly distributed to different data centers to realize cross-cluster data processing, achieve load balancing, and improve the overall performance and response speed of the system.

[0017] In one embodiment, if Figure 2As shown, a cross-cluster data processing method is provided, comprising the following steps S10-S40: S10. Obtain a data processing request, and determine a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request; the initial master cluster is associated with both a read-write domain name identifier and a read-only domain name identifier, and the initial slave cluster is associated with the read-only domain name identifier.

[0018] Understandably, the user sends a data processing request to the server through the client. After receiving the data processing request, the server determines the target processing cluster from the initial primary cluster and initial secondary cluster based on the data processing request. The data processing request is used to indicate that the client user needs to access the data center to read or write data. The initial primary cluster and initial secondary cluster are database clusters with the same deployment version deployed in two data centers. The initial primary cluster refers to the database cluster that supports both data read and data write requirements before the active-active cluster switchover. The initial secondary cluster refers to the database cluster that only supports data read requirements before the active-active cluster switchover.

[0019] A data processing request contains a domain name identifier that represents the user's data processing request. Based on the read / write processing type, it is divided into a read-write domain name identifier (e.g., "rw") and a read-only domain name identifier (e.g., "rr"). The read-write domain name identifier is a domain name character encoding used to represent data writing and editing requests (such as inserting, updating, and deleting data). The read-only domain name identifier is a domain name character encoding used to represent data reading and querying requests. Based on the data processing request, the server can parse the user's data processing request and select a target processing cluster from the initial primary cluster and initial secondary cluster. The target processing cluster is the database cluster that can respond to the data processing request. The initial primary cluster is associated with both a read-write domain name identifier and a read-only domain name identifier, indicating that the initial primary cluster can respond to data processing requests for both data writing and data reading operations. The initial secondary cluster is associated with a read-only domain name identifier, indicating that the initial secondary cluster can only respond to data processing requests for data reading operations, not data writing operations. That is, all data write operations occur on the initial master cluster, while data read operations can occur on the initial master cluster or the initial slave cluster.

[0020] S20: When the target processing cluster is the initial master cluster and the initial master cluster is in an abnormal state, send abnormality reporting information to a preset notification party.

[0021] Understandably, when the target processing cluster is the initial main cluster, the server needs to determine whether the initial main cluster is in a normal state. When the initial main cluster is in a normal state, the server calls the initial main cluster to respond to the data processing request and obtains the data processing result of the data processing request. The data processing result is the result information used to characterize whether the data read or data write operation is successfully performed in the cluster. When the initial main cluster is in an abnormal state, the server sends an abnormal reporting information to the preset notification party. The abnormal state means that the cluster is in an unavailable state where data processing operations cannot be performed, such as a fault state or an overloaded and unresponsive state. The preset notification party refers to a pre-set receiving terminal for receiving abnormal reporting information, such as the terminal device of the operation and maintenance personnel. The abnormal reporting information refers to the prompt information used to characterize the inability of the cluster to respond to the data processing request.

[0022] S30. If a cluster switching instruction corresponding to the abnormal reporting information is received from the preset notification party within the preset waiting time, the business index in the initial slave cluster is canceled according to the cluster switching instruction, and the initial slave cluster after the cancellation of the index following operation is determined as the updated master cluster associated with both the read-write domain name identifier and the read-only domain name identifier.

[0023] Understandably, in a cluster, an index is a document used to store data structures, similar to a data table in a database. The cluster divides the index data into multiple parts and distributes them on different nodes. In active-active mode, the ES clusters of the two data centers can be configured as a cross-cluster replication (CCR) relationship to achieve real-time synchronization of data. In this embodiment, cross-cluster replication between the initial master cluster and the initial slave cluster is achieved through index following, thereby achieving active-active clusters across data centers. Specifically, the business index of the initial master cluster (Leader cluster) is used as the leader index, and the business index of the initial slave cluster (Follower cluster) is used as the follower index. The follower index follows the leader index through configuration information, and the data changes of the initial master cluster will be synchronized to the initial slave cluster in real time.

[0024] After sending the exception report to the pre-set notification source, the server determines whether it has received a cluster switching instruction corresponding to the exception report from the pre-set notification source within the pre-set waiting time. The pre-set waiting time is the maximum period of time set to wait for feedback from the pre-set notification source. The cluster switching instruction is a signal used to transfer the response task of the data processing request from the current master cluster to the slave cluster.

[0025] If the cluster switching instruction corresponding to the abnormal reporting information is not received from the preset notification party within the preset waiting time, the abnormal reporting information is resent to the preset notification party and the number of retries is counted until the cluster switching instruction sent by the preset notification party is received within the preset waiting time, or the retry is stopped when the preset maximum number of retries is reached.

[0026] If a cluster switching instruction corresponding to the exception report is received from the preset notification source within the preset waiting time, the server will perform a de-indexing operation on the business index in the initial slave cluster according to the cluster switching instruction. De-indexing removes the automatic synchronization and dependency between the two clusters at the business index level, allowing the slave cluster to be managed independently of the master cluster. Specifically, the follower index unfollows the leader index, eliminating the dependency between the business index of the initial master cluster and the business index of the initial slave cluster, and no further data synchronization occurs between the initial master and initial slave clusters. The server will identify the initial slave cluster after the de-indexing operation as the update master cluster, establish an association between the update master cluster and the read-write domain name identifier and the read-only domain name identifier, and complete the cluster switching instruction. The update master cluster refers to a database cluster that supports both data read and write requirements after the active-active cluster switchover.

[0027] S40: Respond to the data processing request through the update main cluster to obtain a data processing result of the data processing request.

[0028] As you can understand, after the cluster switchover command completes, the active-active mode temporarily switches to a single-active cluster state, with the updated primary cluster (the original initial secondary cluster) handling all data read and write requests. The server responds to the data processing request by updating the primary cluster and obtains the data processing result of the data processing request.

[0029] This embodiment obtains a data processing request and determines a target processing cluster from the initial master cluster and the initial slave cluster according to the data processing request; the initial master cluster is associated with a read-write domain name identifier and a read-only domain name identifier at the same time, and the initial slave cluster is associated with a read-only domain name identifier; when the target processing cluster is the initial master cluster and the initial master cluster is in an abnormal state, an abnormality reporting information is sent to a preset notification party; if a cluster switching instruction corresponding to the abnormality reporting information is received from the preset notification party within a preset waiting time, the business index in the initial slave cluster is canceled according to the cluster switching instruction, and the initial slave cluster after completing the cancel index following operation is determined as an updated master cluster that is associated with both a read-write domain name identifier and a read-only domain name identifier; the data processing result of the data processing request is obtained by responding to the data processing request by updating the master cluster. In the active-active mode solution of this embodiment, the read-write separation of the master and slave clusters is realized by using the read-write domain name identifier and the read-only domain name identifier, which avoids the irrationality of dual reading and dual writing and improves data processing efficiency. Furthermore, this embodiment ensures data consistency across master and slave clusters through business index tracking. This allows for cross-cluster processing in the event of a cluster failure, efficiently responding to data processing requests and ensuring business continuity. In practical applications, this can prevent business interruptions due to database cluster anomalies in transaction service pages in financial and insurance businesses, or consultation service pages in healthcare businesses, ensuring smooth business service operations.

[0030] In one embodiment, step S10, i.e., determining a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request, includes: S101, extracting a processing domain name identifier from the data processing request; S102: When the processing domain name identifier is a read-only domain name identifier, a target processing cluster is determined from the initial master cluster and the initial slave cluster according to a preset allocation rule; S103: When the processing domain name identifier is a read-write domain name identifier, the initial main cluster is determined as a target processing cluster.

[0031] Understandably, after the server obtains the data processing request, it parses the data processing request and extracts the processing domain name identifier from the data processing request. The processing domain name identifier is a domain name identifier used to represent the user's actual data read and write processing type requirements. The processing domain name identifier is one of the read-write domain name identifier and the read-only domain name identifier. The read-write domain name identifier is associated with the initial master cluster and is used to process data write operations, indicating that when the client sends a data processing request through the read-write domain name identifier, the server will directly respond to the data processing request through the initial master cluster. The read-only domain name identifier is associated with both the initial master cluster and the initial slave cluster and is used to process data read operations, indicating that when the client sends a data processing request through the read-only domain name identifier, the server will assign the initial master cluster or the initial slave cluster to respond to the data processing request to achieve load balancing.

[0032] In one embodiment, when the processing domain name identifier is a read-only domain name identifier, the target processing cluster is determined from the initial master cluster and the initial slave cluster according to the preset allocation rule to achieve load balancing in the financial insurance business and the medical health business. The preset allocation rule is a pre-set criterion for allocating data processing requests between the initial master cluster and the initial slave cluster in proportion. For example, when allocating data processing requests for reading data, the allocation ratio of the initial master cluster is 30%, and the allocation ratio of the initial slave cluster is 70%. At this time, the target processing cluster has a 30% probability of being the initial master cluster and a 70% probability of being the initial slave cluster. When the processing domain name identifier is a read-write domain name identifier, the initial master cluster is directly determined as the target processing cluster.

[0033] This embodiment determines the target processing cluster from the initial master cluster and the initial slave cluster through the processing domain name identifier extracted from the data processing request, realizes cluster read-write separation in active-active mode, and automatically selects the appropriate cluster for data reading operations according to the cluster load situation, realizes a load balancing mechanism, improves the rationality of data processing, and can efficiently respond to data processing requests.

[0034] In one embodiment, step S10, that is, before determining a target processing cluster from the initial master cluster and the initial slave cluster according to the data processing request, includes: S104: Deploy a first cluster in a first data center, deploy a second cluster in a second data center, and determine a remote cluster association relationship between the first cluster and the second cluster; S105: Copy the service index in the first cluster to the second cluster to obtain the service index of the second cluster; S106: Associating the first cluster with the read-write domain name identifier and the read-only domain name identifier, and determining the associated first cluster as the initial primary cluster; S107: Perform an index follow configuration operation on the service index in the second cluster, associate the second cluster after the index follow configuration operation is completed with the read-only domain name identifier, and determine the associated second cluster as the initial slave cluster.

[0035] Understandably, before determining the target processing cluster from the initial master cluster and the initial slave cluster according to the data processing request, the server needs to first determine the initial master cluster and the initial slave cluster in the two data centers.

[0036] First, the server deploys the first cluster in the first data center and the second cluster in the second data center. A data center is a facility used for centralized storage, management, processing, and distribution of large amounts of data. The first and second data centers are located in different regions. The first cluster is an ES cluster that processes business data within the first data center, while the second cluster is an ES cluster that processes business data within the second data center. The server configures the first cluster as a remote cluster (remote_cluster) of the second cluster, and the second cluster as a remote cluster (remote_cluster) of the first cluster, thereby establishing a remote cluster association between the first and second clusters. A remote cluster association is a relationship used to enable cross-cluster search and replication between two clusters. This remote cluster association enables data synchronization and cross-cluster replication between the first and second clusters.

[0037] Then, the server establishes an association relationship between the first cluster and the read-write domain name identifier and the read-only domain name identifier, and determines the first cluster associated with both the read-write domain name identifier and the read-only domain name identifier as the initial primary cluster.

[0038] Next, the business index refers to the index for storing and managing the user's business data in the cluster. The server copies the business index in the first cluster to the second cluster to obtain the business index of the second cluster. At this time, the first cluster and the second cluster have the same business index. In addition, the server performs an index follow configuration operation on the business index in the second cluster, and associates the second cluster with the read-only domain name identifier after completing the index follow configuration operation, and determines the second cluster associated with the read-only domain name identifier as the initial slave cluster. The index follow configuration operation refers to the operation of establishing an automatic synchronization relationship between two clusters at the business index level through information configuration. That is, there is a dependency relationship between the business index of the first cluster and the business index of the second cluster, and data synchronization can be automatically performed between the first cluster and the second cluster.

[0039] In one embodiment, the server uses the service index of the initial leader cluster as the leader index and the service index of the initial follower cluster as the follower index. The server performs index follow configuration by adding the index follow configuration parameter "auto_follow_pattern" to the initial follower cluster. The server can implement index follow configuration by using the following command: PUT / _ccr / auto_follow / <auto_follow_pattern_name> { "remote_cluster" : "<remote_cluster> ", "leader_index_patterns" : [ "*" ], "leader_index_exclusion_patterns": [ ".*" ], "follow_index_pattern" : "{{leader_index}}" } The server implements index follow configuration operations to enable the follower index to follow the leader index. Data changes (such as additions, updates, and deletions) in the initial master cluster are synchronized to the initial slave cluster in real time.

[0040] This implementation deploys master-slave clusters in two data centers and configures remote association. If the initial master cluster (the first cluster) fails, the initial slave cluster can readily take over the master's responsibilities, ensuring uninterrupted service. Furthermore, a data synchronization mechanism based on index tracking ensures data consistency between the initial slave cluster and the business data in the initial master cluster.

[0041] In one embodiment, in step S105, that is, copying the service index in the first cluster to the second cluster to obtain the service index of the second cluster, includes: S1051. Obtain an index information list of the first cluster, and filter out service indexes in the first cluster from the index information list; S1052: Traverse the service indexes in the first cluster and add them to the second cluster, and determine the traversed and added service indexes as the service indexes of the second cluster, so that the service indexes of the second cluster are consistent with the service indexes in the first cluster.

[0042] Understandably, the index information list refers to the data set of all indexes in a cluster. All indexes in a cluster include business indexes and system indexes. A cluster's business indexes are indexes used to store and manage user business data within the cluster, while system indexes are indexes used to store and manage system configuration data and monitoring information within the cluster. When the server obtains the business indexes of a second cluster through replication, it only needs to replicate all business indexes of the first cluster and does not need to replicate the system indexes of the second cluster.

[0043] In one embodiment, the "_cat / indices" command is used to obtain a list of index information for the first cluster. If the service index does not begin with a dot, the "awk" command is used to filter out indexes that do not begin with a dot, thereby filtering out the service indexes for the first cluster. Next, the server traverses the service indexes in the first cluster and creates a service index with the same configuration in the second cluster. The added service index is then determined as the service index for the second cluster, ensuring that the service index for the second cluster is consistent with the service index in the first cluster.

[0044] This embodiment synchronizes the service index of the first cluster to the second cluster, ensuring that the service indexes of the two clusters remain consistent, and provides data support for cross-cluster data read and write processing in the active-active mode.

[0045] In one embodiment, in step S107, that is, associating the second cluster after the index follow configuration operation is completed with the read-only domain name identifier and determining the associated second cluster as the initial slave cluster, the following steps are included: S1071. Regularly obtain the master cluster service data of the initial master cluster and the slave cluster service data of the initial slave cluster; S1072: When there is inconsistency between the master cluster service data and the slave cluster service data, synchronously update the slave cluster service data according to the master cluster service data to obtain synchronously updated slave cluster service data.

[0046] As can be understood, the business index of the initial master cluster (leader cluster) serves as the leader index, and the business index of the initial follower cluster (follower cluster) serves as the follower index, ensuring data consistency between the initial master and follower clusters. The server periodically retrieves the master cluster business data of the initial master cluster and the follower cluster business data of the initial follower cluster, and compares the master and follower data for consistency. This scheduled approach triggers data retrieval and comparison at a pre-set fixed interval. The fixed interval can be set to the default value or adjusted as needed, such as the default value of 1 minute. The master cluster business data refers to the business metadata and business index configuration information in the initial master cluster, while the follower cluster business data refers to the business metadata and business index configuration information in the initial follower cluster. If there is an inconsistency between the master and follower cluster business data, the follower cluster business data is synchronized and updated based on the master cluster business data, resulting in the synchronized and updated follower cluster business data. In practical applications, in financial trading systems, a failure of the initial master cluster may prevent transactions from proceeding. This scheduled synchronization mechanism ensures that the initial follower cluster can take over promptly, ensuring transaction continuity.

[0047] In one embodiment, the server sets a high-frequency scheduled task (executed once a minute) to monitor the business data of the master cluster (the business metadata and index data of the initial master cluster) and the business data of the slave cluster (the business metadata and index data of the initial slave cluster) in the active-active mode. The business metadata includes information such as the cluster's lifecycle policy, templates, and sub-accounts, and the index data includes index configuration item information. When it is discovered that the lifecycle policy, templates, sub-accounts, and index configuration items in the initial master cluster have changed, the server will synchronize and update the corresponding data in the initial slave cluster to ensure that the data on both sides remains consistent. If the initial master cluster deletes a business index, the server will also discover and automatically delete the redundant business indexes in the initial slave cluster, ensuring that the business indexes in the initial master cluster and the initial slave cluster remain consistent.

[0048] This embodiment regularly compares the service data of the initial master cluster and the initial slave cluster. The timed synchronization mechanism promptly detects and corrects data discrepancies, ensuring that the slave cluster's data remains consistent with the master cluster. Furthermore, since the slave cluster maintains data consistency with the master cluster, it facilitates rapid failover to the slave cluster in the event of a master cluster failure, improving the disaster recovery capabilities of the service system.

[0049] In one embodiment, step S30, i.e., performing a cancel index following operation on the service index in the initial slave cluster according to the cluster switching instruction, and determining the initial slave cluster after completing the cancel index following operation as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier, includes: S301, deleting the index following configuration parameters in the initial slave cluster according to the cluster switching instruction; S302: For each business index in the initial slave cluster, sequentially perform operations of pausing following, closing the index, canceling following, and reopening the index, until all business indexes in the initial slave cluster are traversed and the cancellation of index following is confirmed to be completed; S303: Associating the initial slave cluster after the cancellation of the index follow operation with the read-write domain name identifier and the read-only domain name identifier, and determining the associated initial slave cluster as the update master cluster.

[0050] Understandably, after receiving the cluster switch instruction, the server first deletes the index follow configuration parameters in the initial slave cluster according to the cluster switch instruction. Index follow configuration parameters are instruction parameters used to establish an automatic synchronization relationship between two clusters at the business index level, such as the configuration parameter "auto_follow_pattern" added in the initial slave cluster. Next, the server sequentially performs the following operations for each business index in the initial slave cluster: pausing follow (temporarily stopping index synchronization to ensure data consistency), closing the index, canceling follow, and reopening the index (restoring the index to a read-write state). After traversing all business indexes in the initial slave cluster and confirming the completion of the index follow cancellation operation, the business indexes in the initial slave cluster after the index follow cancellation operation are converted from follower indexes to independent regular indexes. Finally, the initial slave cluster after the index follow cancellation operation is associated with the read-write domain name identifier and the read-only domain name identifier, and the associated initial slave cluster is designated as the update master cluster.

[0051] This embodiment switches the initial slave cluster to the update master cluster by canceling the index follow operation and the domain name identifier association operation. The update master cluster processes all data read and data write requests, ensuring business continuity when an exception occurs in the initial master cluster.

[0052] In one embodiment, in step S30, that is, after the initial slave cluster after the cancellation of the index follow operation is determined as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier, the following further comprises: S304: After confirming that the initial master cluster has changed from an abnormal state to a normal state, delete all service indexes in the initial master cluster, and copy the service indexes in the updated master cluster to the initial master cluster to obtain updated service indexes of the initial master cluster; S305: Perform an index follow configuration operation on the update service index in the initial master cluster, associate the initial master cluster after the index follow configuration operation with the read-only domain name identifier, and determine the associated initial master cluster as the update slave cluster.

[0053] Understandably, after the server switches the initial slave cluster to the update master cluster according to the cluster switchover instruction, the active-active mode temporarily remains in a single active-active cluster state, requiring the active-active cluster to be rebuilt after the initial master cluster returns to normal. After confirming that the initial master cluster has returned to normal, the server deletes all existing business indexes in the initial master cluster, copies the business indexes from the update master cluster, and adds them to the initial master cluster, obtaining the updated business indexes of the initial master cluster. The updated business indexes are the business indexes corresponding to the initial master cluster during the active-active cluster rebuilding process. Next, the server adds the configuration parameter "auto_follow_pattern" to the initial master cluster, establishing an automatic synchronization relationship between the two clusters at the business index level, and completes the index follow configuration operation for the updated business indexes in the initial master cluster. Finally, the server associates the initial master cluster with a read-only domain name identifier after the index follow configuration operation, designating the associated initial master cluster as the update slave cluster. An update slave cluster is a database cluster that only supports data reads after the active-active cluster switchover. After the active-active cluster rebuilding, the original master cluster becomes a slave cluster, and the original slave cluster becomes the master cluster.

[0054] In another embodiment, since the data centers corresponding to the two clusters are not completely equivalent, the active-active cluster switching and active-active cluster reconstruction operations can be performed again as needed. After two active-active cluster switching and active-active cluster reconstructions, the original master cluster remains the master cluster, and the original slave cluster remains the slave cluster.

[0055] This embodiment rebuilds the active-active cluster after confirming that the initial master cluster has changed from an abnormal state to a normal state. By cleaning up old data, synchronizing new data, and reconfiguring the follow-up relationship, it achieves role exchange and index synchronization of the master and slave clusters, ensuring the high availability of the active-active cluster.

[0056] It should be understood that the order of execution of the steps in the above embodiments does not necessarily mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0057] In one embodiment, a cross-cluster data processing device is provided, which corresponds one-to-one to the cross-cluster data processing method in the above embodiment. Figure 3 As shown, the cross-cluster data processing device includes a target cluster determination module 10, an abnormality reporting module 20, a cluster switching module 30 and a data processing module 40. The functional modules are described in detail as follows: A target cluster determination module 10 is configured to obtain a data processing request and determine a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request; the initial master cluster is associated with both a read-write domain name identifier and a read-only domain name identifier, and the initial slave cluster is associated with the read-only domain name identifier; The abnormality reporting module 20 is configured to send abnormality reporting information to a preset notification party when the target processing cluster is the initial main cluster and the initial main cluster is in an abnormal state; The cluster switching module 30 is configured to, if a cluster switching instruction corresponding to the abnormal reporting information is received from the preset notifier within a preset waiting time, perform a cancel index following operation on the service index in the initial slave cluster according to the cluster switching instruction, and determine the initial slave cluster after completing the cancel index following operation as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier; The data processing module 40 is configured to respond to the data processing request through the update main cluster to obtain a data processing result of the data processing request.

[0058] In one embodiment, the target cluster determination module 10 includes: A processing domain name identifier extraction unit, configured to extract a processing domain name identifier from the data processing request; a first target cluster determining unit, configured to determine a target processing cluster from the initial master cluster and the initial slave cluster according to a preset allocation rule when the processing domain name identifier is a read-only domain name identifier; The second target cluster determining unit is configured to determine the initial main cluster as a target processing cluster when the processing domain name identifier is a read-write domain name identifier.

[0059] In one embodiment, the target cluster determination module 10 further includes: a cluster deployment unit, configured to deploy a first cluster in a first data center, deploy a second cluster in a second data center, and determine a remote cluster association relationship between the first cluster and the second cluster; A first service index copying unit, configured to copy the service index in the first cluster to the second cluster to obtain the service index of the second cluster; an initial master cluster determining unit, configured to associate the first cluster with the read-write domain name identifier and the read-only domain name identifier, and determine the associated first cluster as the initial master cluster; The initial slave cluster determining unit is configured to perform an index follow configuration operation on the service index in the second cluster, associate the second cluster after the index follow configuration operation with the read-only domain name identifier, and determine the associated second cluster as the initial slave cluster.

[0060] In one embodiment, the target cluster determination module 10 further includes: a service index screening unit, configured to obtain an index information list of the first cluster, and screen the service indexes in the first cluster from the index information list; The service index adding unit is configured to traverse the service indexes in the first cluster and add them to the second cluster, and determine the traversed and added service indexes as the service indexes of the second cluster, so that the service indexes of the second cluster are consistent with the service indexes in the first cluster.

[0061] In one embodiment, the target cluster determination module 10 further includes: a data timing acquisition unit, configured to regularly acquire the master cluster service data of the initial master cluster and the slave cluster service data of the initial slave cluster; The synchronous updating unit is configured to synchronously update the slave cluster service data according to the master cluster service data when there is inconsistency between the master cluster service data and the slave cluster service data, so as to obtain the synchronously updated slave cluster service data.

[0062] In one embodiment, the cluster switching module 30 includes: a configuration parameter deleting unit, configured to delete the index following configuration parameter in the initial slave cluster according to the cluster switching instruction; An index following cancellation processing unit is configured to sequentially perform the operations of pausing following, closing the index, canceling following, and reopening the index for each business index in the initial slave cluster until all business indexes in the initial slave cluster are traversed and the index following cancellation operation is confirmed to be completed; The update master cluster determination unit is used to associate the initial slave cluster after completing the cancel index follow operation with the read-write domain name identifier and the read-only domain name identifier, and determine the associated initial slave cluster as the update master cluster.

[0063] In one embodiment, the cluster switching module 30 further includes: A second service index copying unit is configured to, after confirming that the initial primary cluster has changed from an abnormal state to a normal state, delete all service indexes in the initial primary cluster and copy the service indexes in the updated primary cluster to the initial primary cluster to obtain an updated service index of the initial primary cluster; The update slave cluster determination unit is used to perform an index follow configuration operation on the update service index in the initial master cluster, associate the initial master cluster after the index follow configuration operation with the read-only domain name identifier, and determine the associated initial master cluster as the update slave cluster.

[0064] For the specific definition of the cross-cluster data processing device, please refer to the definition of the cross-cluster data processing method above, and will not be repeated here. Each module in the above-mentioned cross-cluster data processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0065] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer-readable instructions and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The database of the computer device is used to store data involved in the cross-cluster data processing method. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer-readable instructions are executed by the processor, a cross-cluster data processing method is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0066] In one embodiment, a computer device is provided, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the following steps are implemented: Obtaining a data processing request, and determining a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request; the initial master cluster is associated with both a read-write domain name identifier and a read-only domain name identifier, and the initial slave cluster is associated with the read-only domain name identifier; When the target processing cluster is the initial main cluster and the initial main cluster is in an abnormal state, sending abnormality reporting information to a preset notification party; If a cluster switching instruction corresponding to the abnormal reporting information is received from the preset notification party within the preset waiting time, a cancel index following operation is performed on the business index in the initial slave cluster according to the cluster switching instruction, and the initial slave cluster after completing the cancel index following operation is determined as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier; The updating main cluster responds to the data processing request to obtain a data processing result of the data processing request.

[0067] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The computer-readable storage media provided in this embodiment include non-volatile computer-readable storage media and volatile computer-readable storage media. The computer-readable storage media store computer-readable instructions that, when executed by one or more processors, implement the following steps: Obtaining a data processing request, and determining a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request; the initial master cluster is associated with both a read-write domain name identifier and a read-only domain name identifier, and the initial slave cluster is associated with the read-only domain name identifier; When the target processing cluster is the initial main cluster and the initial main cluster is in an abnormal state, sending abnormality reporting information to a preset notification party; If a cluster switching instruction corresponding to the abnormal reporting information is received from the preset notification party within the preset waiting time, a cancel index following operation is performed on the business index in the initial slave cluster according to the cluster switching instruction, and the initial slave cluster after completing the cancel index following operation is determined as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier; The updating main cluster responds to the data processing request to obtain a data processing result of the data processing request.

[0068] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, the computer-readable instructions can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0069] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0070] The non-Company software tools or components that appear in the embodiments of this application are merely examples and do not represent actual use. The above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some of the technical features therein with equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention.

Claims

1. A cross-cluster data processing method, characterized in that: include: Obtaining a data processing request, and determining a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request; The initial master cluster is associated with both a read-write domain name identifier and a read-only domain name identifier, and the initial slave cluster is associated with the read-only domain name identifier; When the target processing cluster is the initial main cluster and the initial main cluster is in an abnormal state, sending abnormality reporting information to a preset notification party; If a cluster switching instruction corresponding to the abnormal reporting information is received from the preset notification party within the preset waiting time, a cancel index following operation is performed on the business index in the initial slave cluster according to the cluster switching instruction, and the initial slave cluster after completing the cancel index following operation is determined as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier; The updating main cluster responds to the data processing request to obtain a data processing result of the data processing request.

2. The cross-cluster data processing method according to claim 1, wherein: Determining a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request includes: Extracting a processing domain name identifier from the data processing request; When the processing domain name identifier is a read-only domain name identifier, a target processing cluster is determined from the initial master cluster and the initial slave cluster according to a preset allocation rule; When the processing domain name identifier is a read-write domain name identifier, the initial main cluster is determined as the target processing cluster.

3. The cross-cluster data processing method according to claim 1, wherein: Before determining a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request, the method includes: Deploy a first cluster in a first data center, deploy a second cluster in a second data center, and determine a remote cluster association relationship between the first cluster and the second cluster; Copying the service index in the first cluster to the second cluster to obtain the service index of the second cluster; Associating the first cluster with the read-write domain name identifier and the read-only domain name identifier, and determining the associated first cluster as the initial primary cluster; An index following configuration operation is performed on the service index in the second cluster, and the second cluster after the index following configuration operation is completed is associated with the read-only domain name identifier, and the associated second cluster is determined as the initial slave cluster.

4. The cross-cluster data processing method according to claim 3, wherein: The copying of the service index in the first cluster to the second cluster to obtain the service index of the second cluster includes: Obtaining an index information list of the first cluster, and filtering out service indexes in the first cluster from the index information list; The service indexes in the first cluster are traversed and added to the second cluster, and the traversed and added service indexes are determined as the service indexes of the second cluster, so that the service indexes of the second cluster are consistent with the service indexes in the first cluster.

5. The cross-cluster data processing method according to claim 3, wherein: The step of associating the second cluster after the index follow configuration operation is completed with the read-only domain name identifier and determining the associated second cluster as the initial slave cluster includes: Regularly acquiring the master cluster service data of the initial master cluster and the slave cluster service data of the initial slave cluster; When there is inconsistency between the master cluster service data and the slave cluster service data, the slave cluster service data is synchronously updated according to the master cluster service data to obtain synchronously updated slave cluster service data.

6. The cross-cluster data processing method according to claim 1, wherein: The performing a cancel index following operation on the service index in the initial slave cluster according to the cluster switching instruction, and determining the initial slave cluster after completing the cancel index following operation as an update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier, includes: Deleting the index following configuration parameter in the initial slave cluster according to the cluster switching instruction; For each business index in the initial slave cluster, sequentially perform the operations of pausing following, closing the index, canceling following, and reopening the index, until all business indexes in the initial slave cluster are traversed and the cancellation of index following is confirmed to be completed; The initial slave cluster after the cancellation of the index follow operation is associated with the read-write domain name identifier and the read-only domain name identifier, and the associated initial slave cluster is determined as the update master cluster.

7. The cross-cluster data processing method according to claim 1, wherein: After the initial slave cluster after the cancellation of the index follow operation is determined as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier, the method further includes: After confirming that the initial main cluster has changed from an abnormal state to a normal state, deleting all business indexes in the initial main cluster, and copying the business indexes in the updated main cluster to the initial main cluster to obtain updated business indexes of the initial main cluster; An index follow configuration operation is performed on the update service index in the initial master cluster, and the initial master cluster after the index follow configuration operation is completed is associated with the read-only domain name identifier, and the associated initial master cluster is determined as the update slave cluster.

8. A cross-cluster data processing device, characterized in that: include: a target cluster determination module, configured to obtain a data processing request and determine a target processing cluster from an initial master cluster and an initial slave cluster according to the data processing request; The initial master cluster is associated with both a read-write domain name identifier and a read-only domain name identifier, and the initial slave cluster is associated with the read-only domain name identifier; an abnormality reporting module, configured to send abnormality reporting information to a preset notification party when the target processing cluster is the initial main cluster and the initial main cluster is in an abnormal state; A cluster switching module is configured to, if a cluster switching instruction corresponding to the abnormal reporting information is received from the preset notification party within a preset waiting time, perform a cancel index following operation on the business index in the initial slave cluster according to the cluster switching instruction, and determine the initial slave cluster after completing the cancel index following operation as the update master cluster associated with both the read-write domain name identifier and the read-only domain name identifier; The data processing module is configured to respond to the data processing request through the update main cluster to obtain a data processing result of the data processing request.

9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein: When the processor executes the computer-readable instructions, the cross-cluster data processing method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to perform the cross-cluster data processing method according to any one of claims 1 to 7.

Images