Data processing method, distributed cluster, device, equipment, medium and product

Abstract

Embodiments of the present application disclose a data processing method, a distributed cluster, an apparatus, an equipment, a medium and a product, relating to the field of big data processing. The method comprises: in response to a read request, determining a target key corresponding to the read request; determining a target slot identifier corresponding to the target key, and determining a target slave node identifier based on the target slot identifier, pre-acquired center limiting information and a pre-acquired slot mapping table, the center limiting information comprising a center identifier; and accessing a slave node corresponding to the target slave node identifier. The technical scheme of the embodiments of the present application can significantly reduce the frequency of client access delay.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of big data processing technology, and in particular to a data processing method, distributed cluster, device, equipment, medium and product. Background Technology

[0002] Clustering enables servers to provide the same service by increasing the number of servers, thereby achieving a stable and efficient state. Among them, distributed server clusters, especially Redis (Remote Dictionary Server) clusters, are widely favored for their numerous features such as replication, high availability, and sharding.

[0003] However, in the process of implementing this invention, the inventors discovered that the client experienced a high frequency of access delays when accessing the distributed cluster. Summary of the Invention

[0004] This invention provides a data processing method to solve the problem of high access latency when clients access distributed systems.

[0005] In a first aspect, embodiments of the present invention provide a data processing method applied to a client based on a target cluster, the target cluster including at least two data centers, the at least two data centers including a first data center and a second data center, the first data center including a master node and a first slave node corresponding to the master node, the second data center including a second slave node corresponding to the master node, the identifiers of the first slave node and the second slave node both including the center identifier of the data center they belong to, the method including:

[0006] In response to a read request, determine the target key corresponding to the read request;

[0007] Determine the target slot identifier corresponding to the target key, and determine the target slave node identifier based on the target slot identifier, the pre-acquired center constraint information, and the pre-acquired slot mapping table, wherein the center constraint information includes the center identifier;

[0008] Access the slave node corresponding to the target slave node identifier.

[0009] Thirdly, embodiments of the present invention also provide a distributed cluster, which is the target cluster described in any embodiment. The target cluster is configured to send center-limiting information and a slot mapping table to the client in response to a service discovery instruction from the client; and respond to the client's read request through the target slave node.

[0010] The central restriction information is used to restrict the client to access only a slave node under one data center for read requests. The target slave node is determined by the client based on the central restriction information, the slot mapping table, and the target slot identifier corresponding to the target key of the read request.

[0011] Thirdly, embodiments of the present invention also provide a data processing apparatus, configured on a client based on a target cluster, the target cluster including at least two data centers, the at least two data centers including a first data center and a second data center, the first data center including a master node and a first slave node corresponding to the master node, the second data center including a second slave node corresponding to the master node, the identifiers of the first slave node and the second slave node both including the center identifier of the data center they belong to, the apparatus including:

[0012] The target key module is used to determine the target key corresponding to the read request in response to the read request;

[0013] The slave node module is used to determine the target slot identifier corresponding to the target key, and to determine the target slave node identifier based on the target slot identifier, the pre-acquired center constraint information and the pre-acquired slot mapping table, wherein the center constraint information includes the center identifier;

[0014] The access module is used to access the slave node corresponding to the target slave node identifier.

[0015] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising:

[0016] One or more processors;

[0017] Memory, used to store one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method provided in any embodiment of the present invention.

[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method provided in any embodiment of the present invention.

[0020] Fifthly, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the data processing method according to any embodiment of the present invention.

[0021] The embodiments of the above invention have the following advantages or beneficial effects:

[0022] Since the central constraint information is used to limit the client's read requests to access slave nodes in only one data center, the target slave node determined based on the central constraint information can only be in one data center each time. Therefore, the client can only access slave nodes in one data center at a time, which optimizes the client's access path, effectively prevents unnecessary cross-center access by the client, avoids network latency risks and data consistency risks caused by unnecessary cross-data center access, and can significantly improve the user experience. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the data processing method provided in an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of the target cluster node distribution provided in an embodiment of the present invention;

[0025] Figure 3 This is another flowchart illustrating the data processing method provided in this embodiment of the invention;

[0026] Figure 4 This is another flowchart illustrating the data processing method provided in this embodiment of the invention;

[0027] Figure 5 This is a schematic diagram of the master node election process provided in an embodiment of the present invention;

[0028] Figure 6 This is a schematic diagram of the process for determining the timeout status of a slave node provided in an embodiment of the present invention;

[0029] Figure 7A This is a schematic diagram of the structure of the data processing device provided in an embodiment of the present invention;

[0030] Figure 7B This is another structural schematic diagram of the data processing device provided in the embodiments of the present invention;

[0031] Figure 7C This is another structural schematic diagram of the data processing device provided in the embodiments of the present invention;

[0032] Figure 7D This is another structural schematic diagram of the data processing device provided in the embodiments of the present invention;

[0033] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0034] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0035] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of the present invention. This embodiment is applicable to situations where a client should access which node in the target cluster for a read request. The method can be executed by a data processing device integrated into the client, which can be implemented in software and / or hardware.

[0036] In this embodiment, the target cluster on which the client is based includes at least two data centers, namely a first data center and a second data center. The first data center includes a master node and a first slave node corresponding to the master node, and the second data center includes a second slave node corresponding to the master node. The identifiers of the first slave node and the second slave node both include the center identifier of the data center in which they are located.

[0037] A client is a terminal entity that actively initiates a request during a specific service interaction. It typically runs on the user's device or in the consumer's process, responsible for user interaction, local logic, and communicating with the target cluster via network protocols to jointly complete a task.

[0038] The target cluster can be a Redis Cluster cluster, supporting data sharding and high availability. Its core components include nodes, slots, master-slave replication, and the Gossip protocol. The Gossip protocol is a communication mechanism used in distributed systems for inter-node state synchronization and fault detection. Its core principle simulates the way humans spread rumors: each node periodically sends its own state information (such as heartbeat, data version, configuration epoch, etc.) to randomly selected other nodes, and after receiving verification information from the receiving nodes, it continues to spread the information to other nodes, ultimately achieving state consistency across the entire cluster. This embodiment extends the Gossip protocol so that the identifier of each node includes a data center identifier. For example, the identifiers of the first slave nodes all include C1, and the identifiers of the second slave nodes all include C2. Thus, the data center to which a node belongs can be determined through its identifier.

[0039] like Figure 2 As shown, the target cluster in this embodiment is a distributed cluster, including at least a first data center and a second data center. Normally, the first data center includes a master node, while the second data center does not. Specifically, the first data center includes a master node and a first slave node corresponding to the master node, while the second data center only includes a second slave node corresponding to the master node.

[0040] A data center is a large physical space that centrally houses computing, storage, and network equipment to support the IT services of an enterprise or organization. In a distributed system, a data center typically refers to a logical server room unit with its own network, power, and cooling systems.

[0041] Each master node and its corresponding first and second master nodes correspond to the same data slices, that is, the same range of slots. In other words, when all their node states are normal, the data they store is the same.

[0042] like Figure 1 As shown, the method specifically includes the following steps:

[0043] S110. In response to the read request, determine the target key corresponding to the read request.

[0044] Upon detecting a read request, the read request is parsed using the existing parsing method to obtain one or more target keys corresponding to the read request.

[0045] S120. Determine the target slot identifier corresponding to the target key, and determine the target slave node identifier based on the target slot identifier, the pre-acquired center constraint information, and the pre-acquired slot mapping table. The center constraint information includes the center identifier.

[0046] The central constraint information includes a central identifier, which is used to restrict clients to accessing slave nodes in only one data center for read requests.

[0047] In one embodiment, the client is configured to, at least upon startup, obtain central constraint information and a slot mapping table from the target cluster via a service discovery operation.

[0048] In one embodiment, the client is configured to retrieve center-bound information and a slot mapping table from the target cluster via a service discovery operation upon startup; then, according to a preset discovery period, it periodically retrieves the latest center-bound information and slot mapping table from the target cluster via a service discovery operation. This embodiment improves client access efficiency by intelligently sensing changes in the target cluster.

[0049] The central constraint information is used to restrict client read requests to access slave nodes within a specified data center. The validity period of the central constraint information can be set to the entire time period from when the client is opened to when it is closed, such as the entire usage process after the client starts. This method allows the target cluster to assign central constraint information to each client based on the load of each data center. The validity period of the central constraint information can also be set to be bound to the client identifier. In this case, regardless of when the client accesses the target cluster, the target slave node identifier is determined based on the same central constraint information. This method eliminates the need for the target cluster to frequently generate central constraint information for clients.

[0050] A slot mapping table is metadata used for data sharding and routing in a distributed cluster. It acts like a "global index" or "address book," explicitly specifying which data should be stored on which node. A slot represents a logical data partition. The cluster divides the entire data space into a fixed number of discrete slots. The slot mapping table defines the correspondence between each slot and a physical node—that is, "which slot is responsible for which node."

[0051] Once the target key is determined, the identifier of the target slot corresponding to the target key is determined. For example, a hash operation is performed on the target key to obtain the hash result, and the modulo result of the hash result is used as the identifier of the target slot.

[0052] Based on the slot mapping table, determine the slave node identifier corresponding to the target slot identifier, and use this slave node identifier as the target slave node identifier.

[0053] In one embodiment, the first data center includes a master node and at least one first slave node corresponding to the master node, and the second data center includes at least one second slave node corresponding to the master node. The target slave node identifier is determined through the following steps:

[0054] Step a1: Based on the target slot identifier and the slot mapping table, determine at least one slave node identifier corresponding to the target slot identifier from the first slave node identifier combination corresponding to the center identifier.

[0055] Based on the target slot identifier and the slot mapping table, determine the first slave node identifier combination corresponding to the target slot identifier; determine all slave node identifiers including the center identifier in the first slave node identifier combination; and use all slave node identifiers as at least one slave node identifier corresponding to the target slot identifier.

[0056] Step a2: Use one of the at least one slave node identifiers as the target slave node identifier.

[0057] Once the at least one slave node identifier is determined, any one of the at least one slave node identifiers can be used as the target slave node identifier; alternatively, based on load balancing rules, the optimal slave node can be selected from the slave nodes corresponding to the at least one slave node identifier, and the identifier of that slave node can be used as the target slave node identifier.

[0058] The load balancing objectives corresponding to the load balancing rules include, but are not limited to, at least one of the following: fastest response, lowest load, closest geographical location, and lowest cost. Fastest response means selecting the slave node with the lowest response latency and fastest processing speed; lowest load means selecting the slave node with the lowest CPU, memory, network, or connection load; closest geographical location means selecting the slave node with the fewest network hops and the closest physical distance; lowest cost means selecting the slave node in the availability zone with the lowest cost.

[0059] In this embodiment, the client perceives the location of the data center through the central identifier. Since the target slave node identifier is determined based on the pre-acquired central identifier, the client can only access the slave node in the data center corresponding to the central identifier, and cannot access nodes in other data centers. This ensures that the client can only access one data center, eliminating the network latency risk and data consistency risk caused by cross-center access.

[0060] In one embodiment, the central qualifying information includes a combination of node identifiers, and the second slave node identifier combination includes the identifier of a slave node connected to each master node in the data center that is allowed to be accessed. The target slave node identifier is determined through the following steps:

[0061] Step b1: Based on the slot mapping table, determine the slave node identifier corresponding to the target slot identifier in the second slave node identifier combination.

[0062] Since the second slave node identifier combination includes the identifier of a slave node connected to each master node in the data center that is allowed to be accessed, under normal circumstances, the second slave node identifier combination includes the data under all data shards, which can meet all the data access needs of the client.

[0063] For example, such as Figure 2 As shown, the second slave node identifier combination may include slave nodes S11, S21 and S31 in the first data center; it may also include slave nodes S12, S22 and S32 in the second data center.

[0064] Regarding the determination of the corresponding slave node identifier for the slot identifier.

[0065] For example, based on the slot mapping table, the slave node identifier corresponding to the slot identifier range of the target slot identifier is determined according to the slot identifier range corresponding to each slave node identifier in the second node identifier combination.

[0066] For example, a third slave node identifier combination corresponding to the target slot identifier is determined based on the slot mapping table; the intersection of the third slave node identifier combination and the second slave node identifier combination is used as the slave node identifier corresponding to the target slot identifier.

[0067] Step b2: Use the slave node identifier corresponding to the target slot identifier as the target slave node identifier.

[0068] In this embodiment, since the second slave node identifier combination is determined by the target cluster, this embodiment does not require the client to be aware of the data center; only the target cluster needs to be aware of the data center. Because the second slave node identifier combination includes one slave node connected to each master node within the allowed data centers, it only includes slave nodes within a single data center. This ensures that the target slave node identifier determined based on the second slave node identifier combination is always within a single data center, effectively preventing unnecessary cross-data center access and thus eliminating network latency and data consistency risks associated with cross-data center access.

[0069] S130, Access the slave node corresponding to the target slave node identifier.

[0070] Once the target slave node identifier is determined, the client directly accesses the slave node corresponding to that target slave node identifier.

[0071] Specifically, while obtaining the central constraint information and slot mapping table through service discovery, the client also obtains node status information, including the node access address. After the target slave node identifier is determined, the client extracts the access address corresponding to the target slave node identifier from the node status information, and then accesses the slave node corresponding to the target slave node identifier based on the access address.

[0072] Of course, the second slave node identifier combination can also be configured to include at least two slave node identifiers corresponding to the master node within the allowed data center. In this way, after the client determines at least two slave node identifiers under a master node corresponding to the target slot identifier, it can access the slave node corresponding to any one of the at least two slave node identifiers; or it can select the optimal slave node from the at least two slave nodes corresponding to the at least two slave node identifiers based on load awareness.

[0073] It should be noted that the write request access method for the client can be implemented using existing technologies, and this embodiment does not impose any specific limitations.

[0074] The technical solution provided by this invention, since the central limitation information is used to limit the client to access the slave node under one data center for read requests, the target slave node determined based on the central limitation information can only be in one data center each time. Therefore, the client can only access the slave node under one data center each time, which optimizes the client's access path, effectively prevents unnecessary cross-center access by the client, avoids network latency risks and data consistency risks caused by unnecessary cross-data center access, and can significantly improve the user experience.

[0075] Figure 3 This is another flowchart illustrating the data processing method provided in this embodiment of the invention. This embodiment adds a post-processing procedure for access failures based on the aforementioned embodiments. For example... Figure 3 As shown, the method includes:

[0076] S210. In response to the read request, determine the target key corresponding to the read request.

[0077] S220. Determine the target slot identifier corresponding to the target key, and determine the target slave node identifier based on the target slot identifier, the pre-acquired center constraint information, and the pre-acquired slot mapping table. The center constraint information includes the center identifier.

[0078] S230, Access the slave node corresponding to the target slave node identifier.

[0079] S240. If the access fails, determine the target application corresponding to the read request, and the application level of the target application.

[0080] Each application on the client corresponds to a predefined application level. This application level indicates the importance of the corresponding application. If access fails, the application identifier of the current application is obtained, and the application level of the current application is determined based on the mapping between this application identifier and the pre-created application identifier-application level.

[0081] S250. Based on the correspondence between the application level and the pre-determined application level and the first access failure handling method, determine the first access failure handling method for the target application. The first access failure handling method is the access failure handling method when all data centers are available.

[0082] "All data centers are available" means that no central-level failures have occurred in the cluster.

[0083] In this embodiment, a corresponding first access failure handling method is set for each application level. Therefore, after the application level is determined, the first access failure handling method corresponding to the current application is determined according to the application level and the pre-determined correspondence between the application level and the first access failure handling method.

[0084] Regarding the first access failure handling method: For example, when the application level falls within the first level, the first access failure handling method is to output a first prompt message indicating access failure to the target application. When the application level falls within the second level, the first access failure handling method is to access the first peer slave node corresponding to the target slave node identifier. The first peer slave node is a peer slave node located in the same data center as the target slave node identifier. The lowest level in the second level range is higher than the highest level in the first level range. Specifically, the slave node corresponding to the target slave node identifier is considered the target slave node, and the first peer slave node is a peer slave node located in the same data center as the target slave node. Other slave nodes under the master node corresponding to the target slave node are all peer slave nodes of the target slave node. It is understood that the target slave node and its peer slave nodes store the same data.

[0085] S260. Process the target application access failure event based on the first access failure handling method.

[0086] If the first access failure handling method is to output a first prompt message to the target application to indicate access failure, then the first prompt message is directly output to the target application, and the target application prompts the user according to the predetermined settings rules.

[0087] If the first access failure handling method is to access the first peer slave node corresponding to the target slave node identifier, then determine the first peer slave node corresponding to the target slave node, extract the access address of the first peer slave node from the status information table, and then access the first peer slave node according to the access address.

[0088] In one embodiment, if the first peer slave node fails to access, the target application may output a target prompt message indicating the access failure to avoid the other peer slave nodes crashing due to switching to another first peer slave node.

[0089] The technical solution provided by this invention determines the first access failure handling method after the target application fails to access the target slave node according to the application level of the target application corresponding to the read request, and completes the processing of the target application access failure event based on the first access failure handling method. In this way, different application levels correspond to different first access failure handling methods, which improves the flexibility and accuracy of slave node access failure handling, and can effectively prevent normal slave nodes from being blocked, delayed or crashed due to excessive access volume.

[0090] Figure 4This is another flowchart illustrating the data processing method provided in this embodiment of the invention. This embodiment adds a post-processing procedure for access failures based on the aforementioned embodiments. For example... Figure 4 As shown, the method includes:

[0091] S310. In response to a read request, determine the target key corresponding to the read request.

[0092] S320. Determine the target slot identifier corresponding to the target key, and determine the target slave node identifier based on the target slot identifier, the pre-acquired center constraint information, and the pre-acquired slot mapping table. The center constraint information includes the center identifier.

[0093] S330, Access the slave node corresponding to the target slave node identifier.

[0094] S340. If the access fails and it is detected that all master nodes are distributed in the second data center, then determine the application level of the target application corresponding to the read request.

[0095] All master nodes are distributed in the second data center, which means that the target cluster has completed the master node switch caused by the central failure. At this time, all master and slave nodes in the first data center are unavailable.

[0096] Specifically, although the client cannot determine the scope of the fault, it can infer that a major fault has occurred through a series of abnormal phenomena. For example, if the client finds that multiple master nodes have timed out or are unreachable, and these multiple master nodes happen to be located on the same IP address (first data center), it will infer that a central-level fault has occurred in the first data center. Of course, when the client triggers service discovery and obtains the status information table, it will show that the new topology of the cluster has undergone a large-scale and drastic change compared to the old topology. For example, the IP addresses of master nodes in multiple slots have changed at the same time, and the new master nodes are concentrated in another network segment.

[0097] In this situation, the client needs to obtain the application identifier of the current application and determine the application level of the current application based on the correspondence between the application identifier and the pre-created application identifier and application level.

[0098] S350. Based on the correspondence between the application level and the pre-determined application level and the second access failure handling method, determine the second access failure handling method for the target application. The second access failure handling method is the access failure handling method in the event of a central-level failure in the first data center.

[0099] This embodiment sets a corresponding second access failure handling method for each application level. Therefore, after the application level is determined, the second access failure handling method corresponding to the current application is determined based on the application level, the pre-determined application level, and the second access failure handling method.

[0100] Regarding the second access failure handling method: For example, when the target application is in the third level range, the second access failure handling method is to output a second prompt message to the target application indicating access failure; when the target application is in the fourth level range, the second access failure handling method is to access the second peer slave node corresponding to the slave node of the target slave node identifier, wherein the second peer slave node is a peer slave node that is not in the same data center as the slave node corresponding to the target slave node identifier; wherein, the lowest level in the fourth level range is higher than the highest level in the third level range.

[0101] The second peer node is a peer node that is not in the same data center as the target peer node. For example, the target peer node belongs to the first data center, but its second peer node belongs to the second data center. Of course, this situation usually means that, under normal circumstances, the second data center includes at least two peer nodes corresponding to each master node. Thus, when all master nodes in the first data center switch to their corresponding peer nodes in the second data center, the second data center includes at least three master nodes and at least one peer node connected to each of these at least three master nodes.

[0102] The lowest level in the fourth level range is higher than the highest level in the third level range. Additionally, the first and third level ranges can be set to be the same or different; if they are different, the highest level in the third level range is higher than the highest level in the first level range; the second and fourth level ranges can also be set to be the same or different.

[0103] S370. Process the target application access failure event based on the second access failure handling method.

[0104] If the second access failure handling method is to output a second prompt message to the target application to indicate access failure, then the second prompt message is directly output to the target application; if the second access failure handling method is to access the second peer slave node of the target slave node, then the second peer slave node of the target slave node is determined, the access address of the second peer slave node is obtained, and then the access address is used to access the target slave node.

[0105] Specifically, if a data center-level failure causes a complete master node switchover, the client will detect the anomaly and retrieve the latest status information table, slot mapping table, and central constraint information through service discovery. At this point, the central constraint information is not for all applications, but only for applications at the fourth level. This is because applications at other levels are not allowed to access the slave nodes of the second data center by default, to prevent excessive access from causing the slave nodes of the second data center to crash. The current target slave node determined based on this central constraint information is the second peer slave node of the original target slave node. After the current target slave node is determined, the access address of the current target slave node is obtained from the latest status information table, and then access is made through that address.

[0106] The technical solution provided by this invention determines a second access failure handling method after the target application fails to access the target slave node according to the application level of the target application corresponding to the read request, and completes the access failure event of the target application based on the second access failure handling method. In this way, different application levels correspond to different second access failure handling methods, which improves the flexibility and accuracy of access failure handling, and can effectively prevent normal slave nodes from being blocked, delayed or crashed due to excessive access volume.

[0107] See Figure 2 The distributed cluster in the data center is configured to send central constraint information and slot mapping table to the client in response to the client's service discovery command; and respond to the client's read request through the target slave node; wherein, the central constraint information is used to restrict the client to access only the slave node under one data center for the read request, and the target slave node is determined by the client based on the central constraint information, the slot mapping table and the target slot identifier corresponding to the target key of the read request.

[0108] Specifically, upon detecting a service discovery command sent by a client, the target cluster sends it center-qualified information, a slot mapping table, and a status information table. The target slave node of the target cluster, upon detecting a read request from a client, will respond to that read request.

[0109] For information on the central constraint information and the method for determining the target slave node, please refer to the aforementioned embodiments; this embodiment will not elaborate further here.

[0110] In one embodiment, the first data center includes at least three master nodes and a first slave node corresponding to each master node, and the second data center includes a second slave node corresponding to each master node; for each master node, after a master node fails, only the first slave node that has a corresponding relationship with the master node is allowed to enter the election voting process.

[0111] like Figure 2As shown, if the master node M1 fails, only the slave node S11 is allowed to enter the election voting process.

[0112] Specifically, such as Figure 5 As shown, for each master node, any slave node, upon detecting a fault, can initiate a master node election process or continue monitoring the master node's status. If a slave node initiates a master node election process, it checks whether the center identifier in its identifier is the same as the center identifier in the master node's identifier. If they are the same, it means they belong to the same data center and can participate in the election, broadcasting a vote request via the Gossip protocol. If not, it abandons the election and logs the result. A normal master node votes for the slave node. If the slave node receives more votes than a predetermined threshold, such as more than half the number of master nodes participating in the vote, it becomes the new master node, updates the cluster metadata, synchronizes data to all its slave nodes, and the election is complete, entering normal working status as the master node. If the election fails, it enters the next round of monitoring. If the master node times out and fails to connect, a new round of election is triggered. If the master node recovers, the election is rolled back, and its slave node role is restored.

[0113] This embodiment extends the RedisCluster Gossip protocol to synchronize data center topology information between nodes, such as by adding a data center identifier field, enabling cross-data center awareness. When the master node fails, slave nodes must pass a triple verification process to participate in master node election. This triple verification includes basic status verification, replication offset verification, and running an ID dictionary. Basic status verification filters out obviously unhealthy slave nodes, replication offset verification selects the strongest among the qualified ones, and running the ID dictionary ensures the uniqueness of the result in the event of a tie. Furthermore, all slave nodes participating in the election must belong to the same data center as the original master node to be allowed into the candidate list, i.e., to participate in the voting process. This prevents cross-data center slave nodes from becoming the master node in the event of a data center-level failure, thus mitigating the risk of split-brain and data inconsistency.

[0114] This embodiment determines whether the slave node initiating the election process and the master node in a disconnected state belong to the same data center. Only after determining that they belong to the same data center is the slave node allowed to enter the election voting process, and only then does the slave node have the opportunity to become the new master node. In this way, the old and new master nodes are located in the same data center, and all master nodes are distributed in the same data center at different times. This fundamentally avoids the situation where master nodes are distributed in different data centers and avoids high latency and erroneous switching. In addition, it avoids the occurrence of the target cluster split-brain phenomenon and ensures the availability of the target cluster.

[0115] In one embodiment, the distributed cluster includes a management device configured to, upon detecting a central-level failure in the first data center, output a prompt message indicating a central-level failure; and, in response to a target switching instruction for a second slave node combination, designate each second slave node in the second slave node combination as a new master node, and designate the original master node corresponding to each second slave node in the second slave node combination as a slave node, wherein the second slave node combination includes a second slave node corresponding to each master node, and the target switching instruction is manually triggered upon the occurrence of a central-level failure in the first data center.

[0116] Specifically, when the control equipment detects a center-level failure in the first data center—for example, all master and slave nodes in the first data center becoming disconnected—it does not directly perform a cross-center master node switch. Instead, it outputs a warning message indicating a center-level failure. At this point, the user can decide based on the actual situation: whether the first data center experienced a temporary data center failure or a major incident such as an earthquake causing the disconnection. If it's the former, the user can continue to wait; if the latter, the user can manually trigger a pre-defined instruction for the second slave node group, promoting each slave node in the second slave node group to master node, while the original master node in the first data center is demoted to a slave node. In this way, for cross-data center master node switching scenarios, automatic switching is not triggered within a preset latency threshold. Any cross-data center switching operation (regardless of whether the latency condition is met) can be configured to require manual confirmation before execution, avoiding automatic cross-data center master node selection operations in the distributed system and preventing frequent cross-data center master node switching.

[0117] To ensure rapid failover in the event of a central-level failure, native cluster failover forced switching commands are supported. In the event of a central-level failure, batch switching operations of master nodes can be performed through the management plane of the management device, for example:

[0118] If the primary master node in the first data center is alive but isolated, the user can trigger the `CLUSTER FAILOVER FORCE` command through the management plane to allow the distributed cluster to skip master-slave status checks and complete the failover of all master nodes within seconds. If all primary master nodes in the first data center are unavailable, the user can trigger the `CLUSTER FAILOVER TAKEOVER` command through the management plane to directly modify the configuration epoch (maxepoch+1), forcibly switching all inactive primary master nodes to the second slave nodes in the second slave node group. The configuration epoch is a monotonically increasing integer in Redis Cluster used to identify the cluster configuration version. The epoch value increases when the cluster topology changes (e.g., during a primary master node switch), ensuring all nodes reach a consensus on the cluster state. Both primary master node failover methods guarantee that the distributed cluster completes the failover of all primary master nodes within one minute in the event of a central-level failure.

[0119] The distributed system of this invention allows automatic switching of master nodes within the same data center, but does not allow automatic switching of master nodes in different data centers; master node switching between different data centers can only be completed through manual triggering; effectively avoiding fatal problems caused by frequent switching of master nodes between different data centers, such as data loss, split-brain, and disordered data reading and ordering.

[0120] Based on the aforementioned embodiments, the slave nodes of the distributed system are configured to cache the replication offset and timestamp of the corresponding master node when a disconnection with the master node is detected. The master node is configured to allow incremental replication operations of the slave node when a connection request from the corresponding slave node is detected, if it is determined that the slave node is in a first timeout state and / or the replication offset is within a predetermined replication backlog cache; otherwise, incremental replication operations of the slave node are prohibited, and a prompt message for full replication is output. The first timeout state refers to the timeout state corresponding to the replication disconnection duration being less than the maximum disconnection time threshold. All nodes are configured with the same maximum disconnection time threshold, and the replication disconnection duration is determined based on the current time and the last replication disconnection time.

[0121] The replication backlog buffer is a circular buffer maintained by the master node to store recently executed write commands. When a slave node disconnects, it can retrieve incremental data from the period of disconnection through this buffer, avoiding full replication. Full replication refers to the slave node synchronizing data by transmitting a complete snapshot of the master node's data (RDB file) and subsequent incremental logs (AOF), suitable for initial synchronization or scenarios with prolonged disconnection. Incremental replication, in contrast, involves the slave node transmitting only the write commands from the period of disconnection (via the replication backlog buffer), suitable for scenarios with brief network interruptions.

[0122] The maximum disconnection time threshold, also known as the replication interruption timeout control parameter, represents the maximum allowed connectionless time after the master and slave nodes disconnect. Upon timeout, a deep check is triggered to dynamically determine the replication type (incremental or full). Its specific value can be set by the user according to the actual situation of the distributed system, such as the size of the master node's replication backlog buffer; optionally, its value is 600 seconds. In this embodiment, the maximum disconnection time threshold is globally effective, meaning all master and slave nodes share the same maximum disconnection time threshold. Furthermore, the update mode for the maximum disconnection time threshold is preferably dynamic hot update, so that after reset, it takes effect immediately without restarting the node.

[0123] The replication offset, also known as the master-slave replication offset difference, is the difference between the replication offset of the master node and the replication offset of the slave node. Incremental replication is allowed if the replication offset is less than the predetermined replication backlog buffer.

[0124] Regarding timeout status monitoring: The master node is configured to periodically check the liveness status of slave nodes via heartbeat. If a slave node's disconnection time exceeds the maximum disconnection time threshold, the slave node is marked as having lost replication connection, and the disconnection time is recorded. Simultaneously, the replication backlog buffer for that slave node is retained until it is overwritten by new data or manually cleared. The slave node is configured to cache the master node's replication offset (cached_master->read_reploff) and timestamp (repl_offset_time) upon disconnection; a scheduled task is started to check the network status; if the network does not recover within a timeout period, it enters a reconnection waiting state.

[0125] Specifically, such as Figure 6 As shown, the slave node periodically sends replication acknowledgment messages (REPLCONFACK) to the master node. If the master node does not respond within a timeout period, it sends a status message to the state machine indicating that the master node's replication connection is broken. The state machine records the last replication disconnection time (last_repl_down_time), sends a notification message about the broken replication connection to the monitoring system, and then uses a calculation module to periodically determine the timeout status. For example, the calculation module determines the difference between the current time and the last replication disconnection time, and uses this difference as the replication disconnection duration. If the replication disconnection duration is less than the maximum disconnection time threshold, the slave node is determined to be in the first timeout state; otherwise, the slave node is determined to be in the second timeout state. It should be noted that the state machine is also called the state machine replication model, which ensures that the slave node's state eventually becomes completely consistent with the master node by having the master and slave nodes execute the same commands in the same order.

[0126] If the replication disconnection duration of a slave node is less than the maximum disconnection time threshold (i.e., the slave node is in the first timeout state), the master node allows the slave node to rebuild the replication connection for incremental replication. If the replication disconnection duration of a slave node is greater than or equal to the maximum disconnection time threshold, and the replication offset is within the predetermined replication backlog buffer, the master node allows the slave node to perform incremental replication. If the replication disconnection duration of a slave node is greater than or equal to the maximum disconnection time threshold, and the replication offset is not within the predetermined replication backlog buffer, the master node prevents the slave node from performing incremental replication and triggers an alarm message for full replication, such as a forced full replication command, to force the slave node to perform full replication, while requiring manual confirmation from the user. Users can choose an appropriate time to initiate synchronization or reject / delay this operation based on service recovery priorities and bandwidth conditions. By manually controlling the timing of high-bandwidth-consuming full replication, it effectively prevents it from crowding out bandwidth for critical service recovery, achieving orderly, controllable, and gradual recovery after data center-level failures.

[0127] In this embodiment, the master node triggers the reconnection, incremental replication, or forces the slave node to perform full replication by using the size relationship between the replication offset and the predetermined replication backlog buffer, as well as the timeout status. This improves the scientific nature and security of the slave node reconnection, incremental replication, and full replication.

[0128] Building upon the aforementioned example, after the master node issues a forced full replication command, it allows the slave nodes to perform full replication, but not immediately. Instead, it notifies the user via an alarm message. The user then semi-automatically completes the full replication of each slave node through the control device. Optionally, the control device is configured to, in response to a target temporary shutdown command, reset the maximum disconnection time threshold to zero, perform incremental replication on slave nodes marked as outdated (ignoring offset differences and timeout limits), and reset the maximum disconnection time threshold to its default value upon completion of incremental replication. The target temporary shutdown command is configured to be manually triggered.

[0129] Users are advised to perform batch repairs of master-slave relationships during off-peak network periods (such as the early morning when traffic is low) to avoid the impact of full replication on peak business periods. Specifically, timeout checks are disabled via cluster commands, setting the maximum disconnection time threshold to zero to indicate forced recovery. Offset differences and timeout limits are ignored, and incremental replication is attempted on all slave nodes marked as outdated. Upon successful recovery, the maximum disconnection time threshold is automatically reset to the default value. By combining this with the semi-automatic full replication method of the management equipment, the risks of full replication are made controllable. Through dynamic control of timeout status, offset difference verification, and manual confirmation, the full replication trigger rate is reduced by 80%, and batch repair capabilities are provided during the maintenance window, avoiding the risk of master node bandwidth storms caused by full replication and reducing operational risks during peak business periods.

[0130] In summary, based on automated high availability, this embodiment of the invention introduces a configurable manual confirmation step at key decision points involving cross-data center resource allocation (master node switching) and large-scale resource consumption (full replication across data centers), giving users the final decision-making power. This significantly improves the security margin and controllability of system operation and maintenance, and prevents serious consequences caused by automated misjudgments.

[0131] The following are embodiments of the data processing apparatus provided in this invention. This apparatus and the data processing method in the foregoing embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the data processing apparatus, please refer to the contents of the above embodiments.

[0132] Figure 7A This is a schematic diagram of the data processing apparatus provided in an embodiment of the present invention. The apparatus is installed on a client side of a target cluster. The target cluster includes at least two data centers, namely a first data center and a second data center. The first data center includes a master node and a first slave node corresponding to the master node. The second data center includes a second slave node corresponding to the master node. The identifiers of both the first and second slave nodes include the center identifier of their respective data centers. Figure 7A As shown, the device includes:

[0133] Target key module 710 is used to determine the target key corresponding to the read request in response to the read request;

[0134] The slave node determination module 720 is used to determine the target slot identifier corresponding to the target key, and to determine the target slave node identifier based on the target slot identifier, the pre-acquired center restriction information and the pre-acquired slot mapping table. The center restriction information is used to limit the client to access slave nodes under one data center for read requests.

[0135] Access module 730 is used to access the slave node corresponding to the target slave node identifier.

[0136] In one embodiment, such as Figure 7B As shown, the device also includes an acquisition module 700;

[0137] The acquisition module is used to obtain the center-defined information and the slot mapping table from the target cluster through service discovery operations;

[0138] The central limitation information is bound to the identifier of the client.

[0139] In one embodiment, the first data center includes a master node and at least one first slave node corresponding to the master node, the second data center includes at least one second slave node corresponding to the master node, and the center definition information includes a center identifier; the slave node module includes:

[0140] The slave node unit is used to determine at least one slave node identifier corresponding to the target slot identifier in the data center corresponding to the center identifier, based on the target slot identifier and the slot mapping table.

[0141] The selection unit is used to select one of the at least one slave node identifiers as the target slave node identifier.

[0142] The selection unit selects the target slave node identifier from the at least one slave node identifier based on the load balancing principle.

[0143] In one embodiment, the central limiting information includes the node identifier combination, and the second slave node identifier combination includes the identifier of a slave node connected to each of the master nodes in the data centers that are allowed to be accessed; the slave node module is used for:

[0144] Based on the slot mapping table, the slave node identifier corresponding to the target slot identifier is determined from the slave node combinations corresponding to the second slave node identifier combination.

[0145] The slave node identifier corresponding to the target slot identifier is used as the target slave node identifier.

[0146] In one embodiment, such as Figure 7C As shown, the device also includes a first response module 740, which is used for:

[0147] If the access fails, determine the target application corresponding to the read request, and the application level of the target application;

[0148] Based on the correspondence between the application level and the pre-determined application level and the first access failure handling method, a first access failure handling method for the target application is determined. The first access failure handling method is the access failure handling method when all data centers are available.

[0149] The target application access failure event is handled based on the first access failure handling method.

[0150] In one embodiment, when the application level is within the first level range, the first access failure handling method is to output a first prompt message to the target application to indicate access failure.

[0151] When the application level is within the second level range, the first access failure handling method is to access the first peer slave node corresponding to the target slave node identifier, wherein the first peer slave node is a peer slave node located in the same data center as the target slave node identifier.

[0152] The lowest level in the second level range is higher than the highest level in the first level range.

[0153] In one embodiment, such as Figure 7D As shown, the device also includes a second response module 750, which is used for:

[0154] If the access fails, and it is detected that all the master nodes are distributed in the second data center, then the application level of the target application corresponding to the read request is determined;

[0155] Based on the correspondence between the application level and the pre-determined application level and the second access failure handling method, a second access failure handling method for the target application is determined. The second access failure handling method is the access failure handling method in the event of a central-level failure in the first data center.

[0156] The target application access failure event is handled based on the second access failure handling method.

[0157] In one embodiment, when the target application is at the third level, the second access failure handling method is to output a second prompt message to the target application to indicate that the access has failed.

[0158] When the target application is in the fourth level range, the second access failure handling method is to access the second peer slave node corresponding to the target slave node identifier. The second peer slave node is a peer slave node that is not in the same data center as the slave node corresponding to the target slave node identifier.

[0159] The lowest level in the fourth level range is higher than the highest level in the third level range.

[0160] The technical solution provided by this invention, since the central limitation information is used to limit the client to access the slave node under one data center for read requests, the target slave node determined based on the central limitation information can only be in one data center each time. Therefore, the client can only access the slave node under one data center each time, which optimizes the client's access path, effectively prevents unnecessary cross-center access by the client, avoids network latency risks and data consistency risks caused by unnecessary cross-data center access, and can significantly improve the user experience.

[0161] The data processing apparatus provided in the embodiments of the present invention can execute the data processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the data processing method.

[0162] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 8 A block diagram of an exemplary server 12 suitable for implementing embodiments of the present invention is shown. Figure 8 The server 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0163] like Figure 8 As shown, server 12 is presented in the form of a general-purpose computing device. The components of server 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0164] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0165] Server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by server 12, including volatile and non-volatile media, removable and non-removable media.

[0166] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 8 Not shown; usually referred to as a "hard drive"). Although Figure 8 As not shown, disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., CD-ROMs, DVD-ROMs, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0167] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0168] Server 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable users to interact with server 12, and / or with any device that enables server 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, server 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of server 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0169] Processing unit 16 executes various functional applications and data processing by running programs stored in system memory 28, such as implementing the data processing method steps provided in this embodiment, which includes:

[0170] In response to a read request, determine the target key corresponding to the read request;

[0171] Determine the target slot identifier corresponding to the target key, and determine the target slave node identifier based on the target slot identifier, the pre-acquired center constraint information, and the pre-acquired slot mapping table, wherein the center constraint information includes the center identifier;

[0172] Access the slave node corresponding to the target slave node identifier.

[0173] Of course, those skilled in the art will understand that the processor can also implement the technical solutions of the data processing methods provided in any embodiment of the present invention.

[0174] This embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the data processing method steps provided in the foregoing embodiments of the present invention. The method includes:

[0175] In response to a read request, determine the target key corresponding to the read request;

[0176] Determine the target slot identifier corresponding to the target key, and determine the target slave node identifier based on the target slot identifier, the pre-acquired center constraint information, and the pre-acquired slot mapping table, wherein the center constraint information includes the center identifier;

[0177] Access the slave node corresponding to the target slave node identifier.

[0178] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0179] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0180] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0181] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0182] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0183] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data processing method provided in any embodiment of this application.

[0184] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0185] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A data processing method, characterized in that, The method is applied to a client based on a target cluster, the target cluster including at least two data centers, the at least two data centers including a first data center and a second data center, the first data center including a master node and a first slave node corresponding to the master node, the second data center including a second slave node corresponding to the master node, the identifiers of the first slave node and the second slave node both including the center identifier of the data center they belong to, the method including: In response to a read request, determine the target key corresponding to the read request; Determine the target slot identifier corresponding to the target key, and determine the target slave node identifier based on the target slot identifier, the pre-acquired center constraint information, and the pre-acquired slot mapping table, wherein the center constraint information includes the center identifier; Access the slave node corresponding to the target slave node identifier.

2. The method according to claim 1, characterized in that, The center constraint information and the slot mapping table are obtained through the following steps: Through service discovery operations, the central constraint information and the slot mapping table are obtained from the target cluster; The central limitation information is bound to the identifier of the client.

3. The method according to claim 1, characterized in that, The first data center includes a master node and at least one first slave node corresponding to the master node, and the second data center includes at least one second slave node corresponding to the master node. The step of determining the target slave node identifier based on the target slot identifier, pre-acquired center constraint information, and a pre-acquired slot mapping table includes: Based on the target slot identifier and the slot mapping table, at least one slave node identifier corresponding to the target slot identifier is determined from the first slave node identifier combination corresponding to the center identifier; Use one of the at least one slave node identifiers as the target slave node identifier.

4. The method according to claim 3, characterized in that, The step of using one of the at least one slave node identifiers as the target slave node identifier includes: Based on the load balancing principle, the target slave node identifier is selected from the at least one slave node identifier.

5. The method according to claim 1, characterized in that, The central restriction information includes the second slave node identifier combination, which includes the identifier of a slave node connected to each of the master nodes in the data center that is allowed to be accessed. The target slave node identifier is determined through the following steps: Based on the slot mapping table, the slave node identifier corresponding to the target slot identifier is determined from the second slave node identifier combination; The slave node identifier corresponding to the target slot identifier is used as the target slave node identifier.

6. The method according to claim 1, characterized in that, After accessing the slave node corresponding to the target slave node identifier, the process further includes: If the access fails, determine the target application corresponding to the read request, and the application level of the target application; Based on the correspondence between the application level and the pre-determined application level and the first access failure handling method, a first access failure handling method for the target application is determined. The first access failure handling method is the access failure handling method when all data centers are available. The target application access failure event is handled based on the first access failure handling method.

7. The method according to claim 6, characterized in that, When the application level is within the first level range, the first access failure handling method is to output a first prompt message indicating access failure to the target application. When the application level is within the second level range, the first access failure handling method is to access the first peer slave node corresponding to the target slave node identifier, wherein the first peer slave node is a peer slave node located in the same data center as the target slave node identifier. The lowest level in the second level range is higher than the highest level in the first level range.

8. The method according to claim 1, characterized in that, After accessing the slave node corresponding to the target slave node identifier, the process further includes: If the access fails, and it is detected that all the master nodes are distributed in the second data center, then the application level of the target application corresponding to the read request is determined; Based on the correspondence between the application level and the pre-determined application level and the second access failure handling method, a second access failure handling method for the target application is determined. The second access failure handling method is the access failure handling method in the event of a central-level failure in the first data center. The target application access failure event is handled based on the second access failure handling method.

9. The method according to claim 8, characterized in that, When the target application is in the third level range, the second access failure handling method is to output a second prompt message to the target application to indicate that the access has failed; When the target application is in the fourth level range, the second access failure handling method is to access the second peer slave node corresponding to the target slave node identifier. The second peer slave node is a peer slave node that is not in the same data center as the slave node corresponding to the target slave node identifier. The lowest level in the fourth level range is higher than the highest level in the third level range.

10. A distributed cluster, characterized in that, The distributed cluster is the target cluster described in any one of claims 1-9; The target cluster is configured to, in response to a client's service discovery command, send center-defined information and a slot mapping table to the client; and respond to the client's read requests through the target slave nodes; The central restriction information is used to restrict the client to access only a slave node under one data center for read requests. The target slave node is determined by the client based on the central restriction information, the slot mapping table, and the target slot identifier corresponding to the target key of the read request.

11. The distributed cluster according to claim 10, characterized in that, The first data center includes at least three master nodes and a first slave node corresponding to each of the master nodes; the second data center includes a second slave node corresponding to each of the master nodes. For each master node, after a master node fails, only the first slave node that has a corresponding relationship with the master node is allowed to enter the election voting process.

12. The distributed cluster according to claim 10, characterized in that, The cluster includes management and control equipment; The control device is configured to output a prompt message indicating a central-level fault when a central-level fault is detected in the first data center. In response to a target switching instruction for a second slave node combination, each second slave node in the second slave node combination is designated as a new master node, and the original master node corresponding to each second slave node in the second slave node combination is designated as a slave node; the second slave node combination includes a second slave node corresponding to each of the master nodes, and the target switching instruction is manually triggered in the event of a central-level failure in the first data center.

13. The distributed cluster according to claim 10, characterized in that, Also includes: The slave node is configured to cache the replication offset and last disconnection time of the master node when it detects a disconnection from the corresponding master node. The master node is configured to allow incremental replication operations of the slave node if it is determined that the slave node is in a first timeout state and / or the replication offset is within a predetermined replication backlog buffer when a connection request is detected from the corresponding slave node. Conversely, incremental replication operations on the slave node are prohibited, and a prompt message for full replication is output. The first timeout state refers to the timeout state corresponding to the replication disconnection duration being less than the maximum disconnection time threshold. All nodes are configured with the same maximum disconnection time threshold, and the replication disconnection duration is determined based on the current time and the last replication disconnection time.

14. The distributed cluster according to claim 10, characterized in that, The cluster includes management and control equipment; The control device is configured to, in response to a target temporary shutdown command, reset the maximum disconnection time threshold to zero, perform incremental replication on slave nodes marked as outdated without ignoring offset differences and timeout limits, and reset the maximum disconnection time threshold to a default value upon completion of the incremental replication. The target temporary shutdown command is configured to be manually triggered.

15. A data processing apparatus, characterized in that, The device is configured on a client based on a target cluster, the target cluster including at least two data centers, the at least two data centers including a first data center and a second data center, the first data center including a master node and a first slave node corresponding to the master node, the second data center including a second slave node corresponding to the master node, the identifiers of the first slave node and the second slave node both including the center identifier of the data center they belong to, the device includes: The target key module is used to determine the target key corresponding to the read request in response to the read request; The slave node module is used to determine the target slot identifier corresponding to the target key, and to determine the target slave node identifier based on the target slot identifier, the pre-acquired center constraint information and the pre-acquired slot mapping table, wherein the center constraint information includes the center identifier; The access module is used to access the slave node corresponding to the target slave node identifier.

16. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method as described in any one of claims 1-9.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the data processing method as described in any one of claims 1-9.

18. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the data processing method according to any one of claims 1-9.

Images