Data processing methods and devices based on cluster networks, electronic equipment, and media

Abstract

Embodiments of the present application provide a kind of data processing method and device based on cluster network, electronic equipment, medium, belong to communication technical field.The method comprises: when the sentinel cluster detects that the running state of first master node is failure state at first time point, determine second master node from first slave node;Other first slave node is as second slave node, and communication connection is established between client and second master node;When first master node responds to transaction request sent by client at third time point, execute transaction request based on distributed lock, and store distributed lock to first log cache, and operation statement corresponding to transaction request is stored to second log cache;When the sentinel cluster detects that the running state of first master node is normal state at fourth time point, data update is carried out to the preset database of second master node based on first log cache and second log cache.The accuracy of data processing in cluster network can be improved in the embodiments of the present application.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a data processing method and apparatus, electronic device, and medium based on a trunking network. Background Technology

[0002] A cluster network refers to the network infrastructure used for communication and data exchange between nodes (or servers) in a computer cluster. For example, in a distributed system providing Redis services (each Redis service can be called a "node"), a Redis cluster can be formed to provide services to terminals. To avoid multiple nodes performing transactional operations (i.e., adding, deleting, and modifying data) on the data simultaneously, a master-slave model is typically used to build the cluster (i.e., the master node provides write operations, and the slave nodes provide read operations). However, in this model, if the master node fails, one of the slave nodes needs to be manually promoted to master; otherwise, the sub-cluster will be unavailable. Therefore, how to reasonably and effectively manage the nodes within the cluster to ensure that the cluster as a whole can provide highly available and high-performance services to the outside world becomes particularly important.

[0003] Currently, related technologies typically employ a sentinel mechanism in the cluster network for node data processing. This involves a designated sentinel node monitoring the master and slave nodes, and electing a new master node when the master node malfunctions to ensure correct data processing during network communication. However, in the event of network failure or instability, the sentinel node is prone to misjudging the master node's operational status, leading to two master nodes in the cluster network. This can result in data inconsistency (i.e., different master nodes in different clusters may write the same data differently, causing inconsistencies) or data loss (i.e., the new master node clears the write commands executed on the old master node during the master-slave switch). Furthermore, the proposed method of adjusting configuration file parameters to avoid these problems does not effectively solve the underlying issues. Summary of the Invention

[0004] The main objective of this application is to propose a data processing method, apparatus, electronic device, and medium based on a cluster network, which can better solve the problems of data inconsistency or data loss when processing data in a cluster network with a sentinel mechanism, and improve the accuracy of data processing in the cluster network.

[0005] To achieve the above objectives, a first aspect of this application proposes a data processing method based on a cluster network, applied to a cluster network including a first master node, at least one first slave node, and a sentinel cluster. The first master node and a first client are located in a first network partition of the cluster network, and the first master node is communicatively connected to the first client. The sentinel cluster is located in a second network partition of the cluster network, and the sentinel cluster is communicatively connected to the first client. The first slave node is located in either the first or second network partition, and the total number of nodes in the first network partition is less than or equal to half of the total number of nodes in the cluster network. The method includes:

[0006] When the sentinel cluster detects that the first master node is in a fault state at a first time point, it selects a second master node based on the first slave node in the third network partition of the cluster network. The fault state indicates that the communication between the first master node and the sentinel cluster is abnormal. The total number of nodes in the third network partition is greater than or equal to half of the total number of nodes in the cluster network. The second network partition is the same as or different from the third network partition.

[0007] The other first slave nodes in the cluster network are designated as second slave nodes, and the second master node completes the master-slave switch at a second time point; wherein the second time point is later than the first time point;

[0008] When the first master node responds to the transaction request sent by the client at a third time point, it executes the transaction request based on a preset distributed lock; wherein, the distributed lock is used to lock the data associated with the operation statement corresponding to the transaction request, and the third time point is later than the first time point;

[0009] The distributed lock is stored in a preset first log cache, and the operation statement corresponding to the transaction request is stored in a preset second log cache;

[0010] When the sentinel cluster detects that the first master node is in a normal operating state at the fourth time point, it updates the preset database of the second master node based on the first log cache and the second log cache; wherein, the normal state is used to indicate that the communication between the first master node and the sentinel cluster is normal, the fourth time point is later than the third time point, and the fourth time point is later than the second time point.

[0011] In some embodiments, updating the preset database of the second master node based on the first log cache and the second log cache includes:

[0012] Release the distributed lock in the first log cache through the first master node, and send the second log cache to the second master node;

[0013] The first transaction operation data at the third time point is obtained by executing the operation statement corresponding to the transaction request through the second master node;

[0014] The preset database is updated based on the data from the first transaction operation.

[0015] In some embodiments, the method further includes:

[0016] The first master node is used as the second slave node, and a full synchronization signal is sent to all second slave nodes based on the updated preset database;

[0017] Each of the second slave nodes responds to the full synchronization signal and updates the data stored in each of the second slave nodes based on the preset database.

[0018] In some embodiments, before determining the second master node based on the first slave node in the third network partition of the cluster network when the sentinel cluster detects that the first master node is in a fault state at a first time point, the method further includes:

[0019] When the first sentinel node in the sentinel cluster detects that the first master node is in a fault state at the first time point, it sends a master node fault voting request to the other sentinel nodes in the sentinel cluster.

[0020] The first sentinel node receives the master node failure vote response returned by the other sentinel nodes, and the master node failure vote response is used to indicate the detection result of the other sentinel nodes on the running status of the first master node at the first time point;

[0021] When the number of fault voting responses from the master node indicating that the first master node's operating status is the fault state is greater than a preset number, the operating status of the first master node at the first time point is determined to be a fault state.

[0022] In some embodiments, the step of selecting a second master node from the first slave node of the third network partition in the cluster network based on the sentinel cluster includes:

[0023] The sentinel master node is determined from the sentinel nodes of the sentinel cluster based on a preset consensus algorithm;

[0024] Based on the sentinel master node's node selection of the first slave node in the third network partition of the cluster network, the second master node is determined.

[0025] In some embodiments, the step of selecting the second master node based on the first slave node of the third network partition in the cluster network using the sentinel master node includes:

[0026] Obtain the node running status and node attribute parameters of each of the first slave nodes in the third network partition;

[0027] Based on the node's operating status and the node's attribute parameters, the first slave node in the third network partition is filtered to determine the second master node.

[0028] In some embodiments, before updating the preset database of the second master node based on the first log cache and the second log cache when the sentinel cluster detects that the first master node is in a normal operating state at the fourth time point, the method further includes:

[0029] Each sentinel node in the sentinel cluster sends a PING command to the first master node based on a preset frequency, and determines multiple response data returned by the master node according to the PING command within a preset time period;

[0030] The operating status of the first master node is detected based on the multiple response data;

[0031] When the second sentry node in the sentry cluster detects that the first master node is in a normal operating state at the fourth time point, it determines that the first master node is in a normal operating state at the first time point.

[0032] To achieve the above objectives, a second aspect of this application provides a data processing apparatus based on a cluster network, applied to a cluster network including a first master node, at least one first slave node, and a sentinel cluster. The first master node and a first client are located in a first network partition of the cluster network, and the first master node is communicatively connected to the first client. The sentinel cluster is located in a second network partition of the cluster network, and the sentinel cluster is communicatively connected to the first client. The first slave node is located in either the first or second network partition, and the total number of nodes in the first network partition is less than or equal to half the total number of nodes in the cluster network. The apparatus includes:

[0033] The master node determination module is used to determine a second master node when the sentinel cluster detects that the first master node is in a fault state at a first time point, based on the sentinel cluster selecting the first slave node in the third network partition of the cluster network; wherein, the fault state is used to indicate that the communication between the first master node and the sentinel cluster is abnormal, the total number of nodes in the third network partition is greater than or equal to half of the total number of nodes in the cluster network, and the second network partition is the same as or different from the third network partition;

[0034] The switching module is used to designate other first slave nodes in the cluster network as second slave nodes and to complete the master-slave switch of the second master node at a second time point; wherein the second time point is later than the first time point;

[0035] The request execution module is used to execute the transaction request based on a preset distributed lock when the first master node responds to the transaction request sent by the client at a third time point; wherein, the distributed lock is used to lock the data associated with the operation statement corresponding to the transaction request, and the third time point is later than the first time point;

[0036] The storage module is used to store the distributed lock in a preset first log cache and to store the operation statement corresponding to the transaction request in a preset second log cache;

[0037] The update module is used to update the preset database of the second master node based on the first log cache and the second log cache when the sentinel cluster detects that the first master node is in a normal operating state at the fourth time point; wherein, the normal state is used to indicate that the communication between the first master node and the sentinel cluster is normal, the fourth time point is later than the third time point, and the fourth time point is later than the second time point.

[0038] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0039] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0040] The data processing method, apparatus, electronic device, and medium based on a cluster network proposed in this application, by using a distributed lock to execute transaction requests on a first master node in a faulty state, ensures that transaction requests executed on the old first master node after a split-brain event (i.e., data inconsistency or data loss) can only be operated by one node, avoiding data inconsistency caused by multiple nodes operating on the same data simultaneously. Furthermore, this application stores the distributed lock of the first master node in a preset first log cache, stores the operation statements corresponding to the transaction requests in a preset second log cache, and synchronizes the first and second log caches to the newly determined second master node to update the preset database of the second master node, thus effectively preventing data loss. Therefore, this application can better solve the problem of data inconsistency or data loss when processing data in a cluster network with a sentinel mechanism, improving the accuracy of data processing in the cluster network. Attached Figure Description

[0041] Figure 1 This is a cluster network deployment diagram using the sentinel mechanism provided in the embodiments of this application;

[0042] Figure 2A This is a schematic diagram of a cluster network provided in an embodiment of this application after a split-brain situation occurs;

[0043] Figure 2B This is another schematic diagram of the cluster network provided in the embodiments of this application after a split-brain situation occurs;

[0044] Figure 3 This is a flowchart of a data processing method based on a cluster network provided in an embodiment of this application;

[0045] Figure 4 This is a schematic diagram of a network partition in a cluster network provided in an embodiment of this application;

[0046] Figure 5A This is a schematic diagram illustrating the principle of subjective offline status of cluster nodes in Sentinel mode.

[0047] Figure 5B This is a schematic diagram illustrating the principle of objective offline status of cluster nodes in Sentinel mode.

[0048] Figure 6 yes Figure 3 A flowchart of step S310 in the process;

[0049] Figure 7 This is a logical diagram illustrating how a sentinel notifies the master node of changes in a cluster under sentinel mode.

[0050] Figure 8 yes Figure 6 A flowchart of step S620 in the process;

[0051] Figure 9 This is a timeline diagram of fault transfer after schizophrenia provided in an embodiment of this application;

[0052] Figure 10 yes Figure 3 A flowchart of step S350 in the process;

[0053] Figure 11 This is a specific logical diagram of the data processing method based on a cluster network provided in the embodiments of this application;

[0054] Figure 12 This is a schematic diagram of a data processing device based on a cluster network provided in an embodiment of this application;

[0055] Figure 13 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0059] First, let's analyze some of the terms used in this application:

[0060] Split-brain: In the field of communications, this is a vivid metaphor, like the "split brain," meaning that what was originally a single "brain" has been split into two or more "brains." In distributed systems, network partitions or node failures cause communication breakdowns between nodes, leading to data consistency issues. In a high-availability cluster, when multiple servers cannot detect each other within a specified time due to network problems, they form new, smaller clusters. Each smaller cluster elects a new master node, providing independent services to the outside world.

[0061] Cluster deployment refers to bringing together multiple servers to provide the same service, making it appear to the client as if there is only one server. This deployment method improves computing speed by using multiple computers for parallel computing and increases system reliability through redundant deployment; even if one server fails, the entire system can still operate normally. However, the physical machines in a cluster deployment may be deployed across different regions, and if network failures occur between these regions, a split-brain problem may occur.

[0062] Redis is an in-memory key-value store database that is widely used in database, caching, or message broker scenarios.

[0063] A Redis cluster is a cluster formed by multiple Redis services electing master and slave nodes. In a Redis cluster, if the master node fails, one of the slave nodes needs to be promoted to the new master node as quickly as possible to ensure the normal operation of the Redis cluster. Since the master-slave mode of a Redis cluster does not have the function of actively electing and switching nodes, manual switching is required, which not only affects business operations but also increases maintenance costs. Therefore, the Sentinel mode is introduced, which can automatically switch over when the master node fails, ensuring high availability of Redis.

[0064] Redis Split-Brain: Redis split-brain refers to an abnormal situation that occurs during the operation of a Redis server, in which a Redis cluster splits into two or more independently running Redis clusters, and they lose communication and data synchronization.

[0065] Redis Master-Slave Mode: In the Redis master-slave architecture, read and write operations are separated. The master node provides write operations, while the slave node only provides read operations, and the master node will synchronize its data with the slave node.

[0066] Sentinel mechanism: Typically deployed on a separate server or container, it avoids sharing hardware resources with the master server, thereby reducing the risk of service interruption due to hardware failure. The Sentinel mechanism can use the Raft protocol, and Sentinel nodes should not be deployed on the same physical machine to prevent hardware failures from affecting all Sentinel nodes. For high availability, Sentinel nodes can form a cluster, monitoring each other to ensure system robustness. Through the above deployment strategies and configurations, the Redis Sentinel mechanism can ensure rapid response in the event of a failure, achieving automated failover, thus improving the stability and availability of the Redis cluster.

[0067] Raft protocol: A strongly consistent, decentralized, and highly available distributed protocol used to solve the consistency problem in distributed systems. The Raft protocol defines three roles for nodes: Leader, Follower, and Candidate. Under normal circumstances, most nodes are in the Follower state, while the Leader handles all client transaction requests. If the Leader fails, the Followers become Candidates, and a new Leader is elected through voting.

[0068] Full synchronization: refers to the process of completely synchronizing all data from the master node to the slave nodes.

[0069] Incremental synchronization: The process of synchronizing only the newly added or changed parts of the business data to the target node.

[0070] Subjective offline: A single sentinel node periodically sends a PING command to the Redis master and slave nodes. If it does not receive a response within a certain period of time, it is determined that the Redis node is subjectively offline.

[0071] Objective shutdown: Multiple sentinel nodes in the sentinel cluster periodically send PING commands to the Redis master and slave nodes. When most sentinel nodes do not receive a response within a certain period of time, the collective decision (i.e., based on the majority rule) determines that the Redis node is objectively offline.

[0072] Distributed locks are used to ensure mutual exclusion access between different processes or node services in a distributed system. This means that the scope of a lock can span multiple processes or node services, ensuring that only one client can access shared resources (such as table data in a database) at any given time.

[0073] A cluster network refers to the network infrastructure used for communication and data exchange between nodes (or servers) in a computer cluster. For example, in a distributed system providing Redis services (each Redis service can be called a "node"), to address single points of failure, improve system availability, achieve load balancing, and horizontal scaling, the same Redis service (each Redis service is referred to as a "node" in this article) is often deployed on multiple physical machines in different regions. This forms a Redis cluster to provide services to the end users. Since each node is equal and provides the same service, and handling client requests from the end users often only requires one Redis service, and to avoid multiple nodes performing transaction operations (i.e., adding, deleting, and modifying data) on the data simultaneously, a master-slave model is used to build the cluster (i.e., the master node provides write operations, and the slave nodes provide read operations). However, in this model, if the master node fails, one of the slave nodes needs to be manually promoted to master; otherwise, the sub-cluster will be unavailable. This drawback significantly reduces system availability. Therefore, it is particularly important to manage the nodes within the cluster in a reasonable and effective manner so that the cluster as a whole can provide highly available and high-performance services to the outside world.

[0074] Currently, related technologies typically employ a sentinel mechanism in cluster networks for node data processing. This involves using designated sentinel nodes to monitor master and slave nodes, and electing a new master node when the master node malfunctions, thus ensuring the correctness of data processing during network communication. For example... Figure 1 As shown, Figure 1 The diagram illustrates a cluster network deployment using a sentinel mechanism. Clients 1 and 2 can communicate with the master node in the cluster network, and the master node connects to slave nodes 1 through 3 via a master-slave mode. Simultaneously, at least one sentinel node (e.g., ...) can be used. Figure 1 Sentinel nodes 1 to 3 in the cluster network monitor each node in real time.

[0075] However, Redis clusters are prone to the following two split-brain behaviors: (1) In the event of network failure or instability, communication between the Redis master node and the sentinel or slave node may be interrupted. At this time, the sentinel may mistakenly believe that the Redis master node has crashed. Then the sentinel starts the election process to elect a new master node from the existing slave nodes. If the network recovers at this time, the old Redis master node is still running, which will result in two master nodes. (2) If the Redis master node "fakes dead", that is, the Redis master node process is still running, but it is not responding. At this time, because the failover strategy of the sentinel node is too sensitive to network anomalies, it is easy to perform the election of the master node under the wrong situation. That is, the false failure causes the Redis cluster to elect an extra master node. Thus, sentinel nodes are prone to misjudging the running status of Redis master nodes, resulting in two master nodes in the cluster network. This can easily lead to data inconsistency (i.e., master nodes in different clusters may write the same data differently, resulting in data inconsistency) or data loss (i.e., the new master node will send a full synchronization (i.e., master-slave replication) command to all slave nodes, causing all slave nodes to re-synchronize. Before this, the data on the instance will be cleared, so during the master-slave switch, the write commands executed on the old master node will also be cleared).

[0076] Based on this, to address the issues of data inconsistency and data loss caused by the split-brain problem in Redis clusters under Sentinel mode, the relevant technical solutions are to avoid the split-brain problem by adjusting the parameters of the configuration file. For example, the adjustment of the configuration file parameters may include: (1) min-slaves-to-write: the number of slave nodes communicating with the master node must be greater than or equal to this value of the master node, otherwise the master node will refuse to write; (2) min-slaves-max-lag: the ACK message delay between the master node and the slave node must be less than this value, otherwise the master node will refuse to write. Regarding the parameter min-slaves-to-write, since the Sentinel cluster will re-elect a new node when it believes that the master node has failed, and the election of the master node is based on the "majority rule", that is, more than half of the slave nodes must elect the same node for that node to become the master node. Therefore, min-slaves-to-write is generally set to more than half of the number of nodes in the cluster (e.g., in a cluster with 5 nodes, min-slaves-to-write is set to 3). Figure 2A and Figure 2B The text lists two scenarios for a cluster of 5 cells after a split-brain effect occurs. For example... Figure 2AAfter a split-brain scenario, since cluster 2 has 3 nodes, new nodes will be created in cluster 2. At this point, the number of slave nodes in cluster 1 communicating with the master node will be 2, which is less than the min-slaves-to-write value, so the master node will be denied write access. Figure 2B After the split-brain event, since cluster 2 has only 2 nodes, which is less than half, no new nodes can be created. Cluster 1, however, has 3 nodes (including the master node itself), indicating that the number of slave nodes communicating with the master in cluster 1 equals the min-slaves-to-write threshold, allowing the master node to write. Regarding the min-slaves-max-lag parameter, since data synchronization between the master and slave nodes occurs at regular intervals, assuming an 8-second synchronization interval, when min-slaves-max-lag is set to 10, meaning the ACK message delay between the master and slave nodes is greater than or equal to 10 seconds, the master node will refuse write operations. At this point, if the min-slaves-max-lag requirement is met, the following scenario will lead to a flaw in this solution: Assuming the master node experiences a brief network failure of 9 seconds, after 8 seconds, the sentinel node will consider the master node to be objectively offline and begin failover; assuming the failover takes 4 seconds, when the master node recovers in the 9th second, the old master node can still provide write operations within the 9th to 12th seconds, causing data generated during this period to be discarded when a full data synchronization is performed after the new master node is elected.

[0077] Therefore, while the Redis split-brain problem can be mitigated to some extent through proper configuration, it is difficult to completely solve. Furthermore, the parameters in the configuration file are difficult to dynamically adjust in real-time based on the actual scenario. These two configuration options also affect cluster availability to some extent; for example, if a slave node fails and the number of available slave nodes is less than the configuration value, even if the master node of the cluster is normal, it will not be able to provide services to the terminals.

[0078] Based on this, the embodiments of this application provide a data processing method and apparatus, electronic device and medium based on a cluster network, which can better solve the problem of data inconsistency or data loss when processing data in a cluster network with a sentinel mechanism, and improve the accuracy of data processing in the cluster network.

[0079] The data processing method based on a cluster network provided in this application relates to the field of communication technology and can be applied to terminals, servers, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; the software can be an application that implements the data processing method based on a cluster network, but is not limited to the above forms.

[0080] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers (PCs), minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0081] Please see Figure 3 , Figure 3 This is an optional flowchart of a data processing method based on a cluster network provided in the embodiments of this application. Figure 3 The method described below may specifically include, but is not limited to, steps S310 to S350. Figure 3 These five steps will be explained in detail.

[0082] Step S310: When the sentinel cluster detects that the first master node is in a fault state at the first time point, it selects the second master node based on the first slave node of the third network partition in the cluster network.

[0083] Step S320: Other first slave nodes in the cluster network are designated as second slave nodes, and the second master node completes the master-slave switch at the second time point;

[0084] Step S330: When the first master node responds to the transaction request sent by the client at the third time point, it executes the transaction request based on the preset distributed lock.

[0085] Step S340: Store the distributed lock in a preset first log cache and store the operation statement corresponding to the transaction request in a preset second log cache;

[0086] Step S350: When the sentinel cluster detects that the first master node is in a normal operating state at the fourth time point, it updates the preset database of the second master node based on the first log cache and the second log cache.

[0087] It should be noted that the data processing method based on cluster network provided in this application can be applied to cluster network (such as Redis cluster). The cluster network includes a first master node, at least one first slave node and a sentinel cluster. The first master node and the first client are located in the first network partition of the cluster network and are communicatively connected. The sentinel cluster is located in the second network partition of the cluster network and is communicatively connected to the first client. The first slave node is located in either the first network partition or the second network partition, and the total number of nodes in the first network partition is less than or equal to half of the total number of nodes in the cluster network.

[0088] It should be noted that because there are many clients (which can be understood as app clients) and they are scattered in different regions, the clients mentioned in this application are not limited. Furthermore, if the sentinel cluster and the primary master node are in the same network partition, the primary master node will not be considered objectively offline. This is because the sentinel node determines whether the primary master node is offline by periodically sending PING heartbeat commands to the primary master node, thus preventing a split-brain phenomenon. A split-brain phenomenon refers to a situation where a single cluster splits into two or more clusters due to a brief network failure. If the sentinel node and the primary master node are in the same network partition, then other slave nodes cannot communicate with the sentinel node, and the sentinel node cannot elect a new master node from these slave nodes. Even in this situation, the system can still provide normal transaction requests and read operations; the only problem is that the service cluster becomes a single point of failure. Therefore, in this application, the primary master node and the sentinel cluster are located in different network partitions.

[0089] It should be noted that, as Figure 4As shown, the first master node and the first client are located in the first network partition of the cluster network. This is because the first master node providing the transaction request must be in the same network partition as the first client, while the sentinel node and other slave nodes must be in a different network partition. Thus, during the period when the first master node experiences a split-brain situation, the first client cannot communicate normally with the sentinel node. Consequently, the first client cannot know who the new master node is and will continue to access its default first master node, unable to connect to the new master node, thus creating the scenario problem that this application aims to solve.

[0090] It should be noted that, in combination Figure 4 Since the first slave node (e.g., slave node 1 and slave node 2) may be located in the same network partition as the old first master node, or it may be located in the same network partition as the new second master node, to better apply this application, it is necessary to ensure that the total number of nodes in the partition where the first master node is located does not exceed half of the total number of nodes in the cluster network. If it exceeds half, the network partition where the second master node is located will be unable to elect a new master node (because the principle of the sentinel mechanism for electing a master node is "majority").

[0091] In step S310 of some embodiments, the first master node refers to the currently selected master node. The running state refers to the communication status of the first master node. The fault state indicates an abnormal communication between the first master node and the sentinel cluster. The first network partition where the first master node is located is different from the third network partition, and the total number of nodes in the third network partition is greater than or equal to half the total number of nodes in the cluster network. The second network partition may be the same as or different from the third network partition. The third network partition is used to select a second master node from its first slave nodes when the first master node experiences a split-brain situation. The second master node refers to the master node re-determined by the sentinel cluster through an automatic failover method.

[0092] It should be noted that when the sentinel cluster detects that the first master node is in a fault state at the first point in time, it can be considered that the first master node is objectively offline at the first point in time, thus resulting in a split-brain situation. Furthermore, to avoid the problems caused by split-brain situations, the method provided in this application is adopted.

[0093] In some embodiments of this application, prior to step S310, the method provided in this application may further include, but is not limited to, the following steps:

[0094] When the first sentinel node in the sentinel cluster detects that the first master node is in a faulty state at the first point in time, it sends a master node fault vote request to the other sentinel nodes in the sentinel cluster.

[0095] The first sentinel node receives master node failure vote responses from other sentinel nodes. These master node failure vote responses are used to indicate the detection results of other sentinel nodes on the running status of the first master node at the first point in time.

[0096] When the number of failure vote responses from the master node indicating that the first master node is in a failure state exceeds a preset number, the first master node is determined to be in a failure state at the first point in time.

[0097] In the above embodiments, this application determines the operating status of the first master node based on the sentinel cluster. Specifically, by default, the sentinel nodes in the sentinel cluster periodically send PING commands to the first master node and the first slave node at a preset frequency (such as once per second or twice per second) to detect the online status of these nodes. Figure 5A As shown, when the first sentinel node in the sentinel cluster (sentinel node 1 in the figure) detects that the first master node (master node in the figure) is in a fault state at the first point in time, that is, there is no heartbeat response to the PING command, the first master node is marked as subjectively offline (if the Redis master node is really down, it needs to start master node switching, including re-election and data synchronization, which consumes a lot of resources, so the judgment of master node offline should be treated with caution. It is possible that the network failure of the sentinel node caused the heartbeat detection of the Redis master node to fail, so the sentinel cluster needs to be introduced to make decisions). Then, as Figure 5B As shown, a sentinel cluster is introduced to jointly determine whether the first sentinel node (as shown in sentinel node 1) is offline. The first sentinel node can send a master node failure vote request to other sentinel nodes in the cluster. These other sentinel nodes will then send a PING command to the first master node (as shown in the master node). The first sentinel node receives the master node failure vote responses from the other sentinel nodes, and these responses indicate the results of the other sentinel nodes' detection of the first master node's operational status at the first time point. If more than half of the sentinel nodes determine that the first master node is subjectively offline (i.e., the number of master node failure vote responses indicating that the first master node's operational status is faulty exceeds a preset number (this preset number can be a preset value or half the number of sentinel nodes in the sentinel cluster, without limitation), then the first master node is marked as objectively offline, meaning its operational status at the first time point is determined to be faulty.

[0098] In other words, sentinel nodes can periodically monitor whether the current master node is malfunctioning and reach a consensus with other sentinel nodes that the master node is indeed malfunctioning through negotiation. Thus, it is not the master node that is malfunctioning, but rather the sentinel node that discovers the malfunction. This situation often occurs when the sentinel node's network is isolated.

[0099] Please see Figure 6 , Figure 6 This is a flowchart illustrating step S310 as provided in an embodiment of this application. In some embodiments of this application, step S310 specifically includes, but is not limited to, steps S610 to S620.

[0100] Step S610: Determine the sentinel master node from the sentinel nodes of the sentinel cluster based on a preset consensus algorithm;

[0101] Step S620: Based on the sentinel master node, select the first slave node of the third network partition in the cluster network to determine the second master node.

[0102] In steps S610 and S620 of some embodiments, when it is determined that the first master node is in a faulty state, i.e., marked as objectively offline and unable to function normally, the sentinel cluster can begin automatic failover operations. Specifically, when the first master node fails, the corresponding first slave node's synchronization connection is interrupted, and master-slave replication stops. Further, sentinel nodes can use a preset consensus algorithm (such as the Raft algorithm) to elect a leader as the sentinel master node. Further, this sentinel master node is responsible for subsequent failover work. Figure 7 As shown, the failover process can specifically include: the sentinel master node selects a first slave node from the first slave node of the third network partition in the cluster network as the second master node (slave node 3 in the figure). The sentinel node can send the connection information of the new second master node to other slave nodes, allowing the slave nodes to execute master-slave replication commands, establish connections with the new second master node, and replicate all data from the new second master node. Additionally, the sentinel node will notify client programs to migrate to the new master node, allowing clients to find the new master node (slave node 3 in the figure) for writing.

[0103] It should be noted that during initialization, the client can obtain the address of the current Redis service's master node by connecting to the sentinel node. However, since the first client and the first master node are in the same network partition, and the sentinel node cannot connect to the first client when the first master node fails, the first client will not know the address of the new master node.

[0104] Please see Figure 8 , Figure 8 This is a flowchart illustrating step S620 as provided in an embodiment of this application. In some embodiments of this application, step S620 specifically includes, but is not limited to, steps S810 to S820.

[0105] Step S810: Obtain the node running status and node attribute parameters of each first slave node in the third network partition;

[0106] Step S820: Based on the node running status and node attribute parameters, the first slave node in the third network partition is filtered to determine the second master node.

[0107] In step S810 of some embodiments, the node running status is used to indicate whether the corresponding first slave node is in a fault state or a normal state. Node attribute parameters are used to indicate parameters related to the attributes of the first slave node, and the node attribute parameters may include slave-priority, slave-node replication offset (used to indicate the progress of the corresponding slave node in replicating data; the larger the offset, the newer the slave node's data is and the closer it is to the original master node's data state), node running identifier (i.e., runid, a random identifier automatically generated when each Redis node starts, and each Redis node has a different runid), etc.

[0108] In step S820 of some embodiments, because the sentinel cluster excludes nodes that are already considered faulty, these slave nodes will not be considered as candidates for the new master node. Therefore, the sentinel master node can first filter the first slave nodes in the third network partition based on the node running status, filtering out first slave nodes in a faulty state, and then filter the remaining first slave nodes based on node attribute parameters. Specifically, the first slave node with the highest slave node priority setting can be selected as the second master node. If there are multiple first slave nodes with the same slave node priority, the sentinel master node can select the first slave node with the largest slave node replication offset as the second master node. If there are at least two first slave nodes with the same slave node replication offset, the sentinel master node can select the first slave node with the smallest or largest runid as the new second master node.

[0109] It should be noted that this application also allows for flexible selection of node attribute parameters, thereby enabling flexible adjustment of the method for determining the second primary node without limitation.

[0110] It should be noted that, to better understand the data processing behavior of different nodes in a split-brain scenario within the cluster, and why a split-brain in a Redis cluster leads to data loss, please refer to [link / reference]. Figure 9 , Figure 9 The diagram shows a timeline of failover after a split-brain scenario. Node 1 is the first master node, and node 2 is the newly determined second master node. After the second master node is determined, it sends a full synchronization (or master-slave replication) command to all slave nodes, causing them to re-synchronize. Full synchronization first clears the data on the nodes. Therefore, the time between the old master node being determined to be objectively offline and the old master node resuming communication with the cluster where the new master node resides (i.e.,...) Figure 9Data executed during the period [t3, t8] will be lost, which is the specific reason for the data loss. During the period [t5, t7], because the client and the sentinel node cannot communicate normally, the client can only access its default first master node, meaning the first master node can receive the client's transaction requests and write data.

[0111] In addition, Figure 9 During the time period [t3, t8], the old master node executes client transaction requests (i.e., client requests to add, delete, or modify database data). The consequence is that it first modifies the cached data in the Redis service, and then Redis persists the data, writing it to the database (e.g., MySQL). This is because clients sharing the same network partition as the first master node can only connect to node 1 and not node 2 during the time period [t5, t7]. Furthermore, as discussed in the previous analysis of data loss, when the new master node performs a full synchronization with all slave nodes (at which point the old master node has been demoted to a slave), the data executed by the old master node during the time period [t3, t8] will be cleared. However, the data already persisted to the database (e.g., MySQL) during this time period will be retained. Therefore, when the new master node persists the data from the Redis cache to the database (e.g., MySQL), data inconsistency will occur. Thus, if clients in other regions initially communicated normally with the cluster where node 2 is located (i.e., the cluster where the sentinel is located), then when node 2 is elected as the new master node, the sentinel will immediately notify these clients that node 2 is the new master node. During the period [t5, t7], these clients will not connect to node 1 but will connect to node 2.

[0112] Based on this, the purpose of the solution in this application is to save the data within the time period [t3, t8] and lock the data executed within the time period [t3, t8] to prevent it from being tampered with by the newly elected master node of other network partitions. After the communication between network partitions returns to normal, the lock is released and the new master node is allowed to update the data within the time period [t3, t8] first, and then all slave nodes are allowed to perform a full synchronization operation.

[0113] In step S320 of some embodiments, after determining the second master node, other first slave nodes in the cluster network can be designated as second slave nodes, and the second master node completes the master-slave switchover at a second time point. The second time point characterizes the time when the second master node completes the master-slave switchover, and the second time point is later than the first time point. Figure 9 As shown, the first time point can be t1, and the second time point can be t5.

[0114] It should be noted that in some other embodiments, this application may also assign a separate network partition to the Redis master node, and the remaining Redis slave nodes and the Sentinel cluster to form another network partition, and the data between the two network partitions cannot be communicated.

[0115] In step S330 of some embodiments, the third time point is any time point within the period from when the sentinel cluster determines that the first master node is objectively offline to when the first master node re-establishes a connection with the sentinel cluster. For example... Figure 9 As shown, the third time point can be any time point within the time interval [t3, t8]. At this time, while the first master node is still responding to the transaction request sent by the client at the third time point, it acquires a distributed lock for each transaction request it receives, and does not release the lock even after the transaction request is completed. The distributed lock is used to lock the data associated with the operation statement corresponding to the transaction request. The third time point is later than the first time point.

[0116] It's important to note that the distributed lock requested by the primary master node can lock the data after the execution of the operation statement corresponding to the transaction request, preventing other nodes besides the primary master node from performing add, delete, or modify operations on it. This distributed lock can be provided by Redis, and in a Redis cluster, regardless of which Redis node generates the distributed lock, its scope applies to the entire cluster.

[0117] In step S340 of some embodiments, the first master node can further store the distributed lock in a preset first log cache (e.g., log cache S1) and store the operation statement corresponding to the transaction request in a preset second log cache (e.g., log cache F1). That is, the first master node can write the transaction request into log cache F1 in the form of an SQL statement. Log cache F1 stores SQL transaction statements (i.e., transaction operations), while log cache S1 stores the distributed lock for each statement in F1. When the first master node performs a transaction operation on the database, such as the operation statement corresponding to the transaction request being "update table1 set name="zhangsan"where id=7", the row data "id=7" in the corresponding table table1 is locked, preventing other nodes besides the first master node from performing add, delete, or modify operations on it. Only after the first master node sends log cache F1 to the new Redis master node and releases the distributed lock can the new Redis master node update its own database with the transaction operation of F1 (e.g., update table1 set name="zhangsan"where id=7). Furthermore, the new Redis master node no longer needs to save the log cache after updating the F1 transaction operations.

[0118] In step S350 of some embodiments, when the sentinel cluster detects that the first master node is in a normal operating state at the fourth time point, that is, when the sentinel node and the first master node resume communication, the second master node can also communicate normally with the first master node, thereby allowing the second master node to update data in its preset database based on the first log cache and the second log cache. Here, "normal state" indicates that communication between the first master node and the sentinel cluster is normal, the fourth time point is later than the third time point, and the fourth time point is later than the second time point. For example... Figure 9 As shown, the fourth time point can be t8.

[0119] In some embodiments, prior to step S350, the method provided in this application may further include:

[0120] Each sentinel node in the sentinel cluster sends a PING command to the first master node based on a preset frequency of data, and determines multiple response data returned by the master node according to the PING command within a preset time period;

[0121] The operating status of the first master node is detected based on multiple response data.

[0122] When the second sentinel node in the sentinel cluster detects that the first master node is in a normal operating state at the fourth time point, it determines that the first master node is in a normal operating state at the first time point.

[0123] In the above embodiments, the first sentinel node and the second sentinel node refer to any node in the sentinel cluster, and they can be the same or different. Each sentinel node in the sentinel cluster periodically sends a PING command to the first master node at a preset frequency (such as once or twice per second). When a sentinel node detects that the first master node is online, it can bring the first master node back into the Redis cluster as a slave node.

[0124] Please see Figure 10 , Figure 10 This is a flowchart illustrating step S350 provided in an embodiment of this application. In some embodiments of this application, step S350 specifically includes, but is not limited to, steps S1010 to S1030.

[0125] Step S1010: Release the distributed lock in the first log cache through the first master node, and send the second log cache to the second master node;

[0126] Step S1020: Execute the operation statement corresponding to the transaction request through the second master node to obtain the first transaction operation data at the third time point;

[0127] Step S1030: Update the preset database based on the data from the first transaction operation.

[0128] In steps S1010 to S1030 of some embodiments, after determining that the first master node has returned to the Redis cluster as a slave node, the first master node can release the distributed lock in the first log cache and send the second log cache to the second master node. Further, the second master node can execute the operation statements corresponding to the transaction requests in the second log cache to obtain the first transaction operation data at the third time point, and update this first transaction operation data to the second master node's preset database.

[0129] In some embodiments, after step S350, the method provided in this application embodiment may further include:

[0130] The first master node is used as the second slave node, and a full synchronization signal is sent to all second slave nodes based on the updated preset database.

[0131] Each second slave node responds to a full synchronization signal and updates the data stored on each second slave node based on a preset database.

[0132] Understandably, after updating the first transaction data to the preset database of the second master node, the sentinel node can notify all current slave nodes to copy all data of the second master node in the preset database.

[0133] In one specific embodiment, such as Figure 11 As shown, Figure 11 This diagram illustrates a complete logical representation of the data processing method based on a cluster network provided in this application. In the process of the Redis cluster (i.e., the cluster network of this application) providing services to clients, both the Redis nodes (i.e., the master and slave nodes included in this application) and the sentinel nodes monitor the network online status of the nodes. The data processing logic is described below for both the Redis node side and the sentinel node side.

[0134] For the Redis node (i.e., the first master node in this application, hereinafter referred to as Redis master node R1), the specific steps may include the following: First, the heartbeats of both Redis master node R1 and the sentinel cluster stop, indicating a Redis split-brain event. However, Redis master node R1 can still communicate with clients at this time. Then, Redis master node R1 continuously responds to client transaction requests. Upon receiving a transaction request, Redis master node R1 acquires a distributed lock, does not release the lock after executing the transaction request, and stores the distributed lock in log cache S1. The transaction request is then written to log cache F1 in the form of an SQL statement. Further, while Redis master node R1 is continuously responding to client transaction requests, the sentinel node periodically sends a PING command to Redis master node R1 once per second to determine if it is online. Redis master node R1 simultaneously listens for heartbeat changes with the sentinel node, waiting to re-establish contact.

[0135] Specifically, for the sentinel node, the process can include the following steps: First, when the first sentinel node detects that the heartbeats of both the first and second sentinel nodes have stopped, the first sentinel node negotiates with other sentinel nodes to reach a consensus that the Redis master node has failed. That is, the sentinel cluster determines that the Redis master node R1 is objectively offline. Then, the sentinel cluster elects a new sentinel master node using the Raft algorithm to handle the failover of the Redis cluster. Next, the sentinel master node elects a slave node from the Redis network partition with a majority of nodes as the new Redis master node R2 (i.e., the second master node in this application, hereinafter referred to as Redis master node R2). Afterwards, the sentinel node notifies all slave nodes to replicate all data from the new Redis master node R2. Finally, upon notification from the sentinel node, the new Redis master node R2 takes over the position of the old Redis master node R1 and continues to respond to client transaction requests.

[0136] Furthermore, the sentinel master node completed the failover of the Redis cluster, and then waited for network communication between the two network partitions to be restored. Specifically, this includes: First, step 1: The sentinel node periodically sends a PING command to the old Redis master node R1 once per second. When the sentinel node detects that the old Redis master node R1 is online, the old Redis master node R1 will return to the Redis cluster as a slave node (the old Redis master node R1 is renamed slave node R1). Then, slave node R1 releases the distributed lock on the log cache S1 and sends the log cache F1 to the new Redis master node R2. The new Redis master node R2 updates the transaction operations of F1 to its local service (i.e., Redis master node R2). Afterwards, the sentinel node notifies all slave nodes to replicate all data from the new Redis master node R2.

[0137] This application provides a data processing method based on a cluster network, which can effectively improve the problems caused by cluster split-brain. The beneficial effects of the improvement can specifically include: (1) Improved data consistency: By using distributed locks, transaction requests executed on the old master node after a split-brain event can only be operated by one node, avoiding data inconsistency caused by multiple nodes operating on the same data at the same time. (2) Avoided data loss: Transaction requests executed on the old master node after a split-brain event are saved in the log cache. After the new master node is elected, the old master node will synchronize this part of the cached log to the new master node. (3) Improved system availability: With the improvement of this application, even if a split-brain event occurs, the ability of Redis to provide services is not affected from the outside. (4) Improved system robustness: In the original solution to cluster split-brain, if the slave node goes down, it may affect the availability of the master node to provide services to clients. However, the solution of this application can avoid the negative impact of the slave node going down on the master node. Therefore, when a split-brain scenario occurs, a distributed lock is added to each transaction operation to ensure that the same data can only be operated on by one master node during the cluster split, preventing different master nodes from potentially writing the same data differently and causing data inconsistency. Furthermore, when a split-brain scenario occurs, the transaction operation logs executed by the old master node are saved, allowing the new master node to synchronously update the data related to the transactions executed by the old master node, preventing data loss. Therefore, this application can better solve the problems of data inconsistency or data loss when processing data in cluster networks with a sentinel mechanism, improving the accuracy of data processing in cluster networks.

[0138] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of a data processing device based on a cluster network provided in an embodiment of this application. The device specifically includes:

[0139] The master node determination module 1210 is used to determine the second master node when the sentinel cluster detects that the first master node is in a fault state at the first time point. The second master node is determined by selecting the first slave node of the third network partition in the cluster network based on the sentinel cluster. The fault state is used to indicate that the communication between the first master node and the sentinel cluster is abnormal, the total number of nodes in the third network partition is greater than or equal to half of the total number of nodes in the cluster network, and the second network partition is the same as or different from the third network partition.

[0140] The switching module 1220 is used to designate other first slave nodes in the cluster network as second slave nodes and complete the master-slave switch of the second master node at a second time point; wherein the second time point is later than the first time point.

[0141] The request execution module 1230 is used to execute the transaction request based on a preset distributed lock when the first master node responds to the transaction request sent by the client at a third time point; wherein, the distributed lock is used to lock the data associated with the operation statement corresponding to the transaction request, and the third time point is later than the first time point;

[0142] Storage module 1240 is used to store the distributed lock in a preset first log cache and store the operation statement corresponding to the transaction request in a preset second log cache;

[0143] The update module 1250 is used to update the preset database of the second master node based on the first log cache and the second log cache when the sentinel cluster detects that the first master node is in a normal operating state at the fourth time point; wherein, the normal state is used to indicate that the communication between the first master node and the sentinel cluster is normal, the fourth time point is later than the third time point, and the fourth time point is later than the second time point.

[0144] It should be noted that the data processing device based on the cluster network in this application embodiment is used to implement the data processing method based on the cluster network in the above embodiment. The data processing device based on the cluster network in this application embodiment corresponds to the aforementioned data processing method based on the cluster network. For the specific processing process, please refer to the aforementioned data processing method based on the cluster network, which will not be repeated here.

[0145] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the data processing method based on a cluster network according to this application. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0146] Please see Figure 13 , Figure 13This illustration shows the hardware structure of an electronic device according to another embodiment, the electronic device comprising:

[0147] The processor 1310 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0148] The memory 1320 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1320 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1320 and is called and executed by the processor 1310 using the data processing method based on a cluster network according to the embodiments of this application.

[0149] The input / output interface 1330 is used to implement information input and output;

[0150] The communication interface 1340 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0151] Bus 1350 transmits information between various components of the device (e.g., processor 1310, memory 1320, input / output interface 1330, and communication interface 1340);

[0152] The processor 1310, memory 1320, input / output interface 1330 and communication interface 1340 are connected to each other within the device via bus 1350.

[0153] This application also provides a computer-readable storage medium storing a computer program for causing a computer to execute the data processing method based on a cluster network described above.

[0154] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0155] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0156] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0157] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0158] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0159] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0160] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0162] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0164] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A data processing method based on a cluster network, characterized by, The method is applied to a cluster network, the cluster network comprises a first master node, at least one first slave node and a sentinel cluster, the first master node is located in a first network partition of the cluster network and communicates with a first client, the sentinel cluster is located in a second network partition of the cluster network and communicates with the first client, and the first slave node is located in the first network partition or the second network partition, and the total number of nodes contained in the first network partition is less than or equal to half of the total number of nodes contained in the cluster network, and the method comprises the following steps: When the sentinel cluster detects that the running state of the first master node at a first time point is a failure state, a second master node is determined based on node selection of the sentinel cluster to the first slave node in a third network partition of the cluster network, wherein the failure state indicates abnormal communication between the first master node and the sentinel cluster, the total number of nodes contained in the third network partition is greater than or equal to half of the total number of nodes contained in the cluster network, and the second network partition is the same as or different from the third network partition; Other first slave nodes in the cluster network are taken as second slave nodes, and the second master node completes master-slave switching at a second time point, wherein the second time point is later than the first time point; When the first master node responds to a transaction request sent by the client at a third time point, the transaction request is executed based on a preset distributed lock, wherein the distributed lock is used to lock data associated with an operation statement corresponding to the transaction request, and the third time point is later than the first time point; The distributed lock is stored in a preset first log cache, and the operation statement corresponding to the transaction request is stored in a preset second log cache; When the sentinel cluster detects that the running state of the first master node at a fourth time point is a normal state, the preset database of the second master node is updated based on the first log cache and the second log cache, wherein the normal state indicates normal communication between the first master node and the sentinel cluster, the fourth time point is later than the third time point, and the fourth time point is later than the second time point.

2. The method of claim 1, wherein, The method further comprises the following steps: The first master node releases the distributed lock in the first log cache, and the second log cache is sent to the second master node; The second master node executes the operation statement corresponding to the transaction request to obtain first transaction operation data at the third time point; The preset database is updated based on the first transaction operation data.

3. The method of claim 2, wherein, The method further comprises the following steps: The first master node is taken as the second slave node, and a full-amount synchronization signal is sent to all the second slave nodes based on the updated preset database. The second slave node responds to the full quantity synchronization signal, and updates the data stored in each second slave node based on the preset database.

4. The method of claim 1, wherein, Before the method further comprises: when the sentinel cluster detects that the running state of the first master node at a first time point is a fault state, based on the sentinel cluster, the first slave node in the third network partition in the cluster network is selected as a node, and a second master node is determined. When the first sentinel node in the sentinel cluster detects that the running state of the first master node at the first time point is a fault state, a master node fault voting request is sent to other sentinel nodes in the sentinel cluster. The first sentinel node receives the master node fault voting response returned by the other sentinel nodes, and the master node fault voting response is used to indicate the detection result of the other sentinel nodes on the running state of the first master node at the first time point. When the number of master node fault voting responses indicating that the running state of the first master node is the fault state is greater than a preset number, it is determined that the running state of the first master node at the first time point is a fault state.

5. The method of claim 1, wherein, The method further comprises: Based on a preset consensus algorithm, a sentinel master node is determined from the sentinel nodes of the sentinel cluster; Based on the sentinel master node, the first slave node in the third network partition in the cluster network is selected as a node, and the second master node is determined.

6. The method of claim 5, wherein, The method further comprises: Obtain the node running state and node attribute parameters of each first slave node in the third network partition; Based on the node running state and the node attribute parameters, the first slave node in the third network partition is selected as a node, and the second master node is determined.

7. The method according to any one of claims 1 to 6, characterized in that, Before the method further comprises: Each sentinel node of the sentinel cluster sends a PING command to the first master node based on preset frequency data, and determines a plurality of response data returned by the master node according to the PING command within a preset time period; According to the plurality of response data, the running state of the first master node is detected; When the second sentinel node in the sentinel cluster detects that the running state of the first master node at the fourth time point is a normal state, it is determined that the running state of the first master node at the first time point is a normal state.

8. A data processing apparatus based on a cluster network, characterized by The application is applied to a cluster network, the cluster network comprises a first master node, at least one first slave node and a sentinel cluster, the first master node is located in a first network partition of the cluster network with a first client, the first master node is in communication connection with the first client, the sentinel cluster is located in a second network partition of the cluster network, the sentinel cluster is in communication connection with the first client, the first slave node is located in the first network partition or the second network partition, and the total number of nodes contained in the first network partition is less than or equal to half of the total number of nodes contained in the cluster network, and the device comprises: The master node determination module is used for determining a second master node based on the sentinel cluster selecting the first slave node in a third network partition of the cluster network when the sentinel cluster detects that the running state of the first master node at a first time point is a fault state; wherein the fault state is used to indicate that the communication between the first master node and the sentinel cluster is abnormal, the total number of nodes contained in the third network partition is greater than or equal to half of the total number of nodes contained in the cluster network, and the second network partition is the same as or different from the third network partition; The switching module is used for switching other first slave nodes in the cluster network as second slave nodes, and completing master-slave switching of the second master node at a second time point; wherein the second time point is later than the first time point; The request execution module is used for executing the transaction request based on a preset distributed lock when the first master node responds to the transaction request sent by the client at a third time point; wherein the distributed lock is used to lock the data associated with the operation statement corresponding to the transaction request, and the third time point is later than the first time point; The storage module is used for storing the distributed lock into a preset first log cache, and storing the operation statement corresponding to the transaction request into a preset second log cache; The update module is used for performing data update on a preset database of the second master node based on the first log cache and the second log cache when the sentinel cluster detects that the running state of the first master node at a fourth time point is a normal state; wherein the normal state is used to indicate that the communication between the first master node and the sentinel cluster is normal, the fourth time point is later than the third time point, and the fourth time point is later than the second time point.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images