Distributed service system, instance splitting method, device, medium, and program product

Abstract

Embodiments of the present disclosure provide a distributed service system, an instance splitting method, a device, a medium, and a program product. In the embodiments of the present disclosure, in response to an instance splitting event, a service instance is scaled out, and no new data is written into the scaled-out service instance by means of a seal instruction (a write-freeze instruction). Subsequently, the scaled-out service instance is split into a plurality of service instances, thereby achieving service instance splitting. In the scaling-out stage, a service may still be provided by an original service node of the service instance, so that data synchronization can be completed while service continuity is ensured. By preventing new data from being written into the scaled-out service instance, the plurality of service instances obtained after splitting commence from an identical state, thereby ensuring data consistency across the service instances before and after splitting.

Description

Distributed service system, instance splitting method, device, medium and program product

[0001] The present disclosure claims priority to Chinese Patent Application No. 202411268487.1, filed on September 10, 2024, with the Chinese Patent Office, entitled “Distributed service system, instance splitting method, device, medium and program product”, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of cloud services, and in particular to a distributed service system, an instance splitting method, a device, a medium and a program product. BACKGROUND

[0003] A distributed service system refers to a system composed of multiple services that cooperate with each other. These services are usually distributed on different hosts or servers and communicate through a network. In order to improve the reliability and disaster recovery capability of the distributed service system, multiple service nodes of the same service instance can be deployed to back up each other. Once a certain service node is abnormal, other service nodes can continue to provide services.

[0004] Multiple service nodes of the same service instance can provide consistent coordination services (such as distributed lock services, etc.). A master node can be selected from multiple service nodes based on a consensus protocol to provide services, and data is synchronized to other service nodes to maintain the state consistency of multiple service nodes. In actual applications, the demand for service instance splitting is real. Therefore, there is an urgent need to provide a service instance splitting scheme to meet actual needs. SUMMARY

[0005] Aspects of the present disclosure provide a distributed service system, an instance splitting method, a device, a medium and a program product to implement splitting of a service instance.

[0006] Embodiments of the present disclosure provide a distributed service system, comprising: a service cluster corresponding to a first service instance; the first service instance is obtained by expanding a second service instance in response to an instance splitting event, and the service cluster comprises multiple service nodes independently providing consistent coordination services; and the multiple service nodes complete data synchronization.

[0007] The master node of the plurality of service nodes is configured to: send a write-prohibition instruction to a slave node of the plurality of service nodes to indicate that the service cluster no longer receives write requests; and send a split instruction to the slave node in a case where a number of first target service nodes corresponding to a plurality of third service instances into which the first service instance is to be split is greater than a corresponding set number, the first target service nodes being service nodes of the plurality of service nodes that acknowledge the write-prohibition instruction; wherein the set number corresponding to any third service instance is greater than or equal to half of a total number of service nodes corresponding to the any third service instance and less than the total number.

[0008] The master node and the slave node that receives the split instruction are configured to split the first service instance into the plurality of third service instances in response to the split instruction.

[0009] Embodiments of the present disclosure further provide an example split method applicable to any service node of a plurality of service nodes corresponding to a first service instance, the first service instance being obtained by expanding a second service instance in response to an instance split event, the plurality of service nodes independently providing consistent coordination services, and data synchronization being completed.

[0010] The method comprises:

[0011] In a case of being the master node, the method further comprises: sending a write-prohibition instruction to a slave node of the plurality of service nodes to make the service cluster no longer receive write requests.

[0012] In a case where a number of first target service nodes corresponding to a plurality of third service instances into which the first service instance is to be split is greater than a corresponding set number, the method further comprises: sending a split instruction to the slave node, so that the master node and the slave node that receives the split instruction split the first service instance into the plurality of third service instances in response to the split instruction.

[0013] The first target service nodes are service nodes of the plurality of service nodes that acknowledge the write-prohibition instruction; the set number corresponding to any third service instance is greater than or equal to half of a total number of service nodes corresponding to the any third service instance and less than the total number.

[0014] Embodiments of the present disclosure further provide an electronic device comprising a memory, a processor and a communication component; wherein the memory is configured to store a computer program;

[0015] The processor is coupled to the memory and the communication component and is configured to execute the computer program to perform steps in the aforementioned example split method.

[0016] The embodiments of the present disclosure further provide a computer readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps in the foregoing example splitting method.

[0017] The embodiments of the present disclosure further provide a computer program product, comprising a computer program, which, when executed by one or more processors, causes the one or more processors to perform the steps in the foregoing example splitting method.

[0018] In the embodiments of the present disclosure, in response to a splitting event, the service instance is scaled out, and the scaled-out service instance is prevented from having new data written in by a write-prohibit instruction, and then the scaled-out service instance is split into multiple service instances, thereby realizing the splitting of the service instance. In the scaling-out stage, the original service nodes of the service instance can still provide services, and the data synchronization can be completed while the service continuity is ensured. By preventing the scaled-out service instance from having new data written in, the multiple service instances after splitting can start splitting from the same state, and the data consistency of the service instances before and after splitting can be ensured. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are included to provide a further understanding of the present disclosure and constitute a part of the present disclosure, illustrate the exemplary embodiments of the present disclosure and serve to explain the present disclosure together with the descriptions. In the drawings:

[0020] FIG. 1 is a schematic diagram of a partition splitting process provided by a conventional scheme;

[0021] FIG. 2 is a schematic diagram of the structure of a distributed service system provided by an embodiment of the present disclosure;

[0022] FIGS. 3 and 4 are schematic diagrams of an instance splitting process provided by an embodiment of the present disclosure;

[0023] FIG. 5 is a schematic diagram of a process in which some data synchronization modes cause data loss provided by an embodiment of the present disclosure;

[0024] FIG. 6 is a schematic diagram of a process of changing members by a joint consensus mechanism provided by an embodiment of the present disclosure;

[0025] FIG. 7 is a comparison diagram of the format of a conventional voting request and the format of a voting request provided by an embodiment of the present disclosure;

[0026] FIG. 8 is a schematic diagram of the space-time changes of a system in an instance splitting process provided by an embodiment of the present disclosure;

[0027] FIG. 9 is a flowchart of an instance splitting method provided by an embodiment of the present disclosure;

[0028] FIG. 10 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] To make the objectives, technical solutions and advantages of the present disclosure clearer, the following will be combined with specific embodiments of the present disclosure and corresponding drawings to clearly and completely describe the technical solutions of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0030] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the related data need to comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation portal for the user to choose authorization or refusal.

[0031] The following will first explain and describe the terms or concepts involved in the present disclosure.

[0032] Consensus protocol: a series of protocols proposed to solve the data consistency problem in a distributed system, such as Two Phase Commitment Protocol (2PC), Three Phase Commitment Protocol (3PC), Paxos consensus protocol and Raft protocol, etc.

[0033] Replicated state machine: in a distributed system, there is a cluster composed of multiple servers. Each server runs a Deterministic Finite State Machine (DFSM), which means that for a given input, the state machine always produces the same output and state transition. The purpose is to let these state machines reach a consistent state so that it appears externally as if only one state machine is working. If it can be ensured that all state machines receive exactly the same sequence of commands and execute them in the same order, then these state machines will eventually reach the same final state. This means that no matter which state machine the client connects to, it will get the same result. Since each state machine runs the same logic independently, the system can tolerate the failure of part of the state machines without affecting the overall service quality.

[0034] In the context of replicated state machines, each state machine maintains a log in which all received commands are recorded. The order of the log is very important as it determines the order in which the state machine executes the commands. Consensus protocols can be used to ensure that the logs on all state machines have a consistent sequence. Through consensus protocols, it can be decided which commands should be added to the log and their order.

[0035] Suppose a replicated state machine system contains three nodes, A, B, and C. These nodes run copies of the same state machine. A client sends a command cmd1 to node A and then sends another command cmd2. To ensure that all nodes execute these two commands in the same order, the system employs a consensus protocol to decide the contents of the log. Specifically, node A receives cmd1 and attempts to write it to the log. Through the consensus protocol, node A informs nodes B and C that it wants to write cmd1 to the log. All nodes (A, B, and C) decide whether to agree to add cmd1 to the log according to the rules of the consensus protocol. Once cmd1 is accepted by a majority of nodes, it is written to the logs of all nodes. Next, node A sends cmd2 to B and C and repeats the above process. Through such a mechanism, the logs on all nodes will contain the same sequence of commands cmd1 and cmd2, ensuring that all state machine copies eventually reach the same final state.

[0036] Shared-nothing architecture: Shared-nothing architecture is a distributed system architecture in which each physical machine in the system works independently and does not share resources such as memory, Central Processing Unit (CPU), or storage resources such as disks with other physical machines, so that there is no single point of contention in the system.

[0037] In a distributed service system, how to achieve correct and efficient state management is the biggest challenge in system architecture design. Generally, distributed systems of the computing type will transfer state management to rely on certain storage products (such as databases, object storage, file storage, etc.), so the problem becomes how a distributed storage system can achieve high reliability and high performance in managing state.

[0038] Distributed storage systems generally use a partition-based scheduling model. Distributed storage systems are divided into a control plane and a data plane, where the data plane divides the user's storage space into several partitions according to certain rules, and at runtime a partition is assigned to a server or a group of servers for service, and a server or a group of servers can load multiple partitions simultaneously. Partition-based distributed storage systems themselves also have Shared-storage architecture and Shared-nothing architecture.

[0039] In the shared storage architecture, the partition does not use the native file system to store the persistent data, but stores the data in a distributed file system at the bottom. The management plane is responsible for partition scheduling, on the one hand, load balancing scheduling, and on the other hand, when a server fails, the management node is responsible for re-scheduling the corresponding partition and quickly migrating to other servers running normally.

[0040] In such a system architecture, the management plane supports state management. In order to have high availability (HA), the management plane usually needs to rely on a replicated state machine system based on Shared-nothing architecture, or the management plane itself is a replicated state machine-based system. In a replicated state machine-based system, a management instance includes multiple independent management nodes, and the multiple management nodes are in a Shared-nothing architecture and do not share resources.

[0041] The multiple management nodes can provide consistent coordination services (such as distributed lock services), a master node can be selected from the multiple management nodes based on a consensus protocol to provide management services, and data is synchronized to other management nodes to maintain the state consistency of the multiple management nodes.

[0042] In addition to the management plane of the distributed storage system described above, the demand for instance splitting exists in any replicated state machine system based on the Shared-nothing architecture, and the service has such a strong demand. The demand for instance splitting includes but is not limited to: (1) insufficient service capacity: as the usage of some tenants grows rapidly, the original system configuration may not be sufficient to support their demand. In this case, these high-load tenants are split from the original instance to create a new instance to serve them exclusively, which can effectively improve the service quality and avoid affecting other tenants; (2) inconsistent disaster recovery needs: different tenants may have different disaster recovery requirements. For example, some high-criticality tenants may require higher-level disaster recovery measures, such as cross-AZ (Availability Zone) redundant deployment, etc. If all tenants are deployed on the same instance, it is difficult to meet such differentiated disaster recovery needs; (3) control the blast radius: in order to reduce the impact range of a single point of failure, when the number of tenants served by an instance is too large, the risk can be dispersed by splitting the instance. In this way, even if an instance fails, the number of affected tenants will be reduced. The blast radius refers to the impact range of an instance failure, which can be understood as the number of affected tenants.

[0043] In the Shared-nothing architecture, the same solution as Shared-storage can also be used for partition storage in the data plane. Each partition is a separate consensus protocol group (such as Raft-Group). In order to allow a single node to be responsible for multiple Raft-Group replicas, a Multi-Raft solution can be used. Based on the Multi-Raft partition system, there is a strong demand for partition splitting and merging.

[0044] As shown in FIG. 1, based on Multi-Raft, partition splitting can be implemented. That is, the data managed in the entire system is sliced in a certain way, and each slice of data has its own replica. The data between these replicas uses the Raft protocol to ensure data consistency. Globally, there are multiple Raft-Groups in the entire system, and any node can be responsible for multiple Raft-Group replicas. As shown in FIG. 1, assume that the data range of partition A (corresponding to a Raft-Group) is [1, 100), which needs to be split into two partitions, one of which has a data range of [1, 50), and the other has a data range of [50, 100). Based on such Multi-Raft, the main steps to implement instance splitting are as follows: (1) first apply to the management component to create a service data range for new partition B; (2) modify the data range boundary of partition A to [1, 50); (3) partition B starts to enable formal service data range [50, 100) request access. Here, the partition splitting process is briefly described. This partition splitting method is a kind of replicated state machine instance splitting mechanism similar to Shared-storage. The main reasons are:

[0045] a) During the splitting process, the original partition A corresponding to the Raft-Group continues to work by modifying the data boundary, and the newly generated partition B is on the same group of nodes as partition A. After the splitting is completed, the management component decides whether the newly generated partition B should be migrated to other nodes according to the global load situation. Before partition B is migrated to other nodes, it shares resources with partition A on the same group of nodes.

[0046] b) The foregoing partition splitting solution is based on Multi-Raft, so the new and old partitions during the splitting process have the advantages of data consistency and service availability under the Shared-storage architecture by sharing a persistent data.

[0047] Figure 1 shows the partition splitting process of the data plane under the Shared-nothing architecture, but the management plane under the Shared-nothing architecture is also based on the Shared-nothing architecture replication state machine system, which has the same management instance splitting requirements as the Shared-storage architecture management plane. However, the partition splitting scheme of the data plane under the Shared-storage architecture shown above does not apply to the management instance splitting of the management plane. This is mainly because the management plane is based on a Shared-nothing architecture replication state machine system, which does not have the concept of Raft-Group. Therefore, how to realize the splitting of service instances (such as the aforementioned management instances) of the Shared-nothing architecture replication state machine system becomes a technical problem to be solved.

[0048] In addition to the aforementioned instance splitting requirements of the management plane of the distributed storage system, any distributed system that adopts a Shared-nothing architecture replication state machine system has service instance splitting requirements. For example, for a distributed lock service system, the service instance is a distributed lock instance, which provides distributed lock services in the form of service pooling, and a same distributed lock instance serves multiple application clusters. Similarly, the service capacity problem, disaster recovery demand problem, and explosion radius problem described above will be encountered, and there is also a requirement for instance splitting.

[0049] In order to realize service instance splitting, in some embodiments of the present disclosure, in response to an instance splitting event, the service instance is expanded, and the expanded service instance is no longer allowed to have new data written into it through a write-inhibit instruction. Then the expanded service instance is split into multiple service instances, realizing the splitting of the service instance. In the expansion stage, the original service nodes of the service instance can still provide services, and data synchronization can be completed while ensuring service continuity. By making the expanded service instance no longer have new data written into it, the multiple service instances after splitting start from the same state, which can ensure the data consistency of the service instances before and after splitting.

[0050] The technical solutions provided by the embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0051] It should be noted that the same reference numerals in the following drawings and embodiments represent the same objects, and therefore, once an object is defined in one drawing or embodiment, it does not need to be further discussed in subsequent drawings and embodiments.

[0052] FIG. 2 is a structural diagram of a distributed service system according to an embodiment of the present disclosure. As shown in FIG. 2, the distributed service system includes a plurality of first service nodes 101 and an operation and maintenance node 20 corresponding to a service instance C0. The plurality of first service nodes 101 can provide consistent coordination services and can reach an agreement through a consensus protocol (such as the Raft protocol) to maintain consistency of the plurality of first service nodes 101.

[0053] The consensus protocol such as the Raft protocol and the Paxos protocol adopts the majority principle to realize master selection and request processing. That is, any request needs to be agreed by more than half of the plurality of service nodes (that is, accepted by more than half of the plurality of service nodes), and the master node will process the request. Therefore, more than half of the plurality of service nodes are normal, that is, the plurality of service nodes can provide service instance supported services, and can tolerate the exception of less than half of the plurality of service nodes. For example, when the plurality of service nodes is 5 or 6, the exception of 2 service nodes can be tolerated. In the embodiments of the present disclosure, the plurality of service nodes refers to at least 3, that is, more than 3. The number of service nodes can be odd or even. Preferably, the number of service nodes is odd, because the tolerance capability of an odd number of service nodes is the same as that of an even number of service nodes which is one more than the odd number. For example, the tolerance capability of 5 service nodes and 6 service nodes is 2 service nodes.

[0054] In the embodiments of the present disclosure, in addition to providing consistent coordination services, the plurality of first service nodes 101 can also provide other services supported by the service instance. For example, the partition scheduling service, the load balancing service, and the data storage service provided by the management and control node of the distributed storage system as shown in the foregoing embodiments.

[0055] The service instance refers to a specific instance providing a specific service. The service instance runs on the plurality of first service nodes 101. The service node specifically undertakes the resources required by the corresponding service. In the embodiments of the present disclosure, the specific content of the service of the service instance is not limited. For example, the service instance can be a management and control instance in a distributed storage system, which provides a partition scheduling service, a load balancing service, and a data storage service, etc. For another example, the service instance can be a distributed lock service instance in a distributed lock system, which provides a distributed lock service, etc.

[0056] In the embodiments of the present disclosure, the plurality of first service nodes 101 are independent of each other, have independent resources, and do not share resources with each other. The resources of the first service node 101 include but are not limited to processing resources such as processors, storage resources such as memories and disks, and input / output (Input / Output, IO) resources, etc.

[0057] In the embodiments of the present disclosure, the operation and maintenance node 20 can monitor an instance splitting event for the service instance C0, and in the case that the instance splitting event for the service instance C0 is monitored, the operation and maintenance node 20 can, in response to the instance splitting event, scale out the service instance C0 to obtain a scaled-out service node.

[0058] In the embodiments of the present disclosure, in order to facilitate description and distinction, the original service instance C0 is defined as a second service instance C0, the service instance C0' after scaling out of the second service instance C0 is defined as a first service instance C0', and the service instance to be split from the first service instance C0' is defined as a third service instance C1-n. Wherein, n represents the nth third service instance. n = 1, 2, …, N. N is the total number of third service instances. Correspondingly, the original service node of the second service instance C0 is the first service node 101, and the service node scaled out in response to the instance splitting event is defined as the second service node 102. Wherein, the number of third service instances C1 is multiple, which means more than 2. For example, the first service instance C0' can be split into 2, 3, 4 or more third service instances C1-n.

[0059] In the embodiments of the present disclosure, the specific implementation form of the instance splitting event is not limited. In some embodiments, the instance splitting event can be triggered by an operation and maintenance personnel. When the operation and maintenance personnel determines that the second service instance C0 needs to be split according to the service performance of the second service instance C0, the operation and maintenance personnel can trigger the instance splitting through a human-computer interaction interface. Correspondingly, when the operation and maintenance node 20 receives the instance splitting instruction provided by the human-computer interaction interface, it is determined that the instance splitting event occurs. The instance splitting event is implemented as receiving the instance splitting instruction.

[0060] In this embodiment, the operation and maintenance personnel can also specify instance splitting information. The instance splitting information is used to indicate how the second service instance C0 is split, and can include: the number of third service instances C1-n to be split from the second service instance C0, the number of service nodes corresponding to each third service instance C1-n, and the resource specification. There is one service node corresponding to one service instance in the third service instance C1-n, which is the aforementioned multiple first service nodes 101.

[0061] The resource specification of the service node is used to represent the amount of hardware resources that the service node can provide, which can be measured by at least one of the following hardware resources: the number of physical cores of the processor and the size of the memory, etc. For example, the resource specification of the second service node is 2 cores and 1G memory, so the second service node occupies 2 processor cores and 1G of memory hardware resources.

[0062] The operation and maintenance node 20 can obtain the instance splitting information submitted through the human-computer interaction interface. Then, the operation and maintenance node 20 can expand the second service instance C0 according to the instance splitting information to obtain a plurality of second service nodes 102. Specifically, the operation and maintenance node 20 can select K second service nodes that meet the resource specification requirements according to the number K of the second service nodes corresponding to the third service instances C1-n and the resource specification, and deploy the target service provided by the second service instance C0 on the second service nodes to realize the expansion of the first service instance C0, and obtain the expanded service instance, i.e., the first service instance C0'. The target service provided by the second service instance C0 can include a consistency coordination service. Of course, it can also include the foregoing data storage service, load balancing service, partition scheduling service, etc.

[0063] In some other embodiments, the operation and maintenance node 20 can autonomously monitor the instance splitting event. The following exemplary embodiments are described in conjunction with several optional implementations.

[0064] Implementation 1: In the case where the capacity of the service instance is insufficient, the tenants with high load can be split from the original instance to create a new instance to serve them exclusively, which can effectively improve the service quality and avoid affecting other tenants. Accordingly, the operation and maintenance node 20 can monitor the load capacity of the second service instance C0. The load capacity of the second service instance C0 is used to represent the idle resources that can be processed by the second service instance C0, which can be represented by the percentage of the idle resources of the second service instance C0 to the total resources of the second service instance C0.

[0065] In implementation 1, a lower limit value of the load capacity can be set. If the load capacity of the second service instance C0 is less than the lower limit value, it indicates that the capacity of the second service instance C0 is insufficient, and the second service instance C0 needs to be split. The lower limit value of the load capacity is the set load capacity. Accordingly, if the operation and maintenance node 20 monitors that the load capacity of the service instance C0 is less than the set load capacity, it is determined that an instance splitting event occurs. Further, the operation and maintenance node 20 can also determine target users with request amounts in a set position from the users of the second service instance C0 in response to the instance splitting event. For example, target users with request amounts in the top M positions can be determined from the users of the second service instance C0, etc. 1≤M is less than N, and is an integer. N represents the total number of users of the second service instance C0. The target users are high-load users.

[0066] Further, the operation and maintenance node 20 can determine the resource specification of the second service nodes corresponding to the third service instances C1-n to be split and the number K of the second service nodes according to the resource requirement information of the target users.

[0067] The resource requirement information of the target user can include resource specifications and disaster recovery requirement information of the service instance required by the target user. The resource specifications of the service instance are used to indicate the resource request amount of the target user, and are used to limit the resource specifications of the service nodes, which can be represented by the resource specifications of the service nodes. The disaster recovery requirement information is used to indicate the disaster recovery capability of the service instance required by the target user, and can be represented by the number of service nodes supporting disaster recovery, which can be used to limit the number of service nodes. Specifically, the number K of the second service nodes can be equal to 2 times the number N of the service nodes supporting disaster recovery plus 1, that is, K=2N+1. For example, the disaster recovery requirement information is 2 service nodes for disaster recovery, and the number of the second service nodes is 5.

[0068] The operation and maintenance node 20 can expand the second service instance C0 according to the resource requirement information of the target user to obtain a plurality of second service nodes 102. Specifically, the operation and maintenance node 20 can determine the number K of the service nodes and the resource specifications of each of the third service instances C1-n according to the resource specifications and the disaster recovery requirement information of the service instance required by the target user. Then, the operation and maintenance node 20 can select K second service nodes satisfying the resource specification requirements according to the number K of the service nodes and the resource specifications of each of the third service instances C1-n, and deploy the target service provided by the second service instance C0 on the second service nodes to realize expansion of the second service instance C0, thereby obtaining the first service instance C0'.

[0069] In this embodiment 1, a new third service instance can be split for the target user with high load to serve the target user exclusively, which can effectively improve the service quality and avoid affecting other users.

[0070] Embodiment 2: In order to reduce the influence range of a single failure point, when the number of tenants served by an instance is too large, the risk can be dispersed by splitting the instance.

[0071] Correspondingly, the operation and maintenance node 20 can monitor the number of users of the second service instance C0, and determine that an instance splitting event occurs when the number of users of the second service instance is greater than or equal to a set user quantity threshold. The operation and maintenance node 20 can expand the second service instance C0 according to a preset explosion radius and resource requirement information of users corresponding to each of a plurality of third service instances C1-n to be split, so as to obtain a plurality of second service nodes 102. The explosion radius can be represented by the maximum number of users affected when the service instance is abnormal.

[0072] Specifically, the operation and maintenance node 20 can determine the number N of the third service instances C1-n to be split and the users corresponding to each of the third service instances C1-n according to the preset explosion radius. The number of users corresponding to each of the third service instances is less than or equal to the preset explosion radius.

[0073] Then, the resource specifications of the second service nodes corresponding to the plurality of service instances C1-n and the number K of the second service nodes can be determined according to the resource requirement information of the users corresponding to the plurality of service instances C1-n respectively. Further, the K second service nodes meeting the resource specification requirements can be selected according to the number K and the resource specifications of the service nodes corresponding to the plurality of service instances C1-n respectively, and the target service provided by the service instance C0 is deployed on the second service nodes to realize the expansion of the second service instance C0, and the first service instance C0' is obtained.

[0074] In this embodiment 2, when the number of users of the second service instance C0 is too large, the risk is dispersed by instance splitting. In this way, even if an abnormality occurs in a certain third service instance, the number of users affected is less than that of the first service instance C0.

[0075] Embodiment 3: Different users can have different disaster recovery requirements. For example, some users with higher criticality can be deployed with zone-level redundancy, some users can need room-level redundancy, and some users can need service node-level redundancy, etc. If all users are deployed on the same service instance, it is difficult to meet such differentiated disaster recovery requirements.

[0076] Correspondingly, the operation and maintenance node 20 can monitor the disaster recovery requirements of the users of the second service instance C0, and determine that an instance splitting event occurs when the disaster recovery requirements of the users of the second service instance C0 meet a set disaster recovery difference condition. For example, the disaster recovery requirements can be divided using disaster recovery levels, and the disaster recovery levels can be divided according to the size of the disaster recovery range, such as zone-level disaster recovery, data center-level disaster recovery, room-level disaster recovery, rack-level disaster recovery, and service node-level disaster recovery, but are not limited thereto. The set disaster recovery difference condition can be realized as a difference across disaster recovery levels, i.e., when the disaster recovery requirements of the users of the first service instance have a difference across disaster recovery levels, it is determined that an instance splitting event occurs.

[0077] Further, the operation and maintenance node 20 can divide the users whose disaster recovery requirement differences are within a set disaster recovery difference range to the same third service instance according to the disaster recovery requirements of the users of the second service instance C0, to determine the number of third service instances to be split. The disaster recovery difference range can be represented as being in the same disaster recovery level. For example, the users whose disaster recovery requirements are in the same disaster recovery level can be divided to the same third service instance. For example, assuming that a data center includes 3 rooms, and 1 room includes 100 racks, the users whose disaster recovery requirements are less than 100 racks can be divided to the rack level. Further, the users of the rack level can be divided to the same third service instance.

[0078] Then, the second service instance C0 can be scaled according to the resource requirement information of the users corresponding to the third service instances to be split, to obtain a plurality of second service nodes 102.

[0079] Specifically, the resource specification and the number K of the second service nodes corresponding to the plurality of third service instances C1-n can be determined according to the resource requirement information of the users corresponding to the plurality of third service instances C1-n respectively. Further, the K second service nodes meeting the resource specification requirement can be selected according to the number K and the resource specification of the service nodes corresponding to the plurality of third service instances C1-n respectively, and the target service provided by the second service instance C0 is deployed on the second service nodes, so as to realize the scaling of the second service instance C0, and obtain the first service instance C0'.

[0080] In the embodiment 3, the users of different disaster recovery levels use different service instances to provide services, so as to meet the disaster recovery requirements of the users.

[0081] The foregoing embodiments 1-3 only exemplarily illustrate the implementation mode of the instance splitting event and the implementation mode of the scaling of the second service instance C0 in response to the instance splitting event, and do not constitute a limitation. The foregoing embodiments 1-3 can be implemented individually, or any two implementation modes can be combined for implementation, or three implementation modes can be combined for implementation. When two or three implementation modes are combined for implementation, the instance splitting event corresponding to any one implementation mode occurs, the instance splitting event is determined to occur, and the service instance scaling mode corresponding to the implementation mode can be used for service instance scaling. If two or three implementation modes are combined for implementation, the instance splitting events corresponding to the plurality of implementation modes all occur, and the service instance scaling modes corresponding to the plurality of implementation modes can be combined for service instance scaling.

[0082] For example, the instance splitting events corresponding to the embodiments 1 and 2 both occur, the target user can be divided into a third service instance alone, and the number and the resource specification of the service nodes of the third service instance can be determined according to the resource requirement information of the target user. Then, the second service nodes meeting the resource specification requirement can be selected according to the number and the resource specification of the service nodes corresponding to the third service instance corresponding to the target user, and the target service provided by the second service instance C0 is deployed on the second service nodes.

[0083] For the other users, the second service instance C0 can be scaled according to the preset explosion radius and the resource requirement information of the users corresponding to the other third service instances to be split, to obtain other second service nodes 102.

[0084] For example, if the instance splitting events corresponding to embodiments 1 and 3 both occur, the target user with the request volume in the set position can be determined from the users of the second service instance C0, and the target user is divided into a third service instance alone; then, according to the disaster recovery needs of the users other than the target user in the second service instance C0, the users with the disaster recovery needs difference within the set disaster recovery difference range are divided into the same third service instance. Then, the second service instance C0 can be expanded according to the resource demand information of the users corresponding to each third service instance, to obtain the first service instance C0'.

[0085] For example, if the instance splitting events corresponding to embodiments 2 and 3 both occur, the users with the disaster recovery needs difference within the set disaster recovery difference range can be divided into the same initial service instance according to the disaster recovery needs of the users other than the target user in the second service instance C0. Then, if there is no case where the user volume in the initial service instance is greater than or equal to the set user volume threshold, the initial service instance is the third service instance, and the second service instance C0 can be directly expanded according to the resource demand information of the users corresponding to each third service instance, to obtain the first service instance C0'.

[0086] If there is a case where the user volume in the initial service instance is greater than or equal to the set user volume threshold, the target service instance with the user volume greater than or equal to the set user volume threshold in the initial service instance is further divided according to the preset explosion radius, to obtain multiple service instances after the division of the target service instance. The third service instance includes the multiple service instances after the division of the target service instance and the initial service instance with the user volume less than the set user volume threshold. Then, the second service instance C0 can be expanded according to the resource demand information of the users corresponding to each third service instance, to obtain the first service instance C0'.

[0087] For example, embodiments 1-3 are combined for implementation, and the instance splitting events corresponding to embodiments 1-3 all occur, the target user with the request volume in the set position can be determined from the users of the second service instance C0, and the target user is divided into a third service instance alone; then, according to the disaster recovery needs of the users other than the target user in the second service instance C0, the users with the disaster recovery needs difference within the set disaster recovery difference range are divided into the same initial service instance. Then, if there is no case where the user volume in the initial service instance is greater than or equal to the set user volume threshold, the third service instance includes the initial service instance and the third service instance corresponding to the target user. Then, the second service instance C0 can be expanded according to the resource demand information of the users corresponding to each third service instance, to obtain the first service instance C0'.

[0088] If the number of users in the initial service instance is greater than or equal to the set user quantity threshold, the target service instance in which the number of users is greater than or equal to the set user quantity threshold is further divided according to the preset explosion radius, and a plurality of third service instances after the division of the target service instance are obtained. The third service instance includes the plurality of service instances after the division of the target service instance, the initial service instance in which the number of users is less than the set user quantity threshold, and the third service instance corresponding to the target user. Then, the first service instance C0' is obtained by expanding the second service instance C0 according to the resource requirement information of the users corresponding to each third service instance.

[0089] In response to the instance splitting event, the second service instance C0 is expanded to obtain an expanded service instance, i.e., the first service instance C0'. The service cluster corresponding to the first service instance C0' is an expanded service cluster. The service cluster includes a plurality of service nodes 10. The plurality of service nodes 10 can include the original first service node 101 and the second service node 102 expanded in response to the instance splitting event. For ease of description, the original first service node 101 and the second service node 102 expanded in response to the instance splitting event are collectively referred to as service nodes 10. The service nodes 10 are independent of each other and have independent resources, and do not share resources with each other. The plurality of service nodes 10 can independently provide consistent coordination services.

[0090] After the second service instance C0 is expanded in response to the instance splitting event, the first service node 101 can also be instructed by the operation and maintenance node 20 to synchronize data to the second service node 102. Specifically, the operation and maintenance node 20 can send a data synchronization instruction to any first service node 101. The data synchronization instruction can include the identifier of the second service node 102. In the embodiment of the present disclosure, the identifier of the service node can uniquely represent a service node, and can be an Internet Protocol (IP) address of the service node, a unique number or identity (ID) in the distributed service system, etc.

[0091] If the first service node receiving the data synchronization instruction is a slave node in the plurality of first service nodes, the data synchronization instruction is forwarded to a master node in the plurality of first service nodes. Further, in response to the data synchronization instruction, the master node can send the stored data to the plurality of second service nodes 102, thereby synchronizing the data to the plurality of second service nodes 102 and realizing data synchronization of the plurality of service nodes 10. If the first service node receiving the data synchronization instruction is a master node, in response to the data synchronization instruction, the master node can send the stored data to the plurality of second service nodes 102, thereby synchronizing the data to the plurality of second service nodes 102 and realizing data synchronization of the plurality of service nodes 10.

[0092] Specifically, the leader in the plurality of first service nodes 101 can send data to the corresponding plurality of second service nodes 102 according to the identification of the plurality of second service nodes 102.

[0093] The second service node 102 can store the data sent by the leader in the first service node 101. The second service node 102 can persistently store the data sent by the leader in the first service node 101, so as to realize data synchronization of the plurality of service nodes 10.

[0094] The plurality of service nodes 10 includes a leader and followers. In the embodiments of the present disclosure, in order to facilitate description and differentiation, the leader in the plurality of service nodes 10 is defined as a first leader, and the leader in the plurality of first service nodes 101 is defined as a second leader. Generally, in the first service instance expansion stage and the plurality of service nodes joint stage, the first leader and the second leader are the same service node, that is, the plurality of first service nodes 101 includes the first leader.

[0095] After the second service node 102 stores the data sent by the first leader, a write success message can be returned to the first leader. The operation and maintenance node 20 can also send the correspondence between the plurality of third service instances to be split and the identification of the service nodes to any first service node 101. Alternatively, the operation and maintenance node 20 can encapsulate the correspondence between the plurality of third service instances to be split and the identification of the service nodes into the aforementioned data synchronization instruction, and send the data synchronization instruction to any first service node 101. If the first service node 101 is the first leader, the correspondence between the plurality of third service instances to be split and the identification of the service nodes can be stored. If the first service node 101 is a follower, the correspondence between the plurality of third service instances to be split and the identification of the service nodes is forwarded to the first leader. The first leader stores the correspondence between the plurality of third service instances to be split and the identification of the service nodes.

[0096] Based on the correspondence between the plurality of third service instances to be split and the identification of the service nodes, the first service node can count the number of service nodes that return the write success message in the service nodes corresponding to the plurality of third service instances to be split respectively.

[0097] In some embodiments, the first leader can determine that the data synchronization of the plurality of service nodes is completed when the number of services that return the write success message is the total number of followers.

[0098] In some embodiments, the consensus protocol adopts a majority principle for master election and request processing. In order to ensure high availability of the distributed service system, a new master can be elected when the master node is abnormal, so that the state of the entire distributed service system can continue to advance. The majority principle is also continued in the data synchronization phase. That is, as long as more than half of the service nodes of the instance are running normally before and after the instance splitting, the service supported by the service instance can be provided. Based on this, the first master node can determine that the data synchronization is completed when the number of service nodes that return the write success message is greater than or equal to a set number R. The set number R is greater than or equal to half of the total number of the plurality of service nodes 10 corresponding to the first service instance C0' and less than the total number of the plurality of service nodes 10.

[0099] However, this data synchronization method can cause data inconsistency before and after the instance splitting. For example, in FIG. 5, an ellipse represents a service node. As shown in FIG. 5, assume that the service nodes {1, 2, 3} are the first service nodes of the second service instance C0 originally; and the service nodes {4, 5, 6} are the second service nodes expanded. The foregoing data synchronization method can cause the data synchronization of the service nodes {1, 2, 3, 4} to be completed, while the data synchronization of the service nodes {5, 6} to fail. In this way, when the instance splits, the service nodes {1, 2, 3} split into one service instance, and the service nodes {4, 5, 6} split into another service instance. However, the service instance corresponding to the service nodes {4, 5, 6} has only the data of the service node {4} that is the latest. If the service node {4} in the service instance corresponding to the service nodes {4, 5, 6} is abnormal, the service instance corresponding to the service nodes {4, 5, 6} causes data loss.

[0100] In order to solve this problem, in some embodiments of the present disclosure, a set number M corresponding to each of the plurality of third service instances C1-n to be split can be set. One third service instance corresponds to one set number M. The set number M corresponding to any third service instance is greater than or equal to half of the total number P of service nodes corresponding to the third service instance and less than the total number P of service nodes corresponding to the third service instance. That is, P / 2≤M<P. Generally, M=P / 2. For example, P=5, and M can be 2.5. The set numbers M corresponding to the plurality of third service instances C1-n can be the same or different, which is determined by the number of service nodes corresponding to each third service instance.

[0101] In the case that the number of service nodes returning the write success message is greater than the corresponding set number M in the service nodes corresponding to the plurality of third service instances C1-n to be split, it is determined that the data synchronization of the plurality of service nodes 10 of the first service instance C0' is completed. In this way, each third service instance after instance splitting has data of more than half of the service nodes, which is consistent with the master node before splitting. Even if an abnormality occurs in a service node whose data synchronization is completed, complete data can be provided by other service nodes whose data synchronization is completed, and the risk of data loss before and after instance splitting can be reduced.

[0102] In some other embodiments, a joint consensus mechanism can be introduced to solve the data loss risk caused by the foregoing instance expansion. The joint consensus mechanism used in the instance splitting scheme provided by the embodiments of the present disclosure will be described first. The joint consensus mechanism is a commonly used method for realizing member change.

[0103] In the process of running the distributed service system, service nodes sometimes fail, and the dynamic addition and deletion of service nodes need to be supported. Member change is to change the node set of the running consistency coordination protocol (such as the Raft protocol) in the service cluster, such as addition, deletion, replacement, etc. The member change process cannot affect the availability of the distributed service system. Member change is also a consistency problem, that is, all service nodes reach an agreement on the new member. In the process of member change, the members participating in voting change.

[0104] The joint consensus mechanism can make the service cluster first switch from the old member (i.e., the original service node) configuration Cold to a transition member configuration, referred to as the joint consensus member configuration Cold,new. The joint consensus member configuration Cold,new is a combination of the old member configuration Cold and the new member (i.e., the new service node) configuration Cnew. Once the joint consensus member configuration Cold,new is submitted, the service instance is switched to the new member configuration Cnew. In the joint consensus member configuration Cold,new, the majority of the old member configuration Cold and the majority of the new member configuration Cnew confirm other service nodes, that is, the majority of the old member configuration Cold and the majority of the new member configuration Cnew confirm that other service nodes belong to the same member group as themselves.

[0105] The working principle of the joint consensus mechanism is described below with reference to FIG. 6. As shown in FIG. 6, assume that the original service node set corresponding to a service instance is Cold={1, 2, 3}. The original service node can provide a consistency coordination service. Any request is considered to have reached consensus as long as any two of the three service nodes accept the request. The service node accepting the request means that the service node writes the request into a log, that is, the request is stored persistently. In the embodiments of the present disclosure, the service node can log the request according to the order in which the request is accepted, and the log number can also be referred to as a log sequence number, which is generally an increasing natural number.

[0106] If it is necessary to change the service nodes of the service instance to {4, 5, 6}, the joint consensus (Joint Consensus) transition phase, that is, the original service node set Cold={1, 2, 3} and the new service node set Cnew={4, 5, 6} work together to elect a leader. In the joint consensus transition phase, for any request received by the service instance, only if the majority of the original service node set Cold accepts the request and the majority of the new service node set Cnew also accepts the request, the leader will process the request. After the joint consensus transition phase, the service instance formally switches to the new service node set Cnew, and the member change is completed. In the whole process, it can be ensured that the original service node and the new service node are not used at the same time, so as to avoid the risk of "split brain". Wherein, the service node in the joint consensus transition phase is in the joint consensus state. That is, in the joint consensus state, only if the majority of the original service node set Cold accepts the request and the majority of the new service node set Cnew also accepts the request, the leader will process the request.

[0107] The foregoing joint consensus mechanism is used to implement the member switching of the service instance, and is not used to implement the splitting of the service instance. The embodiments of the present disclosure can implement a scheme for splitting the service instance. The application of the joint consensus mechanism to the implementation of the service instance splitting scheme provided by the embodiments of the present disclosure is described exemplarily below.

[0108] Specifically, the first master node can modify the stored member group identifier to the identifiers of the plurality of service nodes in the case that the plurality of service nodes are in data synchronization, after determining that the plurality of service nodes are in data synchronization. The service nodes of the same member group correspond to the same service instance. The member group identifier stored by the service node is used to record the identifiers of the service nodes corresponding to the same service instance as the service node. The first master node modifies the stored member group identifier to the identifiers of the plurality of service nodes, that is, the first master node confirms that the plurality of service nodes belong to the same member group, that is, correspond to the same service instance. Among the identifiers of the plurality of service nodes, the identifier of the first service node is originally stored by the first master node, and the identifier of the second service node is provided by the operation and maintenance node. The first master node can also store the correspondence between the plurality of third service instances C1-n to be split and the identifiers of the service nodes.

[0109] Further, the first master node can also send a joint instruction to the slave nodes in the plurality of service nodes. The joint instruction can include: the identifiers of the plurality of service nodes and the correspondence between the plurality of third service instances to be split and the identifiers of the service nodes. Specifically, the joint instruction is used to instruct the slave nodes to modify the stored member group identifier to the identifiers of the plurality of service nodes. The slave nodes refer to all slave nodes of the first master node in the plurality of service nodes. The slave nodes can modify the stored member group identifier to the identifiers of the plurality of service nodes in response to the joint instruction, and store the correspondence between the plurality of third service instances to be split and the identifiers of the service nodes, so that the plurality of service nodes enter a joint consensus state (corresponding to “joint consensus” in FIG. 4). The first service instance in the joint consensus state in FIG. 4 is denoted as Cjoint.

[0110] The service nodes that modify the stored member group identifier to the identifiers of the plurality of service nodes can return a response success message to the first master node. The first master node can determine whether the slave nodes successfully modify the stored member group identifier to the identifiers of the plurality of service nodes according to whether the response success message is received within a set time period. If the first master node receives a response success message of the joint instruction returned by a slave node within a set time period, it is determined that the slave node successfully modifies the stored member group identifier to the identifiers of the plurality of service nodes.

[0111] In the embodiments of the present disclosure, in order to facilitate description and distinction, the service node whose stored member group identifier is successfully modified to the identifier of the plurality of service nodes is defined as a second target service node. The second target service node includes the first master node. Since the consensus protocol adopts the majority principle for master selection and request processing, in order to ensure the high availability of the distributed service system, when the master node is abnormal, a new master node can still be re-elected, so that the state of the entire distributed service system can continue to advance. The majority principle is also continued in the joint consensus stage. That is, as long as more than half of the service nodes of the instance are running normally before and after the instance splitting, the service supported by the service instance can be provided. Therefore, in the joint consensus stage, as long as more than half of the service nodes in each third service instance C1-n to be split are in the joint consensus state, it can be considered that the first service instance C0' before splitting is in the joint consensus state.

[0112] Based on this, for the plurality of third service instances C1-n to be split, a set number M corresponding to each third service instance can be set. The description of the set number M can be referred to the related content of the foregoing embodiments, which will not be described here. Correspondingly, the first master node can also count the number of second target service nodes in the service nodes corresponding to each third service instance, and determine that the plurality of service nodes are in the joint consensus state when the number of second target service nodes corresponding to the plurality of third service instances C1-n to be split is greater than the set number M corresponding to each third service instance. Further, the first master node can also send the first identifier corresponding to the joint consensus state to the slave node. The slave node can determine that the plurality of service nodes are in the joint consensus state according to the first identifier. The role of the joint consensus state will be described in the following embodiments, which will not be described here.

[0113] When the plurality of service nodes 10 are in the joint consensus state (such as the state corresponding to Cjoint in FIG. 6), for any request received in the joint consensus state (defined as a target request for the sake of description), the first master node can forward the target request to the slave node. The slave node receiving the target request can store the target request, such as performing persistent storage on the target request, for example, writing the target request into a log, etc. After the slave node stores the target request, the first master node returns a storage success message. When the first master node receives the storage success message returned by the slave node for the target request within a set time period, it is determined that the slave node has stored the target request, and the slave node can be regarded as a target service node (defined as a third target service node).

[0114] Since the consensus protocol adopts the majority principle for master selection and request processing, in order to ensure the high availability of the distributed service system, a new master node can be re-elected when the master node is abnormal, so that the state of the entire distributed service system can continue to advance. The majority principle is also continued in the joint consensus state. Therefore, in the joint consensus state, as long as more than half of the service nodes in each third service instance C1-n store the target request, after the first service instance is split into multiple third service instances C1-n, more than half of the service nodes corresponding to each third service instance C1-n store the target request, which is consistent with the state of the master node before splitting, so that the data on the new service instance, i.e., the third service instance C1-n, is not lost compared to the first service instance C0', and the data consistency before and after the service instance splitting is guaranteed.

[0115] Based on this, for the multiple third service instances to be split, a set number M corresponding to each third service instance can be set. The description of the set number M can be referred to the related content of the foregoing embodiments, which will not be repeated here. Correspondingly, the first master node can also count the number of third target service nodes in the service nodes corresponding to each third service instance, and determine that the multiple service nodes have reached a consensus on the target request and can process the target request when the number of third target service nodes corresponding to the multiple third service instances is greater than the set number M corresponding to each third service instance. In this way, in the joint consensus state, as long as more than half of the service nodes in each third service instance C1-n store the target request, after splitting into multiple third service instances C1-n, more than half of the service nodes corresponding to each third service instance C1-n are consistent with the state of the master node before splitting, so that the data on the new service instance, i.e., the third service instance C1-n, is not lost compared to the first service instance C0', and the data consistency before and after the service instance splitting is guaranteed.

[0116] Among them, the slave node storing the target request can also process the stored request in the order of request first-in first-out, that is, when the target request is traversed, the target request can be processed to reach a consensus with the first master node and maintain data consistency.

[0117] In the joint consensus mechanism, even if less than half of the service nodes implement data synchronization in the service node corresponding to the third service instance in the foregoing service instance expansion phase, when a new target request (generally a write request) arrives at a service node that is not synchronized with the first master node, the complete data can be obtained based on the target request forwarded by the first master node. Specifically, for the service node that is not synchronized with the first master node, the data with a log sequence number less than that of the target request can be requested from the first master node according to the log sequence number of the target request, so that the complete data is obtained, and the inconsistency between the data of the third service instance after splitting and the data of the service instance before splitting can be prevented.

[0118] For example, it is assumed that the service nodes {1, 2, 3} are the original first service nodes of the second service instance C0, the service nodes {4, 5, 6} are the expanded second service nodes, and the data synchronization of the service nodes {1, 2, 3, 4} is completed, while the data synchronization of the service nodes {5, 6} fails. In the joint consensus state of the service nodes {1, 2, 3, 4, 5, 6}, when a target request arrives, the first master node can forward the target request to the service nodes {5, 6}. The service nodes {5, 6} can request the data with a log sequence number less than that of the target request from the first master node according to the log sequence number of the target request, so that the complete data is obtained. In this way, the state of the service node corresponding to the third service instance C1-n after splitting is consistent with that of the master node before splitting, so that the data on the new service instance, i.e., the third service instance C1-n, is not lost compared with the first service instance C0', and the data consistency before and after the service instance splitting is ensured.

[0119] To prevent the data inconsistency of multiple service nodes, the first master node sends a seal instruction to the slave nodes in the joint consensus state of the multiple service nodes. The seal instruction is used to instruct the multiple service nodes to no longer receive write requests, i.e., to no longer respond to the write requests, and new data is not written in the first service instance. The slave nodes can confirm the seal instruction. Specifically, the slave node receiving the seal request can store the seal instruction, such as performing persistent storage on the seal instruction, for example, writing the seal instruction into a log, and the like. After the slave node stores the seal instruction, the first master node returns a confirmation success message. The first master node determines that the slave node confirms the seal instruction when the confirmation success message for the seal instruction is received from the slave node within a set time length, and the slave node is determined as a target service node (defined as a first target service node). The above process corresponds to the "seal" in FIG. 3 and FIG. 4.

[0120] Since the consensus protocol adopts the majority principle for master selection and request processing, in order to ensure the high availability of the distributed service system, a new master node can still be elected when the master node is abnormal, so that the state of the entire distributed service system can continue to advance. Therefore, in the joint consensus state, as long as more than half of the service nodes in each third service instance C1-n confirm the write-prohibited instruction, the state of the master node determined by each third service instance C1-n after splitting into multiple third service instances C1-n is consistent with the state of the master node before splitting, so that the data on the new service instance, i.e., the third service instance C1-n, is not lost compared to the first service instance C0', ensuring the data consistency before and after the service instance splitting.

[0121] Based on this, for the multiple third service instances to be split, a set number M corresponding to each third service instance can be set. The description of the set number M can be referred to the related content of the foregoing embodiments, which will not be repeated here. Based on the pre-set number M, the first master node can also count the number of first target service nodes in the service nodes corresponding to each third service instance, and determine that the multiple service nodes confirm the write-prohibited instruction when the number of first target service nodes corresponding to the multiple third service instances is greater than the set number M corresponding to each third service instance. In this way, after splitting into multiple third service instances C1-n, the state of the majority of service nodes in the service nodes corresponding to each third service instance C1-n when selecting the master node is consistent with the state of the master node before splitting, so that the data on the new service instance, i.e., the third service instance C1-n, is not lost compared to the first service instance C0', ensuring the data consistency before and after the service instance splitting.

[0122] Further, when the number of first target service nodes corresponding to the multiple third service instances is greater than the set number M corresponding to each third service instance, the first master node can send a splitting instruction to the slave node. The splitting instruction is used to instruct the multiple service nodes to split the first service instance C0' into multiple third service instances C1-n.

[0123] Specifically, when the number of first target service nodes corresponding to the multiple third service instances is greater than the set number M corresponding to each third service instance, the first master node can obtain the correspondence between the multiple third service instances and the identifiers of the service nodes. The correspondence can be provided by the foregoing operation and maintenance node 20 to the first master node. Further, the first master node can generate a splitting instruction according to the correspondence between the multiple third service instances and the identifiers of the service nodes. For example, the correspondence between the multiple third service instances and the identifiers of the service nodes can be encapsulated into the splitting instruction. Further, the first master node can send the splitting instruction to the slave node.

[0124] Correspondingly, the service node obtaining the split instruction can split the first service instance Cjoint in the joint consensus state into a plurality of third service instances C1-n in response to the split instruction. The service node obtaining the split instruction includes the first master node itself and the slave node receiving the split instruction.

[0125] Specifically, any service node A of the first master node and the slave node receiving the split instruction can obtain the identifier of the service node belonging to the same third service instance as the service node A according to the correspondence between the plurality of third service instances and the identifiers of the service nodes carried by the split instruction, and modify the stored member group identifier to the identifier of the service node belonging to the same third service instance as itself, so as to split the first service instance into a plurality of third service instances. For example, as shown in FIG. 3 and FIG. 4, the first service instance can be split into two third service instances C1-1 and C1-2 with service nodes being {1, 2, 3} and {4, 5, 6} respectively. The above process corresponds to “split” in FIG. 3 and FIG. 4.

[0126] In the embodiment, in response to the instance split event, the service instance is expanded, and the expanded service instance is prevented from having new data written in through the write-forbid instruction, and then the expanded service instance is split into a plurality of service instances, thereby realizing the split of the service instance. In the expansion stage, the original service node of the service instance can still provide services, and the data synchronization can be completed while the service continuity is ensured, and by preventing the expanded service instance from having new data written in, the plurality of service instances after the split start from the same state, thereby ensuring the data consistency of the service instances before and after the split.

[0127] In addition, the entire process of the instance split scheme provided by the embodiment of the disclosure does not need to introduce additional management and control dependencies, thereby simplifying the system architecture.

[0128] In some embodiments of the disclosure, during the time period from the completion of the expansion of the service instance to the split of the service instance, the write-forbid instruction makes the distributed service system service unavailable, and the time window of the service unavailability is long. In order to reduce the service unavailability window, in some embodiments, a joint consensus mechanism can be introduced, as shown in FIG. 4, the first master node can prevent the expanded service instance from having new data written in through the write-forbid instruction when the first service instance C0' is in the joint consensus, and then split the first service instance into a plurality of third service instances through the split instruction, thereby realizing the split of the service instance.

[0129] The specific implementation of the joint consensus mechanism for the plurality of service nodes corresponding to the first service instance, the working process of the plurality of service nodes in the joint consensus state, and the specific implementation of the split instruction for splitting the first service instance into a plurality of third service instances can be found in the foregoing embodiments, and thus will not be described here.

[0130] In this embodiment, the original service nodes of the service instance can still provide services during the expansion phase, and data synchronization can be completed while ensuring service continuity. During the time period between the completion of the service instance expansion and the splitting of the service instance, the plurality of service nodes can be in a joint consensus state through the joint consensus mechanism, and the joint consensus mechanism can still be used to process new requests, thereby ensuring service continuity and maintaining data consistency while reducing the unavailability time window.

[0131] For the first master node, the number of service nodes corresponding to each third service instance that have confirmed the split instruction can also be counted. For ease of description and differentiation, the service nodes that have confirmed the split instruction are defined as fourth service nodes. Specifically, if the first master node receives a confirmation success message for the split instruction from a slave node within a set time period, it is determined that the slave node has confirmed the split instruction, and the slave node is taken as a fourth target service node. The specific implementation of the confirmation of the split instruction by the slave node can be found in the foregoing description of the confirmation of the split instruction by the slave node, and thus will not be described here.

[0132] In the joint consensus state, as long as more than half of the service nodes in each third service instance C1-n to be split have confirmed the split instruction, after the splitting into a plurality of third service instances C1-n, each third service instance C1-n has a majority of service nodes providing services.

[0133] Based on this, for the plurality of third service instances to be split, the set number M corresponding to each third service instance can be set. The description of the set number M can be referred to the related content of the foregoing embodiments, which will not be described here. Based on the pre-set set number M, the first master node can further count the number of fourth target service nodes in the service nodes corresponding to each third service instance, and in the case that the number of fourth target service nodes corresponding to the plurality of third service instances is greater than the respective corresponding set number M, it is determined that the plurality of service nodes confirm the splitting instruction, and then the first master node can modify the stored member group identifier to the identifier of the service node corresponding to the same third service instance as the first master node, so that the first master node exits the joint consensus state. In this way, as long as more than half of the service nodes in each third service instance C1-n to be split confirm the splitting instruction, after splitting into a plurality of third service instances C1-n, each third service instance C1-n has a majority of service nodes providing services, which can guarantee the continuity of services.

[0134] However, if the first master node abnormally occurs in the service instance splitting phase, that is, the first master node abnormally occurs in the process of sending the splitting instruction. It may cause some slave nodes not to receive the splitting instruction. If more than half of the service nodes corresponding to a certain third service instance or certain third service instances do not receive the splitting instruction, the splitting of the third service instance will fail. The service node that does not receive the splitting instruction also needs to learn the explicit global state from "somewhere", that is, the information that the global instance splitting has occurred.

[0135] In order to enable the slave node that does not receive the splitting instruction to also learn the information that the global instance splitting has occurred, in some embodiments of the present disclosure, the voting request used for the master node stealing can be improved. The voting request used in the traditional master selection process is shown in the left part of FIG. 7, which can include a three-tuple of epoch, logid and serverid. Among them, the epoch is an integer used to identify a logical time period, and the value of the epoch is increased every time the service cluster experiences a potential master node change. The epoch can ensure that all service nodes in the service cluster know the current election cycle. If a service node receives a voting request with a higher epoch number, it will know that any previous voting commitment has expired, and can reconsider the new voting request.

[0136] The log ID is a reference to a certain specific entry in the election log. Each log entry has a unique index value, and this index is the log ID. The log ID is used to prove that the candidate's log is at least as new (or up-to-date) as the voting request receiver. This is because only the service node with the latest log can become the primary node. Therefore, the candidate needs to provide the log ID of the last entry in its log to prove its qualifications.

[0137] The server ID is an identifier that uniquely identifies each service node in the service cluster, and is used to explicitly indicate which service node is requesting voting, which helps the receiver to determine the specific object of the voting.

[0138] In this embodiment, as shown in the right part of FIG. 7, a target field is added to the traditional voting request. The target field is used to record whether the service cluster is in a joint consensus state. Among them, the first identifier can be used to represent that the service cluster is in a joint consensus state, and the second identifier can be used to represent that the service cluster is not in a joint consensus state. The first identifier and the second identifier are different identifiers, and the specific content of the two is not limited. For example, the first identifier can be "1", and the second identifier can be "0".

[0139] In this embodiment, the first primary node writes the first identifier into the target field when determining that the service cluster enters the joint consensus state, stores the target field, and can also send the first identifier to the slave node. The slave node can write the first identifier into the target field and store the target field.

[0140] The first primary node and the slave node that receives the split instruction can modify the value of the target field to the second identifier after executing the split instruction, to identify that it exits the joint consensus state.

[0141] Based on the target field, the slave node can re-select the master using the improved voting request in the case of abnormality of the first primary node. That is, the slave node can send a voting request to other service nodes belonging to the same member group as the slave node according to the currently stored member group identifier to select the master. Among them, the voting request includes the target field, and the voting request carries the value of the target field.

[0142] Specifically, the slave node can encapsulate the value of the target field stored by itself into the voting request, and send the voting request to other service nodes belonging to the same member group according to the currently stored member group identifier, so as to elect the master. For the slave node that executes the split instruction, the stored member group identifier is the identifier of the service node corresponding to the same third service instance as the slave node, and the value of the target field is the second identifier representing the second consensus state. Even if there are still service nodes in the member group of the slave node that have not received the split instruction, the second identifier corresponding to the target field carried in the received voting request can be used to determine that the third service instance corresponding to the service node sending the voting request, and then the voting request is voted.

[0143] For the slave node that has not received the split instruction, the stored member group identifier is the identifier of the plurality of service nodes, and the value of the target field is the first identifier representing the second consensus state. The slave node sends the voting request to the service node corresponding to the recorded member group identifier, and naturally sends the voting request to the fourth target service node that has received the split instruction.

[0144] If the fourth target service node that has received the split instruction receives the voting request, the value of the target field in the voting request can be obtained. If the value of the target field is the first identifier, it is determined that the slave node sending the voting request is the target slave node, and the target slave node is still in the second consensus state. The fact that the first service instance C0' has split can be informed to the slave node sending the voting request. Specifically, the correspondence between the plurality of third service instances to be split and the identifiers of the service nodes can be returned to the target slave node sending the voting request. In this way, the target slave node can know the fact that the first service instance C0' has split according to the correspondence between the plurality of third service instances to be split and the identifiers of the service nodes, and can determine the identifier of the service node corresponding to the third service instance corresponding to the target slave node according to the correspondence between the plurality of third service instances to be split and the identifiers of the service nodes. Then, the stored member group identifier can be modified to the identifier of the service node corresponding to the third service instance corresponding to the target slave node, so as to realize the splitting of the service instance.

[0145] In this embodiment, by strengthening the information of the voting request, the service node that has not received the split instruction can learn the global splitting information of the service instance from the service node that has received the split instruction, and guide the service node that has not received the split instruction to enter the splitting phase, so as to realize the splitting of the service instance.

[0146] In the distributed service system based on joint consensus provided by the embodiments of the present disclosure, the term number and the log ID mark the space-time before and after the splitting of the service instance in the distributed service system. The time refers to the logical time, which can be represented by the term number. Specifically, as shown in FIG. 8, the space-time of the entire system is continuous from the starting second service instance C0 to the first service instance C0' after expansion, and even to the service instance Cjoint in the joint consensus state. That is, the log sequence on each service node in the distributed service system is completely consistent from the original service instance C0 to the service instance Cjoint in the joint consensus state before splitting, and the term number and the log ID are monotonically increasing synchronously. The first service instance C0' splits into multiple third service instances C1-n (only the third service instances C1-1 and C1-2 are illustrated in FIG. 8). The multiple third service instances C1-n diverge in space-time, independently transition to subsequent states according to their respective space-time from the same starting point, process their respective requests, and achieve the splitting of the services provided by the service instances.

[0147] In addition to the distributed service system provided by the foregoing embodiments, the embodiments of the present disclosure also provide an instance splitting method, which will be exemplarily described below.

[0148] FIG. 9 is a flowchart of an instance splitting method provided by the embodiments of the present disclosure. The instance splitting method is applicable to any service node in the multiple service nodes corresponding to the first service instance, the first service instance being obtained by expanding the second service instance in response to an instance splitting event, the multiple service nodes independently providing consistent coordination services, and the data synchronization being completed. As shown in FIG. 9, the instance splitting method mainly includes:

[0149] 901, in the case of being a master node, sending a write-inhibition instruction to a slave node in the multiple service nodes, so that the service cluster no longer receives a write request.

[0150] 902, in the case where the number of first target service nodes in the service nodes corresponding to the multiple third service instances to be split is greater than the corresponding set number, sending a splitting instruction to the slave node, so that the master node and the slave node receiving the splitting instruction split the first service instance into multiple third service instances in response to the splitting instruction.

[0151] The first target service node is a service node that confirms the write-inhibition instruction; the set number corresponding to any third service instance is greater than or equal to half of the total number of service nodes corresponding to the third service instance, and is less than the total number of service nodes corresponding to the third service instance.

[0152] In the embodiments of the present disclosure, the operation and maintenance node can monitor an instance split event for the second service instance C0, and in response to the instance split event, expand the second service instance C0 to obtain an expanded first service instance C0' and the expanded second service node, when the instance split event for the second service instance C0 is monitored. The specific implementation of the operation and maintenance node monitoring the instance split event can refer to the related content of the foregoing system embodiments, which will not be described here.

[0153] After the second service instance C0 is expanded in response to the instance split event, the operation and maintenance node can also instruct the first service node to synchronize data to the second service node. Specifically, the operation and maintenance node can send a data synchronization instruction to any first service node. The data synchronization instruction can include the identifier of the second service node.

[0154] If the first service node receiving the data synchronization instruction is a slave node in the plurality of first service nodes, the data synchronization instruction is forwarded to a master node in the plurality of first service nodes. Further, in response to the data synchronization instruction, the master node can send the stored data to the plurality of second service nodes, thereby synchronizing the data to the plurality of second service nodes and realizing data synchronization of the plurality of service nodes. If the first service node receiving the data synchronization instruction is a master node, in response to the data synchronization instruction, the master node can send the stored data to the plurality of second service nodes, thereby synchronizing the data to the plurality of second service nodes and realizing data synchronization of the plurality of service nodes.

[0155] Specifically, the master node in the plurality of first service nodes can send data to the plurality of second service nodes corresponding to the identifiers according to the identifiers of the plurality of second service nodes. The second service nodes store the data sent by the master node in the first service nodes. The second service nodes can persistently store the data sent by the master node in the first service nodes, thereby realizing data synchronization of the plurality of service nodes.

[0156] The plurality of service nodes include a master node and a slave node. In the embodiments of the present disclosure, for the convenience of description and distinction, the master node in the plurality of service nodes is defined as a first master node, and the master node in the plurality of first service nodes is defined as a second master node. Generally, in the first service instance expansion phase and the plurality of service node joint phase, the first master node and the second master node are the same service node, that is, the plurality of first service nodes include the first master node.

[0157] After the second service node stores the data sent by the first master node, a write success message can be returned to the first master node. The operation and maintenance node can also send the correspondence between the multiple third service instances to be split from the first service instance and the identities of the service nodes to any first service node. Alternatively, the operation and maintenance node can encapsulate the correspondence between the multiple third service instances and the identities of the service nodes into the aforementioned data synchronization instruction, and send the data synchronization instruction to any first service node. If the first service node is the first master node, the correspondence between the multiple third service instances and the identities of the service nodes can be stored. If the first service node is a slave node, the correspondence between the multiple third service instances and the identities of the service nodes is forwarded to the first master node. The first master node stores the correspondence between the multiple third service instances and the identities of the service nodes.

[0158] Based on the correspondence between the multiple third service instances and the identities of the service nodes, the first service node can count the number of service nodes that return the write success message among the service nodes corresponding to the multiple third service instances to be split.

[0159] In some embodiments, the first master node can determine that the data synchronization of the multiple service nodes is completed when the number of services that return the write success message is equal to the total number of slave nodes.

[0160] In other embodiments, the first master node can determine that the data synchronization is completed when the number of service nodes that return the write success message is greater than or equal to a set number R. The set number R is greater than or equal to half of the total number of service nodes corresponding to the first service instance C0', and is less than the total number of service nodes.

[0161] However, this data synchronization method can cause data inconsistency before and after instance splitting. To solve this problem, in some embodiments of the present disclosure, for the multiple third service instances C1-n to be split, a set number M corresponding to each third service instance can be set. One third service instance corresponds to one set number M. The set number M corresponding to any third service instance is greater than or equal to half of the total number P of service nodes corresponding to the third service instance, and is less than the total number P of service nodes corresponding to the third service instance. That is, P / 2≤M<P. The set numbers M corresponding to the multiple third service instances C1-n can be the same or different, which is determined by the number of service nodes corresponding to each third service instance.

[0162] In a case where the number of service nodes that return the write success message is greater than the corresponding set number M in the service nodes that each correspond to a third service instance C1-n to be split, it is determined that the data synchronization of the plurality of service nodes of the first service instance C0' is completed. In this way, each third service instance after instance splitting has data of more than half of the service nodes, which is consistent with the master node before splitting. Even if an abnormality occurs in a service node whose data synchronization is completed, complete data can be provided by other service nodes whose data synchronization is completed, which can reduce the risk of data loss before and after instance splitting.

[0163] In some other embodiments, a joint consensus mechanism can be introduced to solve the data loss risk caused by the foregoing instance expansion. Specifically, after determining that the data synchronization of the plurality of service nodes is completed, the first master node can modify the stored member group identifier to the identifiers of the plurality of service nodes in the case of data synchronization of the plurality of service nodes. The service nodes of the same member group correspond to the same service instance. The member group identifier stored by the service node is used to record the identifiers of the service nodes that correspond to the same service instance as itself. The first master node modifies the stored member group identifier to the identifiers of the plurality of service nodes, that is, the first master node confirms that the plurality of service nodes belong to the same member group, that is, belong to the same service instance. The identifier of the first service node in the identifiers of the plurality of service nodes is originally stored by the first master node, and the identifier of the second service node is provided by the operation and maintenance node. The first master node can also store the correspondence between the plurality of third service instances C1-n and the identifiers of the service nodes.

[0164] Further, the first master node can also send a joint instruction to the slave nodes in the plurality of service nodes. The joint instruction can include the identifiers of the plurality of service nodes and the correspondence between the plurality of third service instances and the identifiers of the service nodes. Specifically, the joint instruction is used to instruct the slave nodes to modify the member group identifier to the identifiers of the plurality of service nodes. The slave nodes refer to all slave nodes of the first master node in the plurality of service nodes. The slave nodes can modify the stored member group identifier to the identifiers of the plurality of service nodes in response to the joint instruction, and store the correspondence between the plurality of third service instances and the identifiers of the service nodes, so that the plurality of service nodes enter a joint consensus state.

[0165] The service nodes that modify the stored member group identifier to the identifiers of the plurality of service nodes can return a response success message to the first master node. The first master node can determine whether the slave nodes successfully modify the stored member group identifier to the identifiers of the plurality of service nodes according to whether the response success message is received within a set time period. If the first master node receives a response success message of the joint instruction returned by a slave node within a set time period, it is determined that the slave node successfully modifies the stored member group identifier to the identifiers of the plurality of service nodes.

[0166] In the embodiments of the present disclosure, in order to facilitate description and distinction, the service node that modifies the stored member group identifier to the identifier of the plurality of service nodes is defined as a second target service node. The second target service node includes the first master node. Since the consensus protocol adopts the majority principle for master selection and request processing, in order to ensure the high availability of the distributed service system, for the plurality of third service instances C1-n to be split, a set number M corresponding to each third service instance can be set. For the description of the set number M, please refer to the related content of the foregoing embodiments, which will not be described here. Correspondingly, the first master node can also count the number of second target service nodes in the service nodes corresponding to each third service instance, and determine that the plurality of service nodes are in a joint consensus state when the number of second target service nodes corresponding to the plurality of third service instances C1-n is greater than the set number M corresponding to each third service instance. Further, the first master node can also send the first identifier corresponding to the joint consensus state to the slave node. The slave node can determine that the plurality of service nodes are in the joint consensus state according to the first identifier.

[0167] When the plurality of service nodes are in the joint consensus state, for any request received in the joint consensus state (defined as a target request for the sake of description), the first master node can forward the target request to the slave node. The slave node receiving the target request can store the target request, such as performing persistent storage on the target request, for example, writing the target request into a log, etc. After the slave node stores the target request, the first master node returns a storage success message. The first master node receives the storage success message returned by the slave node for the target request within a set time period, determines that the slave node has stored the target request, and takes the slave node as a target service node (defined as a third target service node).

[0168] Since the consensus protocol adopts the majority principle for master selection and request processing, in order to ensure the high availability of the distributed service system, a new master node can be re-elected when the master node is abnormal, so that the state of the entire distributed service system can continue to advance. Therefore, the set number M corresponding to each third service instance can be set. For the description of the set number M, please refer to the related content of the foregoing embodiments, which will not be repeated here. Correspondingly, the first master node can also count the number of third target service nodes in each third service instance corresponding service node, and determine that the plurality of service nodes have reached a consensus on the target request and can process the target request when the number of third target service nodes corresponding to each third service instance is greater than the corresponding set number M. In this way, in the joint consensus state, as long as more than half of the service nodes in each third service instance C1-n store the target request, after splitting into a plurality of third service instances C1-n, more than half of the service nodes in each third service instance C1-n remain consistent with the state of the master node before splitting, so that the data on the new service instance, i.e. the third service instance C1-n, is not lost compared to the first service instance C0', ensuring the data consistency before and after the service instance splitting.

[0169] Among them, the slave node storing the target request can also process the stored request according to the first-in first-out order of the request, that is, when the target request is traversed, the target request can be processed to reach a consensus with the first master node and maintain data consistency.

[0170] Under the joint consensus mechanism, even if less than half of the service nodes in the service node corresponding to the third service instance during the foregoing service instance expansion phase achieve data synchronization, when a new target request (generally a write request) arrives at a service node that has not synchronized data with the first master node, the complete data can be obtained based on the target request forwarded by the first master node. Specifically, for the service node that has not synchronized data with the first master node, the data with a log sequence number less than that of the target request can be requested from the first master node according to the log sequence number of the target request, so as to obtain complete data and prevent the data inconsistency between the third service instance after splitting and the service instance before splitting.

[0171] To prevent the multiple service nodes from being inconsistent, the first master node sends a seal instruction to the slave nodes when the multiple service nodes are in the joint consensus state. The seal instruction is used to instruct the multiple service nodes to no longer receive write requests, that is, to no longer respond to the write requests, and there will be no new data written in the first service instance. The slave nodes can confirm the seal instruction. Specifically, the slave nodes receiving the seal instruction can store the seal instruction, such as persistently storing the seal instruction, for example, writing the seal instruction into a log, and the like. After the slave nodes store the seal instruction, the first master node returns a confirmation success message. When the first master node receives the confirmation success message returned by the slave nodes for the seal instruction within a set time length, it is determined that the slave nodes confirm the seal instruction, and the slave nodes can be taken as target service nodes (defined as first target service nodes).

[0172] Since the consensus protocol adopts the majority principle for master selection and request processing, in order to ensure the high availability of the distributed service system, a new master node can still be elected when the master node is abnormal, so that the state of the entire distributed service system can continue to advance. Therefore, the set number M corresponding to each of the multiple third service instances can be set. The description of the set number M can be referred to the related content of the foregoing embodiments, which will not be described here. Based on the pre-set set number M, the first master node can also count the number of the first target service nodes in the service nodes corresponding to each third service instance, and determine that the multiple service nodes confirm the seal instruction when the number of the first target service nodes corresponding to the multiple third service instances is greater than the set number M corresponding to each third service instance. In this way, after splitting into multiple third service instances C1-n, the state of the majority of the service nodes in the service nodes corresponding to each third service instance C1-n is consistent with the state of the master node before splitting, so that the data on the new service instance, that is, the third service instance C1-n, is not lost compared with the first service instance C0', and the data consistency before and after the service instance splitting is ensured.

[0173] Further, when the number of the first target service nodes corresponding to the multiple third service instances is greater than the set number M corresponding to each third service instance, the first master node can send a split instruction to the slave nodes. The split instruction is used to instruct the multiple service nodes to split the first service instance C0' into multiple third service instances C1-n.

[0174] Specifically, in a case where the number of the first target service nodes corresponding to the plurality of third service instances is greater than the respective corresponding set number M, the first master node can acquire the correspondence between the plurality of third service instances and the identities of the service nodes. The correspondence can be provided by the aforementioned operation and maintenance node to the first master node. Further, the first master node can generate a split instruction according to the correspondence between the plurality of third service instances and the identities of the service nodes. For example, the correspondence between the plurality of third service instances and the identities of the service nodes can be encapsulated into the split instruction. Further, the first master node can send the split instruction to the slave nodes.

[0175] Correspondingly, the first master node and the slave nodes receiving the split instruction can split the first service instance Cjoint in the joint consensus state into the plurality of third service instances C1-n in response to the split instruction.

[0176] Specifically, each of the first master node and the slave nodes receiving the split instruction can acquire the identity of the service node corresponding to the same third service instance according to the correspondence between the plurality of third service instances and the identities of the service nodes carried by the split instruction, and modify the stored member group identity to the identity of the service node corresponding to the same third service instance, thereby splitting the first service instance into the plurality of third service instances.

[0177] In the embodiment, in response to the instance split event, the service instance is expanded, and the expanded service instance is prevented from having new data written thereto by the write-prohibit instruction, and then the expanded service instance is split into a plurality of service instances, thereby realizing the splitting of the service instance. In the expansion stage, the original service node of the service instance can still provide services, and the data synchronization can be completed while the service continuity is ensured, and the expanded service instance is prevented from having new data written thereto, so that the plurality of service instances after the splitting start from the same state, and the data consistency of the service instances before and after the splitting can be ensured.

[0178] In addition, the entire process of the instance split scheme provided by the embodiments of the present disclosure does not need to introduce additional management and control dependencies, and the system architecture is simplified.

[0179] In some embodiments of the present disclosure, during the time period from the completion of the expansion of the service instance to the splitting of the service instance, the write-prohibit instruction makes the distributed service system service unavailable, and the time window of the service unavailability is long. In order to reduce the service unavailability window, in some embodiments, a joint consensus mechanism can be introduced, and the first master node can make the expanded service instance prevent new data from being written thereto by the write-prohibit instruction in a case where the first service instance C0' is in the joint consensus, and then split the first service instance into a plurality of third service instances by the split instruction, thereby realizing the splitting of the service instance.

[0180] The specific implementation of the joint consensus mechanism for the plurality of service nodes corresponding to the first service instance, the working process of the plurality of service nodes in the joint consensus state, and the specific implementation of the split instruction for splitting the first service instance into a plurality of third service instances can be found in the foregoing embodiments, and thus will not be described here.

[0181] In this embodiment, the original service nodes of the service instance can still provide services during the expansion phase, and data synchronization can be completed while ensuring service continuity. During the time period between the completion of the service instance expansion and the splitting of the service instance, the plurality of service nodes can be in a joint consensus state through the joint consensus mechanism, and the joint consensus mechanism can still be used to process new requests, thereby ensuring service continuity and maintaining data consistency while reducing the unavailability time window.

[0182] For the first master node, the number of service nodes corresponding to each third service instance that have confirmed the split instruction can also be counted. For ease of description and differentiation, the service nodes that have confirmed the split instruction are defined as fourth service nodes. Specifically, if the first master node receives a confirmation success message for the split instruction from a slave node within a set time period, it is determined that the slave node has confirmed the split instruction, and the slave node can be regarded as a fourth target service node. The specific implementation of the confirmation of the split instruction by the slave node can be found in the foregoing description of the confirmation of the split instruction by the slave node, and thus will not be described here.

[0183] In the joint consensus state, as long as more than half of the service nodes in each third service instance C1-n to be split have confirmed the split instruction, after the plurality of third service instances C1-n are split, each third service instance C1-n has a majority of service nodes providing services.

[0184] Based on this, for the plurality of third service instances to be split, the set number M corresponding to each third service instance can be set. The description of the set number M can be referred to the related content of the foregoing embodiments, which will not be described here. Based on the pre-set number M, the first master node can further count the number of fourth target service nodes in the service nodes corresponding to each third service instance, and in the case that the number of fourth target service nodes corresponding to the plurality of third service instances is greater than the respective set number M, it is determined that the plurality of service nodes confirm the splitting instruction, and then the first master node can modify the stored member group identifier to the identifier of the service node corresponding to the same third service instance as itself, so that the first master node exits the joint consensus state. In this way, as long as more than half of the service nodes in each third service instance C1-n to be split confirm the splitting instruction, after splitting into a plurality of third service instances C1-n, each third service instance C1-n has a majority of service nodes providing services, which can guarantee the continuity of services.

[0185] However, if the first master node abnormally occurs in the service instance splitting stage, that is, the first master node abnormally occurs in the process of sending the splitting instruction. It may cause some slave nodes not to receive the splitting instruction. If more than half of the service nodes corresponding to a certain third service instance or certain third service instances do not receive the splitting instruction, the splitting of the third service instance will fail. The service node that does not receive the splitting instruction also needs to learn the explicit global state from "somewhere", that is, the information that the global instance splitting has occurred.

[0186] In order to enable the slave node that does not receive the splitting instruction to also learn the information that the global instance splitting has occurred, in some embodiments of the present disclosure, the voting request for the service node to use the master can be improved. A target field is added to the traditional voting request. The target field is used to record whether the service cluster is in the joint consensus state. Among them, the first identifier is used to represent that the service cluster is in the joint consensus state, and the second identifier is used to represent that the service cluster is not in the joint consensus state. The first identifier and the second identifier are different identifiers, and the specific content of the two is not limited. For example, the first identifier can be "1", and the second identifier can be "0".

[0187] In the present embodiment, the first master node writes the first identifier into the target field when determining that the service cluster enters the joint consensus state, and stores the target field. The first master node can also send the first identifier to the slave node. The slave node can write the first identifier into the target field and store the target field. The first master node and the slave node that receives the splitting instruction can modify the value of the target field to the second identifier after executing the splitting instruction, so as to identify that it exits the joint consensus state.

[0188] Based on the target field, the slave node can reelect the master in the case of abnormality of the first master node by using the improved voting request. That is, the slave node can send a voting request to other service nodes belonging to the same member group according to the currently stored member group identifier to elect the master. The voting request includes the target field, and the voting request carries the value of the target field.

[0189] Specifically, the slave node encapsulates the value of the target field stored by itself into the voting request, and sends the voting request to other service nodes belonging to the same member group according to the currently stored member group identifier to elect the master. For the slave node that executes the split instruction, the stored member group identifier is the identifier of the service node corresponding to the same third service instance as itself, and the value of the target field is the second identifier representing the state of not being in the joint consensus. Even if there are still service nodes in the member group of the slave node that have not received the split instruction, the second identifier corresponding to the target field carried by the received voting request can be used to determine that the third service instance corresponding to the service node sending the voting request, and then the voting request is voted.

[0190] For the slave node that has not received the split instruction, the stored member group identifier is the identifier of the plurality of service nodes, and the value of the target field is the first identifier representing the state of not being in the joint consensus. The slave node sends a voting request to the service node corresponding to the member group identifier recorded by the slave node, and naturally sends the voting request to the fourth target service node that has received the split request.

[0191] If the fourth target service node that has received the split request receives the voting request, the value of the target field can be obtained from the voting request. If the value of the target field is the first identifier, it is determined that the slave node sending the voting request is the target slave node, and the target slave node is still in the joint consensus state. The fact that the first service instance C0' has split can be informed. Specifically, the correspondence between the plurality of third service instances and the identifiers of the service nodes can be returned to the target slave node. In this way, the target slave node can know the fact that the first service instance C0' has split according to the correspondence between the plurality of third service instances and the identifiers of the service nodes, and can determine the identifier of the service node corresponding to the third service instance corresponding to the target slave node according to the correspondence between the plurality of third service instances and the identifiers of the service nodes. Then, the stored member group identifier can be modified to the identifier of the service node corresponding to the third service instance corresponding to the target slave node, and the split of the service instance is realized.

[0192] In this embodiment, by strengthening the information of the voting request, the service node that has not received the split instruction can be supported, the global split information of the service instance can be learned from the service node that has received the split instruction, and the service node that has not received the split instruction can be guided to enter the split phase, and the split of the service instance is realized.

[0193] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can also be executed by different devices as the execution subject. For example, the execution subject of steps 901 and 902 can be device A; for another example, the execution subject of step 901 can be device A, and the execution subject of step 902 can be device B; and the like.

[0194] In addition, in some of the processes described in the above embodiments and the accompanying drawings, a plurality of operations appearing in a certain order are included, but it should be clearly understood that these operations can be executed in the order appearing in the text or in parallel, and the serial numbers of the operations, such as 901, 902, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel.

[0195] Correspondingly, the embodiments of the present disclosure also provide a computer readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the above example splitting method.

[0196] The embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by one or more processors, causes the one or more processors to perform the steps of the above example splitting method. In the embodiments of the present disclosure, the specific implementation form of the computer program product is not limited. In some embodiments, the computer program product can be implemented as an application (APP), a mini program, a computer side client, a program module, a plug-in, an installation package, a software development kit (SDK), an image file of an optical disc (such as an ISO file), a plug-in, or a software in the form of software as a service (SaaS), etc., but is not limited thereto.

[0197] FIG. 10 is a structural schematic diagram of an electronic device provided by the embodiments of the present disclosure. As shown in FIG. 10, the electronic device includes a memory 100a, a processor 100b, and a communication component 100c. The memory 100a is configured to store a computer program.

[0198] The processor 100b is coupled to the memory 100a and the communication component 100c, and is configured to execute the computer program to perform the steps of the example splitting method provided by the above embodiments. For the specific implementation of each step, please refer to the related description of the above embodiments, which will not be repeated here.

[0199] In some optional embodiments, as shown in FIG. 10, the electronic device can further include optional components such as a communication component 100c, a power supply component 100d, a display component 100e, an audio component 100f, and the like. Only some components are shown in FIG. 10, and it does not mean that the electronic device must include all components shown in FIG. 10, nor does it mean that the electronic device can only include the components shown in FIG. 10.

[0200] In addition, the components in the dashed box in FIG. 10 are optional components, not mandatory components, and can be determined according to the product form of the electronic device. The electronic device of the present embodiment can be implemented as a terminal device such as a desktop computer, a notebook computer, a mobile phone, or an Internet of Things device; or a server device such as a traditional server, a cloud server, or a server cluster.

[0201] In the embodiments of the present disclosure, the memory is used to store a computer program and can be configured to store other various data to support the operation on the device where it is located. The processor can execute the computer program stored in the memory to implement the corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Electrical Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0202] In the embodiments of the present disclosure, the processor can be any hardware processing device that can execute the logic of the above method. Optionally, the processor can be a central processing unit (CPU), a graphics processing unit (GPU), or a microcontroller unit (MCU); it can also be a programmable device such as a field-programmable gate array (FPGA), a programmable array logic (PAL), a general array logic (GAL), a complex programmable logic device (CPLD), etc.; or an advanced reduced instruction set compute (RISC) processor (Advanced RISC Machines, ARM) or a system on chip (SoC), etc., but is not limited thereto.

[0203] In the embodiments of the present disclosure, the communication component is configured to facilitate communication between the device where it is located and other devices in a wired or wireless manner. The device where the communication component is located can access a wireless network based on a communication standard, such as wireless fidelity (WiFi), 2G or 3G, 4G, 5G, or a combination thereof. In an example embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component can also be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra wide band (UWB) technology, Bluetooth (BT) technology, or other technologies.

[0204] In embodiments of the present disclosure, the display component can include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the display component includes the touch panel, the display component can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touch or a slide action, but also detect a duration and a pressure related to a touch or a slide operation.

[0205] In embodiments of the present disclosure, the power supply component is configured to provide power to various components of the device in which the power supply component is located. The power supply component can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0206] In embodiments of the present disclosure, the audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) that is configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in a memory or transmitted via the communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals. For example, for a device with a language interaction function, voice interaction with a user can be implemented through the audio component, etc.

[0207] It should be noted that the "first", "second", and the like descriptions herein are used to distinguish different messages, devices, modules, and the like, and do not represent a sequence, nor limit the "first" and "second" to be different types.

[0208] Those skilled in the art should understand that embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, Compact Disc Read-Only Memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0209] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0210] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks.

[0211] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0212] In one typical configuration, the computing device includes one or more processors (CPU's and the like), input / output interfaces, network interfaces, and memory.

[0213] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) for example, for storage of information and instructions to be executed by the processor. The memory can also include non-volatile memory, such as read only memory (ROM) and / or flash memory for storage of static information and instructions that are not to be changed, or require only moderate changes. The memory is an example of computer readable media.

[0214] The storage media of the computer is a readable storage media, also can be called readable media. The readable storage media includes permanent and non-permanent, movable and non-movable media, and can realize information storage by any method or technology. The information can be computer readable instructions, data structure, program module or other data. Examples of the storage media of the computer include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital video disc (DVD) or other optical storage, magnetic cassette, disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition in this paper, the computer readable medium does not include transitory computer readable media, such as modulated data signals and carriers.

[0215] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that the process, method, product or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, product or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, product or equipment including the above element.

[0216] The above is only an embodiment of the present disclosure and is not intended to limit the present disclosure. The present disclosure can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present disclosure shall be included in the scope of claims of the present disclosure.

Claims

1. A distributed service system, characterized by, The method comprises: a service cluster corresponding to a first service instance; the first service instance is obtained by expanding a second service instance in response to an instance splitting event; the service cluster comprises a plurality of service nodes independently providing a consistent coordination service; data synchronization of the plurality of service nodes is completed; a master node in the plurality of service nodes is configured to send a write-inhibition instruction to a slave node in the plurality of service nodes to indicate that the service cluster no longer receives a write request; and in a case where a number of first target service nodes corresponding to each of a plurality of third service instances to be split from the first service instance is greater than a corresponding set number, a splitting instruction is sent to the slave node; the first target service node is a service node in the plurality of service nodes that confirms the write-inhibition instruction; wherein a set number corresponding to any third service instance is greater than or equal to half of a total number of service nodes corresponding to the any third service instance and is less than the total number; the master node and the slave node that receives the splitting instruction are configured to split the first service instance into the plurality of third service instances in response to the splitting instruction.

2. The system of claim 1, wherein, The master node is further configured to: in a case where data synchronization of the plurality of service nodes is completed, modify a stored member group identifier to an identifier of the plurality of service nodes, and store a correspondence between the plurality of third service instances to be split and the identifiers of the service nodes; and send a joint instruction to the slave node; the slave node is configured to modify the stored member group identifier to the identifier of the plurality of service nodes and store the correspondence in response to the joint instruction, so that the plurality of service nodes enter a joint consensus state.

3. The system of claim 2, wherein, The master node is further configured to: in a case where a number of second target service nodes corresponding to each of the plurality of third service instances to be split is greater than a corresponding set number, determine that the plurality of service nodes enter the joint consensus state; the second target service node is a service node in the plurality of service nodes that successfully modifies the stored member group identifier to the identifier of the plurality of service nodes; for a target request received in the joint consensus state, forward the target request to the slave node; in a case where a number of third target service nodes corresponding to each of the plurality of third service instances to be split is greater than a corresponding set number, determine that the plurality of service nodes reach a consensus on the target request and process the target request; the third target service node is a service node in the plurality of service nodes that stores the target request.

4. The system of claim 3, wherein, The master node is further configured to: in a case where a number of fourth target service nodes corresponding to each of the plurality of third service instances to be split is greater than a corresponding set number, modify the stored member group identifier to an identifier of the plurality of first service nodes, so that the master node exits the joint consensus state; the fourth target service node is a service node in the plurality of service nodes that confirms the splitting instruction.

5. The system of claim 4, wherein, The master node is further configured to: in a case where it is determined that the plurality of service nodes enter a joint consensus state, send a first identifier representing that the plurality of service nodes are in the joint consensus state to the slave node; The slave node is configured to, in a case where the master node is abnormal, send a voting request to other service nodes belonging to the same member group as the slave node according to the currently stored member group identifier and the stored value of the target field, so as to select a master node; The voting request carries the value of the target field, and the target field is used to record whether the plurality of service nodes are in a joint consensus state; The fourth target service node is configured to, in a case where the voting request with the value of the target field being the first identifier is received, return the correspondence to a target slave node, the target slave node being the slave node sending the voting request with the value of the target field being the first identifier; The target slave node is configured to modify the stored member group identifier to an identifier of a service node corresponding to the same third service instance as the target slave node according to the correspondence.

6. The system according to any one of claims 1-5, characterized in that, The distributed service system further comprises an operation and maintenance node; the plurality of service nodes comprise a plurality of first service nodes corresponding to the first service instance before being scaled out, and a plurality of second service nodes scaled out in response to the instance split event; and the plurality of first service nodes comprise the master node; The operation and maintenance node is configured to, in response to the instance split event, scale out the first service instance to obtain the plurality of second service nodes, and instruct the plurality of first service nodes to synchronize data to the plurality of second service nodes; The master node is configured to send the stored data to the plurality of second service nodes to synchronize the data to the plurality of second service nodes.

7. An example splitting method, applicable to any service node in a plurality of service nodes corresponding to a first service instance, characterized in that, The first service instance is obtained by scaling out a second service instance in response to an instance split event, the plurality of service nodes independently provide consistent coordination services, and data synchronization is completed; The method comprises: In a case where the master node, send a write-inhibition instruction to a slave node in the plurality of service nodes to instruct the service cluster to no longer receive a write request; In a case where a number of first target service nodes corresponding to a plurality of third service instances to which the first service instance is to be split is greater than a corresponding set number, send a split instruction to the slave node, so that the master node and the slave node receiving the split instruction split the first service instance into a plurality of third service instances in response to the split instruction; The first target service node is a service node in the plurality of service nodes that confirms the write-inhibition instruction; the set number corresponding to any third service instance is greater than or equal to half of a total number of service nodes corresponding to the any third service instance, and is less than the total number.

8. The method of claim 7, wherein, The method further comprises: Obtaining a correspondence between the plurality of third service instances to be split and identifiers of service nodes; Generating the split instruction according to the correspondence.

9. The method according to claim 7 or 8, characterized in that, The master node splits the first service instance into a plurality of third service instances in response to the split instruction, comprising: obtaining the correspondence relationship from the split instruction; obtaining the identity of the service node corresponding to the third service instance of the same as the master node according to the correspondence relationship; modifying the stored member group identity to the identity of the service node corresponding to the third service instance of the same as the master node, so as to split the third service instance corresponding to the member group to which the master node belongs from the first service instance; wherein the service nodes belonging to the same member group correspond to the same third service instance.

10. The method according to any one of claims 7-9, characterized in that, The method further comprises: in the case where the data synchronization of the plurality of service nodes is completed, modifying the stored member group identity to the identity of the plurality of service nodes, and storing the correspondence relationship; sending a joint instruction to the slave node, so that the slave node modifies the stored member group identity to the identity of the plurality of service nodes and stores the correspondence relationship in response to the joint instruction, so that the plurality of service nodes enter a joint consensus state.

11. The method of claim 10, wherein, The method further comprises: in the case where the number of the second target service nodes corresponding to the plurality of third service instances to be split respectively is greater than the corresponding set number, determining that the plurality of service nodes enter the joint consensus state; the second target service node is a service node in the plurality of service nodes which successfully modifies the stored member group identity to the identity of the plurality of service nodes; forwarding the target request received in the joint consensus state to the slave node; in the case where the number of the third target service nodes corresponding to the plurality of third service instances to be split respectively is greater than the corresponding set number, determining that the plurality of service nodes reach consensus on the target request and processing the target request; the third target service node is a service node in the plurality of service nodes which stores the target request.

12. The method according to claim 10 or 11, characterized in that, The method further comprises: in the case where the number of the fourth target service nodes corresponding to the plurality of third service instances to be split respectively is greater than the corresponding set number, modifying the stored member group identity to the identity of the plurality of first service nodes, so that the master node exits the joint consensus state; the fourth target service node is a service node in the plurality of service nodes which confirms the split instruction.

13. The method according to any one of claims 10-12, characterized in that, The method further comprises: in the case where it is determined that the plurality of service nodes enter the joint consensus state, sending a first identity representing that the plurality of service nodes are in the joint consensus state to the slave node, so that the slave node writes the first identity into a target field stored; the target field is used to record whether the plurality of service nodes are in the joint consensus state; The slave node sends a voting request to other service nodes belonging to the same member group to select a master node according to the currently stored member group identifier and the stored value of the target field in the case of an exception of the master node; the voting request carries the value of the target field, so that a fourth target service node in the other service nodes returns the correspondence to a target slave node in the case of receiving the voting request with the value of the target field being the first identifier; the target slave node is a slave node sending the voting request with the value of the target field being the first identifier; The target slave node modifies the stored member group identifier to an identifier of a service node corresponding to the same third service instance as the slave node according to the correspondence.

14. An electronic device, comprising: Comprise: a memory, a processor and a communication component; wherein the memory is configured to store a computer program; The processor is coupled to the memory and the communication component, and is configured to execute the computer program to perform the steps in the method of any one of claims 7-13.

15. A computer readable storage medium having stored thereon computer instructions, wherein, When the computer instructions are executed by one or more processors, the one or more processors are caused to perform the steps in the method of any one of claims 7-13.

16. A computer program product, characterised in that, Comprise a computer program, when the computer program is executed by one or more processors, the one or more processors are caused to perform the steps in the method of any one of claims 7-13.

Images