Distributed KV cluster routing view updating method, system and program product

Abstract

The invention relates to the technical field of distributed software systems, and discloses a distributed KV cluster routing view updating method, a distributed KV cluster routing view updating system and a distributed KV cluster routing view updating program product. Bidirectional connection with a client is established through a backup node, so that the load is dispersed, the load pressure of a main control node is reduced, and the timeliness of information transmission is ensured. Furthermore, whether the local routing view of the local main control node needs to be updated or not is judged according to the received first routing view updating information, so that the consistency of the local view and the main control node view is further ensured. Meanwhile, unnecessary view updating operation is avoided, the running efficiency of the system is improved, and waste of system resources is reduced. Furthermore, the client can better adapt to the change of the cluster by updating the local routing view in time, so that the availability and the stability of the whole distributed KV cluster system are improved, and the high availability requirement of the system is met.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed software systems, and in particular to a distributed KV cluster routing view updating method, system and program product. Background Art

[0002] In a distributed KV (Key-Value) database, multiple servers hold databases (DBs), and data is distributed across these servers using hashes or ranges. To determine which server to send an update or delete request for a key, the client relies on a routing view to determine the service address.

[0003] View updates in distributed systems involve coordinated communication between multiple nodes. When a server undergoes a service update, migration, or failure, the routing view changes. This requires a robust messaging mechanism to communicate view changes to the client for updates. Traditional messaging systems, such as message queues and publish-subscribe systems like Kafka, can propagate view update messages, ensuring that all nodes in the system receive and apply view changes. However, these systems have significant drawbacks.

[0004] Traditional view updates rely primarily on clients sending requests to the master control node (MasterLeader). This places a heavy load on the backup node (MasterLeader). Furthermore, if the MasterSlave node doesn't send timely notifications, clients struggle to obtain the latest view information, resulting in significant update latency and failing to meet high system availability requirements. Summary of the Invention

[0005] In view of this, the present invention provides a distributed KV cluster routing view update method, system and program product to solve the problem that in the traditional view update method, the backup node is heavily loaded and the main control node does not notify the message in time, resulting in a long update delay and failing to meet the high availability requirements of the system.

[0006] In a first aspect, the present invention provides a distributed KV cluster routing view update method for any client, each client is connected to a server, and each client is integrated with a target thread; the method comprises:

[0007] When any client is in the non-listening view update state, the target thread is controlled to start and the target thread is used to detect the nodes in the KV cluster on the server. When the main control node and the backup node of the main control node in the KV cluster are detected, the first address information of the backup node is obtained, and the non-listening view update state is updated to the need to initiate listening state. After a two-way connection is established with the backup node based on the first address information, the need to initiate listening state is updated to the waiting to receive view update information state. When the first routing view update information of the main control node is received from the backup node, the first routing view update information is used to determine whether the local routing view of the local main control node needs to be updated. If the local routing view needs to be updated, the first routing view update information is used to update the local routing view to obtain the updated local routing view information.

[0008] The distributed KV cluster routing view update method provided by the present invention can detect nodes in the server-side KV cluster by actively starting a target thread when the client is in a non-listening view update state, so that the client can perceive cluster changes more quickly. Furthermore, after detecting the main control node and the backup node, the first address information of the backup node is obtained, which provides the necessary conditions for the client to establish a connection with the backup node. At the same time, updating the non-listening view update state to the need to initiate listening state helps the client to perform subsequent listening and update operations in an orderly manner, avoids confusion and disorder in operations, and improves the execution efficiency of the system. Furthermore, a two-way connection is established based on the first address information of the backup node, which disperses the load and reduces the load pressure on the main control node. At the same time, the need to initiate listening state is updated to the waiting to receive view update information state, so that the client can focus on receiving view update information, reducing the delay caused by waiting for view update, improving the response speed of the distributed KV cluster system, and meeting the high availability requirements of the system. Furthermore, through the two-way connection, the client can promptly receive the routing view update information sent by the backup node, ensuring the timeliness of information transmission. At the same time, based on the received first routing view update information, it can be determined whether the local routing view of the local master control node needs to be updated, which can further ensure the consistency of the local view with the master control node view. At the same time, unnecessary view update operations can be avoided, improving the operating efficiency of the system and reducing the waste of system resources. Furthermore, by timely updating the local routing view, the client can better adapt to changes in the cluster, ensuring that the client can still perform data operations normally when the cluster state changes, thereby helping to improve the availability and stability of the entire distributed KV cluster system and meet the high availability requirements of the system.

[0009] In an optional implementation, using the first routing view update information to determine whether the local routing view of the local master control node needs to be updated includes:

[0010] Obtaining local routing view information of the local master control node; determining whether the first routing view update information matches the local routing view information; when the first routing view update information does not match the local routing view information, determining that the local routing view needs to be updated; when the first routing view update information matches the local routing view information, determining that the local routing view does not need to be updated.

[0011] The distributed KV cluster routing view update method provided by the present invention can accurately determine whether the local view needs to be updated by obtaining local routing view information and matching it with the received update information, thereby avoiding unnecessary update operations, reducing the waste of system resources, improving the accuracy and efficiency of view updates, and ensuring that the client always has the latest and accurate view information.

[0012] In an optional embodiment, the method further includes:

[0013] When a change in the master control node in the KV cluster is detected, the second address information of the changed master control node is obtained; the local routing view is updated according to the second address information, the updated local routing view information is obtained, and a two-way connection is re-established with the backup node of the changed master control node.

[0014] The distributed KV cluster routing view updating method provided by the present invention, when a change in the main control node is detected, the client updates the local routing view by obtaining the changed address information of the main control node and re-establishes a two-way connection with the new backup node, so that the client can adapt to the change of the main control node in the cluster in a timely manner and maintain the connection with the backup node, thereby ensuring that the view update information can continue to be received after the main control node changes, ensuring the continuity and stability of the system when the main control node changes, reducing the update delay and data inconsistency problems caused by node changes, and further improving the high availability of the system.

[0015] In an optional embodiment, the method further includes:

[0016] When a master control node in a KV cluster is detected, the third address information of the master control node is obtained; an operation request containing local routing view information is sent to the server, so that the server determines whether the local routing view information matches the current routing view information of the master control node based on the operation request; when a routing view information mismatch instruction is received from the server, a two-way connection is established with the master control node based on the third address information, and an active update request is sent to the master control node; second routing view update information is received from the master control node, and the local routing view information is updated using the second routing view update information to obtain updated local routing view information.

[0017] The distributed KV cluster routing view update method provided by the present invention can proactively obtain the latest view from the master control node when the client receives a routing view information mismatch instruction sent by the server, avoiding the long delay caused by waiting for notification from the master control node, greatly reducing the update latency, improving the system's response speed, ensuring that the client can obtain accurate view information in a timely manner, and thus performing data operations more efficiently, thereby improving the overall performance and availability of the system.

[0018] In a second aspect, the present invention provides a distributed KV cluster routing view update method for a server connected to multiple clients; the method includes:

[0019] The method detects whether the routing view of the master control node in the KV cluster has changed; when the routing view has changed, determines whether a backup node of the master control node has established a bidirectional connection with multiple clients; when the backup node has established a bidirectional connection with multiple clients, uses the backup node to send first routing view update information of the master control node to multiple clients in a state of waiting to receive the view update information, so that the multiple clients in the state of waiting to receive the view update information determine local routing view information based on the first routing view update information.

[0020] The distributed KV cluster routing view update method provided by the present invention determines the connection status between the backup node and the client upon detecting a change in the routing view of the primary control node. The backup node then sends update information to the client waiting to receive view update information, distributing the task of propagating view update information to the backup node, thereby reducing the load on the primary control node. This method also ensures that the client can receive view update information in a timely manner, improving the efficiency of view update information propagation, enabling the client to more quickly obtain the latest view information and update local view information in a timely manner. This reduces client request errors and retries caused by view inconsistencies, thereby improving the system's response speed. Furthermore, it enables the client to more quickly adapt to cluster changes and maintain synchronization with the cluster state, thereby improving the availability and performance of the entire distributed KV cluster system.

[0021] In an optional embodiment, the method further includes:

[0022] When the main control node changes, it determines whether the backup node of the changed main control node receives the connection request of the client; when receiving the connection request of the client, it controls the backup node of the changed main control node to re-establish a two-way connection with the client.

[0023] The distributed KV cluster routing view update method provided by the present invention can, when the main control node changes, determine whether the backup node has received the client's connection request, and control the backup node to re-establish a two-way connection with the client when the request is received. This ensures that when the main control node changes, the connection between the client and the backup node can be restored and rebuilt in time, ensuring that the client can still obtain view update information through the backup node, improving the adaptability and recovery ability of the system when the node changes, ensuring the continuity of view update information transmission, and helping to improve the stability and reliability of the system.

[0024] In an optional embodiment, the method further includes:

[0025] Upon receiving an operation request from any client, the local routing view information of the client is determined according to the operation request; the local routing view information is matched with the current routing view information of the master control node; when the local routing view information does not match the current routing view information of the master control node, a routing view information mismatch instruction is sent to the corresponding client; upon receiving an active update request from the client, second routing view update information of the master control node is sent to the client according to the active update request, so that the client updates the local routing view information using the second routing view update information, thereby obtaining updated local routing view information.

[0026] The distributed KV cluster routing view update method provided by the present invention enables the server to determine whether the client's local view matches the master control node's current view upon receiving a client operation request, and perform corresponding processing based on the matching result. This effectively manages client view updates and promptly detects and resolves view inconsistencies. Compared with traditional methods, this method avoids data operation errors caused by view inconsistencies and improves the accuracy of data operations. Furthermore, through interaction with the client, timely view updates are achieved, reducing update latency, improving the system's response speed and overall performance, and meeting the system's high availability requirements.

[0027] In a third aspect, the present invention provides a distributed KV cluster routing view update system, comprising: a server and multiple clients, each client being connected to the server, and each client being integrated with a target thread; each client being used to execute the distributed KV cluster routing view update method of the above-mentioned first aspect or any corresponding embodiment thereof; and the server being used to execute the distributed KV cluster routing view update method of the above-mentioned second aspect or any corresponding embodiment thereof.

[0028] The distributed KV cluster routing view update system provided by the present invention solves the problem that the backup node is heavily loaded and the main control node fails to notify the message in time, resulting in a long update delay and failing to meet the high availability requirements of the system.

[0029] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the distributed KV cluster routing view update method of the above-mentioned first aspect or any corresponding embodiment thereof, or the distributed KV cluster routing view update method of the above-mentioned second aspect or any corresponding embodiment thereof.

[0030] In a fifth aspect, the present invention provides a computer program product comprising computer instructions, the computer instructions being used to enable a computer to execute the distributed KV cluster routing view update method of the above-mentioned first aspect or any corresponding embodiment thereof, or the distributed KV cluster routing view update method of the above-mentioned second aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 is a structural block diagram of a distributed KV cluster routing view update system according to an embodiment of the present invention;

[0033] Figure 2 1 is a flow chart of a distributed KV cluster routing view updating method according to an embodiment of the present invention;

[0034] Figure 3 1 is a flow chart of another distributed KV cluster routing view updating method according to an embodiment of the present invention;

[0035] Figure 4 This is a flow chart of establishing a stream connection by a client according to an embodiment of the present invention;

[0036] Figure 5 This is an overall flow chart of a client executing a request according to an embodiment of the present invention;

[0037] Figure 6 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0039] An embodiment of the present invention provides a distributed KV cluster routing view update method, which establishes a connection with a backup node and receives routing view update information, and determines whether an update is required based on the received routing view update information, thereby reducing the load pressure on the main control node, reducing the delay caused by waiting for view updates, improving the response speed of the distributed KV cluster system, and meeting the high availability requirements of the system.

[0040] According to an embodiment of the present invention, an embodiment of a distributed KV cluster routing view update method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0041] In this embodiment, a distributed KV cluster routing view update method is provided for Figure 1 In any client 11 shown, each client 11 is connected to the server 12, and each client 11 is integrated with a target thread.

[0042] Figure 2 is a flow chart of a distributed KV cluster routing view updating method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0043] Step S201: When any client is in a state of not monitoring view updates, the target thread is controlled to start and the target thread is used to detect nodes in the KV cluster in the server.

[0044] Specifically, the client 11 is in the non-watching view update state in the initial state or certain specific situations (such as just started, abnormal connection recovery, etc.). In this embodiment, the client 11 is set to the initial state "NO_WATCH", that is, the non-watching view update state.

[0045] Furthermore, in order to promptly obtain routing view update information of the KV cluster in the server 12, the client 11 needs to proactively learn about the cluster status. Therefore, in this embodiment, when the client 11 is not monitoring view updates, the target thread is controlled to start.

[0046] Furthermore, after starting the target thread, the target thread can send a probe request to the server 12 via a network protocol (such as TCP, UDP, etc.) to attempt to discover nodes in the KV cluster. For example, the target thread can send a node query request to a specific port of the server 12. After receiving the request, the server 12 returns node information, and the client 11 determines the existence and status of the node based on the returned information.

[0047] By starting the target thread, the client 11 can periodically or continuously detect the nodes in the server KV cluster to discover the existence of the master control node (Master Leader) and its backup node (Master Slave) in the KV cluster.

[0048] Step S202: When the master control node and the backup node of the master control node in the KV cluster are detected, the first address information of the backup node is obtained, and the view update state is updated from the non-monitoring state to the need-to-initiate-monitoring state.

[0049] Among them, the master control node (Master Leader) represents the node that plays the main control and management role in the entire distributed KV cluster; the backup node (Master Slave), also known as the slave node or standby node, mainly serves as a backup for the master control node.

[0050] Specifically, after the client 11 successfully detects the master control node and its corresponding backup node in the KV cluster, the address of the backup node (such as IP address and port number) can be extracted from the node information returned by the server 12 as the first address information.

[0051] Furthermore, the client 11 may update its own status flag from "not watching view updates (NO_WATCH)" to "need to initiate watching (NEED_WATCH)", indicating that the client is ready to actively initiate a connection with the backup node to watch for view updates.

[0052] Step S203: After a bidirectional connection is established with the backup node based on the first address information, the update needs to initiate a listening state to a state of waiting to receive view update information.

[0053] Specifically, the client 11 may use the acquired first address information of the backup node and a network programming interface (such as Socket programming) to try to establish a connection with the backup node.

[0054] During the connection process, the client 11 and the backup node may perform a handshake protocol to ensure that both parties can communicate normally.

[0055] Furthermore, when client 11 successfully establishes a bidirectional connection with the backup node, it can now receive routing view updates from the backup node and can also send confirmation messages or other necessary requests to the backup node. Furthermore, this bidirectional connection distributes the load, reducing the pressure on the primary control node.

[0056] Furthermore, the status of client 11 can be changed from "NEED_WATCH" to "WATCHED," indicating that client 11 begins monitoring data sent by the backup node. Furthermore, in the "WATCHED" state, client 11 can focus on receiving view updates, reducing the latency caused by waiting for view updates, improving the responsiveness of the distributed KV cluster system, and meeting the high availability requirements of the system.

[0057] Step S204: upon receiving the first routing view update information of the master control node sent by the backup node, the backup node uses the first routing view update information to determine whether the local routing view of the local master control node needs to be updated.

[0058] Specifically, since the view information of the backup node and the main control node remains consistent, when the client 11 is in the "waiting to receive view update information state", if the routing view of the main control node in the KV cluster changes, the backup node can send the first routing view update information of the main control node to the client through the two-way connection channel established with the client 11.

[0059] Furthermore, the client 11 may further determine whether the local routing view of the local master control node needs to be updated based on the received first routing view update information, thereby further ensuring the consistency between the local view and the master control node view through the determination.

[0060] Step S205: When the local routing view needs to be updated, the local routing view is updated using the first routing view update information to obtain updated local routing view information.

[0061] If it is determined that the local routing view needs to be updated, the client 11 may replace or modify the locally stored routing view according to the received first routing view update information to ensure that the local view is consistent with the view of the master control node. Figure 1 Therefore, data requests can be accurately routed to the correct service node.

[0062] Specifically, when it is determined that the local routing view needs to be updated, the client 11 may parse the received first routing view update information and extract the routing rules, node information and other contents therein.

[0063] Then, the client 11 may update the corresponding content in the local routing view according to the parsed content, such as directly replacing the corresponding content in the local routing view with the corresponding content in the parsed content, or may merge or modify the corresponding content in the local routing view according to the parsed content.

[0064] Finally, after the update is completed, the client 11 can obtain the updated local routing view information.

[0065] Furthermore, if it is determined through judgment that the local routing view does not need to be updated, the original local routing view information is still used as the current local routing view information.

[0066] The distributed KV cluster routing view update method provided in this embodiment, when the client is in the non-listening view update state, actively initiates a target thread to detect nodes within the server-side KV cluster, enabling the client to more quickly perceive cluster changes. Furthermore, after detecting the primary control node and backup node, the first address information of the backup node is obtained, providing the necessary conditions for the client to establish a connection with the backup node. Simultaneously, updating the non-listening view update state to the need-listening state helps the client execute subsequent listening and update operations in an orderly manner, avoiding confusion and disorder in operations and improving the execution efficiency of the system. Furthermore, establishing a bidirectional connection based on the backup node's first address information distributes the load and reduces the load pressure on the primary control node. Simultaneously, updating the need-listening state to the waiting-to-receive view update information state allows the client to focus on receiving view update information, reducing the delay caused by waiting for view updates, improving the responsiveness of the distributed KV cluster system, and meeting the system's high availability requirements. Furthermore, through the bidirectional connection, the client can promptly receive routing view update information sent by the backup node, ensuring the timeliness of information transmission. At the same time, based on the received first routing view update information, it can be determined whether the local routing view of the local master control node needs to be updated, which can further ensure the consistency of the local view with the master control node view. At the same time, unnecessary view update operations can be avoided, improving the operating efficiency of the system and reducing the waste of system resources. Furthermore, by timely updating the local routing view, the client can better adapt to changes in the cluster, ensuring that the client can still perform data operations normally when the cluster state changes, thereby helping to improve the availability and stability of the entire distributed KV cluster system and meet the high availability requirements of the system.

[0067] In some optional implementations, determining whether the local routing view of the local master control node needs to be updated by using the first routing view update information in step S204 includes:

[0068] Step a1: Acquire local routing view information of the local main control node.

[0069] Step a2: Determine whether the first routing view update information matches the local routing view information.

[0070] Step a3: When the first routing view update information does not match the local routing view information, it is determined that the local routing view needs to be updated.

[0071] Step a4: When the first routing view update information matches the local routing view information, it is determined that the local routing view does not need to be updated.

[0072] Specifically, the client 11 locally stores routing view information about the local master control node. This information records the cluster routing situation currently known by the client 11, such as the location of each node, data distribution, and the like.

[0073] After receiving the first routing view update information sent by the backup node in the KV cluster, the client 11 first reads the local routing view information of the local master control node from the local storage.

[0074] Secondly, the acquired local routing view information can be compared with the received first routing view update information, such as version number comparison, data content comparison, etc. Furthermore, the version number can intuitively reflect the update status of the view. Therefore, in this embodiment, the version number is used as the main judgment basis.

[0075] If the version number in the first routing view update information is higher than the version number in the local routing view information, it means that the routing view of the server 12 has changed, but the local view of the client 11 has not kept up with it in time. At this time, in order to ensure that the client can correctly interact with the cluster and obtain the latest data and services, the local routing view needs to be updated. Otherwise, it means that the local routing view is already the latest and does not need to be updated, which avoids unnecessary update operations, reduces the waste of system resources, improves the accuracy and efficiency of view updates, and ensures that the client always has the latest and accurate view information.

[0076] In some optional embodiments, the above method further includes the following steps:

[0077] Step b1: When a change in the main control node in the KV cluster is detected, the second address information of the changed main control node is obtained.

[0078] Step b2: updating the local routing view according to the second address information, obtaining updated local routing view information, and re-establishing a bidirectional connection with the backup node of the changed master control node.

[0079] Specifically, when the client 11 uses the target thread to detect the nodes in the KV cluster in the server 12, if it is found that a cluster master switching event occurs in the KV cluster, that is, the main control node in the KV cluster has changed, the local routing view of the local control node needs to be updated in time.

[0080] First, the client 11 obtains the second address information of the changed master control node in the KV cluster. The specific process can be referred to the description of the acquisition of the first address information in the above steps S201 and S202, which will not be repeated here.

[0081] Next, client 11 can update the local routing view based on the obtained second address information. Specifically, the local routing view records the client's knowledge of each node in the cluster and related information about data routing. Therefore, the client can integrate the obtained second address information into the local routing view, thereby obtaining updated local routing view information.

[0082] When integrating the second address information into the local routing view, other related routing rules, node relationships and other information may also be adjusted accordingly.

[0083] Finally, after updating the local routing view, since the main control node in the KV cluster changes, its corresponding backup node also changes. At this time, the client 11 will cut off the two-way connection established with the previous backup node and return to the above step S202, repeating the operations of steps S202 to S203 to enable the client 11 to re-establish a two-way connection with the backup node corresponding to the changed main control node in the KV cluster in the server 12.

[0084] By re-establishing a two-way connection, the client 11 can promptly adapt to changes in the master control node in the KV cluster in the server 12 and maintain a connection with its backup node, thereby ensuring that the view update information can continue to be received after the master control node in the KV cluster changes, ensuring the continuity and stability of the KV cluster system when the master control node changes, reducing the update delay and data inconsistency problems caused by node changes, and further improving the high availability of the KV cluster system.

[0085] In some optional embodiments, the above method further includes the following steps:

[0086] Step c1: When a main control node in the KV cluster is detected, third address information of the main control node is obtained.

[0087] Step c2: sending an operation request including the local routing view information to the server, so that the server determines whether the local routing view information matches the current routing view information of the main control node based on the operation request.

[0088] Step c3: upon receiving the routing view information mismatch instruction sent by the server, a bidirectional connection is established with the main control node based on the third address information, and an active update request is sent to the main control node.

[0089] Step c4: receiving the second routing view update information sent by the main control node, and using the second routing view update information to update the local routing view information, to obtain updated local routing view information.

[0090] Specifically, when the client 11 detects nodes in the KV cluster of the server 12 using the target thread, if a master control node in the KV cluster is detected, the third address information of the master control node is obtained. The specific process can be referred to the relevant description of the acquisition of the first address information in steps S201 and S202 above, and will not be repeated here.

[0091] Furthermore, when the client 11 performs a specific operation (such as reading, writing, updating or deleting data on the distributed KV cluster), the locally stored routing view information can be attached to the operation request.

[0092] Furthermore, after the client 11 sends an operation request containing local routing view information to the server 12, the server 12 can extract the corresponding local routing view information based on the received operation request, and further determine whether the local routing view information matches the current routing view information of the main control node in the distributed KV cluster.

[0093] Furthermore, if the local routing view information does not match the current routing view information of the master control node in the distributed KV cluster, the server 12 may send a routing view information mismatch instruction to the client 11 .

[0094] After receiving the routing view information mismatch instruction, client 11 may determine that the current local routing view information is outdated and needs to be updated. At this time, client 11 may attempt to establish a bidirectional connection with the master control node through a network programming interface (such as Socket programming) and a network protocol (such as TCP) based on the third address information of the master control node obtained.

[0095] Furthermore, after successfully establishing a bidirectional connection with the master control node in the distributed KV cluster, client 11 can send an active update request to the master control node, explicitly informing the master control node that it needs to obtain the latest routing view information in order to update its local routing view. By sending an active update request, the long delay caused by waiting for notification from the master control node is avoided, significantly reducing update latency and improving system responsiveness.

[0096] Furthermore, after receiving the active update request from the client, the main control node may send the latest routing view information (ie, the second routing view update information) to the client 11 .

[0097] Finally, after receiving the second routing view update information, the client 11 may parse the received second routing view update information and update the locally stored routing view information according to the parsed information to obtain updated local routing view information.

[0098] The updating method may refer to the updating method in the above step S205 and will not be described in detail here.

[0099] Through the above process, the local routing view is ensured to be consistent with the current routing view of the main control node, so that the client can accurately route data requests to the correct node in subsequent operations, ensuring the normal operation of the distributed KV cluster system and improving the overall performance and availability of the system.

[0100] In this embodiment, a distributed KV cluster routing view update method is provided for Figure 1 The server 12 is shown, and the server 12 is connected to multiple clients 11.

[0101] Figure 3 is a flow chart of a distributed KV cluster routing view updating method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0102] Step S301: Detect whether the routing view of the main control node in the KV cluster has changed.

[0103] Specifically, the server 12 can detect in real time whether the routing view of the main control node in the KV cluster changes.

[0104] For example, the detection mechanism within the server 12 may periodically check routing view related data (such as version number, node information, data distribution configuration, etc.) on the main control node to determine whether the routing view has changed.

[0105] Alternatively, the server 12 may also adopt an event-driven approach. When an operation occurs on the main control node that may cause a change in the routing view (such as adding a node, deleting a node, migrating data, etc.), the main control node may send a corresponding event notification to the server, and the server 12 may immediately check the routing view after receiving the notification.

[0106] Step S302: When the routing view changes, it is determined whether the backup node of the main control node has established bidirectional connections with multiple clients.

[0107] Specifically, in order to reduce the load pressure of the main control node, the server 12 can send the routing view information to the client 11 in a timely manner through the backup node of the main control node in the KV cluster.

[0108] Therefore, when the server 12 detects that the routing view of the primary control node has changed, in order to ensure that the updated information can be reliably transmitted to the client 11 through the backup node, it is necessary to determine the connection status between the backup node of the current primary control node and the client 11.

[0109] In step S303, when the backup node establishes a bidirectional connection with multiple clients, the backup node sends first routing view update information from the primary control node to the multiple clients in a state of waiting to receive the view update information, so that the multiple clients in the state of waiting to receive the view update information determine local routing view information based on the first routing view update information.

[0110] Specifically, if it is determined that the backup node has successfully established bidirectional connections with multiple clients, the backup node may send the first routing view update information after the change of the primary control node to each client 11 in the "WATCHED" state.

[0111] Furthermore, after receiving the first routing view update information, the client 11 can determine whether the local routing view information needs to be updated based on the first routing view update information and determine the final local routing view information. The specific process can be referred to the update process description of step S205 and steps a1 to a4 above, and will not be repeated here.

[0112] The distributed KV cluster routing view update method provided in this embodiment, upon detecting a change in the routing view of the primary control node, determines the connection status between the backup node and the client, and sends update information via the backup node to clients waiting to receive view update information. This distributes the task of propagating view update information to the backup node, reducing the load on the primary control node. At the same time, it ensures that clients can receive view update information in a timely manner, improving the efficiency of view update information propagation, enabling clients to more quickly obtain the latest view information and update local view information in a timely manner, reducing client request errors and retries caused by view inconsistencies, thereby improving the system's response speed. Furthermore, it enables clients to adapt to cluster changes more quickly and maintain synchronization with the cluster state, thereby improving the availability and performance of the entire distributed KV cluster system.

[0113] In some optional embodiments, the above method further includes the following steps:

[0114] Step d1: When the main control node changes, it is determined whether the backup node of the changed main control node receives a connection request from the client.

[0115] Step d2: upon receiving a connection request from the client, the backup node of the primary control node whose control has changed re-establishes a two-way connection with the client.

[0116] Specifically, when the server 12 detects that the main control node in the KV cluster has changed, in order to ensure the continuity and stability of the KV cluster system when the main control node changes, and reduce the update delay and data inconsistency problems caused by node changes, it is necessary to determine whether the backup node of the changed main control node has received the client's connection request.

[0117] The backup node may continuously monitor the network port and wait for the connection request from the client 11 .

[0118] Furthermore, when the server 12 confirms that the backup node has received the client's connection request, it can control the backup node to re-establish a two-way connection with the client 11. The specific process can refer to the relevant descriptions in the above steps b1 to b2, which will not be repeated here.

[0119] By re-establishing a two-way connection, it is ensured that the connection between the client and the backup node can be restored and rebuilt in time when the main control node changes, ensuring that the client can still obtain view update information through the backup node, improving the system's adaptability and recovery capabilities when nodes change, ensuring the continuity of view update information transmission, and helping to improve the stability and reliability of the system.

[0120] In some optional embodiments, the above method further includes the following steps:

[0121] Step e1: When an operation request sent by any client is received, local routing view information of the client is determined according to the operation request.

[0122] Step e2: Match the local routing view information with the current routing view information of the main control node.

[0123] Step e3: When the local routing view information does not match the current routing view information of the main control node, a routing view information mismatch instruction is sent to the corresponding client.

[0124] In step e4, upon receiving the active update request sent by the client, the master control node sends the second routing view update information to the client according to the active update request, so that the client updates the local routing view information using the second routing view update information to obtain the updated local routing view information.

[0125] Specifically, the server 12 may monitor the network port at all times, waiting for the client 11 to send an operation request.

[0126] Furthermore, when the client 11 needs to read, write, or update data in the distributed KV cluster, it can send an operation request to the server. In addition to containing specific operation instructions (such as reading the value of a key, writing a key-value pair, etc.), the operation request can also carry the client's current local routing view information.

[0127] Furthermore, after receiving the operation request sent by the client 11 , the server 12 may parse the operation request and extract the corresponding local routing view information.

[0128] Furthermore, after obtaining the local routing view information of the client 11 , the server 12 may obtain its current routing view information from the main control node in the KV cluster.

[0129] Furthermore, the local routing view information of the client 11 is matched with the current routing view information of the master control node, and a corresponding instruction can be sent to the client 11 based on the matching result. For example, if the local routing view information does not match the current routing view information of the master control node, a routing view information mismatch instruction can be sent to the corresponding client 11; otherwise, a corresponding routing view information match instruction can be sent to the corresponding client 11.

[0130] Furthermore, the client 11 can process the local routing view information according to the received instruction. Specifically, if the client 11 receives an instruction indicating that the view information does not match, it means that the current local routing view information is outdated and needs to be updated. At this time, the client 11 sends an active update request to the server 12.

[0131] Furthermore, after receiving the active update request from client 11, server 12 can send the current routing view information of the master control node in the KV cluster, i.e., the second routing view update information, to client 11, thereby causing client 11 to update its local routing view information based on the received second routing view update information and obtain the updated local routing view information. The update method can refer to the update method in step S205 above and will not be repeated here.

[0132] By determining whether the client's local view matches the master control node's current view and taking appropriate action based on the matching result, this approach effectively manages client view updates and promptly detects and resolves view inconsistencies. Compared to traditional approaches, this approach avoids data manipulation errors caused by view inconsistencies and improves data manipulation accuracy. Furthermore, through interaction with the client, timely view updates are achieved, reducing update latency, improving system responsiveness and overall performance, and meeting high-availability requirements.

[0133] In this embodiment, a distributed KV cluster routing view update system is provided, such as Figure 1 As shown, the distributed KV cluster routing view update system 1 includes a client 11 and a server 12 .

[0134] In this embodiment, there is no specific limitation on the number of clients 11, which can be set according to actual needs.

[0135] Furthermore, each client 11 is connected to the server 12 , and each client 11 is integrated with a corresponding target thread.

[0136] Specifically, each client 11 is used to execute the distributed KV cluster routing view updating method for the client 11 provided in the above embodiment of the present invention.

[0137] The server 12 is used to execute the distributed KV cluster routing view updating method for the server 12 provided in the above embodiment of the present invention.

[0138] The distributed KV cluster routing view update system provided in this embodiment solves the problem that the backup node is heavily loaded and the main control node fails to notify the message in a timely manner, resulting in a long update delay and failing to meet the high availability requirements of the system. By executing the corresponding distributed KV cluster routing view update method on the client and the server respectively,

[0139] In one example, Figure 4 and Figure 5 As shown, a distributed KV cluster routing view update method is provided, including the following steps:

[0140] Step 1: The client establishes the initial state "NO_WATCH" (not listening for view updates), waits for the MasterLeader to be determined, and connects to the MasterSlave.

[0141] Step 2: The client starts a timer thread to detect changes in the MasterLeader and Slave identities and determine the addresses of the leader and slave. If the timer thread detects the emergence of a MasterLeader or a master switch, it updates the routing information of the local master and notifies the client to reestablish the stream connection.

[0142] Step 3: When the client determines the identities of the leader and slave or the cluster switches to the master, the state changes to "NEED_WATCH" (need to initiate monitoring), and a two-way stream connection is initiated to the MasterSlave to monitor view updates.

[0143] Step 4: After the client establishes a connection with the MasterSlave, the status changes to "WATCHED" and waits to receive view update information.

[0144] Step 5: The views of the MasterSlave and Leader nodes are consistent. When the routing view of a service changes, the Slave node notifies all listening clients via a bidirectional streaming connection. Upon receiving the change message, the client updates its local view. Unlike traditional update methods, this allows users to subscribe to changes in the MasterSlave view, reducing the load on the Leader node.

[0145] Step 6: When the client initiates a request to the service, if it finds that the view version does not match, it immediately initiates a request to the MasterLeader to actively update the view to reduce latency.

[0146] In some optional implementations, the above-mentioned distributed KV cluster routing view update method specifically includes the following steps:

[0147] Step 1: Create a client and initialize the listener state to "NO_WATCH". Start a scheduled thread to poll the Master Leader and Master Slave in the current cluster. Set the local listener state to "NEED_WATCH".

[0148] Step 2.1: After the scheduled thread completes polling, it determines the current leader and slave, and the client initiates a bidirectional streaming connection to the slave. If the scheduled thread detects a cluster leader switch, it changes the listening state to "NEED_WATCH," closes the currently changed slave connection, and re-initiates the streaming connection.

[0149] Step 2.2: After the client establishes a bidirectional streaming connection, it sets the listening state to "WATCHED" and waits for the view change message notification from the MasterSlave.

[0150] Step 2.3: After receiving the view change message, the client will compare the local view version number. If the version number in the message is higher than the local version, it will be updated.

[0151] Step 3.1: If a master switch occurs in the cluster, the new master slave closes all currently established stream connections. If the leader becomes a slave, it needs to wait for the client to initiate a bidirectional stream connection.

[0152] Step 3.2: After the client sends a service request, the server compares the view version information. If it lags behind the current view version, it returns an error code. After the client detects the error code carried in the response body of the request, it initiates an active request to the MasterLeader to update the local view information.

[0153] Step 4.1: If the client is destroyed, the established bidirectional stream connection is actively closed.

[0154] This example provides a distributed KV cluster routing view update method that reduces the load on the master leader by establishing a bidirectional streaming connection with the master slave to subscribe to view change information. Furthermore, if the master slave fails to provide timely notifications, the client can proactively send a view update request to the master leader to obtain the latest view information from the leader, reducing update latency. Therefore, this example reduces the load on the master leader in active-standby mode and reduces the latency of view message transmission. Furthermore, by combining active and passive updates, high availability is achieved.

[0155] The embodiment of the present invention also provides a computer device for executing the above Figures 2 to 3 The distributed KV cluster routing view update method shown.

[0156] See also Figure 6 , Figure 6is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of a GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0157] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0158] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0159] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0160] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0161] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0162] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0163] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0164] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A distributed KV cluster routing view update method, characterized in that: For any client, each client is connected to the server, and each client is integrated with a target thread; the method includes: When any of the clients is in a state of not listening to view updates, controlling the target thread to start and using the target thread to detect nodes in the KV cluster in the server; When a master control node and a backup node of the master control node in the KV cluster are detected, first address information of the backup node is obtained, and the non-listening view update state is updated to a listening state; After a bidirectional connection is established with the backup node based on the first address information, the state of needing to initiate monitoring is updated to a state of waiting to receive view update information; When receiving the first routing view update information of the master control node sent by the backup node, determining whether the local routing view of the local master control node needs to be updated by using the first routing view update information; When the local routing view needs to be updated, the local routing view is updated using the first routing view update information to obtain updated local routing view information.

2. The method according to claim 1, characterized in that Determining whether the local routing view of the local master control node needs to be updated by using the first routing view update information includes: Obtaining local routing view information of the local master control node; Determining whether the first routing view update information matches the local routing view information; When the first routing view update information does not match the local routing view information, determining that the local routing view needs to be updated; When the first routing view update information matches the local routing view information, it is determined that the local routing view does not need to be updated.

3. The method according to claim 1, characterized in that The method further comprises: When detecting that the master control node in the KV cluster changes, obtaining second address information of the changed master control node; The local routing view is updated according to the second address information to obtain updated local routing view information, and a bidirectional connection is re-established with the backup node of the changed main control node.

4. The method according to claim 1 or 3, characterized in that The method further comprises: When the master control node in the KV cluster is detected, obtaining third address information of the master control node; Sending an operation request including the local routing view information to the server, so that the server determines whether the local routing view information matches the current routing view information of the master control node based on the operation request; When receiving the routing view information mismatch instruction sent by the server, establishing a bidirectional connection with the main control node based on the third address information, and sending an active update request to the main control node; The second routing view update information sent by the main control node is received, and the local routing view information is updated using the second routing view update information to obtain updated local routing view information.

5. A distributed KV cluster routing view update method, characterized in that: Used for a server, the server is connected to multiple clients; the method includes: Check whether the routing view of the master control node in the KV cluster has changed; When the routing view changes, determining whether a backup node of the master control node has established a bidirectional connection with the multiple clients; When the backup node establishes a bidirectional connection with the multiple clients, the backup node sends first routing view update information of the primary control node to the multiple clients in a state of waiting to receive the view update information, so that the multiple clients in the state of waiting to receive the view update information determine local routing view information based on the first routing view update information.

6. The method according to claim 5, characterized in that The method further comprises: When the main control node changes, determining whether the backup node of the changed main control node receives the connection request of the client; When receiving the connection request from the client, the backup node of the primary control node whose control has changed re-establishes a bidirectional connection with the client.

7. The method according to claim 5 or 6, characterized in that The method further comprises: When receiving an operation request sent by any client, determining the local routing view information of the client according to the operation request; Matching the local routing view information with the current routing view information of the master control node; When the local routing view information does not match the current routing view information of the master control node, sending a routing view information mismatch instruction to the corresponding client; When receiving the active update request sent by the client, the second routing view update information of the main control node is sent to the client according to the active update request, so that the client updates the local routing view information using the second routing view update information to obtain the updated local routing view information.

8. A distributed KV cluster routing view update system, characterized in that: The system comprises: a server and a plurality of clients, each client being connected to the server, and each client being integrated with a target thread; Each client is configured to execute the distributed KV cluster routing view updating method according to any one of claims 1 to 4; The server is used to execute the distributed KV cluster routing view updating method according to any one of claims 5 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the distributed KV cluster routing view updating method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the distributed KV cluster routing view updating method according to any one of claims 1 to 7.

Images