Dynamic node transfer method

Abstract

The invention provides a dynamic node transfer method, which comprises the following steps of: establishing connection between a server and a Zookeeper, executing S2 when the server is online, and executing S5 when the server is offline; registering a data node containing a current machine I P under a specified directory of the Zookeeper; monitoring the change of the data node under the specified directory through the Zookeeper; based on the Zookeeper, the server can perform data interaction and state synchronization with the Zookeeper, so that when the machine is online or offline, the capability of automatically transferring mapping between the machine and a driver is provided, the risk of data loss can be effectively reduced, the requirement for manual configuration is reduced, the relationship between the IP and node mapping does not need to be manually maintained, and the working efficiency is improved. The problem of online data loss caused by configuration errors due to manual omission is avoided, and the probability of occurrence of online problems is reduced.

Description

Technical Field

[0001] The present invention relates to a method for dynamic node transfer, specifically a method for dynamic node transfer based on Zookeeper, belonging to the technical field of data processing. Background Art

[0002] Zookeeper: It is a distributed application coordination service and uses a data model based on the familiar file system directory tree structure; it is mainly used to provide consistency services for distributed applications. It is a project of the Apache Software Foundation and is usually used to maintain configuration information, naming services, distributed synchronization, and provide group services.

[0003] Node: A service may consist of multiple machines forming a cluster, and each of these multiple machines is simply referred to as a node.

[0004] Service online / offline: It refers to the application on the machine starting to provide services or stopping to provide services.

[0005] Online car-hailing is a mode of transportation service based on Internet technology. Users can reserve vehicles through an application (App) on their smartphones. The vehicles providing services are usually driven by private car owners or professional drivers; with the continuous development of online car-hailing, the industry vitality has gradually increased, and more and more people have joined the online car-hailing industry. However, it is found during the operation of the online car-hailing server system that some data needs to be calculated in the server memory, such as the driver's order information. Then, the data of the same driver needs to be sent to the same machine. When the machine crashes or restarts, the driver data sent to this machine will be lost.

[0006] The existing solutions to solve this data loss problem are usually: sending the data of the same driver to two machines, with one of the two machines as a backup. When one of the machines goes offline, there is still the data backed up on the other machine. Although it can, to a certain extent, avoid the problem of data loss, it will cause a waste of half of the machine resources, and manual configuration is required to map different drivers to different machine IPs. When performing operations such as expanding and shrinking the server cluster, the mapping relationship between drivers and machine IPs needs to be manually maintained, which greatly increases the probability of online problems. Therefore, a method for dynamic node transfer is proposed. Summary of the Invention

[0007] In view of this, the present invention provides a method for dynamic node transfer to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0008] The technical solution of the embodiment of the present invention is implemented as follows: A method for dynamic node transfer includes the following steps:

[0009] S1, establish a connection between the server and Zookeeper, execute S2 when the server is online, and execute S5 when the server is offline;

[0010] S2. Register a data node containing the current machine IP in the specified directory of Zookeeper;

[0011] S3, monitor the changes of data nodes in the specified directory through Zookeeper;

[0012] S4. When the data node changes, the mapping relationship between the current driver and the service IP is assigned to the newly launched IP data node;

[0013] S5. Remove the data node of the current machine IP in the specified directory of Zookeeper;

[0014] S6. Monitor the changes of data nodes in the specified directory through Zookeeper;

[0015] S7. When the data node changes, the offline IP mapping relationship in the mapping relationship between the current driver and the service IP is assigned to other service IP data nodes.

[0016] Further preferably, the server includes a computing service for matching the corresponding mapped service IP data node according to the driver ID, and after the matching is completed, pushing the data to the corresponding machine IP for subsequent data processing.

[0017] Further preferably, in S1, when establishing the connection between the server and Zookeeper, an application coordination service named zk is created and instantiated to serve as a bridge for communication between the driver user terminal and the Zookeeper cluster.

[0018] Further preferably, in S2, after the data node including the current machine IP is registered, a listener is added to the specified directory to be triggered when the data node changes.

[0019] Further preferably, the step S3 further includes the following steps:

[0020] S31. When the data node under the specified directory changes, the listener is triggered;

[0021] S32, sending a notification to the computing service through zk;

[0022] S33. The computing service pulls all data nodes under the specified directory.

[0023] Further preferably, in S4, before allocating the mapping relationship between the current driver and the service IP, the mapping relationship between the current driver and the service IP is segmented according to the number of newly launched servers for subsequent sequential allocation.

[0024] Further preferably, in S6, the change of data in the specified directory after removing the data node is monitored through the listener in the specified directory.

[0025] Further preferably, in S7, the following steps are further included:

[0026] S71. When the data in the specified directory changes, the listener is triggered;

[0027] S72. Send a notification to the computing service through zk;

[0028] S73. The computing service pulls all the data nodes in the specified directory.

[0029] Further preferably, the change of the data node includes the increase and decrease of the IP address in the specified directory.

[0030] Due to the above technical solutions, the embodiments of the present invention have the following advantages: By using Zookeeper as the basis, the server can perform data interaction and status synchronization with Zookeeper, so as to provide the ability to automatically transfer the mapping between machines and drivers when the machine goes online or offline, which can effectively reduce the risk of data loss, reduce the need for manual configuration, and there is no need to manually maintain the mapping relationship between IP and nodes, avoiding the problem of online data loss caused by configuration errors due to human negligence and reducing the probability of online problems.

[0031] The above summary is only for the purpose of the specification and is not limited in any way. In addition to the above-described illustrative aspects, embodiments and features, other aspects, embodiments and features of the present invention will be readily apparent by referring to the drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0033] Figure 1 It is the logical structure diagram of the method of the present invention;

[0034] Figure 2 This is a schematic diagram of the step flow of the present invention. Detailed implementation manners

[0035] In the following, only some exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present invention. Therefore, the drawings and the description are considered to be exemplary in nature rather than restrictive.

[0036] It should be noted that terms such as "first", "second", "symmetric", "array", etc. are only used for the purpose of distinguishing descriptions and position descriptions, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "symmetric", etc. can explicitly or implicitly include one or more of such features; similarly, when certain features are not limited in quantity by words such as "two", "three", etc., it should be noted that such features also belong to explicitly or implicitly including one or more feature quantities.

[0037] The embodiments of the present invention will be described in detail below with reference to the drawings.

[0038] As Figure 1 - Figure 2 shown, the embodiments of the present invention provide a method for dynamic node transfer, including the following steps:

[0039] S1. Establish a connection between the server and Zookeeper, execute S2 when the server goes online, and execute S5 when the server goes offline;

[0040] S2. Register a data node containing the current machine's IP in the specified directory of Zookeeper;

[0041] S3. Listen for changes in the data nodes in the specified directory through Zookeeper;

[0042] S4. After the data node changes, allocate the mapping relationship between the current driver and the service IP to the newly online IP data node;

[0043] S5. Remove the data node of the current machine's IP in the specified directory of Zookeeper;

[0044] S6. Listen for changes in the data nodes in the specified directory through Zookeeper;

[0045] S7. After the data node changes, in the mapping relationship between the current driver and the service IP, allocate the mapping relationship of the offline IP to other service IP data nodes.

[0046] In one embodiment, the server includes a computing service for matching the corresponding mapped service IP data node according to the driver ID, and after the matching is completed, pushing the data to the corresponding machine IP for subsequent data processing.

[0047] In one embodiment, in S1, when establishing the connection between the server and Zookeeper, an application coordination service named zk is created and instantiated as a bridge for the driver client to communicate with the Zookeeper cluster.

[0048] The steps for establishing the connection between the server and Zookeeper are as follows:

[0049] Create a Zookeeper client instance: First, create an instance of the ZooKeeper object and name it zk.

[0050] Connect to the Zookeeper cluster: When creating the zk instance, the address and port of the Zookeeper server, as well as a listener to handle connection events, need to be provided.

[0051] Set the listener: Set a listener on the zk instance to listen for changes in the IP data nodes in a specific directory in the Zookeeper cluster.

[0052] Instantiate the application coordination service: Instantiate the zk service to make it a medium for communication between the server and the Zookeeper cluster.

[0053] The code example is as follows:

[0054]

[0055]

[0056]

[0057]

[0058] In this instance, the ApplicationCoordinator class creates and instantiates the zk service, which serves as a bridge for communication between the server and the Zookeeper cluster, is responsible for listening to and processing node data change events, and notifies the computing service.

[0059] The functions of creating an application coordination service named zk and instantiating it are as follows:

[0060] Establish a communication bridge:

[0061] The zk instance serves as a bridge for the driver client to communicate with the Zookeeper cluster, enabling the server to interact with Zookeeper for data and status synchronization.

[0062] Distributed coordination:

[0063] Zookeeper provides a distributed coordination service. Through the zk instance, the server can participate in the coordination work of the distributed system, such as allocating the mapping relationship between the driver and the service IP.

[0064] Status monitoring:

[0065] The server can set a watcher through the zk instance to monitor data changes or node status changes of specific nodes in Zookeeper, so as to achieve dynamic response.

[0066] Data consistency guarantee:

[0067] Zookeeper ensures data consistency in the distributed system. Through the zk instance, the server can read and write data, and Zookeeper is responsible for the atomicity and consistency of these operations.

[0068] Service discovery:

[0069] The server can register its own service in Zookeeper through the zk instance, and other services or clients can query Zookeeper to discover and connect to these services.

[0070] Configuration management:

[0071] Zookeeper can be used as a storage center for configuration information. The server can obtain configuration information through the zk instance and be notified when the configuration is updated, achieving centralized management and dynamic update of configuration information.

[0072] Naming service:

[0073] Zookeeper provides a naming service. The server can create uniquely named nodes through the zk instance to identify resources or services in the cluster.

[0074] In one embodiment, in S2, after the data node containing the current machine IP is registered, a watcher is added in the specified directory to be triggered when the data node changes.

[0075] After the watcher is triggered, Zookeeper will asynchronously notify the server of the occurrence of the compute service event for subsequent processing of the data node using the compute service.

[0076] In one embodiment, in S3, the following steps are further included:

[0077] S31. When the data node in the specified directory changes, the listener is triggered;

[0078] S32. Send a notification to the computing service through zk;

[0079] S33. The computing service pulls all the data nodes in the specified directory;

[0080] When the listener is triggered, first send a notification to the computing service through zk, and then the computing service pulls all the data nodes in the specified directory, so as to allocate the mapping relationship between the current drivers and the service IPs according to the number of newly online servers subsequently.

[0081] In one embodiment, in S4, before allocating the mapping relationship between the current drivers and the service IPs, divide the mapping relationship between the current drivers and the service IPs successively according to the number of newly online servers, so as to allocate them successively subsequently;

[0082] By dividing the mapping relationship between the current drivers and the service IPs according to the number of newly online servers, the subsequently divided data is successively allocated to different newly online IP nodes, so as to ensure that the number of drivers allocated to each machine is relatively average.

[0083] In one embodiment, in S6, the listener in the specified directory is used to monitor the data changes in the specified directory after removing the data node;

[0084] By using the listener to monitor the data changes in the specified directory, the monitoring process is the same as when the data and the server go online.

[0085] In one embodiment, in S7, the following steps are further included:

[0086] S71. When the data in the specified directory changes, the listener is triggered;

[0087] S72. Send a notification to the computing service through zk;

[0088] S73. The computing service pulls all the data nodes in the specified directory;

[0089] When the listener is triggered, first send a notification to the computing service through zk, and then the computing service pulls all the data nodes in the specified directory, so as to allocate the mapping relationship of the offline IP in the mapping relationship between the current drivers and the service IPs to other service IPs subsequently, so as to ensure that the data of the drivers on the original offline machine will not be lost and can be allocated to other machines.

[0090] In one embodiment, the change of the data node includes the increase and decrease of the IP addresses in the specified directory;

[0091] By monitoring the increase and decrease of the number of IP addresses, the allocation operation of the IP addresses can be automatically completed through the knowledge computing service.

[0092] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various changes or substitutions, and these should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for dynamic node transfer, characterized in that: The following steps are involved: S1, establish a connection between the server and Zookeeper, execute S2 when the server is online, and execute S5 when the server is offline; S2. Register a data node containing the current machine IP in the specified directory of Zookeeper; S3, monitor the changes of data nodes in the specified directory through Zookeeper; S4. When the data node changes, the mapping relationship between the current driver and the service IP is assigned to the newly launched IP data node; S5. Remove the data node of the current machine IP in the specified directory of Zookeeper; S6. Monitor the changes of data nodes in the specified directory through Zookeeper; S7. When the data node changes, the offline IP mapping relationship in the mapping relationship between the current driver and the service IP is assigned to other service IP data nodes.

2. The method for dynamic node transfer according to claim 1, characterized in that: The server includes a computing service for matching the corresponding mapped service IP data node according to the driver ID, and after the matching is completed, pushing the data to the corresponding machine IP for subsequent data processing.

3. The method for dynamic node transfer according to claim 1, characterized in that: In S1, when establishing the connection between the server and Zookeeper, an application coordination service named zk is created and instantiated as a bridge for communication between the driver user end and the Zookeeper cluster.

4. The method for dynamic node transfer according to claim 3, characterized in that: In S2, after the data node containing the current machine IP is registered, a listener is added to the specified directory to be triggered when the data node changes.

5. The method for dynamic node transfer according to claim 4, characterized in that: The S3 further includes the following steps: S31. When the data node under the specified directory changes, the listener is triggered; S32, sending a notification to the computing service through zk; S33. The computing service pulls all data nodes under the specified directory.

6. The method for dynamic node transfer according to claim 5, characterized in that: In S4, before allocating the mapping relationship between the current driver and the service IP, the mapping relationship between the current driver and the service IP is divided in sequence according to the number of newly online servers, so as to be allocated in sequence later.

7. The method for dynamic node transfer according to claim 3, characterized in that: In S6, the data changes in the designated directory after the data node is removed are monitored through the listener in the designated directory.

8. The method for dynamic node transfer according to claim 7, characterized in that: The S7 further includes the following steps: S71. When the data in the specified directory changes, the listener is triggered; S72. Send a notification to the computing service through zk; S73. The computing service pulls all data nodes under the specified directory.

9. The method for dynamic node transfer according to claim 1, characterized in that: The data node changes include the increase and decrease of IP addresses under the specified directory.

Images