Zookeeper node monitoring method, device and system

Abstract

The invention discloses a Zookeeper node monitoring method, which is executed by any node server in a Zookeeper cluster, and comprises the following steps of: determining a node type and a node acquisition state of a current node server; when the node type of the current node server is a slave node and the node acquisition state is a working state, determining whether the current node server triggers a preset data acquisition event; if so, collecting snapshot data of the current node server; and sending the snapshot data of the current node server to a monitoring server, so that the monitoring server monitors the node server based on the snapshot data. According to the method and the device, collection of the Zookeeper node data is automatically realized in a mode of not interfering the Zookeeper service in a production environment, and the slave node in the working state is selected to execute the data collection work, so that the service performance of the Zookeeper cluster is ensured, the problem can be conveniently found in time, the monitoring analysis work of the node data is executed by the independently deployed monitoring server, and the monitoring efficiency of the Zookeeper cluster is improved. And the influence of the monitoring logic on the cluster service performance is further reduced.

Description

Technical Field

[0001] This application relates to the technical field of Zookeeper node monitoring, and in particular, to a method, device, and system for monitoring Zookeeper nodes. Background Art

[0002] ZooKeeper is an open-source distributed coordination service. It is software that provides a consistency service for distributed applications. Distributed applications can implement functions such as data publishing / subscribing, load balancing, naming services, distributed coordination / notification, cluster management, master node election, distributed locks, and distributed queues based on ZooKeeper. Therefore, the application of ZooKeeper clusters is becoming more and more extensive, and the monitoring of ZooKeeper cluster nodes is particularly important.

[0003] Currently, there are mainly two types of ZooKeeper node monitoring solutions. One is to monitor the ZooKeeper nodes by periodically traversing and reading them through a monitoring program. However, directly reading and traversing the ZooKeeper in the production environment may pose performance risks to the ZooKeeper service and affect the service quality of ZooKeeper in the production environment.

[0004] The other is to manually download the ZooKeeper snapshot data in the production environment, convert it to a text format through a dedicated program after downloading, and read and analyze it manually. However, the manual analysis method is inefficient, cannot be executed automatically at regular intervals, and is not convenient for timely discovery of problems. Summary of the Invention

[0005] Embodiments of this application provide a method, device, and system for monitoring Zookeeper nodes to improve the timeliness of ZooKeeper node monitoring and reduce the impact on the ZooKeeper real-time service.

[0006] Embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a method for monitoring Zookeeper nodes, which is executed by a current node server in a ZooKeeper cluster, and the current node server is any one of the node servers in the ZooKeeper cluster. Among them, the method includes:

[0008] Determine the node type and node collection status of the current node server;

[0009] When the node type of the current node server is a slave node and the node collection status is the working state, determine whether the current node server triggers a preset data collection event;

[0010] When a preset data collection event is triggered on the current node server, snapshot data of the current node server is collected;

[0011] Send the snapshot data of the current node server to the monitoring server so that the monitoring server can monitor the node server based on the snapshot data.

[0012] Optionally, the determining the node type and node collection status of the current node server includes:

[0013] Determine whether the node type of the current node server is a slave node;

[0014] When the node type of the current node server is a slave node, determine whether the current node server is a working node, where the working node refers to a node used to execute data collection tasks;

[0015] When the current node server is a working node, determine that the node collection status of the current node server is the working state;

[0016] When the current node server is a slave node and not a working node, determine that the node collection status of the current node server is the standby state.

[0017] Optionally, the working node is obtained in the following way:

[0018] When the node type of the current node server is a slave node, negotiate with other slave nodes in the Zookeeper cluster through a negotiation mechanism to determine the working node.

[0019] Optionally, the when the node type of the current node server is a slave node, negotiating with other slave nodes in the Zookeeper cluster through a negotiation mechanism to determine the working node includes:

[0020] Determine the service status of the current node server;

[0021] When the node type of the current node server is a slave node and the service status of the current node server is normal, participate in the competition for the working node through a negotiation mechanism;

[0022] Otherwise, exit the competition for the working node.

[0023] Optionally, the determining whether the current node server triggers a preset data collection event includes:

[0024] Monitor the specified directory of the current node server;

[0025] If new snapshot data is written to the specified directory, determine that the current node server triggers a preset data collection event;

[0026] Otherwise, it is determined that the current node server has not triggered the preset data collection event.

[0027] In a second aspect, an embodiment of the present application further provides a Zookeeper node monitoring method, which is executed by a Zookeeper monitoring server. Wherein, the method includes:

[0028] Receiving snapshot data of node servers in the Zookeeper cluster;

[0029] Based on the snapshot data of the node servers, using a preset analysis strategy to monitor and analyze the node servers in the Zookeeper cluster, and obtaining the monitoring results of the node servers in the Zookeeper cluster;

[0030] Wherein, the snapshot data is collected based on the Zookeeper node monitoring method according to any one of the foregoing first aspects.

[0031] Optionally, the monitoring and analyzing the node servers in the Zookeeper cluster based on the snapshot data of the node servers by using a preset analysis strategy includes:

[0032] Converting the format of the snapshot data of the node servers to obtain the snapshot data after format conversion;

[0033] Updating the preset node information database according to the snapshot data after format conversion;

[0034] Based on the updated preset node information database, using a preset analysis strategy to monitor and analyze the node servers in the Zookeeper cluster;

[0035] Wherein, the preset analysis strategy includes at least one dimension of node comparison analysis, unauthorized use analysis, capacity analysis, and sub-node quantity analysis.

[0036] In a third aspect, an embodiment of the present application further provides a Zookeeper node monitoring device, which is applied to the current node server in the Zookeeper cluster. Wherein, the device includes:

[0037] A first determination unit, configured to determine the node type and node collection status of the current node server;

[0038] A second determination unit, configured to determine whether the current node server has triggered a preset data collection event when the node type of the current node server is a slave node and the node collection status is a working state;

[0039] An acquisition unit, configured to acquire snapshot data of the current node server when a preset data acquisition event is triggered by the current node server;

[0040] A sending unit, configured to send the snapshot data of the current node server to a monitoring server, so that the monitoring server monitors the node server based on the snapshot data.

[0041] In a fourth aspect, an embodiment of the present application further provides a Zookeeper node monitoring device, which is applied to a Zookeeper monitoring server. The device includes:

[0042] A receiving unit, configured to receive snapshot data of a node server in a Zookeeper cluster;

[0043] A monitoring unit, configured to monitor and analyze the node servers in the Zookeeper cluster by using a preset analysis strategy based on the snapshot data of the node servers, so as to obtain a monitoring result of the node servers in the Zookeeper cluster;

[0044] The snapshot data is acquired based on the Zookeeper node monitoring method according to any one of the foregoing first aspects.

[0045] In a fifth aspect, an embodiment of the present application further provides a Zookeeper node monitoring system. The Zookeeper node monitoring system includes a Zookeeper cluster and a Zookeeper monitoring server. The Zookeeper cluster includes multiple node servers. Any one of the node servers is configured to execute the method according to any one of the foregoing first aspects, and the Zookeeper monitoring server is configured to execute the method according to any one of the foregoing second aspects.

[0046] The above at least one technical solution adopted in the embodiments of the present application can achieve the following beneficial effects: The Zookeeper node monitoring method in the embodiments of the present application is executed by the current node server in the Zookeeper cluster, and the current node server is any one node server in the Zookeeper cluster. First, determine the node type and node collection status of the current node server; then, when the node type of the current node server is a slave node and the node collection status is the working state, determine whether the current node server triggers a preset data collection event; after that, when the current node server triggers a preset data collection event, collect the snapshot data of the current node server; finally, send the snapshot data of the current node server to the monitoring server so that the monitoring server can monitor the node server based on the snapshot data. The Zookeeper node monitoring method in the embodiments of the present application automatically realizes the collection of Zookeeper node data in a way that does not interfere with the Zookeeper service in the production environment. Based on the characteristics of the Zookeeper cluster, select the slave nodes in the working state to perform the data collection work, which ensures the service performance of the Zookeeper cluster, facilitates timely problem discovery, and hands over the monitoring and analysis work of the node data to the independently deployed monitoring server, which further reduces the impact of the monitoring logic on the service performance of the Zookeeper cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0048] Figure 1 It is a schematic flowchart of a Zookeeper node monitoring method in the embodiments of the present application;

[0049] Figure 2 It is a schematic flowchart of another Zookeeper node monitoring method in the embodiments of the present application;

[0050] Figure 3 It is a schematic structural diagram of a Zookeeper node monitoring device in the embodiments of the present application;

[0051] Figure 4 It is a schematic structural diagram of another Zookeeper node monitoring device in the embodiments of the present application;

[0052] Figure 5 It is a schematic architecture diagram of a Zookeeper node monitoring system in the embodiments of the present application;

[0053] Figure 6This is a schematic structural diagram of an electronic device in an embodiment of the present application. Detailed implementation manners

[0054] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and the corresponding drawings. Apparently, the described embodiments are only a part rather than all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0055] The following will, with reference to the drawings, elaborate on the technical solutions provided by each embodiment of the present application.

[0056] An embodiment of the present application provides a Zookeeper node monitoring method, which is executed by the current node server in the Zookeeper cluster. The current node server is any one of the node servers in the Zookeeper cluster, such as Figure 1 As shown, a flowchart of a Zookeeper node monitoring method in an embodiment of the present application is provided. The method at least includes the following steps S110 to S140:

[0057] Step S110: Determine the node type and node collection status of the current node server.

[0058] The Zookeeper node monitoring method of the embodiment of the present application can be executed by any one of the node servers in the Zookeeper cluster. Specifically, it can be executed by the data collection and transmission module deployed on the node server. Here, the data collection and transmission module can be implemented based on existing log collection tools, and customized development is carried out on the existing log collection tools in combination with the actual application scenario of the present application.

[0059] The above log collection tool can adopt, for example, the Flume tool. Flume is a distributed, reliable and highly available service for effectively collecting, aggregating and moving a large amount of log data. It has a simple and flexible architecture based on stream data, good reliability mechanisms, failover and recovery mechanisms, and strong fault tolerance. It supports customizing various data senders in the system to collect data; at the same time, Flume provides simple processing of data and has the ability to write to various data receivers.

[0060] In the embodiment of the present application, Flume Agents can be installed and configured on each node server in the Zookeeper cluster. Each Agent is configured with a Source (data source), a Channel (transmission channel), and a Sink (destination), which serves as the basis for collecting and transmitting node data in the present application.

[0061] The Zookeeper cluster adopts a master-slave mode. The cluster includes a master node (Leader) and at least one slave node (Follower). The master node is mainly responsible for managing the entire cluster, ensuring global data consistency, handling data transaction-related operations, and forwarding non-transaction data operations to the slave nodes. The slave nodes are mainly responsible for pulling data from the master node in real time to maintain data consistency, handling non-transaction-related data operations, and forwarding data transaction operations to the master node.

[0062] Based on this, when monitoring Zookeeper nodes in the embodiments of this application, it is necessary to first determine the node type of the current node server. Here, the node type refers to the node role in the above-mentioned Zookeeper cluster, such as whether it is a master node or a slave node. In addition, the embodiments of this application also need to determine the node collection status of the current node server. Here, the node collection status is a status defined according to the specific application scenario of this application, used to reflect whether the current node server has the ability or qualification to collect node data.

[0063] Step S120, when the node type of the current node server is a slave node and the node collection status is the working state, determine whether the current node server triggers a preset data collection event.

[0064] Since the master node is mainly responsible for the management of the Zookeeper cluster and operations related to transactions, in order not to affect the service performance of the Zookeeper cluster, the data collection logic in the embodiments of this application can be executed by the slave nodes. Also, since there are generally multiple slave nodes in the Zookeeper cluster, and based on the characteristic of automatic data replication in the Zookeeper cluster, normally, the data in the master node and multiple slave nodes is consistent. Therefore, it is not necessary for all slave nodes to execute data collection work, and having multiple slave nodes execute data collection work may also cause data inconsistency problems. Therefore, in combination with the node collection status, a slave node in the working state can be selected to execute data collection work.

[0065] Based on this, if the node type of the current node server is a slave node and the node collection status is the working state, it means that the current node server has the qualification to execute data collection work. Therefore, it can be further determined whether the current node server has triggered a preset data collection event, that is, whether it is necessary to start executing data collection work. For example, when new node data is generated in the current node server, it can trigger the execution. Of course, those skilled in the art can also flexibly define the preset data collection event according to the actual business scenario, and no specific limitation is made here.

[0066] Step S130: When a preset data collection event is triggered on the current node server, collect snapshot data of the current node server.

[0067] If a preset data collection event is triggered on the current node server, the data collection and transmission tool deployed in the current node server can execute the data collection task. Here, the collected data is the snapshot data of the current node server, rather than the full volume of data of the current node server. Snapshot data is a very core operating mechanism in the ZooKeeper data storage. The snapshot data in the embodiments of this application can be understood as the data used to record the operation sequence data on the ZooKeeper server. Compared with directly collecting the full volume of data, it can greatly reduce the occupation of memory resources and transmission resources and reduce the transmission time-consuming.

[0068] Step S140: Send the snapshot data of the current node server to the monitoring server so that the monitoring server can monitor the node server based on the snapshot data.

[0069] After the snapshot data of the current node server is collected, the data collection and transmission tool can transmit the collected snapshot data to the monitoring server, and the monitoring server can then perform monitoring and analysis on the node server based on the snapshot data.

[0070] The Zookeeper node monitoring method in the embodiments of this application automatically realizes the collection of Zookeeper node data in a way that does not interfere with the Zookeeper service in the production environment. Based on the characteristics of the Zookeeper cluster, it selects the slave node in the working state to perform the data collection work, ensuring the service performance of the Zookeeper cluster, facilitating the timely discovery of problems, and handing over the monitoring and analysis work of the node data to the independently deployed monitoring server, which further reduces the impact of the monitoring logic on the service performance of the Zookeeper cluster.

[0071] In some embodiments of this application, the determination of the node type and node collection status of the current node server includes: determining whether the node type of the current node server is a slave node; when the node type of the current node server is a slave node, determining whether the current node server is a working node, where the working node refers to the node used to execute the data collection task; when the current node server is a working node, determining that the node collection status of the current node server is the working state; when the current node server is a slave node and not a working node, determining that the node collection status of the current node server is the standby state.

[0072] When determining the node type and node collection status of the current node server, it is possible to first determine whether the current node server is the master node or the slave node. If the current node server is the master node, the current node server can directly skip the subsequent data collection logic and not perform the data collection work, thereby avoiding potential service performance issues caused by the master node performing data collection work.

[0073] If the current node server is a slave node, it is possible to further determine whether the current node server is a working node, that is, a slave node capable of performing data collection tasks. If it is a working node, the corresponding data collection status is the working state. When a subsequent data collection event is triggered, the current node server can collect the newly generated snapshot data on the current node server. If it is not a working node, the corresponding data collection status can be the standby state.

[0074] Through the above judgment logic, on the one hand, the slave nodes in the working state perform the data collection work, avoiding potential service performance issues caused by the master node performing data collection work. On the other hand, only one working node performs the data collection work, avoiding data inconsistency issues caused by multiple nodes collecting data simultaneously.

[0075] In some embodiments of the present application, the working node is obtained in the following manner: when the node type of the current node server is a slave node, through a negotiation mechanism, negotiate with other slave nodes in the Zookeeper cluster to determine the working node.

[0076] The working node in the embodiments of the present application is selected from multiple slave nodes. Specifically, a current working node can be selected through the multi-node negotiation mechanism of the Zookeeper cluster. For example, if the current node server belongs to a slave node, it can negotiate with other slave nodes in the Zookeeper cluster to participate in the election of the working node, thereby determining the working node. Of course, specifically how to conduct the negotiation and election can be flexibly determined by those skilled in the art in combination with the existing technology, and no specific limitation is made here.

[0077] In some embodiments of the present application, when the node type of the current node server is a slave node, negotiating with other slave nodes in the Zookeeper cluster through a negotiation mechanism to determine the working node includes: determining the service status of the current node server; when the node type of the current node server is a slave node and the service status of the current node server is normal, participate in the competition for the working node through the negotiation mechanism; otherwise, exit the competition for the working node.

[0078] When determining the working nodes, it is also possible to monitor the service status of the current node server in real time. Here, the service status mainly refers to whether the node server can establish a connection normally. For example, the Flume Agent can detect the connection status of the node through the connection port. If the connection is normal, it can participate in the election of the working nodes. If the connection is abnormal, it will withdraw from the competition for the working nodes.

[0079] For the elected working nodes, it is also possible to continuously monitor their status. When a working node exits the working state due to an exception, a new election for the working nodes can be carried out among the remaining normal slave nodes.

[0080] In some embodiments of the present application, the determination of whether the current node server triggers a preset data collection event includes: monitoring a specified directory of the current node server; if new snapshot data is written to the specified directory, it is determined that the current node server triggers the preset data collection event; otherwise, it is determined that the current node server does not trigger the preset data collection event.

[0081] The snapshot files generated on the node server are usually stored in the specified directory of the node server. Therefore, the specified directory of the current node server can be monitored in real time. If it is monitored that new snapshot files are written to the specified directory, it is considered that the preset data collection event is triggered, and the current working node collects the snapshot data through the Flume Agent.

[0082] Since the data between the master and slave nodes in the Zookeeper cluster is automatically synchronized and replicated, therefore, when the current node server meets the collection requirements, it can collect the snapshot files in the local specified directory.

[0083] The embodiments of the present application also provide another Zookeeper node monitoring method, which is executed by the Zookeeper monitoring server, as Figure 2 shown, provides a schematic flowchart of another Zookeeper node monitoring method in the embodiments of the present application. The method at least includes the following steps S210 to step S220:

[0084] Step S210, receiving the snapshot data of the node server in the Zookeeper cluster;

[0085] Step S220, based on the snapshot data of the node server, using a preset analysis strategy to monitor and analyze the node servers in the Zookeeper cluster, and obtaining the monitoring results of the node servers in the Zookeeper cluster;

[0086] Among them, the snapshot data is collected based on any one of the foregoing Zookeeper node monitoring methods.

[0087] The Zookeeper node monitoring method according to the embodiments of the present application can be executed by a Zookeeper monitoring server. A corresponding Flume Agent is also deployed in the Zookeeper monitoring server, which is used to receive the snapshot data sent by the Zookeeper node and perform internal processing, internal and external transmission and other operations.

[0088] Based on the snapshot data sent by the node server, certain analysis strategies can be adopted to monitor and analyze the node servers in the Zookeeper cluster, so as to realize the real-time monitoring of the Zookeeper nodes and facilitate the timely discovery of problems.

[0089] In some embodiments of the present application, the monitoring and analysis of the node servers in the Zookeeper cluster by using a preset analysis strategy based on the snapshot data of the node server includes: converting the format of the snapshot data of the node server to obtain the snapshot data after format conversion; updating a preset node information database according to the snapshot data after format conversion; based on the updated preset node information database, using a preset analysis strategy to monitor and analyze the node servers in the Zookeeper cluster; wherein, the preset analysis strategy includes at least one dimension of node comparison analysis, unauthorized use analysis, capacity analysis, and sub-node quantity analysis.

[0090] Since the snapshot files generated by Zookeeper nodes are usually binary files, before processing the snapshot data of the node server, the binary snapshot files can be automatically converted into a readable format, and then the preset node information database can be updated according to the converted snapshot data. The preset node information database here can include the local storage of node information, or can include the remote persistent storage of node information such as ElasticSearch storage.

[0091] Finally, according to the node information stored in the preset node information database, different preset analysis strategies are adopted to monitor and analyze the node servers in the Zookeeper cluster. The preset analysis strategies here mainly can include multiple dimensions such as node comparison analysis, unauthorized use analysis, capacity analysis, and sub-node quantity analysis according to the needs of the actual business scenario.

[0092] The node comparison analysis is mainly used to monitor whether major changes have occurred in the Zookeeper cluster. For example, it can be judged based on the change in the number of nodes and the change time. The unauthorized use analysis is mainly to monitor whether there is any unauthorized use behavior between the producers and consumers in the Zookeeper cluster. Since there is a theoretical authorization process between the producers and consumers, if a consumer uses the producer's data without authorization, it belongs to unauthorized use. In addition, the access priority of Zookeeper is to access the local computer room, so the cross-computer room access situation can also be monitored. The capacity analysis is mainly to monitor the number of nodes in each computer room and the number of consumers corresponding to each producer. The child node quantity analysis is mainly to monitor whether the number of child nodes is too large and whether they are not deleted in time. If any of the above dimensions shows an abnormality, an alarm can be issued for timely handling.

[0093] Of course, for the specific monitoring dimensions to be set, those skilled in the art can flexibly set them according to the actual application scenarios and will not be listed one by one here.

[0094] The embodiment of the present application also provides a Zookeeper node monitoring device 300, which is applied to the current node server in the Zookeeper cluster, as Figure 3 shown, which provides a structural schematic diagram of a Zookeeper node monitoring device in the embodiment of the present application. The device 300 includes: a first determination unit 310, a second determination unit 320, a collection unit 330, and a sending unit 340, where:

[0095] The first determination unit 310 is used to determine the node type and node collection status of the current node server;

[0096] The second determination unit 320 is used to determine whether the current node server triggers a preset data collection event when the node type of the current node server is a slave node and the node collection status is the working state;

[0097] The collection unit 330 is used to collect the snapshot data of the current node server when the current node server triggers a preset data collection event;

[0098] The sending unit 340 is used to send the snapshot data of the current node server to the monitoring server so that the monitoring server monitors the node server based on the snapshot data.

[0099] In some embodiments of the present application, the first determination unit 310 is specifically configured to: determine whether the node type of the current node server is a slave node; in the case where the node type of the current node server is a slave node, determine whether the current node server is a working node, where the working node refers to a node used to execute a data collection task; in the case where the current node server is a working node, determine that the node collection status of the current node server is the working state; in the case where the current node server is a slave node and not a working node, determine that the node collection status of the current node server is the standby state.

[0100] In some embodiments of the present application, the working node is obtained in the following manner: in the case where the node type of the current node server is a slave node, negotiate with other slave nodes in the Zookeeper cluster through a negotiation mechanism to determine the working node.

[0101] In some embodiments of the present application, the working node is obtained in the following manner: determine the service status of the current node server; in the case where the node type of the current node server is a slave node and the service status of the current node server is normal, participate in the competition for the working node through a negotiation mechanism; otherwise, exit the competition for the working node.

[0102] In some embodiments of the present application, the second determination unit 320 is specifically configured to: monitor a specified directory of the current node server; if new snapshot data is written to the specified directory, determine that the current node server triggers a preset data collection event; otherwise, determine that the current node server does not trigger a preset data collection event.

[0103] It can be understood that the above Zookeeper node monitoring device can implement each step of the Zookeeper node monitoring method executed by the current node server in the Zookeeper cluster in the foregoing embodiments. The relevant explanations regarding the Zookeeper node monitoring method are applicable to the Zookeeper node monitoring device and will not be elaborated herein.

[0104] The embodiments of the present application further provide another Zookeeper node monitoring device 400, which is applied to a Zookeeper monitoring server, as Figure 4 shown, provides a schematic structural diagram of another Zookeeper node monitoring device in the embodiments of the present application. The device 400 includes: a receiving unit 410 and a monitoring unit 420, where:

[0105] The receiving unit 410 is configured to receive snapshot data of a node server in the Zookeeper cluster;

[0106] The monitoring unit 420 is configured to perform monitoring and analysis on the node servers in the Zookeeper cluster based on the snapshot data of the node servers, using a preset analysis strategy, to obtain the monitoring results of the node servers in the Zookeeper cluster;

[0107] Wherein, the snapshot data is collected based on the Zookeeper node monitoring method described in any one of the foregoing items.

[0108] In some embodiments of the present application, the monitoring unit 420 is specifically configured to: perform format conversion on the snapshot data of the node servers to obtain the snapshot data after format conversion; update the preset node information database according to the snapshot data after format conversion; based on the updated preset node information database, perform monitoring and analysis on the node servers in the Zookeeper cluster using a preset analysis strategy; wherein, the preset analysis strategy includes at least one dimension of node comparison analysis, unauthorized use analysis, capacity analysis, and child node quantity analysis.

[0109] It can be understood that the above Zookeeper node monitoring device can implement each step of the Zookeeper node monitoring method executed by the Zookeeper monitoring server provided in the foregoing embodiments. The relevant explanations regarding the Zookeeper node monitoring method are applicable to the Zookeeper node monitoring device and will not be elaborated here.

[0110] An embodiment of the present application further provides a Zookeeper node monitoring system. The Zookeeper node monitoring system includes a Zookeeper cluster and a Zookeeper monitoring server. The Zookeeper cluster includes multiple node servers, and any one of the node servers is configured to execute the method described in any one of the foregoing embodiments of the first aspect. The Zookeeper monitoring server is configured to execute the method described in any one of the foregoing embodiments of the second aspect.

[0111] As Figure 5 shown, there is provided a schematic diagram of the architecture of a Zookeeper node monitoring system in an embodiment of the present application. The Zookeeper node monitoring system in the embodiment of the present application mainly includes a Zookeeper cluster and a Zookeeper monitoring server. In addition, it may further include ElasticSearch storage. The Zookeeper cluster may be one or more, and each Zookeeper cluster is composed of multiple node servers.

[0112] In the embodiments of the present application, corresponding Flume Agents are respectively installed and configured in each node server and the monitoring server. The Flume Agent deployed in the node server includes a ZookeeperSnapshot Source (Zookeeper snapshot data source) and a Memory Channel (transmission channel) customized and developed based on actual business requirements, which are used to collect snapshot data and transmit it to the monitoring server. The Flume Agent deployed in the monitoring server includes an Avro sink, a MemoryChannel (transmission channel), and a Zookeeper Snapshot sink (Zookeeper snapshot data destination), which are used to implement operations such as data reception, decompression processing, format conversion, and internal and external transmission.

[0113] The Zookeeper monitoring server updates the processed data to the local memory and the remote ElasticSearch storage, and realizes multi-dimensional monitoring and analysis functions such as node comparison analysis, unauthorized use analysis, capacity analysis, and sub-node quantity analysis through the ElasticSearch storage. Finally, the monitoring and analysis results can also be displayed through a visualization tool.

[0114] Figure 6 It is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 6 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0115] The processor, the network interface, and the memory can be interconnected through the internal bus, and the internal bus can be an ISA (Industry Standard Architecture) bus, a PCI (PeripheralComponent Interconnect) bus, or an EISA (Extended Industry StandardArchitecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 6 only a bidirectional arrow is used in

[0116] A memory for storing programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0117] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a Zookeeper node monitoring device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0118] Determine the node type and node collection status of the current node server;

[0119] When the node type of the current node server is a slave node and the node collection status is the working state, determine whether the current node server triggers a preset data collection event;

[0120] When the current node server triggers a preset data collection event, collect the snapshot data of the current node server;

[0121] Send the snapshot data of the current node server to the monitoring server so that the monitoring server monitors the node server based on the snapshot data.

[0122] The above as in this application Figure 1The method executed by the Zookeeper node monitoring device disclosed in the illustrated embodiment can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method can be completed by the integrated logic circuit in the hardware of the processor or instructions in software form. The above-mentioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0123] The electronic device can also execute Figure 1 the method executed by the Zookeeper node monitoring device in Figure 1 the illustrated embodiment and implement the functions of the Zookeeper node monitoring device in

[0124] The embodiments of the present application also propose a computer-readable storage medium that stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including multiple application programs, can enable the electronic device to execute Figure 1 the method executed by the Zookeeper node monitoring device in the illustrated embodiment, and specifically used to execute:

[0125] Determine the node type and node collection status of the current node server;

[0126] When the node type of the current node server is a slave node and the node collection status is in a working state, determine whether the current node server triggers a preset data collection event;

[0127] When the current node server triggers a preset data collection event, collect the snapshot data of the current node server;

[0128] Send the snapshot data of the current node server to the monitoring server so that the monitoring server can monitor the node server based on the snapshot data.

[0129] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0130] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0131] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1A step that specifies a function in one or more boxes.

[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0134] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0135] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0136] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0138] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A Zookeeper node monitoring method, which is executed by the current node server in the Zookeeper cluster. The current node server is any one of the node servers in the Zookeeper cluster. Among them, The method includes: Determine the node type and node collection status of the current node server; When the node type of the current node server is a slave node and the node collection status is the working state, determine whether the current node server triggers a preset data collection event; When the current node server triggers a preset data collection event, collect the snapshot data of the current node server; Send the snapshot data of the current node server to the monitoring server, so that the monitoring server monitors the node server based on the snapshot data.

2. The method according to claim 1, wherein The determining the node type and node collection status of the current node server includes: Determine whether the node type of the current node server is a slave node; When the node type of the current node server is a slave node, determine whether the current node server is a working node, and the working node refers to a node used to execute the data collection task; When the current node server is a working node, determine that the node collection status of the current node server is the working state; When the current node server is a slave node and not a working node, determine that the node collection status of the current node server is the standby state.

3. The method according to claim 2, wherein The working node is obtained in the following way: When the node type of the current node server is a slave node, negotiate with other slave nodes in the Zookeeper cluster through a negotiation mechanism to determine the working node.

4. The method according to claim 3, wherein The when the node type of the current node server is a slave node, negotiating with other slave nodes in the Zookeeper cluster through a negotiation mechanism to determine the working node includes: Determine the service status of the current node server; When the node type of the current node server is a slave node and the service status of the current node server is normal, participate in the competition for the working node through a negotiation mechanism; Otherwise, exit the competition for the working node.

5. The method according to claim 1, wherein The determining whether the current node server triggers a preset data collection event includes: Monitor the specified directory of the current node server; If new snapshot data is written to the specified directory, determine that the current node server triggers a preset data collection event; Otherwise, determine that the current node server does not trigger a preset data collection event.

6. A Zookeeper node monitoring method, which is executed by a Zookeeper monitoring server, wherein, The method includes: Receive the snapshot data of the node server in the Zookeeper cluster; Based on the snapshot data of the node server, use a preset analysis strategy to monitor and analyze the node servers in the Zookeeper cluster to obtain the monitoring results of the node servers in the Zookeeper cluster; Wherein, the snapshot data is collected based on the Zookeeper node monitoring method according to any one of claims 1 to 5.

7. The method according to claim 6, wherein The using a preset analysis strategy to monitor and analyze the node servers in the Zookeeper cluster based on the snapshot data of the node server includes: Convert the format of the snapshot data of the node server to obtain the snapshot data after format conversion; Update the preset node information database according to the snapshot data after format conversion; Based on the updated preset node information database, the node servers in the Zookeeper cluster are monitored and analyzed using preset analysis strategies; Among them, the preset analysis strategy includes at least one dimension of node comparison analysis, unauthorized use analysis, capacity analysis, and sub-node quantity analysis.

8. A Zookeeper node monitoring device is applied to the current node server in a Zookeeper cluster, where The device includes: A first determination unit for determining the node type and node collection status of the current node server; A second determination unit for determining whether the current node server triggers a preset data collection event when the node type of the current node server is a slave node and the node collection status is the working state; A collection unit for collecting snapshot data of the current node server when the current node server triggers a preset data collection event; A sending unit for sending the snapshot data of the current node server to the monitoring server so that the monitoring server monitors the node server based on the snapshot data.

9. A Zookeeper node monitoring device is applied to a Zookeeper monitoring server, where The device includes: A receiving unit for receiving snapshot data of the node servers in the Zookeeper cluster; A monitoring unit for monitoring and analyzing the node servers in the Zookeeper cluster using preset analysis strategies based on the snapshot data of the node servers to obtain the monitoring results of the node servers in the Zookeeper cluster; Among them, the snapshot data is collected based on the Zookeeper node monitoring method according to any one of claims 1 to 5.

10. A Zookeeper node monitoring system, the Zookeeper node monitoring system includes a Zookeeper cluster and a Zookeeper monitoring server, the Zookeeper cluster includes multiple node servers, any one of the node servers is used to execute the method according to any one of claims 1 to 5, and the Zookeeper monitoring server is used to execute the method according to any one of claims 6 to 7.

Images