A method, server and server cluster for service monitoring and instance scaling

Abstract

A service monitoring method comprises: obtaining node information of a plurality of nodes to be monitored, and obtaining at least one node information set based on the node information of the plurality of nodes to be monitored, wherein the node information set comprises node information of at least one node to be monitored; starting at least one service instance based on the at least one node information set, the service instance corresponding to the node information set, the service instance being used for monitoring each node to be monitored in the corresponding node information set; creating at least one instance monitoring node, the instance monitoring node corresponding to the service instance, the instance monitoring node being used for monitoring a state of the corresponding service instance; and monitoring the at least one service instance based on the at least one node information set and the at least one instance monitoring node. In this way, each node to be monitored is intelligently monitored by one instance monitoring, each instance monitoring different nodes to be monitored, which can reduce the problem of data consistency and the problem of double data storage, and can realize high availability of the service.

Description

Technical Field

[0001] This invention relates to the field of information technology (IT), and more particularly to a service monitoring and instance expansion method, a server, and a server cluster. Background Technology

[0002] Prometheus is an open-source system monitoring and alerting system. It's a model based on centralized rule calculation, unified analysis, and alerting, providing a service to monitor system status in real time. Prometheus boasts advantages such as ease of management, a powerful data model, the powerful query language PromQL, high efficiency, scalability, ease of integration, visualization, and openness. However, during Prometheus operation, service interruptions or inability to view historical data can cause the service to fail to meet user needs. Therefore, ensuring its availability is a crucial technical challenge that needs to be addressed. Summary of the Invention

[0003] To address the problems existing in the prior art, embodiments of this application provide a service monitoring method, a service instance expansion method, a server, a server cluster, a computer storage medium, and a product containing a computer program, which can enhance the high availability of service monitoring.

[0004] In a first aspect, embodiments of this application provide a service monitoring method, the method comprising: acquiring node information of multiple nodes to be monitored, and obtaining at least one node information set based on the node information of the multiple nodes to be monitored, wherein the node information set includes node information of at least one node to be monitored; starting at least one service instance based on the at least one node information set, the service instance corresponding to the node information set, the service instance being used to monitor each node to be monitored in the corresponding node information set; creating at least one instance monitoring node, the instance monitoring node corresponding to the service instance, the instance monitoring node being used to monitor the status of the corresponding service instance; and monitoring at least one service instance based on the at least one node information set and the at least one instance monitoring node.

[0005] In this way, by acquiring the node information of the nodes to be monitored, storing the node information in a set, and starting a service instance based on the set of node information, the service instance can monitor the nodes corresponding to the node information. This achieves high availability of the monitoring service.

[0006] In one possible implementation, the method further includes: monitoring data generated by the storage service instance monitoring the node to be monitored.

[0007] This allows the monitoring data to be stored, making it convenient to use when needed later.

[0008] In one possible implementation, at least one set of node information is obtained based on the node information of multiple nodes to be monitored, including: determining the number of node information sets based on the number of nodes corresponding to the node information and the number of node information records in a node information set indicated by the first configuration information; assigning an identifier to each node information set; and creating a node information set based on the identifier of the node information set, wherein the name of the node information set includes at least the identifier of the node information set.

[0009] In this way, after determining the identifier of the node information set, creating the node information set based on this identifier can make the creation of the node information set more stable.

[0010] In one possible implementation, when recording node information in a node information set, the method further includes: allocating the node information set with the node information information to be recorded; and recording the node information set with the node information to be recorded in the name of the node information set, wherein the name of the node information set includes the identifier of the node information set and the node information set with the node information to be recorded.

[0011] In one possible implementation, at least one service instance is monitored based on at least one set of node information and at least one instance monitoring node, including: starting the service instance based on the identifier of the set of node information; obtaining the node information recorded in the set of node information based on the identifier of the set of node information; and the service instance monitoring the node to be monitored corresponding to the node information.

[0012] In this way, the service instance launched based on the node information set obtains the node information included in this set, and then monitors the nodes corresponding to that node information. This allows the service instance to more clearly identify the nodes that need to be monitored.

[0013] In one possible implementation, monitoring at least one service instance is performed based on at least one set of node information and at least one instance monitoring node. The method further includes expanding the service instance if the number of nodes in the set of node information is greater than the number of instance monitoring nodes that have detected the service instance as alive.

[0014] Thus, if the number of nodes in the node information set exceeds the number of monitored nodes of the service instance that are currently alive, it indicates that some nodes have not yet been monitored. In this case, the service instance can be expanded by starting a new instance to monitor the unmonitored nodes. This allows the service instance to adapt to changes in the number of nodes.

[0015] Secondly, embodiments of this application provide a service instance expansion method, characterized in that the method includes: acquiring at least one node information set and at least one instance monitoring node, wherein the instance monitoring node is used to monitor the status of the corresponding service instance, the service instance corresponds to the node information set, the node information set includes node information of at least one node to be monitored, and the service instance is used to monitor each node to be monitored in the corresponding node information set; if the number of the node information set is greater than the number of instance monitoring nodes that have monitored the service instance to be alive, then the service instance is expanded.

[0016] In this way, when the number of nodes to be monitored changes, the number of service instances can increase accordingly, thus achieving the effect that the number of service instances changes dynamically with the increase of the number of nodes to be monitored.

[0017] In one possible implementation, the service instance is expanded by: determining the newly added set of node information, and starting the service instance corresponding to the number of newly added node information sets based on the newly added set of node information.

[0018] In this way, as the number of nodes to be monitored increases, the number of service instances can be increased accordingly, thereby achieving dynamic expansion.

[0019] In one possible implementation, the method further includes: if the number of node information sets is less than the number of monitoring nodes that are monitoring service instances in a live state, then determine the service instances in an idle state; stop or delete the service instances in an idle state.

[0020] In this way, as the number of nodes to be monitored decreases, the number of service instances can also decrease. By stopping or deleting idle service instances, resource consumption can be reduced.

[0021] Thirdly, embodiments of this application provide a server cluster, including: multiple nodes to be monitored, each node including node information; a first cluster, used to acquire the node information of the multiple nodes to be monitored, and based on the node information, start at least one service instance, the service instance being used to monitor at least one node to be monitored; a second cluster, used to create at least one instance monitoring node, the instance monitoring node corresponding to the service instance, the instance monitoring node being used to monitor the status of the corresponding service instance; the first cluster is also used to monitor at least one service instance based on the node information and the instance monitoring node.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including computer-readable instructions that, when read and executed by a computer, cause the computer to perform the method described in either the first or second aspect.

[0023] Fifthly, embodiments of this application provide a server, including a processor and a memory, wherein the memory is connected to the processor, and the memory stores computer program instructions, which, when executed by the processor, perform the method described in either the first or second aspect.

[0024] Sixthly, embodiments of this application provide a product comprising a computer program that, when the computer program product is run on a processor, causes the processor to perform the method as described in either the first or second aspect.

[0025] It is understood that the beneficial effects of the third to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the architecture of a service monitoring system provided in an embodiment of this application;

[0028] Figure 2 This is a schematic diagram of a registration process provided in an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of a node in an operation registration center provided in an embodiment of this application;

[0030] Figure 4This is a schematic diagram of the structure of a scheduling center provided in an embodiment of this application;

[0031] Figure 5 This is a schematic diagram illustrating the interoperability of various modules within a scheduling center, as provided in an embodiment of this application.

[0032] Figure 6 This is a schematic diagram illustrating the operation of a data registration module provided in an embodiment of this application;

[0033] Figure 7 This is a flowchart illustrating the operation of a data modification module provided in an embodiment of this application;

[0034] Figure 8 This is a flowchart illustrating the operation of an instance monitoring module provided in an embodiment of this application;

[0035] Figure 9 This is a flowchart illustrating the operation of an instance scheduling module provided in an embodiment of this application;

[0036] Figure 10 This is a schematic diagram of a service monitoring system provided in an embodiment of this application;

[0037] Figure 11 This is a flowchart illustrating a service monitoring method provided in an embodiment of this application;

[0038] Figure 12 This is a schematic diagram of a service instance extension method provided in an embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0041] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0042] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0043] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0044] To facilitate understanding of the embodiments of this application, the following will provide further explanation and description with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0045] First, the technical terms involved in the embodiments of this application will be introduced:

[0046] 1. High availability, also known as high availability, is used to describe a system that is specifically designed to reduce downtime and maintain a high level of service availability.

[0047] 2. Active-active is a resource-saving disaster recovery solution where two data centers, one primary and one backup, simultaneously handle business operations, acting as backups for each other. When one data center fails, the other can take over the business, ensuring continued service operation.

[0048] Next, the technical solutions provided in the embodiments of this application will be introduced.

[0049] When designing services, high availability is a consideration, and high availability is usually a matter of architecture.

[0050] In one related technology, an architecture to enhance the high availability of Prometheus implements a primary-backup dual-storage system, achieving both data persistence and Prometheus scalability. However, in this architecture, because both primary and backup data need to be persisted, the data is stored twice, resulting in significant waste of storage space. Furthermore, since a data node can be monitored by multiple Prometheus instances—for example, when a data node is simultaneously monitored by Prometheus instances P1 and P2—a failure in either instance (e.g., P1) will cause inconsistencies in the monitoring data generated by the multiple Prometheus instances (P1 and P2), leading to a lack of data consistency guarantees across the system. Additionally, since each Prometheus instance monitors a fixed number of nodes, it becomes impossible to monitor new nodes.

[0051] In view of this, this application provides a service monitoring system that obtains node information of the nodes to be monitored through a scheduling center. The scheduling center determines the identifier of the registration file based on the configuration information and the obtained node information. The registration file is then determined in the registration center based on the identifier of the registration file and the node information. The monitoring center can launch the corresponding instance based on the identifier of the registration file obtained from the scheduling center. Each instance obtains the node information of the nodes to be monitored through the registration center to monitor different nodes (the nodes recorded in the registration file), thereby achieving high availability of monitoring.

[0052] For example, Figure 1 An architecture of a service monitoring system provided in an embodiment of this application is illustrated. For example... Figure 1 As shown, the architecture 100 of the service monitoring system includes a business system 110, a scheduling center 120, a registration center 130, a monitoring center 140, and a storage center 150. The service monitoring system 100 can be a cluster, and the business system 110, scheduling center 120, registration center 130, and monitoring center 140 can be sub-clusters of the cluster.

[0053] The business system 110 may include multiple (e.g., n, n≥1) nodes. Nodes can be any device, server, or equipment cluster with computing and processing capabilities. When the business system 110 includes multiple nodes, each node can complete a portion of the business, thus the business in the business system 110 is completed collaboratively by multiple nodes. Alternatively, each node can complete all the business, with multiple nodes providing services in a cluster, and one node pre-designated as the management node within the cluster. Each node has its own node information, including its Internet Protocol (IP) address and port. Each node is a node to be monitored, and the management node in the business system 110 can send the node information of each node in the business system 110 to the dispatch center 120.

[0054] The dispatch center 120 can be a cluster with multiple nodes. A configuration file is pre-configured in the dispatch center 120, which specifies the number of node information records in a registration file. Since each node has its own node information, and there is a one-to-one correspondence between nodes and node information, the number of node information records is the same as the number of nodes. The registration file is used to record the node information; therefore, the registration file can also be called a node information set. Accordingly, the node information in the registration file is the node information recorded in the node information set. After the dispatch center 120 obtains the node information sent by each node to be monitored in the business system 110, it can determine the identifiers of the registration files to be created in the registration center 130 based on the number of node information records and the configuration file. For example, the number of registration files to be created in the registration center 130 (i.e., the number of identifiers) can be the rounded-up value obtained by dividing the ratio between the number of nodes to be monitored and the number configured in the configuration information. The dispatch center 120 can send the identifier of the registration file required by the registration center 130 and the node information of the node to be monitored obtained by the dispatch center 120 to the registration center 130; and send the identifier of the registration file to the monitoring center 140. For an example, please refer to [reference needed]. Figure 2 The business system 110 sends the node information of the nodes to be monitored to the dispatch center 120. After receiving the node information, the dispatch center 120 can generate an identifier for the registration file. During the generation of the identifier, operations such as adding, modifying, and deleting identifiers can be performed. After generating the identifier, the system can send the identifier to the registration center 130, so that the registration center 130 can register the nodes based on the identifier.

[0055] After receiving the identifier of the registration file to be created and the node information of the node to be monitored from the scheduling center 120, the registration center 130 can create a corresponding number of registration files on one or more of its nodes, and record the identifier of the corresponding registration file and randomly record the node information of the node to be monitored in each registration file. For example, the registration file created by the registration center 130 can be in the style of a folder, which can include the folder's identifier (such as name) and the node information of the nodes it records. Alternatively, the registration file created by the registration center 130 can be a document, which can include the document's identifier (such as name) and the node information of the nodes it records. For example, the registration center can be ZooKeeper. ZooKeeper is a distributed, open-source distributed application coordination service that can provide services through a cluster. This embodiment uses a ZooKeeper cluster as an example for description. In a ZooKeeper cluster, there are master nodes and slave nodes. The master node is used to receive information sent by the scheduling center 120, generate registration files, and synchronize the registration files to the slave nodes. The registration files record data in the form of data nodes. In ZooKeeper, data nodes can be viewed as a tree structure (or directory). Each data node in the tree is called a znode (i.e., a ZooKeeper node). A znode can have multiple child nodes. In essence, all stored data in ZooKeeper is composed of znodes, and its overall structure is similar to a Linux file system, stored in a tree structure, with the root path starting with " / ". After the master node receives information sent by the scheduling center 120, it can configure a first data node " / info" (hereinafter referred to as the info node) and a second data node " / node" (hereinafter referred to as the node node). The first data node " / info" is used to store the identifier of the registration file, and the second data node " / node" is used to store the node information of the node to be monitored. For example, if the IP address of the node to be monitored, 0.0.0.1, and the port is 80, then the node can be represented as " / node / 0.0.0.1:80". The info node is used to represent the identifier of the registered file. For example, if the info1 node carries (i.e., records) node 1, then the info node can be represented as " / info / info1 / node / 0.0.0.1:80". When the info node carries multiple node nodes, such as if the info1 node carries node 1 and node 2, then the info node can be represented as " / info / info1 / node / 0.0.0.1:80" or " / info / info1 / node / 0.0.0.2:80".The registry center 130 can also modify the created registry files. For example, if a registry file contains only one node information under the data node " / info", and the business system 110 adds a new node, this node sends its node information to the scheduling center 120. The scheduling center 120 can instruct the registry center 130 to add node information under the data node " / info". For instance, if the data node info2 carries node 3, represented as " / info / info2 / node / 0.0.0.3:80", and the business system 110 adds node 4, then ZooKeeper can modify the data node info2, representing it as " / info / info2 / node / 0.0.0.3:80" or " / info / info2 / node / 0.0.0.4:80".

[0056] After receiving the identifier of the registration file sent by the scheduling center 120, the monitoring center 140 can launch (i.e., create and start) a service monitoring instance (or simply instance) based on the identifier of the registration file. There is a one-to-one correspondence between instances and identifiers in the registration files; that is, one identifier in a registration file corresponds to one instance, and one instance corresponds to one identifier in a registration file. After launching an instance, each instance can periodically (e.g., every minute) send a request message to the registration center 130, which may carry the identifier of the registration file corresponding to that instance. After receiving the request message from the instance, the registration center 130 can query the registration file within it based on the identifier of the registration file corresponding to the instance, obtain the node information recorded in the registration file indicated by the identifier, and return the node information to the requesting instance. After receiving the node information returned by the registration center 130, the instance launched by the monitoring center 140 can monitor the corresponding node according to the IP address and port contained in the node information, and send the monitoring data generated during the monitoring process to the storage center 150.

[0057] Furthermore, when an instance started by monitoring center 140 sends a request to registry center 130, it can also send its status information to registry center 130. This status information indicates whether the requesting instance is alive. After receiving this information, registry center 130 can record the status information, creating a status file representing the instance's status. For example, the registry center can create a third data node related to the instance's status within the file. This third data node can indicate the instance's status information; therefore, it can also be called an instance monitoring node. For example, monitoring center 140 can send a request to registry center 130 (using ZooKeeper as an example) for instance 1 started by Prometheus. The request includes information that instance 1 is alive. When ZooKeeper receives the request and confirms that instance 1 is alive, it can create a third data node, named " / alive," which records the status of instance 1 as " / alive / true." For example, as shown... Figure 3 As shown, the scheduling center 120 can create and modify node nodes and info nodes in the registration center 130. The service monitoring instance started by the monitoring center 140 can read the info nodes in the registration center 130 to obtain the node information that needs to be monitored. At the same time, the instance can also create an alive node (i.e., an instance monitoring node) in the registration center. The scheduling center 120 can also monitor the alive node.

[0058] Storage center 140 can receive and store monitoring data sent by monitoring center 140 for subsequent viewing and maintenance. The monitoring data may include the running status of nodes in the business system 110 monitored by the instance. When the running status differs from the normal running status, it can promptly alert maintenance personnel to perform maintenance. For example, storage center 150 can choose a time-series database like InfluxDB. Compared to traditional relational databases, the biggest feature of time-series databases is that data is stored in chronological order. Because monitoring data is obtained in chronological order, choosing a time-series database for storage is a better choice, as storing monitoring data in chronological order makes querying more convenient. Of course, a time-series database is just one example of storage; storage center 150 can also choose a relational database (such as MySQL) or a non-relational database (such as Redis) for storage.

[0059] This achieves the monitoring of monitoring nodes by the monitoring center. During program execution, the number of nodes in business system 110 may change as business needs evolve. Therefore, when the number of nodes increases, the newly added nodes also need to be monitored, requiring the addition of new instances. For example, if the number of users for a certain business grows continuously with business operations, and the existing nodes in the business system cannot handle the high concurrency requests from a large number of users, then adding nodes can reduce the pressure on a single server. Conversely, when the number of nodes decreases, some instances may become idle and need to be stopped or deleted. For instance, if a business experiences a significant loss of users due to poor management, then a large number of servers are not needed to maintain it, and resources can be saved by reducing the number of servers. Therefore, the number of instances needs to be adjusted.

[0060] In view of this, the scheduling center 120 can also periodically (e.g., every 10 seconds, 1 minute, etc.) obtain the number of instances with a live status recorded in the registration center 130, and the number of registration files (identifiers of registration files). After obtaining the information, these two numbers can be compared. If the number of live instances is less than the number of registration files, it indicates that there are nodes to be monitored that are not being monitored by the monitoring center 140. In this case, a new registration file identifier can be created and sent to both the registration center 130 and the monitoring center 140, so that the registration center 130 records the node information set and the monitoring center 140 creates new instances to monitor the unmonitored nodes. If the number of live instances is greater than the number of nodes in the node information set, it indicates that there are idle instances or multiple instances monitoring the same node. In this case, the nodes monitored by each instance can be obtained, and the monitoring center 140 can be instructed to stop (or delete) duplicate or redundant instances. Specifically, the scheduling center 120 can obtain the configuration information of each instance in the monitoring center 140. This configuration information contains the node information of the nodes monitored by the instance. If the configuration information does not contain any node information, it is determined that the instance is not monitoring any nodes, indicating that the instance is in an idle state. At this time, the scheduling center 120 can instruct the monitoring center 140 to delete the idle instance.

[0061] For example, when reducing a node in business system 110, business system 110 can send the IP address and port of the reduced node to scheduling center 120. Scheduling center 120 then sends the node information of that node to registration center 130. Registration center 130 searches its own recorded node information, finds the corresponding node information, and deletes it. When adding a node to business system 110, business system 110 can send the node information of the newly added node to scheduling center 120. Scheduling center 120 obtains the number of registration files and the number of node information in the registration center, and calculates a first ratio between the existing number of node information and the number of registration files. If the first ratio is less than the second ratio configured in the registration file in the configuration file (the number of nodes corresponding to the node information that a registration file can record), it indicates that the existing registration files can still record node information. At this time, based on the number of registration files, the second ratio, and the existing number of node information, the number of node information that the registration files can still record can be determined. If the number of node information recordsable in the existing registration file is greater than the number of new nodes, it indicates that the existing registration file can record the node information of the new nodes. In this case, the scheduling center 120 can send the node information of the new nodes to the registration center 130. If the number of node information recordsable in the existing registration file is less than the number of new nodes, it indicates that the existing registration file is insufficient to record the node information of the new nodes. In this case, a new registration file identifier can be created, and the identifier of the new registration file along with the node information of the new nodes can be sent to the registration center 130. The registration center 130 records the node information of the new nodes based on the received information.

[0062] Therefore, this architecture, by adding a scheduling center and a registration center, allows the instances launched by the monitoring center to be scheduled according to the scheduling center when monitoring nodes, thus enabling the monitoring center to expand its instances based on the nodes to be monitored. Each monitoring instance can monitor multiple nodes, and each node can only be monitored by one instance, which solves the data consistency problem. Furthermore, since each node can be monitored by only one instance, the monitoring data is stored in a single copy, reducing the need for double data storage.

[0063] For example, Figure 4 A schematic diagram of a dispatch center structure is shown. Figure 4 As shown, the scheduling center 120 may include a data receiving module 121, a data registration module 122, a data modification module 123, an instance monitoring module 124, and an instance scheduling module 125. Please refer to [the relevant documentation / reference]. Figure 5In the dispatch center 120, the data receiving module 121 connects to the business system 110, the data registration module 122 and the data modification module 123 connect to the registration center 130, the instance monitoring module 124 connects to the monitoring center 140 to monitor each instance, and the instance scheduling module 125 connects to the monitoring center 140 to notify the instances in the monitoring center 140 to be scheduled.

[0064] The data receiving module 121 can be used to receive node information from each node in the business system 110.

[0065] The data registration module 122 can be used to send the node information received by the data receiving module 121 to the registration center 130 for registration. For example, such as... Figure 6 As shown, the data registration module 122 can be used to read the configuration file in the scheduling center 120, which specifies the number of node information that a registration file can record. The configuration file can be pre-set or acquired in real time. Based on the node information received by the data receiving module 121 and the configuration information in the configuration file, the identifier of the registration file to be created in the registration center 130 is determined. The identifier of the registration file and the node information of the nodes to be monitored are sent to the registration center 130.

[0066] The data modification module 123 can be used to modify the identifier of the registration file when the number of nodes in the business system 110 changes. For example, such as... Figure 7As shown, when the number of nodes in business system 110 changes, the management node in business system 110 sends the changed node information to scheduling center 120. Data receiving module 121 in scheduling center 120 receives the node information. Data modification module 122 can modify the identifier of the registration file created by data registration module 122 based on the node information obtained by data receiving module 121. After modification, data registration module 122 sends the identifier of the registration file and the node information of the node to be monitored to registration center 130. For example, if the number of nodes in business system 110 increases from 6 to 8, and one registration file records two node information entries, then instead of the original requirement of 3 registration files to record node information, it should be modified to require 4 registration files to record node information (i.e., the number of info data nodes in ZooKeeper should be changed from 3 to 4). At this time, the node information of the newly added node can be received through the data receiving module 121, and the data modification module 123 can create a new identifier "info4" for the registration file. The newly created identifier of the registration file and the node information of the newly added node are sent to the registration center 130 through the data registration module 122, so that the registration file in the registration center 130 is changed from "info1, info2, info3" to "info1, info2, info3, info4".

[0067] For example, if the number of nodes in business system 110 is reduced from 8 to 6, and each registration file records information for two nodes, then the requirement of 4 registration files should be changed to 3 (i.e., the number of "info" data nodes in ZooKeeper should be changed from 4 to 3). In this case, data modification module 123 can change the identifiers of the registration files obtained by the data registration module from "info1, info2, info3, info4" to "info1, info2, info3". Of course, as the number of nodes in business system 110 increases, data modification module 123 can also correspondingly increase the number of "info" identifiers in the registration files.

[0068] The instance monitoring module 124 can be used to monitor instances in the monitoring center 140. For example, such as... Figure 8As shown, the instance monitoring module 124 can periodically (e.g., every 10 seconds, 1 minute, etc.) obtain the number of registration files (registration file identifiers) (i.e., the number of info nodes) in the registration center 130, and the number of instances corresponding to the instance status information (e.g., the number of live nodes in ZooKeeper). If the number of registration files and the number of instances are inconsistent, the instance scheduling module 125 is notified to modify the number of instances. Specifically, when the number of info nodes equals the number of live nodes, no processing is performed, and the number of info nodes and live nodes continues to be periodically obtained. When the number of info nodes is not equal to the number of live nodes, it is determined whether the number of info nodes is less than the number of live nodes. If the number of info nodes is less than the number of live nodes, it indicates that there are service monitoring instances in an idle state. At this time, an idle service monitoring instance can be obtained, and after obtaining an idle service monitoring instance, it can be stopped or deleted. If the number of info nodes is not less than the number of live nodes, it indicates that there are nodes to be monitored that have not yet been monitored by the service monitoring instance. At this time, a new service monitoring instance can be started to monitor the unmonitored nodes.

[0069] The instance scheduling module 125 can be used to send the identifier of the registration file to the monitoring center 140, so that the monitoring center 140 can start the instance based on the identifier of the registration file. For example, as shown... Figure 9 As shown, after obtaining the identifier of the registration file, the data registration module 122 can send the identifier to the instance scheduling module 125. The instance scheduling module then sends the received identifier to the monitoring center 140, enabling the monitoring center 140 to launch an instance based on the received identifier. After launching the instance, the instance can send a request to the registration center 130, including the identifier of the registration file. The registration center 130 can obtain the node information corresponding to the identifier and return the node information to the requesting instance. The instance can then monitor the node corresponding to the received node information.

[0070] The above is an introduction to the architecture of a service monitoring system provided in the embodiments of this application. Next, an example will be used to describe the architecture of this service monitoring system. For example, Figure 10 A schematic diagram of a service monitoring system is shown. Figure 10As shown, there are three business nodes in the business system: Node 1, Node 2, and Node 3. These three nodes work together to complete the business operations in business system 110. Nodes 1, 2, and 3 are servers with computing capabilities. Node 1's IP address and port (i.e., node information) are 0.0.0.1:80, Node 2's IP address and port (i.e., node information) are 0.0.0.2:80, Node 3's IP address and port (i.e., node information) are 0.0.0.3:80, and Node 4's IP address and port (i.e., node information) are 0.0.0.4:80. Node 1 acts as the management node, sending the node information of each node in business system 110 to the scheduling center 120. After receiving the node information from each node, the scheduling center 120 can read the pre-configured configuration file. The configuration file contains a registration file that can record two nodes. The registration file is used to record node information. Based on the number of nodes receiving node information (4) and the configuration information in the configuration registration file, the number of registration files can be obtained, which is 2. Then, the identifiers of the registration files can be determined as "info1" and "info2". The registration file corresponding to the identifier "info1" is used to record the node information of node 1 and node 2, and the registration file corresponding to the identifier "info2" is used to record the node information of node 3 and node 4. Next, the scheduling center 120 sends the identifiers "info1" and "info2" of the registration files to the monitoring center 140, and sends the identifiers "info1" and "info2" of the registration files, as well as the registration file corresponding to "info1" used to record the node information of node 1 and node 2, and the registration file corresponding to "info2" used to record the node information of node 3 and node 4, to the registration center 130. Registration center 130 is a ZooKeeper cluster with a master node zk1 and slave nodes zk2 and zk3. After receiving information from scheduling center 120, master node zk1 creates registration files info1 and info2. Info1 records node information for nodes 1 and 2, and info2 records node information for nodes 3 and 4. After completing the registration file creation, the information in master node zk1 is synchronized to slave nodes zk2 and zk3. Monitoring center 140 is Prometheus. After receiving the identifiers "info1" and "info2" from the registration files sent by scheduling center 120, Prometheus can start instance P1 based on identifier "info1" and instance P2 based on identifier "info2".After starting an instance, each instance can send a request message to a random slave node of the registry center 130. The request message carries an identifier for the registration file. Upon receiving the request message, the slave node can read the registration file corresponding to the identifier, thereby obtaining the node information recorded in the registration file. The slave node then returns the node information to the instance that sent the request message (node ​​information 1 and node information 2 are returned to instance P1, and node information 3 and node information 4 are returned to instance P2). After obtaining the node information, the instance can monitor the corresponding nodes. Specifically, instance P1 monitors the node with node information "0.0.0.1:80" (node ​​1) and the node with node information "0.0.0.2:80" (node ​​2), while instance P2 monitors the node with node information "0.0.0.3:80" (node ​​3) and the node with node information "0.0.0.4:80" (node ​​4). Each instance monitors a node and generates monitoring data. The monitoring center 140 can transmit the generated monitoring data to the storage center 150 for storage.

[0071] The above is an introduction to a service monitoring system provided by the embodiments of this application. The following is based on... Figures 1 to 10 The content described herein introduces a monitoring method provided by an embodiment of this application.

[0072] For example, Figure 11 The flowchart of a service monitoring method is shown. It should be understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. Figure 11 As shown, the monitoring method may include the following steps:

[0073] S1101: Obtain node information of multiple nodes to be monitored, and based on the node information of multiple nodes to be monitored, obtain at least one set of node information, wherein the set of node information includes the node information of at least one node to be monitored.

[0074] In this embodiment, node information of the node to be monitored can be obtained, for example, by obtaining node information sent by the node to be monitored through the aforementioned scheduling center 120. For instance, a node to be monitored has an IP address of 0.0.0.1 and a port of 80. This node can send its node information to the scheduling center 120, which then retrieves the node information. After obtaining the node information, pre-configured first configuration information in the scheduling center 120 can also be obtained. This first configuration information specifies a node information set containing a number of node information entries, for example, a node information set containing two node information entries. Based on the obtained node information and the content configured in the first configuration, the identifier of the node information set can be obtained. Then, based on the obtained identifier of the node information set, a node information set is generated, and the node information is recorded in the generated node information set. For example, after obtaining the node information and the identifier of the node information set, the aforementioned scheduling center 120 can send the node information and the identifier of the node information set to the registration center 130. The registration center 130 can generate a node information set based on the received identifier of the node information set and record the node information in the generated node information set.

[0075] For example, if there are 6 nodes to be monitored, then the obtained node information will be 6, represented as "node1", "node2", "node3", "node4", "node5", and "node6". If the first configuration information includes a node information set recording two node information sets, then 3 node information sets can be obtained based on this information, identified as "info1", "info2", and "info3". Each identifier represents a node information set containing the node information of two nodes to be monitored. For example, the node information set corresponding to "info1" records nodes "node1" and "node2", the node information set corresponding to "info2" records nodes "node3" and "node4", and the node information set corresponding to "info3" records nodes "node5" and "node6". The registry center 130 selects ZooKeeper. After ZooKeeper receives the node information and the identifiers of the node information sets, it records the node information as data nodes. Specifically, the node information set generated by ZooKeeper records node information in the form of data nodes, which can be recorded as / info / info1 / node / 0.0.0.1:80, / info / info1 / node / 0.0.0.2:80, / info / info2 / node / 0.0.0.3:80, / info / info2 / node / 0.0.0.4:80, / info / info3 / node / 0.0.0.5:80, and / info / info3 / node / 0.0.0.6:80.

[0076] S1102: Based on at least one set of node information, start at least one service instance. The service instance corresponds to the set of node information and is used to monitor each node to be monitored in the corresponding set of node information.

[0077] In this embodiment, service monitoring instances can be started based on the identifiers of the obtained node information sets, with each instance corresponding to (i.e., bound to) an identifier of a node information set. For example, after obtaining the identifiers "info1", "info2", and "info3" of the node information sets, the aforementioned scheduling center 120 sends the identifiers of the node information sets to the monitoring center 140. After receiving the identifiers of the node information sets, the monitoring center 140 starts the corresponding instances based on the identifiers of the node information sets, which are represented as instance 1, instance 2, and instance 3, respectively. Instance 1 corresponds to the identifier "info1", instance 2 corresponds to the identifier "info2", and instance 3 corresponds to the identifier "info3".

[0078] S1103: Create at least one instance monitoring node. Each instance monitoring node corresponds to a service instance and is used to monitor the status of the corresponding service instance. In this embodiment, each service monitoring instance can create an instance monitoring node after startup, and each instance monitoring node corresponds to an instance. The instance monitoring node can record the status of the instance, such as whether the instance is alive or idle.

[0079] S1104: Monitor at least one service instance based on at least one set of node information and at least one instance monitoring node.

[0080] In this embodiment, after starting the instance, the instance does not know which specific node to be monitored. Therefore, each instance can look up the node information recorded in the node information set corresponding to the instance, based on the identifier of the node information set corresponding to the instance. After obtaining the node information, the node to be monitored corresponding to the node information can be monitored.

[0081] For example, if instance 1 corresponds to the identifier "info1" in the node information set, then instance 1 can send a request message to the registration center 130, which includes the identifier "info1" in the node information set. After receiving the request from instance 1, the registration center 130 can query the node information recorded in the node information set corresponding to the identifier "info1" in the request message, obtaining "0.0.0.1:80" and "0.0.0.2:80", and return these two node information to instance 1. After obtaining the node information, instance 1 monitors the nodes "0.0.0.1:80" and "0.0.0.2:80".

[0082] In this way, the node information of the node to be monitored is obtained through the scheduling center. The scheduling center determines the identifier of the registration file based on the configuration information and the obtained node information. The registration file is determined in the registration center based on the identifier of the registration file and the node information. The monitoring center can start the corresponding instance based on the identifier of the registration file obtained from the scheduling center. Each instance obtains the node information of the node to be monitored through the registration center to monitor different nodes (the nodes recorded in the registration file).

[0083] Furthermore, this application also provides a method for dynamically adjusting monitoring instances, applicable when the number of nodes in a business system changes. For example, Figure 12 A schematic diagram illustrating a method for extending a service instance according to an embodiment of this application is shown. Figure 12 As shown, extending the service instance may include the following steps:

[0084] S1201: Get the first number of instances that are alive. These instances are used to monitor the nodes to be monitored.

[0085] In this embodiment, when each instance sends a request to the registration center 130, it can include its own status information in the request, which indicates whether the instance is alive. After receiving the request, the registration center 130 can record the liveness status of the instance. At this time, the scheduling center 120 can obtain the first number of live instances recorded in the registration center 130.

[0086] For example, instance 1 sends a request to registry center 130, carrying its own status information, indicating it is alive. After receiving the request, registry center 130 records instance 1 and its status. If registry center 130 is a ZooKeeper, a data node named " / alive / true" can be created, representing instance 1's liveness. The scheduling center can then retrieve the number of " / alive / true" nodes.

[0087] S1202: Obtain the second number of registration files. Registration files are used to record node information of the nodes to be monitored.

[0088] In this embodiment, the aforementioned scheduling center 120 can obtain the number of identifiers of the registration files recorded in the registration center 130 in real time or periodically.

[0089] S1203: Adjust the number of instances if the first quantity is different from the second quantity.

[0090] In this embodiment, after obtaining the number of registration file identifiers and the number of live instances, these two numbers can be compared. If the number of registration file identifiers equals the number of live instances, it indicates that all existing nodes in the business system 110 are being monitored by instances, and each instance has monitored nodes; no adjustment is needed. If the number of registration file identifiers is less than the number of live instances, it indicates that some started instances are idle and not monitoring any nodes to be monitored, resulting in wasted resources. In this case, the scheduling center 120 can instruct the monitoring center 140 to stop / delete the idle instances. If the number of registration file identifiers exceeds the number of live instances, it indicates that some nodes to be monitored have not yet been monitored by the monitoring center 140. In this case, the registration file identifier can be sent to the monitoring center 140, and the monitoring center 140 can restart the instance based on the registration file identifier. Therefore, by recording the instance liveness status in the registry center, the registry center can determine whether there are idle instances in the system or unmonitored nodes in the business system based on the number of identifiers of the registration files and the number of live instances recorded in the registry center. This allows the instances launched by the monitoring center to be scheduled by the scheduling center, thereby enabling the instances launched by the monitoring center to be expanded according to the nodes to be monitored.

[0091] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. Furthermore, in some possible implementations, each step in the above embodiments may be selectively executed according to actual circumstances; it may be partially or fully executed, without limitation here. All or part of any feature of any embodiment of this application can be freely and arbitrarily combined without contradiction. The combined technical solutions are also within the scope of this application.

[0092] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0093] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments. It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0094] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0095] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0096] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

Claims

1. A service monitoring method, characterized in that, The method includes: Obtain node information of multiple nodes to be monitored, and based on the node information of the multiple nodes to be monitored, obtain at least one set of node information, wherein the set of node information includes node information of at least one node to be monitored. Based on the at least one node information set, at least one service instance is started, the service instance corresponding to the node information set, and the service instance is used to monitor each node to be monitored in the corresponding node information set; Create at least one instance monitoring node, which corresponds to the service instance. The instance monitoring node is used to monitor the status of the corresponding service instance, including whether the instance is alive. Based on the at least one node information set and the at least one instance monitoring node, the at least one service instance is monitored, including: if the number of nodes in the node information set is greater than the number of instance monitoring nodes that monitor the service instance as alive, the service instance is expanded; if the number of nodes in the node information set is less than the number of instance monitoring nodes that monitor the service instance as alive, the service instance in the idle state is stopped or deleted.

2. The method according to claim 1, characterized in that, The method further includes storing monitoring data generated by the service instance monitoring the node to be monitored.

3. The method according to claim 1 or 2, characterized in that, Based on the node information of the multiple nodes to be monitored, at least one set of node information is obtained, including: Based on the number of nodes corresponding to the node information and the number of nodes recorded in a node information set indicated by the first configuration information, the number of nodes in the node information set is determined; Each node information set is assigned an identifier; Based on the identifier of the node information set, the node information set is created, wherein the name of the node information set includes at least the identifier of the node information set.

4. The method according to claim 3, characterized in that, The creation of at least one instance monitoring node specifically includes: Obtain the status of each service instance, and based on the status of the service instance, create at least one instance monitoring node, wherein each service instance corresponds to one instance monitoring node.

5. The method according to claim 1, characterized in that, The expansion of the service instance includes: Determine the newly added set of node information, and based on the newly added set of node information, start the service instance corresponding to the number of newly added node information sets.

6. A server cluster, characterized in that, include: Multiple nodes to be monitored, wherein the nodes to be monitored include node information; The first cluster is used to obtain node information of the plurality of nodes to be monitored, and based on the node information, to start at least one service instance, the service instance being used to monitor at least one of the nodes to be monitored; The second cluster is used to create at least one instance monitoring node, which corresponds to the service instance. The instance monitoring node is used to monitor the status of the corresponding service instance, including whether the instance is alive. The first cluster is further configured to monitor the at least one service instance based on the node information and the instance monitoring nodes, including: if the number of nodes in the node information set is greater than the number of instance monitoring nodes that have detected the service instance as alive, then the service instance is expanded; If the number of nodes in the node information set is less than the number of monitoring nodes that detect the service instance as alive, then the service instance that is in an idle state will be stopped or deleted.

7. A server comprising a memory and a processor, the memory being connected to the processor, the memory storing computer program instructions which, when executed by the processor, perform the method as described in any one of claims 1 to 5.

Images