MongoDB database cluster instance monitoring method and device

Abstract

The embodiment of the invention provides a MongoDB database cluster instance monitoring method and device, and the method comprises the steps: obtaining monitoring data of a MongoDB database cluster instance from an InfluxDB database, obtaining a replica set and a non-replica set from the monitoring data of the MongoDB database cluster instance based on a node query table, determining whether the replica set is abnormal or not based on a first judgment rule, and if yes, determining whether the replica set is abnormal or not based on a second judgment rule; if it is determined that the replica set is abnormal, the abnormal state of the replica set is stored in a node query table, if a normally-running Mongos node exists in a non-replica set, a fragment cluster is obtained from monitoring data of a MongoDB database cluster instance based on the node query table, and whether the fragment cluster is abnormal or not is determined based on a second judgment rule; and if it is determined that the fragmented cluster is abnormal, storing the abnormal state of the fragmented cluster in a node query table. By setting different judgment rules, the replica set and the fragmentation cluster are judged, and whether the replica set and the fragmentation cluster are abnormal or not is determined, so that overall monitoring of the replica set and the fragmentation cluster is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing environments, and in particular to a monitoring method and device for a MongoDB database cluster instance. Background Art

[0002] The existing Telegraf open source indicator collection tool provides multiple monitoring input plug-ins for the MongoDB distributed file storage NoSQL database. These plug-ins include monitoring of a single MongoDB database service node, and its fields generally refer to specific data fields in different monitoring statistics. Specifically, Telegraf provides the following monitoring plug-ins: mongodb_db_stats: used to monitor various statistics of the database, and the fields covered include the monitoring indicators of the database. mongodb_col_stats: used to monitor the performance and status of the collection, and its fields include the monitoring indicators of the collection. mongodb_shard_stats: used to monitor the relevant information of the shard. Although it includes monitoring of the shard, it can currently only collect the following indicators: in_use (integer): indicates the storage space in use; available (integer): indicates the available storage space; created (integer): indicates the creation time of the shard; refreshing (integer): indicates the number of shards being refreshed.

[0003] However, the mongodb_shard_stats plug-in cannot monitor the status of shard nodes, let alone fully reflect the health of the entire shard cluster. And from the analysis of existing monitoring indicators, the state monitoring indicator and the member_status monitoring indicator can monitor the role status of a single node, and can only determine the status of a single node, but cannot determine whether the replica set and shard cluster in the MongoDB database are abnormal as a whole, that is, it cannot monitor the replica set and shard cluster in the MongoDB database. Summary of the invention

[0004] The present invention provides a method and device for monitoring a MongoDB database cluster instance, which solves the problem in the prior art that a MongoDB database replica set and a shard cluster cannot be monitored.

[0005] According to one aspect of the present invention, a method for monitoring a MongoDB database cluster instance is provided, comprising:

[0006] Get monitoring data of MongoDB database cluster instances from InfluxDB database;

[0007] Based on the node query table, obtaining replica sets and non-replica sets from monitoring data of the MongoDB database cluster instance;

[0008] Based on the first judgment rule, determining whether the replica set is abnormal;

[0009] If it is determined that the replica set is abnormal, the abnormal state of the replica set is stored in the node query table;

[0010] If there is a normally operating Mongos node in the non-replica set, then based on the node query table, a shard cluster is obtained from the monitoring data of the MongoDB database cluster instance;

[0011] Based on the second judgment rule, determine whether the shard cluster is abnormal;

[0012] If it is determined that an abnormality occurs in the shard cluster, the abnormal state of the shard cluster is stored in the node query table.

[0013] In one embodiment, determining whether the replica set is abnormal based on the first judgment rule includes:

[0014] If the number of nodes in the replica set that are running normally is greater than or equal to the preset number of nodes, then obtaining the instance role of the node in the replica set; wherein the instance role indicates whether the node is a master node or a slave node;

[0015] If the instance roles of the nodes of the replica set do not satisfy one master node and two slave nodes or one master node and one slave node, it is determined that an abnormality occurs in the replica set.

[0016] In one embodiment, the monitoring method of the MongoDB database cluster instance further includes:

[0017] If the number of nodes in the replica set that are operating normally is less than the preset number of nodes, it is determined that an abnormality has occurred in the replica set.

[0018] In one embodiment, obtaining monitoring data of a MongoDB database cluster instance from an InfluxDB database includes:

[0019] Obtain the creation time of the monitoring data of the MongoDB database cluster instance;

[0020] If the difference between the current time and the creation time is greater than a preset difference, monitoring data of the MongoDB database cluster instance is obtained from the InfluxDB database.

[0021] In one embodiment, determining whether the shard cluster is abnormal based on the second judgment rule includes:

[0022] Obtain the instance role of the shard node in the shard cluster;

[0023] If the instance role of the shard node in the shard cluster does not satisfy one master node and at least one slave node, it is determined that an abnormality occurs in the shard node and the abnormal state of the shard node is stored in the node query table.

[0024] In one embodiment, the monitoring method of the MongoDB database cluster instance further includes:

[0025] If the instance role of the shard node in the shard cluster satisfies one master node and at least one slave node, determine the instance role of the config server node in the shard cluster;

[0026] If the instance role of the config server node in the sharded cluster does not satisfy one master node and at least one slave node, it is determined that an abnormality occurs in the config server node and the abnormal state of the config server node is stored in the node query table.

[0027] In one embodiment, the monitoring method of the MongoDB database cluster instance further includes:

[0028] Determine a target Mongos node in the non-replica set; wherein the target Mongos node is used to provide a routing function for the shard cluster;

[0029] If none of the target Mongos nodes are operating normally, it is determined that an abnormality has occurred in the shard cluster and the abnormal state of the shard cluster is stored in the node query table.

[0030] In one embodiment, before acquiring the replica set and the non-replica set from the monitoring data of the MongoDB database cluster instance based on the node query table, the monitoring method of the MongoDB database cluster instance further includes:

[0031] If the node in the monitoring data of the MongoDB database cluster instance has no instance role, sending a preset number of reply requests to the node in the monitoring data of the MongoDB database cluster instance;

[0032] If the number of reply responses sent by the node in the monitoring data of the MongoDB database cluster instance does not meet the preset number of times, it is determined that the network connection of the node in the monitoring data of the MongoDB database cluster instance is abnormal and the abnormal network connection status of the node in the monitoring data of the MongoDB database cluster instance is stored in the node query table.

[0033] In one embodiment, the monitoring method of the MongoDB database cluster instance further includes:

[0034] If the number of reply responses sent by the node in the monitoring data of the MongoDB database cluster instance meets the preset number, obtaining the key of the MongoDB database;

[0035] Connecting a Mongo client to the MongoDB database according to the key of the MongoDB database;

[0036] Obtain the response data of the Mongo client;

[0037] If the response data of the Mongo client indicates that the connection with the MongoDB database is unsuccessful, it is determined that the service of the MongoDB database is abnormal and the service abnormality state of the MongoDB database is stored in the node query table.

[0038] According to another aspect of the present invention, a monitoring device for a MongoDB database cluster instance is provided, comprising:

[0039] The acquisition module is used to obtain the monitoring data of the MongoDB database cluster instance from the InfluxDB database;

[0040] A query module, used to obtain replica sets and non-replica sets from the monitoring data of the MongoDB database cluster instance based on a node query table;

[0041] A first judgment module, used to determine whether the replica set is abnormal based on a first judgment rule;

[0042] A replica set exception module, configured to store the abnormal state of the replica set in the node query table if it is determined that the replica set is abnormal;

[0043] A shard cluster acquisition module, configured to acquire a shard cluster from the monitoring data of the MongoDB database cluster instance based on the node query table if there is a normally operating Mongos node in the non-replica set;

[0044] A second judgment module, used to determine whether the shard cluster is abnormal based on a second judgment rule;

[0045] The storage module is used to store the abnormal state of the shard cluster in the node query table if it is determined that the shard cluster is abnormal.

[0046] The embodiment of the present invention provides a monitoring method and device for a MongoDB database cluster instance, including: obtaining monitoring data of a MongoDB database cluster instance from an InfluxDB database, obtaining a replica set and a non-replica set from the monitoring data of the MongoDB database cluster instance based on a node query table, determining whether the replica set is abnormal based on a first judgment rule, if it is determined that the replica set is abnormal, storing the abnormal state of the replica set in the node query table, if there is a normally operating Mongos node in the non-replica set, obtaining a shard cluster from the monitoring data of the MongoDB database cluster instance based on the node query table, determining whether the shard cluster is abnormal based on a second judgment rule, if it is determined that the shard cluster is abnormal, storing the abnormal state of the shard cluster in the node query table. By setting different judgment rules, the replica set and the shard cluster are judged and it is determined whether the replica set and the shard cluster are abnormal, so that it can be determined whether the replica set and the shard cluster as a whole are abnormal, so as to realize monitoring of the replica set and the shard cluster as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flowchart of a method for monitoring a MongoDB database cluster instance provided by an exemplary embodiment of the present invention.

[0048] Figure 2 It is a flowchart of a replica set abnormality determination method provided by an exemplary embodiment of the present invention.

[0049] Figure 3 It is a flowchart of a method for determining anomalies of a sharded cluster provided by an exemplary embodiment of the present invention.

[0050] Figure 4 It is a schematic diagram of the structure of a monitoring device for a MongoDB database cluster instance provided by an exemplary embodiment of the present invention.

[0051] Figure 5 It is a structural diagram of a monitoring device for a MongoDB database cluster instance provided by another exemplary embodiment of the present invention.

[0052] Figure 6 is a structural diagram of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0053] Telegraf is an open source indicator collection tool based on plug-ins. It is mainly used in conjunction with a time series database for performance monitoring. Usually, Telegraf captures a batch of indicator data at intervals and sends the data to a time series database or other custom output. Usually, it captures a batch of indicator data (such as the CPU usage of the machine, disk IO, network conditions, the number of sessions on the MySQL database server, the number of connections on the MongoDB database server, etc.) at intervals and sends them to a time series database, message queue, or custom exports to a certain place for downstream application processing (such as alarms).

[0054] InfluxDB is an open source distributed time series, event, and indicator database. Written in Go, it does not require external dependencies. Its design goal is to achieve distributed and horizontal scalability. InfluxDB is used to store and query data, handle data processing or monitoring and alerting purposes in the background, user dashboards, and visualize and explore data.

[0055] A cloud database is a database that is optimized or deployed in a virtual computing environment. It can achieve advantages such as pay-as-you-go, on-demand expansion, high availability, and storage integration. Users can easily complete database application, create a cloud database, and destroy a cloud database in the cloud database console.

[0056] MongoDB is a NoSQL database based on distributed file storage, which is used to store massive data and achieve high availability and scalability. In the MongoDB database, Replica Set and Sharded Cluster are the two main ways to achieve high availability and scalability. Replica Set provides redundancy and high availability and is the basis of all production deployments. MongoDB replication requires at least two nodes. One of them is the master node, which is responsible for processing client requests, and the rest are slave nodes, which are responsible for replicating the data on the master node. MongoDB sharding refers to the process of splitting data and distributing it on different machines. Each shard contains a subset of the shard data, and each shard can be deployed as a replica set. The Config Server configuration server stores the metadata and configuration settings of the cluster. Query Routers, i.e. mongos, act as query routers and provide an interface between the client application and the sharded cluster. The monitoring of the MongoDB database cluster is the monitoring of the replica set and the sharded cluster. Multiple cloud database MongoDB database instances are deployed in the cloud computing environment, and the cluster status is monitored to facilitate the operation and maintenance management of the MongoDB database.

[0057] XXL-JOB is a lightweight distributed task scheduling framework.

[0058] Figure 1 FIG. 1 is a flow chart of a method for monitoring a MongoDB database cluster instance provided by an exemplary embodiment of the present invention. Figure 1 As shown in the figure, the monitoring method of the MongoDB database cluster instance includes:

[0059] Step 110: Get the monitoring data of the MongoDB database cluster instance from the InfluxDB database.

[0060] In an embodiment of the present invention, based on the original Telegraf MongoDB database monitoring and collection input plug-in, the monitoring data of the database is collected regularly and reported to the InfluxDB database. The back-end data service layer processes the cluster node status data in the MongoDB database in the InfluxDB database, and the front-end browser client provides users with the ability to view the database instance status and monitor the abnormal status of the alarm instance.

[0061] First, deploy the Telegraf client collection tool on the MongoDB database server host node, configure the telegraf.conf configuration file, and configure the monitoring collection indicators, collection cycle, and collection monitoring database output to the InfluxDB database and other information.

[0062] Specifically, configure service, account, region, resource_id, resource_name, instance_id, and user_ip in the global_tags configuration item to identify the uniqueness of the MongoDB service node, configure the interval monitoring and collection cycle in the agent, configure the local MongoDB service address in inputs.mongodb, and configure the user name, password, and other information that needs to be reported to the InfluxDB database in outputs.influxdb.

[0063] Among them, global_tags is a configuration item used to identify the uniqueness of the service node of the MongoDB database. This item usually includes multiple tags to accurately identify and distinguish different nodes during monitoring and data analysis. service: The name of the service, used to identify a specific service. With this tag, you can quickly find the corresponding service instance. account: Account information, which may be used to distinguish service nodes for different users or tenants. region: Regional information, identifying the geographical region where the service node is located (such as: us-east-1, eu-west-1). This helps optimize data transmission and reduce latency. resource_id: Resource ID, used to uniquely identify resources in cloud services. It is usually a unique identifier generated by the platform. resource_name: Resource name, which is a user-defined name used to describe the function or purpose of the resource. instance_id: Instance ID, used to identify a specific service instance. In a cloud environment, each instance usually has a unique identifier. user_ip: User IP address, which identifies the IP address of the user connecting to the MongoDB service, which helps with security auditing and access control.

[0064] The agent configuration item is used to set the parameters related to monitoring collection. inputs.mongodb is used to configure the relevant information of the MongoDB service so that the monitoring tool can connect to the MongoDB instance and collect data. The outputs.influxdb configuration item is used to specify the output target of the monitoring data, and send the collected data and information to the InfluxDB database for storage and analysis. This information includes user name, password and other information.

[0065] Then, the backend Java data service layer configures the XXLJOB scheduled task to call the monitoring method of the monitoring data of the MongoDB database cluster instance, that is, an automated mechanism is set in the system, which regularly checks the health status and performance indicators of the MongoDB cluster to ensure the stability and availability of the system.

[0066] Then, the database cluster instance monitoring status method of the backend Java data service layer processes the status data of the MongoDB cluster nodes collected by InfluxDB, calculates the cluster status of the entire database cluster instance based on the status data of the real node, and saves the node status of the MongoDB database cluster.

[0067] Specifically, the monitoring data of the MongoDB database cluster instance obtained is for all cluster instances in the existing cloud environment, including instances in operation and abnormal operation, and does not include the monitoring data of database cluster instances that have been deleted by the user. Then, the node information instance_node table (node ​​query table) of the MongoDB database instance is queried according to the database instance instance. Among them, the database instance instance means that in database management, the instance refers to the specific operating environment of a database. For MongoDB, an instance can be a separate database server or a node in a cluster. Instances usually have their specific configuration, data storage, users and permission settings. The instance_node table is used to store node information related to the MongoDB database instance, which includes: the IP address and port number of the node, the status of the node (such as: online, offline), the role of the node (such as: master node or slave node), load and performance indicators.

[0068] Step 120: Based on the node query table, obtain the replica set and non-replica set from the monitoring data of the MongoDB database cluster instance.

[0069] In an embodiment of the present invention, a shard cluster for MongoDB includes a shard node and a config server node, and the Mongos node is a routing node, which is a non-replica set. For ease of management, the Mongos instance is listed separately for query. Therefore, it is necessary to monitor three types of monitoring nodes: monitoring nodes of replica sets, monitoring nodes of shard clusters, and monitoring nodes of non-replica sets. Through the node query table, the monitoring list of replica sets and the monitoring list of non-replica sets can be queried, and each monitoring list contains monitoring nodes. In the shard architecture of MongoDB, the mongos node is a routing service responsible for routing requests between the client and the shard cluster. It acts as an intermediary between the client and the entire shard cluster. Mongos receives requests from the client and forwards the requests to the appropriate shard node based on the routing information provided by the configuration server (config server). This means that the client does not need to know the specific shard location and can interact directly with mongos.

[0070] Step 130: Based on the first judgment rule, determine whether the replica set is abnormal.

[0071] Step 140: If it is determined that an abnormality occurs in the replica set, the instance status of the replica set is recorded as abnormal in the node query table.

[0072] In the embodiment of the present invention, if it is determined that an abnormality occurs in the replica set, the abnormal state of the replica set is stored in the node query table, that is, by setting the instance state of the replica set to ERROR, the instance with the problem can be quickly identified, which is convenient for operation and maintenance personnel to monitor and troubleshoot. ERROR is used to indicate an abnormal state.

[0073] Step 150: If there is a normally operating Mongos node in the non-replica set, the shard cluster is obtained from the monitoring data of the MongoDB database cluster instance based on the node query table.

[0074] In the embodiment of the present invention, it is possible to determine whether a Mongos node is abnormal by obtaining system performance indicators such as CPU usage, memory usage, network traffic, and disk I / O of the Mongos node. If an abnormal performance indicator appears, it means that the Mongos node is abnormal, otherwise it is normal. In addition, the response time of the Mongos node to the request can also be obtained. If there is an abnormal delay, it may indicate that the Mongos node is faulty.

[0075] Optionally, it is possible to determine whether the Mongos node is communicating normally with all shard nodes and config server nodes. Specifically, the Mongos node is controlled to send routing requests to the shard node or config server node. If the Mongos node cannot obtain the response data sent by the shard node or config server node within a preset time period, it means that the current network is unavailable or the configuration is wrong, which may cause the Mongos node to be unable to route the request, that is, it is determined that the Mongos node is abnormal.

[0076] Step 160: Based on the second judgment rule, determine whether the shard cluster is abnormal.

[0077] Step 170: If it is determined that an abnormality occurs in the shard cluster, the abnormal state of the shard cluster is stored in the node query table.

[0078] In an embodiment of the present invention, if it is determined that an abnormality occurs in a sharded cluster, the abnormal state of the sharded cluster is stored in a node query table, that is, by setting the instance state of the sharded cluster to ERROR, the instance with the problem can be quickly identified, making it easier for operation and maintenance personnel to monitor and troubleshoot.

[0079] The embodiment of the present invention provides a monitoring method for a MongoDB database cluster instance, including: obtaining monitoring data of a MongoDB database cluster instance from an InfluxDB database, obtaining a replica set and a non-replica set from the monitoring data of the MongoDB database cluster instance based on a node query table, determining whether the replica set is abnormal based on a first judgment rule, if it is determined that the replica set is abnormal, storing the abnormal state of the replica set in the node query table, if there is a normally operating Mongos node in the non-replica set, obtaining a shard cluster from the monitoring data of the MongoDB database cluster instance based on the node query table, determining whether the shard cluster is abnormal based on a second judgment rule, if it is determined that the shard cluster is abnormal, storing the abnormal state of the shard cluster in the node query table. By setting different judgment rules, the replica set and the shard cluster are judged and it is determined whether the replica set and the shard cluster are abnormal, so that it can be determined whether the replica set and the shard cluster as a whole are abnormal, so as to realize monitoring of the replica set and the shard cluster as a whole. The present invention processes the status data of the MongoDB cluster nodes collected by Telegraf and stored in the InfluxDB database through the Java back-end data service layer, calculates the cluster status of the monitoring data of the entire database cluster instance according to the status data of the real node, and saves the node status of the MongoDB database cluster for the front-end browser client to display the monitoring data and node status of the MongoDB database cluster instance, thereby solving the problem that it is difficult to monitor the replica set and shard cluster status of the MongoDB database by relying solely on Telegraf, and facilitating the operation and maintenance management of the MongoDB database.

[0080] Figure 2 FIG. 1 is a flow chart of a replica set abnormality determination method provided by an exemplary embodiment of the present invention. Figure 2 As shown, step 130 may include:

[0081] Step 131: Determine whether the number of nodes in the replica set that are operating normally is greater than or equal to a preset number of nodes.

[0082] In an embodiment of the present invention, if the result of step 131 is yes, that is, the number of nodes running normally in the replica set is greater than or equal to the preset number of nodes, step 132 is executed; if the result of step 131 is no, that is, the number of nodes running normally in the replica set is less than the preset number of nodes, step 134 is executed.

[0083] In an embodiment of the present invention, whether an abnormality occurs in a replica set is determined by the instance role of the node in the replica set. Specifically, if the number of nodes in the replica set that are operating normally is greater than the preset number of nodes, the instance role of the node in the replica set is obtained, where the instance role indicates whether the node is a master node or a slave node. The preset number of nodes is 2. Taking a 3-node replica set as an example, if the number of nodes that are operating normally is greater than or equal to 2 nodes, it means that at least 2 nodes are operating normally.

[0084] Step 132: Obtain the instance role of the node in the replica set, where the instance role indicates whether the node is a master node or a slave node.

[0085] Step 133: Determine whether the instance role of the node of the replica set does not satisfy one master node and two slave nodes or one master node and one slave node.

[0086] In an embodiment of the present invention, if the judgment result of step 133 is yes, that is, the instance role of the node of the replica set does not satisfy one master node and two slave nodes or one master node and one slave node, then step 134 is executed; if the judgment result of step 133 is no, that is, the instance role of the node of the replica set satisfies one master node and two slave nodes or one master node and one slave node, then step 135 is executed.

[0087] Step 134: Determine that an abnormality occurs in the replica set.

[0088] Step 135: Determine that the replica set is normal.

[0089] In an embodiment of the present invention, a replica set must have a master node to process write requests. If there is no master node, the entire replica set will not be able to process write operations, resulting in service interruption. And there must be at least one slave node to ensure that failover can be performed when the master node fails. If there is only a master node and no slave node, the replica set will not be able to elect a new master node when the master node fails, resulting in service unavailability. Therefore, if the instance role of the node of the replica set does not satisfy one master node and two slave nodes or one master node and one slave node, it is determined that the replica set is abnormal. Taking a 3-node replica set as an example, the instance role of one master and two slaves or one master and one slave both indicate that the replica set is in a normal state, and the instance state of the replica set in the node query table is recorded as RUNNING normal. In addition, the replica set nodes or replica set instances with more than 3 nodes are the same as the 3-node judgment, and will not be repeated again.

[0090] In one embodiment, the monitoring method of a MongoDB database cluster instance may be specifically implemented as follows: if the number of nodes in a replica set that are operating normally is less than a preset number of nodes, it is determined that an abnormality has occurred in the replica set.

[0091] In the embodiment of the present invention, if the normal operation status of the node is less than 2, the replica set is abnormal, and the status of the replica set is recorded as ERROR abnormal in the node query table.

[0092] In one embodiment, step 110 can be specifically implemented as follows: obtaining the creation time of the monitoring data of the MongoDB database cluster instance; if the difference between the current time and the creation time is greater than a preset difference, obtaining the monitoring data of the MongoDB database cluster instance from the InfluxDB database.

[0093] In an embodiment of the present invention, the creation time of the MongoDB cluster instance is obtained, and it is determined whether the instance was created within the past 5 minutes. Whether the MongoDB database cluster instance is a new instance is determined by judging the difference between the current time and the creation time. For example, if the current time is 4 o'clock and the creation time is 3 o'clock, then the difference between the current time and the creation time is less than 5 minutes, which means that the MongoDB database cluster instance is a new instance. Therefore, it is necessary to query again in the next round of scheduled tasks to obtain the MongoDB database cluster instance whose difference between the current time and the creation time is greater than the preset difference. If the MongoDB cluster instance is identified as a new instance, the monitoring data generated since its startup will be retained. To ensure the integrity of the monitoring data, we will retain the latest 5 minutes of monitoring data to prevent the situation where the monitoring data is not generated because the database has not been fully started. This ensures that during instance initialization, we can effectively capture and record the required monitoring information.

[0094] Figure 3 FIG. 1 is a flow chart of a method for determining anomalies of a sharded cluster provided by an exemplary embodiment of the present invention. Figure 3 As shown, step 160 may include:

[0095] Step 161: Get the instance role of the shard node in the shard cluster.

[0096] In the embodiment of the present invention, in a shard cluster, a shard node (shard node) refers to a node that stores and manages data shards. Sharding is the division of a data set into multiple parts (or fragments) for distributed storage and processing. Each shard node is responsible for processing a specific data shard, thereby achieving load balancing and improving the scalability and fault tolerance of the system.

[0097] Step 162: Determine whether the instance role of the shard node in the shard cluster does not satisfy one master node and at least one slave node.

[0098] In the embodiment of the present application, if the judgment of step 162 is yes, that is, it is determined that the instance role of the shard node in the shard cluster does not satisfy one master node and at least one slave node, then execute step 163. If the judgment of step 162 is no, that is, it is determined that the instance role of the shard node in the shard cluster satisfies one master node and at least one slave node, then execute step 165.

[0099] Step 163: Determine if an abnormality occurs in the shard node and store the abnormal state of the shard node in the node query table.

[0100] In the embodiment of the present invention, if there is a normally operating Mongos node in the non-replica set, it means that the shard cluster can perform routing normally, so the status of the node of the shard cluster can be determined.

[0101] Specifically, if there is a bad node in the shard node of the shard cluster, there is a problem in the data acquisition of the shard shard in the shard cluster, the shard node is abnormal, and the abnormal state of the shard node is recorded in the node query table and the state of the shard node is marked as ERROR abnormal.

[0102] The method for determining whether a shard node in a shard cluster has a bad node specifically includes: if the instance role of the shard node in the shard cluster does not satisfy one master node and at least one slave node, it is determined that the shard node is abnormal, and then the shard cluster is abnormal.

[0103] Step 164: Determine that the shard cluster is abnormal.

[0104] Step 165: Determine the instance role of the config server node in the sharded cluster.

[0105] In an embodiment of the present invention, in a sharded cluster, a config server node (configuration server node) is a dedicated node for storing and managing cluster configuration. The config server node stores the metadata of the cluster, including shard information, node status, routing information, and distribution of data shards. The config server provides information to guide the client on how to route requests to the correct shard node. The client obtains the current shard information by querying the config server to ensure that the request can be correctly sent to the corresponding shard node. The config server monitors the status of each node, including online / offline, load status, etc., so as to update the configuration and routing information of the cluster in a timely manner.

[0106] Step 166: Determine whether the instance role of the config server node in the sharded cluster does not satisfy one master node and at least one slave node.

[0107] In the embodiment of the present invention, if the result of step 166 is yes, that is, it is determined that the instance role of the configserver node in the sharded cluster does not satisfy one master node and at least one slave node, then step 167 is executed. If the result of step 166 is no, that is, it is determined that the instance role of the config server node in the sharded cluster satisfies one master node and at least one slave node, then step 168 is executed.

[0108] Step 167: Determine that an abnormality occurs in the config server node and store the abnormal state of the config server node in the node query table.

[0109] In the embodiment of the present invention, after executing step 167, step 164 is executed again. That is, an abnormality in the config server node can lead to an abnormality in the shard cluster.

[0110] Step 168: Determine that the shard cluster is normal.

[0111] In an embodiment of the present invention, if the instance role of the shard node in the shard cluster satisfies one master node and at least one slave node, it means that the shard node in the shard cluster is normal, and then the instance role of the config server node in the shard cluster is determined; if the instance role of the config server node in the shard cluster does not satisfy one master node and at least one slave node, it is determined that an abnormality has occurred in the config server node, and the abnormal status of the config server node is recorded in the node query table and the status of the config server node is marked as ERROR abnormality.

[0112] In one embodiment, the monitoring method of a MongoDB database cluster instance can be specifically implemented as follows: determining a target Mongos node in a non-replica set; wherein the target Mongos node is used to provide a routing function for a sharded cluster; if none of the target Mongos nodes operate normally, it is determined that an abnormality has occurred in the sharded cluster and the abnormal state of the sharded cluster is stored in a node query table.

[0113] In the embodiment of the present invention, if all Mongos routing nodes of the shard cluster are abnormal, the shard cluster cannot perform routing normally, and the shard cluster status is recorded as ERROR abnormal in the node query table.

[0114] In one embodiment, before step 120, the monitoring method of the MongoDB database cluster instance can be specifically implemented as follows: if the node in the monitoring data of the MongoDB database cluster instance does not have an instance role, a preset number of reply requests are sent to the node in the monitoring data of the MongoDB database cluster instance; if the number of reply responses sent by the node in the monitoring data of the MongoDB database cluster instance does not meet the preset number, it is determined that the network connection of the node in the monitoring data of the MongoDB database cluster instance is abnormal and the abnormal network connection status of the node in the monitoring data of the MongoDB database cluster instance is stored in the node query table.

[0115] In the embodiment of the present invention, the monitoring collection and reporting cycle of Telegraf is 30 seconds. This step queries the node state data in the monitoring data of the MongoDB database cluster instance in the past 1 minute. If the acquired node state data monitoring role data is empty, that is, there is no instance role, it is necessary to further check the node status. First, check the network factors by executing ping 4 times to the MongoDB service node. If the network is abnormal, record the instance node status instance_node table instance node status DB_DOWN, and record the network abnormality.

[0116] Specifically, a preset number of reply requests are sent to the node in the monitoring data of the MongoDB database cluster instance. If the number of reply responses sent by the node in the monitoring data of the MongoDB database cluster instance does not meet the preset number, it is determined that the network connection of the node in the monitoring data of the MongoDB database cluster instance is abnormal and the network connection abnormality status of the node in the monitoring data of the MongoDB database cluster instance is stored in the node query table, where the preset number of times is 4.

[0117] In one embodiment, the monitoring method of a MongoDB database cluster instance can be specifically implemented as follows: if the number of reply responses sent by a node in the monitoring data of the MongoDB database cluster instance meets a preset number, then obtaining the key of the MongoDB database; connecting the Mongo client to the MongoDB database according to the key of the MongoDB database; obtaining the response data of the Mongo client; if the response data of the Mongo client indicates that the connection with the MongoDB database is unsuccessful, then determining that the service of the MongoDB database is abnormal and storing the service abnormality status of the MongoDB database in the node query table.

[0118] In an embodiment of the present invention, if the network connection is normal, the service of the MongoDB database is connected through MongoClient (Mongo client). If the service of the MongoDB database is abnormal, the node status of the monitoring data of the MongoDB database cluster instance is recorded in the node query table as DB_DOWN. If the acquired state node monitoring role data is not empty, that is, the node in the monitoring data of the MongoDB database cluster instance has no instance role, then the instance_node table is updated, and the node role and the node status of the monitoring data of the MongoDB database cluster instance are updated to RUNNING normal operation state. The nodes in the monitoring data of the above-mentioned MongoDB database cluster instance include nodes of the replica set, non-replica set nodes and nodes of the sharded cluster.

[0119] Among them, based on the node query table, obtaining replica sets and non-replica sets from the monitoring data of the MongoDB database cluster instance includes:

[0120] If the node query table indicates that the node status of the monitoring data of the MongoDB database cluster instance is in a normal operating state, the replica set and the non-replica set are obtained from the monitoring data of the MongoDB database cluster instance.

[0121] Figure 4 FIG. 1 is a schematic diagram of a monitoring device for a MongoDB database cluster instance provided by an exemplary embodiment of the present invention. Figure 4 As shown, the monitoring device 20 of the MongoDB database cluster instance includes: an acquisition module 201, which is used to acquire the monitoring data of the MongoDB database cluster instance from the InfluxDB database; a query module 202, which is used to acquire the replica set and the non-replica set from the monitoring data of the MongoDB database cluster instance based on the node query table; a first judgment module 203, which is used to determine whether the replica set is abnormal based on the first judgment rule; a replica set abnormality module 204, which is used to store the abnormal state of the replica set in the node query table if it is determined that the replica set is abnormal; a shard cluster acquisition module 205, which is used to acquire the shard cluster from the monitoring data of the MongoDB database cluster instance based on the node query table if there is a normally operating Mongos node in the non-replica set; a second judgment module 206, which is used to determine whether the shard cluster is abnormal based on the second judgment rule; a storage module 207, which is used to store the abnormal state of the shard cluster in the node query table if it is determined that the shard cluster is abnormal.

[0122] The embodiment of the present invention provides a monitoring device for a MongoDB database cluster instance, comprising: an acquisition module that acquires monitoring data of a MongoDB database cluster instance from an InfluxDB database, a query module that acquires a replica set and a non-replica set from the monitoring data of the MongoDB database cluster instance based on a node query table, a first judgment module that determines whether the replica set is abnormal based on a first judgment rule, and a replica set abnormal module that stores the abnormal state of the replica set in the node query table if it determines that the replica set is abnormal, a shard cluster acquisition module that acquires a shard cluster from the monitoring data of the MongoDB database cluster instance based on the node query table if there is a normally operating Mongos node in the non-replica set, a second judgment module that determines whether the shard cluster is abnormal based on a second judgment rule, and a storage module that stores the abnormal state of the shard cluster in the node query table if it determines that the shard cluster is abnormal. By setting different judgment rules, the replica set and the shard cluster are judged and it is determined whether the replica set and the shard cluster are abnormal, so that it can be determined whether the replica set and the shard cluster as a whole are abnormal, so as to realize monitoring of the replica set and the shard cluster as a whole.

[0123] Figure 5 FIG. 1 is a schematic diagram of a monitoring device for a MongoDB database cluster instance provided by another exemplary embodiment of the present invention. Figure 5 As shown, the first judgment module 203 may include: a role acquisition unit 2031, which is used to obtain the instance role of the node in the replica set if the number of nodes in the replica set that are normally operating is greater than or equal to a preset number of nodes; wherein the instance role indicates whether the node is a master node or a slave node; and a determination unit 2032, which is used to determine that an abnormality has occurred in the replica set if the instance role of the node in the replica set does not satisfy one master node and two slave nodes or one master node and one slave node.

[0124] In one embodiment, the monitoring device 20 of the MongoDB database cluster instance can be specifically configured as follows: if the number of nodes in the replica set that are operating normally is less than a preset number of nodes, it is determined that an abnormality has occurred in the replica set and the abnormal state of the replica set is stored in the node query table.

[0125] In one embodiment, the acquisition module 201 may include: a time acquisition unit 2011, used to obtain the creation time of the monitoring data of the MongoDB database cluster instance; an acquisition subunit 2012, used to obtain the monitoring data of the MongoDB database cluster instance from the InfluxDB database if the difference between the current time and the creation time is greater than a preset difference.

[0126] In one embodiment, the second judgment module 206 may include: a node acquisition unit 2061, used to obtain the instance role of the shard node in the shard cluster; an exception unit 2062, used to determine that an exception occurs in the shard node and store the abnormal state of the shard node in the node query table if the instance role of the shard node in the shard cluster does not satisfy one master node and at least one slave node.

[0127] In one embodiment, the second judgment module 206 can be specifically configured as follows: if the instance role of the shard node in the shard cluster satisfies one master node and at least one slave node, then the instance role of the config server node in the shard cluster is determined; if the instance role of the config server node in the shard cluster does not satisfy one master node and at least one slave node, then it is determined that an abnormality has occurred in the config server node and the abnormal state of the config server node is stored in the node query table.

[0128] In one embodiment, the monitoring device 20 of the MongoDB database cluster instance can be specifically configured as follows: determining a target Mongos node in a non-replica set; wherein the target Mongos node is used to provide routing function for the sharded cluster; if none of the target Mongos nodes operate normally, it is determined that an abnormality has occurred in the sharded cluster and the abnormal state of the sharded cluster is stored in a node query table.

[0129] In one embodiment, before the query module 202, the monitoring 20 of the MongoDB database cluster instance can be specifically configured as follows: if the node in the monitoring data of the MongoDB database cluster instance has no instance role, a preset number of reply requests are sent to the node in the monitoring data of the MongoDB database cluster instance; if the number of reply responses sent by the node in the monitoring data of the MongoDB database cluster instance does not meet the preset number, it is determined that the network connection of the node in the monitoring data of the MongoDB database cluster instance is abnormal and the abnormal network connection status of the node in the monitoring data of the MongoDB database cluster instance is stored in the node query table.

[0130] In one embodiment, the monitoring 20 of the MongoDB database cluster instance can be specifically configured as follows: if the number of reply responses sent by the node in the monitoring data of the MongoDB database cluster instance meets the preset number, the key of the MongoDB database is obtained; according to the key of the MongoDB database, the Mongo client is connected to the MongoDB database; the response data of the Mongo client is obtained; if the response data of the Mongo client indicates that the connection with the MongoDB database is unsuccessful, the service of the MongoDB database is determined to be abnormal and the service abnormality status of the MongoDB database is stored in the node query table.

[0131] Figure 6 A block diagram of an electronic device according to an embodiment of the present invention is illustrated.

[0132] like Figure 6 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0133] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0134] The memory 12 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the monitoring method of the MongoDB database cluster instance of each embodiment of the present invention described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0135] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0136] When the electronic device is a stand-alone device, the input device 13 may be a communication network connector, which is used to receive the collected input signals from the first device and the second device.

[0137] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0138] The output device 14 can output various information to the outside, including the determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0139] Of course, to simplify, Figure 6 Only some of the components related to the present invention in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 10 may also include any other appropriate components.

[0140] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0141] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0142] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A monitoring method for a MongoDB database cluster instance, characterized in that: include: Get monitoring data of MongoDB database cluster instances from InfluxDB database; Based on the node query table, obtaining replica sets and non-replica sets from monitoring data of the MongoDB database cluster instance; Based on the first judgment rule, determining whether the replica set is abnormal; If it is determined that the replica set is abnormal, the abnormal state of the replica set is stored in the node query table; If there is a normally operating Mongos node in the non-replica set, then based on the node query table, a shard cluster is obtained from the monitoring data of the MongoDB database cluster instance; Based on the second judgment rule, determine whether the shard cluster is abnormal; If it is determined that an abnormality occurs in the shard cluster, the abnormal state of the shard cluster is stored in the node query table.

2. The monitoring method of a MongoDB database cluster instance according to claim 1, characterized in that: The determining whether the replica set is abnormal based on the first judgment rule includes: If the number of nodes in the replica set that are running normally is greater than or equal to the preset number of nodes, then obtaining the instance role of the node in the replica set; wherein the instance role indicates whether the node is a master node or a slave node; If the instance roles of the nodes of the replica set do not satisfy one master node and two slave nodes or one master node and one slave node, it is determined that an abnormality occurs in the replica set.

3. The monitoring method of the MongoDB database cluster instance according to claim 2, characterized in that: Also includes: If the number of nodes in the replica set that are operating normally is less than the preset number of nodes, it is determined that an abnormality has occurred in the replica set.

4. The monitoring method of a MongoDB database cluster instance according to claim 1, characterized in that: The monitoring data of the MongoDB database cluster instance obtained from the InfluxDB database includes: Obtain the creation time of the monitoring data of the MongoDB database cluster instance; If the difference between the current time and the creation time is greater than a preset difference, monitoring data of the MongoDB database cluster instance is obtained from the InfluxDB database.

5. The monitoring method of a MongoDB database cluster instance according to claim 1, characterized in that: The determining whether the shard cluster is abnormal based on the second judgment rule includes: Obtain the instance role of the shard node in the shard cluster; If the instance role of the shard node in the shard cluster does not satisfy one master node and at least one slave node, it is determined that an abnormality occurs in the shard node and the abnormal state of the shard node is stored in the node query table.

6. The monitoring method of a MongoDB database cluster instance according to claim 5, characterized in that: Also includes: If the instance role of the shard node in the shard cluster satisfies one master node and at least one slave node, determine the instance role of the config server node in the shard cluster; If the instance role of the config server node in the sharded cluster does not satisfy one master node and at least one slave node, it is determined that an abnormality occurs in the config server node and the abnormal state of the config server node is stored in the node query table.

7. The monitoring method of a MongoDB database cluster instance according to claim 1, characterized in that: Also includes: Determine a target Mongos node in the non-replica set; wherein the target Mongos node is used to provide a routing function for the shard cluster; If none of the target Mongos nodes are operating normally, it is determined that an abnormality has occurred in the shard cluster and the abnormal state of the shard cluster is stored in the node query table.

8. The monitoring method of a MongoDB database cluster instance according to claim 1, characterized in that: Before obtaining the replica set and the non-replica set from the monitoring data of the MongoDB database cluster instance based on the node query table, the method further includes: If the node in the monitoring data of the MongoDB database cluster instance has no instance role, sending a preset number of reply requests to the node in the monitoring data of the MongoDB database cluster instance; If the number of reply responses sent by the node in the monitoring data of the MongoDB database cluster instance does not meet the preset number of times, it is determined that the network connection of the node in the monitoring data of the MongoDB database cluster instance is abnormal and the abnormal network connection status of the node in the monitoring data of the MongoDB database cluster instance is stored in the node query table.

9. The monitoring method of a MongoDB database cluster instance according to claim 8, characterized in that: Also includes: If the number of reply responses sent by the node in the monitoring data of the MongoDB database cluster instance meets the preset number, obtaining the key of the MongoDB database; Connecting a Mongo client to the MongoDB database according to the key of the MongoDB database; Obtain the response data of the Mongo client; If the response data of the Mongo client indicates that the connection with the MongoDB database is unsuccessful, it is determined that the service of the MongoDB database is abnormal and the service abnormality state of the MongoDB database is stored in the node query table.

10. A monitoring device for a MongoDB database cluster instance, characterized in that: include: The acquisition module is used to obtain the monitoring data of the MongoDB database cluster instance from the InfluxDB database; A query module, used to obtain replica sets and non-replica sets from the monitoring data of the MongoDB database cluster instance based on a node query table; A first judgment module, used to determine whether the replica set is abnormal based on a first judgment rule; A replica set exception module, configured to store the abnormal state of the replica set in the node query table if it is determined that the replica set is abnormal; A shard cluster acquisition module, configured to acquire a shard cluster from the monitoring data of the MongoDB database cluster instance based on the node query table if there is a normally operating Mongos node in the non-replica set; A second judgment module, used to determine whether the shard cluster is abnormal based on a second judgment rule; The storage module is used to store the abnormal state of the shard cluster in the node query table if it is determined that the shard cluster is abnormal.

Images