Database monitoring method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a database monitoring method and system, electronic equipment and a storage medium. The method comprises the following steps: a server determines a target monitoring index, and determines a monitoring request corresponding to the target monitoring index according to the target monitoring index; the server side sends the monitoring request to a proxy side, and the proxy side determines a client side according to the monitoring request and sends a query request corresponding to the monitoring request to the client side; the client searches target data corresponding to the query request in a database instance according to the query request, and sends the target data to the agent; and the agent side determines a monitoring result corresponding to the target monitoring index in the target data, and sends the monitoring result to the server side. According to the technical scheme provided by the embodiment of the invention, the monitoring index iteration process can be optimized, the monitoring index maintenance cost is reduced, and the dependence of an external platform is reduced or eliminated.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a database monitoring method, system, electronic device and storage medium. Background Art

[0002] With the rapid development of computer and network technology, databases are widely used in various fields, providing query, addition, modification and deletion functions for various applications and services.

[0003] By monitoring the database, you can easily observe the operating status of the database and promptly discover problems with the database operation to ensure the healthy operation of the database, optimize database performance, prevent database failures, and ensure data security.

[0004] In existing database monitoring methods, monitoring scripts are usually encapsulated in a client agent program, or the monitoring scripts are directly deployed on the client, and data corresponding to monitoring indicators are obtained by scheduling the client agent program or the monitoring script.

[0005] However, when there are a large number of clients and monitoring indicators are iterated (that is, monitoring indicators are updated or added), it is necessary to update the agent program or monitoring script of each client, and it is necessary to rely on an external platform for batch updates, which leads to problems such as cumbersome monitoring indicator iteration process, high monitoring indicator maintenance cost, and strong dependence on external platforms. Summary of the invention

[0006] The embodiments of the present invention provide a database monitoring method, system, electronic device and storage medium, which can optimize the monitoring indicator iteration process, reduce the monitoring indicator maintenance cost, and reduce or eliminate the dependence on external platforms.

[0007] According to one aspect of the present invention, there is provided a database monitoring method, comprising:

[0008] The server determines the target monitoring indicator, and determines the monitoring request corresponding to the target monitoring indicator based on the target monitoring indicator;

[0009] The server sends the monitoring request to the agent, and the agent determines the client based on the monitoring request and sends a query request corresponding to the monitoring request to the client;

[0010] According to the query request, the client searches for the target data corresponding to the query request in the database instance and sends the target data to the proxy;

[0011] The agent determines the monitoring results corresponding to the target monitoring indicators in the target data and sends the monitoring results to the server.

[0012] According to another aspect of the present invention, a database monitoring system is provided, the database monitoring system comprising a server, an agent and a client; wherein:

[0013] The server is used to determine the target monitoring indicator and, based on the target monitoring indicator, determine the monitoring request corresponding to the target monitoring indicator;

[0014] The server is also used to send monitoring requests to the agent;

[0015] The agent is used to determine the client according to the monitoring request and send the query request corresponding to the monitoring request to the client;

[0016] The client is used to search for target data corresponding to the query request in the database instance according to the query request, and send the target data to the proxy end;

[0017] The agent is also used to determine the monitoring results corresponding to the target monitoring indicators in the target data and send the monitoring results to the server.

[0018] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the database monitoring method described in any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the database monitoring method described in any embodiment of the present invention when executed.

[0023] The technical solution of the embodiment of the present invention determines the target monitoring indicator through the server, and determines the monitoring request corresponding to the target monitoring indicator based on the target monitoring indicator, so as to send the monitoring request to the agent, and then determines the client according to the monitoring request through the agent, and sends the query request corresponding to the monitoring request to the client, and then searches the database instance for the target data corresponding to the query request through the client according to the query request, and sends the target data to the agent, and finally determines the monitoring result corresponding to the target monitoring indicator in the target data through the agent, and then sends the monitoring result to the server, which solves the problems of cumbersome monitoring indicator iteration process, high monitoring indicator maintenance cost and strong external platform dependence, and can optimize the monitoring indicator iteration process, reduce the monitoring indicator maintenance cost, and reduce or eliminate the dependence on external platforms.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of a database monitoring method provided by Embodiment 1 of the present invention;

[0027] Figure 2 is a flow chart of a database monitoring method provided by Embodiment 2 of the present invention;

[0028] Figure 3a-3c is a specific example flow chart of a database monitoring method provided by Embodiment 3 of the present invention;

[0029] Figure 4 is a schematic diagram of a database monitoring system provided by Embodiment 4 of the present invention;

[0030] Figure 5 It is a structural schematic diagram of an electronic device for implementing the database monitoring method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] Figure 1 This is a flowchart of a database monitoring method provided by the first embodiment of the present invention. This embodiment can be applied to optimizing the monitoring indicator iteration process and reducing the monitoring indicator maintenance cost. The method can be executed by a database monitoring system, which can be implemented by software and / or hardware, and can generally be directly integrated in an electronic device that executes the method. The electronic device can be a terminal device or a server device. The embodiment of the present invention does not limit the type of electronic device that executes the database monitoring method. Specifically, Figure 1 As shown, the database monitoring method may specifically include the following steps:

[0035] S110. The server determines a target monitoring indicator, and determines a monitoring request corresponding to the target monitoring indicator according to the target monitoring indicator.

[0036] The server, that is, the server corresponding to the database, can store and manage the data in the database. It is understandable that one server can correspond to multiple databases.

[0037] The target monitoring indicator may be any indicator for monitoring the database. Exemplarily, the target monitoring indicator may be a performance indicator of the database, such as response time, query speed, etc., or a configuration indicator of the database, such as table space usage, lock status, etc., or other indicators, such as user permissions, abnormal behavior, database status, resource utilization, etc., which are not limited in the embodiments of the present invention.

[0038] The monitoring request may be a request for monitoring a monitoring indicator. Exemplarily, the monitoring request may be an API (Application Programming Interface) request. It is understandable that multiple monitoring indicators may be determined on the server side, and any monitoring indicator may have a corresponding monitoring request.

[0039] In the embodiment of the present invention, the target monitoring indicator is determined by the server, and the monitoring request corresponding to the target monitoring indicator is determined according to the target monitoring indicator, so as to send the monitoring request to the agent. In this way, for the addition, deletion and update of the monitoring indicator, the monitoring indicator can be added, deleted and updated on the server, without the need to maintain the local agent program or script of the client, thereby optimizing the monitoring indicator iteration process.

[0040] S120, the server sends the monitoring request to the agent, the agent determines the client according to the monitoring request, and sends a query request corresponding to the monitoring request to the client.

[0041] The agent is located between the server and the client and can be used to process the monitoring request of the server. It is understandable that one server can correspond to one agent, and one agent can correspond to multiple clients.

[0042] The query request may be a request determined by the agent according to the monitoring request to query the corresponding monitoring indicator in the database.

[0043] Specifically, before the proxy sends the query request corresponding to the monitoring request to the client, the proxy may also determine the query request corresponding to the monitoring request according to the monitoring request.

[0044] In an embodiment of the present invention, after receiving the monitoring request sent by the server, the agent can determine the client corresponding to the monitoring request according to the monitoring request, and determine the query request corresponding to the monitoring request according to the monitoring request, so as to send the query request to the client.

[0045] S130: The client searches for target data corresponding to the query request in the database instance according to the query request, and sends the target data to the proxy.

[0046] The client is the client corresponding to the database. It is understandable that a client can have a database instance corresponding to it.

[0047] The target data may be data found in a database instance and corresponding to the query request. For example, if the query request is to query the usage rate of a table space, the target data may include table space usage rate data. If the query request is to query the capacity of a table in a database, the target data may include the capacity data of the table in the database.

[0048] In the embodiment of the present invention, after the client receives the query request sent by the agent, the client can further search the database instance for target data corresponding to the query request according to the query request to send the target data to the agent.

[0049] S140: The agent determines the monitoring result corresponding to the target monitoring indicator in the target data, and sends the monitoring result to the server.

[0050] The monitoring result may be a result obtained by monitoring the monitoring indicator. Specifically, the monitoring result may include monitoring items and data corresponding to the monitoring items.

[0051] For example, if the target monitoring indicator is table space usage, the monitoring result may include the name of the table space and the usage data corresponding to the table space. If the target monitoring indicator is the lock status of a certain version of the database, the monitoring result may include the version number of the database and the lock status data of the database of that version.

[0052] In an embodiment of the present invention, after the agent receives the target data sent by the client, the agent can further determine the monitoring results corresponding to the target monitoring indicators in the target data, and send the monitoring results to the server, so that the server can perform further processing on the monitoring results (such as analysis, storage, etc.).

[0053] The technical solution of this embodiment determines the target monitoring indicator through the server, and determines the monitoring request corresponding to the target monitoring indicator based on the target monitoring indicator, so as to send the monitoring request to the agent, and then determines the client according to the monitoring request through the agent, and sends the query request corresponding to the monitoring request to the client, and then searches the database instance for the target data corresponding to the query request through the client according to the query request, and sends the target data to the agent, and finally determines the monitoring result corresponding to the target monitoring indicator in the target data through the agent, and then sends the monitoring result to the server, which solves the problems of cumbersome monitoring indicator iteration process, high monitoring indicator maintenance cost and strong external platform dependence, and can optimize the monitoring indicator iteration process, reduce the monitoring indicator maintenance cost, and reduce or eliminate the dependence on external platforms.

[0054] Embodiment 2

[0055] Figure 2 1 is a flowchart of a database monitoring method provided by Embodiment 2 of the present invention. This embodiment further refines the above-mentioned technical solutions and provides a variety of specific optional implementation methods. The technical solution in this embodiment can be combined with the various optional solutions in one or more of the above-mentioned embodiments. Figure 2 As shown, the method may include the following steps:

[0056] S210. The server determines a target monitoring indicator, and determines a monitoring request corresponding to the target monitoring indicator according to the target monitoring indicator; wherein the target monitoring indicator includes a parent monitoring indicator and a child monitoring indicator.

[0057] Among them, the parent monitoring indicator can be a general monitoring indicator with a larger scope. For example, when you want to monitor the lock status of a certain version of the database, the parent monitoring indicator can be the lock status of each database in the database instance where the database of that version is located. For another example, when you want to monitor the usage rate of a table space, the parent monitoring indicator can be the usage rate of all table spaces in the database instance corresponding to the table space.

[0058] Among them, the sub-monitoring indicator can be a specific monitoring indicator corresponding to the parent monitoring indicator. For example, when you want to monitor the lock status of a certain version of the database, the sub-monitoring indicator can be the lock status of the database of that version. For another example, when you want to monitor the table space usage rate, the sub-monitoring indicator can be the usage rate of a specific table space.

[0059] S220. The server determines a database query statement corresponding to the parent monitoring indicator, and sends the database query statement to the agent.

[0060] The database query statement may be a statement for querying a database, for example, an SQL statement.

[0061] Specifically, the database query statement corresponding to the parent monitoring indicator is a statement used to query the data corresponding to the parent monitoring indicator in the database.

[0062] In an embodiment of the present invention, after determining the parent monitoring indicator in the target monitoring indicator, the server may further determine a database query statement corresponding to the parent monitoring indicator, so as to send the database query statement to the agent.

[0063] It is understandable that after determining the database query statement, the server can directly synchronize the database query statement to the agent, or send the database query statement to the agent while sending a monitoring request to the agent, which is not limited in this embodiment of the present invention.

[0064] S230: The server determines the value-selecting rule corresponding to the sub-monitoring indicator, and sends the value-selecting rule to the agent.

[0065] Among them, the value-taking rule may be a rule for obtaining data corresponding to a sub-monitoring indicator.

[0066] In an embodiment of the present invention, after determining the sub-monitoring indicator corresponding to the parent monitoring indicator, the server may further determine the value-taking rule corresponding to the sub-monitoring indicator, and send the value-taking rule corresponding to the sub-monitoring indicator to the agent.

[0067] It is understandable that after determining the value-taking rule corresponding to the sub-monitoring indicator, the server can directly synchronize the value-taking rule to the agent, or send the value-taking rule to the agent while sending a monitoring request to the agent. The embodiment of the present invention does not limit this.

[0068] It should be noted that the embodiment of the present invention does not limit the order of S220 and S230, that is, S220 and S230 can be performed simultaneously.

[0069] Optionally, the server determines the value-taking rules of the sub-monitoring indicator, which may include: the server determines the indicator type of the sub-monitoring indicator; wherein the indicator type includes a first type and a second type; when the indicator type is the first type, the value-taking rules of the sub-monitoring indicator include attribute information of the sub-monitoring indicator; when the indicator type is the second type, the value-taking rules of the sub-monitoring indicator include location information of the first sub-monitoring item corresponding to the sub-monitoring indicator, definition information of the second sub-monitoring item and value information of the second sub-monitoring item.

[0070] The indicator type may be a type of a sub-monitoring indicator. The first type may be a sub-monitoring indicator with a fixed key value. The attribute information of the sub-monitoring indicator may be the key value information of the sub-monitoring indicator.

[0071] The second type may be a sub-monitoring indicator without a fixed key value. The first sub-monitoring item may be a sub-monitoring item in the sub-monitoring indicator. The second sub-monitoring item may be another sub-monitoring item in the sub-monitoring indicator.

[0072] Specifically, the first sub-monitoring item may be an automatically discovered sub-monitoring item, that is, it needs to be automatically discovered in the query result of the parent monitoring indicator. The second sub-monitoring item may be a dynamic key value type sub-monitoring item, that is, the key value of the second sub-monitoring item is dynamic.

[0073] The location information of the first sub-monitoring item may be the location information of the first sub-monitoring item in the query result of the parent monitoring indicator. The definition information of the second sub-monitoring item may be used to determine the second sub-monitoring item. The value information of the second sub-monitoring item may be the information for obtaining the data corresponding to the second sub-monitoring item.

[0074] Exemplarily, assuming that the sub-monitoring indicator is the lock status of a certain version of a database, the key value of the sub-monitoring indicator is the version number of the database, and the indicator type of the sub-monitoring indicator is the first type.

[0075] In another example, suppose the sub-monitoring indicator is a specific tablespace usage rate. However, since the name of the tablespace is set by the user, the name of the tablespace cannot be known before the tablespace is monitored. Therefore, the sub-monitoring indicator does not have a fixed key value. Then the indicator type of the sub-monitoring indicator is the second type. The first sub-monitoring item can be the name of the tablespace, and the second sub-monitoring item can be the usage rate of the tablespace.

[0076] Specifically, the server may determine the indicator type of the sub-monitoring indicator, and determine the value-taking rule of the sub-monitoring indicator according to the indicator type of the sub-monitoring indicator.

[0077] S240, the server sends the monitoring request to the agent, the agent determines the client according to the monitoring request, and sends the database query statement to the client.

[0078] In an optional implementation, the parent monitoring indicator can be of ODBC type, so after receiving the monitoring request corresponding to the parent monitoring indicator, the agent can connect to the client through the ODBC driver. ODBC is a database connection driver that can connect to the corresponding database by configuring the connection string and authentication information.

[0079] S250: The client searches for target data in the database instance according to the database query statement, and sends the target data to the proxy.

[0080] Target data, that is, the query result corresponding to the parent monitoring indicator.

[0081] It should be noted that the target data can be a two-dimensional table, which can include multiple sub-monitoring indicators and data corresponding to the sub-monitoring indicators. That is, it is impossible to perform monitoring trend chart and alarm map analysis based on the query results corresponding to the parent monitoring indicator.

[0082] S260. The agent determines the monitoring result corresponding to the sub-monitoring indicator in the target data according to the value selection rule.

[0083] In an embodiment of the present invention, after the agent receives the target data sent by the client, the agent can further determine the monitoring results corresponding to the sub-monitoring indicators in the target data according to the value selection rules, and send the monitoring results to the server, so that the server can perform further processing on the monitoring results.

[0084] Optionally, the agent determines the monitoring result corresponding to the sub-monitoring indicator in the target data according to the value-selecting rules, which may include: when the indicator type is the first type, the agent determines the data of the sub-monitoring indicator in the target data according to the attribute information of the sub-monitoring indicator; and determines the data of the sub-monitoring indicator as the monitoring result corresponding to the sub-monitoring indicator.

[0085] Specifically, when the indicator type of the sub-monitoring indicator is the first type, the agent can determine the data of the sub-monitoring indicator in the target data according to the attribute information of the sub-monitoring indicator to determine the data of the sub-monitoring indicator as the monitoring result corresponding to the sub-monitoring indicator.

[0086] Specifically, when the sub-monitoring indicator is a sub-monitoring indicator with a fixed key value, kv (key:value) matching can be performed in the target data based on the attribute information of the sub-monitoring indicator (that is, the key value information of the sub-monitoring indicator), that is, the value corresponding to the key value of the sub-monitoring indicator is searched, and the found value is determined as the data of the sub-monitoring indicator.

[0087] For example, assuming that you want to monitor the lock status of a MySQL 5.0.27 version database, the parent monitoring indicator can be the lock status of each database in the database instance where the database of this version is located. The query result corresponding to the parent monitoring indicator, that is, the target data, can be shown in Table 1.

[0088] Table 1 Target data

[0089] Database version number Lock Status MySQL 5.0.27 State 1 MySQL 8.0.34 State 3

[0090] Therefore, the sub-monitoring indicator is the lock status of the MySQL5.0.27 database, and the attribute information of the sub-monitoring indicator is MySQL5.0.27. Then, according to the attribute information of the sub-monitoring indicator MySQL5.0.27, kv matching is performed in the target data to determine that the data of the sub-monitoring indicator is "status 1". In this way, the monitoring result corresponding to the sub-monitoring indicator can be determined to be "status 1".

[0091] Optionally, the agent determines the monitoring result corresponding to the sub-monitoring indicator in the target data according to the value-taking rule, which may include: when the indicator type is the second type, determining the first sub-monitoring item in the target data according to the location information of the first sub-monitoring item; generating the second sub-monitoring item according to the definition information of the first sub-monitoring item and the second sub-monitoring item; determining the data of the second sub-monitoring item in the target data according to the value-taking information of the second sub-monitoring item and the second sub-monitoring item; and determining the data of the first sub-monitoring item and the second sub-monitoring item as the monitoring result corresponding to the sub-monitoring indicator.

[0092] Among them, the number of the first sub-monitoring items can be one or more, and the second sub-monitoring items and the first sub-monitoring items are in one-to-one correspondence.

[0093] Specifically, when the indicator type of the sub-monitoring indicator is the second type, the agent can determine the first sub-monitoring item in the target data based on the location information of the first sub-monitoring item, and generate the second sub-monitoring item based on the definition information of the first sub-monitoring item and the second sub-monitoring item, so as to determine the data of the second sub-monitoring item in the target data based on the value information of the second sub-monitoring item and the second sub-monitoring item, thereby determining the data of the first sub-monitoring item and the second sub-monitoring item as the monitoring results corresponding to the sub-monitoring indicator.

[0094] Specifically, the location information of the first sub-monitoring item may be a column in a two-dimensional table of the target data. That is, according to the location information of the first sub-monitoring item, determining the first sub-monitoring item in the target data may be determining the first sub-monitoring item in a corresponding column of the target data.

[0095] For example, assuming that you want to monitor the table space usage, the parent monitoring indicator is the usage of all table spaces in the database instance corresponding to the table space, then the query result corresponding to the parent monitoring indicator, that is, the target data is shown in Table 2.

[0096] Table 2 Target data

[0097] name Utilization rate (%) type nameA 30 Type 1 nameB 75 Type 1 nameC 66 Type 2

[0098] Therefore, if the sub-monitoring indicator is a specific table space usage rate, the location information of the first sub-monitoring item may include the name column in the target data, and it can be determined that the number of the first sub-monitoring items is three, namely nameA, nameB, and nameC.

[0099] Specifically, the definition information of the second sub-monitoring item may be a definition related to the first sub-monitoring item. After the first sub-monitoring item is determined, the second sub-monitoring item may be generated according to the definition information of the first sub-monitoring item and the second sub-monitoring item.

[0100] Exemplarily, assuming that the definition information of the second sub-monitoring item includes name.rate, then when the first sub-monitoring item is nameA, nameB, and nameC, the second sub-monitoring item may be nameA.rate, nameB.rate, and nameC.rate.

[0101] Specifically, the value information of the second sub-monitoring item can be a JSONPath expression for obtaining the value of the second sub-monitoring item in the target data. Thus, according to the value information of the second sub-monitoring item and the second sub-monitoring item, the data of the second sub-monitoring item can be determined in the target data. It can be understood that the data of the second sub-monitoring item and the second sub-monitoring item can be matched through the value information of the second sub-monitoring item.

[0102] Exemplarily, when the second sub-monitoring item is nameA.rate, nameB.rate and nameC.rate, the data of the second sub-monitoring item determined in the target data may be 30, 75, and 66.

[0103] Specifically, the monitoring result corresponding to the sub-monitoring indicator may be composed of data of the first sub-monitoring item and the second sub-monitoring item corresponding to the first sub-monitoring item.

[0104] For example, the first sub-monitoring item is nameA, nameB, nameC, and the data of the second sub-monitoring item can be 30, 75, and 66. Then the monitoring results corresponding to the sub-monitoring indicators are "the tablespace utilization rate of nameA is 30", "the tablespace utilization rate of nameB is 75", and "the tablespace utilization rate of nameC is 66".

[0105] Optionally, before the agent determines the monitoring results corresponding to the target monitoring indicators in the target data, it may also include: the agent serializes the target data; the agent determines the monitoring results corresponding to the target monitoring indicators in the target data, which may include: the agent determines the monitoring results corresponding to the target monitoring indicators in the serialized target data by string segmentation.

[0106] The serialization process may be to convert the target data into data in the JSON serial format, that is, to serialize the target data into data of the JSON data type. String segmentation, that is, to cut the JSON string, may be implemented by calling the JSON Split method.

[0107] Specifically, after the proxy receives the target data sent by the client, the proxy may further serialize the target data, and determine the monitoring result corresponding to the target monitoring indicator in the serialized target data by string segmentation.

[0108] Optionally, after the target data is serialized, the serialized target data may be further formatted in a standard manner, and in the target data formatted in a standard manner, the monitoring result corresponding to the target monitoring indicator may be determined by string segmentation.

[0109] The technical solution of this embodiment determines the target monitoring indicator through the server, and determines the database query statement corresponding to the parent monitoring indicator and the value-taking rule corresponding to the child monitoring indicator, and then determines the monitoring request corresponding to the target monitoring indicator according to the target monitoring indicator, and then sends the database query statement, the value-taking rule and the monitoring request to the agent, determines the client according to the monitoring request through the agent, and sends the database query statement to the client, searches the target data in the database instance according to the database query statement through the client, and sends the target data to the agent, and finally determines the monitoring result corresponding to the child monitoring indicator in the target data according to the value-taking rule through the agent, which solves the problem of reduced database performance, greater instantaneous pressure and more database resource occupation caused by more database queries at the same time when there are more monitoring indicators, can optimize the value-taking method of monitoring indicators, reduce the number of queries to the database, and reduce the pressure of querying and taking values ​​of monitoring indicators in the database.

[0110] Embodiment 3

[0111] In order to enable those skilled in the art to better understand the database monitoring method provided by this embodiment, a specific example is used below for illustration.

[0112] The database monitoring system is the "eye" of database operation and maintenance management. By monitoring the database, it is easy to observe the operation status of the database, discover database operation problems in time, and deal with them. Currently, there are two common database monitoring methods:

[0113] (1) Agents based on database component media are similar to Zabbix and Prometheus monitoring. In the database local agent, the monitoring script is encapsulated in the agent, and the local monitoring indicators are switched on and off through external control files. The server schedules the client agent, and obtains the corresponding value of the monitoring indicator through the client local connection 127.0.0.1 (loopback address) and port and returns it to the server.

[0114] (2) Based on the local monitoring script of the database, the monitoring scripts of various indicators are deployed locally in the database, and the local crontab scheduling script is used to obtain the corresponding value of the monitoring item and return it to the server through the local client connection 127.0.0.1 and port.

[0115] When there are many clients (such as thousands or tens of thousands) and the client monitoring indicators need to be iterated, these two monitoring methods have the following main problems:

[0116] The monitoring indicator iteration process is cumbersome and highly dependent on external factors. In other words, when monitoring indicators are updated or added, it is necessary to rely on an external automated operation platform to update the local agent or script version of each client. In addition, when there are a large number of clients, the update may fail due to historical network changes, OS authentication expiration, and other issues.

[0117] As the number of client monitoring indicators increases, more and more SQL queries are issued at the same time, which will bring instantaneous QPS (Queries Per Second) pressure to the database and occupy more database resources. For example, when monitoring the size of tablespaces in the database, if you want to monitor the size of 100 tablespaces, there will be 100 SQL queries at the same time. In this case, database monitoring becomes a burden on the database system.

[0118] For example, taking obtaining the usage rate of the EXAMPLE_DATA_TBS_PART01 tablespace in the Oracle database as an example, in the client database local agent or script, it is necessary to define the SQL for obtaining the value of the specified tablespace, and then define the K value that matches the EXAMPLE_DATA_TBS_PART01 tablespace and the server-side monitoring item in the local kv configuration file (such as EXAMPLE_DATA_TBS_PART01_USED_PCT). The server matches the K value in the client's local kv configuration file according to the K value of the monitoring item, and then calls the query statement corresponding to the client's local K value, connects to the database query, and returns the query results to the server.

[0119] The above database monitoring method has low monitoring value efficiency and is not friendly to the database. Every time a tablespace name is added, deleted, or modified, the local script definition must be manually updated and reloaded. The monitoring item iteration cost and query cost are both high. Therefore, in order to solve the above technical problems, an embodiment of the present invention provides a database monitoring method.

[0120] Figure 3a-3c is a specific example flow chart of a database monitoring method provided by Embodiment 3 of the present invention, such as Figure 3a-3c As shown, the method may specifically include:

[0121] Step 1: Define the monitoring indicators required by different types of database users through the server-side monitoring template definition module, which are specifically divided into parent monitoring indicators and child monitoring indicators related to parent monitoring indicators. Among them, the generation and value of child monitoring indicators are all based on the return results after processing of parent monitoring indicators.

[0122] Step 2: remotely connect to the database value acquisition module through the proxy end, receive the monitoring request from the server end, authenticate and connect to the target client business database, and forward the monitoring request to the target client business database for execution.

[0123] Step 3: The agent preprocesses the returned result module to receive the result returned by the target client's business database query and perform serialization processing on it.

[0124] Step 4: Process the serialized results through the proxy sub-monitoring item kv matching module to obtain the sub-monitoring indicators related to the parent monitoring indicator and the values ​​of the sub-monitoring indicators.

[0125] Specifically, the parent monitoring is generally a two-dimensional table query, and the query returns a two-dimensional table of data; the child monitoring related to the parent monitoring is divided into two scenarios: (1) child monitoring of fixed key value type, which can perform kv matching and value extraction based on the return result of the parent monitoring; (2) child monitoring of automatic discovery type, which determines the key value column based on the return result of the parent monitoring, and generates several child monitoring items of dynamic key value type based on the key value column. The generated child monitoring items then perform kv matching and value extraction based on the corresponding key value and the return result of the parent monitoring.

[0126] Step 5: The proxy preprocesses the result return module and returns the result after kv matching to the server.

[0127] The above technical solution can centrally maintain and manage the monitoring indicators of the client database through the server-side monitoring template definition module: for the addition, deletion and update of client monitoring items, it is only necessary to add, delete and update the monitoring item definition of the monitoring template on the server, and there is no need to maintain the agent program or script locally on the client; for monitoring items with undetermined key values, it is only necessary to define the sub-monitoring of the automatic discovery type related to the parent monitoring and the sub-monitoring items of the dynamic key value type in the template on the server. When the result returned by the parent monitoring changes subsequently, no manual maintenance is required, and the sub-monitoring list will be automatically updated; through the agent-side remote connection database value acquisition module, the business database can be remotely connected to obtain the monitoring results, without relying on local agent programs or scripts, eliminating the dependence on external automated operation platforms; through the agent-side preprocessing return result module, the two-dimensional data table is serialized to facilitate the cutting and value acquisition of sub-monitoring items; through the agent-sub-monitoring item kv matching module, the sub-monitoring item can directly consume the query return result of the parent monitoring item, which greatly reduces the pressure on the client business database to query and acquire monitoring items.

[0128] The following is a detailed explanation using the monitoring of the top 10 large capacity tables of the Oracle database as an example. In this example, the data serialization method is JSON serialization, and the database remote connection method is ODBC connection.

[0129] The server-side Oracle monitoring template defines the remote connection (ODBC) type parent monitoring item Top_Table_Info (that is, the parent monitoring indicator) and determines the query SQL corresponding to the monitoring item Top_Table_Info.

[0130] The agent-side remote connection (ODBC) business database A takes values, and the results are shown in Table 3.

[0131] Table 3 Value results

[0132] TABLE_NAME SEGMENT_TYPE SIZE_B SCHEMA01.BIGTABLE01 TABLE PARTITION 4618520559616 SCHEMA01.BIGTABLE02 TABLE 3006377623552 ... ... ...

[0133] The proxy preprocesses the returned results of the Table type and serializes them into the JSON data type.

[0134] The server-side Oracle monitoring template defines the related sub-monitoring Discovery_Top_Table (that is, the first sub-monitoring item) of the parent monitoring item Top_Table_Info automatic discovery type, extracts TABLE_NAME through the JSONPath expression, and formats the extracted TABLE_NAME sequence through the JS script.

[0135] The server-side Oracle monitoring template defines the related sub-monitoring {#TABLE_NAME}_Size (that is, the second sub-monitoring item) of the dynamic key value type of the parent monitoring item Top_Table_Info, and extracts the value of the SIZE_B field corresponding to the specific table name through the JSONPath expression.

[0136] For example, the child monitoring item obtains the value of SIZE_B column 4618520559616 from the parent monitoring item Top_Table_Info through the JSONPath expression.

[0137] Optionally, the server can perform unit conversion and truncation when displaying the value of SCHEMA01.BIGTABLE01_Size, displaying 4618520559616bytes as 4,301.33GB.

[0138] The above technical solution, for the monitoring of the Top 10 large tables in Oracle database, actually only queries the business database once to get the return results of the Top 10 large tables. The generation and value of sub-monitoring are completed on the agent and server. The monitoring method based on remote connection type monitoring and return result data serialization and cutting values ​​is relatively efficient, with low query pressure on the business database. The maintenance definition of monitoring items on the server is completed at the template level, and does not rely on the client local agent program or script. The above example uses an efficient and low-cost method for monitoring the Top 10 large tables in Oracle database.

[0139] The technical solution of this embodiment solves the problems of complicated monitoring item iteration process and strong external dependence, optimizes the monitoring item iteration process, reduces or eliminates the dependence on external automated operations; solves the problem of the impact of the increase in monitoring items on database performance, optimizes the monitoring item value acquisition method, and greatly reduces the number of queries to the business database; for scenarios that require both convenient iteration of monitoring items and low cost of business database value acquisition, such as health status monitoring of each node in the cluster under a distributed database architecture, as nodes are added or deleted, the value needs to be acquired once, and the status information of all nodes can be accurately obtained without the need for additional maintenance of monitoring items for added and deleted nodes.

[0140] Embodiment 4

[0141] Figure 4 is a schematic diagram of a database monitoring system provided by Embodiment 4 of the present invention, such as Figure 4 As shown, the system includes: a server 410, an agent 420 and a client 430, wherein:

[0142] The server 410 is used to determine the target monitoring indicator and determine the monitoring request corresponding to the target monitoring indicator according to the target monitoring indicator;

[0143] The server 410 is also used to send the monitoring request to the agent 420;

[0144] The agent 420 is used to determine the client 430 according to the monitoring request and send a query request corresponding to the monitoring request to the client 430;

[0145] The client 430 is used to search for target data corresponding to the query request in the database instance according to the query request, and send the target data to the proxy 420;

[0146] The agent end 420 is also used to determine the monitoring result corresponding to the target monitoring indicator in the target data, and send the monitoring result to the server end 410.

[0147] The technical solution of this embodiment determines the target monitoring indicator through the server, and determines the monitoring request corresponding to the target monitoring indicator based on the target monitoring indicator, so as to send the monitoring request to the agent, and then determines the client according to the monitoring request through the agent, and sends the query request corresponding to the monitoring request to the client, and then searches the database instance for the target data corresponding to the query request through the client according to the query request, and sends the target data to the agent, and finally determines the monitoring result corresponding to the target monitoring indicator in the target data through the agent, and then sends the monitoring result to the server, which solves the problems of cumbersome monitoring indicator iteration process, high monitoring indicator maintenance cost and strong external platform dependence, and can optimize the monitoring indicator iteration process, reduce the monitoring indicator maintenance cost, and reduce or eliminate the dependence on external platforms.

[0148] Optionally, the target monitoring indicator includes a parent monitoring indicator; the server 410 can also be specifically used to: determine the database query statement corresponding to the parent monitoring indicator, and send the database query statement to the agent 420; the agent 420 can be specifically used to: send the database query statement to the client 430; the client 430 can be specifically used to: search for target data in the database instance according to the database query statement.

[0149] Optionally, the target monitoring indicator also includes a sub-monitoring indicator corresponding to the parent monitoring indicator; the server 410 can also be specifically used to: determine the value-taking rules of the sub-monitoring indicator, and send the value-taking rules to the agent 420; the agent 420 can be specifically used to: determine the monitoring results corresponding to the sub-monitoring indicator in the target data according to the value-taking rules.

[0150] Optionally, the server 410 can be specifically used to: determine the indicator type of the sub-monitoring indicator; wherein the indicator type includes a first type and a second type; when the indicator type is the first type, the value-taking rule of the sub-monitoring indicator includes the attribute information of the sub-monitoring indicator; when the indicator type is the second type, the value-taking rule of the sub-monitoring indicator includes the location information of the first sub-monitoring item corresponding to the sub-monitoring indicator, the definition information of the second sub-monitoring item and the value information of the second sub-monitoring item.

[0151] Optionally, the agent 420 may be specifically used to: when the indicator type is the first type, determine the data of the sub-monitoring indicator in the target data according to the attribute information of the sub-monitoring indicator; and determine the data of the sub-monitoring indicator as the monitoring result corresponding to the sub-monitoring indicator.

[0152] Optionally, the agent 420 can also be specifically used for: when the indicator type is the second type, determining the first sub-monitoring item in the target data according to the location information of the first sub-monitoring item; wherein the number of the first sub-monitoring item is one or more; generating the second sub-monitoring item according to the definition information of the first sub-monitoring item and the second sub-monitoring item; wherein the second sub-monitoring item and the first sub-monitoring item are one-to-one corresponding; determining the data of the second sub-monitoring item in the target data according to the value information of the second sub-monitoring item and the second sub-monitoring item; and determining the data of the first sub-monitoring item and the second sub-monitoring item as the monitoring result corresponding to the sub-monitoring indicator.

[0153] Optionally, the proxy end 420 may also be specifically used to: perform serialization processing on the target data; and determine the monitoring result corresponding to the target monitoring indicator in the serialized target data by character string segmentation.

[0154] The database monitoring system provided in the embodiment of the present invention can execute the database monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0155] Embodiment 5

[0156] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0157] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0158] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0159] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a database monitoring method.

[0160] In some embodiments, the database monitoring method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the database monitoring method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the database monitoring method in any other appropriate manner (e.g., by means of firmware).

[0161] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0162] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0163] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0164] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0165] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0166] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0167] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0168] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A database monitoring method, characterized in that: include: The server determines a target monitoring indicator, and determines a monitoring request corresponding to the target monitoring indicator according to the target monitoring indicator; The server sends the monitoring request to the agent, and the agent determines the client according to the monitoring request and sends a query request corresponding to the monitoring request to the client; The client searches for target data corresponding to the query request in the database instance according to the query request, and sends the target data to the proxy end; The agent determines the monitoring result corresponding to the target monitoring indicator in the target data, and sends the monitoring result to the server.

2. The method according to claim 1, characterized in that The target monitoring indicator includes a parent monitoring indicator; The method further comprises: The server determines a database query statement corresponding to the parent monitoring indicator, and sends the database query statement to the agent; The agent sends a query request corresponding to the monitoring request to the client, including: The proxy sends the database query statement to the client; The client searches for target data corresponding to the query request in the database instance according to the query request, including: The client searches for the target data in the database instance according to the database query statement.

3. The method according to claim 2, characterized in that The target monitoring indicator also includes a sub-monitoring indicator corresponding to the parent monitoring indicator; The method further comprises: The server determines a value-taking rule corresponding to the sub-monitoring indicator, and sends the value-taking rule to the agent; The agent determines the monitoring result corresponding to the target monitoring indicator in the target data, including: The agent determines the monitoring result corresponding to the sub-monitoring indicator in the target data according to the value-taking rule.

4. The method according to claim 3, characterized in that The server determines the value rule of the sub-monitoring indicator, including: The server determines the indicator type of the sub-monitoring indicator; wherein the indicator type includes a first type and a second type; When the indicator type is the first type, the value selection rule of the sub-monitoring indicator includes attribute information of the sub-monitoring indicator; When the indicator type is the second type, the value selection rule of the sub-monitoring indicator includes location information of the first sub-monitoring item corresponding to the sub-monitoring indicator, definition information of the second sub-monitoring item, and value selection information of the second sub-monitoring item.

5. The method according to claim 4, characterized in that The agent determines the monitoring result corresponding to the sub-monitoring indicator in the target data according to the value selection rule, including: When the indicator type is the first type, the agent determines the data of the sub-monitoring indicator in the target data according to the attribute information of the sub-monitoring indicator; The agent determines the data of the sub-monitoring indicator as the monitoring result corresponding to the sub-monitoring indicator.

6. The method according to claim 4, characterized in that The agent determines the monitoring result corresponding to the sub-monitoring indicator in the target data according to the value selection rule, including: When the indicator type is the second type, the agent determines the first sub-monitoring item in the target data according to the location information of the first sub-monitoring item; wherein the number of the first sub-monitoring item is one or more; The agent generates the second sub-monitoring item according to definition information of the first sub-monitoring item and the second sub-monitoring item; wherein the second sub-monitoring item and the first sub-monitoring item are in one-to-one correspondence; The agent determines the data of the second sub-monitoring item in the target data according to the second sub-monitoring item and the value information of the second sub-monitoring item; The agent determines the data of the first sub-monitoring item and the second sub-monitoring item as the monitoring result corresponding to the sub-monitoring indicator.

7. The method according to claim 1, characterized in that: Before the agent determines the monitoring result corresponding to the target monitoring indicator in the target data, the method further includes: The agent performs serialization processing on the target data; The agent determines the monitoring result corresponding to the target monitoring indicator in the target data, including: The agent determines the monitoring result corresponding to the target monitoring indicator by string segmentation in the target data after serialization.

8. A database monitoring system, characterized in that: The database monitoring system includes a server, an agent and a client; wherein: The server is used to determine a target monitoring indicator and, based on the target monitoring indicator, determine a monitoring request corresponding to the target monitoring indicator; The server is further configured to send the monitoring request to the agent; The proxy end is used to determine the client according to the monitoring request, and send a query request corresponding to the monitoring request to the client; The client is used to search the target data corresponding to the query request in the database instance according to the query request, and send the target data to the proxy end; The agent end is also used to determine the monitoring result corresponding to the target monitoring indicator in the target data, and send the monitoring result to the server end.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the database monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the database monitoring method according to any one of claims 1 to 7 when executed.

Images