Performance diagnosis method, device and equipment of distributed database and storage medium

Abstract

The application discloses a performance diagnosis method, device and equipment of a distributed database and a storage medium, and the method comprises the following steps: determining the data table statistical information of each data table in at least two dimensions within at least one statistical period for each storage node and a coordination node in the distributed database; determining the network delay information between the storage nodes and the coordination node, between the storage nodes and between the coordination node and a client within the at least one statistical period; determining the operating system statistical information of each storage node and the coordination node in at least two dimensions within the at least one statistical period; and performing performance diagnosis of the distributed database according to the data table statistical information, the network delay information and the operating system statistical information within each statistical period. The application solves the bottleneck problem of difficult performance problem positioning in the distributed database scenario, and realizes efficient performance diagnosis of the distributed database.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of performance diagnosis of distributed databases, and in particular to a performance diagnosis method, device and equipment of distributed databases and a storage medium. BACKGROUND

[0002] The database is located between the operating system and the application, and the distributed database also involves network communication. When a performance problem of the distributed database is encountered, the database manufacturer is usually contacted by the application layer, and the database manufacturer needs to comprehensively analyze and locate from multiple angles such as network, memory, IO, database pressure, database access history statistics, and the like, and to locate and delimit between the application layer, the operating system and the network provider. Such problems are usually difficult to locate and take a long time.

[0003] Therefore, there is an urgent need for a method for efficiently diagnosing performance problems of distributed databases under a distributed transaction library. SUMMARY

[0004] The present application provides a performance diagnosis method, device and equipment of distributed databases and a storage medium to achieve efficient performance diagnosis of distributed databases.

[0005] In a first aspect, the present application provides a performance diagnosis method of distributed databases, which comprises:

[0006] For each storage node and coordination node in the distributed database, determine the data table statistical information of each data table in at least two dimensions within at least one statistical period;

[0007] Determine the network delay information between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients within at least one statistical period;

[0008] Determine the operating system statistical information of each storage node and coordination node in at least two dimensions within at least one statistical period;

[0009] According to the data table statistical information, the network delay information and the operating system statistical information within each statistical period, the performance of the distributed database is diagnosed.

[0010] In a second aspect, the present application also provides a performance diagnosis device of distributed databases, which comprises:

[0011] A data table statistical information determination module is configured to determine the data table statistical information of each data table in at least two dimensions within at least one statistical period for each storage node and coordination node in the distributed database;

[0012] a network delay information determination module configured to determine network delay information between the storage nodes and the coordination node, between the storage nodes, and between the coordination node and the client, within at least one statistical period;

[0013] an operating system statistical information determination module configured to determine operating system statistical information of the storage nodes and the coordination node in at least two dimensions within at least one statistical period;

[0014] a performance diagnosis module configured to perform performance diagnosis of the distributed database according to the data table statistical information, the network delay information, and the operating system statistical information within each statistical period.

[0015] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the performance diagnosis method of the distributed database according to any of the embodiments of the present application when executing the program.

[0016] In a fourth aspect, a storage medium storing computer executable instructions is provided, and the computer executable instructions are used to perform the performance diagnosis method of the distributed database according to any of the embodiments of the present application when executed by a computer processor.

[0017] The technical solution of the embodiments of the present application determines the data table statistical information, the network delay information, and the operating system statistical information corresponding to each statistical period of the storage nodes and the coordination node, performs performance diagnosis of the distributed database, solves the bottleneck problem of difficult performance problem positioning in the distributed database scenario, and realizes efficient performance diagnosis of the distributed database.

[0018] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is a flowchart of a performance diagnosis method of a distributed database provided by the first embodiment of the present application;

[0021] Figure 2 is a structural schematic diagram of a distributed database provided by the second embodiment of the present application;

[0022] Figure 3 is a flow chart of a performance diagnosis method of a distributed database provided by Embodiment Two of the present application;

[0023] Figure 4 is a structural schematic diagram of a performance diagnosis device of a distributed database provided by Embodiment Three of the present application;

[0024] Figure 5 is a structural schematic diagram of an electronic device provided by Embodiment Four of the present application. DETAILED DESCRIPTION

[0025] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by the personnel in the art without creative labor should belong to the protection scope of the present application.

[0026] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices. In the embodiments of the present application, some software, components, models and the like in the prior art may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the implementation of the technical solutions of the present application, but does not mean that the applicant has or will necessarily use the scheme.

[0027] The acquisition, transmission, storage, use, processing and the like of data in the technical solutions of the present application all comply with the relevant provisions of national laws and regulations.

[0028] Embodiment One

[0029] Figure 1A flowchart of a performance diagnosis method of a distributed database is provided for Embodiment One of the present application. The embodiment can be applied to the performance diagnosis of a distributed database. The method can be executed by a performance diagnosis device of a distributed database. The performance diagnosis device of a distributed database can be implemented in the form of hardware and / or software. The performance diagnosis device of a distributed database can be configured in a server and used in cooperation with at least two virtual machines.

[0030] As shown in Figure 1 , the method comprises the following steps.

[0031] In S110, the data table statistical information of each data table in at least two dimensions in at least one statistical period is determined for each storage node and coordination node in the distributed database.

[0032] In the embodiment, the storage node is a node in the distributed database responsible for actually storing data and processing operations directly related to data. The coordination node is a node in the distributed database responsible for coordinating the cooperation of each node. In the distributed database, the storage node and the coordination node work together to support the high availability, high scalability and efficient data processing capability of the distributed system.

[0033] In the embodiment, the statistical period is a preset time period. The data table statistical information represents the runtime information in the distributed database, including data state statistics and operation behavior statistics, and is used to record the active / dead record number and information such as addition, deletion, modification and query of each data table.

[0034] Referring to Figure 2 , a structural diagram of a distributed database is shown. The client, coordination node and storage node of the distributed database usually constitute a three-layer structure of “client layer-coordination layer-storage layer”. Each layer has clear responsibilities and interacts through a network protocol (such as TCP / IP or HTTP protocol) to realize network data interaction communication. Among them, the client layer is the entrance for users or application programs to interact with the database, responsible for initiating requests (queries, transactions, etc.) and receiving results; the coordination layer is the “middle layer” connecting the client and the storage node, responsible for global scheduling and cooperation; the storage layer is the actual storage and processing layer of data, composed of multiple storage nodes.

[0035] Taking a cross-shard query request as an example, the data interaction steps are as follows:

[0036] Firstly, the client sends a request (such as "query the orders of users with IDs 100 and 200") to the coordination node without specifying the data storage location. Further, the coordination node parses the request and sends a subquery request to the storage node according to the parsed data. Further, the storage node queries the local data according to the subquery request and returns the query result to the coordination node. Then, the coordination node merges the query results, sorts or filters them according to the client's requirements, and returns the final result to the client, so that the client can obtain the query result.

[0037] It should be noted that each storage node and coordination node in the distributed database contains multiple data tables. For each storage node, the data table statistical information of each data table of the storage node in at least two dimensions in at least one statistical period is determined. For each coordination node, the data table statistical information of each data table of the storage node in at least two dimensions in at least one statistical period is determined.

[0038] In this embodiment, by determining these data table statistical information, the performance bottleneck of the distributed database can be quickly located.

[0039] As an optional but non-limiting implementation manner, the dimensions of the data table statistical information include: the number of active records, the number of dead records, the number of inserted rows, the number of deleted rows, the number of updated rows, the number of index scan times, the number of index scan captured rows, the number of sequential scan times, and the number of sequential scan captured rows.

[0040] Among them, the number of active records is the total number of records in the data table in the "effective state", reflecting the actual data amount of the data table; the number of dead records is the total number of records in the data table in the "invalid state"; the number of inserted rows is the total number of newly added records in the data table, reflecting the data write frequency; the number of deleted rows is the total number of deleted records in the data table; the number of updated rows is the total number of modified records in the data table; the number of index scan times is the number of queries through index positioning records; the number of index scan captured times is the total number of records actually read through index scan; the number of sequential scan times is the number of queries for complete scanning of the data table; and the number of sequential scan captured times is the total number of records actually read in the complete scanning process of the data table.

[0041] The above data table statistical information comprehensively reflects the running state of the data table from three aspects of data state (active / dead records), operation behavior (addition / deletion / modification), and query efficiency (index / sequential scan). By determining these data table statistical information, the performance bottleneck of the distributed database can be quickly located.

[0042] As an optional but non-limiting implementation manner, determining the data table statistical information of each data table in at least two dimensions in at least one statistical period includes but is not limited to the following steps:

[0043] determining current data table statistical information of the target data table in at least two dimensions at a current time point of a current statistical period, and historical data table statistical information of the target data table in at least two dimensions at a historical time point of a previous statistical period;

[0044] determining data table statistical information of the target data table in at least two dimensions in the current statistical period according to a difference between the current data table statistical information and the historical data table statistical information.

[0045] In the embodiment, the target data table is a data table currently being processed, and the target data table can be any data table in the coordination node or the storage node. The current data table statistical information is statistical information of the target data table acquired at a current time point of a current statistical period, and the historical data table statistical information is statistical information of the target data table acquired at a historical time point of a previous statistical period. The statistical period can be one hour, one day, one week, etc.

[0046] Specifically, the data table statistical information of the target data table in at least two dimensions in the current statistical period is determined according to a difference between the current data table statistical information and the historical data table statistical information.

[0047] Taking the number of inserted rows in the data table statistical information as an example, the number of inserted rows saved by the target data table at 0 o'clock on February 2 and the number of inserted rows saved by the target data table at 0 o'clock on February 1 are acquired, and a difference between the number of inserted rows saved by the target data table at 0 o'clock on February 2 and the number of inserted rows saved by the target data table at 0 o'clock on February 1 is taken as the number of inserted rows generated by the target data table on February 1. In this way, the data table statistical information of each dimension of the target data table in each statistical period can be acquired.

[0048] It should be noted that each coordination node and storage node contains multiple data tables, and for each data table, the operation of determining the current data table statistical information of the target data table in at least two dimensions at a current time point of a current statistical period, and the historical data table statistical information of the target data table in at least two dimensions at a historical time point of a previous statistical period, and determining the data table statistical information of the target data table in at least two dimensions in the current statistical period according to a difference between the current data table statistical information and the historical data table statistical information is repeated, so that the data table statistical information of each dimension of each data table in each statistical period can be determined.

[0049] S120, determining network delay information between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in at least one statistical period.

[0050] In the embodiment, the network delay information refers to a time required for data to be transmitted from one node to another node.

[0051] It should be noted that in the distributed database of the embodiment, the data interaction between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients is frequent and the network traffic is large, and therefore, network delays can exist between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients.

[0052] Therefore, by determining the network delay information between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients of the distributed database in each statistical period, the performance bottleneck of the distributed database can be quickly located.

[0053] As an optional but non-limiting implementation manner, the determining the network delay information between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in at least one statistical period comprises:

[0054] The network diagnostic test is performed on the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in the current statistical period by a script deployed in the operating system of the distributed database to obtain the network delay information in the current statistical period.

[0055] In the embodiment, the network diagnostic test is a diagnostic test on network faults or network performance to obtain network connectivity, delay, packet loss, and the like. The network diagnostic test comprises a basic connectivity test, a bandwidth and throughput test, and a network performance test, and the like.

[0056] In the embodiment, a script can be deployed in the operating system of the distributed database, for example, when there is data interaction, a ping delay test is performed on the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in the distributed database by deploying a ping command script to obtain the network delay information in the current statistical period. The smaller the network delay information value is, the faster the communication response between the two nodes is, and the larger the network delay information value is, the slower the communication response between the two nodes is. If the network delay information is timed out, it indicates that the network communication between the two nodes is interrupted.

[0057] In the embodiment, the network diagnostic test is automatically performed by the script deployed in the operating system of the distributed database to obtain the network delay information between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in the current statistical period in real time, and the performance diagnosis efficiency of the distributed database is improved.

[0058] As an optional but non-limiting implementation manner, the network diagnostic test performed on the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients comprises but is not limited to the following steps:

[0059] According to the node network structure of the distributed database, the network diagnosis test is performed on the storage node and the coordination node connected thereto;

[0060] In the node network structure, if there is interaction between the storage node and the coordination node, the storage node is connected to the coordination node;

[0061] If it is determined that the joint query condition is met, the network diagnosis test is performed on each storage node that has interaction during the joint query;

[0062] The network diagnosis test is performed between the coordination node and the client.

[0063] In this embodiment, the network diagnosis test is divided into three independent processes, which are the process of performing the network diagnosis test on the storage node and the coordination node, the process of performing the network diagnosis test between each storage node, and the process of performing the network diagnosis test between the coordination node and the client. It should be noted that the execution order of the three independent processes of the network diagnosis test is not limited in this embodiment.

[0064] In this embodiment, in the node network structure of the distributed database, if there is data interaction between the storage node and the coordination node, the storage node is connected to the coordination node. The network diagnosis test is performed on the storage node and the coordination node connected thereto, and the corresponding network delay information is obtained.

[0065] It should be noted that the joint query condition is the existence of broadcast or redistribution of the data table. In the data query process, sometimes the data of the query object is distributed in multiple different storage nodes, at which time the data table of the query object in different storage nodes needs to be broadcasted respectively to realize the data aggregation and query of the query object. Among them, the broadcast is to copy and send the data in the data table in one storage node to other storage nodes, and the redistribution is that the storage node receives the broadcasted data table from other storage nodes, and then the data table is redistributed in each storage node. Through the broadcast and redistribution of the data table, each data table corresponding to the same query object and distributed in different storage nodes is concentrated in the same storage node for processing, which avoids frequent remote data interaction to ensure that the relevant data information can be matched and processed in the same storage node.

[0066] In this embodiment, in the distributed database, when multiple distributed tables are jointly queried, the broadcast or redistribution of the data table is involved, and the storage nodes that have interaction need to be determined. For example, if it is received in the distributed database system that "broadcast table A to node 2 and node 3", there is interaction between node 1 where table A is located and node 2 and node 3.

[0067] Interacting storage nodes generate a large amount of data transmission. The network performance between these nodes directly impacts query efficiency and may even cause query timeouts. Therefore, network latency between storage nodes is a significant factor affecting distributed database performance during federated queries. If the federated query conditions are met, network diagnostics are performed on each storage node that interacts during the query to obtain the corresponding network latency information.

[0068] In this embodiment, the network delay between the client and the coordination node may also cause the query timeout. Therefore, a network diagnostic test is performed between the coordination node and the client to obtain corresponding network delay information.

[0069] In this embodiment, by performing network diagnostic tests on storage nodes and coordination nodes, network diagnostic tests between storage nodes, and network diagnostic tests between coordination nodes and clients, network delay information during the interaction of each node is obtained, providing a basis for subsequent rapid positioning of performance bottlenecks in the distributed database.

[0070] S130: Determine operating system statistical information of at least two dimensions of each storage node and coordination node within at least one statistical period.

[0071] In this embodiment, during data exchange, each storage node and coordination node experiences operating system usage, such as resource consumption and resource occupancy. This process significantly impacts the performance of the distributed database. Therefore, it is necessary to determine operating system statistics for each storage node and coordination node during each statistical period to provide a basis for quickly locating performance bottlenecks in the distributed database.

[0072] As an optional but non-limiting implementation, the dimensions of the operating system statistical information include: system performance statistical information, disk performance statistical information, network connection status statistical information, grid interface statistical information, and network card configuration statistical information;

[0073] The system performance statistical information includes at least one of central processing unit occupation, memory occupation, swap space activity, block device read-write performance, and system kernel activity; the disk performance statistical information includes at least one of user process occupation time, kernel space process occupation time, central processing unit waiting time, and central processing unit idle time; the network connection state statistical information includes at least one of protocol connection number, protocol connection state distribution, protocol connection duration, protocol reconnection number, protocol timeout number, and protocol connection rejection number; the grid interface statistical information includes at least one of total byte reception number, total byte transmission number, error packet reception number, discarded packet number, data packet transmission failure number, and data packet transmission conflict number; and the network card configuration statistical information includes at least one of interface state, error packet reception number, discarded packet number, data packet transmission failure number, and data transmission conflict number.

[0074] In the embodiment, the system performance statistical information is statistical data of system overall running state and resource utilization; the disk performance statistical information is statistical data of disk read-write efficiency and load state; the network connection state statistical information is statistical data of network connection state, number, and type; the grid interface statistical information is statistical data of network interface transmission data volume, error rate, and other underlying transmission indexes; and the network card configuration statistical information is statistical data of network interface card static configuration parameter and dynamic configuration state.

[0075] Specifically, the system performance statistical information of each storage node and coordination node can be obtained by a vmstat (Virtual Memory Statistics) tool, including central processing unit occupation, memory occupation, swap space activity, block device read-write performance, and system kernel activity.

[0076] Specifically, the disk performance statistical information of each storage node and coordination node can be obtained by an iostat (Input / Output Statistics) tool, including user process occupation time, kernel space process occupation time, central processing unit waiting time, and central processing unit idle time.

[0077] Specifically, the network connection state statistical information of each storage node and coordination node can be obtained by a netstat (Network Statistics) tool, including protocol connection number, protocol connection state distribution, protocol connection duration, protocol reconnection number, protocol timeout number, and protocol connection rejection number.

[0078] Specifically, the grid interface statistics of each storage node and the coordination node can be obtained by using an ethtool (Ethernet Tool) tool, including the total number of received bytes, the total number of sent bytes, the number of received error packets, the number of discarded packets, the number of failed packet transmissions, and the number of packet transmission conflicts.

[0079] Specifically, the network card configuration statistics of each storage node and the coordination node can be obtained by using an ifconfig (interface configuration) tool, including the interface state, the number of received error packets, the number of discarded packets, the number of failed packet transmissions, and the number of data transmission conflicts.

[0080] Further, according to the obtained system performance statistics, disk performance statistics, network connection state statistics, grid interface statistics, and network card configuration statistics, it can be determined whether there is a problem such as a receiving and transmitting buffer overflow or a hardware error caused by a network abnormal time point.

[0081] In this embodiment, by determining the operating system statistics of each storage node and the coordination node in each statistical period, the performance bottleneck of the distributed database can be quickly located.

[0082] S140, according to the data table statistics, network delay information, and operating system statistics in each statistical period, the performance of the distributed database is diagnosed.

[0083] In this embodiment, if the network performance is abnormal, the performance bottleneck of the distributed database can be quickly located according to the data table statistics, network delay information, and operating system statistics in each statistical period, and the performance diagnosis efficiency of the distributed database is improved.

[0084] As an optional but non-limiting implementation manner, according to the data table statistics, network delay information, and operating system statistics in each statistical period, the performance of the distributed database is diagnosed, including:

[0085] If it is determined that the number of times that the target statistical information is greater than or equal to the target threshold value in the continuous statistical period exceeds the number threshold value, a performance alarm is triggered, and the target statistical information is any one of the data table statistics, the network delay information, and the operating system statistics.

[0086] Specifically, taking the network delay information as an example, and taking the network delay information time between nodes as 10 milliseconds as an example, the performance alarm triggering condition can be configured as that the number of times that the network delay information is greater than or equal to 100 milliseconds in the statistical period continuously exceeds 5 times, that is, the network delay performance alarm can be triggered, and the performance bottleneck can be quickly located according to the network delay performance alarm.

[0087] It should be noted that the memory consumption, central processing unit consumption, database execution time, etc. can configure performance alarm triggering conditions according to statistical information, so as to facilitate timely discovery of problems and rapid positioning of closed loop.

[0088] As an optional but non-limiting implementation, according to the data table statistical information, network delay information and operating system statistical information in each statistical period, the performance diagnosis of the distributed database further includes but is not limited to the following steps:

[0089] Determine the target threshold and target weight of the target statistical information; the target statistical information is at least two of the data table statistical information, the network delay information and the operating system statistical information;

[0090] Determine the difference between the target statistical information and the target threshold, and determine the first statistical value according to the ratio of the difference to the target threshold;

[0091] According to the target weight, the first statistical value is weighted and summed to determine the second statistical value;

[0092] If it is determined that the number of times that the second statistical value is greater than or equal to the statistical information threshold in the continuous statistical period exceeds the number threshold, the performance alarm is triggered.

[0093] Further, the performance alarm triggering condition can also be determined according to multiple target statistical information. In order to avoid the influence of the large difference in numerical range caused by the different units of the data table statistical information, the network delay information and the operating system statistical information, the ratio of the difference between the target statistical information and the target threshold to the target threshold is calculated, and the data table statistical information, the network delay information and the operating system statistical information are unified to a value between 0 and 1, so as to set the threshold value of the performance alarm triggering condition and ensure the stability of the performance alarm triggering.

[0094] Specifically, the difference between the target statistical information and the target threshold is calculated, and the first statistical value corresponding to each target statistical information is determined according to the ratio of the difference to the target threshold. Further, the corresponding target weight of each target statistical information can be set according to the importance of each statistical information in the distributed system, and the second statistical value is obtained by weighted sum of each first statistical value according to the corresponding target weight of each target statistical information. If the number of times that the second statistical value is greater than or equal to the statistical information threshold in the continuous statistical period exceeds the number threshold, the performance alarm is triggered.

[0095] The technical scheme of the embodiment of the present application determines the data table statistical information, network delay information and operating system statistical information in each statistical period corresponding to the storage node, the coordination node and the client, performs performance diagnosis of the distributed database, solves the bottleneck problem of difficult performance problem positioning in the distributed database scenario, and realizes efficient performance diagnosis of the distributed database.

[0096] Embodiment two

[0097] Figure 3 A flowchart of a performance diagnosis method of a distributed database provided by the second embodiment of the present application is shown in the figure. The second embodiment of the present application is further specific on the basis of the above-mentioned embodiment.

[0098] As Figure 3 shown, the method comprises the following steps.

[0099] S210, for each storage node and coordination node in the distributed database, determining data table statistical information of each data table in at least two dimensions in at least one statistical period.

[0100] S220, determining network delay information between the storage node and the coordination node, between the storage nodes and between the coordination node and the client in at least one statistical period.

[0101] S230, determining operating system statistical information of each storage node and coordination node in at least two dimensions in at least one statistical period.

[0102] S240, drawing a database pressure diagnosis graph according to the data table statistical information in each statistical period.

[0103] In this embodiment, the database pressure diagnosis graph is used to represent the pressure distribution of the data table statistical information of each coordination node and storage node.

[0104] Specifically, the collected data table statistical information of each coordination node and storage node in each statistical period is drawn into the corresponding database pressure diagnosis graph, wherein the database pressure diagnosis graph can be presented in the form of a line graph or a curve.

[0105] In actual application, once a feedback that the database execution performance is slow in a certain statistical period is received, the historical statistical information of the database pressure can be determined according to the database pressure diagnosis graph, and then it can be directly confirmed whether the database pressure is caused by the surge, such as the surge of the number of active records or the surge of the number of times of adding, deleting, modifying and inquiring.

[0106] S250, drawing a network diagnosis graph according to the network delay information in each statistical period.

[0107] In this embodiment, the network diagnosis graph is used to represent the network delay distribution or transmission delay distribution between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients.

[0108] Specifically, the network delay information between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients collected in each statistical period is drawn into a corresponding network diagnosis graph, which can be presented in the form of a line graph or a curve.

[0109] It should be noted that the network delay information exists between two nodes, and therefore, the network diagnosis graph is drawn according to the network structure layout of the distributed database. According to the network delay information on the connection line between two nodes in the network diagnosis graph, whether there is a network performance anomaly between the two nodes can be determined. If there is a network performance anomaly, the location of the network performance anomaly can be determined according to the sudden increase of the network delay curve in the network diagnosis graph.

[0110] S260, draw an operating system pressure diagnosis graph according to the operating system statistical information in each statistical period.

[0111] In this embodiment, the operating system pressure diagnosis graph is used to represent the pressure distribution of the operating system statistical information of each coordination node and storage node.

[0112] Specifically, the operating system statistical information of each coordination node and storage node collected in each statistical period is drawn into a corresponding operating system pressure diagnosis graph, which can be presented in the form of a line graph or a curve.

[0113] In actual application, once a feedback that the operating system execution performance is slow in a statistical period is received, the historical statistical information of the operating system pressure can be determined according to the operating system pressure diagnosis graph, and whether the operating system pressure is caused by the surge of the operating system pressure can be confirmed.

[0114] It should be noted that the operating system pressure diagnosis graph is drawn according to the network structure layout of the distributed database, and in actual application, in response to the operation of triggering a node in the network structure layout of the distributed database, the operating system pressure diagnosis graph of the corresponding node is obtained. The operating system pressure diagnosis graph displays the pressure distribution of the IO pressure, memory consumption, CPU consumption, swap usage, socket receiving and sending buffer occupancy, and network retransmission number on the node in a graphical form. Once a feedback that the distributed database execution performance is slow in a statistical period is received, the location of the performance anomaly can be determined according to the sudden increase of the curve in the operating system pressure diagnosis graph.

[0115] S270, according to the database pressure diagnosis graph, the network diagnosis graph and the operating system pressure diagnosis graph, performance diagnosis of the distributed database is performed.

[0116] In the embodiment, when network performance is abnormal, according to the database pressure diagnosis graph, the network diagnosis graph and the operating system pressure diagnosis graph, the performance bottleneck of the distributed database can be quickly located.

[0117] As an optional but not limited implementation, according to the database pressure diagnosis graph, the network diagnosis graph and the operating system pressure diagnosis graph, performance diagnosis of the distributed database includes but is not limited to the following steps:

[0118] In the node network structure of the distributed database, the database pressure diagnosis graph corresponding to each storage node and coordination node is respectively embedded into each storage node and coordination node, and the operating system pressure diagnosis graph corresponding to each storage node and coordination node is respectively embedded into each storage node and coordination node.

[0119] The network diagnosis graph is embedded into the connection line between the nodes matched with the network diagnosis graph, and a performance diagnosis graph is obtained.

[0120] According to the performance diagnosis graph, performance diagnosis of the distributed database is performed.

[0121] In the embodiment, the performance diagnosis graph is used to represent the pressure distribution of each node and between nodes in each statistical period.

[0122] Specifically, the database pressure diagnosis graph is embedded into each storage node and coordination node, the operating system pressure diagnosis graph is embedded into each storage node and coordination node, and the network diagnosis graph is embedded into the connection line between nodes, and a performance diagnosis graph is obtained.

[0123] In the embodiment, the data table statistical information, the network delay information and the operating system statistical information are all displayed in a graphical form, and the sudden increase of database pressure, network pressure and operating system pressure is represented as a curve or burr with a steep increase in the performance diagnosis graph. Further, in combination with automatic performance alarm, the performance problem positioning efficiency of the distributed database can be improved.

[0124] In actual application, if performance problem feedback of the distributed database is received, performance alarm of the performance problem time point is determined by using the performance diagnosis method of the distributed database. According to the performance alarm type and the corresponding target diagnosis graph, the position of the performance abnormality is determined.

[0125] At a time point, there is a performance alarm, and a corresponding statistical period is determined according to the time point. In the statistical period, if there is a network delay performance alarm, a position of a performance anomaly is determined according to a network diagnosis graph; if it is an operating system stress performance alarm, such as a memory or central processing unit resource exhaustion performance alarm, the position of the performance anomaly is determined according to an operating system stress diagnosis graph; if it is a database stress performance alarm, the position of the performance anomaly is determined according to the operating system stress diagnosis graph.

[0126] Through the above method, comprehensive diagnosis of the distributed database performance problem is formed from the aspects of the database, the network delay and the operating system, and detailed and reliable historical statistical information is provided for the positioning of the distributed database performance problem. The historical statistical information is displayed in a graphical manner, the performance alarm function is combined, the performance bottleneck of the distributed database is quickly positioned, and the performance diagnosis efficiency of the distributed database is improved.

[0127] The technical scheme of the embodiment of the application determines the data table statistical information, the network delay information and the operating system statistical information in each statistical period corresponding to the storage nodes, the coordination nodes and the clients, performs the performance diagnosis of the distributed database, solves the bottleneck problem of the positioning of the performance problem in the distributed database scenario, and realizes the efficient performance diagnosis of the distributed database.

[0128] Embodiment three

[0129] Figure 4 A structure schematic diagram of a performance diagnosis device of a distributed database provided by the embodiment three is shown in the figure. Figure 4 As shown in the figure, the device comprises:

[0130] A data table statistical information determination module 310 is configured to determine at least two dimensions of data table statistical information of each data table in at least one statistical period for each storage node and coordination node in the distributed database.

[0131] A network delay information determination module 320 is configured to determine network delay information between the storage nodes and the coordination nodes, between the storage nodes and between the coordination nodes and the clients in at least one statistical period.

[0132] An operating system statistical information determination module 330 is configured to determine at least two dimensions of operating system statistical information of each storage node and coordination node in at least one statistical period.

[0133] A performance diagnosis module 340 is configured to perform the performance diagnosis of the distributed database according to the data table statistical information, the network delay information and the operating system statistical information in each statistical period.

[0134] On the basis of the above embodiment, optionally, the device further comprises:

[0135] determining the data table statistics information of the target data table in at least two dimensions in at least one statistical period, comprising:

[0136] determining current data table statistics information of the target data table in at least two dimensions at a current time in a current statistical period, and historical data table statistics information of the target data table in at least two dimensions at a historical time in a previous statistical period;

[0137] determining the data table statistics information of the target data table in at least two dimensions in the current statistical period according to a difference between the current data table statistics information and the historical data table statistics information.

[0138] On the basis of the above-mentioned embodiments, optionally, the apparatus further comprises:

[0139] The dimensions of the data table statistics information include: the number of active records, the number of dead records, the number of inserted rows, the number of deleted rows, the number of updated rows, the number of index scan times, the number of index scan captured rows, the number of sequential scan times, and the number of sequential scan captured rows.

[0140] On the basis of the above-mentioned embodiments, optionally, the apparatus further comprises:

[0141] determining the network delay information between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in at least one statistical period, comprising:

[0142] performing network diagnosis tests between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in the current statistical period through scripts deployed in the operating system of the distributed database to obtain the network delay information in the current statistical period.

[0143] On the basis of the above-mentioned embodiments, optionally, the apparatus further comprises:

[0144] performing network diagnosis tests between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients, comprising:

[0145] performing network diagnosis tests on the storage nodes and the coordination nodes connected thereto according to the node network structure of the distributed database;

[0146] In the node network structure, if there is interaction between the storage nodes and the coordination nodes, the storage nodes and the coordination nodes are connected;

[0147] If it is determined that the joint query condition is met, performing network diagnosis tests between the storage nodes that interact during the joint query;

[0148] performing network diagnosis tests between the coordination nodes and the clients.

[0149] In the above embodiment, the device further comprises:

[0150] According to the data table statistical information, network delay information and operating system statistical information in each statistical period, the performance of the distributed database is diagnosed, including:

[0151] According to the data table statistical information in each statistical period, a database stress diagnosis graph is drawn;

[0152] According to the network delay information in each statistical period, a network diagnosis graph is drawn;

[0153] According to the operating system statistical information in each statistical period, an operating system stress diagnosis graph is drawn;

[0154] According to the database stress diagnosis graph, the network diagnosis graph and the operating system stress diagnosis graph, the performance of the distributed database is diagnosed.

[0155] In the above embodiment, the device further comprises:

[0156] According to the database stress diagnosis graph, the network diagnosis graph and the operating system stress diagnosis graph, the performance of the distributed database is diagnosed, including:

[0157] In the node network structure of the distributed database, the database stress diagnosis graph corresponding to each storage node and coordination node is respectively embedded into each storage node and coordination node, and the operating system stress diagnosis graph corresponding to each storage node and coordination node is respectively embedded into each storage node and coordination node;

[0158] The network diagnosis graph is embedded into the connection line between the nodes matched with the network diagnosis graph, to obtain a performance diagnosis graph;

[0159] According to the performance diagnosis graph, the performance of the distributed database is diagnosed.

[0160] The technical scheme of the embodiment of the application determines the data table statistical information, network delay information and operating system statistical information in each statistical period corresponding to the storage node, the coordination node and the client, diagnoses the performance of the distributed database, solves the bottleneck problem of difficult performance problem positioning in the distributed database scenario, and realizes efficient performance diagnosis of the distributed database.

[0161] The distributed database performance diagnosis device provided by the embodiment of the application can execute the distributed database performance diagnosis method provided by any embodiment of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0162] Embodiment four

[0163] Figure 5 A structural diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0164] As shown in Figure 5 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., connected in communication with the at least one processor 11, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0165] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0166] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the performance diagnosis method of the distributed database.

[0167] In some embodiments, the performance diagnosis method of a distributed database can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the performance diagnosis method of a distributed database as described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the performance diagnosis method of a distributed database by way of other any suitable means, e.g., by way of firmware.

[0168] The various embodiments of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0169] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed, can implement the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0170] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0171] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0172] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0173] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0174] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0175] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A performance diagnosis method of a distributed database, characterized by, The method comprises the following steps: determining data table statistical information of each data table in at least two dimensions in at least one statistical period for each storage node and coordination node in a distributed database; determining network delay information between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in at least one statistical period; determining operating system statistical information of each storage node and coordination node in at least two dimensions in at least one statistical period; performing performance diagnosis of the distributed database according to the data table statistical information, the network delay information, and the operating system statistical information in each statistical period; determining data table statistical information of each data table in at least two dimensions in at least one statistical period, comprising: determining current data table statistical information of a target data table in at least two dimensions at a current time in a current statistical period, and historical data table statistical information of the target data table in at least two dimensions at a historical time in a previous statistical period; determining data table statistical information of the target data table in at least two dimensions in the current statistical period according to a difference between the current data table statistical information and the historical data table statistical information.

2. The method of claim 1, wherein, The dimensions of the data table statistical information include: the number of active records, the number of dead records, the number of inserted rows, the number of deleted rows, the number of updated rows, the number of index scan times, the number of index scan captured rows, the number of sequential scan times, and the number of sequential scan captured rows.

3. The method of claim 1, wherein, determining network delay information between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in at least one statistical period, comprising: performing network diagnosis testing between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients in the current statistical period by a script deployed in an operating system of the distributed database to obtain network delay information in the current statistical period.

4. The method of claim 3, wherein, performing network diagnosis testing between the storage nodes and the coordination nodes, between the storage nodes, and between the coordination nodes and the clients, comprising: performing network diagnosis testing on the storage nodes and the coordination nodes connected thereto according to a node network structure of the distributed database; wherein, in the node network structure, if there is interaction between the storage nodes and the coordination nodes, the storage nodes and the coordination nodes are connected; if it is determined that the joint query condition is met, performing network diagnosis testing between the storage nodes that interact during the joint query; performing network diagnosis testing between the coordination nodes and the clients.

5. The method of claim 1, wherein, performing performance diagnosis of the distributed database according to the data table statistical information, the network delay information, and the operating system statistical information in each statistical period, comprising: drawing a database pressure diagnosis graph according to the data table statistical information in each statistical period; drawing a network diagnosis graph according to the network delay information in each statistical period; drawing an operating system pressure diagnosis graph according to the operating system statistical information in each statistical period; performing performance diagnosis of the distributed database according to the database pressure diagnosis graph, the network diagnosis graph, and the operating system pressure diagnosis graph.

6. The method of claim 5, wherein, performing performance diagnosis of the distributed database according to the database pressure diagnosis graph, the network diagnosis graph, and the operating system pressure diagnosis graph, comprising: In the node network structure of the distributed database, the database stress diagnosis graph corresponding to each storage node and coordination node is embedded into each storage node and coordination node respectively, and the operating system stress diagnosis graph corresponding to each storage node and coordination node is embedded into each storage node and coordination node respectively; The network diagnosis graph is embedded into the connection line between the nodes matched with the network diagnosis graph, to obtain a performance diagnosis graph; According to the performance diagnosis graph, the performance of the distributed database is diagnosed.

7. A performance diagnosis apparatus of a distributed database, characterized by comprising: The method comprises: A data table statistical information determination module is configured to determine at least two dimensions of data table statistical information of each data table in at least one statistical period for each storage node and coordination node in the distributed database; A network delay information determination module is configured to determine network delay information between storage nodes and coordination nodes, between storage nodes, and between coordination nodes and clients in at least one statistical period; An operating system statistical information determination module is configured to determine at least two dimensions of operating system statistical information of each storage node and coordination node in at least one statistical period; A performance diagnosis module is configured to diagnose the performance of the distributed database according to the data table statistical information, the network delay information, and the operating system statistical information in each statistical period; Determining at least two dimensions of data table statistical information of each data table in at least one statistical period comprises: Determining at least two dimensions of current data table statistical information of a target data table at a current time in a current statistical period, and at least two dimensions of historical data table statistical information of the target data table at a historical time in a previous statistical period; Determining at least two dimensions of data table statistical information of the target data table in the current statistical period according to the difference between the current data table statistical information and the historical data table statistical information.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the performance diagnosis method of the distributed database according to any one of claims 1-6.

9. A storage medium storing computer-executable instructions, wherein: The computer executable instructions, when executed by the computer processor, are used to execute the performance diagnosis method of the distributed database according to any one of claims 1-6.

Images