Distributed link-based database monitoring method, device, equipment and medium

Abstract

The application discloses a database monitoring method and device based on a distributed link, equipment and a storage medium, and belongs to the technical field of computers. After a business transaction request is executed on a database node, database stack information returned by the database node is acquired, wherein the database stack information comprises database access connection information, structured query language statement information corresponding to the business transaction request, and request execution result information. According to the call stack information of the database access node and the database stack information, each log item of the distributed link log of the database access node is generated, so that the distributed link becomes a full-link coverage in a true sense.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a database monitoring method, apparatus, device, and storage medium based on distributed links. Background Technology

[0002] With the continuous development of information technology, distributed systems can meet the requirements of high availability and high performance. Under this architecture, distributed tracing technology has become a key element in ensuring system stability and performance.

[0003] In related technologies, distributed tracing solutions do not define standards and implementation methods for database access. They only treat the database as the last black-box access node at the end of the link. All database business transactions occurring on it return an overall result based on the call stack. The distributed tracing log generated by the database access node based on this result cannot reflect the detailed information generated when the database executes business transaction requests. Summary of the Invention

[0004] The main objective of this application is to provide a database monitoring method, apparatus, device, and storage medium based on distributed links, aiming to solve the problem of difficulty in implementing database monitoring under distributed links.

[0005] To achieve the above objectives, this application provides a database monitoring method based on distributed links, applied to database access nodes, the method comprising:

[0006] After the database node executes the business transaction request, the database stack information returned by the database node is obtained. The database stack information includes database access connection information, structured query language statement information corresponding to the business transaction request, and request execution result information.

[0007] Based on the call stack information and database stack information of the database access node, generate each log entry of the distributed tracing log of the database access node.

[0008] Optionally, the database access connection information includes the database connection information, the database name, and the database connection username.

[0009] Optionally, based on the call stack information and database stack information of the database access node, log entries of the distributed cascading log of the database access node are generated, including:

[0010] Based on the database type, obtain the corresponding database connection information template;

[0011] Based on the database connection information, fill in the database connection information template to generate the database connection information item in the distributed link log.

[0012] Optionally, the Structured Query Language (SCL) statement information includes the SCL statements, the statement execution order, and a list of database objects corresponding to each SCL statement; the list of database objects includes at least one of the following: a database table, a database view, and a stored procedure.

[0013] Optionally, based on the call stack information and database stack information of the database access node, log entries of the distributed cascading log of the database access node are generated, including:

[0014] If the length of the Structured Query Language (SCL) statement is greater than the preset truncation value, the SCL statement will be truncated at the preset truncation value to obtain the processed SCL statement with the preset truncation value.

[0015] The constant parts of the processed Structured Query Language statement are replaced with preset characters to obtain a standardized Structured Query Language statement.

[0016] Based on standardized Structured Query Language (SCL) statements, generate SCL text items for distributed link logs.

[0017] Optionally, after replacing the constant portion of the Structured Query Language (SCL) statement with preset characters to obtain a standardized SCL statement, the method further includes:

[0018] Using a hash algorithm, obtain the hash value corresponding to a standardized Structured Query Language statement;

[0019] Based on the hash value, generate a Structured Query Language statement identification flag item in the distributed link log;

[0020] Generate a hash algorithm version entry in the distributed link log based on the version number corresponding to the hash algorithm.

[0021] Optionally, the request execution result information includes the start execution time and execution response time corresponding to the Structured Query Language statement.

[0022] Secondly, to achieve the above objectives, this application also provides a database monitoring device based on distributed links, comprising:

[0023] The information acquisition module is used to acquire the database stack information returned by the database node after the database node executes the business transaction request. The database stack information includes database access connection information, structured query language statement information corresponding to the business transaction request, and request execution result information.

[0024] The log generation module is used to generate log entries for the distributed trajectories log of the database access node based on the call stack information and database stack information of the database access node.

[0025] Thirdly, to achieve the above objectives, this application also provides a database monitoring device based on distributed links, including: a processor, a memory, and a database monitoring program based on distributed links stored in the memory. The database monitoring program based on distributed links is executed by the processor to implement the steps of the database monitoring method based on distributed links described above.

[0026] Fourthly, to achieve the above objectives, this application also provides a computer-readable storage medium storing a database monitoring program based on a distributed link, which, when executed by a processor, implements the aforementioned database monitoring method based on a distributed link.

[0027] This technical solution generates a distributed log for database analysis by collecting complete database stack information returned after the database executes business transaction requests. It implements a log design for business transaction access at the database node layer under distributed conditions, making the distributed log truly cover the entire database chain. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the structure of the device in this application.

[0029] Figure 2 This is a first flowchart illustrating the first embodiment of the method of this application.

[0030] Figure 3 This is a schematic diagram of the second process of the first embodiment of the method of this application.

[0031] Figure 4 This is a schematic diagram of the third process of the first embodiment of the method of this application.

[0032] Figure 5 This is an application scenario diagram for Example 1 of this application.

[0033] Figure 6 This is an application scenario diagram for Example 2 of this application.

[0034] Figure 7 This is the first implementation effect diagram of Example 2 of this application.

[0035] Figure 8 The second implementation effect diagram of Example 2 in this application.

[0036] Figure 9 The third implementation effect diagram of Example 2 in this application.

[0037] Figure 10 The fourth implementation effect diagram of Example 2 in this application.

[0038] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0039] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0040] In related technologies, distributed tracing solutions do not define standards and implementation methods for database access. They only treat the database as the last black-box access node at the end of the link. All database business transactions occurring on it return an overall result based on the call stack. The distributed tracing log generated by the database access node based on this result cannot reflect the detailed information generated when the database executes business transaction requests.

[0041] This application proposes a solution to generate distributed logs for database analysis by collecting complete database stack information generated during the execution of business transactions. It implements a log design for business transaction access at the database node layer under distributed conditions, enabling true full-link coverage of the distributed network.

[0042] The following embodiments of this application will describe the database monitoring method, apparatus, device, and storage medium based on distributed links used in the technical implementation of this application:

[0043] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a database monitoring device based on a distributed link in the hardware operating environment involved in the embodiments of this application.

[0044] like Figure 1 As shown, the distributed link-based database monitoring device may include: a processor 1001, such as a CPU; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a voice pickup module, such as a microphone array; optionally, the user interface 1003 may also be a display screen or an input unit such as a keyboard. The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Alternatively, the memory 1005 may be a storage device independent of the aforementioned processor 1001.

[0045] It is understood that the database monitoring device based on distributed links may also include a network interface 1004, which may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). Optionally, the database monitoring device based on distributed links may also include RF (Radio Frequency) circuitry, sensors, audio circuitry, Wi-Fi modules, etc.

[0046] Those skilled in the art will understand that Figure 1 The distributed link-based database monitoring device structure shown in the figure does not constitute a limitation on the distributed link-based database monitoring device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0047] Based on, but not limited to, the hardware structure of the database monitoring device based on distributed links, this application provides a first embodiment of a database monitoring method based on distributed links. (Refer to...) Figure 2 , Figure 2 A flowchart illustrating the first embodiment of the database monitoring method based on distributed links of this application is shown.

[0048] It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0049] In this embodiment, the database monitoring method based on distributed links includes:

[0050] Step S100: After the database node executes the business transaction request, obtain the database stack information returned by the database node.

[0051] The database stack information includes database access connection information, structured query language statement information corresponding to the business transaction request, and request execution result information.

[0052] You can refer to Figure 5This diagram illustrates the generation process of distributed link logs during distributed link tracing in this embodiment. Specifically, the business application gateway, as the first node accessing the business transaction request, generates and sends TraceContext-AB to application AP1. After application AP1 returns a response, the business application gateway generates distributed link log A. Simultaneously, upon receiving the request from the business application gateway, application AP1 processes TraceContext-AB in the request. When calling application AP2, it generates and sends TraceContext-BC to application AP2. After application AP2 returns a response, it generates distributed link log B and sends a related request to the DB access application node. The DB access application node, as the last hop of the request, receives and processes TraceContext-CD in the request from application AP2, and accesses the database DB to obtain data according to the related request. Simultaneously, it collects the DBSTACK information returned by the database DB when executing the related access request, i.e., the database stack information.

[0053] In this step, the database stack information is used to track and record various details during the database access process. It provides a complete picture of database access during business transactions. Among them, the database access connection information records connection-related information of the database node, including database connection information, such as the database server's IP information, port number information, etc., as well as the database name and the username connected to the database.

[0054] The SQL (Structured Query Language) statement information corresponding to a business transaction request represents the instructions executed by database nodes in a distributed system during business transactions, including operations such as querying, updating, inserting, and deleting data. These instructions are typically written in SQL to tell the database what operation to perform. Specifically, the SQL statement information may include the specific SQL statement, the execution order of the SQL statements, and a list of database objects corresponding to the SQL statements. The list of database objects can include database tables, database views, and stored procedures, etc.

[0055] The request execution result information is the execution result of the database node for the corresponding SQL statement, including the start time of the SQL statement execution and the response time.

[0056] Understandably, the DBSTACK information returned by the database node contains detailed information about the database node when executing related access requests. Based on this information, a distributed tracing log describing the database in detail can be generated.

[0057] Step S200: Based on the call stack information and database stack information of the database access node, generate each log entry of the distributed link log of the database access node.

[0058] In this step, the call stack information contains the call information of the database node, which may include the database node identifier, the timestamp of sending the access request to the database node, the response time of the database node in responding to the request, and the return code of the database node in processing the request-related business (whether the business processing was successful, etc.).

[0059] Understandably, the call stack information provides the overall result information of the database node's execution of related access requests, while the database stack information provides detailed information (down to the SQL statement level) of the database node during the execution of related access requests. This step generates log entries in a distributed cascading log containing complete information about the database node's response to the related requests, based on the call stack information and the database stack information of the database access node.

[0060] Furthermore, refer to Figure 2 and Figure 3 In one specific implementation, step S200 includes:

[0061] Step S210: Obtain the corresponding database connection information template according to the database type.

[0062] Step S220: Fill in the database connection information template based on the database connection information to generate the database connection information item in the distributed link log.

[0063] In the above steps, the database type refers to the type of database management system corresponding to the database. Generally speaking, databases are usually managed by database management systems, and there are many types of database management systems, such as Oracle, MySQL, DB2, etc.

[0064] The constant part of an SQL statement includes numeric and character constants. For example, in "SELECT * FROM TABLEA WHERE BID = 'asd'", 'asd' is the constant part of the statement, which usually contains sensitive information related to business transactions.

[0065] The above steps define different database connection information templates for different types of database management systems, and fill the corresponding database connection information templates with the database connection information returned by the database nodes to generate the corresponding database connection information items.

[0066] The above steps generate different database connection information items based on the type of database management system corresponding to the database node. This allows the log analysis end to identify the database type based on the information item. At the same time, it eliminates the need to define separate log items related to the database type, effectively controlling the size of the distributed link log and preventing the distributed link log from becoming too large and affecting the performance of the distributed system.

[0067] Step S230: If the length of the structured query language statement is greater than the preset truncation value, the structured query language statement is truncated at the preset truncation value to obtain the processed structured query language statement with the preset truncation value.

[0068] In this step, the obtained SQL statement is truncated at a preset truncation value; that is, only the portion before the truncation value is retained, and the excess portion is cut off. It is understood that in this embodiment, when analyzing distributed logs, longer SQL statements can cause the distributed logs to become too large, impacting system performance. Therefore, to balance the size and content of the distributed logs, a truncation value of 256 is preferred.

[0069] Step S240: Replace the constant part of the processed Structured Query Language statement with preset characters to obtain a standardized Structured Query Language statement.

[0070] Step S250: Generate the structured query language statement text item in the distributed link log based on the standardized structured query language statement.

[0071] In the above steps, the constant part of the SQL statement includes the numeric part and character constants in the SQL statement. For example, in "SELECT * FROM TABLEA WHERE BID = 'asd'", 'asd' is the constant part of the statement, which usually contains sensitive information related to business transactions.

[0072] Specifically, the above steps target the constant portion of the intercepted SQL statement, use preset character replacements to obtain a standardized SQL statement, and use it as a log item in the distributed link log, that is, a Structured Query Language statement text item.

[0073] The above steps, by standardizing the SQL statements returned by the database nodes, not only enable the tracking of the SQL statements executed by the database nodes, but also ensure that the capacity of the distributed log does not become too large, and avoid the leakage of sensitive information.

[0074] Furthermore, refer to Figure 4 In one specific embodiment, after step S240, the method further includes:

[0075] Step S260: Use a hash algorithm to obtain the hash value corresponding to the standardized Structured Query Language statement.

[0076] Step S270: Generate a Structured Query Language statement identification flag item in the distributed link log based on the hash value.

[0077] Step S280: Generate the hash algorithm version entry in the distributed link log based on the version number corresponding to the hash algorithm.

[0078] A hash algorithm is a mathematical function that converts input data into a fixed-length string. It accepts input, performs calculations, and produces a unique, fixed-length output, commonly called a hash value. It's worth noting that the same SQL statement will produce different hash values ​​using different hash algorithms; therefore, it's necessary to record the version number of the hash algorithm.

[0079] The above steps utilize the characteristic that "the same SQL statement using the same hash algorithm will have the same hash value, while different SQL statements using different hash algorithms will have different hash values" to use the hash value as a unique identifier for the SQL statement executed by the database.

[0080] Specifically, the above steps convert standardized SQL statements into fixed-length hash values, which are then used as SQL statement identification markers in the distributed link log, and the hash algorithm version number is used as the hash algorithm version entry in the distributed link log. Understandably, to ensure the uniqueness of the standardized SQL statements—that is, to ensure the extracted SQL statements can fully express the information of the original SQL statements—the preset truncation value cannot be too small. At the same time, the computational efficiency of the hash algorithm must be considered. Therefore, unlike the truncation value used to generate the structured query language statement text items, the truncation value for the SQL statement identification marker is preferably set to 3000.

[0081] Understandably, the SQL statement identification flag serves as a unique identifier for the SQL statements executed by the database node, playing a crucial role in distributed log analysis of database-related data in other log analysis systems. To better understand this embodiment, an example is provided below:

[0082] Imagine a large business transaction platform processing millions of transactions daily. To ensure high availability and performance, this platform employs a distributed architecture, with the database being a core component. In this scenario, analyzing logs to pinpoint the problem is crucial if system performance issues or database access anomalies occur. The SQL statement identification markers for database node execution provided in this embodiment offer key clues in the logs. Suppose that at some point, the system's task processing speed suddenly drops, causing delays for users. By analyzing the distributed logs, the specific database node accessed by the business transaction can be traced, and the specific SQL statements executed by the database can be accurately located based on the SQL statement identification markers in the distributed logs. This means that it's possible to accurately trace which specific batch of SQL statements caused the performance problem. For example, a complex query or an unoptimized database operation might cause a slowdown in database response.

[0083] This embodiment generates a distributed log for database nodes based on the database stack information and call stack information returned by the database nodes when executing business transaction requests. It implements a log design scheme for business transaction access at the database node layer under distributed conditions, making the distributed log truly cover the entire link.

[0084] It is easy to understand, of course, that although a logical order is shown in the above embodiments, in some cases the steps shown or described may be performed in a different order than that shown here.

[0085] To enable those skilled in the art to better understand the scope of protection of the claims of this application, specific implementation examples in specific application scenarios are used to explain and illustrate the technical solutions described in the claims of this application. It should be understood that the following examples are only used to explain this application and are not intended to limit the scope of protection of the claims of this application.

[0086] like Figure 5 As shown, Example 1 is a specific application scenario of this application:

[0087] Figure 5This is the system architecture diagram for this example. The diagram illustrates the generation process of the distributed tracing log in this application's distributed tracing process. Specifically: The business application gateway, as the first node accessing the business transaction request, generates and sends TraceContext-AB to application AP1. After application AP1 returns a response, the business application gateway generates distributed tracing log A. Simultaneously, upon receiving a request from the business application gateway, application AP1 processes the TraceContext-AB in the request. When calling application AP2, it generates and sends TraceContext-BC to application AP2. After application AP2 returns a response, it generates distributed tracing log B. This process continues. When the database access application node, as the last hop in the request, receives and processes the TraceContext-CD in the application AP2 request, accesses the database DB to obtain data according to the relevant request, and simultaneously collects the DBSTACK information returned by the database DB when executing the relevant access request. Based on the DBSTACK information, it generates database distributed tracing log D.

[0088] In the above process, TraceContext serves as the trace log information, containing crucial information for tracing business transaction requests, such as a unique ID identifying the request, the request's start timestamp, and the parent node identifier. TraceContext is passed between multiple nodes as a request traverses the system. Each node retrieves the TraceContext and adds its own identification information to it to trace the execution path of the entire business transaction request. Through TraceContext, the entire execution process of a business transaction request can be chained together, enabling end-to-end tracing of requests in a distributed system. Each distributed trace node can generate its own distributed trace log based on the TraceContext and the response returned by the next node.

[0089] Specifically, the distributed tracing logs generated by each node contain log items as shown in Table 1. The BusinessId and TraceId are generated by the business application gateway based on the business transaction request definition. The call stack information in Table 1 contains the call information for the next hop of the node. It can be understood that by associating the traceid and other information from the tracing logs with the call stack information, application-level distributed tracing can be achieved.

[0090]

[0091] Understandably, for DB access applications, the next node is the database DB node. If the distributed tracing log D is generated using the field information in Table 1, it cannot fully reflect the detailed information of the data returned by the database node after executing related requests. Therefore, this example defines DBSTACK log entries as shown in Table 2 based on the database stack DBSTACK information returned after the database DB executes related requests. Based on the DBSTACK log entries shown in Table 2 and the distributed tracing log entries shown in Table 1, the database distributed tracing log D is generated to achieve distributed tracing at the database level.

[0092] As shown in Table 2, the functions of each defined field are as follows:

[0093] `platfInfo`: This field identifies the framework platform type used to implement the database distributed stack trace scheme. For example, in this case, the DB access application can utilize the improved SUMMER framework platform to obtain database stack trace information, taking the first four characters. If no framework platform is used, then "OTHR" is entered. This field provides a common standard for distributed stack trace designs at different database levels. Different framework platforms or applications can refer to this standard to implement database distributed stack traces, allowing for log differentiation based on the framework platform type when parsing distributed stack trace logs later.

[0094]

[0095] dbConnInfo: This field represents database connection information. Different types of database management systems have different connection methods, therefore, different database connection information needs to be defined according to different database management systems. For example:

[0096] For the Oracle database management system, database connection information is defined as follows:

[0097] [IP]:[Port];[ServiceName] For example: "192.168.1.22:1521;SeviceXXXX", where, for Oracle database management system, the [ServiceName] field represents the service name provided by Oracle database management system, which is used to help DB access applications access the corresponding database instance.

[0098] For MySQL databases, the database connection information is defined as follows:

[0099] IP address access method:

[0100] [IP1]:[Port],[IP2]:[Port],[IP3]:[Port]; [InstName]; For example, "192.168.1.11:6446,192.168.2.22:6446,192.168.3.33:6446;testinst", where [InstName] represents the specific database instance name.

[0101] Understandably, to accommodate the size of the distributed database log, Table 2 does not directly define the database management system type. By defining different database connection information for different database management systems, the database management system type can be identified during subsequent distributed database log parsing. It is worth mentioning that this field, through the appropriate use of symbols such as ":", "," and ";", allows for the identification of the content of different fields in the database connection information during subsequent distributed database log parsing.

[0102] `dbName`: Represents the database name. This field only takes the first 16 characters of the database name; any part exceeding 16 characters needs to be truncated, and no padding is required if it is insufficient. This is because the definition of this field needs to be closely related to the database naming length definition in the database development specifications of the operating environment, and it should not be too long. The best practice in this example is a length of 16 bytes.

[0103] userName: Database connection username, containing basic information about the database connection.

[0104] sqlObjInfo: A list of database objects executed by the SQL statement, including tables, views, stored procedures, etc., separated by commas, taking the first 64 characters of the list. This field is used to establish the association between business transactions and database objects. For example, the BusinessId field of the distributed log of the database can determine the information of a business transaction, and then the sqlObjInfo field of the distributed log can be used to determine the database objects related to that business transaction.

[0105] sqlSeq: This example uses a 4-character string to indicate the execution sequence number of the SQL statements in this call. A business transaction typically involves multiple database interactions, so it's necessary to define the execution order of each SQL statement. By using the statement order and the start and end flags of each database transaction, the number of database transactions executed in a single business transaction can be determined.

[0106] sqlTxt: This field only takes the first 256 characters of the SQL statement executed by the database node; any excess is truncated. This field can be used to visualize the SQL statement. To avoid displaying sensitive information, this example replaces numbers and character constants in the SQL text with the "?" character.

[0107] Additionally, the STATEMENT, COMMIT, and ROLLBACK behaviors of a database can be defined as special sqlTxts to serve as a delimiter for a database transaction.

[0108] `sqlID`: Extracts the first N characters of the SQL statement body executed by the database node (DB node), hashes them to obtain a unique identifier, and uses this hash value as the `sqlID` field. Generally, the length of SQL statements executed by the database node varies depending on different business needs; some analytical statements can even exceed 100KB. Therefore, from a log capacity perspective, recording such a large number of SQL statements is impractical. Furthermore, variable-length SQL statement logs cause variable-length dbstack logs, hindering subsequent processing of the database distributed log. Therefore, it is necessary to standardize the SQL statements into fixed-length structures according to certain processing logic, and then calculate the fixed-length `sqlID`. Understandably, the `sqlID` can then be used to associate the database distributed log with various database distributed log parsing platforms. For SQL statement standardization, the best practice in this example is: convert SQL statements to lowercase, replace numeric or character constants with "?" symbols, remove spaces, and use the first N = 3000 characters.

[0109] sqlIDVer: The hash algorithm version number is used to distinguish and associate different versions of sqlID when the algorithm version changes. The same SQL statement may produce different hash values ​​under different hash algorithm versions, but the hash value calculated under the same hash algorithm version is consistent. In particular, sqlID is the core link between SQL statements in the database distributed log and other log parsing platforms, so in practical applications, hash algorithm version updates should be minimized as much as possible.

[0110] sqlStartTimestamp: The time when the SQL statement started execution, expressed in milliseconds, using a Unix timestamp. Add spaces if necessary, for example: "1547438399837".

[0111] sqlResponseTime: The difference in timestamps between the database response to the SQL statement and the timestamp of the SQL statement being issued, in milliseconds. Enter "0" if there is no response, indicating a timeout.

[0112] sqlReturnCode: The SQL return code returned by the database (DB) to the application node accessing the DB. Enter "SUC0000" if empty or if the peer returned success, "TIMEOUT" if a timeout occurred, "ASYNCAL" for asynchronous calls, and the original SQLCODE for other exceptions. If the original SQLCODE has sub-error codes, separate them with ":", e.g., SQL0911N:68; if multiple error codes are returned, separate them with ",", e.g., "ORA-XXXX,ORA-XXXX".

[0113] sqlEftRows: The number of rows affected by the SQL statement. If the value cannot be obtained or there are no affected rows, enter "0".

[0114] sqlRem: Reserved field, can be set to empty.

[0115] Understandably, a business transaction request may involve multiple database accesses. Each time a database access occurs, the database (DB) returns DBSTACK information. Therefore, to avoid the application being frequently invoked during a business transaction request and thus consuming resources, this example can centrally obtain all DBSTACK information after a business transaction request ends, in order to generate a distributed database log.

[0116] This example establishes a standard for database SQL access logs, creating a unified log standard for the generated distributed data link logs. It also standardizes the field lengths of each log entry, minimizing log space usage while ensuring maximum log information content.

[0117] like Figure 6 As shown, based on Example 1, Example 2 is another specific application scenario of this application:

[0118] In this example, the front-end application acts as the first node receiving business transaction requests, responsible for sending these requests to the next node, application service 1. After application service 1 processes the business transaction request, it returns a response to the front-end application. The front-end application then generates a distributed tracing log E and outputs it to the Kafka cluster. This means that, according to the node access path corresponding to the business transaction request, subsequent application services receive the request information from the previous node, pass the business transaction request and tracing log information to the next application node, generate their own distributed tracing log based on the response information returned by the next node, and output it to the Kafka cluster.

[0119] When a business transaction request is routed to the application service node 2 that accesses the database, application service 2 generates a distributed database log G based on the DBSTACK information returned by the database service executing the business transaction request, and uploads the log to the Kafka cluster.

[0120] After the distributed logs related to business transaction requests are uploaded to the Kafka cluster, the logs undergo disk persistence and preprocessing. For example, the distributed logs of each node are sorted according to the order in which business transaction requests access the nodes before being persisted to disk. Figure 6 The DB1 and DB2 databases shown here use a database product that supports big data types, such as HBase, for log persistence because the link logs are massive amounts of data.

[0121] The distributed link platform is responsible for interpreting log fields and building data analysis models for distributed link logs. In this example application scenario, the distributed link platform can parse the database distributed link logs generated by application service 2, and perform data association and processing based on the database distributed link logs, the configuration information platform, and the database operation and maintenance management platform. Specifically, the configuration information platform can contain the correspondence between database administrators and each database. Understandably, the database administrator corresponding to each database can obtain the association between business transactions and the detailed information of the databases accessed based on the database distributed link logs parsed by the distributed link platform, achieving more refined and accurate database management.

[0122] It is worth mentioning that the distributed link platform will also monitor the SQL statements corresponding to the SQLID log entries in the distributed link log according to the preset alarm rules. When the SQL statement is found to match the alarm rules, alarm information is generated, summarized on DB3, and sent to the alarm platform through the background program.

[0123] In summary, this example uses SQLID to establish a connection between business transactions and various database platforms, enabling the integration of external and internal database monitoring.

[0124] Finally, based on the data result set processed by the distributed link platform, a graphical interface display of each functional scenario of the present invention is implemented, that is... Figure 6 The platform showcases front-end scene functional interfaces. The specific display effect is as follows:

[0125] Figure 7 This is an interface for displaying DBSTACK information in the database distributed link log.

[0126] Figure 8This page displays information related to database access for a single business transaction. When a single business transaction experiences a timeout or excessively long response time, it is necessary to identify which part of the business transaction takes the longest time. If the problem is caused by a database node, the problematic SQL statement can be quickly located through this page.

[0127] Figure 9 To demonstrate the associations between important business transactions and their corresponding databases and objects, it is necessary to understand the databases and their associated objects when performing maintenance on important business transactions. This ensures that database objects are not subject to planned changes during the maintenance period and that associated objects can be quickly identified when problems occur.

[0128] Figure 10 This section displays the relationships between database objects and the business transactions accessing them. When making changes or maintaining a database table, it's necessary to confirm which business transactions will be affected if the table becomes unavailable. Additionally, it allows you to view which SQL statements are associated with this database table object.

[0129] Understandably, database distributed logs record every SQL statement generated when business transactions access the database. Therefore, in a busy system with numerous database access interactions, the volume of database distributed logs can be very large. To ensure that the recording of database distributed logs does not affect business transactions, restrictions need to be placed on log uploads. Specifically, database distributed logs should be uploaded to the Kafka cluster asynchronously to avoid impacting the execution of other tasks in the distributed system's application services. To avoid consuming system resources due to frequent distributed log uploads, a "caching + timed" approach can be used. Specifically, distributed logs generated by each node can be placed in a cache in the order of generation. When the cache is full, the upload is triggered, and the cache is cleared. Furthermore, specifically, if performance issues or other abnormal situations occur during distributed log uploads, writing must be stopped and a failure flag must be set to avoid affecting the operation of the application itself.

[0130] As can be seen from the above embodiments, this application, through the design of a distributed link log scheme at the distributed link database node layer, enables the distributed link to achieve true full-link coverage. By analyzing the distributed link logs of the database, a complete association between business transactions and database objects is established, giving the database and related objects business attribute information. Through the correlation between external and internal database monitoring, the root cause analysis of abnormal business transaction problems is achieved, enabling rapid location of the problem at the database object level.

[0131] Based on the same inventive concept, this application also provides a database monitoring device based on distributed links, the device comprising:

[0132] The information acquisition module is used to acquire the database stack information returned by the database node after the database node executes the business transaction request. The database stack information includes database access connection information, structured query language statement information corresponding to the business transaction request, and request execution result information.

[0133] The log generation module is used to generate log entries for the distributed trajectories log of the database access node based on the call stack information and database stack information of the database access node.

[0134] It should be noted that the various embodiments of the database monitoring device based on distributed links in this embodiment, and the technical effects they achieve, can be referred to the various implementation methods of the database monitoring method based on distributed links in the foregoing embodiments, and will not be repeated here.

[0135] Furthermore, embodiments of this application also propose a computer storage medium storing a database monitoring program based on a distributed link. When executed by a processor, the database monitoring program based on a distributed link implements the steps of the database monitoring method based on a distributed link described above. Therefore, it will not be repeated here. Additionally, the beneficial effects of using the same method will not be repeated here either. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed to execute on a single computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0136] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0137] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0139] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A database monitoring method based on distributed links, characterized in that, Applied to a database access node, the method includes: After the database node executes the business transaction request, the database stack information returned by the database node is obtained; wherein, the database stack information includes database access connection information, structured query language statement information corresponding to the business transaction request, and request execution result information; Based on the call stack information of the database access node and the database stack information, generate each log entry of the distributed link log of the database access node; The step of generating log entries for the distributed link log of the database access node based on the call stack information and the database stack information includes: If the length of the structured query language statement is greater than the preset truncation value, the structured query language statement is truncated at the preset truncation value to obtain a processed structured query language statement with a length of the preset truncation value. The constant part of the processed structured query language statement is replaced with preset characters to obtain a standardized structured query language statement. Based on the standardized structured query language statements, generate structured query language statement text items in the distributed link log; The hash value corresponding to the standardized structured query language statement is obtained using a hash algorithm; Based on the hash value, generate a Structured Query Language statement identification flag item in the distributed link log; Based on the version number corresponding to the hash algorithm, a hash algorithm version entry is generated in the distributed link log.

2. The database monitoring method based on distributed links according to claim 1, characterized in that, The database access connection information includes the database connection information, database name, and database connection username.

3. The database monitoring method based on distributed links according to claim 2, characterized in that, The step of generating log entries for the distributed link log of the database access node based on the call stack information and the database stack information includes: Based on the type of the database, obtain the corresponding database connection information template; Based on the database connection information, fill in the database connection information template to generate the database connection information item in the distributed link log.

4. The database monitoring method based on distributed links according to claim 1, characterized in that, The Structured Query Language (SCL) statement information includes the SCL statements, the statement execution order, and a list of database objects corresponding to each SCL statement; the list of database objects includes at least one of database tables, database views, and stored procedures.

5. The database monitoring method based on distributed links according to claim 1, characterized in that, The request execution result information includes the start execution time and execution response time corresponding to the structured query language statement.

6. A database monitoring device based on distributed links, characterized in that, The device includes: The information acquisition module is used to acquire the database stack information returned by the database node after the database node executes the business transaction request; wherein, the database stack information includes database access connection information, structured query language statement information corresponding to the business transaction request, and request execution result information; The log generation module is used to generate each log entry of the distributed link log of the database access node based on the call stack information of the database access node and the database stack information; Based on the call stack information of the database access node and the database stack information, generate the following log entries for the distributed link log of the database access node: If the length of the structured query language statement is greater than the preset truncation value, the structured query language statement is truncated at the preset truncation value to obtain a processed structured query language statement with a length of the preset truncation value. The constant part of the processed structured query language statement is replaced with preset characters to obtain a standardized structured query language statement. Based on the standardized structured query language statements, generate structured query language statement text items in the distributed link log; The hash value corresponding to the standardized structured query language statement is obtained using a hash algorithm; Based on the hash value, generate a Structured Query Language statement identification flag item in the distributed link log; Based on the version number corresponding to the hash algorithm, a hash algorithm version entry is generated in the distributed link log.

7. A database monitoring device based on distributed links, characterized in that, include: A processor, a memory, and a distributed link-based database monitoring program stored in the memory, wherein the distributed link-based database monitoring program is executed by the processor to implement the steps of the distributed link-based database monitoring method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a database monitoring program based on a distributed link, which, when executed by a processor, implements the database monitoring method based on a distributed link as described in any one of claims 1 to 5.

Images