Data query method, device, server and storage medium

Abstract

The embodiment of the present application discloses a data query method, device, server and storage medium. The method includes: obtaining first slow query data determined by an application system, storing the first slow query data in a database of the slow query distributed system; obtaining second slow query data that meets preset conditions from the slow query data included in the database according to the slow query configuration parameters corresponding to the application system; and sending the second slow query data to a target device corresponding to the application system. The slow query distributed system can send the slow query data that meets the preset conditions to the target device used by the operation and maintenance personnel of the application system, so that the operation and maintenance personnel can subsequently obtain the slow query data that needs to be optimized in a timely manner based on the slow query data sent by the slow query distributed system, and then optimize the slow query data in a timely manner, and finally achieve the purpose of improving the processing efficiency of the database of the application system.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a data query method, device, server and storage medium. Background Art

[0002] The rapid development of communication and network technologies has promoted the geometric growth of online application systems. The service content of online application systems has also developed from the original single static to multiple dynamic interactive modes, improving the interactivity and richness of user experience. However, when a large number of users experience the same online service, they often encounter busy servers. Therefore, multiple servers are often used to provide services for the same application and distribute the applications. While distributed deployment improves the service capabilities of online application systems, it also increases the complexity of the service process of online application systems. The database processing efficiency of online application systems is an important link that affects the service capabilities of online application systems. In related methods, the processing efficiency of the database can be improved by obtaining slow query data that operates on the database and optimizing the slow query data. Therefore, how to obtain the slow query data that needs to be optimized in a timely manner is a technical problem that needs to be solved. Summary of the invention

[0003] In view of the above problems, the present application proposes a data query method, device, server and storage medium to timely obtain slow query data that needs to be optimized.

[0004] In a first aspect, an embodiment of the present application provides a data query method, which is applied to a slow query distributed system, the method comprising: obtaining first slow query data determined by an application system, and storing the first slow query data in a database of the slow query distributed system; obtaining second slow query data that meets preset conditions from the slow query data included in the database according to slow query configuration parameters corresponding to the application system; and sending the second slow query data to a target device corresponding to the application system, the target device being a device used by operation and maintenance personnel of the application system.

[0005] It can be seen that in the embodiment of the present application, the slow query distributed system can directly find the slow query data that meets the preset conditions according to the slow query configuration parameters corresponding to the application system, and send the slow query data that meets the preset conditions to the target device corresponding to the application system, so that the slow query distributed system can monitor the slow query data generated by the application system and obtain the slow query data of the preset conditions in a timely manner, and send the slow query data that meets the preset conditions to the target device used by the operation and maintenance personnel of the application system, so that the operation and maintenance personnel can obtain the slow query data that needs to be optimized in a timely manner, and then optimize the slow query data in a timely manner, and finally achieve the purpose of improving the processing efficiency of the database of the application system.

[0006] In a second aspect, an embodiment of the present application provides a data query method, which is applied to an application system, the method comprising: in response to an operation on a database of the application system, obtaining an operation parameter of an operation statement corresponding to the operation; if the operation statement is determined to be first slow query data based on the operation parameter and a slow query configuration parameter corresponding to the application system; then sending the first slow query data to a slow query distributed system; wherein the first slow query data is used to: trigger the slow query distributed system to store the first slow query data in a database of the slow query distributed system; according to the slow query configuration parameter corresponding to the application system, obtaining second slow query data that meets a preset condition from the slow query data included in the database, and sending the second slow query data to a target device corresponding to the application system, the target device being a device used by an operation and maintenance personnel of the application system.

[0007] It can be seen that in an embodiment of the present application, after the application system sends the slow query data to the slow query distributed system, the slow query distributed system can be triggered to directly find the slow query data that meets the preset conditions according to the slow query configuration parameters corresponding to the application system, and send the slow query data that meets the preset conditions to the target device corresponding to the application system, so that the slow query distributed system can monitor the slow query data generated by the application system and obtain the slow query data of the preset conditions in a timely manner, and send the slow query data that meets the preset conditions to the target device used by the operation and maintenance personnel of the application system, so that the operation and maintenance personnel can subsequently obtain the slow query data that needs to be optimized in a timely manner, and then optimize the slow query data in a timely manner, and ultimately achieve the purpose of improving the processing efficiency of the database of the application system.

[0008] In a third aspect, an embodiment of the present application provides a data query device, which runs in a slow query distributed system, and the device includes: a first data acquisition unit, which is used to acquire first slow query data determined by an application system, and store the first slow query data in a database of the slow query distributed system; a second data acquisition unit, which is used to acquire second slow query data that meets preset conditions from the slow query data included in the database according to slow query configuration parameters corresponding to the application system; and a data sending unit, which is used to send the second slow query data to a target device corresponding to the application system, and the target device is a device used by operation and maintenance personnel of the application system.

[0009] In a fourth aspect, an embodiment of the present application provides a server, comprising one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0010] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, wherein the above method is executed when the program code is run. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram of a flow chart of an existing example is shown;

[0013] Figure 2 A flow chart of a data query method proposed in one embodiment of the present application is shown;

[0014] Figure 3 A schematic diagram showing an application scenario of the method corresponding to S110-S150 proposed in an embodiment of the present application;

[0015] Figure 4 A timing diagram showing the access of an application system to a slow query distributed system according to an embodiment of the present application is shown;

[0016] Figure 5 A flow chart of a data query method proposed in another embodiment of the present application is shown;

[0017] Figure 6 A flow chart of a data query method proposed in another embodiment of the present application is shown;

[0018] Figure 7 A schematic diagram showing another embodiment of the present application for pushing modified slow query configuration parameters is shown;

[0019] Figure 8 A schematic diagram showing another embodiment of the present application for pushing top-n slow query data is shown;

[0020] Fig. 9 A flow chart of a data query method proposed in another embodiment of the present application is shown;

[0021] Fig.10 A flow chart of a data query method proposed in another embodiment of the present application is shown;

[0022] Fig.11 A schematic diagram of obtaining slow query configuration parameters proposed in another embodiment of the present application is shown;

[0023] Fig.12 A schematic diagram showing an implementation process of the method corresponding to S540-S570 proposed in yet another embodiment of the present application;

[0024] Fig.13 A structural block diagram of a data query device proposed in an embodiment of the present application is shown;

[0025] Fig.14 A structural block diagram of a data query device proposed in an embodiment of the present application is shown;

[0026] Fig.15 A structural block diagram of a data query system proposed in an embodiment of the present application is shown;

[0027] Fig.16 A structural block diagram of a server used by the present application to execute a data query method according to an embodiment of the present application is shown;

[0028] Fig.17 The present application shows a storage unit for storing or carrying a program code for implementing a data query method according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] For example, Figure 1 As shown, first select the relational database that needs to enable slow query, configure the slow query related parameters in the configuration file of the relational database, such as to enable slow query in the MySQL database, you need to add the slow query related parameters in the configuration file my.cnf, and then restart the relational database, so that the slow query related parameters configured in the configuration file take effect. When the relational database executes sql (Structured Query Language), if the execution time of sql is greater than the slow query threshold in the configured slow query related parameters, the slow query sql is recorded in the slow query log, so that the administrator of the relational database can query the slow query data through the client corresponding to the relational database. To obtain the slow query data for operating the database by the above method, it is necessary to set the slow query related parameters for each database and restart the database, which is relatively inconvenient and inefficient for medium and large software companies with many databases.

[0031] Therefore, in order to solve the above problems, the present application proposes a data query method, device, server and storage medium. The relevant contents will be described in detail below with reference to the accompanying drawings.

[0032] See also Figure 2 A data query method provided in an embodiment of the present application is applied to a data query system, the data query system including an application system and a slow query distributed system, and the method specifically includes the following steps:

[0033] Step S110: the application system obtains operation parameters of an operation statement corresponding to the operation in response to an operation on a database of the application system.

[0034] In the embodiment of the present application, the application system may include a single server or a server cluster. The database of the application system is used to store data generated during the execution of related services by the application system.

[0035] The operation on the database of the application system is performed in the process of using the application system to execute related business. The operation on the database of the application system includes at least one of inserting, updating, deleting, querying, adding, and modifying.

[0036] The operation statement corresponding to the operation includes a SQL statement, and the operation parameter of the operation statement includes at least one of the execution time of the SQL statement and the operation statement type of the SQL statement.

[0037] Step S120: If the operation statement is determined to be first slow query data based on the operation parameters and the slow query configuration parameters corresponding to the application system, the application system sends the first slow query data to the slow query distributed system.

[0038] The slow query configuration parameters corresponding to the application system are pre-set for the application system in the slow query distributed system. Different application systems can configure different slow query configuration parameters according to their own characteristics, such as by time period or by SQL type.

[0039] Among them, slow query data refers to SQL statements whose execution time is greater than or equal to the preset time.

[0040] After the application system obtains the operation parameters corresponding to the SQL statement, it can compare the operation parameters corresponding to the SQL statement with the slow query configuration parameters corresponding to the application system, perform slow query verification on the SQL statement, and determine whether the SQL statement is slow query data.

[0041] If it is determined that the execution time of the SQL statement is greater than or equal to the preset time based on the operation parameters corresponding to the SQL statement and the slow query configuration parameters corresponding to the application system, the SQL statement is sent to the slow query distributed system so that the slow query distributed system stores the SQL statement.

[0042] Step S130: the slow query distributed system obtains the first slow query data determined by the application system, and stores the first slow query data in a database of the slow query distributed system.

[0043] The database of the slow query distributed system is used to store the slow query data corresponding to different application systems. The slow query distributed system may include a single server or a server cluster.

[0044] After the slow query distributed system obtains the first slow query data, the first slow query data may be stored in the database of the slow query distributed system. At this time, if multiple application systems simultaneously send corresponding first slow query data to the slow query distributed system, the slow query distributed system may store the first slow query data determined by different application systems in the databases of different slow query distributed systems when storing the first slow query data, so that the first slow query data determined by the specified application system can be easily obtained in subsequent use.

[0045] Step S140: the slow query distributed system acquires second slow query data satisfying a preset condition from the slow query data included in the database of the slow query distributed system according to the slow query configuration parameters corresponding to the application system.

[0046] The preset condition is a pre-set condition for obtaining a specified amount of slow query data from a database of the slow query distributed system. The second slow query data is the determined slow query data corresponding to the slow query configuration parameter.

[0047] As a method, the slow query distributed system can obtain the second slow query data that meets the preset condition from the slow query data included in the database of the slow query distributed system according to the slow query configuration parameters at a fixed time. The timing can be a preset time period, and whenever the preset time period is reached, the slow query distributed system obtains the second slow query data that meets the preset condition from the slow query data included in the database of the slow query distributed system according to the slow query configuration parameters.

[0048] Step S150: the slow query distributed system sends the second slow query data to a target device corresponding to the application system, where the target device is a device used by an operation and maintenance personnel of the application system.

[0049] After obtaining the second slow query data, the slow query distributed system may send the second slow query data to a device used by an operation and maintenance personnel of the application system.

[0050] In the embodiment of the present application, the application scenario of the method corresponding to step S110-step S150 can be as follows: Figure 3 shown. Figure 3 In the example, the user device corresponding to the application system is the device used by the user who uses the business functions provided by the application system to complete the corresponding business; the message queue middleware is used to realize the decoupling of the application system and the slow query distributed system, wherein the message queue middleware may include a single server or a server cluster. There are multiple message queues for storing different data in the message queue middleware. The specific subject queue of the message queue middleware is the message queue for storing the slow query data of the application system. When the application system generates slow query data, the application system will send the slow query data to the specific subject queue. When the slow query data arrives in the specific subject queue, the slow query distributed system can read the slow query data from the specific subject queue.

[0051] Specifically, when an operation is performed on the database of the application system, it can be determined whether slow query data is generated based on the slow query configuration parameters corresponding to the application system. When it is determined that slow query data is generated, the application system can asynchronously send the slow query data to the specific topic queue of the message queue middleware. The slow query distributed system subscribes to the specific topic queue of the message queue middleware. When slow query data arrives in the specific topic queue of the message queue middleware, the slow query distributed system can obtain the slow query data from the specific topic queue of the message queue middleware and store the slow query data in the database of the slow query distributed system, so that the slow query distributed system can periodically calculate the Top-n slow query data and send the calculated Top-n slow query data to the equipment used by the operation and maintenance personnel of the application system. Among them, the Top-n slow query data are the top n slow query data in the slow query data.

[0052] Of course, before the application system sends slow query data to the slow query distributed system, the application system needs to be connected to the slow query distributed system. The timing diagram of the application system connecting to the slow query distributed system is as follows: Figure 4As shown. First, before starting the application system, you need to add the slow query distributed system client jar package dependency and perform basic configuration on slow query. After the configuration is completed, start the application system. Then the application system operation and maintenance personnel log in to the slow query distributed system, fill in the application system's relevant configuration items on slow query, and submit them for review. The slow query distributed system operation and maintenance personnel review the application system's access request. If the review is passed, the application system's relevant configuration items on slow query will take effect and generate slow query configuration parameters. Therefore, the slow query distributed system can request to call the interface in the slow query client jar package to push the application system's slow query configuration parameters to the application system in real time. When the application system receives the slow query configuration parameters pushed by the slow query distributed system, it can filter out the slow query data according to the slow query configuration parameters and send the slow query data to the message queue middleware.

[0053] Among them, the jar package can be understood as a third-party library function. After relying on the jar package, you can call the method provided in the jar package to implement the corresponding function of the method.

[0054] A data query method provided by an embodiment of the present application, a slow query distributed system can directly find slow query data that meets preset conditions according to the slow query configuration parameters corresponding to the application system, and send the slow query data that meets the preset conditions to a target device corresponding to the application system, so that the slow query distributed system can monitor the slow query data generated by the application system and timely obtain the slow query data of the preset conditions, and send the slow query data that meets the preset conditions to the target device used by the operation and maintenance personnel of the application system, so that the operation and maintenance personnel can subsequently timely obtain the slow query data that needs to be optimized based on the slow query data sent by the slow query distributed system, and then timely optimize the slow query data, thereby ultimately achieving the purpose of improving the processing efficiency of the database of the application system.

[0055] See also Figure 5 , a data query method provided in an embodiment of the present application is applied to a slow query distributed system, the method comprising:

[0056] Step S210: Acquire first slow query data determined by the application system, and store the first slow query data in a database of the slow query distributed system.

[0057] In an embodiment of the present application, the slow query distributed system may obtain the first slow query data periodically or in real time.

[0058] As one of the ways, when the slow query distributed system obtains the first slow query data regularly, the time for obtaining the first slow query data can be pre-set, and then when the pre-set time is reached, the slow query distributed system starts to obtain the first slow query data. The pre-set time can be set based on historical data. For example, after analyzing the historical data, the time is set to 3 pm every day, then the slow query distributed system will start to obtain the first slow query data at 3 pm every day.

[0059] As another method, when the slow query distributed system obtains the first slow query data in real time, each time the application system has new first slow query data determined, it sends a data acquisition signal to the slow query distributed system. When the slow query distributed system receives the data acquisition signal, it can obtain the first slow query data determined by the application system in real time.

[0060] Step S220: acquiring second slow query data satisfying a preset condition from the slow query data included in the database according to the slow query configuration parameters corresponding to the application system.

[0061] Step S230: sending the second slow query data to a target device corresponding to the application system, where the target device is a device used by an operation and maintenance personnel of the application system.

[0062] A data query method provided by an embodiment of the present application, a slow query distributed system can directly find slow query data that meets preset conditions according to the slow query configuration parameters corresponding to the application system, and send the slow query data that meets the preset conditions to a target device corresponding to the application system, so that the slow query distributed system can monitor the slow query data generated by the application system and timely obtain the slow query data of the preset conditions, and send the slow query data that meets the preset conditions to the target device used by the operation and maintenance personnel of the application system, so that the operation and maintenance personnel can subsequently timely obtain the slow query data that needs to be optimized based on the slow query data sent by the slow query distributed system, and then timely optimize the slow query data, thereby ultimately achieving the purpose of improving the processing efficiency of the database of the application system.

[0063] See also Figure 6 , a data query method provided in an embodiment of the present application is applied to a slow query distributed system, the method comprising:

[0064] Step S310: Obtain the slow query configuration parameters.

[0065] Step S320: Send the slow query configuration parameters to the application system, so that the application system determines the first slow query data based on the slow query configuration parameters.

[0066] As a way, the modified slow query configuration parameters corresponding to the application system are obtained, and the modified slow query configuration parameters corresponding to the application system are sent to the application system.

[0067] In the embodiment of the present application, the operation and maintenance personnel of the application system can log in to the slow query distributed system and modify the slow query configuration parameters corresponding to the application system. When the operation and maintenance personnel of the slow query distributed system review and approve, the slow query distributed system can actively push the modified slow query configuration parameters corresponding to the application system to the application system, so that the application system can obtain and use the modified slow query configuration parameters without restarting. Figure 7 As shown, the slow query distributed system can obtain the access or configuration modification application of the operation and maintenance personnel of the application system, and then the operation and maintenance personnel of the slow query distributed system approves the access or configuration modification application. If the approval is passed, the slow query distributed system can remotely access the application system and push the modified slow query configuration parameters to the application system.

[0068] Among them, when the operation and maintenance personnel of the application system set the slow query configuration parameters unreasonably, or when the operation and maintenance personnel of the application system change, the operation and maintenance personnel of the application system can log in to the slow query distributed system and modify the slow query configuration parameters corresponding to the application system. Through the above method, the slow query configuration parameters can be turned on, off or modified. Whenever there is a change in the slow query configuration parameters, the slow query distributed system can push the latest slow query configuration parameters to the application system without restarting the application system.

[0069] Step S330: acquiring the first slow query data determined by the application system from a pre-subscribed specific subject queue, wherein the specific subject queue is a queue in a message queue middleware.

[0070] Step S340: Arrange the third slow query data in the database in descending order according to the execution time, the application system identifier corresponding to the third slow query data is the target identifier, the operation statement type corresponding to the third slow query data is the target operation statement type, and the slow query data is an operation statement whose execution time is greater than or equal to the preset time.

[0071] The third slow query data is slow query data stored in the database, belonging to the same application system and having the same corresponding operation statement type.

[0072] The slow query configuration parameters may include the target identifier of the application system and the target operation statement type. Exemplarily, the slow query configuration parameters may be as shown in Table 1:

[0073] Table 1

[0074]

[0075]

[0076] As shown in the table above, slow query configuration parameters may include the target identifier of the application system, the name of the application system, whether to enable slow query configuration, Top-n configuration (time period start time, time period end time, sql type, Top-n value), contact information, configuration parameters (time period start time, time period end time, sql type, preset duration), etc.

[0077] In the embodiment of the present application, different application systems have different slow query configuration parameters. The differences here are reflected in that different application systems, the same sql type, have different preset durations set in different time periods; the same application system, the same sql type, have different preset durations set in different time periods; different application systems, different sql types, have different preset durations set in different time periods.

[0078] For example, assuming that the ID of a third-party application system is "A" and its Chinese name is "A Management System", the slow query configuration is as follows:

[0079] Preset duration configuration:

[0080] 8:00-18:00: Set the preset duration of insert type SQL to 500 milliseconds, the preset duration of update type SQL to 600 milliseconds, the preset duration of select type SQL to 400 milliseconds, and the preset duration of delete type SQL to 100 milliseconds.

[0081] Other time ranges: Regardless of SQL type (i.e. SQL type is all), the default duration is set to 800 milliseconds.

[0082] For example, different application systems, different SQL types, and preset durations set in different time periods are shown in Table 2:

[0083] Table 2

[0084]

[0085] Optionally, when the slow query distributed system stores the slow query data of the application system, it may be stored in the structure of the above slow query configuration parameters. Exemplarily, the data structure of the slow query data may be as shown in Table 3:

[0086] Table 3

[0087]

[0088]

[0089] The data structure of the slow query data shown above can be used to know the execution time of each slow query data, the corresponding SQL type and the corresponding application system, so that the third slow query data can be obtained from the slow query data included in the database of the slow query distributed system according to the identification of the application system and the type of the slow query data, and then the third slow query data can be arranged in descending order according to the corresponding execution time to obtain the third slow query data in descending order.

[0090] Step S350: acquiring the third slow query data at a specified position from the third slow query data arranged in descending order, to obtain the second slow query data.

[0091] In the embodiment of the present application, the third slow query data at the specified position is the preset Top-n slow query data. For example, the Top-n slow query data is the top 5 slow query data, that is, after each calculation of the Top-n, the 5 third slow query data with the longest SQL execution time will be retained, and these 5 third slow query data will be determined as the second slow query data that meets the preset conditions.

[0092] Each time the second slow query data is determined, the second slow query data determined each time may be stored. When the second slow query data is stored, it may be stored according to the data structure shown in Table 4:

[0093] Table 4

[0094]

[0095]

[0096] As an approach, specific steps of step S350 may include: if the second slow query data acquired this time is different from the second slow query data acquired last time, sending the second slow query data acquired this time to a target device corresponding to the application system.

[0097] If the second slow query data obtained this time is different from the second slow query data obtained last time, the second slow query data obtained this time can be sent to the target device corresponding to the application system according to the application system identifier, time period and SQL type, so as to achieve the effect of timely warning. Figure 8As shown, the slow query distributed system subscribes to a specific topic queue of the message queue middleware to obtain the slow query data of the application system. The slow query distributed system can periodically calculate whether the Top-n slow query data of each type has changed. If there is a change, the current Top-n slow query data will be notified to the operation and maintenance personnel of the application system via email or SMS. Through the above method, when the Top-n slow query data of the application system changes, the operation and maintenance personnel of the application system can be alerted in time.

[0098] Step S360: Send the second slow query data to the target device corresponding to the application system via email or text message.

[0099] In an embodiment of the present application, the slow query distributed system can send the second slow query data to the mailbox of the operation and maintenance personnel of the application system via email, or send the second slow query data to the mobile phone reserved by the operation and maintenance personnel of the application system via text message, or other methods can be used as long as the alarm information can be notified to the relevant responsible persons of the application system smoothly and promptly.

[0100] The embodiment of the present application provides a data query method, which can directly obtain the corresponding slow query data that meets the preset conditions after arranging the slow query data in descending order according to the execution time of the slow query data, so that the subsequent slow query distributed system can send the slow query data to the operation and maintenance personnel of the application system in time for maintenance. Furthermore, the Top-n slow query data obtained this time can be compared with the Top-n slow query obtained last time, and then when the Top-n slow query data of the application system changes, the operation and maintenance personnel of the application system can be alerted in time, so that the operation and maintenance personnel of the application system can optimize the slow query data in time.

[0101] See also Fig. 9 , a data query method provided in an embodiment of the present application is applied to an application system, and the method includes:

[0102] Step S410: In response to an operation on a database of the application system, an operation parameter of an operation statement corresponding to the operation is obtained.

[0103] Step S420: If it is determined that the operation statement is first slow query data based on the operation parameters and the slow query configuration parameters corresponding to the application system, the first slow query data is sent to the slow query distributed system.

[0104] In an embodiment of the present application, a slow query verification is performed on the operation statement according to the execution time of the operation statement, the type of the operation statement, and the slow query configuration parameters of the application system. If the operation statement passes the slow query verification, the operation statement is determined to be the first slow query data.

[0105] The first slow query data is used for:

[0106] The slow query distributed system is triggered to store the first slow query data in a database of the slow query distributed system; according to the slow query configuration parameters corresponding to the application system, second slow query data that meets preset conditions is obtained from the slow query data included in the database, and the second slow query data is sent to a target device corresponding to the application system, where the target device is a device used by operation and maintenance personnel of the application system.

[0107] In an embodiment of the present application, the application system may send the first slow query data to the slow query distributed system by calling an http interface provided by the slow query distributed system.

[0108] The data query method provided by the embodiment of the present application does not require the configuration of a slow query log for the database of the application system, and does not invade the database of the application system. The slow query distributed system can directly find the slow query data that meets the preset conditions according to the slow query configuration parameters corresponding to the application system, and send the slow query data that meets the preset conditions to the operation and maintenance personnel of the application system, so that the operation and maintenance personnel of the application system can monitor the slow query data generated by the application system, and at the same time, the operation and maintenance personnel of the application system can locate the slow query data of the application system in time, so that the slow query configuration parameters can be modified based on the slow query data of the application system in the future, so as to improve the processing efficiency of the database of the application system.

[0109] See also Fig.10 , a data query method provided in an embodiment of the present application is applied to an application system, and the method includes:

[0110] Step S510: Load the client jar package of the slow query distributed system and read the basic configuration.

[0111] In the embodiment of the present application, the basic configuration may include the address of the message queue middleware, the specific topic pair of publish / subscribe, the address of the slow query distributed system, etc. The application system only needs to perform simple configuration, such as the address of the message queue middleware, the specific topic pair of publish / subscribe, the address of the slow query distributed system, etc.

[0112] When the client jar package of the slow query distributed system is detected to be upgraded, the upgrade message of the client jar package of the slow query distributed system can be sent to the operation and maintenance personnel of the application system, who will decide whether to upgrade and when to upgrade. The new version of the client jar package of the slow query distributed system is fully compatible with the old version. When the client jar package of the slow query distributed system is upgraded, the latest client jar package of the slow query distributed system can be pushed to the Maven central library, and an email can be sent to notify the operation and maintenance personnel of the application system of the upgraded version of the client jar package of the slow query distributed system and the new and modified contents of the new version. The operation and maintenance personnel of the application system will decide whether to upgrade and when to upgrade.

[0113] If the application system is a Maven project, you only need to modify the version of the slow query distributed system client jar package in the pom.xml (or build.gradle) file (generally, current application systems are Maven or Gradle projects). If not, you need to download the latest version of the jar package from the Maven central library and manually add it to the application system. Finally, redeploy the application system containing the new version of the client jar package to the production environment to complete the upgrade of the application system's slow query distributed system client jar package.

[0114] Step S520: Based on the basic configuration and the client jar package, access the slow query distributed system to obtain slow query configuration parameters corresponding to the application system.

[0115] In the embodiment of the present application, the application system can remotely access the slow query distributed system through the interface provided in the client jar package and the address of the slow query distributed system to obtain the slow query configuration parameters corresponding to the application system. Fig.11 As shown, when the application system starts, the application system can load the slow query distributed system client jar, read the slow query basic configuration, obtain the slow query distributed system domain name, and then remotely request the slow query distributed system, obtain the slow query configuration parameters, and store the slow query configuration parameters in the local jvm memory of the application system.

[0116] Step S530: In response to the operation on the database of the application system, an operation parameter of the operation statement corresponding to the operation is obtained.

[0117] In the embodiment of the present application, the operation parameters of the operation statement include an execution time period, an operation statement type, and an execution duration.

[0118] Step S540: Based on the execution time period of the operation statement and the time period configured in each configuration parameter, perform time period verification on the multiple configuration parameters; and use the configuration parameter that passes the time period verification among the multiple configuration parameters as the first candidate configuration parameter.

[0119] The slow query configuration parameters include multiple configuration parameters, each of which includes a configured time period, a configured operation statement type, and a preset duration.

[0120] Among them, when the start time of the execution period of the operation statement is later than the start time of the time period configured in the configuration parameters, and the end time of the execution period of the operation statement is earlier than the end time of the time period configured in the configuration parameters, it means that the configuration parameters pass the time period verification.

[0121] In an embodiment of the present application, a configuration parameter set in the slow query configuration parameter is traversed, and a time period verification is performed on each configuration parameter based on the execution time period of the operation statement and the time period configured in each configuration parameter. If the start time of the execution time period of the operation statement is later than the start time of the time period configured in a certain configuration parameter, and the end time of the execution time period of the operation statement is earlier than the end time of the time period configured in the configuration parameter, the configuration parameter passes the time period verification, and the configuration parameter is added to the first candidate configuration parameter for subsequent operation statement type verification.

[0122] Step S550: Based on the operation statement type of the operation statement and the operation statement type configured in the first candidate configuration parameter, perform operation statement type verification on the first candidate configuration parameter; and use the configuration parameter in the first candidate configuration parameter that passes the operation statement type verification as the second candidate configuration parameter.

[0123] Among them, if the operation statement type of the operation statement is the same as the operation statement type configured in the first candidate configuration parameter, it means that the first candidate configuration parameter passes the operation statement type verification; or, if the operation statement type of the operation statement is different from the operation statement types configured in all the first candidate configuration parameters, and the operation statement type configured in the first candidate configuration parameter is a full statement type, it means that the first candidate configuration parameter passes the operation statement type verification.

[0124] In an embodiment of the present application, the operation statement type of the operation statement may include four types: insert, update, delete, and select. The first candidate configuration parameter is traversed to determine whether the operation statement type of the operation statement is consistent with the operation statement type configured in the first candidate configuration parameter. If the operation statement type configured in the first candidate configuration parameter is consistent with the operation statement type of the operation statement, the preset duration configured in the first candidate configuration parameter is used to verify the duration of the operation statement; if the operation statement type configured in the first candidate configuration parameter is not consistent with the operation statement type of the operation statement, and there is a first candidate configuration parameter of the full statement type in the operation statement type configured in the first candidate configuration parameter, the preset duration configured in the first candidate configuration parameter is used to verify the duration of the operation statement.

[0125] Step S560: If the execution time of the operation statement is greater than or equal to the preset time configured in the second candidate configuration parameter, it is determined that the operation statement is the first slow query data.

[0126] In an embodiment of the present application, if the execution duration of the operation statement is greater than or equal to the preset duration configured in the configuration parameters for determining the duration verification, the operation statement is determined to be the first slow query data, and the first slow query data needs to be sent to a specific topic pair column of the message queue.

[0127] In the embodiment of the present application, the implementation process of the method corresponding to step S540-step S560 can be as follows: Fig.12 When the application system obtains the execution time period of the SQL statement, it will initiate asynchronous slow query verification. First, the time period verification will be performed. The specific implementation process of the time period verification can be as shown in step S540. If the time period verification passes, the SQL type judgment will continue. The specific implementation process of the SQL type judgment can be as shown in step S550. When the SQL type verification passes, the duration verification is performed. If the SQL statement passes the duration verification, it is determined that the SQL statement is the second slow query data, and the SQL statement is sent to the message queue middleware.

[0128] Step S570: Send the first slow query data to the slow query distributed system.

[0129] The data query method provided by the embodiment of the present application only needs basic configuration for the application system, does not need to configure a slow query log for the database of the application system, and does not infringe on the database of the application system. As long as an operation on the database of the application system is detected, the operation parameters of the corresponding operation statement can be obtained, so that it can be quickly determined whether the operation statement is slow query data according to the corresponding operation parameters.

[0130] See also Fig.13 , a data query device 600 provided in an embodiment of the present application runs in a slow query distributed system, and the data query device 600 includes:

[0131] The first data acquisition unit 610 is configured to acquire first slow query data determined by the application system, and store the first slow query data in a database of the slow query distributed system.

[0132] The second data acquisition unit 620 is used to acquire second slow query data that meets a preset condition from the slow query data included in the database of the slow query distributed system according to the slow query configuration parameters corresponding to the application system.

[0133] As a method, the slow query configuration parameters include the target identifier of the application system and the target operation statement type. The second data acquisition unit 620 is used to arrange the third slow query data in the database in descending order according to the execution time, the application system identifier corresponding to the third slow query data is the target identifier, the operation statement type corresponding to the third slow query data is the target operation statement type, and the slow query data is an operation statement with an execution time greater than or equal to a preset time; from the third slow query data arranged in descending order, the third slow query data at a specified position is obtained to obtain the second slow query data.

[0134] Optionally, the second data acquisition unit 620 is configured to send the second slow query data acquired this time to a target device corresponding to the application system if the second slow query data acquired this time is different from the second slow query data acquired last time.

[0135] The data sending unit 630 is used to send the second slow query data to a target device corresponding to the application system, where the target device is a device used by an operation and maintenance personnel of the application system.

[0136] See also Fig.14 , a data query device 700 provided in an embodiment of the present application runs in an application system, and the data query device 700 includes:

[0137] The parameter acquisition unit 710 is used to obtain, in response to an operation on a database of the application system, an operation parameter of an operation statement corresponding to the operation.

[0138] The determination unit 720 is used for sending the first slow query data to a slow query distributed system if it is determined that the operation statement is the first slow query data based on the operation parameters and the slow query configuration parameters corresponding to the application system; wherein the first slow query data is used to: trigger the slow query distributed system to store the first slow query data in a database of the slow query distributed system; obtain second slow query data that meets preset conditions from the slow query data included in the database according to the slow query configuration parameters corresponding to the application system, and send the second slow query data to a target device corresponding to the application system, where the target device is a device used by operation and maintenance personnel of the application system.

[0139] As a method, the slow query configuration parameters include multiple configuration parameters, each of which includes a configured time period, a configured operation statement type, and a preset duration. The determination unit 720 is used to perform time period verification on the multiple configuration parameters based on the execution time period of the operation statement and the time period configured in each configuration parameter; use the configuration parameter that passes the time period verification among the multiple configuration parameters as the first candidate configuration parameter; perform operation statement type verification on the first candidate configuration parameter based on the operation statement type of the operation statement and the operation statement type configured in the first candidate configuration parameter; use the configuration parameter that passes the operation statement type verification among the first candidate configuration parameter as the second candidate configuration parameter; if the execution duration of the operation statement is greater than or equal to the preset duration configured in the second candidate configuration parameter, determine that the operation statement is the first slow query data.

[0140] Among them, when the start time of the execution period of the operation statement is later than the start time of the time period configured in the configuration parameters, and the end time of the execution period of the operation statement is earlier than the end time of the time period configured in the configuration parameters, it means that the configuration parameters pass the time period verification.

[0141] When the operation statement type of the operation statement is the same as the operation statement type configured in the first candidate configuration parameter, it indicates that the first candidate configuration parameter passes the operation statement type verification; or, when the operation statement type of the operation statement is different from the operation statement types configured in all the first candidate configuration parameters, and the operation statement type configured in the first candidate configuration parameter is a full statement type, it indicates that the first candidate configuration parameter passes the operation statement type verification.

[0142] See also Fig.15 , an embodiment of the present application provides a data query system 800, the data query system 800 includes an application system 810 and a slow query distributed system 820:

[0143] The application system 810 is used to obtain operation parameters of an operation statement corresponding to the operation in response to an operation on a database of the application system 810.

[0144] The application system 810 is configured to send the first slow query data to the slow query distributed system 820 if it is determined that the operation statement is first slow query data based on the operation parameter and the slow query configuration parameter corresponding to the application system 810 .

[0145] The slow query distributed system 820 is used to obtain the first slow query data determined by the application system 810, and store the first slow query data in a database of the slow query distributed system 820;

[0146] The slow query distributed system 820 is used to obtain second slow query data that meets a preset condition from the slow query data included in the database of the slow query distributed system 820 according to the slow query configuration parameters corresponding to the application system 810 .

[0147] The slow query distributed system 820 is used to send the second slow query data to a target device corresponding to the application system 810, where the target device is a device used by an operation and maintenance personnel of the application system.

[0148] It should be noted that the device embodiment in the present application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.

[0149] The following will be combined Fig.16 A server provided by this application is described.

[0150] See also Fig.16 Based on the above data query method and device, the embodiment of the present application also provides another server 900 that can execute the above data query method. The server 900 includes one or more (only one is shown in the figure) processors 902, a memory 904 and a network module 906 that are coupled to each other. Among them, the memory 904 stores a program that can execute the content of the above embodiment, and the processor 902 can execute the program stored in the memory 904.

[0151] Among them, the processor 902 may include one or more processing cores. The processor 902 uses various interfaces and lines to connect various parts within the entire server 900, and executes various functions and processes data of the server 900 by running or executing instructions, programs, code sets or instruction sets stored in the memory 904, and calling data stored in the memory 904. Optionally, the processor 902 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 902 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 902, and may be implemented separately through a communication chip.

[0152] The memory 904 may include a random access memory (RAM) or a read-only memory (ROM). The memory 904 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 904 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the server 900 during use.

[0153] The network module 906 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, such as communicating with an audio playback device. The network module 906 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM) cards, memories, etc. The network module 906 may communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices via wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. For example, the network module 906 may exchange information with a base station.

[0154] Please refer to Fig.17 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 1100 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0155] The computer readable storage medium 1100 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 1100 includes a non-transitory computer-readable storage medium. The computer readable storage medium 1100 has storage space for program code 1110 that executes any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 1110 can be compressed, for example, in an appropriate form.

[0156] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A data query method, characterized in that: Applied to a slow query distributed system, the method includes: Acquire first slow query data determined by the application system, and store the first slow query data in a database of the slow query distributed system, wherein the first slow query data is an operation statement corresponding to an operation on the database of the application system, and the execution time of the operation statement is greater than or equal to a preset time, wherein before judging that the execution time of the operation statement is greater than or equal to the preset time, it also includes: sequentially performing time period verification and operation statement type verification on the operation statement, wherein the operation statement type verification and time period verification are asynchronous verification; According to the slow query configuration parameters corresponding to the application system, second slow query data satisfying a preset condition is acquired from the slow query data included in the database; The second slow query data is sent to a target device corresponding to the application system, where the target device is a device used by an operation and maintenance personnel of the application system.

2. The method according to claim 1, characterized in that The slow query configuration parameters include a target identifier of the application system and a target operation statement type; and obtaining second slow query data that meets a preset condition from the slow query data included in the database according to the slow query configuration parameters corresponding to the application system includes: Arrange the third slow query data in the database in descending order according to the execution time, the application system identifier corresponding to the third slow query data is the target identifier, the operation statement type corresponding to the third slow query data is the target operation statement type, and the slow query data is an operation statement with an execution time greater than or equal to a preset time; The third slow query data at a specified position is obtained from the third slow query data arranged in descending order to obtain the second slow query data.

3. The method according to claim 2, characterized in that The sending the second slow query data to a target device corresponding to the application system includes: If the second slow query data obtained this time is different from the second slow query data obtained last time, the second slow query data obtained this time is sent to the target device corresponding to the application system.

4. A data query method, characterized in that: Applied to an application system, the method comprises: In response to an operation on a database of the application system, obtaining an operation parameter of an operation statement corresponding to the operation; If it is determined that the operation statement is the first slow query data based on the operation parameter and the slow query configuration parameter corresponding to the application system, the first slow query data is sent to the slow query distributed system, and the first slow query data is an operation statement whose execution time of the operation statement is greater than or equal to the preset time, wherein before judging that the execution time of the operation statement is greater than or equal to the preset time, it also includes: sequentially performing time period verification and operation statement type verification on the operation statement, and the operation statement type verification and time period verification are asynchronous verification; The first slow query data is used for: The slow query distributed system is triggered to store the first slow query data in a database of the slow query distributed system; according to the slow query configuration parameters corresponding to the application system, second slow query data that meets preset conditions is obtained from the slow query data included in the database, and the second slow query data is sent to a target device corresponding to the application system, where the target device is a device used by operation and maintenance personnel of the application system.

5. The method according to claim 4, characterized in that The slow query configuration parameters include multiple configuration parameters, each of which includes a configured time period, a configured operation statement type, and a preset duration; the operation parameters of the operation statement include an execution time period, an operation statement type, and an execution duration; Before sending the first slow query data to the slow query distributed system, the method further includes: Based on the execution time period of the operation statement and the time period configured in each configuration parameter, the multiple configuration parameters are subject to time period verification; and the configuration parameter that passes the time period verification among the multiple configuration parameters is used as the first candidate configuration parameter; Based on the operation statement type of the operation statement and the operation statement type configured in the first candidate configuration parameter, perform operation statement type verification on the first candidate configuration parameter; and use the configuration parameter in the first candidate configuration parameter that passes the operation statement type verification as the second candidate configuration parameter; If the execution time of the operation statement is greater than or equal to the preset time configured in the second candidate configuration parameter, it is determined that the operation statement is the first slow query data.

6. The method according to claim 5, characterized in that When the start time of the execution period of the operation statement is later than the start time of the time period configured in the configuration parameters, and the end time of the execution period of the operation statement is earlier than the end time of the time period configured in the configuration parameters, it indicates that the configuration parameters pass the time period verification.

7. The method according to claim 5, characterized in that If the operation statement type of the operation statement is the same as the operation statement type configured in the first candidate configuration parameter, it indicates that the first candidate configuration parameter passes the operation statement type verification; Alternatively, when the operation statement type of the operation statement is different from the operation statement types configured in all the first candidate configuration parameters, and the operation statement type configured in the first candidate configuration parameter is a full statement type, it means that the first candidate configuration parameter passes the operation statement type verification.

8. A data query device, characterized in that: Running in a slow query distributed system, the device includes: A first data acquisition unit is used to acquire first slow query data determined by the application system, and store the first slow query data in a database of the slow query distributed system, wherein the first slow query data is an operation statement whose execution time is greater than or equal to a preset time, wherein before judging that the execution time of the operation statement is greater than or equal to the preset time, the method further includes: sequentially performing time period verification and operation statement type verification on the operation statement, wherein the operation statement type verification and time period verification are asynchronous verifications; A second data acquisition unit, configured to acquire second slow query data satisfying a preset condition from the slow query data included in the database according to the slow query configuration parameters corresponding to the application system; A data sending unit is used to send the second slow query data to a target device corresponding to the application system, where the target device is a device used by an operation and maintenance personnel of the application system.

9. A server, characterized in that: The method comprises one or more processors; one or more programs are stored in a memory and configured to be executed by the one or more processors to perform any one of the methods of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images