A query log processing method, device, equipment and medium

CN120743950BActive Publication Date: 2026-09-22BEIJING YOUTEJIE INFORMATION TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510844944.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-06-23
Publication Date
2026-09-22
Estimated Expiration
2045-06-23

AI Technical Summary

Technical Problem

[0004]本发明提供了一种查询日志处理方法、装置、设备及介质,以解决相关技术中的查询日志处理方案依赖于人工操作,时间成本和人力成本较高,而且准确性难以保证的问题

Benefits of technology

[0018]根据本发明的另一方面,提供了一种计算机程序产品,所述计算机程序产品包括计算机程序,所述计算机程序在被处理器执行时实现本发明任一实施例所述的查询日志处理方法。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120743950B_ABST
    Figure CN120743950B_ABST
Patent Text Reader

Abstract

The application discloses a kind of query log processing method, device, equipment and medium.Therein, method includes: by log acquisition component, each query log is collected from query log file, and each query log is stored to first storage component;Timely from each to-be-processed query log in first storage component acquisition, according to the key query information of each to-be-processed query log, each to-be-processed query log is sorted, according to the time stamp of adjacent query log, the time consumption of each to-be-processed query log is determined, and is stored to second storage component;Key query information includes query time stamp, query thread information, query resource information and query address information;When obtaining time consumption statistics request, determine the statistical result corresponding to time consumption statistics request.The application embodiment can be automatically based on the time stamp of adjacent query log, quickly and accurately process query log, determine the time consumption of query log.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a query log processing method, apparatus, device, and medium. Background Technology

[0002] Business data generated in an enterprise's business systems is typically stored in a database hosted on a data storage device. The data storage device can retrieve specific business data from the database by executing a query statement entered by a business user, and then provide that data to the user. Each time the data storage device executes a query operation, it generates a query log to record the query process. To effectively monitor these data query operations, the generated query logs need to be processed to determine their execution time. The query log execution time refers to the duration it takes for the data storage device to execute the data query operation described in the log.

[0003] In related technologies, a common query log processing solution involves technicians processing query logs generated in data storage devices based on statistical data and system resource usage to determine the time consumed by each query. However, this solution relies on manual operation, resulting in high time and labor costs, and its accuracy is difficult to guarantee. Summary of the Invention

[0004] This invention provides a query log processing method, apparatus, device, and medium to solve the problems in related technologies where query log processing schemes rely on manual operation, resulting in high time and labor costs, and difficulty in guaranteeing accuracy.

[0005] According to one aspect of the present invention, a query log processing method is provided, comprising:

[0006] The log collection component collects each query log from the query log file and stores each query log in the first storage component;

[0007] Each pending query log is periodically retrieved from the first storage component. The pending query logs are sorted according to their key query information. The time consumed by each pending query log is determined based on the query timestamps of adjacent query logs. Each pending query log, its key query information, and its time consumed are then stored in the second storage component. The key query information includes query timestamps, query thread information, query resource information, and query address information.

[0008] When a time consumption statistics request is received, the statistical results corresponding to the time consumption statistics request are determined based on the various query logs stored in the second storage component, the key query information of each query log, and the time consumption.

[0009] According to another aspect of the present invention, a query log processing apparatus is provided, comprising:

[0010] The log collection module is used to collect various query logs from the query log file through the log collection component and store each query log in the first storage component;

[0011] The time consumption determination module is used to periodically retrieve each pending query log from the first storage component, sort each pending query log according to the key query information of each pending query log, determine the time consumption of each pending query log according to the query timestamps of adjacent query logs, and store each pending query log, the key query information of each pending query log, and the time consumption in the second storage component accordingly; wherein, the key query information includes query timestamp, query thread information, query resource information, and query address information;

[0012] The time consumption statistics module is used to determine the statistical results corresponding to the time consumption statistics request based on the various query logs stored in the second storage component, the key query information of each query log, and the time consumption when a time consumption statistics request is obtained.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor;

[0015] and a memory communicatively connected to the at least one processor;

[0016] The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the query log processing method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the query log processing method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the query log processing method described in any embodiment of the present invention.

[0019] The technical solution of this invention involves collecting various query logs from a query log file using a log collection component and storing each query log in a first storage component. Periodically, it retrieves each pending query log from the first storage component, sorts the pending query logs based on their key query information, determines the time consumed by each pending query log based on the query timestamps of adjacent query logs, and stores each pending query log, its key query information, and its time consumed in a second storage component. The key query information includes query timestamps, query thread information, query resource information, and query address information. When a time consumption statistics request is received, it calculates the time consumed based on the query logs and query timestamps stored in the second storage component. This method automatically sorts query logs based on key query information and time consumption, and quickly and accurately determines the time consumption statistics request. It solves the problems of traditional query log processing solutions relying on manual operation, resulting in high time and labor costs, and difficulty in guaranteeing accuracy. It can automatically sort query logs based on key query information and quickly and accurately determine the time consumption of query logs based on the query timestamps of adjacent query logs. It can then perform statistical analysis on the determined time-consuming query logs to obtain corresponding results. This achieves automatic processing of query logs generated in data storage devices based on the query timestamps of adjacent query logs, determining the time consumption of query logs, reducing the time and labor costs of query log processing, and improving the accuracy of query log processing.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a query log processing method provided in Embodiment 1 of the present invention.

[0023] Figure 2 This is a flowchart of a query log processing method provided in Embodiment 2 of the present invention.

[0024] Figure 3 This is a schematic diagram of the structure of a query log processing device provided in Embodiment 3 of the present invention.

[0025] Figure 4 A schematic diagram of the structure of an electronic device for implementing the query log processing method of this embodiment of the invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "target," "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising," "including," and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] Example 1

[0029] Figure 1 This is a flowchart illustrating a query log processing method according to Embodiment 1 of the present invention. This embodiment is applicable to processing query logs generated in a data storage device and determining the time consumed by the query logs. This method can be executed by a query log processing device, which can be implemented in hardware and / or software and can be configured in an electronic device installed within an enterprise. The electronic device can be an electronic device installed within the enterprise for processing query logs generated in the data storage device. For example... Figure 1 As shown, the method includes:

[0030] Step 101: Collect each query log from the query log file using the log collection component, and store each query log in the first storage component.

[0031] Optionally, electronic devices can connect to data storage devices within the enterprise, enabling information exchange. Data storage devices are electronic devices within the enterprise used to store business data. These devices contain a database. Business data generated by the enterprise's business systems is stored in the database within the data storage device. The enterprise's business systems can be servers responsible for processing the enterprise's business. Business data can be data related to the enterprise's business. Data query statements are code statements that, once executed, are used to retrieve specific business data from the database within the data storage device. The data storage device can retrieve specific business data from the database by executing data query statements and then provide this data to users who need it. When needing to retrieve specific business data from the database, users can input data query statements to do so. The data storage device can retrieve specific business data from the database by executing data query statements and then provide this data to users. Users can be business personnel within the enterprise.

[0032] Optionally, the data storage device generates a query log to record the data query operation each time it executes a user-inputted data query statement that retrieves the specified business data from the database and then provides that data to the user. A data query operation is defined as the process of retrieving the specified business data from the database and providing that data to the user by executing the user-inputted data query statement. The query log can contain information describing the data query operation. The query log's execution time refers to the duration during which the data storage device executes the data query operation described in the log.

[0033] Optionally, for each data query operation, the query log can consist of parameter names and values ​​of multiple parameters related to the data query operation recorded by the data storage device. These parameters include, but are not limited to: query start time, user and host information, data query statement information, execution thread information, source information, Internet Protocol (IP) address, query duration, query lock wait time, number of rows returned by the query, and number of rows scanned by the query. The query start time can be the time when the data storage device begins executing the data query operation. User and host information can be the identification information of the user instructing the data storage device to execute the data query operation and the identification information of the host used by the user. The data query statement information can be the data query statement executed by the data storage device during the execution of the data query operation. The execution thread information can be the numerical ID of the thread in the data storage device used to execute the data query operation. The source information can be information identifying the source of the data query statement executed by the data storage device during the execution of the data query operation. The IP address can be the IP address of the host used by the user instructing the data storage device to execute the data query operation. The query duration can be the duration for the data storage device to execute the data query statement during the execution of the data query operation. The query lock wait time can be the duration during which the data storage device waits while executing a data query statement. The number of rows returned by the query can be the number of rows of business data retrieved by the data storage device during the data query operation. The number of rows scanned by the query can be the number of rows of business data scanned by the data storage device during the data query operation.

[0034] Optionally, the data storage device includes a query log file. The query log file can be a file used to store query logs for all data query operations. Each time the data storage device executes a data query operation—which retrieves specified business data from the database by running a user-inputted query statement and then provides that data to the user—it records the parameter values ​​of multiple parameters related to the data query operation. The parameter names and values ​​are arranged in a preset order to form a sequence. Delimiters are inserted between the parameter names and values ​​in this sequence. This sequence, with delimiters inserted, is considered the query log for the data query operation and is stored in the query log file. The delimiter can be a symbol used to separate different information. For example, the delimiter can be a space or a tab. The preset order can be a pre-defined order in which the parameter names and values ​​are arranged.

[0035] Optionally, the data storage device can generate query logs in either a general query log mode or a slow query log mode.

[0036] Optionally, the general query log mode can refer to the data storage device recording the query start time, user and host information, data query statement information, execution thread information, source information, and IP address parameter values ​​related to the data query operation each time it executes a data query operation that retrieves the specified business data from the database and then provides the specified business data to the user. The parameter names and values ​​of the query start time, user and host information, data query statement information, execution thread information, source information, and IP address are arranged in a preset order to obtain a sequence. Separators are inserted between the parameter names and values ​​of the query start time, user and host information, data query statement information, execution thread information, source information, and IP address in the sequence. The sequence after inserting the separators is determined as the query log of the data query operation and stored in the query log file.

[0037] Optionally, the slow query log mode refers to the data storage device recording the following parameters related to the data query operation each time it executes a user-inputted data query statement that retrieves the specified business data from the database and then provides the data to the user: query start time, user and host information, data query statement information, execution thread information, source information, IP address, query time, query lock wait time, number of rows returned, and number of rows scanned. These parameters are arranged in a preset order to form a sequence. Separators are inserted between the parameters in this sequence, and the sequence after the separators are used to define the query log of the data query operation and store it in a query log file.

[0038] Optionally, the data storage device is configured with pattern indication information. This pattern indication information can be used to instruct the data storage device to generate query logs according to either a general query log pattern or a slow query log pattern. The pattern indication information can be set by the target user in the data storage device. The target user can be a technical personnel responsible for managing the data storage device. The target user sets the pattern indication information in the data storage device and can update the pattern indication information according to business needs. The pattern indication information is either general query or slow query. If the pattern indication information is general query, it indicates that the data storage device needs to generate query logs according to the general query log pattern. If the pattern indication information is general query, it indicates that the data storage device needs to generate query logs according to the slow query log pattern.

[0039] Optionally, when the pattern indication information in the data storage device is a general query, the data storage device generates a query log according to the general query log pattern. When the pattern indication information in the data storage device is a slow query, the data storage device generates a query log according to the slow query log pattern.

[0040] Optionally, for each data query operation, the query log should at least include the parameter names and values ​​for query start time, user and host information, data query statement, execution thread, source information, and IP address. The query log may also include the parameter names and values ​​for query duration, query lock wait time, number of rows returned, and number of rows scanned.

[0041] Optionally, the parameter name for the query start time in the query log of a data query operation can be a pre-set string used to identify the query start time. For example, the parameter name for the query start time is "timestamp". The parameter value for the query start time in the query log of a data query operation can be the time recorded by the data storage device when the data query operation began.

[0042] Optionally, the parameter names for user and host information in the query log of a data query operation can be pre-set strings used to identify user and host information. For example, the parameter name for user and host information is "user_host". The parameter values ​​for user and host information in the query log of a data query operation can be the identification information of the user who instructs the data storage device to perform the data query operation and the identification information of the host used by the user, recorded by the data storage device.

[0043] Optionally, the parameter name of the data query statement information in the query log of the data query operation can be a pre-set string used to identify the data query statement information. For example, the parameter name of the data query statement information is "sql_text". The parameter value of the data query statement information in the query log of the data query operation can be the data query statement recorded by the data storage device during the execution of the data query operation.

[0044] Optionally, the parameter name for the execution thread information in the query log of a data query operation can be a pre-set string used to identify the execution thread. For example, the parameter name for the execution thread information is "thread_id". The parameter value for the execution thread information in the query log of a data query operation can be a numerical ID recorded by the data storage device as the thread used to perform the data query operation.

[0045] Optionally, the parameter name for the source information in the query log of a data query operation can be a pre-set string used to identify the source information. For example, the parameter name for the source information is "source". The parameter value for the source information in the query log of a data query operation can be information recorded by the data storage device to identify the source of the data query statement executed by the data storage device during the data query operation.

[0046] Optionally, the parameter name for the IP address in the query log of a data query operation can be a pre-set string used to identify the IP address. For example, the parameter name for the IP address is "ip". The parameter value for the IP address in the query log of a data query operation can be the IP address of the host used by the user who instructs the data storage device to perform the data query operation, as recorded by the data storage device.

[0047] Optionally, the parameter name for query time in the query log of a data query operation can be a pre-set string used to identify the query time. For example, the parameter name for query time is "query_time". The parameter value for query time in the query log of a data query operation can be the duration recorded by the data storage device during the execution of the data query operation, showing the time taken for the data storage device to run the data query statement.

[0048] Optionally, the parameter name for the query lock wait time in the query log of a data query operation can be a pre-set string used to identify the query lock wait time. For example, the parameter name for the query lock wait time is "lock_time". The parameter value for the query lock wait time in the query log of a data query operation can be the duration recorded by the data storage device during the execution of the data query statement.

[0049] Optionally, the parameter name for the number of rows returned in the query log of a data query operation can be a pre-set string used to identify the number of rows returned. For example, the parameter name for the number of rows returned is "rows_sent". The parameter value for the number of rows returned in the query log of a data query operation can be the number of rows of business data obtained by the data storage device during the execution of the data query operation, as recorded by the data storage device.

[0050] Optionally, the parameter name for the number of rows scanned in the query log of a data query operation can be a pre-set string used to identify the number of rows scanned. For example, the parameter name for the number of rows scanned is "rows_examined". The parameter value for the number of rows scanned in the query log of a data query operation can be the number of rows of business data scanned by the data storage device during the execution of the data query operation, as recorded by the data storage device.

[0051] Optionally, the log acquisition component can be a software or hardware module within an electronic device used to detect whether a data storage device stores query logs in a query log file, and to acquire the stored query logs each time the data storage device is detected to have stored query logs in the query log file, and to store the acquired query logs in a first storage component. Acquiring various query logs from the query log file through the log acquisition component and storing them in the first storage component includes: performing the following operations through the log acquisition component: detecting whether a data storage device stores query logs in a query log file; acquiring the stored query logs each time the data storage device is detected to have stored query logs in the query log file; and storing the acquired query logs in the first storage component. It is possible to detect whether a data storage device stores query logs in a query log file, acquire the stored query logs each time the data storage device is detected to have stored query logs in the query log file, and then store the acquired query logs in the first storage component, thereby acquiring various query logs from the query log file and storing them in the first storage component. The first storage component can be a memory within an electronic device used to store the various query logs acquired from the data storage device.

[0052] Optionally, a preset probe can be used to detect whether the data storage device stores query logs to a query log file, and to retrieve the stored query logs each time this is detected. The preset probe can be a pre-configured software or hardware module used to detect whether the data storage device stores query logs to a query log file and to retrieve the stored query logs each time this is detected.

[0053] Optionally, after storing each query log in the first storage component, the method further includes: detecting whether each query log stored in the first storage component is an invalid query log or a duplicate query log, and deleting the detected invalid query log and duplicate query log.

[0054] Optionally, an invalid query log can refer to a query log that does not contain parameter names and values ​​for query start time, user and host information, data query statement information, execution thread information, source information, or IP address. A duplicate query log can refer to multiple query logs that contain the same parameter names and values ​​for query start time, execution thread information, source information, and IP address.

[0055] Optionally, it can detect whether the query logs stored in the first storage component are invalid query logs and delete the detected invalid query logs. It can also detect whether there are multiple query logs in the first storage component that contain the same query start time, execution thread information, source information, and IP address parameter names and values, and for the detected multiple query logs containing the same query start time, execution thread information, source information, and IP address parameter names and values, retain one of the multiple query logs and delete the others.

[0056] Step 102: Periodically retrieve each pending query log from the first storage component, sort each pending query log according to the key query information of each pending query log, determine the time consumed by each pending query log according to the query timestamps of adjacent query logs, and store each pending query log, the key query information of each pending query log, and the time consumed in the second storage component.

[0057] The key query information includes query timestamp, query thread information, query resource information, and query address information.

[0058] Optionally, the pending query log can be a query log that needs to be processed at the current moment. The key query information in the pending query log can be information needed when processing the pending query log. This key query information includes query timestamp, query thread information, query resource information, and query address information. The query timestamp can be the parameter value of the query start time in the pending query log, i.e., the time recorded by the data storage device when the data query operation began. The query thread information can be the parameter value of the execution thread information in the pending query log, i.e., the numerical ID of the thread in the data storage device used to execute the data query operation, recorded by the data storage device. The query resource information can be the parameter value of the source information in the pending query log, i.e., the information recorded by the data storage device to identify the source of the data query statement executed during the data query operation. The query address information can be the parameter value of the IP address in the pending query log, i.e., the IP address of the host used by the user instructing the data storage device to execute the data query operation, recorded by the data storage device.

[0059] Optionally, the adjacent query log of the pending query log refers to the query log that describes the next data query operation executed after the data query operation described in the pending query log is completed. Typically, the time difference between the query timestamp of the adjacent query log and the query timestamp of the pending query log is the time consumed by the pending query log. The second storage component can be a memory located in an electronic device used to store each completed query log, as well as key query information and time consumed by each completed query log.

[0060] Optionally, the system periodically retrieves each pending query log from the first storage component, sorts each pending query log based on its key query information, determines the time consumed by each pending query log based on the query timestamps of adjacent query logs, and stores each pending query log, its key query information, and its time consumed in the second storage component. This includes: periodically performing the following operations at preset time intervals: retrieving each pending query log from the first storage component; extracting key query information from each pending query log using a feature extraction component, and converting the query timestamp format in the key query information of each pending query log to a target time format; sorting each pending query log based on the query timestamp and query thread information in the key query information of each pending query log using a time consumption calculation component, and determining the time consumed by each pending query log based on the query timestamps of adjacent query logs; and storing each pending query log, its key query information, and its time consumed in the second storage component.

[0061] Optionally, the preset time interval can be a pre-set time interval. For example, the preset time interval can be 10 minutes, 20 minutes, 30 minutes, 1 hour, 2 hours, 12 hours, or 24 hours. The following operations can be performed once every preset time interval: retrieve each pending query log from the first storage component; extract key query information from each pending query log using a feature extraction component, and convert the query timestamp format in the key query information of each pending query log into a target time format; sort each pending query log according to the query timestamp and query thread information in the key query information of each pending query log using a time consumption calculation component, and determine the time consumption of each pending query log based on the query timestamps of its adjacent query logs; store each pending query log, its key query information, and its time consumption in the second storage component.

[0062] Optionally, each query log in the first storage component can be retrieved, and each retrieved query log can be identified as a query log to be processed, thereby obtaining each query log to be processed from the first storage component.

[0063] Optionally, the feature extraction component can be a software or hardware module installed in an electronic device to extract key query information from query logs and convert the query timestamp format in the key query information of the query logs into a target time format. The target time format can be a pre-set time format. For example, a query timestamp in the target time format is "2011-09-12 13:00:42.000". For each query log to be processed, the feature extraction component can determine the separator between the parameter name and parameter value of each parameter in the query log, the parameter name and parameter value of each parameter, identify and extract the query timestamp, query thread information, query resource information, and query address information in the query log, and convert the query timestamp format of the query log to be processed into the target time format.

[0064] Optionally, the time consumption calculation component can be a software or hardware module set in an electronic device for sorting each pending query log according to the query timestamp and query thread information in the key query information of each pending query log, and determining the time consumption of each pending query log according to the query timestamps of adjacent query logs of each pending query log.

[0065] Optionally, the pending query logs are sorted based on the query timestamp and query thread information in the key query information of each pending query log. This includes sorting the pending query logs in chronological order of their query timestamps to form a sequence of pending query logs. When multiple pending query logs have the same query timestamp, they are sorted according to their query thread information. Sorting pending query logs with the same query timestamp based on their query thread information includes sorting them in ascending order of the query thread information values.

[0066] Optionally, the time consumption of each pending query log is determined based on the query timestamps of adjacent query logs, including: performing the following operations for each pending query log: determining the adjacent query logs of the pending query log, and identifying the pending query log as the previous row of the adjacent query log; obtaining the query timestamp of the previous row of the adjacent query log through a log analysis function, calculating the time difference between the adjacent query log and the previous row of the adjacent query log, and obtaining the time consumption of the pending query log.

[0067] Typically, for each pending query log, the query logs whose query timestamps are later than the query timestamp of the pending query log have the smallest time difference with the query timestamp of the pending query log, and whose query thread information, query resource information, and query address information are the same as those of the pending query log are considered to be adjacent query logs of the pending query log.

[0068] Optionally, determining the adjacent query logs of the pending query log includes: identifying the target pending query log among the sorted pending query logs as the adjacent query logs of the pending query log; wherein, the query timestamp of the target pending query log is a time later than the query timestamp of the pending query log, the target pending query log is the other pending query log with the smallest time difference between its query timestamp and the query timestamp of the pending query log among the other pending query logs whose query timestamps are later than the query timestamp of the pending query log, the query thread information of the target pending query log is the same as the query thread information of the pending query log, the query resource information of the target pending query log is the same as the query resource information of the pending query log, and the query address information of the target pending query log is the same as the query address information of the pending query log.

[0069] Optionally, the preceding line of the query log can refer to a query log describing the data query operation executed before the data query operation described in the query log. The log analysis function can be a pre-defined function in the electronic device used to manipulate the query log. After determining the preceding line of the query log, the log analysis function can be called to obtain the query timestamp of the preceding line of the query log, calculate the time difference between the query log and the preceding line of the query log, and obtain the time consumed by the preceding line of the query log.

[0070] Optionally, after determining that the query log to be processed is the previous row of the adjacent query log, the log analysis function can be called to obtain the query timestamp of the previous row of the adjacent query log, calculate the time difference between the adjacent query log and the previous row of the adjacent query log, and obtain the time consumed by the query log to be processed.

[0071] Optionally, for each pending query log whose time consumption has been determined, the key query information and time consumption of each pending query log can be stored in the second storage component.

[0072] Optionally, for pending query logs in each pending query log where adjacent query logs cannot be determined or where the time consumption cannot be determined, the pending query logs can be re-stored in the first storage component, so that the pending query logs can continue to be processed in subsequent processing.

[0073] Step 103: When a time consumption statistics request is obtained, the statistical result corresponding to the time consumption statistics request is determined based on the various query logs stored in the second storage component, the key query information of each query log, and the time consumption.

[0074] Optionally, a time-consuming statistics request can be a request sent by a user to request specified statistical operations on each query log, key query information of each query log, and time consumption stored in the second storage component. Time-consuming statistics requests can include average time-consuming statistics requests, timeout log statistics requests, and user log statistics requests.

[0075] Optionally, when a time consumption statistics request is obtained, the statistical result corresponding to the time consumption statistics request is determined based on the various query logs stored in the second storage component, the key query information of each query log, and the time consumption. This includes: when an average time consumption statistics request is obtained, the time consumption of each target query log corresponding to the target information of the average time consumption statistics request is obtained from the second storage component, the average time consumption of each target query log is calculated, and the statistical result corresponding to the average time consumption statistics request is obtained.

[0076] Optionally, the average time consumption statistics request can be a request to calculate the average time consumption of various query logs containing target information in the second storage component. The average time consumption statistics request contains target information. Target information can be the identifier of a specified user, the identifier of a host, or other information indicating the type of query log. The target query log corresponding to the target information in the average time consumption statistics request can refer to the query logs containing target information in the second storage component. It can detect whether the electronic device has received the average time consumption statistics request sent by the user, and after each detection that the electronic device has received the average time consumption statistics request sent by the user, it can obtain the average time consumption statistics request received by the electronic device, thereby obtaining the average time consumption statistics request. It can obtain the time consumption of each target query log corresponding to the target information in the average time consumption statistics request from the second storage component, calculate the average time consumption of each target query log, and determine the calculated average time consumption of each target query log as the statistical result corresponding to the average time consumption statistics request. The statistical result corresponding to the average time consumption statistics request can be fed back to the user who sent the average time consumption statistics request.

[0077] Optionally, when a timeout statistics request is obtained, the statistical result corresponding to the timeout statistics request is determined based on the various query logs stored in the second storage component, the key query information of each query log, and the timeout. This includes: when a timeout log statistics request is obtained, the various timeout query logs in the second storage component are obtained based on the timeout threshold of the timeout log statistics request, and the statistical result corresponding to the timeout log statistics request is obtained.

[0078] Optionally, a timeout log statistics request can be a request to collect statistics on various timeout query logs in the second storage component whose execution time exceeds a timeout threshold. The average execution time statistics request includes a timeout threshold. The timeout threshold can be a specified time threshold. Timeout query logs can refer to query logs whose execution time exceeds the timeout threshold. It can detect whether the electronic device receives a timeout log statistics request sent by the user, and after each detection that the electronic device has received a timeout log statistics request sent by the user, it can obtain the received timeout log statistics request. It can retrieve various timeout query logs whose execution time exceeds the timeout threshold from the second storage component, and determine the retrieved timeout query logs as the statistical results corresponding to the timeout log statistics request. The statistical results corresponding to the timeout log statistics request can be fed back to the user who sent the timeout log statistics request.

[0079] Optionally, when a time consumption statistics request is obtained, the statistical result corresponding to the time consumption statistics request is determined based on the various query logs, key query information of each query log, and time consumption stored in the second storage component. This includes: when a user log statistics request is obtained, retrieving various query logs containing target user information, key query information of each query log containing target user information, and time consumption from the second storage component to obtain the statistical result corresponding to the user log statistics request.

[0080] Optionally, a user log statistics request can be a request to collect statistics on various query logs containing target user information, key query information of each query log containing target user information, and the execution time. The user log statistics request contains target user information. Target information can be the identifier information of a specified user. It can detect whether the electronic device has received a user log statistics request sent by the user, and after each detection that the electronic device has received a user log statistics request, it can obtain the user log statistics request received by the electronic device. It can obtain various query logs containing target user information, key query information of each query log containing target user information, and the execution time from the second storage component, and determine the obtained various query logs containing target user information, key query information, and execution time as the statistical results corresponding to the user log statistics request. The statistical results corresponding to the user log statistics request can be fed back to the user who sent the user log statistics request.

[0081] The technical solution of this invention involves collecting various query logs from a query log file using a log collection component and storing each query log in a first storage component. Periodically, it retrieves each pending query log from the first storage component, sorts the pending query logs based on their key query information, determines the time consumed by each pending query log based on the query timestamps of adjacent query logs, and stores each pending query log, its key query information, and its time consumed in a second storage component. The key query information includes query timestamps, query thread information, query resource information, and query address information. When a time consumption statistics request is received, it calculates the time consumed based on the query logs and query timestamps stored in the second storage component. This method automatically sorts query logs based on key query information and time consumption, and quickly and accurately determines the time consumption statistics request. It solves the problems of traditional query log processing solutions relying on manual operation, resulting in high time and labor costs, and difficulty in guaranteeing accuracy. It can automatically sort query logs based on key query information and quickly and accurately determine the time consumption of query logs based on the query timestamps of adjacent query logs. It can then perform statistical analysis on the determined time-consuming query logs to obtain corresponding results. This achieves automatic processing of query logs generated in data storage devices based on the query timestamps of adjacent query logs, determining the time consumption of query logs, reducing the time and labor costs of query log processing, and improving the accuracy of query log processing.

[0082] The technical solution of this invention can deeply analyze the time differences in query logs and achieve precise monitoring of database performance.

[0083] Optionally, in a specific instance, the database-specific configuration logic in the data storage device may include:

[0084] slow_query_log=ON: Enable slow query logging.

[0085] long_query_time = N: Sets the time threshold for recording slow queries (unit: seconds, N is a custom value).

[0086] log_queries_not_using_indexes=ON: Log queries that do not use indexes.

[0087] log_throttle_queries_not_using_indexes=N: Limits the frequency of queries that do not use indexes.

[0088] general_log=ON: Enable general query logging (optional, enable as needed).

[0089] Optionally, in a specific instance, the log analysis function's operations on the query log include dynamic field injection. This can be achieved through the parameter p= <num>Specify the number of rows to backtrack, and it will automatically generate a line with the suffix _p. <num>Temporary fields. For example: "timestamp_p1", "timestamp_p2", etc. If multiple rows need to be backtracked (e.g., p = 1-3), the generated field naming rule is: "<field name>_p <num>", where "<field name>" is the original field name, <num>" represents the row number to be traced. The operation logic for dynamic field injection may include:

[0090] autoregress timestamp p=1: Copy the timestamp field from the previous row to the current row and generate a new field timestamp_p1.

[0091] autoregress timestamp p=1-3: Copy the timestamp field from the previous three rows to the current row, generating new fields timestamp_p1, timestamp_p2, and timestamp_p3.

[0092] Optionally, in a specific instance, the log analysis function's operations on the query logs include interpolation. The logic for interpolation based on temporary fields may include:

[0093] |eval time_diff=timestamp-timestamp_p1: Calculates the difference between the timestamp of the current row and the previous row, and stores the result in the time_diff field.

[0094] Optionally, in a specific instance, the log analysis function's operations on the query log include status comparison. The logic for comparing field values ​​between different rows may include:

[0095] |where apache.clientip==apache.clientip_p1: Filters out log records where the current line has the same clientip field value as the previous line.

[0096] Optionally, in a specific instance, the log analysis function's operations on the query logs may include complex condition filtering. The logic for filtering based on multiple conditions may include:

[0097] |where apache.clientip==apache.clientip_p1&×tamp-timestamp_p1>3*30*24*3600*1000: Filters out log records where the current line has the same clientip field value as the previous line and the time difference is greater than 3 months.

[0098] Optionally, in a specific instance, the logic for the log analysis function to perform cross-row field backtracking correlation on the query log may include:

[0099] |autoregress timestamp as timestamp_p p=1: Copies the timestamp field from the previous row to the current row, generating a new field timestamp_p1.

[0100] |where isnum(timestamp): Ensures that the timestamp field is a numeric type.

[0101] |table timestamp,timestamp_p,raw message: Outputs timestamp, timestamp_p, and raw log message.

[0102] |evaltim = timestamp - timestamp_p: Calculates the difference between the timestamp of the current row and the previous row, and stores the result in the tim field.

[0103] |sort by timestamp: Sort by timestamp in ascending order.

[0104] |table timestamp,timestamp_p,tim,raw message: Outputs the timestamp, the previous line timestamp, the time difference, and the raw log information.

[0105] Optionally, in a specific instance, the processing logic for the average time consumption statistics request may include:

[0106] |stats avg(tim)by sql_text.

[0107] Optionally, in a specific instance, the processing logic for timeout log statistics requests may include:

[0108] |where tim>1000.

[0109] Optionally, in a specific instance, the processing logic for user log statistics requests may include:

[0110] |stats avg(tim),count by user_host.

[0111] Example 2

[0112] Figure 2 This is a flowchart illustrating a query log processing method provided in Embodiment 2 of the present invention. This embodiment of the present invention can be combined with various optional solutions from one or more of the above embodiments. For example... Figure 2 As shown, the method includes:

[0113] Step 201: Collect each query log from the query log file using the log collection component, store each query log in the first storage component, detect whether each query log stored in the first storage component is an invalid query log or a duplicate query log, and delete the detected invalid query log and duplicate query log.

[0114] Step 202: Periodically retrieve each pending query log from the first storage component, sort each pending query log according to the key query information of each pending query log, determine the time consumed by each pending query log according to the query timestamps of adjacent query logs, and store each pending query log, the key query information of each pending query log, and the time consumed in the second storage component.

[0115] The key query information includes query timestamp, query thread information, query resource information, and query address information.

[0116] Step 203: When an average time consumption statistics request is obtained, the time consumption of each target query log corresponding to the target information of the average time consumption statistics request is obtained from the second storage component, the average time consumption of each target query log is calculated, and the statistical result corresponding to the average time consumption statistics request is obtained.

[0117] Step 204: When a timeout log statistics request is received, according to the timeout threshold of the timeout log statistics request, obtain each timeout query log in the second storage component to obtain the statistical results corresponding to the timeout log statistics request.

[0118] Step 205: When a user log statistics request is received, retrieve the various query logs containing the target user information, the key query information of the various query logs containing the target user information, and the time consumption from the second storage component to obtain the statistical results corresponding to the user log statistics request.

[0119] The technical solution of this invention can automatically sort query logs based on key query information, quickly and accurately determine the query log's processing time based on the query timestamps of adjacent query logs, and perform statistical analysis on the determined time-consuming query logs to obtain corresponding statistical results. This achieves automatic processing of query logs generated in the data storage device based on the query timestamps of adjacent query logs, determining the query log's processing time, reducing the time and manpower costs of query log processing, and improving the accuracy of query log processing.

[0120] Example 3

[0121] Figure 3 This is a schematic diagram of a query log processing device provided in Embodiment 3 of the present invention. The device can be configured in an electronic device. Figure 3 As shown, the device includes: a log collection module 301, a time consumption determination module 302, and a time consumption statistics module 303.

[0122] The system includes a log collection module 301, which collects various query logs from a query log file using a log collection component and stores them in a first storage component. A time consumption determination module 302 periodically retrieves each pending query log from the first storage component, sorts the pending query logs based on their key query information, determines the time consumption of each pending query log based on the query timestamps of adjacent query logs, and stores each pending query log, its key query information, and its time consumption in a second storage component. The key query information includes query timestamps, query thread information, query resource information, and query address information. A time consumption statistics module 303, when a time consumption statistics request is received, determines the statistical result corresponding to the request based on the various query logs, their key query information, and their time consumption stored in the second storage component.

[0123] The technical solution of this invention involves collecting various query logs from a query log file using a log collection component and storing each query log in a first storage component. Periodically, it retrieves each pending query log from the first storage component, sorts the pending query logs based on their key query information, determines the time consumed by each pending query log based on the query timestamps of adjacent query logs, and stores each pending query log, its key query information, and its time consumed in a second storage component. The key query information includes query timestamps, query thread information, query resource information, and query address information. When a time consumption statistics request is received, it calculates the time consumed based on the query logs and query timestamps stored in the second storage component. This method automatically sorts query logs based on key query information and time consumption, and quickly and accurately determines the time consumption statistics request. It solves the problems of traditional query log processing solutions relying on manual operation, resulting in high time and labor costs, and difficulty in guaranteeing accuracy. It can automatically sort query logs based on key query information and quickly and accurately determine the time consumption of query logs based on the query timestamps of adjacent query logs. It can then perform statistical analysis on the determined time-consuming query logs to obtain corresponding results. This achieves automatic processing of query logs generated in data storage devices based on the query timestamps of adjacent query logs, determining the time consumption of query logs, reducing the time and labor costs of query log processing, and improving the accuracy of query log processing.

[0124] In an optional embodiment of the present invention, the log collection module 301 may be further configured to: detect whether each query log stored in the first storage component is an invalid query log or a duplicate query log, and delete the detected invalid query log and duplicate query log.

[0125] In an optional embodiment of the present invention, the time consumption determination module 302 is specifically configured to: periodically perform the following operations at preset time intervals: obtain each pending query log from the first storage component; extract key query information of each pending query log from each pending query log using a feature extraction component, and convert the query timestamp format in the key query information of each pending query log into a target time format; sort each pending query log according to the query timestamp and query thread information in the key query information of each pending query log using a time consumption calculation component, and determine the time consumption of each pending query log according to the query timestamps of adjacent query logs of each pending query log; and store each pending query log, the key query information of each pending query log, and the time consumption in the second storage component.

[0126] In an optional embodiment of the present invention, the time consumption determination module 302, when performing the operation of sorting each query log to be processed according to the query timestamp and query thread information in the key query information of each query log to be processed, is specifically used to: sort each query log to be processed in order of query timestamp from front to back to form a sequence of query logs to be processed; wherein, when there are multiple query logs to be processed with the same query timestamp, the query logs to be processed with the same query timestamp are sorted according to the query thread information.

[0127] In an optional embodiment of the present invention, the time consumption determination module 302, when performing the operation of determining the time consumption of each pending query log based on the query timestamps of adjacent query logs of each pending query log, is specifically configured to: perform the following operations for each pending query log: determine the adjacent query logs of the pending query log, and determine the pending query log as the query log preceding the adjacent query log; obtain the query timestamp of the preceding query log of the adjacent query log through a log analysis function, calculate the time difference between the adjacent query log and the query timestamp of the preceding query log of the adjacent query log, and obtain the time consumption of the pending query log.

[0128] In an optional embodiment of the present invention, the time consumption statistics module 303 is specifically used to: when an average time consumption statistics request is obtained, retrieve the time consumption of each target query log corresponding to the target information of the average time consumption statistics request from the second storage component, calculate the average time consumption of each target query log, and obtain the statistical result corresponding to the average time consumption statistics request.

[0129] In an optional embodiment of the present invention, the time consumption statistics module 303 is specifically used to: when a timeout log statistics request is obtained, obtain each timeout query log in the second storage component according to the timeout threshold of the timeout log statistics request, and obtain the statistical result corresponding to the timeout log statistics request.

[0130] The query log processing device provided in this embodiment of the invention can execute the query log processing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0131] Example 4

[0132] Figure 4 A schematic diagram of an electronic device 10, which can be used to implement the query log processing method of embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0133] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0134] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as query log processing methods.

[0136] In some embodiments, the query log processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the query log processing method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the query log processing method by any other suitable means (e.g., by means of firmware).

[0137] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or electronic device.

[0139] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0140] To provide user interaction, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).

[0141] The systems and technologies described herein can be implemented in computing systems that include back-end components (e.g., as data electronic devices), or computing systems that include middleware components (e.g., application electronic devices), or computing systems that include front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0142] A computing system can include clients and electronic devices. Clients and electronic devices are generally geographically separated and typically interact via communication networks. The client-electronic device relationship is created by computer programs running on the respective computers and establishing a client-electronic device relationship between them. Electronic devices can be cloud electronic devices, also known as cloud computing electronic devices or cloud servers, which are hosting products within the cloud computing service ecosystem. These address the shortcomings of traditional physical hosting and VPS services, such as high management difficulty and weak business scalability.

[0143] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0144] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.< / num> < / num> < / num> < / num>

Claims

1. A query log processing method, characterized in that, include: The log collection component collects each query log from the query log file and stores each query log in the first storage component; The system periodically retrieves each pending query log from the first storage component, sorts each pending query log based on its key query information, determines the time consumed by each pending query log based on the query timestamps of adjacent query logs, and stores each pending query log, its key query information, and its time consumed in the second storage component. The key query information includes query timestamps, query thread information, query resource information, and query address information. At preset time intervals, the system performs the following operations: retrieves each pending query log from the first storage component; extracts key query information from each pending query log using a feature extraction component, converts the query timestamp format in the key query information to a target time format; sorts each pending query log based on its query timestamps and query thread information using a time consumption calculation component, determines the time consumed by each pending query log based on the query timestamps of adjacent query logs; and stores each pending query log, its key query information, and its time consumed in the second storage component. When a time consumption statistics request is obtained, the statistical result corresponding to the time consumption statistics request is determined based on the various query logs stored in the second storage component, the key query information of each query log, and the time consumption. Specifically, when an average time consumption statistics request is obtained, the time consumption of each target query log corresponding to the target information of the average time consumption statistics request is obtained from the second storage component, the average time consumption of each target query log is calculated, and the statistical result corresponding to the average time consumption statistics request is obtained.

2. The query log processing method according to claim 1, characterized in that, After storing each query log in the first storage component, the following is also included: Detect whether each query log stored in the first storage component is an invalid query log or a duplicate query log, and delete the detected invalid query log and duplicate query log.

3. The query log processing method according to claim 1, characterized in that, The pending query logs are sorted based on the query timestamps and query thread information in the key query information of each pending query log, including: The pending query logs are sorted in order of query timestamp from front to back to form a sequence of pending query logs; when there are multiple pending query logs with the same query timestamp, they are sorted according to the query thread information.

4. The query log processing method according to claim 3, characterized in that, The time consumed by each pending query log is determined based on the query timestamps of its adjacent query logs, including: For each pending query log, perform the following operations: Determine the adjacent query logs of the query log to be processed, and determine the query log to be processed as the previous line of the adjacent query log; By using the log analysis function, the query timestamp of the previous row of the adjacent query log is obtained, and the time difference between the adjacent query log and the query timestamp of the previous row of the adjacent query log is calculated to obtain the time consumption of the query log to be processed.

5. The query log processing method according to claim 1, characterized in that, When a time consumption statistics request is received, based on the various query logs stored in the second storage component, the key query information of each query log, and the time consumption, the statistical results corresponding to the time consumption statistics request are determined, including: When a timeout log statistics request is received, the timeout query logs in the second storage component are obtained according to the timeout threshold of the timeout log statistics request, and the statistical results corresponding to the timeout log statistics request are obtained.

6. A query log processing device, characterized in that, include: The log collection module is used to collect various query logs from the query log file through the log collection component and store each query log in the first storage component; The time consumption determination module is used to periodically retrieve each pending query log from the first storage component, sort each pending query log according to its key query information, determine the time consumption of each pending query log based on the query timestamps of adjacent query logs, and store each pending query log, its key query information, and its time consumption in the second storage component. The key query information includes query timestamps, query thread information, query resource information, and query address information. At preset time intervals, the module performs the following operations: retrieving each pending query log from the first storage component. The system extracts key query information from each pending query log using a feature extraction component, converting the query timestamp format in the key query information to a target time format. It then sorts the pending query logs based on the query timestamps and query thread information in the key query information, determining the time consumption of each pending query log based on the query timestamps of its adjacent logs. Finally, it stores each pending query log, its key query information, and the corresponding time consumption in a second storage component. The time consumption statistics module is used to determine the statistical results corresponding to the time consumption statistics request based on the various query logs, key query information and time consumption of each query log stored in the second storage component when a time consumption statistics request is obtained; wherein, when an average time consumption statistics request is obtained, the time consumption of each target query log corresponding to the target information of the average time consumption statistics request is obtained from the second storage component, the average time consumption of each target query log is calculated, and the statistical results corresponding to the average time consumption statistics request are obtained.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the query log processing method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the query log processing method according to any one of claims 1-5.

Citation Information

Patent Citations

  • Log query method and device, computer equipment and storage medium

    CN111767252A

  • Log processing method and device, equipment and storage medium

    CN114116811A