A method, apparatus, and medium for analyzing slow queries in a system.

Abstract

This application relates to the field of ElasticSearch system maintenance and discloses a method, apparatus, and medium for analyzing slow queries. The method includes: acquiring target models from a model library to analyze slow query tasks based on these target models; wherein the model library is a database storing timeout models corresponding to timeout query tasks; analyzing target models based on their query conditions and data volume, and corresponding query task examples, and obtaining model scores to exclude query tasks that time out due to preemption of computing resources; and analyzing target models with scores higher than a threshold to determine the reasons for query task timeouts. This application analyzes and scores the models of timeout query tasks in the model library to remove misclassified models from the database, facilitating system analysis and improvement based on actual timeout query tasks.

Description

Technical Field

[0001] This application relates to the field of ElasticSearch system maintenance, and in particular to a method, apparatus, and medium for analyzing slow queries in a system. Background Technology

[0002] With the development of big data, non-relational databases, such as Elasticsearch (a distributed full-text search engine), have rapidly emerged, meeting enterprises' needs for storing and querying massive amounts of document data and gaining widespread application. However, due to rapid product feature iterations and large data volumes, Elasticsearch systems may experience performance bottlenecks or system failures during data writing and querying. In particular, the data querying process, involving numerous system nodes and a large amount of data, is prone to slow queries, leading to query timeouts.

[0003] In practice, it's necessary to analyze timeout queries to optimize Elasticsearch system performance and improve query speed. Currently, timeout queries are primarily identified through the official slow query logs. However, because each node in the Elasticsearch system independently determines timeout queries and only records completed timeouts, the slow query logs suffer from low timeliness and disorganized storage. Furthermore, due to limited Elasticsearch system resources, high-load query tasks can degrade the efficiency of other queries, causing the system to classify affected normal queries as slow queries, thus hindering subsequent analysis and improvements to the Elasticsearch system.

[0004] Therefore, it is evident that providing an accurate and efficient method for analyzing slow queries in a system to prevent misjudgments of query tasks due to insufficient system computing resources is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, and medium for analyzing slow queries in a system, so as to accurately and efficiently obtain timed-out query tasks in the ElasticSearch system, and to analyze and improve the ElasticSearch system based on the timed-out query tasks.

[0006] To address the aforementioned technical problems, this application provides a method for analyzing slow queries in a system, including:

[0007] Obtain the target model from the model library, wherein the model library is a database used to store timeout models corresponding to timeout query tasks, the timeout query task is a query task whose query time is greater than a first threshold time obtained by using the ElasticSearch system query interface, and the target model is a timeout model in the database that meets a first preset condition.

[0008] The target models are analyzed based on the query conditions and data volume of each target model, as well as the query task examples corresponding to each target model, and a model score is obtained.

[0009] The target models whose model scores are higher than the threshold are analyzed to determine the reasons for the query task timeout.

[0010] Preferred options also include:

[0011] Retrieve the timeout query tasks in each ElasticSearch node within each detection period;

[0012] Determine whether the timeout model corresponding to the timeout query task exists in the model library;

[0013] If the timeout model exists, the timeout query task is stored in the sample library corresponding to the timeout model;

[0014] If the timeout model does not exist, then the timeout model is stored in the model library.

[0015] Preferably, the first preset condition is:

[0016] No analysis was performed within the second threshold time period;

[0017] Or the change in the query sample corresponding to the target model is greater than the sample threshold.

[0018] Preferably, the analysis of the target models based on the query conditions and data volume of each target model, and the query task examples corresponding to each target model, includes:

[0019] Construct a tree structure corresponding to the target model based on the description language of the target model to determine the number of query conditions and the number of expensive query categories of the target model;

[0020] Obtain a query task example corresponding to the target model, and determine the total length and data volume of the query conditions based on the tree structure and the query task example;

[0021] The model score of the target model is calculated based on the number of query conditions, the number of expensive query categories, the total length of the query conditions, and the amount of data.

[0022] Preferably, the step of calculating the model score of the target model based on the number of query conditions, the number of expensive query categories, the total length of the query conditions, and the data volume includes:

[0023] Determine the weights corresponding to the number of query conditions, the number of expensive query categories, the total length of query conditions, and the amount of data; wherein the weights of the number of expensive query categories, the amount of data, the total length of query conditions, and the number of query conditions decrease in that order.

[0024] Preferably, determining whether the timeout model corresponding to the timeout query task exists in the model library includes:

[0025] Construct a detection model corresponding to the timeout query task, and calculate the detection identifier of the detection model, wherein the detection identifier is MD5;

[0026] Determine whether the detection identifier is the same as the identifier of each timeout model;

[0027] If the identifier is different from that of each of the timeout models, then it is determined that there is no timeout model in the model library corresponding to the timeout query task.

[0028] Preferably, after the step of analyzing the target models whose model scores are higher than the threshold, the method further includes: optimizing the query task of the ElasticSearch system based on the timeout reason.

[0029] To address the aforementioned technical problems, this application also provides a system slow query analysis device, comprising:

[0030] The acquisition module is used to acquire the target model in the model library, wherein the model library is a database used to store the timeout model corresponding to the timeout query task, the timeout query task is a query task whose query time is greater than a first threshold time obtained by using the ElasticSearch system query interface, and the target model is the timeout model in the database that meets the first preset condition.

[0031] The scoring module is used to analyze the target models based on the query conditions and data volume of each target model and the query task examples corresponding to each target model, and to obtain model scores.

[0032] The analysis module is used to analyze the target models whose model scores are higher than a threshold in order to determine the reasons for the timeout of the query task.

[0033] To address the aforementioned technical problems, this application also provides a system slow query analysis device, including a memory for storing computer programs;

[0034] A processor is used to implement the steps of the system slow query analysis method when executing the computer program.

[0035] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the system slow query analysis method.

[0036] This application provides a method for analyzing slow queries in a system, including: acquiring target models from a model library to analyze slow query tasks based on the target models; wherein the model library is a database used to store timeout models corresponding to timeout query tasks; timeout query tasks are query tasks whose query time exceeds a first threshold time; and target models are timeout models in the database that meet a first preset condition. The method analyzes the target models based on the query conditions and data volume of each target model, and the corresponding query task examples, and obtains a model score to exclude query tasks that time out due to preemption of computing resources, preventing misjudgments of query tasks from interfering with subsequent analysis. Target models with model scores higher than the threshold are analyzed to determine the cause of the query task timeout. Therefore, the technical solution provided in this application analyzes and scores the models of timeout query tasks in the model library to remove misjudged models from the database, facilitating the analysis and improvement of the ElasticSearch system based on actual timeout query tasks.

[0037] In addition, this application also provides a system slow query analysis device and medium, which correspond to the above method and have the same effect. Attached Figure Description

[0038] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A flowchart illustrating a system slow query analysis method provided in this application embodiment;

[0040] Figure 2 This is a structural diagram of a system slow query analysis device provided in an embodiment of this application;

[0041] Figure 3 A structural diagram of a system slow query analysis device provided in another embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0043] The core of this application is to provide a method, apparatus, and medium for analyzing slow queries in a system, so as to accurately and efficiently obtain timed-out query tasks in the ElasticSearch system, and to analyze and improve the ElasticSearch system based on the timed-out query tasks.

[0044] During the operation of the Elasticsearch system, data querying is prone to slow speeds and timeouts due to the large number of system nodes and data volume involved. To optimize the Elasticsearch system, it is necessary to identify and analyze timeout query tasks. However, timeout tasks include those caused by their own logical problems and those caused by system resource preemption. The latter does not reflect problems within the Elasticsearch system; therefore, it needs to be removed during analysis. To address this issue, this application provides a method for slow query analysis, including: obtaining target models from a model library to analyze slow query tasks based on the target models. The model library is a database storing timeout models corresponding to timeout query tasks; timeout query tasks are those with query times exceeding a first threshold; and target models are timeout models in the database that meet a first preset condition. The target models are analyzed based on their query conditions and data volume, along with corresponding query task examples, and a model score is obtained. This process eliminates query tasks that time out due to preemption of computing resources, preventing misjudgments from interfering with subsequent analysis. Target models with scores higher than a threshold are analyzed to determine the reasons for query timeouts. Therefore, the technical solution provided in this application analyzes and scores models of timed-out query tasks in the model library to remove misjudged models from the database, facilitating the analysis and improvement of the ElasticSearch system based on actual timed-out query tasks.

[0045] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Figure 1 A flowchart of a system slow query analysis method provided in an embodiment of this application is shown below. Figure 1 As shown, the method includes:

[0047] S10: Obtain the target model from the model library, where the model library is a database used to store the timeout models corresponding to the timeout query tasks, the timeout query task is a query task obtained by using the ElasticSearch system query interface whose query time is greater than the first threshold time, and the target model is the timeout model in the database that meets the first preset condition.

[0048] In practice, the slow query analysis method provided in this application can be executed on the models in the model library once every preset time interval, or it can be executed when the change in the models in the model library is greater than the threshold. There is no limitation here.

[0049] It is understandable that the target model is a timeout model in the database that meets the first preset condition. The first preset condition includes models that have never been analyzed and scored, or models whose corresponding query sample changes exceed a sample threshold. It is important to note that since the scores of each target model are determined based on model information and query task instances, and the overall resource volume of the ElasticSearch system is constantly changing, it is necessary to re-analyze models that have already been analyzed and scored. Therefore, the aforementioned first preset condition also includes models that have not undergone analysis within the second threshold time.

[0050] It's important to note that while Elasticsearch provides slow query logs, these logs often don't show currently incomplete, timed-out tasks, impacting scoring results. To address this, a data collection task can be executed every preset time interval (e.g., every minute). The currently executing query tasks can be retrieved using Elasticsearch's task management API.

[0051] The request command is as follows: GET_tasks?actions=*search&detailed

[0052] Example of response result:

[0053] {"nodes":{"oTUltX4IQMOUUVeiohTt8A":{"name":"H5dfFeA","transport_address":"127.0.0.1:9300","host":"127.0.0.1","ip":127. 0.0.1:9300","tasks":{"oTUltX4IQMOUUVeiohTt8A:464":{"node":"oTUltX4IQMOUUVeiohTt8A","id":464,"type":"transport","action ":"indices:data / read / search", "description":"indices[test], types[test], search_type[QUERY_THEN_FETCH], source[{\"query\": ...}]", "start_time_in_millis":1483478610008, "running_time_in_nanos":13991383, "cancellable":true, "cancelled":false}}}}}

[0054] The description contains the specific query information, start_time_in_millis is the query start timestamp, and running_time_in_nanos is the query time (nanoseconds). This filters query information whose query time exceeds a preset time.

[0055] S11: Analyze the target models based on the query conditions and data volume of each target model, as well as the query task examples corresponding to each target model, and obtain the model score.

[0056] Understandably, to ensure the scoring more accurately reflects the merits of each target model, it's possible to analyze the target models by combining their query conditions, data volume, and corresponding query task examples, and then obtain a model score. Specifically, the analysis of the target models includes: constructing a tree structure corresponding to the target model based on its description language to determine the number of query conditions and expensive query categories; obtaining query task examples corresponding to the target model, and determining the total length of query conditions and data volume based on the tree structure and query task examples; and calculating the model score based on the number of query conditions, the number of expensive query categories, the total length of query conditions, and the data volume.

[0057] S12: Analyze the target models whose model scores are higher than the threshold to determine the reasons for the timeout of the query task.

[0058] Understandably, after obtaining the scores for each target model, it is necessary to determine whether the score of each target model is greater than a threshold. If it is not greater than the threshold, it indicates that the timeout query task corresponding to the target model is due to insufficient system resources, and no analysis of the task is required. If it is greater than the threshold, it indicates that the timeout query task corresponding to the target model has a structural or logical problem, and analysis is needed to determine the cause of the query task's timeout.

[0059] Understandably, after obtaining the timeout reason, it is also necessary to optimize the query tasks of the ElasticSearch system based on the timeout reason.

[0060] This application provides a method for analyzing slow queries in a system, including: acquiring target models from a model library to analyze slow query tasks based on the target models; wherein the model library is a database used to store timeout models corresponding to timeout query tasks; timeout query tasks are query tasks whose query time exceeds a first threshold time; and target models are timeout models in the database that meet a first preset condition. The method analyzes the target models based on the query conditions and data volume of each target model, and the corresponding query task examples, and obtains a model score to exclude query tasks that time out due to preemption of computing resources, preventing misjudgments of query tasks from interfering with subsequent analysis. Target models with model scores higher than the threshold are analyzed to determine the cause of the query task timeout. Therefore, the technical solution provided in this application analyzes and scores the models of timeout query tasks in the model library to remove misjudged models from the database, facilitating the analysis and improvement of the ElasticSearch system based on actual timeout query tasks.

[0061] Understandably, in order to improve the efficiency of query task analysis and avoid useless work, it is necessary to determine whether there is a timeout model in the model library before obtaining the model of the timeout query task, including: obtaining the timeout query tasks in each ElasticSearch node within each detection period.

[0062] Determine if a timeout model corresponding to the timeout query task exists in the model library;

[0063] If a timeout model exists, the timeout query task will be stored in the sample library corresponding to the timeout model.

[0064] If no timeout model exists, the timeout model will be stored in the model library.

[0065] Specifically, determining whether a timeout model exists in the model library corresponding to the timeout query task includes:

[0066] Construct a detection model corresponding to the timeout query task, and calculate the detection identifier of the detection model, where the detection identifier is MD5;

[0067] Determine whether the detection identifier is the same as the identifier of each timeout model;

[0068] If the identifiers of each timeout model are different, it is determined that there is no timeout model in the model library corresponding to the timeout query task.

[0069] In practice, the source content in the description is parsed. For example, the original query statement is:

[0070] {"query":{"bool":{"must":[{"term":{"name":"test"}},{"range":{"time":{"gte":"2022-06-21 00:00:00","lte":"2022-06-22 00:00:00"}}}]}

[0071] }}

[0072] When building the model, clear the specific query conditions and remove other non-query conditions. The statement is:

[0073] {"query":{"bool":{"must":[{"term":{"name":null}}, {"range":{"time":null}}]}}}

[0074] Check if the model exists in the existing model library; if not, insert the model into the model library.

[0075] As a preferred embodiment, the analysis of the target models based on the query conditions and data volume of each target model, and the query task examples corresponding to each target model, includes:

[0076] Construct a tree structure corresponding to the target model based on the description language of the target model to determine the number of query conditions and the number of expensive query categories in the target model;

[0077] Obtain query task examples corresponding to the target model, and determine the total length and data volume of query conditions based on the tree structure and query task examples;

[0078] The model score of the target model is calculated based on the number of query conditions, the number of expensive query categories, the total length of query conditions, and the amount of data.

[0079] In practical implementation, the target model is analyzed one by one, including analysis from four aspects: the number of expensive query categories, the total number of query conditions, the length of query conditions, and the amount of data. Specifically, this includes:

[0080] (1) Model statement constructs tree structure

[0081] The model statements are in JSON format and are constructed into a tree structure in the code for easy traversal and searching.

[0082] (2) Get the parent field name of the leaf node, count the number of expensive query categories and the total number of query conditions. For example, the parent field name of the leaf node is term, range, the total number of query conditions is 2, and the expensive query categories include: wildcard, script, fuzzy, regexp, prefix. When within this range, the count is incremented by 1 for each one.

[0083] (3) Obtain the sample list corresponding to the model, sum the condition lengths of the leaf nodes of each sample, and calculate the average condition length of the entire sample list.

[0084] For each sample in the same model, calculate the total length of the query conditions. For example, if the length of the condition "test" is 4, and the lengths of "gte" and "lte" within the range are both 19, the total length of the query conditions is 23. If there are multiple samples, calculate the average of the total lengths of the conditions across all samples.

[0085] (4) Obtain the sample list corresponding to the model and query the amount of indexed data in each sample.

[0086] The model example records query information (describe), which includes a list of indexes queried, the amount of data in this index list in Elasticsearch queried, and the average value, which tells us the amount of data queried in this query example.

[0087] In a preferred embodiment, the model score of the target model is calculated based on the number of query conditions, the number of expensive query categories, the total length of query conditions, and the amount of data, including:

[0088] Determine the weights corresponding to the number of query conditions, the number of expensive query categories, the total length of query conditions, and the amount of data; among them, the weights of the number of expensive query categories, the amount of data, the total length of query conditions, and the number of query conditions decrease in that order.

[0089] Based on project experience, the following settings are proposed for implementation (subject to adjustment):

[0090] An expensive query category scores 5 points; the highest score for data volume is 3 points, with 0.1 points awarded for every 10 million data points; the highest score for query condition length is 2 points, with 0.1 points awarded for every 128 characters; and the highest score for total query conditions is 1 points, with 0.1 points awarded for each condition. For example, if a model contains one expensive query, has 30 million data points, a total query condition length of 1024, and a total of 5 query conditions, then the score would be 5 + min(3, (30 million / 10 million * 0.1)) + min(2, (1024 / 128 * 0.1)) + min(1, 5 * 0.1) = 6.6.

[0091] Furthermore, after the scoring is completed, the scoring results of each model and the values ​​of each dimension are stored in the scoring database. This facilitates subsequent sorting of the scoring database by the scoring results, selecting the top-scoring timeout query tasks, and combining this with business priorities for analysis.

[0092] The above embodiments have described the system slow query analysis method in detail. This application also provides embodiments corresponding to the system slow query analysis device. It should be noted that this application describes the device embodiments from two perspectives: one based on functional modules and the other based on hardware.

[0093] Figure 2 A structural diagram of a system slow query analysis device provided in this application embodiment includes:

[0094] The acquisition module 10 is used to acquire the target model in the model library. The model library is a database used to store the timeout models corresponding to the timeout query tasks. The timeout query task is a query task whose query time is greater than a first threshold time obtained by using the ElasticSearch system query interface. The target model is a timeout model in the database that meets the first preset condition.

[0095] The scoring module 11 is used to analyze the target models based on the query conditions and data volume of each target model and the query task examples corresponding to each target model, and to obtain the model score.

[0096] Analysis module 12 is used to analyze target models whose model scores are higher than a threshold in order to determine the reasons for timeouts in query tasks.

[0097] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0098] This application provides a slow query analysis device, comprising: acquiring target models from a model library to analyze slow query tasks based on the target models; wherein the model library is a database for storing timeout models corresponding to timeout query tasks; timeout query tasks are query tasks whose query time exceeds a first threshold time; and target models are timeout models in the database that meet a first preset condition. The device analyzes the target models based on the query conditions and data volume of each target model, and on the corresponding query task examples, and obtains a model score to exclude query tasks that time out due to preemption of computing resources, preventing misjudgments of query tasks from interfering with subsequent analysis. Target models with model scores higher than the threshold are analyzed to determine the cause of the query task timeout. Therefore, the technical solution provided in this application analyzes and scores the models of timeout query tasks in the model library to remove misjudged models from the database, thereby facilitating the analysis and improvement of the ElasticSearch system based on actual timeout query tasks.

[0099] Figure 3 A structural diagram of a system slow query analysis device provided in another embodiment of this application is shown below. Figure 3 As shown, the slow query analysis device of the system includes: a memory 20 for storing computer programs;

[0100] The processor 21 is used to implement the steps of the slow query analysis method of the system as described in the above embodiment when executing a computer program.

[0101] The smart devices provided in this embodiment may include, but are not limited to, smartphones, tablets, laptops, or desktop computers.

[0102] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0103] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 20 is used to store at least the following computer program 201, which, after being loaded and executed by the processor 21, is capable of implementing the relevant steps of the system slow query analysis method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, and the storage method may be temporary or permanent storage. The operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data provided in the system slow query analysis method.

[0104] In some embodiments, the system slow query analysis device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0105] Those skilled in the art will understand that Figure 3 The structure shown does not constitute a limitation on the system's slow query analysis apparatus and may include more or fewer components than illustrated.

[0106] The system slow query analysis device provided in this application embodiment includes a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method:

[0107] Obtain the target model from the model library, where the model library is a database used to store timeout models corresponding to timeout query tasks, the timeout query task is a query task whose query time is greater than a first threshold time obtained by using the ElasticSearch system query interface, and the target model is a timeout model in the database that meets the first preset condition.

[0108] The target models are analyzed based on the query conditions and data volume of each target model, as well as the corresponding query task examples, and a model score is obtained.

[0109] Analyze target models with scores higher than a threshold to determine the reasons for query timeouts.

[0110] Finally, this application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiments.

[0111] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] The foregoing has provided a detailed description of a system slow query analysis method, apparatus, and medium provided in this application. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0113] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for analyzing slow queries in a system, characterized in that, include: Obtain the target model from the model library, wherein the model library is a database used to store timeout models corresponding to timeout query tasks, the timeout query task is a query task whose query time is greater than a first threshold time obtained by using the ElasticSearch system query interface, and the target model is a timeout model in the database that meets a first preset condition. The target models are analyzed based on the query conditions and data volume of each target model, as well as the query task examples corresponding to each target model, and a model score is obtained. The target models whose model scores are higher than a threshold are analyzed to determine the reasons for query timeouts. The first preset condition is: No analysis was performed within the second threshold time period; Or the change in the query sample corresponding to the target model is greater than the sample threshold.

2. The system slow query analysis method according to claim 1, characterized in that, Also includes: Retrieve the timeout query tasks in each ElasticSearch node within each detection period; Determine whether the timeout model corresponding to the timeout query task exists in the model library; If the timeout model exists, the timeout query task is stored in the sample library corresponding to the timeout model; If the timeout model does not exist, then the timeout model is stored in the model library.

3. The system slow query analysis method according to claim 1, characterized in that, The analysis of the target models based on the query conditions and data volume of each target model, and the query task examples corresponding to each target model, includes: Construct a tree structure corresponding to the target model based on the description language of the target model to determine the number of query conditions and the number of expensive query categories of the target model; Obtain a query task example corresponding to the target model, and determine the total length and data volume of the query conditions based on the tree structure and the query task example; The model score of the target model is calculated based on the number of query conditions, the number of expensive query categories, the total length of the query conditions, and the amount of data.

4. The system slow query analysis method according to claim 3, characterized in that, The calculation of the model score of the target model based on the number of query conditions, the number of expensive query categories, the total length of query conditions, and the amount of data includes: Determine the weights corresponding to the number of query conditions, the number of expensive query categories, the total length of query conditions, and the amount of data; wherein the weights of the number of expensive query categories, the amount of data, the total length of query conditions, and the number of query conditions decrease in that order.

5. The system slow query analysis method according to claim 2, characterized in that, The step of determining whether the timeout model corresponding to the timeout query task exists in the model library includes: Construct a detection model corresponding to the timeout query task, and calculate the detection identifier of the detection model, wherein the detection identifier is MD5; Determine whether the detection identifier is the same as the identifier of each timeout model; If the identifier is different from that of each of the timeout models, then it is determined that there is no timeout model in the model library corresponding to the timeout query task.

6. The system slow query analysis method according to claim 1, characterized in that, After the step of analyzing the target models whose model scores are higher than the threshold, the method further includes: optimizing the query tasks of the ElasticSearch system based on the timeout reasons.

7. A system slow query analysis device, characterized in that, include: The acquisition module is used to acquire the target model in the model library, wherein the model library is a database used to store the timeout model corresponding to the timeout query task, the timeout query task is a query task whose query time is greater than a first threshold time obtained by using the ElasticSearch system query interface, and the target model is the timeout model in the database that meets the first preset condition. The scoring module is used to analyze the target models based on the query conditions and data volume of each target model and the query task examples corresponding to each target model, and to obtain model scores. The analysis module is used to analyze the target models whose model scores are higher than a threshold in order to determine the reasons for the timeout of the query task; The first preset condition is: No analysis was performed within the second threshold time period; Or the change in the query sample corresponding to the target model is greater than the sample threshold.

8. A system slow query analysis device, characterized in that, Includes memory used to store computer programs; A processor, configured to implement the steps of the system slow query analysis method as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the system slow query analysis method as described in any one of claims 1 to 6.

Images