Performance analysis method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a performance analysis method, an electronic device and a storage medium to realize automatic analysis of abnormal queries in a database. The method comprises: obtaining a query statement and performance data thereof; determining a query template based on the query statement, analyzing the query template according to the performance data of the query statement, and determining performance statistical data of the query template; classifying the query template based on a performance threshold and the performance statistical data, and determining a performance category to which each query template belongs; and performing outlier analysis on each performance category based on the performance statistical data, and determining an abnormal query template and an abnormal reason. The cause of performance abnormality in the database can be automatically analyzed, and the analysis efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a performance analysis method, an electronic device, and a storage medium. Background Technology

[0002] An analytical database is a cloud-hosted petabyte (PB) level high-concurrency data warehouse, specifically designed to serve the OLAP (On-Line Transaction Processing) domain. It employs a relational data storage model, allowing for flexible and free computational analysis using SQL (Structured Query Language) without the need for pre-modeling.

[0003] Analytical databases handle a large number of complex data query tasks from users. Due to various factors such as normal system fluctuations, normal changes in query load, elastic scaling of cloud databases, software version upgrades, and execution plan changes, the performance of the same data query statement can vary with each execution. Analyzing the causes of these performance variations is costly and impractical due to the massive user query load. Summary of the Invention

[0004] This application provides a performance analysis method to automatically analyze abnormal queries in a database.

[0005] Accordingly, embodiments of this application also provide an electronic device and a storage medium to ensure the implementation and application of the above system.

[0006] To address the aforementioned problems, this application discloses a performance analysis method, the method comprising:

[0007] Retrieve the query statement and its performance data;

[0008] A query template is determined based on the query statement, and the query template is analyzed according to the performance data of the query statement to determine the performance statistics of the query template.

[0009] The query templates are categorized based on performance thresholds and performance statistics to determine the performance category to which each query template belongs;

[0010] Outlier analysis is performed on each performance category based on performance statistics to identify anomalous query templates and the causes of the anomalies.

[0011] Optionally, obtaining the query statement and its performance data includes:

[0012] The database determines the query statement and its performance data, which includes at least one of the following: response time information, CPU time consumption information, storage time consumption information, and memory usage information.

[0013] The performance data is used to generate the performance vector of the query statement.

[0014] Optionally, determining the query template based on the query statement includes:

[0015] Replace the constants in the query statement to obtain the query statement with the replaced constants;

[0016] The query statement that replaces constants is regularized to determine the corresponding query template.

[0017] Optionally, the step of analyzing the query template based on the performance data of the query statement to determine the performance statistics of the query template includes:

[0018] Determine each query statement corresponding to the performance template to form a query set, and determine the performance data of each query statement corresponding to the query set.

[0019] Statistical aggregation analysis is performed on the performance data of each query statement to determine the performance statistics of the query template. The statistical performance data includes response time statistics and running frequency statistics.

[0020] Optionally, the classification of query templates based on performance thresholds and performance statistics to determine the performance category to which each query template belongs includes:

[0021] Obtain performance thresholds, including response time thresholds and operating frequency thresholds;

[0022] Divide into more than one performance category based on performance thresholds;

[0023] The performance category to which each query template belongs is determined based on performance statistics.

[0024] Optionally, the outlier analysis performed on each performance category based on performance statistics to determine anomalous query templates and causes of anomalies includes:

[0025] Determine the anomaly classification model corresponding to each performance category;

[0026] The performance statistics of the performance templates corresponding to the performance categories are input into the anomaly classification model for analysis to determine the query templates and causes of anomalies.

[0027] Optional, also includes:

[0028] The query template and cause of the anomaly are verified, and the verification results are obtained.

[0029] The anomaly type is determined based on the verification results.

[0030] Optionally, the verification of the query template and cause of the anomaly includes the following verification results:

[0031] For the query template corresponding to the anomaly, analyze the anomaly change information;

[0032] Consistency analysis is performed using the aforementioned abnormal change information and causes to determine the corresponding verification results.

[0033] This application also discloses a performance analysis device, wherein the task scheduling device includes:

[0034] The data acquisition module is used to acquire query statements and their performance data;

[0035] The clustering analysis module is used to determine a query template based on the query statement, analyze the query template according to the performance data of the query statement, and determine the performance statistics of the query template.

[0036] The classification module is used to classify query templates based on performance thresholds and performance statistics, and to determine the performance category to which each query template belongs;

[0037] The anomaly analysis module is used to perform outlier analysis on each performance category based on performance statistics, and to determine the query template and cause of the anomaly.

[0038] This application also discloses an electronic device, including: a processor; and a memory storing executable code thereon, wherein when the executable code is executed by the processor, the method described in this application is performed.

[0039] This application also discloses one or more machine-readable media storing executable code thereon, which, when executed by a processor, performs the method described in this application.

[0040] Compared with the prior art, the embodiments of this application have the following advantages:

[0041] In this embodiment, query statements and their performance data from the database are obtained. Then, query templates are determined based on the query statements. The query templates are analyzed according to the performance data of the query statements to determine the performance statistics of the query templates. This allows for the clustering and statistical analysis of the query statements. The query templates are then classified based on performance thresholds and performance statistics to determine the performance category to which each query template belongs. Based on the clustering, the query templates are further classified. Outlier analysis is performed on each performance category based on the performance statistics to identify abnormal query templates and their causes. This enables automatic analysis of the causes of performance anomalies in the database, improving analysis efficiency. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the steps of an embodiment of the performance analysis method of this application;

[0043] Figure 2 This is a schematic diagram illustrating an example of query template classification according to an embodiment of this application;

[0044] Figure 3 This is a schematic diagram illustrating an example of outlier analysis according to an embodiment of this application;

[0045] Figure 4 This is a flowchart illustrating the steps of another embodiment of the performance analysis method of this application;

[0046] Figure 5 This is a schematic diagram of the structure of an exemplary device provided in one embodiment of this application. Detailed Implementation

[0047] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] The embodiments of this application can be applied to cloud-based database scenarios such as analytical databases. They enable performance analysis of database queries, allowing for the identification of performance anomalies and their causes.

[0049] Reference Figure 1 The diagram shows a flowchart of an embodiment of a performance analysis method according to this application.

[0050] Step 102: Obtain the query statement and its performance data.

[0051] It is possible to obtain database query statements and their corresponding performance data over a period of time. In some examples, obtaining query statements and their performance data includes: determining the query statements and their performance data based on the database; and generating a performance vector for the query statements using the performance data. Each user query statement and its performance data can be obtained through various methods, such as logs, based on the database kernel's monitoring mechanism. The performance data includes at least one of the following: response time information, CPU time consumption information, storage time consumption information, memory usage information, etc. The performance data of each query statement can be used to construct a performance vector. Thus, each query record can form a set, denoted as: {query_i, metrics_i}, where query_i is the query statement, such as an SQL statement, and metrics_i is the performance data vector representation.

[0052] Step 104: Determine a query template based on the query statement, analyze the query template according to the performance data of the query statement, and determine the performance statistics of the query template.

[0053] After obtaining the query statements and their performance data, the query statements can be categorized. Database queries have temporal and spatial characteristics; essentially, similar query statements will run repeatedly over a period of time and often exhibit a certain periodicity. Therefore, the queries are first clustered to form query templates, which serve as the basic units for performance monitoring and analysis. The query template is standardized information describing the structural form of the query statement; by replacing the constant parameters in the template, the corresponding query statement can be generated.

[0054] Therefore, determining the template corresponding to a query statement can be achieved by replacing constants in the query statement. In one optional embodiment, determining the query template based on the query statement includes: replacing constants in the query statement to obtain a query statement with replaced constants; performing regularization processing on the query statement with replaced constants to determine the corresponding query template. First, the constants in the query statement are determined, and then the constants are replaced with a wildcard, which can be set according to requirements, such as "?" or other symbols.

[0055] Then, regularization can be applied to the query statement with constant replacements. Regularization rules can be pre-defined, generated based on the query statement's syntax. In other examples, a syntax tree can be constructed, such as building an SQL syntax tree for the SQL query statement. In one example, the query statement with constant replacements can be used to traverse the syntax tree nodes to determine the query template. For example, for an SQL statement with constant replacements, a deterministic traversal of the SQL syntax tree nodes can be performed to form a QueryPattern template class. In other examples, the query statement with constant replacements can be regularized according to preset rules. For example, variables in the query statement with constant replacements can be sorted in a certain order, such as lexicographical order, where lexicographical order refers to the order in which they appear in a dictionary. Then, variable aliases are removed, and the keywords of the query statement are sorted according to fixed rules to obtain the corresponding query string as the query template.

[0056] Taking SQL query statements as an example, regularization can be used to replace constants in the SQL statement with wildcards, generating a Query Pattern template class.

[0057] After determining each query template, the query template can be analyzed according to the performance data of the query statements to determine the performance statistics of the query template. Specifically, each query statement corresponding to the query template and its performance data can be identified. This performance data is then used to perform performance analysis on the query template to determine the performance statistics of the query template. In one optional embodiment, analyzing the query template according to the performance data of the query statements to determine the performance statistics of the query template includes: identifying each query statement corresponding to the performance template, forming a query combination, and determining the performance data of each query statement corresponding to the query combination; performing statistical aggregation analysis on the performance data of each query statement to determine the performance statistics of the query template, wherein the statistical performance data includes response time information and running frequency information.

[0058] After analyzing the query templates for constant substitution queries, a query set can be generated for each template. Query statements belonging to that template are placed into this set, and the performance data of each query statement within the set also constitutes a performance data set. Statistical analysis is then performed on the performance data in these sets. For each type of performance data, separate statistical analysis can be conducted, analyzing the maximum (max), minimum (min), mean (avg), variance (std), and percentile for each type of performance data. For example, using the response time information of each query statement within the template, the maximum, minimum, mean, variance, and percentile of the response time can be calculated as response time statistics. Similarly, for other information such as CPU time consumption, storage time consumption, and memory usage, the corresponding maximum, minimum, mean, variance, and percentile can be calculated to determine CPU time consumption statistics, storage time consumption statistics, and memory usage statistics, respectively. A query template can correspond to multiple query statements, and each query statement may be executed n times within a certain period (e.g., a specified time interval), such as a periodic query statement. Therefore, the frequency information of the query statements can also be determined, and based on the number of times and time of all query statements, the running frequency of the query template can be counted to obtain running frequency statistics.

[0059] For example, perform statistical aggregation calculations on the performance data of all SQL queries in a Query Pattern template class. If a Query Pattern template class contains two SQL queries that may have been run n times and m times respectively, then perform statistical calculations on the performance data corresponding to these (n+m) queries to determine the statistical performance data.

[0060] This allows for the clustering of query statements, forming the basic unit for performance monitoring and analysis—the query template. Statistical analysis is then performed according to the query template to determine its temporal and spatial characteristics.

[0061] Step 106: Classify the query templates based on performance thresholds and performance statistics to determine the performance category to which each query template belongs.

[0062] Based on the query templates obtained through the above clustering, these templates can be categorized. Performance thresholds can be set based on temporal and spatial characteristics, including a response time threshold (Threshold_RT) and a runtime threshold (Threhold_Feq). These thresholds are used to categorize query templates according to their response time and runtime dimensions, respectively, to determine the performance category to which each query template belongs.

[0063] In one optional embodiment, classifying query templates based on performance thresholds and performance statistics to determine the performance category to which each query template belongs includes: obtaining performance thresholds, which include response time thresholds and running frequency thresholds; dividing more than one performance category based on performance thresholds; and determining the performance category to which each query template belongs based on performance statistics.

[0064] Based on response time and runtime thresholds, more than one performance category can be defined, such as four. The performance category of a query template is determined based on its performance statistics. In one example, the four performance categories can be defined based on response time and runtime thresholds as follows: Category 1 (greater than both response time and runtime thresholds), Category 2 (less than both response time and runtime thresholds), Category 3 (less than both response time and runtime thresholds), Category 4 (greater than both response time and runtime thresholds), and so on. In other examples, the above can also be represented by coordinates in quadrants. For each template class P_i, the average response time avg(metric_rt) within the analysis period T and the corresponding query frequency F correspond to the four quadrants of the plane. Each quadrant contains a series of query template classes, such as... Figure 2 The diagram illustrates a classification coordinate system. Query templates with both response time statistics exceeding the response time threshold and execution frequency statistics exceeding the execution frequency threshold are located in the first quadrant, classifying them as the first category—long queries that run repeatedly. Query templates with both response time statistics falling below the response time threshold and execution frequency statistics exceeding the execution frequency threshold are located in the second quadrant, classifying them as the second category—short queries that run repeatedly. Query templates with both response time statistics falling below the response time threshold and execution frequency threshold are located in the third quadrant, classifying them as the third category—short queries that run occasionally. Query templates with both response time statistics exceeding the response time threshold and execution frequency threshold are located in the fourth quadrant, classifying them as the fourth category—long queries that run occasionally.

[0065] This allows us to determine the category of each query template based on performance statistics, and then further classify the clustered queries according to performance data such as time consumption and frequency.

[0066] Step 108: Perform outlier analysis on each performance category based on performance statistics to identify abnormal query templates and the causes of the anomalies.

[0067] Under each performance category, outlier analysis can be performed on performance templates based on performance statistics. Outlier analysis can be performed through methods such as clustering to identify outlier and abnormal query templates and to analyze the reasons for the anomalies.

[0068] Unsupervised models can be constructed for classification analysis. These models can be built using various algorithms, such as weighted (Nearest-neighbor), clustering, statistical, subspace, or other classic algorithms. Weighted algorithms include K-NN clustering, Local Outlier Factor (LOF), and Connectivity-Based Outlier Factor (COF). Clustering algorithms include Cluster-based Local Outlier Factor (CBLOF) and Local Density Vluster-based Outlier Factor (LDCOF). Statistical algorithms include Histogram-based Outlier Score (HBOS). Subspace algorithms include Robust Principal Component Analysis (rPCA). Other classic algorithms can be single-class classification algorithms such as One-Class-SVM (Support Vector Machines).

[0069] This application embodiment allows for the training of unsupervised models for each performance category. Within each performance category, an unsupervised model can be constructed based on performance data across various dimensions, and trained using the corresponding model parameters and samples. Taking an unsupervised model of the HBOS algorithm as an example, the following processing can be performed on the response time dimension:

[0070] First, within two consecutive analysis periods T1 and T2, calculate the relative values ​​and percentage proportions of the changes in various performance indicators corresponding to the same query template, and denot them as vectors.<diff_metric_i> .

[0071] For example, the relative change in maximum response time is:

[0072] diff_metric_rt_max=metric_rt_max_t1–metric_rt_max_t2

[0073] The percentage change in maximum response time is:

[0074] diff_pct_metric_rt_max=(metric_rt_max_t1–metric_rt_max_t2) / metric_rt_max_t1

[0075] Then the vector can be...<diff_metric_i> For the set of values ​​across each dimension, calculate their distribution histogram to determine the threshold for outlier filtering in the unsupervised model. For example, statistically analyze the percentage distribution of the maximum response time change over two consecutive analysis periods T1 and T2 for all query templates corresponding to repeatedly run long queries in the first category (i.e., the first quadrant). Figure 3 As shown, the threshold can be determined based on the screening ratio. Figure 3 In the example, the percentage distribution of the maximum response time variation follows a normal distribution. The outlier threshold parameter can be set to N times the variance of the corresponding distribution. Figure 3 In the offline threshold, outlier thresholds are marked with dashed lines; the slashed portion represents the outliers that exceed the offline threshold.

[0076] For performance data across each dimension, distribution information is calculated using appropriate clustering algorithms. Outlier thresholds are then determined based on this distribution information, and a corresponding multi-dimensional unsupervised model is trained using these outlier thresholds and the algorithm. This unsupervised model can be used as an anomaly classification model, and anomaly analysis can be performed on query templates within that performance category based on performance statistics.

[0077] In one optional embodiment, the step of performing outlier analysis on each performance category based on performance statistics to determine anomalous query templates and causes includes: determining an anomaly classification model corresponding to each performance category; inputting the performance statistics of the performance templates corresponding to the performance categories into the anomaly classification model for analysis to determine anomalous query templates and causes. The anomaly classification model can analyze the performance parameters of each performance template, and through methods such as clustering statistics, determine query templates whose performance statistics meet outlier thresholds. Specifically, anomalous query templates can be filtered based on one or more performance thresholds, and the causes of anomalies can be determined based on the met outlier thresholds. This eliminates the need for manual analysis of each query anomaly, improving the efficiency of anomaly analysis.

[0078] In this embodiment, an anomaly classification model is constructed using an unsupervised model. Unsupervised models possess interpretability and semantic information, thus enabling the identification of abnormal query templates and causes. This embodiment also verifies the cause of the anomaly. The abnormal query template and cause can be verified, and the verification result is used to locate the anomaly type. The abnormal query statement can be identified through system investigation methods such as querying system logs, and the cause of the anomaly can be determined. This result is then matched with the result of the anomaly classification model to determine the verification result. This verification result can be anomaly matching, anomaly non-matching, etc. Based on this verification result, the anomaly type can be located, such as group anomaly, individual anomaly, etc., which can be excluded accordingly.

[0079] The process of verifying the abnormal query template and its cause includes: analyzing the change information of the query set corresponding to the abnormal query template; and performing consistency analysis using the change information and the cause of the abnormality to determine the corresponding verification result. The cause of the abnormality for each query statement corresponding to the abnormal query template can be determined through system investigation methods such as querying system logs. For ease of distinction, this is referred to as abnormal change information. System investigation methods can identify the set of query statements affected by potential factors. For example, a change in the storage structure of a database table will affect all queries using that table, thus enabling the analysis of corresponding abnormal change information, such as performance anomalies caused by storage structure changes. Then, the abnormal change information is matched with the cause of the abnormality. Consistency analysis can be performed using the abnormal change information and the cause of the abnormality to analyze whether there are consistent changes in performance, such as an increase or decrease in certain performance data, to determine the corresponding verification result, such as change matching or change mismatch verification results.

[0080] If a consistent change occurs, it's considered a group anomaly, such as the database table structure causing performance issues in all queries. Manual analysis can further pinpoint the cause of the anomaly. If the change isn't consistent, it's considered an individual anomaly. For example, if only a very small number of queries on a single table show anomalies, it's unnecessary to consider table-specific anomalies. Whether these anomalous queries share commonalities can help eliminate some of the analytical work required for manual analysis and pinpoint the cause.

[0081] In summary, this method obtains database query statements and their performance data, then determines query templates based on these statements, analyzes the query templates according to their performance data, and determines performance statistics for each template. This allows for the clustering and statistical analysis of query statements. Furthermore, query templates are categorized based on performance thresholds and statistical data to determine the performance category of each template. Further categorization is performed on the clustered templates, and outlier analysis is conducted for each performance category based on the statistical data to identify anomalous query templates and their causes. This approach automatically analyzes the causes of performance anomalies in the database, reducing manual analysis and improving efficiency.

[0082] Based on the above embodiments, this application also provides a performance analysis method that can automatically cluster query statements, analyze anomalies and their causes, and may also verify anomalies, locate common anomalies, and reduce the workload of manual analysis.

[0083] Reference Figure 4 The flowchart illustrates the steps of another embodiment of the performance analysis method of this application.

[0084] Step 402: Obtain the query statement and its performance data.

[0085] The step of obtaining the query statement and its performance data includes: determining the query statement and its performance data based on the database, wherein the performance data includes at least one of the following: response time information, CPU time consumption information, storage time consumption information, and memory usage information; and generating a performance vector of the query statement using the performance data.

[0086] Step 404: Replace the constants in the query statement to obtain the query statement with the replaced constants.

[0087] Step 406: Perform regularization processing on the query statement that replaces constants to determine the corresponding query template.

[0088] Step 408: Determine each query statement corresponding to the performance template to form a query set, and determine the performance data of each query statement corresponding to the query set.

[0089] The step of classifying query templates based on performance thresholds and performance statistics to determine the performance category to which each query template belongs includes: obtaining performance thresholds, which include response time thresholds and running frequency thresholds; dividing more than one performance category based on performance thresholds; and determining the performance category to which each query template belongs based on performance statistics.

[0090] Step 410: Perform statistical aggregation analysis on the performance data of each query statement to determine the performance statistics of the query template. The statistical performance data includes: response time statistics and running frequency statistics.

[0091] Step 412: Determine the anomaly classification model corresponding to each performance category.

[0092] Step 414: Input the performance statistics of the performance template corresponding to the performance category into the anomaly classification model for analysis to determine the query template and cause of the anomaly.

[0093] Step 416: Verify the query template and cause of the anomaly, and obtain the verification result.

[0094] The verification of the query template and cause of the anomaly includes: analyzing the abnormal change information of the query set corresponding to the query template of the anomaly; and performing consistency analysis using the abnormal change information and the cause of the anomaly to determine the corresponding verification result.

[0095] Step 418: Locate the anomaly type based on the verification result.

[0096] In this embodiment, the query load based on the database exhibits temporal and spatial locality, meaning that essentially identical query statements will run repeatedly over a period of time and often exhibit certain periodic characteristics. First, the query statements are regularized and clustered to form the basic unit for performance monitoring and analysis, namely, the query template. Then, the clustered query templates are further classified according to performance indicators such as time consumption and frequency, and a customized unsupervised model is established for each category of query templates.

[0097] Based on the fundamental assumption that most performance changes are within the bounds of normal and reasonable fluctuations, this application performs attribution analysis on query performance changes using multi-dimensional features and filters out outliers in each feature dimension—that is, data with significantly abnormal performance changes. This greatly reduces the magnitude of abnormal loads that cannot be reasonably attributed. The embodiments of this application improve the efficiency of anomaly analysis, and the unsupervised model possesses interpretability and scene semantics, facilitating further analysis and problem localization.

[0098] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0099] Based on the above embodiments, this embodiment also provides a performance analysis device, which can be applied to electronic devices, such as server-side electronic devices.

[0100] The data acquisition module is used to acquire query statements and their performance data;

[0101] The clustering analysis module is used to determine a query template based on the query statement, analyze the query template according to the performance data of the query statement, and determine the performance statistics of the query template.

[0102] The classification module is used to classify query templates based on performance thresholds and performance statistics, and to determine the performance category to which each query template belongs;

[0103] The anomaly analysis module is used to perform outlier analysis on each performance category based on performance statistics, and to determine the query template and cause of the anomaly.

[0104] In summary, this method involves acquiring database query statements and their performance data, determining query templates based on these statements, analyzing the performance data of the query templates to determine performance statistics, clustering and statistically analyzing the query statements, classifying the query templates based on performance thresholds and performance statistics to determine the performance category of each template, reclassifying the query templates based on clustering, and performing outlier analysis on each performance category based on performance statistics to identify abnormal query templates and their causes. This approach can automatically analyze the causes of performance anomalies in the database, improving analysis efficiency.

[0105] The data acquisition module is used to determine the query statement and its performance data based on the database. The performance data includes at least one of the following: response time information, CPU time consumption information, storage time consumption information, and memory usage information; and to generate a performance vector of the query statement using the performance data.

[0106] The clustering analysis module includes: a template analysis submodule and a performance statistics submodule, wherein:

[0107] The template analysis submodule is used to replace constants in the query statement to obtain a query statement with the replaced constants; and to perform regularization processing on the query statement with the replaced constants to determine the corresponding query template.

[0108] The performance statistics submodule is used to determine each query statement corresponding to the performance template, form a query set, determine the performance data of each query statement corresponding to the query set, perform statistical aggregation analysis on the performance data of each query statement, and determine the performance statistics data of the query template. The statistical performance data includes: response time statistics data and running frequency statistics data.

[0109] The classification module is used to obtain performance thresholds, which include response time thresholds and running frequency thresholds; to divide more than one performance category based on the performance thresholds; and to determine the performance category to which each query template belongs based on performance statistics.

[0110] The anomaly analysis module is used to determine the anomaly classification model corresponding to each performance category; the performance statistics of the performance template corresponding to the performance category are input into the anomaly classification model for analysis to determine the query template and cause of the anomaly.

[0111] The anomaly analysis module is also used to verify the query template and cause of the anomaly, and to locate the anomaly type based on the verification result.

[0112] The anomaly analysis module is used to analyze the anomaly change information of the query set corresponding to the query template of the anomaly.

[0113] Consistency analysis is performed using the aforementioned abnormal change information and causes to determine the corresponding verification results.

[0114] In this embodiment, the query load based on the database exhibits temporal and spatial locality, meaning that essentially identical query statements will run repeatedly over a period of time and often exhibit certain periodic characteristics. First, the query statements are regularized and clustered to form the basic unit for performance monitoring and analysis, namely, the query template. Then, the clustered query templates are further classified according to performance indicators such as time consumption and frequency, and a customized unsupervised model is established for each category of query templates.

[0115] Based on the fundamental assumption that most performance changes are within the bounds of normal and reasonable fluctuations, this application performs attribution analysis on query performance changes using multi-dimensional features and filters out outliers in each feature dimension—that is, data with significantly abnormal performance changes. This greatly reduces the magnitude of abnormal loads that cannot be reasonably attributed. The embodiments of this application improve the efficiency of anomaly analysis, and the unsupervised model possesses interpretability and scene semantics, facilitating further analysis and problem localization.

[0116] This application also provides a non-volatile readable storage medium storing one or more modules (programs). When these modules are applied to a device, they enable the device to execute the instructions for the method steps in this application.

[0117] This application provides one or more machine-readable media storing instructions that, when executed by one or more processors, cause an electronic device to perform one or more of the methods described in the above embodiments. In this application, the electronic device includes devices such as servers and terminal devices.

[0118] Embodiments of this disclosure can be implemented as an apparatus with any suitable hardware, firmware, software, or any combination thereof, configured as desired, and the apparatus may include electronic devices such as servers (clusters) and terminals. Figure 5 An exemplary apparatus 500 is schematically shown that can be used to implement the various embodiments described in this application.

[0119] In one embodiment, Figure 5 An exemplary device 500 is shown, which includes one or more processors 502, a control module (chipset) 504 coupled to at least one of the processors 502, a memory 506 coupled to the control module 504, a non-volatile memory (NVM) / storage device 508 coupled to the control module 504, one or more input / output devices 510 coupled to the control module 504, and a network interface 512 coupled to the control module 504.

[0120] Processor 502 may include one or more single-core or multi-core processors, and processor 502 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, device 500 can serve as a server, terminal, or other device as described in the embodiments of this application.

[0121] In some embodiments, the apparatus 500 may include one or more computer-readable media (e.g., memory 506 or NVM / storage device 508) having instructions 514 and one or more processors 502 that are combined with the one or more computer-readable media and configured to execute the instructions 514 to implement the module and thus perform the actions described in this disclosure.

[0122] In one embodiment, the control module 504 may include any suitable interface controller to provide any suitable interface to at least one of the processors 502 and / or any suitable device or component communicating with the control module 504.

[0123] The control module 504 may include a memory controller module to provide an interface to the memory 506. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0124] Memory 506 may be used, for example, to load and store data and / or instructions 514 for device 500. In one embodiment, memory 506 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, memory 506 may include double data rate type quad synchronous dynamic random access memory (DDR5 SDRAM).

[0125] In one embodiment, the control module 504 may include one or more input / output controllers to provide an interface to the NVM / storage device 508 and (one or more) input / output devices 510.

[0126] For example, NVM / storage device 508 may be used to store data and / or instructions 514. NVM / storage device 508 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives (HDDs), one or more optical disc drives (CDs), and / or one or more digital universal optical disc (DVD) drives).

[0127] NVM / storage device 508 may include storage resources that are part of a device on which device 500 is mounted, or that are accessible to the device but do not necessarily have to be part of the device. For example, NVM / storage device 508 may be accessed via a network through one or more input / output devices 510.

[0128] One or more input / output devices 510 may provide an interface for device 500 to communicate with any other suitable device. Input / output devices 510 may include communication components, audio components, sensor components, etc. A network interface 512 may provide an interface for device 500 to communicate via one or more networks. Device 500 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, etc., or combinations thereof.

[0129] In one embodiment, at least one of the processors 502 may be logically packaged with one or more controllers (e.g., memory controller modules) of the control module 504. In one embodiment, at least one of the processors 502 may be logically packaged with one or more controllers of the control module 504 to form a system-in-package (SiP). In one embodiment, at least one of the processors 502 may be integrated with the logic of one or more controllers of the control module 504 on the same die. In one embodiment, at least one of the processors 502 may be integrated with the logic of one or more controllers of the control module 504 on the same die to form a system-on-a-chip (SoC).

[0130] In various embodiments, device 500 may be, but is not limited to, a terminal device such as a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, device 500 may have more or fewer components and / or different architectures. For example, in some embodiments, device 500 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0131] The detection device can use a main control chip as a processor or control module, and sensor data, position information, etc. can be stored in a memory or NVM / storage device. The sensor group can be used as an input / output device, and the communication interface can include a network interface.

[0132] This application also provides an electronic device, including: a processor; and a memory storing executable code thereon. When the executable code is executed, the processor performs one or more methods as described in this application embodiment. In this application embodiment, the memory can store various types of data, such as target files, file-application association data, and user behavior data, thereby providing a data foundation for various processing operations.

[0133] This application also provides one or more machine-readable media having executable code stored thereon, which, when executed, causes a processor to perform one or more of the methods described in this application.

[0134] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0135] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0136] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0139] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0140] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0141] The above provides a detailed description of a performance analysis method and apparatus, an electronic device, and a storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A performance analysis method, the method comprising: The database determines the query statement and its performance data, which includes at least one of the following: response time information, CPU time consumption information, storage time consumption information, and memory usage information. Based on the query statement, a query template is determined, and each query statement corresponding to the query template is determined to form a query set of the query template. The performance data of each query statement corresponding to the query set is determined. The query template is the basic unit for performance monitoring and analysis. Statistical aggregation analysis is performed on the performance data of each query statement to determine the performance statistics of the query template. The performance statistics include: response time statistics and running frequency statistics. The query templates are categorized based on performance thresholds and performance statistics to determine the performance category to which each query template belongs; Outlier analysis is performed on each performance category based on performance statistics to identify anomalous query templates and the causes of the anomalies.

2. The method according to claim 1, wherein, Also includes: The performance data is used to generate the performance vector of the query statement.

3. The method according to claim 1, wherein, The process of determining the query template based on the query statement includes: Replace the constants in the query statement to obtain the query statement with the replaced constants; The query statement that replaces constants is regularized to determine the corresponding query template.

4. The method according to claim 3, wherein, The process of classifying query templates based on performance thresholds and performance statistics to determine the performance category to which each query template belongs includes: Obtain performance thresholds, including response time thresholds and operating frequency thresholds; Divide into more than one performance category based on performance thresholds; The performance category to which each query template belongs is determined based on performance statistics.

5. The method according to claim 1, wherein, The outlier analysis performed on each performance category based on performance statistics to identify anomalous query templates and their causes includes: Determine the anomaly classification model corresponding to each performance category; The performance statistics of the performance templates corresponding to the performance categories are input into the anomaly classification model for analysis to determine the query templates and causes of anomalies.

6. The method according to claim 5, wherein, Also includes: The query template and cause of the anomaly are verified, and the verification results are obtained. The anomaly type is determined based on the verification results.

7. The method according to claim 6, wherein, The verification of the query template and cause of the anomaly includes the following verification results: For the query template corresponding to the anomaly, analyze the anomaly change information; Consistency analysis is performed using the aforementioned abnormal change information and causes to determine the corresponding verification results.

8. An electronic device, comprising: processor; And a memory, on which executable code is stored, which, when executed by a processor, performs the method as described in any one of claims 1-7.

9. One or more machine-readable media having executable code stored thereon, which, when executed by a processor, performs the method as described in any one of claims 1-7.

Images