Evaluation method, device and storage medium for database performance testing

Abstract

The present application discloses an evaluation method, device, and storage medium for database performance testing. The method includes: determining a set of evaluation objects corresponding to the database to be tested, and based on the test requirements, determining the performance indicator system to be tested and the data indicator system to be tested; determining the performance indicator weight vector corresponding to the performance indicator system to be tested; determining the subjective data indicator weight vector and the objective data indicator weight vector corresponding to the data indicator system to be tested; based on the subjective data indicator weight vector and the objective data indicator weight vector, determining the data indicator weight vector corresponding to the data indicator system to be tested; based on the data indicator weight vector, determining the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested, and determining the comprehensive evaluation index of the database to be tested based on the performance evaluation index and the performance indicator weight vector. Through the above method, the present application implements a set of performance test evaluation solutions that can select a suitable database for the application system.

Description

Technical Field

[0001] The present application relates to the technical field of database testing, and in particular to an evaluation method, device, and storage medium for database performance testing. Background Art

[0002] Database performance comparison testing uses automated testing tools to simulate various normal, peak, and abnormal load conditions to test various database performance indicators. The main types of performance testing include load testing and stress testing. Load testing measures the changes in various system performance indicators as the load gradually increases, with the goal of determining system performance under various workloads. Stress testing tests the changes in various system performance indicators under high load conditions, as well as functional risks and stability under peak loads.

[0003] For application systems, database performance directly impacts the performance of the application, and different databases offer varying performance benefits. Database performance comparison testing can help you select the appropriate database for your application, providing proactive guidance for improving and enhancing application performance while also reliably ensuring the quality of your application.

[0004] To select the right database for an application system, a comprehensive database performance testing solution should be applied to the databases being compared under the same specific conditions and based on real-world data. However, there is currently no suitable database performance testing evaluation solution that can be used to select the right database for an application system. Summary of the Invention

[0005] The embodiments of the present application provide an evaluation method, device, and storage medium for database performance testing, which solves the technical problem that there is currently no suitable evaluation solution for database performance testing and can select a suitable database for an application system.

[0006] In a first aspect, an embodiment of the present application provides an evaluation method for database performance testing, characterized in that the method includes: determining a set of evaluation objects corresponding to the database to be tested, and based on the test requirements, determining a performance indicator system to be tested and a data indicator system to be tested; wherein the performance indicator system to be tested includes performance indicators for evaluating the database to be tested; the data indicator system to be tested includes data indicators for evaluating single performance; performance indicators include: query performance, deletion performance, update performance, and insertion performance; data indicators include: average response time, average throughput, average CPU usage, average memory occupancy, and abnormality rate; based on a preset performance indicator related judgment matrix, an algorithm is constructed through a preset performance weight vector to determine The performance indicator weight vector corresponding to the performance indicator system to be tested; based on the preset data indicator related judgment matrix, an algorithm is constructed through the preset data weight vector to determine the subjective data indicator weight vector corresponding to the data indicator system to be tested, and based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined; based on the subjective data indicator weight vector and the objective data indicator weight vector, the data indicator weight vector corresponding to the data indicator system to be tested is determined; based on the data indicator weight vector, the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested is determined, and based on the performance evaluation index and the performance indicator weight vector, the comprehensive evaluation index of the database to be tested is determined.

[0007] In one implementation of the present application, before determining the performance indicator weight vector corresponding to the performance indicator system to be measured based on a preset performance indicator related judgment matrix and using a preset performance weight vector construction algorithm, the method further includes: determining the relative importance value of each performance indicator relative to other performance indicators based on a preset judgment matrix element value table; and constructing a performance indicator related judgment matrix based on the relative importance values, which is expressed by the following formula:

[0008]

[0009] Among them, Λ is the performance index related judgment matrix, λ i,j Indicates performance index I i For performance index I j The relative importance of λ i,j λ j,i =1(,j=1,2,3,4).

[0010] In one implementation of the present application, the performance weight vector construction algorithm is represented by the following formula:

[0011]

[0012] Where W is the performance indicator weight vector, α is the construction vector, and α is expressed by the following formula:

[0013]

[0014] In one implementation of the present application, after determining the performance indicator weight vector corresponding to the performance indicator system to be measured, the method also includes: performing a rationality evaluation on the performance indicator weight vector, specifically including: calculating the consistency value of the performance indicator related judgment matrix and the performance indicator weight vector through a preset consistency indicator calculation formula; based on the consistency value, determining the consistency ratio of the performance indicator related judgment matrix and the performance indicator weight vector, and when the consistency ratio is less than a preset threshold, determining that the performance indicator weight vector passes the rationality evaluation.

[0015] In one implementation of the present application, the consistency index calculation formula is expressed by the following formula:

[0016]

[0017] Among them, CI is the consistency index, W is the performance index weight vector, Λ is the performance index correlation judgment matrix, w i is the i-th performance indicator weight coefficient in the performance indicator weight vector;

[0018] The consistency ratio is calculated by the following formula:

[0019]

[0020] Among them, CR is the consistency ratio and CI is the consistency index.

[0021] In one implementation of the present application, based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined, specifically including: constructing a measured data matrix based on the measured data obtained from the performance test of the database to be tested; performing a normalization transformation on the measured data matrix to obtain a corresponding normalized matrix, and determining the corresponding indicator proportion matrix based on the normalized matrix; based on the indicator proportion matrix, determining the information entropy of the data indicator through a preset data indicator information entropy calculation formula; based on the information entropy of the data indicator, determining the objective data indicator weight vector corresponding to the data indicator system to be tested through a preset objective data indicator weight vector calculation formula.

[0022] In one implementation of the present application, a normalization transformation is performed on the measured data matrix, which is implemented by the following formula:

[0023]

[0024] Among them, Θ k is the normalized matrix, Represents the database object D to be testedi In terms of performance index I k Corresponding data indicators during the process Measured data of

[0025] Performance Index I k The corresponding measured data matrix is expressed by the following formula:

[0026]

[0027] Among them, k represents the measured data matrix;

[0028] The indicator weight matrix is determined by the following formula:

[0029]

[0030] Among them, k is the indicator weight matrix;

[0031] The calculation formula for data indicator information entropy is expressed by the following formula:

[0032]

[0033] in, is the information entropy of the data indicator, if Then in the calculation, let

[0034] In one implementation of the present application, the performance evaluation index corresponding to the performance indicator is determined by the following formula:

[0035]

[0036] Among them, p(I k ,D i ) is the database to be tested D i Performance Indicators I k Corresponding performance evaluation index; Performance index I k Corresponding data indicator weight vector W k The j-th data indicator weight coefficient in ;

[0037] The comprehensive evaluation index of the database to be tested is determined by the following formula:

[0038]

[0039] Among them, p(D i ) represents the comprehensive evaluation index corresponding to the i-th database to be tested, w j is the jth performance indicator weight coefficient in the performance indicator weight vector.

[0040] In a second aspect, an embodiment of the present application also provides an evaluation device for database performance testing, characterized in that the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: determine the set of evaluation objects corresponding to the database to be tested, and based on the test requirements, determine the performance indicator system to be tested and the data indicator system to be tested; wherein the performance indicator system to be tested includes performance indicators for evaluating the database to be tested; the data indicator system to be tested includes data indicators for evaluating single performance; the performance indicators include: query performance, deletion performance, update performance, and insertion performance; the data indicators include: average response time, average throughput, average CPU usage, and average memory occupancy , abnormality rate; based on the preset performance indicator related judgment matrix, the algorithm is constructed through the preset performance weight vector to determine the performance indicator weight vector corresponding to the performance indicator system to be tested; based on the preset data indicator related judgment matrix, the algorithm is constructed through the preset data weight vector to determine the subjective data indicator weight vector corresponding to the data indicator system to be tested, and based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined; based on the subjective data indicator weight vector and the objective data indicator weight vector, the data indicator weight vector corresponding to the data indicator system to be tested is determined; based on the data indicator weight vector, the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested is determined, and based on the performance evaluation index and the performance indicator weight vector, the comprehensive evaluation index of the database to be tested is determined.

[0041] On the third aspect, the embodiment of the present application also provides a non-volatile computer storage medium for evaluating database performance testing, which stores computer executable instructions, characterized in that the computer executable instructions are set to: determine the evaluation object set corresponding to the database to be tested, and based on the test requirements, determine the performance indicator system to be tested and the data indicator system to be tested; wherein the performance indicator system to be tested includes performance indicators for evaluating the database to be tested; the data indicator system to be tested includes data indicators for evaluating single performance; performance indicators include: query performance, deletion performance, update performance, and insertion performance; data indicators include: average response time, average throughput, average CPU usage, average memory occupancy, and abnormality rate; based on a preset performance indicator related judgment matrix, through pre- A performance weight vector construction algorithm is set to determine the performance indicator weight vector corresponding to the performance indicator system to be tested; based on the preset data indicator related judgment matrix, the subjective data indicator weight vector corresponding to the data indicator system to be tested is determined through the preset data weight vector construction algorithm, and based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined; based on the subjective data indicator weight vector and the objective data indicator weight vector, the data indicator weight vector corresponding to the data indicator system to be tested is determined; based on the data indicator weight vector, the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested is determined, and based on the performance evaluation index and the performance indicator weight vector, the comprehensive evaluation index of the database to be tested is determined.

[0042] The embodiments of the present application provide an evaluation method, device, and storage medium for database performance testing, which have the following beneficial effects:

[0043] 1. This method is based on Jmeter measured data and can complete the standardized processing of measured data. It can fully integrate the measured data and deeply mine the information contained therein to improve the utilization rate of data.

[0044] 2. This method establishes a systematic and hierarchical multi-level indicator evaluation system according to the test requirements and combines the hierarchical analysis method and the entropy weight method to determine the weight coefficients of each level of the evaluation system. It can not only reflect the subjective intention of the decision maker, but also make full use of objective measurement data. It can more scientifically quantify the importance of each indicator and improve the rationality and accuracy of the performance comparison test results.

[0045] 3. This method completes the quantitative scoring of the database in multiple performance tests by constructing a reasonable indicator evaluation system and combining it with an effective weight calculation method, achieving an objective, reasonable and comprehensive evaluation of the database, providing scientific reference and positive guidance for the application system to select a suitable database, improve and enhance the performance of the application system, and ensure and increase the quality and reliability of the application system. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 A flowchart of an evaluation method for database performance testing provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the internal structure of an evaluation device for database performance testing provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0050] Comparing performance between different databases requires the use of automated testing tools. Currently, there are many relevant testing tools, with Jmeter being a prominent database testing tool, primarily used for performance testing. It is also mature in implementing various interface calls and offers excellent support for common interfaces such as HTTP(S), WebService, JDBC, JAVA, and FTP. As open source software, Jmeter is highly extensible and versatile, supporting numerous plug-ins, enabling performance testing of various applications or services, a graphical interface and command-line mode, and multiple ways to present performance test results, including graphical analysis. Using Jmeter for database performance comparison testing is primarily aimed at obtaining time-sensitive data (response time) and resource-sensitive data (throughput, CPU, and memory usage) for different databases under specific conditions. This provides reasonable data support for subsequent database performance comparisons. To ensure the rigor of performance testing, consistency in the test environment and test conditions is essential, primarily including the operating system, application software version, hardware configuration, test data size, concurrency settings, and the process of increasing and changing concurrency during testing.

[0051] After using JMeter to perform performance tests on different databases and obtain real-world data, it's necessary to determine an indicator evaluation system and corresponding weighting coefficients. When using some indicators to comprehensively evaluate different databases, not all indicators are equally important. In a comprehensive evaluation model, the weighting coefficient reflects the importance of the evaluation indicator; a larger weighting coefficient indicates greater importance. There are three main methods for determining indicator weighting coefficients: subjective weighting, objective weighting, and combined integrated weighting. The Analytic Hierarchy Process (AHP) is a subjective weighting method. It is a systematic, hierarchical analysis method that combines qualitative and quantitative analysis. Its key feature is that it transforms human judgment into a comparison of the importance of several factors through the establishment of a hierarchical structure, thereby transforming difficult-to-quantify qualitative judgments into actionable comparisons of importance. The weighting coefficients of the indicators are then determined through further calculation. The Entropy Weighting Method (EWM) is an objective weighting method. Its key feature is that it calculates the entropy weight of each indicator based on the degree of dispersion of the data by processing known data and using information entropy to obtain a more objective indicator weight. Based on the idea of combined integrated property rights confirmation method, the property rights confirmation scheme formed by combining the hierarchical analysis method and the entropy weight method can not only reflect the subjective needs and wishes of decision makers, but also make full use of actual measurement data and objectively reflect the importance of each indicator.

[0052] The embodiments of the present application provide an evaluation method, device, and storage medium for database performance testing, which solves the technical problem that there is currently no suitable evaluation solution for database performance testing and can select a suitable database for an application system.

[0053] The technical solutions proposed in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0054] Figure 1 This is a flow chart of an evaluation method for database performance testing provided in an embodiment of the present application. Figure 1 As shown, an evaluation method for database performance testing provided in an embodiment of the present application specifically includes the following steps:

[0055] Step 101: Determine the evaluation object set corresponding to the database to be tested, and based on the test requirements, determine the performance indicator system to be tested and the data indicator system to be tested.

[0056] In one embodiment of the present application, the evaluation object set consisting of n databases to be tested is D={D1, D2, ..., D n}, where D i Represents different database objects to be tested (i=1, 2, ..., n).

[0057] In one embodiment of the present application, based on the test requirements, the performance indicator system to be tested includes performance indicators for evaluating the database to be tested; the data indicator system to be tested includes data indicators for evaluating single performance; the performance indicators include: query performance, deletion performance, update performance, and insert performance; the data indicators include: average response time, average throughput, average CPU usage, average memory occupancy, and exception rate.

[0058] In one embodiment of the present application, the performance indicator system to be measured can be expressed as I = {I1, I2, I3, I4}, where I1, I2, I3, and I4 represent the query performance, deletion performance, update performance, and insertion performance of the database, respectively. The performance indicator weight vector of the performance indicator system to be measured can be expressed as

[0059] W = [w1, w2, w3, w4] T ,

[0060] where w j ≥0 indicates performance index I j The weight coefficients (j=1, 2, 3, 4) when evaluating the database and satisfying

[0061] In one embodiment of the present application, the index system of the data to be measured can be expressed as in They represent average response time, average throughput, average CPU usage, average memory usage, and exception rate respectively.

[0062] It should be noted that among the above five data indicators, the abnormality rate of all tested databases in a single performance test may be zero, which results in the abnormality rate data indicator not being able to play its due evaluation role in evaluating the single performance test. In this case, abandoning this data indicator will not affect the final performance comparison test. Therefore, without loss of generality, when describing the solution below, it can be assumed that the abnormality rate of all tested databases in a single performance test is not zero. This performance indicator system to be tested is used to evaluate performance indicator I k The data indicator weight vector can be expressed as

[0063]

[0064] in Represents data indicators In evaluating individual performance test I k The weight coefficient when (j = 1, 2, 3, 4, 5), and satisfy

[0065] Step 102: Based on a preset performance indicator related judgment matrix, a preset performance weight vector construction algorithm is used to determine a performance indicator weight vector corresponding to the performance indicator system to be measured.

[0066] In one embodiment of the present application, after determining the performance indicator system to be measured, based on a preset performance indicator related judgment matrix, a preset performance weight vector construction algorithm is used to determine the performance indicator weight vector corresponding to the performance indicator system to be measured.

[0067] Specifically, based on the performance indicator system I to be tested, the performance indicator related judgment matrix is determined by scoring by experts or testers:

[0068]

[0069] where λ i,j Indicates that when evaluating the database to be tested, the performance index I i For performance index I j The relative importance value of can only be selected from the preset judgment matrix element value table. From the value characteristics of the elements in the performance index related judgment matrix Λ, it can be seen that λ i,j λ j,i =1(,j=1,2,3,4), so we only need to determine the 6 elements in the upper triangle of the performance index related judgment matrix Λ to determine the entire matrix.

[0070] The judgment matrix element value table is as follows:

[0071]

[0072] Furthermore, the construction vector α is determined based on the performance index correlation judgment matrix Λ, and α is expressed by the following formula:

[0073]

[0074] Furthermore, based on the construction vector α, the performance indicator weight vector is determined through the performance weight vector construction algorithm. The performance weight vector construction algorithm is expressed by the following formula:

[0075]

[0076] In one embodiment of the present application, since the above-mentioned evaluation process involves multiple indicators, in the process of judging the importance of the indicators, that is, when determining the performance indicator-related judgment matrix Λ, inconsistent judgment results are likely to occur. Therefore, after determining the performance indicator weight vector W, a consistency check is required to evaluate the rationality of the performance indicator weight vector determination process.

[0077] In one embodiment of the present application, the consistency check specifically includes: calculating the consistency value of the performance indicator related judgment matrix and the performance indicator weight vector through a preset consistency indicator calculation formula; based on the consistency value, determining the consistency ratio of the performance indicator related judgment matrix and the performance indicator weight vector, and when the consistency ratio is less than a preset threshold, determining that the performance indicator weight vector passes the rationality assessment, otherwise it is necessary to correct the performance indicator related judgment matrix to re-determine the performance indicator weight vector.

[0078] In one implementation of the present application, the consistency index calculation formula is expressed by the following formula:

[0079]

[0080] Among them, CI is the consistency index, W is the performance index weight vector, Λ is the performance index correlation judgment matrix, w i is the i-th performance indicator weight coefficient in the performance indicator weight vector;

[0081] The consistency ratio is calculated by the following formula:

[0082]

[0083] Among them, CR is the consistency ratio and CI is the consistency index.

[0084] Step 103: Based on the preset data indicator related judgment matrix, the algorithm is constructed through the preset data weight vector to determine the subjective data indicator weight vector corresponding to the data indicator system to be tested, and based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined.

[0085] In one embodiment of the present application, after determining the data indicator system to be measured, firstly, based on the preset data indicator related judgment matrix, an algorithm is constructed through a preset data weight vector to determine the subjective data indicator weight vector corresponding to the data indicator system to be measured.

[0086] Specifically, according to the indicator system of the data to be tested Determine the data indicator related judgment matrix through scoring by experts or testers:

[0087]

[0088] in, Indicates that the performance index I k When, data indicators For indicators The relative importance of the value can only be selected from the judgment matrix element value table. It is worth noting that only the matrix Λk The values of the 10 elements in the upper triangle can determine the value of the entire matrix.

[0089] Furthermore, based on the data indicator correlation judgment matrix Λ k Determine the data index related construction vector β k , β k It is expressed by the following formula:

[0090]

[0091] Furthermore, the vector β is constructed based on the correlation of data indicators. k , through the data weight vector construction algorithm, determine the subjective data indicator weight vector; the data weight vector construction algorithm is expressed by the following formula:

[0092]

[0093] Similarly, it is necessary to conduct a rationality assessment of the process of determining the subjective data indicator weight vector, specifically including: calculating the data consistency value of the data indicator related judgment matrix and the subjective data indicator weight vector through the preset data consistency indicator calculation formula; based on the data consistency value, determining the data consistency ratio of the data indicator related judgment matrix and the subjective data indicator weight vector, and when the data consistency ratio is less than the preset threshold, determining that the subjective data indicator weight vector has passed the rationality assessment, otherwise it is necessary to revise the data indicator related judgment matrix to re-determine the subjective data indicator weight vector.

[0094] In one implementation of the present application, the data consistency index calculation formula is expressed by the following formula:

[0095]

[0096] Among them, CI k is the data consistency indicator, is the weight vector of subjective data indicators, Λ k is the data indicator related judgment matrix, is the i-th data indicator weight coefficient in the data indicator weight vector;

[0097] The data consistency ratio is calculated by the following formula:

[0098]

[0099] Among them, CR k is the data consistency ratio, CI k It is an indicator of data consistency.

[0100] In one embodiment of the present application, after determining the subjective data indicator weight vector, it is also necessary to determine the objective data indicator weight vector corresponding to the data indicator system to be tested based on the measured data obtained from the performance test of the database to be tested.

[0101] Specifically, Jmeter is used to test the database object to be tested according to the four performance indicators in the performance indicator system I to be tested, and the test data indicator system is collected respectively. The five types of measured data shown in the figure are used to construct the measured data matrix:

[0102]

[0103] Among them, k represents the measured data matrix, Represents the database object D to be tested i In terms of performance index I k Corresponding data indicators during the process Measured data.

[0104] Furthermore, the measured data matrix is normalized to obtain the corresponding normalized matrix:

[0105]

[0106] Among them, Θ k is a standardized matrix.

[0107] Furthermore, based on the standardized matrix, the corresponding indicator weight matrix is determined:

[0108]

[0109] Among them, k is the indicator weight matrix;

[0110] Furthermore, based on the indicator weight matrix, the information entropy of the data indicator is determined by a preset data indicator information entropy calculation formula; wherein the data indicator information entropy calculation formula is expressed by the following formula:

[0111]

[0112] in, is the information entropy of the data indicator, if Then in the calculation, let

[0113] Furthermore, based on the information entropy of the data indicators, the objective data indicator weight vector corresponding to the data indicator system to be measured is determined by a preset objective data indicator weight vector calculation formula; wherein the objective data indicator weight vector calculation formula is expressed by the following formula:

[0114]

[0115] It is understandable that Performance index I k Corresponding data indicator system to be tested The corresponding objective data indicator weight vector.

[0116] Step 104: Determine the data indicator weight vector corresponding to the data indicator system to be measured based on the subjective data indicator weight vector and the objective data indicator weight vector.

[0117] In one embodiment of the present application, after determining the subjective data indicator weight vector and the objective data indicator weight vector, the data indicator weight vector corresponding to the data indicator system to be measured is determined by the following formula:

[0118]

[0119] Step 105: Determine the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested based on the data indicator weight vector, and determine the comprehensive evaluation index of the database to be tested based on the performance evaluation index and the performance indicator weight vector.

[0120] In one embodiment of the present application, after determining the performance indicator weight vector and the data indicator weight vector, the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested is determined based on the data indicator weight vector, and the comprehensive evaluation index of the database to be tested is determined based on the performance evaluation index and the performance indicator weight vector; wherein the performance evaluation index corresponding to the performance indicator is determined by the following formula:

[0121]

[0122] Among them, p(I k ,D i ) is the database to be tested D i Performance Indicators I k Corresponding performance evaluation index; Performance index I k Corresponding data indicator weight vector W k The j-th data indicator weight coefficient in ;

[0123] The comprehensive evaluation index of the database to be tested is determined by the following formula:

[0124]

[0125] Among them, p(D i ) represents the comprehensive evaluation index corresponding to the i-th database to be tested, w jis the jth performance indicator weight coefficient in the performance indicator weight vector.

[0126] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, this application embodiment also provides an evaluation device for database performance testing, the structure of which is as follows: Figure 2 shown.

[0127] Figure 2 This is a schematic diagram of the internal structure of an evaluation device for database performance testing provided in an embodiment of the present application. Figure 2 As shown, the equipment includes:

[0128] at least one processor 201;

[0129] and, a memory 202 communicatively coupled to the at least one processor;

[0130] The memory 202 stores instructions that can be executed by at least one processor, and the instructions are executed by the at least one processor 201 to enable the at least one processor 201 to:

[0131] Determine the set of evaluation objects corresponding to the database to be tested, and based on the test requirements, determine the performance indicator system and data indicator system to be tested; the performance indicator system to be tested includes performance indicators for evaluating the database to be tested; the data indicator system to be tested includes data indicators for evaluating individual performance items; performance indicators include: query performance, delete performance, update performance, and insert performance; data indicators include: average response time, average throughput, average CPU usage, average memory occupancy, and exception rate;

[0132] Based on the preset performance indicator related judgment matrix, an algorithm is constructed through a preset performance weight vector to determine the performance indicator weight vector corresponding to the performance indicator system to be tested;

[0133] Based on the preset data indicator correlation judgment matrix, the algorithm is constructed through the preset data weight vector to determine the subjective data indicator weight vector corresponding to the data indicator system to be tested, and based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined;

[0134] Determine the data indicator weight vector corresponding to the data indicator system to be measured based on the subjective data indicator weight vector and the objective data indicator weight vector;

[0135] Based on the data indicator weight vector, the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested is determined, and based on the performance evaluation index and the performance indicator weight vector, the comprehensive evaluation index of the database to be tested is determined.

[0136] Some embodiments of the present application provide corresponding Figure 1 A non-volatile computer storage medium for evaluating database performance testing stores computer executable instructions, wherein the computer executable instructions are configured as follows:

[0137] Determine the set of evaluation objects corresponding to the database to be tested, and based on the test requirements, determine the performance indicator system and data indicator system to be tested; the performance indicator system to be tested includes performance indicators for evaluating the database to be tested; the data indicator system to be tested includes data indicators for evaluating individual performance items; performance indicators include: query performance, delete performance, update performance, and insert performance; data indicators include: average response time, average throughput, average CPU usage, average memory occupancy, and exception rate;

[0138] Based on the preset performance indicator related judgment matrix, an algorithm is constructed through a preset performance weight vector to determine the performance indicator weight vector corresponding to the performance indicator system to be tested;

[0139] Based on the preset data indicator correlation judgment matrix, the algorithm is constructed through the preset data weight vector to determine the subjective data indicator weight vector corresponding to the data indicator system to be tested, and based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined;

[0140] Determine the data indicator weight vector corresponding to the data indicator system to be measured based on the subjective data indicator weight vector and the objective data indicator weight vector;

[0141] Based on the data indicator weight vector, the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested is determined, and based on the performance evaluation index and the performance indicator weight vector, the comprehensive evaluation index of the database to be tested is determined.

[0142] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from the other embodiments. In particular, the IoT device and media embodiments are generally similar to the method embodiments, so their description is relatively simple. For relevant portions, refer to the description of the method embodiments.

[0143] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0144] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0145] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0148] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0149] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0150] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0151] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0152] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for evaluating database performance testing, characterized in that: The method comprises: Determine the set of evaluation objects corresponding to the database to be tested, and based on the test requirements, determine the performance indicator system to be tested and the data indicator system to be tested; wherein the performance indicator system to be tested includes performance indicators for evaluating the database to be tested; the data indicator system to be tested includes data indicators for evaluating individual performance items; the performance indicators include: query performance, deletion performance, update performance, and insertion performance; the data indicators include: average response time, average throughput, average CPU usage, average memory occupancy, and exception rate; Based on the preset performance indicator related judgment matrix, an algorithm is constructed through a preset performance weight vector to determine the performance indicator weight vector corresponding to the performance indicator system to be tested; Based on the preset data indicator correlation judgment matrix, the algorithm is constructed through the preset data weight vector to determine the subjective data indicator weight vector corresponding to the data indicator system to be tested, and based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined; Determining a data indicator weight vector corresponding to the data indicator system to be measured based on the subjective data indicator weight vector and the objective data indicator weight vector; Based on the data indicator weight vector, the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested is determined, and based on the performance evaluation index and the performance indicator weight vector, the comprehensive evaluation index of the database to be tested is determined.

2. The evaluation method for database performance testing according to claim 1, characterized in that: Before determining the performance indicator weight vector corresponding to the performance indicator system to be measured based on the preset performance indicator correlation judgment matrix and using the preset performance weight vector construction algorithm, the method further includes: Based on the preset judgment matrix element value table, determine the relative importance of each performance indicator relative to other performance indicators; Based on the relative importance values, the performance indicator correlation judgment matrix is constructed, which is expressed by the following formula: Among them, Λ is the performance index related judgment matrix, λ i,j Indicates performance index I i For performance index I j The relative importance of λ i,j λ j,i =1(i,j=1,2,3,4).

3. The evaluation method for database performance testing according to claim 2, characterized in that: The performance weight vector construction algorithm is expressed by the following formula: Where W is the performance indicator weight vector, α is the construction vector, and α is expressed by the following formula:

4. The evaluation method for database performance testing according to claim 3, characterized in that: After determining the performance indicator weight vector corresponding to the performance indicator system to be measured, the method further includes: Performing a rationality assessment on the performance indicator weight vector, specifically including: Calculate the consistency value of the performance indicator correlation judgment matrix and the performance indicator weight vector using a preset consistency indicator calculation formula; Based on the consistency value, a consistency ratio between the performance indicator correlation judgment matrix and the performance indicator weight vector is determined, and when the consistency ratio is less than a preset threshold, it is determined that the performance indicator weight vector passes the rationality evaluation.

5. The evaluation method for database performance testing according to claim 4, characterized in that: The consistency index calculation formula is expressed by the following formula: Among them, CI is the consistency index, W is the performance index weight vector, Λ is the performance index correlation judgment matrix, w i is the i-th performance indicator weight coefficient in the performance indicator weight vector; The consistency ratio is calculated by the following formula: Among them, CR is the consistency ratio and CI is the consistency index.

6. The evaluation method for database performance testing according to claim 1, characterized in that: Based on the measured data obtained from the performance test of the database to be tested, determine the objective data indicator weight vector corresponding to the data indicator system to be tested, including: Construct a measured data matrix based on the measured data obtained from the performance test of the database to be tested; Performing a standardization transformation on the measured data matrix to obtain a corresponding standardized matrix, and determining a corresponding indicator weight matrix based on the standardized matrix; Based on the indicator weight matrix, the information entropy of the data indicator is determined by a preset data indicator information entropy calculation formula; Based on the information entropy of the data indicators, the objective data indicator weight vector corresponding to the data indicator system to be measured is determined through a preset objective data indicator weight vector calculation formula.

7. The evaluation method for database performance testing according to claim 6, characterized in that: The measured data matrix is subjected to a standardized transformation, which is achieved by the following formula: Among them, Θ k is the normalized matrix, Represents the database object D to be tested i In terms of performance index I k Corresponding data indicators during the process Measured data of Performance Index I k The corresponding measured data matrix is expressed by the following formula: Among them, k represents the measured data matrix; The indicator weight matrix is determined by the following formula: Among them, k is the indicator weight matrix; The calculation formula for data indicator information entropy is expressed by the following formula: in, is the information entropy of the data indicator, if Then in the calculation, let 8. The method for evaluating database performance testing according to claim 7, wherein: The performance evaluation index corresponding to the performance indicator is determined by the following formula: Among them, p(I k ,D i ) is the database to be tested D i Performance Indicators I k Corresponding performance evaluation index; Performance index I k Corresponding data indicator weight vector W k The j-th data indicator weight coefficient in ; The comprehensive evaluation index of the database to be tested is determined by the following formula: Among them, p(D i ) represents the comprehensive evaluation index corresponding to the i-th database to be tested, w j is the jth performance indicator weight coefficient in the performance indicator weight vector.

9. An evaluation device for database performance testing, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Determine the set of evaluation objects corresponding to the database to be tested, and based on the test requirements, determine the performance indicator system to be tested and the data indicator system to be tested; wherein the performance indicator system to be tested includes performance indicators for evaluating the database to be tested; the data indicator system to be tested includes data indicators for evaluating individual performance items; the performance indicators include: query performance, deletion performance, update performance, and insertion performance; the data indicators include: average response time, average throughput, average CPU usage, average memory occupancy, and exception rate; Based on the preset performance indicator related judgment matrix, an algorithm is constructed through a preset performance weight vector to determine the performance indicator weight vector corresponding to the performance indicator system to be tested; Based on the preset data indicator correlation judgment matrix, the algorithm is constructed through the preset data weight vector to determine the subjective data indicator weight vector corresponding to the data indicator system to be tested, and based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined; Determining a data indicator weight vector corresponding to the data indicator system to be measured based on the subjective data indicator weight vector and the objective data indicator weight vector; Based on the data indicator weight vector, the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested is determined, and based on the performance evaluation index and the performance indicator weight vector, the comprehensive evaluation index of the database to be tested is determined.

10. A non-volatile computer storage medium for evaluating database performance testing, storing computer-executable instructions, characterized in that: The computer executable instructions are configured to: Determine the set of evaluation objects corresponding to the database to be tested, and based on the test requirements, determine the performance indicator system to be tested and the data indicator system to be tested; wherein the performance indicator system to be tested includes performance indicators for evaluating the database to be tested; the data indicator system to be tested includes data indicators for evaluating individual performance items; the performance indicators include: query performance, deletion performance, update performance, and insertion performance; the data indicators include: average response time, average throughput, average CPU usage, average memory occupancy, and exception rate; Based on the preset performance indicator related judgment matrix, an algorithm is constructed through a preset performance weight vector to determine the performance indicator weight vector corresponding to the performance indicator system to be tested; Based on the preset data indicator correlation judgment matrix, the algorithm is constructed through the preset data weight vector to determine the subjective data indicator weight vector corresponding to the data indicator system to be tested, and based on the measured data obtained from the performance test of the database to be tested, the objective data indicator weight vector corresponding to the data indicator system to be tested is determined; Determining a data indicator weight vector corresponding to the data indicator system to be measured based on the subjective data indicator weight vector and the objective data indicator weight vector; Based on the data indicator weight vector, the performance evaluation index corresponding to each performance indicator in the performance indicator system to be tested is determined, and based on the performance evaluation index and the performance indicator weight vector, the comprehensive evaluation index of the database to be tested is determined.

Images