Database testing methods, apparatuses, devices, and media

Abstract

The application provides a database test method, device, equipment and medium, and is applied to the technical field of database test. The method comprises the following steps: detecting kernel code updating of a to-be-tested database, analyzing the kernel code, determining a difference code and a corresponding to-be-tested object; executing a plurality of to-be-tested use cases on the to-be-tested database to obtain an execution result, the to-be-tested use case being a test use case containing the to-be-tested object and being taken from a preset use case set; determining a to-be-analyzed result from an abnormal result of the execution result based on a to-be-tested object updating link indicated by the difference code, the updating link representing a mapping relationship between a first return value before the to-be-tested object is updated and a second return value after the to-be-tested object is updated; obtaining a clustering cluster based on an abnormal feature of the to-be-analyzed result; for each cluster, determining a target use case based on a use case feature of an abnormal use case; and determining an abnormal reason and generating a test report based on the target use case and the abnormal feature.

Description

Technical Field

[0001] This application relates to the field of database testing technology, and specifically to a database testing method, apparatus, device, and medium. Background Technology

[0002] Database management systems are widely used in critical industries such as finance, telecommunications, and government, and their stability and performance directly impact business continuity. To ensure database quality during version iteration, feature optimization, and defect fixing, development teams typically maintain a large number of test cases that describe various test scenarios, including database initialization, statement execution, result verification, and exception handling, to continuously verify the database's various functions.

[0003] In the process of realizing the concept of this application, at least the following problems were found in the related technologies: full regression testing is inefficient, which leads to a long testing time after the database is updated, affecting the normal use of the database. Summary of the Invention

[0004] In view of the above problems, this application provides a database testing method, apparatus, device and medium.

[0005] According to a first aspect of this application, a database testing method is provided, comprising: in response to detecting an update in the kernel code of a database under test, parsing the kernel code to determine the existence of updated difference code and a test object corresponding to the difference code; executing multiple test cases on the database under test to obtain execution results for each of the multiple test cases, wherein the test cases are test cases including the test object, and the test cases are obtained from a pre-set test case set; and determining an analysis result from multiple abnormal results in the execution results based on the update chain of the test object indicated by the difference code, wherein the update chain represents the test object before the update. The mapping relationship between the first return value and the updated second return value is used to determine the execution result as an abnormal result when the execution result indicates that the test case failed. The test case with the abnormal result is then identified as the abnormal test case corresponding to the abnormal result. Based on the abnormal features indicated by each of the multiple results to be analyzed, the multiple results to be analyzed are clustered to obtain multiple clusters. For each cluster, a target test case is determined from the multiple abnormal test cases in the cluster based on the test case features of each of the multiple abnormal test cases in the cluster. Based on the multiple target test cases and the abnormal features corresponding to the multiple target test cases, the cause of the abnormality is determined, and a test report is generated based on the cause of the abnormality.

[0006] According to an embodiment of this application, the database testing method further includes: performing structured parsing on the test cases of each of the multiple execution results to obtain test case execution plans corresponding to the test cases; comparing the test case execution plans with a benchmark execution plan to obtain a comparison result, wherein the benchmark execution plan is determined from an execution plan library, the execution plan library includes multiple execution plans obtained by executing the test cases using multiple versions of the database under test, and the benchmark execution plan is determined based on the performance of each of the multiple execution plans; and if it is determined that the comparison result indicates that the test case execution plan is inconsistent with the benchmark execution plan, generating structural difference information between the test case execution plan and the benchmark execution plan based on the comparison result, and sending the structural difference information to the test personnel of the database under test.

[0007] According to an embodiment of this application, the comparison of the use case execution plan with the baseline execution plan to obtain a comparison result includes: determining a first topology corresponding to the use case execution plan and a second topology corresponding to the baseline execution plan based on the use case execution plan and the baseline execution plan; determining topological similarity based on multiple nodes of the first topology and the second topology and the connection relationships between the multiple nodes; comparing the execution logic of the use case execution plan and the baseline execution plan to determine abnormal change events, wherein the execution logic includes access paths and the connection order of multi-table accesses, and the abnormal change events include changes in access paths and changes in the connection order of multi-table accesses; and determining the comparison result based on the topological similarity and the abnormal change events.

[0008] According to an embodiment of this application, the database testing method further includes: for each of the above execution results, determining the frequency of inconsistency between the use case execution plan of the above execution result and the above baseline execution plan; and when the frequency of inconsistency is greater than a frequency threshold, adjusting the use case weight of the above execution result, wherein the use case weight is used to scale the use case features.

[0009] According to an embodiment of this application, the database testing method further includes: when the database language of the database application to be tested undergoes a specification change, comparing a first specification document before the specification change and a second specification document after the specification change to determine at least one change information; determining a changed test case from a plurality of test cases in the test case set, wherein the changed test case includes at least one of the change information; updating the changed test case based on the at least one of the change information included in the changed test case to obtain an updated test case; and updating the changed test cases in the test case set to the updated test cases.

[0010] According to an embodiment of this application, updating the change use case based on at least one of the change information included in the change use case to obtain an updated use case includes: determining the change type of the change information based on the change information, wherein the change type includes syntax change, function function change, and target value change; if the change type is determined to be syntax change, replacing the first function in the change use case with the second function based on the first function before the change and the second function after the change, thereby obtaining the updated use case;

[0011] If the change type is determined to be a function function change, based on the change information, the target function with the function function change is determined, as well as the first function and the second function of the target function. The first function is the function function before the function change, and the second function is the function function after the function change. Based on the first expected result in the change use case, the first function, and the second function, the second expected result after the function function change is calculated. The first expected result is updated to the second expected result to obtain the update use case. If the change type is determined to be a target value change, the first expected result is updated based on the change information to obtain the update use case.

[0012] According to an embodiment of this application, the use case characteristics include stack depth, log integrity, and use case script line count. For each cluster, determining a target use case from multiple abnormal use cases within that cluster based on their respective use case characteristics includes: determining a depth median for each cluster based on the stack depth of each abnormal use case within that cluster; and determining the target use case from multiple abnormal use cases within that cluster based on selection criteria, wherein the selection criteria include at least one of the following: the stack depth of the abnormal use case is within a depth range; the log integrity of the abnormal use case is higher than an integrity threshold; the number of use case script lines for the abnormal use case is less than a script size threshold. The depth range is determined based on the depth median and a preset offset, and the script size threshold is determined based on the number of use case script lines for each of the multiple abnormal use cases and a preset percentage value.

[0013] A second aspect of this application provides a database testing apparatus, comprising: a code parsing module, configured to parse the kernel code of a database under test in response to detecting an update, and determine the existence of updated difference code and the test object corresponding to the difference code; a test case execution module, configured to execute multiple test cases on the database under test, and obtain the execution results of each of the multiple test cases, wherein the test cases include the test object, and the test cases are obtained from a pre-set test case set; and an anomaly determination module, configured to determine the result to be analyzed from multiple anomaly results in the execution results based on the update chain of the test object indicated by the difference code, wherein the update chain represents the first return of the test object before the update. The mapping relationship between the return value and the updated second return value is used to determine the execution result as an abnormal result when the execution result indicates that the test case under test has failed. The test case under test with the abnormal result is the abnormal test case corresponding to the abnormal result. The clustering module is used to cluster the multiple results under test based on the abnormal features indicated by each of the multiple results under test, to obtain multiple clusters. The target test case determination module is used to determine the target test case from the multiple abnormal test cases in the cluster for each cluster based on the test case features of each of the multiple abnormal test cases in the cluster. The report generation module is used to determine the cause of the abnormality based on the multiple target test cases and the abnormal features corresponding to the multiple target test cases, and generate a test report based on the cause of the abnormality.

[0014] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0015] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0016] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0017] According to the embodiments of this application, by employing a technique that responds to the detection of a kernel code update in the database under test, parses the kernel code to determine the difference code and the corresponding test object, thereby enabling the pre-screening of test cases affected by the update from a large number of test cases, allowing test execution to focus on relevant test cases rather than running all test cases; executes test cases containing the test object on the database under test and obtains the execution results; determines the results to be analyzed from the abnormal results based on the update chain indicated by the difference code; clusters abnormal test cases based on abnormal features to classify abnormal test cases; and for each cluster, determines the most representative target test case with low analysis difficulty based on test case features; and then determines the cause of the abnormality and generates a test report based on the target test case and abnormal features, the technical means at least partially overcome the technical problems of low execution efficiency of regression testing, difficulty in analyzing failed test cases, and difficulty in predicting performance degradation in related technologies, thereby achieving the technical effects of improving the accuracy and efficiency of database regression testing, while enabling the test report to comprehensively cover various abnormal problems and avoid duplication and redundancy, and reduce the difficulty of analysis. Attached Figure Description

[0018] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments of this application with reference to the accompanying drawings.

[0019] Figure 1 The illustration shows an application scenario diagram of the database testing method, apparatus, device, and medium according to embodiments of this application.

[0020] Figure 2 A flowchart illustrating a database testing method according to an embodiment of this application is shown schematically.

[0021] Figure 3 A flowchart illustrating a database testing method according to another embodiment of this application is shown.

[0022] Figure 4 A schematic block diagram of a database testing apparatus according to an embodiment of this application is shown.

[0023] Figure 5 A block diagram schematically illustrates an electronic device suitable for implementing a database testing method according to an embodiment of this application. Detailed Implementation

[0024] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms "comprising," "including," etc., as used herein indicate the presence of the above-described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0028] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0029] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0030] When testing databases using relevant technologies, if the database product undergoes specification changes, some test cases may experience false failures due to mismatches between expected outputs and updated behaviors when using pre-maintained test cases. In cases of code changes to the database product, these changes may cause the query optimizer to choose alternative execution plans, leading to performance degradation. Since related tests typically only verify the correctness of results, it's difficult to detect performance degradation. Furthermore, analyzing failed test cases requires meticulous examination of logs and stack traces, resulting in inefficient backtracking.

[0031] This application provides a database testing method, comprising: in response to detecting an update in the kernel code of the database under test, parsing the kernel code to determine the existence of updated difference code and the object under test corresponding to the difference code; executing multiple test cases on the database under test to obtain the execution results of each test case, wherein the test cases are test cases including the object under test, and the test cases are obtained from a pre-set test case set; and determining the results to be analyzed from multiple abnormal results in the execution results based on the update chain of the object under test indicated by the difference code, wherein the update chain represents the first update of the object under test before the update. The mapping relationship between the return value and the updated second return value is used to determine the execution result as an abnormal result when the execution result indicates that the test case failed. The test case with the abnormal result is the abnormal test case corresponding to the abnormal result. Based on the abnormal characteristics indicated by each of the multiple test results, the multiple test results are clustered to obtain multiple clusters. For each cluster, the target test case is determined from the multiple abnormal test cases in the cluster based on the test case characteristics of each of the multiple abnormal test cases in the cluster. Based on the multiple target test cases and the abnormal characteristics corresponding to the multiple target test cases, the cause of the abnormality is determined, and a test report is generated based on the cause of the abnormality.

[0032] Figure 1 The illustration shows an application scenario diagram of the database testing method, apparatus, device, and medium according to embodiments of this application.

[0033] like Figure 1As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0034] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0035] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0036] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0037] It should be noted that the database testing method provided in this application embodiment can generally be executed by server 105. Correspondingly, the database testing device provided in this application embodiment can generally be located in server 105. The database testing method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the database testing device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0038] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0039] The following will be based on Figure 1 The described scene, through Figures 2-3 The database testing method of the embodiments of this application will be described in detail.

[0040] Figure 2 A flowchart illustrating a database testing method according to an embodiment of this application is shown schematically.

[0041] like Figure 2 As shown, the database testing method of this embodiment includes operations S210 to S260.

[0042] In operation S210, in response to the detection of an update to the kernel code of the database under test, the kernel code is parsed to determine the existence of updated difference code and the object under test corresponding to the difference code.

[0043] According to embodiments of this application, kernel code refers to the source code of the core functions of the database management system, including the implementation code of modules such as the storage engine, query optimizer, and transaction manager. Difference code refers to code segments that have changed in this update, such as newly added functions, modified logical branches, or deleted statements.

[0044] The objects under test include database kernel functions, modules, or data structures that have calling relationships or data dependencies with the differing code. For example, when the internal implementation of a function changes, the function itself becomes the object under test; when the field definition or memory layout of a data structure changes, the read and write functions involved in that data structure become the objects under test; when the core logic of a module changes, the interface functions provided by that module become the objects under test.

[0045] Taking the index module update of a database system as an example, the developers submitted performance optimization code for the index page splitting logic, involving implementation changes to the index insertion function and the index page splitting function. By analyzing the difference information in the code submission record, it was determined that the differing code consisted of the conditional statements regarding page splitting and the implementation of the splitting operation in the aforementioned two functions. The corresponding objects under test were the two functions themselves and the data structures related to page splitting.

[0046] Taking the cost estimation model adjustment of a database system query optimizer as an example, the developers modified the cost calculation function for selecting the multi-table join order. This involved the loop variable and cost accumulation logic in the cost calculation function, as well as the system table statistics data structure on which cost estimation depends. By analyzing the difference code, the objects under test were identified as the newly added and changed field types in the cost calculation function and the statistics structure.

[0047] In operation S220, multiple test cases are executed on the database to be tested, and the execution results of each test case are obtained. The test cases are test cases that include the object to be tested, and the test cases are obtained from a pre-set test case set.

[0048] According to an embodiment of this application, a test case is a test description document used to verify specific functions of a database. Each test case includes at least one of the following operations: database initialization operation, statement execution, result verification, and exception capture, and also includes the expected output after performing the above operations.

[0049] A pre-defined test case set refers to a collection of all pre-written and maintained test cases used for continuous regression verification of various functions of the database. Multiple test cases to be tested are selected from the pre-defined test case set based on the differing code and the object under test, and are related to the current code change.

[0050] Taking the change of index-related functions in the database under test as an example, the preset test case set contains functional verification test cases for various data operation statements. When the insertion-related functions of the index module change, test cases containing index insertion operations are obtained from the preset test case set as test cases to be tested, and these test cases are executed on the updated database to verify whether the index insertion function is normal.

[0051] Taking a change in the query optimizer of the database under test as an example, the preset test case set contains functional verification test cases for various complex query statements. When the cost estimation related functions are adjusted, test cases containing multi-table join queries and nested queries are obtained from the preset test case set as test cases. These test cases are then executed on the updated database to verify whether the query optimizer can generate the correct execution plan.

[0052] In operation S230, based on the update link of the test object indicated by the difference code, the result to be analyzed is determined from multiple abnormal results in the execution result. The update link represents the mapping relationship between the first return value of the test object before the update and the second return value after the update. If the execution result indicates that the test case failed to execute, the execution result is determined to be an abnormal result, and the test case with the abnormal result is the abnormal test case corresponding to the abnormal result.

[0053] According to the embodiments of this application, the update link refers to the correspondence between the behavior changes of the object under test before and after the code change, specifically manifested as the mapping of the return results of the same operation under the old version and the new version.

[0054] Execution results include two states: successful execution and failed execution. Execution failure mainly includes two situations: the test case is interrupted during execution due to code exceptions such as process crashes or lock timeouts, and the execution result is inconsistent with expectations. An inconsistent execution result means that the test case has been fully executed, but its output is inconsistent with the pre-set expected result. By filtering out execution results that are inconsistent with expectations due to specification updates, only the results caused by code updates, process crashes, and lock timeouts can be retained as the analysis results for subsequent cluster analysis.

[0055] For example, after executing multiple test cases on the database under test, several abnormal results are obtained. Some of these abnormal results are cases where the test cases were executed completely, but the returned error codes were inconsistent with the expected error codes; for example, the expected error code was A, but the actual error code was B. The execution results of these test cases are identified as results to be analyzed, and the corresponding test cases are designated as test cases to be analyzed.

[0056] For example, after executing the test cases, some test cases may complete the execution but the number of data rows returned may not be consistent with expectations. This is considered an execution result that does not meet expectations and is retained as a result to be analyzed. The corresponding test cases are test cases to be analyzed.

[0057] In operation S240, based on the abnormal features indicated by each of the multiple results to be analyzed, multiple clusters are obtained.

[0058] According to embodiments of this application, anomaly features refer to multi-dimensional information that can characterize the reasons for the failure of the use case to be analyzed, including error codes, process exception stacks, key log information, and assertion failure types. Clustering refers to the process of grouping use cases to be analyzed with similar anomaly features into the same category, and each category is called a cluster.

[0059] Taking the 200 results obtained after executing the test database as an example, the error codes and exception stack information of each test case are collected as anomaly features. Initial grouping is performed based on the error codes, grouping test cases with the same error codes into the same candidate cluster, followed by secondary subdivision based on stack features. For example, if 50 test cases return exception error codes indicating that the lock cannot be acquired and their stacks all point to lock management-related functions, these 50 test cases are grouped into the same cluster, representing the "lock resource contention" failure mode.

[0060] Taking another test scenario as an example, 100 test cases to be analyzed returned exception error codes indicating duplicate key values, but their stack traces differed: the stack traces of 60 test cases pointed to functions related to index uniqueness verification, while the stack traces of the other 40 test cases pointed to functions related to data relationship constraint verification. After clustering based on the exception characteristics, these 100 test cases were divided into two clusters, corresponding to two different failure modes: "index uniqueness constraint conflict" and "data relationship constraint conflict."

[0061] In operation S250, for each cluster, the target use case is determined from the multiple abnormal use cases in the cluster based on the use case characteristics of each of the multiple abnormal use cases in the cluster.

[0062] According to embodiments of this application, use case characteristics refer to information that reflects the attributes of an abnormal use case itself, used to measure the representativeness and analysis difficulty of the abnormal use case among similar anomalies. The target use case refers to the most representative use case with the lowest analysis difficulty selected from multiple use cases to be analyzed within the same cluster.

[0063] Taking a cluster containing multiple use cases to be analyzed as an example, based on the pre-set use case feature filtering conditions, the most representative use case with the lowest analysis difficulty is selected from the cluster as the target use case of the cluster, which makes it easier for developers to quickly locate the problem.

[0064] In operation S260, based on multiple target use cases and the abnormal characteristics corresponding to the multiple target use cases, the cause of the abnormality is determined, and a test report is generated based on the cause of the abnormality.

[0065] According to embodiments of this application, the cause of an anomaly refers to the root cause that leads to the execution result of the test case being analyzed not meeting expectations. The test report is a document summarizing all anomaly types, their corresponding target test cases, and analysis results, used to provide developers with a basis for fault localization.

[0066] Taking a regression test of the database under test as an example, three clusters were obtained after clustering: Cluster A contains 50 test cases to be analyzed, all involving lock acquisition failures; Cluster B contains 30 test cases to be analyzed, all involving index uniqueness constraint conflicts; and Cluster C contains 20 test cases to be analyzed, all involving memory resource allocation anomalies. For Cluster A, based on its anomaly characteristics, such as the error code indicating lock acquisition failure and the stack pointing to lock management-related functions, the cause of the anomaly was determined to be "lock acquisition failure or timeout". The target test cases and their anomaly characteristics of Cluster A were selected to generate Cluster A entries in the test report. The final test report contains only three records, each corresponding to a fault mode and its target test case.

[0067] According to the embodiments of this application, by detecting a kernel code update in the database under test, parsing the kernel code to determine the difference code and the corresponding object under test, executing test cases containing the object under test on the database under test, and obtaining the execution results, it is possible to pre-select test cases affected by the update from a large number of test cases, so that test execution focuses on relevant test cases rather than running all test cases. The update chain based on the difference code indicates that the results to be analyzed are determined from the abnormal results, and multiple results to be analyzed are clustered based on abnormal characteristics. For each cluster, the target test case is determined based on the test case characteristics of the abnormal test cases. Then, based on the target test case and abnormal characteristics, the cause of the abnormality is determined and a test report is generated. This technical means categorizes hundreds or thousands of abnormal test cases into several typical failure modes and presents the analysis results in a concise report containing representative target test cases. Therefore, it at least partially overcomes the technical problems in related technologies, such as low execution efficiency of full regression testing, waste of resources caused by repeated execution of test cases unrelated to the current modification, low analysis efficiency caused by the need to check abnormal test cases one by one, and lengthy test reports. In this way, it achieves the technical effects of improving the accuracy and efficiency of database regression testing, reducing the amount of test execution, shortening the fault location time, and reducing the difficulty of test report analysis.

[0068] According to an embodiment of this application, the use case characteristics include stack depth, log integrity, and use case script line count. For each cluster, based on the use case characteristics of multiple abnormal use cases in the cluster, a target use case is determined from the multiple abnormal use cases in the cluster, including: for each cluster, determining the depth median based on the stack depth of multiple abnormal use cases in the cluster; and determining the target use case from the multiple abnormal use cases in the cluster based on selection conditions, the selection conditions including at least one of the following: the stack depth of the abnormal use case is within the depth range, the log integrity of the abnormal use case is higher than the integrity threshold, and the number of use case script lines of the abnormal use case is less than the script size threshold. The depth range is determined based on the depth median and a preset offset amplitude, and the script size threshold is determined based on the number of use case script lines of multiple abnormal use cases and a preset value percentage.

[0069] According to embodiments of this application, anomaly features refer to multi-dimensional information that can characterize the reasons for failure of the use case to be analyzed, including error codes, process exception stacks, key log information, and assertion failure types.

[0070] Before clustering abnormal features, the abnormal features can be encoded into feature vectors, and then clustered. Specifically, different encoding strategies can be adopted for different abnormal features: error codes can be directly used as category features, stack information can be sliced ​​and used as sequence features, and log keywords can be encoded using a bag-of-words model. The encoded feature vectors are then clustered.

[0071] According to embodiments of this application, the DBSCAN algorithm can be used for clustering. This algorithm can automatically discover clusters of arbitrary shapes without pre-specifying the number of clusters. Considering the characteristics of failed data in the database, the neighborhood radius and minimum sample size are adjusted to effectively separate typical failure modes.

[0072] Taking the 200 results obtained after executing the test database as an example, error codes and exception stack information for each test case are collected as exception features. Error codes are used as category features, stack information is sliced ​​and used as sequence features, and log keywords are encoded using a bag-of-words model to form the feature vector for each test case. All 200 feature vectors are input into the DBSCAN clustering model, which automatically groups test cases with similar exception features into the same cluster. For example, if 50 test cases return exception error codes indicating that the lock cannot be acquired and the stacks all point to lock management-related functions, these 50 test cases are grouped into the same cluster, representing the "lock cannot be acquired" failure mode.

[0073] After clustering, for each cluster, a target use case needs to be selected. Selection criteria include: stack depth within a preset range, complete log information, and the fewest lines of script in the use case. Taking a cluster containing 45 use cases to be analyzed as an example, the stack depth, log completeness, and number of lines of script for each use case are collected. The median stack depth of all use cases within the cluster is calculated, and the median ± 2 is used as the depth range. Use cases with stack depths within this range are selected, and those with log completeness higher than the integrity threshold and script lines less than the script size threshold are chosen as the target use case for that cluster. Taking another cluster containing 80 use cases to be analyzed as an example, the error codes for all use cases in this cluster indicate that the lock cannot be acquired, and the stacks all point to lock management-related functions. Some use cases involve deep stacks due to multi-level nested function calls, making analysis more difficult. Other use cases have moderate stack depths and logs recording the complete lock waiting process. Based on the above selection criteria, use cases with moderate stack depth, complete log information, and the fewest lines of script are selected as target use cases, facilitating rapid problem reproduction and localization by developers.

[0074] Figure 3 A flowchart illustrating a database testing method according to another embodiment of this application is shown.

[0075] like Figure 3 As shown, in Figure 2 Based on the illustrated embodiment, the database testing method may further include operation S0 and operation S3, wherein the dashed boxes of operation S0 and operation S3 indicate that the above operations are optional operations. That is, depending on different testing requirements, the embodiments of this application may include operation S1~operation S2, operation S0~operation S2, operation S1~operation S3 or operation S0~operation S3.

[0076] According to the embodiments of this application, operation S0 is used for product specification change parsing and adaptive adjustment of test case expectations. In operation S0, when the database language of the database application to be tested undergoes a specification change, the first specification document before the specification change and the second specification document after the specification change are compared to determine at least one change information; changed test cases are determined from multiple test cases in the test case set, and the changed test cases include at least one change information; the changed test cases are updated based on the at least one change information included in the changed test cases to obtain updated test cases; and the changed test cases in the test case set are updated to updated test cases.

[0077] According to embodiments of this application, product release specification changes may include changes to the database new version feature description and the requirements specification document. In the event of a specification change, the system receives the relevant documents, compares the first specification document before the change with the second specification document after the change, and extracts structured change information. This structured change information may include change type, change object, change description, and effective version. The change type includes adding syntax, modifying function behavior, deprecating features, adjusting error codes, and changing default values; the change object includes function names, keywords, system table names, and error codes; the change description includes old behavior, new behavior, and an example Structured Query Language (SQL); and the effective version indicates which version the change took effect from.

[0078] Based on the extracted change information, the test case set is traversed to identify test cases affected by the updates indicated in the change information, i.e., change test cases. According to the change type, corresponding update strategies are executed on the change test cases, including syntax replacement, recalculation of expected results, error code updates, and assertion logic modifications, resulting in updated test cases. Detailed logs of all updates are recorded, and the original change test cases in the test case set are updated to updated test cases, generating a specification change impact report.

[0079] For example, when a database system releases a new version of its specification document, using this document as the second specification document, the system compares the locally stored first specification document with the second specification document and can obtain the following changes: a string processing function changes from returning the number of bytes to returning the number of characters; an error code is changed from -206 to -217; and the default value of a system table field is changed from NULL to 0.

[0080] Based on the extracted change information, the test case set is traversed to retrieve all test cases containing the above string processing functions, all test cases expected to produce a -206 error code, and all test cases that query the system table and involve the above-mentioned changed fields. These test cases are then identified as change test cases.

[0081] For test cases containing this string processing function, the system calls the old version to execute the SQL to obtain the old result, then combines this with the updated rule for the number of characters returned to calculate the new result, and updates the expected output of the test case. For test cases where the assertion expects an error code of -206, the expected error code is updated to -217. For test cases involving the aforementioned system table fields, the query is re-executed to capture the new default values, and the assertion statement is updated. After completing the above update operations, the updated test cases are obtained.

[0082] After the update is complete, replace the original changed test cases in the test case set with the updated test cases.

[0083] According to the embodiments of this application, by automatically sensing changes in product specifications and adaptively adjusting test case expectations, false alarms caused by specification changes can be filtered out, allowing failure analysis to focus on real code defects. At the same time, it replaces the heavy work of manually modifying tens of thousands of test cases one by one, greatly improving the efficiency of test case maintenance and ensuring that test assets remain effective as the product evolves.

[0084] According to embodiments of this application, updating a change use case based on at least one change information included in the change use case to obtain an updated use case includes: determining the change type of the change information based on the change information, where the change type includes syntax change, function function change, and target value change; if the change type is determined to be syntax change, replacing the first function in the change use case with the second function based on the first function before the change and the second function after the change, to obtain an updated use case; if the change type is determined to be function function change, determining the target function with function function change based on the change information, as well as the first function and the second function of the target function, where the first function is the function function before the function change and the second function is the function function after the function change; calculating the second expected result after the function function change based on the first expected result, the first function, and the second function in the change use case; updating the first expected result to the second expected result, to obtain an updated use case; and if the change type is determined to be target value change, updating the first expected result based on the change information, to obtain an updated use case.

[0085] According to embodiments of this application, the types of changes include syntax changes, function functionality changes, and target value changes. Syntax changes refer to changes at the syntax level of the database language, such as changes to function names, keywords, or the obsolescence of functions. Function functionality changes refer to changes in the semantics of functions. Target value changes refer to changes in expected target values ​​such as error code values ​​or default values ​​for system table fields.

[0086] Specifically, syntax changes can include adding new syntax, function functionality changes can include modifying function behavior or deprecating function functionality, and target value changes can include adjusting error codes or changing system table default values.

[0087] Depending on the type of change, different adjustment strategies are implemented for affected use cases. For example, if a function is deprecated due to a syntax change, that function in the use case (function 1) is replaced with the recommended new function (function 2). The original SQL is then executed using the old version of the database under test to obtain the old results. The new results are then calculated based on the change rules, and the expected output of the use case is updated. If the change involves a target value, the query is re-executed to capture the new default value. The assertions in the use case are automatically updated, where the original default value is the first expected result, and the new default value is the second expected result. All adjustments are logged in detail, including the original use case, the adjusted use case, the basis for the adjustment, and the adjustment time. The use cases are marked as "automatically adapted".

[0088] Specifically, taking a specification change of a database product as an example, the product specification document records the following changes: Function A, which is not recommended for use, is deprecated and replaced by Function B; Function C changes its return value from bytes to characters; Error code -206 is adjusted to -217; The default value of the `nrows` field in the system table `systables` is changed from NULL to 0. For syntax changes, the system replaces the first function A in the change test cases with the second function B. For function functionality changes, the system identifies all change test cases containing function C, executes the SQL in the old version test cases to obtain the first expected result, and then calculates the second expected result based on the rule of "new version returning character count," updating the test case expectation from the first expected result to the second expected result. For error code changes, the system updates the expectation of all test cases with an assertion expectation of -206 to -217. For default value changes, the system re-executes the query to capture the new default value and updates the assertion statements of the test cases.

[0089] According to embodiments of this application, by adopting corresponding adjustment strategies for different change types, use cases are expected to automatically adapt to new specifications. This can automatically filter out a large number of pseudo-failures caused by specification changes before analysis, allowing subsequent failure analysis to focus on real code defects. Simultaneously, through automated recalculation of expected results (including calling the old version to obtain the runtime context and then combining it with change rules for deduction, and using a simulation execution engine to recalculate based on the new specifications), complex semantic change scenarios can be accurately handled, replacing manual analysis and modification, significantly reducing use case maintenance costs, and improving the reliability and efficiency of regression testing.

[0090] According to the embodiments of this application, Figure 3 Operation S1 is used for difference code impact domain analysis. In operation S1, in response to the code change submission of the kernel of the database under test, the impact code domain of this change is determined through static code analysis technology. Based on the pre-generated test cases and code coverage path mapping relationship, the target test case set associated with the impact code domain is selected from the test case set, and the test cases in the target test case set are prioritized according to the degree of impact.

[0091] According to an embodiment of this application, a code change submission for the kernel of the database to be tested is received. By parsing the submitted discrepancy information and combining it with the compilation dependencies of the source code, static code analysis tools are used to determine the functions, modules, and data structures involved in the change. A pre-built test case and code coverage path mapping library is queried, and the list of affected code domains is matched with the mapping library to filter out all test cases that cover any function or line of code in the affected code domain, or whose access paths to dependent database objects have changed due to the code change. This generates a set of target test cases to be executed. The set of target test cases to be executed is prioritized, with the following factors considered: in terms of module criticality, test cases involving the storage engine core have the highest priority, followed by those involving query optimizer cost calculation, and those involving management tools or compatibility interfaces have the lowest priority. In terms of change popularity, if the number of times a changed function has been modified exceeds a threshold in the past period, the priority of test cases covering that function is increased; in terms of impact scope, test cases covering more lines of changed code are prioritized over those covering fewer lines. The sorted list of test cases will serve as the basis for subsequent execution. At the same time, the use cases that have been adjusted in step S0 (if they are also within the scope of this screening) are marked as "requiring special attention" because their expectations have been updated and the execution results can verify the correctness of the adjustments.

[0092] Taking a code change to the database under test as an example, the developers submitted performance optimization code for the index page splitting logic. By analyzing the difference information in the code commit record, it was determined that the functions involved in this change were the index insertion function and the index page splitting function. A test case and code coverage path mapping database was queried to filter out test cases covering any of the above functions, generating a target test case set of 120 test cases to be executed. Among them, 15 test cases involve the core logic of index page splitting, 80 test cases involve the index insertion path, and 25 test cases only indirectly depend on related data structures. Based on the module criticality, the 15 test cases involving the core logic were set to the highest priority, the 80 test cases involving the insertion path were set to the second highest priority, and the 25 indirectly dependent test cases were set to the lowest priority. The sorted test case list serves as the basis for subsequent execution.

[0093] By using the above methods, the target test case set affected by the update can be accurately selected from the full test case set based on the actual changes in the code. This allows the test execution to focus on the relevant test cases rather than running the full set, greatly reducing the waste of computing resources caused by repeatedly executing test cases that are not related to the current modification. As a result, the regression testing cycle can be significantly shortened while ensuring sufficient test coverage.

[0094] According to the embodiments of this application, Figure 3Operation S2 in the model is used for multi-dimensional failure clustering analysis. In operation S2, the target test case set is executed in parallel, and log information of failed test cases is collected. Multi-dimensional failure features are extracted from the log information, and the failure features are input into the clustering model to classify multiple failed test cases and generate a representative test case for each category of failure.

[0095] According to an embodiment of this application, the target test case set selected in step S1 is executed in parallel on a test cluster. Each test node runs an independent database instance and is configured with a unified log collection agent. The execution status of all test cases is monitored, and once a test case failure is detected, the log collector is immediately triggered to collect multi-dimensional failure features. Before inputting the failure features into the clustering model, pseudo-failure filtering is first performed: the features of failed test cases are compared with the specification change knowledge base. If it is found that the object involved in a failed test case is marked as "adjusted" in a recent specification change, and the failure features match the change features, then the test case is marked as a "pseudo-failure caused by specification change" and removed from the clustering analysis process. The feature information of the remaining failed test cases is uniformly encoded, and the encoded feature vectors are input into the clustering model.

[0096] Taking a regression test on the database under test as an example, 500 target test cases were selected and executed in parallel, of which 200 test cases failed. Error codes and exception stack traces for each failed test case were collected as anomaly features. Before inputting the failure features into the clustering model, the features of the failed test cases were compared with a specification change knowledge base. It was found that 50 of these test cases involved specification changes to a string processing function and were marked as "pseudo-failures caused by specification changes" and removed from the clustering process. The error codes of the remaining 150 failed test cases were used as category features, the stack traces were sliced ​​and used as sequence features, and log keywords were encoded using a bag-of-words model to form the feature vector for each test case. All 150 feature vectors were input into the DBSCAN clustering model, which automatically merged test cases with similar anomaly features into the same cluster. After clustering, for each cluster, a test case with the most significant features and the simplest execution steps was selected as a representative test case for developers to quickly reproduce and locate the problem.

[0097] According to embodiments of this application, by categorizing multiple abnormal use cases and selecting the target use case with the most obvious abnormal characteristics and easy reproduction and backtracking from each category, testers are no longer required to check the logs and stack traces of each failed use case individually, thus shortening the fault localization time. Furthermore, the test report only needs to include representative use cases and analysis results for each cluster, avoiding the lengthy presentation of multiple use cases with the same problem repeatedly, effectively reducing the difficulty of analyzing the test report.

[0098] According to the embodiments of this application, Figure 3Operation S3 is used for structured comparison of execution plans and performance degradation warning. In operation S3, for the execution results, the test cases of each of the multiple execution results are structured and parsed to obtain the test case execution plans corresponding to each test case; the test case execution plans are compared with the baseline execution plans to obtain the comparison results. The baseline execution plans are determined from the execution plan library, which includes multiple execution plans obtained by executing non-abnormal test cases using multiple versions of the database under test. The baseline execution plans are determined based on the performance of each of the multiple execution plans; and if the comparison results indicate that the test case execution plans and the baseline execution plans are inconsistent, structural difference information between the test case execution plans and the baseline execution plans is generated based on the comparison results, and the structural difference information is sent to the testers of the database under test.

[0099] According to embodiments of this application, the execution results include both abnormal and non-abnormal results. For both abnormal and non-abnormal results, a structured comparison of the execution plan and performance degradation warnings are performed. The execution plan of a test case is the execution step strategy generated by the database to complete a specified SQL query, containing core information such as table access order, scanning method, index usage rules, and join logic. The baseline execution plan is determined from the execution plan library, which stores multiple execution plans obtained by executing non-abnormal test cases on multiple versions of the database under test. The baseline execution plan is determined based on the performance of each of the multiple execution plans; for example, the one with the shortest actual execution time is selected as the baseline. Structural difference information refers to the records of differences between the current execution plan and the baseline execution plan in terms of plan tree topology, table access paths, and join order.

[0100] Specifically, the baseline execution plan is initially run on a recognized, stable, and reliable released database version, running all test cases and capturing the performance fingerprint of each test case. This performance fingerprint is then used as the initial baseline version. The baseline execution plan is determined from a set of validated, stable, and acceptable execution plans. For newly generated execution plans, performance comparisons can be performed based on metrics such as execution time, CPU consumption, and I / O overhead through an evolutionary mechanism. Only when a new execution plan is found to significantly outperform existing baseline execution plans and has no regression risk will it be marked as acceptable and included in the baseline set.

[0101] Specifically, before test execution, the performance fingerprint of the baseline version is loaded from the performance baseline library. The baseline library is constructed as follows: when running each test case on the baseline version of the database under test, the execution plan is obtained using the SET EXPLAIN command. Simultaneously, resource metrics such as actual execution time, disk I / O, and memory usage are recorded, and the execution plan is parsed into a structured JSON format (containing operator nodes, join types, index information, cost estimates, etc.) and stored. For the same test case, if the execution plan drifts due to changes in data volume, multiple versions of the baseline can be maintained. When executing each test case in the current version, the execution plan is also captured using the SET EXPLAIN command and parsed into the same structured format as the baseline. The current execution plan is compared with the baseline execution plan in a structured manner, generating an execution plan difference report. If the comparison reveals a change in the execution plan and the estimated cost or actual execution time of the new plan is worse than the baseline plan by more than a preset threshold (e.g., a 20% increase in cost or a 50% increase in time), it is determined to be a performance regression event. The test case is marked as high-risk, and the merge request for the current code change is automatically blocked via an API call to the code version management system. At the same time, a performance comparison report containing details of the structured differences, execution plan diagrams for both versions, and descriptions of the business loads that may be affected will be pushed to the developer who submitted the change.

[0102] According to embodiments of this application, by comparing the structured differences in the execution plan in real time during the functional testing phase, performance degradation caused by changes in optimizer strategy can be effectively captured, ensuring the service quality after the database goes live.

[0103] According to an embodiment of this application, comparing a use case execution plan with a baseline execution plan to obtain a comparison result includes: determining a first topology corresponding to the use case execution plan and a second topology corresponding to the baseline execution plan based on the use case execution plan and the baseline execution plan; determining topological similarity based on multiple nodes of each of the first and second topologies and the connection relationships between the multiple nodes; comparing the execution logic of each of the use case execution plan and the baseline execution plan to determine abnormal change events, wherein the execution logic includes access paths and the connection order of multi-table accesses, and the abnormal change events include changes in access paths and changes in the connection order of multi-table accesses; and determining the comparison result based on the topological similarity and the abnormal change events.

[0104] According to embodiments of this application, the structured comparison of execution plans includes plan tree topology comparison, access path comparison, join order comparison, and cost estimation comparison. Topology comparison uses a tree edit distance algorithm to calculate the similarity between two plan trees; if the topology difference exceeds a threshold, it is marked as a structural change. Access path comparison compares the usage of base tables and indexes to check for situations such as full table scans replacing index scans or index type changes. Join order comparison, for multi-table joins, compares whether the driving table and join order have changed. Cost estimation comparison compares the total cost estimated by the optimizer and the cost of each sub-operation; if the new plan cost exceeds the baseline plan cost by a certain proportion, it is considered abnormal.

[0105] Taking an execution plan comparison of a database product as an example, when the current version executes a multi-table join query, the topology of the execution plan generated by the optimizer differs from the baseline version: In the baseline version, the optimizer selected table A as the driving table, retrieved data through an index scan, and then performed a hash join with table B; in the current version, the optimizer selected table B as the driving table, first performing a full table scan and then performing a nested loop join with table A. The topology comparison used a tree edit distance algorithm to calculate the similarity between the two plan trees, identifying node additions / deletions and changes in operator types, marking these as structural changes. Access path comparison revealed that the index scan used in the original version has been changed to a full table scan in the current version. Join order comparison showed that the driving table changed from table A to table B, and the join order changed. Cost estimation comparison revealed that the current version's optimizer estimated the total cost to be more than 30% higher than the baseline plan cost. Based on the comparison results of these four dimensions, it was determined that this execution plan change carries a risk of performance degradation.

[0106] According to an embodiment of this application, for each execution result, the frequency of inconsistency between the use case execution plan and the baseline execution plan of the execution result is determined; and if the frequency of inconsistency is greater than a frequency threshold, the use case weight of the execution result is adjusted, and the use case weight is used to scale the use case features.

[0107] According to embodiments of this application, the difference frequency refers to the frequency with which the test case execution plan differs from the baseline execution plan in multiple tests. The frequency threshold is used to determine whether this frequency reaches a critical value requiring weight adjustment. The test case weight refers to the importance of the test case in subsequent test analysis, used to scale test case characteristics. Performance regression information is fed back to the test case and code coverage path mapping relationship in step S1. If a test case frequently experiences performance regression, i.e., the difference frequency of the test case is greater than the frequency threshold, its associated code module is assigned a higher weight during subsequent screening in S1, i.e., the weight of the test case is adjusted to achieve dynamic learning. Simultaneously, performance regression events are associated with specification change information. If it is found that the performance regression is caused by a specification change (such as a change in optimizer behavior due to new syntax), a re-evaluation of the relevant specification change is triggered.

[0108] Specifically, taking multiple regression tests of a certain database as an example, a certain test case experienced performance regression in the regression tests after the first three code changes. This means that the execution plan of this test case repeatedly deviated from the baseline execution plan, indicating frequent performance regressions. The system feeds this performance regression information back to the test case and code coverage path mapping relationship. When performing regression tests after subsequent code changes, the code module associated with this test case is given higher weight in the subsequent S1 filtering, ensuring that this test case is executed first in subsequent tests. Simultaneously, the system performs correlation analysis with the specification change knowledge base. If it finds that the performance regression was caused by a specification change (such as a change in optimizer behavior due to new syntax), it triggers a re-evaluation of the relevant specification change and pushes the evaluation results to the specification change management personnel.

[0109] According to the embodiments of this application, the above method can realize dynamic feedback of performance regression information to the test case selection stage, forming a closed-loop optimization, so that test resources are continuously tilted towards high-risk test cases, thereby improving the accuracy and efficiency of regression testing.

[0110] Based on the above database testing method, this application also provides a database testing apparatus. The following will combine... Figure 4 The device is described in detail.

[0111] Figure 4 A schematic block diagram of a database testing apparatus according to an embodiment of this application is shown.

[0112] like Figure 4 As shown, the database testing device 400 of this embodiment includes a code parsing module 410, a test case execution module 420, an exception determination module 430, a clustering module 440, a target test case determination module 450, and a report generation module 460.

[0113] The code parsing module 410 is used to parse the kernel code in response to detecting an update in the kernel code of the database under test, and to determine the existence of updated difference code and the object under test corresponding to the difference code. In one embodiment, the code parsing module 410 can be used to perform the operation S210 described above, which will not be repeated here.

[0114] The test case execution module 420 is used to execute multiple test cases on the database to be tested, and obtain the execution results of each test case. The test cases are test cases that include the object to be tested, and the test cases are obtained from a pre-set test case set. In one embodiment, the test case execution module 420 can be used to execute the operation S220 described above, which will not be repeated here.

[0115] The anomaly determination module 430 is used to determine the result to be analyzed from multiple abnormal results in the execution results based on the update link of the test object indicated by the difference code. The update link represents the mapping relationship between the first return value of the test object before the update and the second return value after the update. When the execution result indicates that the test case failed to execute, the execution result is determined to be an abnormal result, and the test case with the abnormal result is the abnormal test case corresponding to the abnormal result. In one embodiment, the anomaly determination module 430 can be used to perform the operation S230 described above, which will not be repeated here.

[0116] The clustering module 440 is used to cluster multiple analysis results based on the abnormal features indicated by each of the multiple analysis results, thereby obtaining multiple clusters. In one embodiment, the clustering module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0117] The target use case determination module 450 is used to determine a target use case from multiple abnormal use cases in each cluster, based on the use case characteristics of each abnormal use case in the cluster. In one embodiment, the target use case determination module 450 can be used to perform the operation S250 described above, which will not be repeated here.

[0118] The report generation module 460 is used to determine the cause of an anomaly based on multiple target use cases and the anomaly characteristics corresponding to the multiple target use cases, and to generate a test report based on the cause of the anomaly. In one embodiment, the report generation module 460 can be used to perform the operation S230 described above, which will not be repeated here.

[0119] According to an embodiment of this application, the database testing apparatus 400 further includes an execution plan comparison module.

[0120] The execution plan comparison module includes a structured parsing submodule, an execution plan comparison submodule, and a structured difference generation submodule.

[0121] The structured parsing submodule is used to perform structured parsing on the test cases of multiple execution results to obtain the test case execution plan corresponding to each test case.

[0122] The execution plan comparison submodule is used to compare the test case execution plan with the baseline execution plan to obtain the comparison result. The baseline execution plan is determined from the execution plan library, which includes multiple execution plans obtained by executing the test cases using multiple versions of the database to be tested. The baseline execution plan is determined based on the performance of each of the multiple execution plans.

[0123] The structural difference generation submodule is used to generate structural difference information between the test case execution plan and the baseline execution plan based on the comparison results when it is determined that the comparison results represent inconsistencies between the test case execution plan and the baseline execution plan, and then send the structural difference information to the testers in the database to be tested.

[0124] According to an embodiment of this application, the execution plan comparison submodule includes a topology comparison unit, an execution logic comparison unit, and a result determination unit.

[0125] The topology comparison unit is used to determine the first topology corresponding to the use case execution plan and the second topology corresponding to the baseline execution plan based on the use case execution plan and the baseline execution plan; and to determine the topology similarity based on multiple nodes of the first topology and the second topology and the connection relationships between the multiple nodes.

[0126] The execution logic comparison unit is used to compare the execution logic of the test case execution plan and the baseline execution plan to determine abnormal change events. The execution logic includes access paths and the connection order of multi-table accesses. Abnormal change events include changes in access paths and changes in the connection order of multi-table accesses.

[0127] The result determination unit is used to determine the comparison results based on topological similarity and abnormal change events.

[0128] According to embodiments of this application, the execution plan comparison module further includes a difference frequency statistics submodule and a use case weight adjustment submodule.

[0129] The difference frequency statistics submodule is used to determine the difference frequency between the use case execution plan and the baseline execution plan for each execution result.

[0130] The use case weight adjustment submodule is used to adjust the use case weights of the execution results when the difference frequency is greater than the frequency threshold. The use case weights are used to scale the use case features.

[0131] According to an embodiment of this application, the database testing apparatus 400 further includes a specification change parsing module.

[0132] The specification change parsing module includes a change information determination submodule, a change use case determination submodule, and a use case update submodule.

[0133] The Change Information Determination Submodule is used to compare the first specification document before the specification change and the second specification document after the specification change when there is a specification change in the database language of the database application under test, and to determine at least one change information.

[0134] The Change Test Case Determination Submodule is used to determine change test cases from multiple test cases in the test case set. Each change test case includes at least one change information.

[0135] The test case update submodule is used to update the changed test cases based on at least one change information included in the changed test cases, to obtain updated test cases, and to update the changed test cases in the test case set to updated test cases.

[0136] According to embodiments of this application, the use case update submodule includes a change type determination unit, a syntax change processing unit, a function functionality change processing unit, and a target value change processing unit.

[0137] The change type determination unit is used to determine the change type of the change information based on the change information. The change type includes syntax change, function function change, and target value change.

[0138] The syntax change processing unit is used to, when the change type is determined to be a syntax change, replace the first function in the change test case with the second function based on the first function before the change and the second function after the change, to obtain the update test case.

[0139] The function change processing unit is used to determine the target function with function change when the change type is determined to be function change, based on the change information, the first function and the second function of the target function, the first function being the function function before the change, and the second function being the function function after the change; calculate the second expected result after the function change based on the first expected result, the first function and the second function in the change test case; update the first expected result to the second expected result to obtain the update test case.

[0140] The target value change processing unit is used to update the first expected result based on the change information when the change type is determined to be a target value change, thereby obtaining an updated use case.

[0141] According to an embodiment of this application, the target use case determination module 450 includes a depth median determination submodule and a target use case selection submodule.

[0142] The Deep Median Determination submodule is used to determine the deep median for each cluster based on the stack depth of each of the multiple exception test cases in the cluster.

[0143] The target use case selection submodule is used to determine the target use case from multiple abnormal use cases in the cluster based on selection conditions. The selection conditions include at least one of the following: the stack depth of the abnormal use case is within the depth range, the log integrity of the abnormal use case is higher than the integrity threshold, and the number of lines of the use case script of the abnormal use case is less than the script size threshold. The depth range is determined based on the median depth and the preset offset amplitude, and the script size threshold is determined based on the number of lines of the use case script of each of the multiple abnormal use cases and the preset value percentage.

[0144] According to embodiments of this application, any multiple modules among the code parsing module 410, test case execution module 420, exception determination module 430, clustering module 440, target test case determination module 450, and report generation module 460 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the code parsing module 410, test case execution module 420, exception determination module 430, clustering module 440, target test case determination module 450, and report generation module 460 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the code parsing module 410, test case execution module 420, exception determination module 430, clustering module 440, target test case determination module 450, and report generation module 460 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0145] Figure 5 A block diagram schematically illustrates an electronic device suitable for implementing a database testing method according to an embodiment of this application.

[0146] like Figure 5 As shown, an electronic device 500 according to an embodiment of this application includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0147] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 502 and / or RAM 503. It should be noted that the aforementioned programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in one or more of the aforementioned memories.

[0148] According to embodiments of this application, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0149] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0150] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 502 and / or RAM 503 and / or one or more memories other than ROM 502 and RAM 503 described above.

[0151] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0152] When the computer program is executed by the processor 501, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0153] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0154] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0155] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0157] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0158] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A database testing method, characterized in that, The method includes: In response to the detection of an update to the kernel code of the database under test, the kernel code is parsed to determine the existence of updated difference code and the object under test corresponding to the difference code; Multiple test cases are executed on the database to be tested to obtain the execution results of each test case. The test cases are test cases that include the object to be tested, and the test cases are obtained from a pre-set set of test cases. Based on the update path of the test object indicated by the difference code, the result to be analyzed is determined from multiple abnormal results in the execution results. The update path represents the mapping relationship between the first return value of the test object before the update and the second return value after the update. When the execution result indicates that the test case failed, the execution result is determined to be an abnormal result, and the test case with the abnormal result is the abnormal test case corresponding to the abnormal result. Based on the abnormal features indicated by each of the multiple results to be analyzed, the multiple results to be analyzed are clustered to obtain multiple clusters; For each cluster, based on the use case characteristics of each of the multiple anomalous use cases in the cluster, a target use case is determined from the multiple anomalous use cases in the cluster; and Based on multiple target use cases and the anomaly characteristics corresponding to the multiple target use cases, the cause of the anomaly is determined, and a test report is generated based on the cause of the anomaly.

2. The method according to claim 1, characterized in that, The method further includes: For the multiple execution results, the test cases to be tested for each of the multiple execution results are structured and parsed to obtain the test case execution plan corresponding to each test case; The test case execution plan is compared with a baseline execution plan to obtain a comparison result. The baseline execution plan is determined from an execution plan library, which includes multiple execution plans obtained by executing the test cases using multiple versions of the database under test. The baseline execution plan is determined based on the performance of each of the multiple execution plans. If the comparison result indicates that the use case execution plan is inconsistent with the baseline execution plan, structural difference information between the use case execution plan and the baseline execution plan is generated based on the comparison result, and the structural difference information is sent to the testers in the database to be tested.

3. The method according to claim 2, characterized in that, The step of comparing the use case execution plan with the baseline execution plan to obtain the comparison result includes: Based on the use case execution plan and the baseline execution plan, a first topology corresponding to the use case execution plan and a second topology corresponding to the baseline execution plan are determined; Based on multiple nodes in the first topology and the second topology, and the connection relationships between the multiple nodes, the topological similarity is determined. The execution logic of the use case execution plan and the baseline execution plan are compared to identify abnormal change events. The execution logic includes access paths and the join order of multi-table accesses. The abnormal change events include changes in access paths and changes in the join order of multi-table accesses. The comparison result is determined based on the topological similarity and the abnormal change event.

4. The method according to claim 2 or 3, characterized in that, The method further includes: For each execution result, determine the frequency of discrepancies between the use case execution plan and the baseline execution plan for that execution result; and If the difference frequency is greater than a frequency threshold, the use case weight of the execution result is adjusted, and the use case weight is used to scale the use case features.

5. The method according to claim 1, characterized in that, The method further includes: In the event of a specification change in the database language of the database application under test, the first specification document before the specification change and the second specification document after the specification change are compared to determine at least one change information. A modified test case is determined from a plurality of test cases in the test case set, the modified test case including at least one of the modified information; Based on at least one of the change information included in the change use case, the change use case is updated to obtain an updated use case; and Update the changed test cases in the test case set to the updated test cases.

6. The method according to claim 5, characterized in that, The step of updating the change use case based on at least one of the change information included in the change use case to obtain an updated use case includes: Based on the change information, the change type of the change information is determined, and the change type includes syntax change, function function change, and target value change; If the change type is determined to be a syntax change, the update use case is obtained by replacing the first function in the change use case with the second function based on the first function before the change and the second function after the change. If the change type is determined to be a function function change, based on the change information, the target function with the function function change is determined, as well as the first function and the second function of the target function. The first function is the function function before the function change, and the second function is the function function after the function change. Based on the first expected result, the first function, and the second function in the changed use case, calculate the second expected result after the function function change; The first expected result is updated to the second expected result to obtain the updated use case; and If the change type is determined to be a target value change, the first expected result is updated based on the change information to obtain the updated use case.

7. The method according to claim 1, characterized in that, The use case characteristics include stack depth, log integrity, and number of lines of use case script; For each cluster, determining the target use case from the multiple abnormal use cases in the cluster based on their respective use case characteristics includes: For each cluster, the median depth is determined based on the stack depth of each of the multiple abnormal use cases in the cluster; as well as Based on selection criteria, the target use case is determined from multiple abnormal use cases in the cluster. The selection criteria include at least one of the following: the stack depth of the abnormal use case is within a depth range, the log integrity of the abnormal use case is higher than an integrity threshold, and the number of use case script lines of the abnormal use case is less than a script size threshold. The depth range is determined based on the median depth and a preset offset, and the script size threshold is determined based on the number of use case script lines of each of the multiple abnormal use cases and a preset value percentage.

8. A database testing device, characterized in that, The device includes: The code parsing module is used to parse the kernel code in response to the detection of an update in the kernel code of the database to be tested, and to determine the existence of updated difference code and the object to be tested corresponding to the difference code; The test case execution module is used to execute multiple test cases on the database to be tested, and obtain the execution results of each of the multiple test cases. The test cases are test cases that include the object to be tested, and the test cases are obtained from a pre-set test case set. An anomaly determination module is used to determine the result to be analyzed from multiple abnormal results in the execution results based on the update link of the object under test indicated by the difference code. The update link represents the mapping relationship between the first return value of the object under test before the update and the second return value after the update. When the execution result indicates that the test case under test has failed, the execution result is determined to be an abnormal result, and the test case under test with the abnormal result is the abnormal test case corresponding to the abnormal result. The clustering module is used to cluster the multiple results to be analyzed based on the abnormal features indicated by each of the multiple results to be analyzed, so as to obtain multiple clusters; The target use case determination module is configured to, for each cluster, determine a target use case from the multiple abnormal use cases in the cluster based on the use case characteristics of each abnormal use case in the cluster; and The report generation module is used to determine the cause of the anomaly based on multiple target use cases and the anomaly characteristics corresponding to the multiple target use cases, and to generate a test report based on the cause of the anomaly.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images