Test method of database management system, related device and medium

Abstract

The invention provides a test method of a database management system, a related device and a medium. The method comprises the steps of obtaining a first use case; determining a target verification strategy corresponding to the first use case in the plurality of candidate verification strategies; based on the first use case, constructing a reference use case by using a construction rule; applying the first use case to the target corpus to obtain a first query result, and applying the reference use case to the target corpus to obtain a second query result; based on a comparison condition, comparing the first query result with the second query result to obtain a singleton comparison result; performing variation on the first use case, and returning to the step of constructing the reference use case by using the construction rule based on the first use case until the variation frequency of the first use case reaches a preset frequency; and determining a test result of the database management system based on the comparison results of the single examples. The test coverage rate of the database management system can be improved. The method and the device can be applied to various scenes such as big data, database management and computer technologies.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a testing method, related apparatus and medium for a database management system. Background Technology

[0002] Currently, testing of query engines for Database Management Systems (DBMS) often includes usability testing and correctness testing. Usability testing verifies whether inputting query statements (SQL statements) into the DBMS yields correct results, while correctness testing verifies whether the results obtained from inputting query statements are correct. Related technologies often utilize manual verification for DBMS correctness testing. However, in practice, it's difficult to exhaustively verify all query statements; correctness testing is usually limited to a finite number of queries, resulting in low test coverage. Furthermore, manual verification becomes inefficient when dealing with large-scale DBMS systems, further reducing testing efficiency. Summary of the Invention

[0003] This disclosure provides a testing method, related apparatus, and medium for a database management system, which can improve the test coverage of the database management system.

[0004] According to one aspect of this disclosure, a testing method for a database management system is provided, the method comprising:

[0005] Obtain the first use case;

[0006] Among multiple candidate verification strategies, a target verification strategy corresponding to the first use case is determined, wherein the target verification strategy includes a construction rule for constructing a reference use case based on the first use case, and a comparison condition between a first query result corresponding to the first use case and a second query result corresponding to the reference use case;

[0007] Based on the first use case, the reference use case is constructed using the construction rules;

[0008] The first use case is applied to the target corpus to obtain the first query result, and the reference use case is applied to the target corpus to obtain the second query result;

[0009] Based on the comparison conditions, the first query result and the second query result are compared to obtain a single-instance comparison result;

[0010] The first use case is mutated, and the step of constructing the reference use case based on the first use case and using the construction rules is returned, until the number of mutations of the first use case reaches a predetermined number;

[0011] Based on the comparison results of multiple single instances, the test results of the database management system are determined.

[0012] According to one aspect of this disclosure, a testing apparatus for a database management system is provided, the apparatus comprising:

[0013] The acquisition unit is used to acquire the first use case;

[0014] The first determining unit is configured to determine the target verification strategy corresponding to the first use case from among multiple candidate verification strategies, wherein the target verification strategy includes a construction rule for constructing a reference use case based on the first use case, and a comparison condition between a first query result corresponding to the first use case and a second query result corresponding to the reference use case.

[0015] A construction unit is configured to construct the reference use case based on the first use case and using the construction rules.

[0016] The query unit is configured to apply the first use case to the target corpus to obtain the first query result, and apply the reference use case to the target corpus to obtain the second query result;

[0017] The comparison unit is used to compare the first query result and the second query result based on the comparison conditions to obtain a single-instance comparison result;

[0018] The mutation unit is used to mutate the first use case and return the step of constructing the reference use case based on the first use case and using the construction rules, until the number of mutations of the first use case reaches a predetermined number.

[0019] The second determining unit is used to determine the test results of the database management system based on multiple single-instance comparison results.

[0020] Optionally, the first determining unit is used to:

[0021] Determine the query statement structure for the first use case;

[0022] Based on predetermined verification rules, the query field of the first use case is verified to obtain the verification result, which is used to indicate whether the target field exists in the first use case.

[0023] Based on the query statement structure and the verification results, the target verification strategy is determined from multiple candidate verification strategies.

[0024] Optionally, the target verification strategy includes multiple construction rules for constructing multiple reference use cases based on the first use case, and comparison conditions for a first query result corresponding to the first use case and multiple second query results corresponding to the multiple reference use cases; the comparison conditions include a first comparison sub-condition between multiple second query results and a second comparison sub-condition between the first query result and each second query result;

[0025] The comparison unit is used for:

[0026] Based on the first comparison sub-condition, the first comparison sub-result is obtained by comparing each pair of multiple second query results;

[0027] Based on the second comparison sub-condition, a comparison is made between the first query result and each of the second query results to obtain the second comparison sub-result;

[0028] The singleton comparison result is generated based on the first comparison result and the second comparison result.

[0029] Optionally, the first use case includes a first segmentation method and a target query condition, the reference use case includes a second segmentation method and the target query condition, and the construction rule is to convert the first segmentation method in the first use case into the second segmentation method while keeping the target query condition unchanged;

[0030] The query unit is used for:

[0031] The first segmentation method is applied to the target corpus to obtain the first segmented corpus segment;

[0032] Apply the target query condition to each of the first segmented corpus segments to obtain the first subquery result corresponding to each of the first segmented corpus segments;

[0033] The first subquery results corresponding to each of the first segmented corpus segments are integrated into the first query result;

[0034] The second segmentation method is applied to the target corpus to obtain the second segmented corpus segment;

[0035] Apply the target query condition to each of the second segmented corpus segments to obtain the second subquery result corresponding to each of the second segmented corpus segments;

[0036] The results of the second subquery corresponding to each of the second segmented corpus segments are integrated into the second query result.

[0037] Optionally, the target verification strategy includes multiple construction rules for constructing multiple reference use cases based on the first use case, and comparison conditions for the first query result corresponding to the first use case and multiple second query results corresponding to the multiple reference use cases;

[0038] The comparison unit is used for:

[0039] Based on the comparison conditions, the second query results corresponding to the multiple reference use cases are merged to obtain the merged second query result, and the first query result and the merged second query result are compared.

[0040] If the first query result is the same as the merged second query result, then the single-instance comparison result is determined to be a positive result.

[0041] If the first query result is different from the merged second query result, then the singleton comparison result is determined to be a negative result.

[0042] Optionally, the construction rule is used to indicate that when the first use case has a first field and the target position of the first use case does not have a target function field, the reference use case is constructed using the second field corresponding to the first field;

[0043] The construction unit is used for:

[0044] Based on the construction rules, field detection is performed on the first use case;

[0045] If it is determined that the first use case has a first field and the target position of the first use case does not have a target function field, then based on the second field, the first segmentation method in the first use case is converted into a second segmentation method, while keeping the target query conditions in the first use case unchanged, to obtain the reference use case.

[0046] Optionally, the construction unit is used for:

[0047] Detect the query engine filtering module of the database management system;

[0048] If it is determined that there is optimization logic for the first use case in the query engine filtering module, then the position of at least one condition field of the first use case is adjusted using the construction rules to obtain the reference use case.

[0049] Optionally, the mutation unit is used for:

[0050] For the first use case, the target number is determined within the first use case;

[0051] In the first use case, the target number is replaced with a number to obtain the modified first use case.

[0052] Optionally, the step of replacing the target number includes:

[0053] The target number is increased or decreased according to a predetermined rule, so as to replace the target number with the increased or decreased target number;

[0054] or;

[0055] Take the first value from the preset first set and replace the target number with the first value.

[0056] Optionally, the mutation unit is used for:

[0057] For the first use case, the target operator is determined in the first use case;

[0058] Take a reference operator that is different from the target operator from the preset second set;

[0059] In the first use case, the target operator is replaced with the reference operator to obtain the mutated first use case.

[0060] Optionally, the mutation unit is used for:

[0061] Perform a non-empty check on the first use case;

[0062] If it is determined that the first use case contains a non-empty logical field, then the non-empty logical field of the first use case is replaced with an empty logical field to obtain the mutated first use case;

[0063] If it is determined that the first use case contains an empty logical field, then the empty logical field of the first use case is replaced with a non-empty logical field to obtain the mutated first use case.

[0064] Optionally, the mutation unit is used for:

[0065] For the first use case, the target expression is determined from multiple candidate expressions of the first use case;

[0066] In the first use case, the target expression is deleted to obtain the mutated first use case.

[0067] Optionally, the mutation unit is used for:

[0068] Construct new expressions;

[0069] The newly added expression is embedded into the first use case to obtain the mutated first use case.

[0070] Optionally, embedding the new expression into the first use case includes:

[0071] The newly added expression is checked, and after the expression check is passed, the newly added expression is embedded into the first use case;

[0072] or;

[0073] The newly added expression is embedded into the first use case to obtain an intermediate query statement. The intermediate query statement is then subjected to syntax verification. After the syntax verification passes, the intermediate query statement is identified as the modified first use case.

[0074] Optionally, the mutation unit is used for:

[0075] In the first use case, determine the target logical field used to indicate the sorting method of the first query results;

[0076] Take out a sorting logical field that is different from the target logical field from the preset third set;

[0077] In the first use case, the target logical field is replaced with the sorting logical field to obtain the modified first use case.

[0078] Optionally, the predetermined number of times is determined in the following way:

[0079] Determine the query statement structure of the first use case and determine the total amount of data in the database management system;

[0080] The predetermined number of times is determined based on the structure of the query statement and the total amount of data.

[0081] Optionally, determining the predetermined number of times based on the query statement structure and the total amount of data includes:

[0082] Based on the structure of the query statement, determine the first score;

[0083] Based on the total amount of data, determine the second score;

[0084] The predetermined number of times is determined based on the first score and the second score.

[0085] Optionally, the testing apparatus for the database management system further includes an update unit, which is used for:

[0086] When the singleton comparison result is negative, determine the mutation count update strategy for the first use case;

[0087] The predetermined number of mutations is updated based on the mutation number update strategy to increase the predetermined number of mutations, and the updated predetermined number of mutations is not greater than the first value.

[0088] Optionally, the construction unit is used for:

[0089] Based on the construction rules, field detection is performed on the first use case; if it is determined that the first use case does not include an aggregate function field, the conditional expression in the first use case is pre-expressed to obtain the reference use case;

[0090] The first query result indicates the number of data items retrieved, and the second query result indicates the total number of data items retrieved; the comparison unit is used for:

[0091] Based on the comparison conditions, the data items in the first query result are statistically analyzed to obtain the data item statistical results;

[0092] If it is determined that the statistical result of the data item is the same as the second query result, then the single-case comparison result is determined to be a positive result.

[0093] Optionally, the second determining unit is used to:

[0094] Determine the first proportion of singleton comparison results that are negative among the multiple singleton comparison results;

[0095] Based on the first percentage, the test results of the database management system are determined, wherein the test results are used to indicate the degree of logical defects in the database management system.

[0096] Optionally, the construction unit is used for:

[0097] Based on the construction rules, target data items are extracted from the target corpus; the target data items are embedded into the first use case to obtain the reference use case;

[0098] The comparison unit is used for:

[0099] If it is determined that the target data item is not found in the second query result, then the singleton comparison result is determined to be negative.

[0100] According to one aspect of this disclosure, an electronic device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement a test method for a database management system as described above.

[0101] According to one aspect of this disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program that, when executed by a processor, implements the test method for the database management system as described above.

[0102] According to one aspect of this disclosure, a computer program product is provided, comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the database management system testing method as described above.

[0103] In this embodiment, a consistency check is used to test the correctness of SQL queries in a database management system. Specifically, firstly, a first use case is obtained; then, a target verification strategy corresponding to the first use case is determined from multiple candidate verification strategies. The target verification strategy includes construction rules for constructing a reference use case based on the first use case, and comparison conditions for the first query result corresponding to the first use case and the second query result corresponding to the reference use case. Different methods are provided to construct logically equivalent SQL query statements for consistency verification for different SQL query statements, making it easy to construct a reference use case based on the first use case using the construction rules, thus improving the standardization and efficiency of reference use case construction. Further, the first use case is applied to the target corpus to obtain a first query result, and the reference use case is applied to the target corpus to obtain a second query result. Based on the comparison conditions, the first query result and the second query result are compared to obtain a singleton comparison result. If the singleton comparison result indicates that the first query result and the second query result are inconsistent, logical defects in the database management system can be exposed. Furthermore, the first test case is mutated, and more test cases are automatically expanded for testing using an SQL mutation strategy. The process then returns the steps for constructing reference test cases based on the first test case and using construction rules, until the first test case has been mutated a predetermined number of times. This method enables rapid expansion of the first test case and solves the problem of low efficiency in manual SQL query correctness verification. Finally, based on the comparison results of multiple single cases, the test results of the database management system are determined, enabling coverage of more SQL verification scenarios during testing and improving the test coverage of the database management system.

[0104] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0105] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0106] Figure 1 This is a system architecture diagram of the database management system testing method applied according to an embodiment of the present disclosure;

[0107] Figure 2 A schematic diagram is shown illustrating the application of a database management system testing method according to an embodiment of the present disclosure in a database testing scenario;

[0108] Figure 3 This is a flowchart of a testing method for a database management system according to an embodiment of the present disclosure;

[0109] Figure 4 This is a flowchart illustrating how a target verification strategy is determined from multiple candidate verification strategies, according to an embodiment of this disclosure.

[0110] Figure 5 This is a schematic diagram illustrating the implementation process of determining the target verification strategy from multiple candidate verification strategies according to an embodiment of this disclosure;

[0111] Figure 6 This is a flowchart illustrating the generation of singleton comparison results according to an embodiment of this disclosure;

[0112] Figure 7 This is a flowchart illustrating the generation of a first query result and a second query result according to an embodiment of this disclosure;

[0113] Figure 8 This is a flowchart illustrating the generation of singleton comparison results according to another embodiment of this disclosure;

[0114] Figure 9 This is a flowchart illustrating the construction of a reference use case according to an embodiment of this disclosure;

[0115] Figure 10 This is a flowchart illustrating the construction of a reference use case for another embodiment of this disclosure;

[0116] Figure 11 This is a flowchart illustrating the construction of a reference use case according to another embodiment of this disclosure;

[0117] Figure 12 This is a flowchart illustrating the construction of a reference use case according to another embodiment of this disclosure;

[0118] Figure 13 This is a flowchart illustrating a variation of a first use case according to an embodiment of this disclosure;

[0119] Figure 14This is a flowchart illustrating a variation of the first use case according to another embodiment of this disclosure;

[0120] Figure 15 This is a flowchart illustrating a variation of the first use case according to another embodiment of this disclosure;

[0121] Figure 16 This is a flowchart illustrating a variation of the first use case according to another embodiment of this disclosure;

[0122] Figure 17 This is a flowchart illustrating a variation of the first use case according to another embodiment of this disclosure;

[0123] Figure 18 This is a flowchart illustrating a variation of the first use case according to another embodiment of this disclosure;

[0124] Figure 19 This is a flowchart illustrating the determination of a predetermined number of times according to an embodiment of the present disclosure;

[0125] Figure 20 This is a flowchart illustrating the number of times a predetermined number of updates is performed according to an embodiment of this disclosure;

[0126] Figure 21A This is a schematic diagram illustrating the process of determining a predetermined number of times according to an embodiment of the present disclosure;

[0127] Figure 21B This is a schematic diagram illustrating the implementation process of updating a predetermined number of times according to an embodiment of the present disclosure;

[0128] Figure 22 This is a schematic diagram illustrating the implementation details of a testing method for a database management system according to an embodiment of the present disclosure;

[0129] Figure 23 This is a block diagram of a test apparatus for a database management system according to an embodiment of the present disclosure;

[0130] Figure 24 This is a terminal structure diagram of a testing method for a database management system according to an embodiment of the present disclosure;

[0131] Figure 25 This is a server structure diagram of a testing method for a database management system according to an embodiment of the present disclosure. Detailed Implementation

[0132] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0133] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0134] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Pre-trained models, also known as large models or foundational models, can be widely applied to downstream tasks in various areas of AI after fine-tuning. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning. With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0135] A Database Management System (DBMS) is a large software application that manipulates and manages databases, used for creating, using, and maintaining them.

[0136] Structured Query Language (SQL) is a special-purpose programming language used to access, query, update, and manage relational database systems.

[0137] System architecture and scenario description of the embodiments disclosed herein

[0138] Figure 1 This is a system architecture diagram applied to the testing method of the database management system according to embodiments of the present disclosure. It includes an object terminal 140, an Internet 130, a gateway 120, a database management system server 110, and a target corpus 150, etc.

[0139] The target terminal 140 includes various forms such as desktop computers, laptops, PDAs (Personal Digital Assistants), mobile phones, in-vehicle terminals, home theater terminals, and dedicated terminals. Furthermore, it can be a single device or a collection of multiple devices. The target terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data. The target terminal 140 includes a database query platform, which is used by the target to input various forms of query statements and request the database management system to search for matching data in the target corpus 150 based on the input query statements. The database query platform is also used by the target to input sample query statements for testing and to perform query correctness tests on the database management system based on the sample query statements.

[0140] Database management system server 110 refers to a computer system that can provide certain services to object terminal 140. Compared with ordinary object terminal 140, database management system server 110 has higher requirements in terms of stability, security, and performance. Database management system server 110 can be a high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines). Server 110 contains various types of services, and the implementation of each service of database management system server 110 is often associated with some intermediate databases or storage media. Database management system server 110 is used to search for matching query statements in target corpus 150 based on query statement samples input by the object, and also to perform query correctness tests on database management system based on sample query statements input by the object, so as to expose logical defects in database management system.

[0141] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from object terminal 140 to database management system server 110 are forwarded to the corresponding server via gateway 120. Messages sent from database management system server 110 to object terminal 140 are also forwarded to the corresponding object terminal 140 via gateway 120.

[0142] The embodiments disclosed herein can be applied in various scenarios, such as Figure 2 The database test scenario shown is an example.

[0143] like Figure 2As shown, when performing correctness testing on a database management system, firstly, test cases containing multiple seed SQL statements are input into the test case translation unit (IR translation). The IR translation unit analyzes the statement structure of each seed SQL statement, extracting its structure while preserving its framework, thus removing its specific content. Next, the statement structure of the seed SQL statements is input into the syntax-preserving mutation unit. The mutation unit performs mutation operations on the statement structure of the seed SQL statements through various methods such as insertion, deletion, and addition, to expand into more statement mutation structures equivalent to the seed SQL statements. Further, the semantics-guided instantiation unit analyzes the generated statement mutation structures, filling in the specific content of each seed SQL statement into the mutation structures, resulting in mutated SQL statements that differ in form from the seed SQL statements but have the same meaning. Finally, the seed SQL statement and the variant SQL statement are input into the test unit (fuzzing). The test unit uses consistency comparison rules to construct consistency comparison SQL corresponding to the seed SQL statement and consistency comparison SQL corresponding to the variant SQL statement. Based on the comparison of the query results of the seed SQL statement and the consistency comparison SQL, as well as the comparison of the query results of the variant SQL statement and the corresponding consistency comparison SQL, it is determined whether there are logical defects in the database management system, so as to realize the query correctness test of the database management system.

[0144] General Description of Embodiments in this Disclosure

[0145] According to one embodiment of this disclosure, a testing method for a database management system is provided.

[0146] The testing methods for this database management system are generally applied in business scenarios where the query accuracy of the database management system is highly critical, for example... Figure 2 The diagram illustrates a database testing scenario. This disclosure provides a scheme for constructing reference test cases based on an input first test case to perform correctness testing on a database management system, and for automatically mutating the first test cases to generate more first test cases for testing. This approach can improve the test coverage of the database management system during correctness testing.

[0147] like Figure 3 As shown, a testing method for a database management system according to an embodiment of this disclosure may include:

[0148] Step 310: Obtain the first use case;

[0149] Step 320: Determine the target verification strategy corresponding to the first use case from multiple candidate verification strategies;

[0150] Step 330: Based on the first use case, construct a reference use case using the construction rules;

[0151] Step 340: Apply the first use case to the target corpus to obtain the first query result, and apply the reference use case to the target corpus to obtain the second query result;

[0152] Step 350: Based on the comparison conditions, compare the first query result and the second query result to obtain the singleton comparison result;

[0153] Step 360: Mutate the first use case and return the steps of constructing a reference use case based on the first use case using the construction rules, until the number of mutations of the first use case reaches the predetermined number;

[0154] Step 370: Based on the comparison results of multiple singletons, determine the test results of the database management system.

[0155] Steps 310-370 are described in detail below.

[0156] In step 310, the first use case is obtained.

[0157] The first use case refers to the SQL query statement used to input into a database management system to query the target corpus.

[0158] The target corpus refers to multiple corpus information that can be linked through a database management system, used to provide matching query results for queries input into the database management system.

[0159] For example, if a first use case indicates: find results greater than 0 in the target corpus [0,1,2,3,5], then running the first use case in the database management system and applying it to the target corpus will yield the query result [1,2,3,5].

[0160] In the specific implementation of this embodiment, since the first use case is pre-constructed for the correctness testing of the database management system, multiple pre-constructed first use cases can be obtained with authorization.

[0161] It should be noted that the query conditions in different first use cases may differ to some extent, and the SQL statement format (query statement structure) of the first use case may also differ to some extent.

[0162] In step 320, the target verification strategy corresponding to the first use case is determined from multiple candidate verification strategies.

[0163] Since the same validation strategy is often not applicable to query statements with various SQL statement formats, in order to improve test coverage, this disclosure proposes to design strategies applicable to query statements with different SQL statement formats, i.e., construct multiple candidate validation strategies.

[0164] Each candidate validation strategy is used to provide a construction rule corresponding to a SQL statement format based on the SQL statement format of the SQL query statement, so as to generate a reference SQL statement (i.e., reference test case) for consistency testing corresponding to the SQL query statement using the construction rule. In addition, the candidate validation strategy is also used to provide the relationship that the SQL query statement and the reference SQL statement should satisfy when applied to the same corpus.

[0165] Based on this, the target verification strategy of this disclosure consists of two parts: the target verification strategy includes the construction rules for constructing reference use cases based on the first use case, and the comparison conditions for the first query result corresponding to the first use case and the second query result corresponding to the reference use case.

[0166] The construction rules are used to instruct how to construct a reference use case that matches the first use case. The construction rules involve information such as the format judgment of the first use case and the structural framework of the reference use case.

[0167] The comparison condition is used to indicate under what conditions the first query result corresponding to the first use case and the second query result corresponding to the reference use case are equivalent.

[0168] In this specific implementation, since the multiple candidate verification strategies are designed based on the query statement structure (SQL statement format) of each first use case, when selecting the target verification strategy, the format of the first use case can first be parsed to determine the query statement structure of the first use case. Then, the candidate verification strategy that matches the query statement structure is selected as the target verification strategy.

[0169] In step 330, a reference use case is constructed based on the first use case and using the construction rules.

[0170] In the specific implementation of this embodiment, based on the first use case, the SQL statement format of the first use case is adjusted using construction rules, and the query conditions in the first use case are added to the adjusted SQL statement format without changing them, to obtain the reference use case.

[0171] In step 340, the first use case is applied to the target corpus to obtain the first query result, and the reference use case is applied to the target corpus to obtain the second query result.

[0172] In this specific implementation, firstly, a first use case is executed in the database management system and applied to the target corpus to find results in the target corpus that match the query conditions in the first use case, thus obtaining a first query result. Next, a reference use case is executed in the database management system and applied to the target corpus to find results in the target corpus that match the query conditions in the reference use case, thus obtaining a second query result.

[0173] In step 350, the first query result and the second query result are compared based on the comparison conditions to obtain a singleton comparison result.

[0174] The singleton comparison result is used to affirm or negate the first and second query results. A positive result indicates that the first and second query results meet the equivalence requirements specified in the comparison conditions. A negative result indicates that the first and second query results do not meet the equivalence requirements specified in the comparison conditions.

[0175] It should be noted that for a given SQL statement format, the way reference test cases are constructed based on the first test case and the comparison conditions are mutually matched.

[0176] To save space, the various scenarios of constructing reference use cases based on the construction rules of the target verification strategy and the various scenarios of obtaining single-instance comparison results based on comparison conditions in the embodiments of this disclosure will be described in detail below, and will not be repeated here.

[0177] In step 360, the first use case is mutated, and the steps of constructing a reference use case based on the first use case and using the construction rules are returned until the number of mutations of the first use case reaches a predetermined number.

[0178] Variation refers to adjusting the statement structure of the first test case, or adding, deleting, or modifying the query conditions in the first test case, in order to expand more query statements for testing.

[0179] The number of mutations refers to the cumulative number of mutations performed on the first use case.

[0180] The predetermined number of times is used to limit the maximum number of mutations to be performed on the first use case.

[0181] To save space, the specific process of modifying the first use case in this embodiment will be described in detail below, and will not be repeated here.

[0182] Furthermore, after a new use case is mutated, the new use case replaces the original first use case, and the process returns to the step of constructing a reference use case based on the first use case using construction rules. This allows for the construction of a corresponding reference use case using the new first use case, thereby generating more singleton comparison results to expand the SQL query statements used for testing the database management system. However, since the mutation methods of each first use case are often diverse, exhausting all mutation possibilities would result in an excessive number of mutated new first use cases, leading to significant computational resource consumption. Therefore, this embodiment limits the maximum number of mutations for the first use case; that is, when the number of mutations for the first use case reaches a predetermined number, further mutations are stopped. Specifically, for each SQL query statement mutated from the first use case, the mutation count of the first use case is incremented by 1.

[0183] It's important to note that the mutation counts of the first use case and the SQL query statements mutated from it are not accumulated together. For the first use case, mutations are only counted when the first use case is used for mutation. Mutations of SQL query statements mutated from the first use case, however, are included in the mutation count of those mutated SQL query statements.

[0184] Step 370: Based on the comparison results of multiple singletons, determine the test results of the database management system.

[0185] The test results are used to indicate the extent of logical flaws in the database management system.

[0186] In this specific implementation, firstly, a first proportion of singleton comparison results with a negative result is determined among multiple singleton comparison results; then, based on the first proportion, the test result of the database management system is determined. The test result indicates the degree of logical defect in the database management system.

[0187] Specifically, first, all singleton comparison results are counted to obtain the total number of singleton comparison results. Next, the singleton comparison results that are negative are counted to obtain the total number of negative results. Further, the total number of negative results is divided by the total number of singleton comparison results to obtain the first proportion. Finally, based on the magnitude of the first proportion, the corresponding logical defect level in the proportion interval and logical defect severity table is looked up, and the logical defect severity corresponding to the proportion interval containing the first proportion is taken as the logical defect severity of the database management system, thus obtaining the test result.

[0188] For example, the table showing the correspondence between percentage ranges and the degree of logical defects indicates that: when the first percentage is between 0-30%, the logical defect is minor (high query accuracy); when the first percentage is between 31-60%, the logical defect is moderate (average query accuracy); and when the first percentage is between 61-100%, the logical defect is severe (low query accuracy). Based on this, when the first percentage is 54%, the test result for the database management system is that it has a moderate defect and the query accuracy is average.

[0189] Through steps 310-370 above, this embodiment of the disclosure considers using consistency detection to test the correctness of SQL queries in a database management system. SQL query statements are used as test cases, and corresponding consistency verification strategies are designed for different test cases to construct comparison test cases. The query results obtained by putting the test cases and comparison test cases into the same dataset are compared, achieving the goal of automatically testing the query correctness of the database management system. Furthermore, this disclosure also provides an automated mutation method for test cases during automated testing to generate more test cases for verification. Specifically, firstly, a first test case is obtained; then, a target verification strategy corresponding to the first test case is determined from multiple candidate verification strategies. The target verification strategy includes construction rules for constructing reference test cases based on the first test case, and comparison conditions for the first query result corresponding to the first test case and the second query result corresponding to the reference test case. Different methods are provided to construct logically equivalent SQL query statements for consistency verification for different SQL query statements, so that reference test cases can be easily constructed based on the first test case using the construction rules, improving the standardization and efficiency of reference test case construction. Furthermore, the first use case is applied to the target corpus to obtain the first query result, and the reference use case is applied to the target corpus to obtain the second query result. Based on the comparison conditions, the first query result and the second query result are compared to obtain a singleton comparison result. If the singleton comparison result indicates that the first query result and the second query result are inconsistent, then a logical defect in the database management system can be exposed. Further, the first use case is mutated, and more test cases are automatically expanded for testing through an SQL mutation strategy. The steps of constructing a reference use case based on the first use case and using construction rules are returned until the first use case has been mutated a predetermined number of times. This method can achieve rapid expansion of the first use case and solves the problem of low efficiency in manual SQL query correctness verification. Finally, based on multiple singleton comparison results, the test result of the database management system is determined, which can cover more SQL verification scenarios during testing and improve the test coverage of the database management system.

[0190] The above is a general description of steps 310-340. Since steps 310 and 370 have been detailed in the above general description, the specific implementations of steps 320, 330, 340, 350 and 360 will be described in detail below.

[0191] Detailed description of step 320

[0192] In step 320, a target verification strategy corresponding to the first use case is determined from multiple candidate verification strategies. The target verification strategy includes construction rules for constructing a reference use case based on the first use case, and comparison conditions between the first query result corresponding to the first use case and the second query result corresponding to the reference use case.

[0193] Please refer to Figure 4 In one embodiment, step 320 specifically includes, but is not limited to, the following steps 410-430:

[0194] Step 410: Determine the query statement structure for the first use case;

[0195] Step 420: Perform query field validation on the first use case based on the predetermined validation rules to obtain the validation results;

[0196] Step 430: Based on the query statement structure and the test results, determine the target validation strategy from multiple candidate validation strategies.

[0197] Steps 410-430 are described in detail below.

[0198] In step 410, the query statement structure of the first use case is determined.

[0199] The query statement structure is used to indicate the SQL statement format for the first use case.

[0200] In the specific implementation of this embodiment, the statement format of the first use case is parsed to determine the field types contained in the first use case, and the query statement structure of the first use case is determined based on the field types, operators, and connectors contained in the first use case.

[0201] In step 420, the query fields of the first use case are validated based on the predetermined validation rules to obtain the validation results.

[0202] Predefined validation rules are used to check whether certain logical fields exist in the first use case. For example, predefined validation rules are used to check whether there are aggregate function fields (count function, sum function), fields with a limit on the number of returned results (limit limit), or fields with a deduplication limit (distinct limit), etc.

[0203] The test results are used to indicate whether the target field exists in the first use case.

[0204] In this specific implementation, the query fields of the first use case are checked based on predetermined checking rules to determine whether certain logical fields (target fields) exist in the first use case. When certain target fields exist in the first use case, the check result will indicate the target fields in the first use case and their location. When no target fields exist in the first use case, the check result will indicate that no target fields exist in the first use case.

[0205] In step 430, the target validation strategy is determined from multiple candidate validation strategies based on the query statement structure and the validation results.

[0206] In the specific implementation of this embodiment, since different candidate validation strategies have their own applicable SQL statement usage conditions, the query statement structure and the test results are compared with the usage conditions of each candidate validation strategy. Among the multiple candidate validation strategies, the one whose usage conditions include the query statement structure and the test results is determined as the target validation strategy.

[0207] For example, candidate validation strategy 1 is suitable for SQL statements that do not contain aggregate functions, while candidate validation strategy 2 is suitable for SQL statements that contain the count aggregate function.

[0208] like Figure 5 The diagram shows the correspondence between each candidate validation strategy and the first use case. Specifically, first use cases that satisfy the same SQL statement structure are integrated into a first use case set. For first use case set 1, a corresponding candidate validation strategy 1 is set. For first use case set 2, a corresponding candidate validation strategy 2 is set; ...; for first use case set N, a corresponding candidate validation strategy N is set, so that in steps 330-350 below, the query correctness of different forms of first use cases is validated according to different validation strategies.

[0209] The advantage of this embodiment is that, for query statements with different SQL statement formats, strategies applicable to each SQL statement format are designed, that is, multiple candidate verification strategies are constructed, and for the first use case, the target verification strategy that matches it is selected according to the query field verification results of the first use case and the SQL statement structure, which can improve the efficiency and accuracy of strategy selection.

[0210] Detailed description of steps 330-350

[0211] To improve test coverage and efficiency, this disclosure provides a scheme for testing the query correctness of a database management system based on result consistency detection. Result consistency detection involves applying the construction rules of the corresponding target verification strategy to any first use case, constructing a logically equivalent reference use case, and comparing the query results of the reference use case and the first use case in the same dataset according to the comparison conditions in the target verification strategy. This exposes logical flaws in the query correctness of the database management system when inconsistencies in the query results occur.

[0212] In this embodiment of the disclosure, considering the different SQL statement formats of each first use case, a corresponding consistency verification strategy (target verification strategy) is provided for each type of first use case. This enables the solution to support the coverage of more types of SQL statements and allows the test to cover more query scenarios and more logic on the query engine, thereby improving test coverage and the ability to discover problems in the database management system.

[0213] It should be noted that in each candidate verification strategy of this disclosure embodiment, the first query result of the first use case is required to be non-random, that is, the first query result obtained by executing the same first use case multiple times should be the same, so as to reduce test error.

[0214] The following will describe in detail the reference use case construction process for various first use cases supported by the embodiments of this disclosure, and the process of obtaining single-case comparison results based on the comparison of the first query result and the second query result.

[0215] In one embodiment, the target verification strategy includes multiple construction rules for constructing multiple reference use cases based on a first use case, and comparison conditions for a first query result corresponding to the first use case and multiple second query results corresponding to the multiple reference use cases; the comparison conditions include a first comparison sub-condition between the multiple second query results and a second comparison sub-condition between the first query result and each second query result.

[0216] The first comparison sub-condition is used to limit the conditions required for pairwise equivalence of the second query results, while the second comparison sub-condition is used to limit the conditions required for mutual equivalence between the first query results and each of the second query results.

[0217] In the specific implementation of this embodiment, firstly, the SQL statement structure of the first use case is adjusted according to the construction rules corresponding to each reference use case, while retaining the query conditions present in the first use case, to obtain the reference use cases. Next, the first use case and each reference use case are applied to the target corpus, enabling searches for multiple identical data to obtain the first query result corresponding to the first use case and the second query results corresponding to each reference use case.

[0218] Please refer to Figure 6 In this embodiment, step 350 specifically includes, but is not limited to, the following steps 610-630:

[0219] Step 610: Based on the first comparison sub-condition, compare each pair of the multiple second query results to obtain the first comparison sub-result;

[0220] Step 620: Based on the second comparison sub-condition, compare the first query result and each second query result to obtain the second comparison sub-result;

[0221] Step 630: Generate a singleton comparison result based on the first comparison result and the second comparison result.

[0222] Steps 610-630 are described in detail below.

[0223] In step 610, for every two second query results, firstly, the first comparison sub-condition corresponding to these two second query results is determined. Then, the two second query results are compared according to the first comparison sub-condition. If the two second query results satisfy the equivalence relation defined by the comparison condition, the first comparison sub-result is determined to be positive; if the two second query results do not satisfy the equivalence relation defined by the comparison condition, the first comparison sub-result is determined to be negative.

[0224] In step 620, for each second query result, firstly, the second comparison sub-condition corresponding to this second query result and the first query result is determined. Then, according to the second comparison sub-condition, the second query result and the first query result are compared. If the second query result and the first query result satisfy the equivalence relation defined by the comparison condition, the second comparison sub-result is determined to be positive; if the second query result and the first query result do not satisfy the equivalence relation defined by the comparison condition, the second comparison sub-result is determined to be negative.

[0225] In step 630, if either the first or second comparison sub-result contains a negative result, the generated singleton comparison result is negative, indicating that the database management system has detected a logical defect. If both the first and second comparison sub-results are positive, the generated singleton comparison result is positive, indicating that the database management system has not detected a logical defect.

[0226] For example, for the first use case src: select c0,c1 from table0 where c0>0; three construction rules are designed, and corresponding reference use cases are generated according to the three construction rules. Here, the first use case refers to finding the result corresponding to c0 being greater than 0 in the target corpus table0.

[0227] Construction rule 1: sum(count(1))group by a union all sum(count(1))group by b; The generated reference test case SQL 1 is "SELECT sum(sub_0.c)FROM(SELECT count(*)AS c FROM table0WHERE c0>0GROUP BY c1 UNION ALL SELECT count(*)AS c FROM table0 WHERE c0>0GROUP BY c0)AS sub_0". Here, reference test case SQL 1 represents, for the target corpus table0, searching for results where c0 is greater than 0 in all datasets corresponding to all values ​​of c1 using a row-wise summation method, and taking the union of the results where c0 is greater than 0 in all datasets corresponding to all values ​​of c0 using a row-wise summation method.

[0228] Construction rule 2: sum(count(1)) group by a,b; the generated reference test case sql2 is "SELECT sum(sub_0.c) FROM(SELECT count(*) AS c FROM table0 WHERE c0>0 GROUP BY c1,c0) AS sub_0", where the reference test case sql2 means that for the target corpus table0, in a row-by-row summation manner, the result of c0 being greater than 0 is found in the dataset corresponding to all values ​​of c1 and c0.

[0229] Construct rule 3: count(1); The generated reference test case sql3 is "SELECT count(*) AS c FROM table0 WHERE c0>0", where the reference test case sql3 means to traverse all the data of the target corpus table0 and directly accumulate the result that c0 is greater than 0.

[0230] Set the first comparison sub-condition to "verify sql1 = 2 * sql2 = 2 sql3", which means it is necessary to verify whether the second query result corresponding to the reference test case sql1 is twice the second query result of the reference test case sql2 and twice the second query result of the reference test case sql3, and verify whether the second query result of the reference test case sql2 is the same as the second query result of the reference test case sql3.

[0231] Therefore, the singleton comparison result is considered positive only when the second query results of the three reference use cases simultaneously satisfy sql1 = 2 * sql2 = 2sql3. Otherwise, it indicates a logical flaw in the database management system.

[0232] The advantage of this embodiment is that it takes into account the design of multiple construction rules to construct multiple reference use cases based on the first use case, and at the same time limits the comparison method between multiple second query results of multiple reference use cases, as well as the comparison method between the query results of each reference use case and the first use case. This can extend the result consistency detection from singular form comparison to plural form comparison, realize the expansion of the test scenario of the database management system, enable more SQL query statements to be tested, and improve test coverage.

[0233] Since SQL queries often consist of multiple statement format fields and multiple query condition fields, performing a query on the target corpus as a whole when the SQL query is composed of multiple query condition parts often leads to low query efficiency. Therefore, this disclosure proposes a data query scheme that segments the target corpus based on SQL queries, which can improve the efficiency of obtaining query results.

[0234] The first use case includes a first segmentation method and a target query condition. The reference use case includes a second segmentation method and a target query condition. The construction rule is to convert the first segmentation method in the first use case into the second segmentation method while keeping the target query condition unchanged.

[0235] The first segmentation method is used to divide the multiple independent conditional expressions contained in the first use case. The target query condition is used to indicate the query restrictions corresponding to each independent conditional expression. The second segmentation method is used to divide the multiple independent conditional expressions contained in the reference use case.

[0236] Please refer to Figure 7 In this embodiment, step 340 specifically includes, but is not limited to, the following steps 710-760:

[0237] Step 710: Apply the first segmentation method to the target corpus to obtain the first segmented corpus segment;

[0238] Step 720: Apply the target query condition to each first segmented corpus segment to obtain the first subquery result corresponding to each first segmented corpus segment;

[0239] Step 730: Integrate the first subquery results corresponding to each first segmented corpus segment into a first query result;

[0240] Step 740: Apply the second segmentation method to the target corpus to obtain the second segmented corpus segments;

[0241] Step 750: Apply the target query condition to each second segment of the corpus to obtain the second subquery result corresponding to each second segment of the corpus;

[0242] Step 760: Integrate the second subquery results corresponding to each second segment of the corpus into a second query result.

[0243] Steps 710-760 are described in detail below.

[0244] The first segment of the corpus refers to a portion of the target corpus. The corpus contained in different first segments does not overlap.

[0245] In step 710, the first segmentation method is applied to the target corpus, and the target corpus is divided into multiple corpus segments according to the division rules indicated by the first segmentation method, to obtain the first segmented corpus segment.

[0246] In step 720, for each first segmented corpus segment, the target query conditions applied to the first segmented corpus segment are applied, and the target query conditions are executed in the database management system so as to query the corpus segment that matches the target query conditions in the first segmented corpus segment and obtain the first subquery result corresponding to the first segmented corpus segment.

[0247] In step 730, the union of the first subquery results corresponding to each first segmented corpus segment is taken to obtain the first query result.

[0248] The specific implementation process of steps 740-760 is similar to that of steps 710-730 above. To save space, it will not be described in detail again.

[0249] For example, regarding the reference use case SQL1 above, "SELECT sum(sub_0.c) FROM (SELECT count(*)ASc FROM table0 WHERE c0>0 GROUP BY c1 UNION ALL SELECT count(*)AS c FROM table0 WHERE c0>0 GROUP BY c0)AS sub_0", reference use case SQL1 searches for results where c0 is greater than 0 based on c1, and searches for results where c0 is greater than 0 based on c0. Therefore, according to the first segmentation method (based on the value of c1 and the value of c0), the target corpus is segmented into corpus groups corresponding to each value of c1 and corpus groups corresponding to each value of c0, resulting in multiple first-segmented corpus segments. Further, for each first-segmented corpus segment, the corresponding query condition "c0 greater than 0" is applied, thus obtaining multiple first-subquery results.

[0250] The advantage of this embodiment is that by segmenting the target corpus based on the SQL query statement, the target corpus is divided into multiple subcorpora, and queries are performed in each subcorpus according to the query conditions in the SQL query statement, which can effectively improve the efficiency of obtaining query results.

[0251] In SQL, it is assumed that by adding `true`, `false`, and `is null` conditions to a SQL query, creating three separate SQL queries each containing `true`, `false`, and `is null`, the results of the original SQL query in the dataset should be equivalent to the union of the results of the queries containing the original queries and the results of the queries containing the original queries. Based on this, this disclosure provides a reference test case construction scheme based on adding condition fields to a first test case, and provides a corresponding comparison scheme for the first and second query results, which can improve test coverage.

[0252] The target verification strategy includes multiple construction rules for constructing multiple reference use cases based on the first use case, as well as the comparison conditions for the first query result corresponding to the first use case and the multiple second query results corresponding to the multiple reference use cases.

[0253] Please refer to Figure 8 In one embodiment, step 350 specifically includes, but is not limited to, the following steps 810-830:

[0254] Step 810: Based on the comparison conditions, merge the multiple second query results corresponding to multiple reference use cases to obtain the merged second query result, and compare the first query result with the merged second query result;

[0255] Step 820: If the first query result is the same as the merged second query result, then the singleton comparison result is determined to be positive.

[0256] Step 830: If the first query result is different from the merged second query result, then the singleton comparison result is determined to be a negative result.

[0257] Steps 810-830 are described in detail below.

[0258] In step 810, based on the comparison conditions, the union of multiple second query results corresponding to multiple reference use cases is performed to merge all the second query results, resulting in a merged second query result. Next, it is determined whether the first query result and the second query result are the same.

[0259] In step 820, if the first query result is the same as the merged second query result, it indicates that the query results of the first use case and the reference use case are equivalent and meet expectations. Then, the singleton comparison result is determined to be a positive result.

[0260] In step 830, if the first query result is the same as the merged second query result, it indicates that the query results of the first use case and the reference use case are not equivalent and do not meet expectations. The database management system may have logical defects, so the singleton comparison result is determined to be a negative result.

[0261] It should be noted that the specific construction rules applicable to the first use case that does not contain aggregate functions and has deduplication field restrictions are somewhat different from the specific construction rules applicable to the first use case that contains the count() aggregate function.

[0262] For example, for the first use case "SELECT c0 FROM table0 WHERE c0>0". When the first use case does not contain aggregate functions and there is a deduplication field restriction, the construction rules are used to construct the reference use cases "SELECT c0 FROM table0 WHERE(c0>0) AND c0>100" (equivalent to inserting a true condition), "SELECT c0 FROM table0 WHERE(c0>0) AND NOT c0>100" (equivalent to inserting an FSLSE condition), and "SELECT c0 FROM table0 WHERE(c0>0) AND c0>100 IS NULL".

[0263] For the first use case "SELECT count(*) AS c FROM table0 WHERE c0>0", where "AS c" refers to an alias for the column in the query result, when the first use case contains the count() function and there is no deduplication field restriction, the reference use cases constructed using the construction rules are: Reference Use Case 1 "SELECT count(*) AS c FROM table0 WHERE(c0>0) AND c1>100"; Reference Use Case 2 "SELECT count(*) AS c FROM table0 WHERE(c0>0) AND NOT c1>100"; Reference Use Case 3 "SELECT count(*) AS c FROM table0 WHERE(c0>0) AND c1>100 IS NULL)".

[0264] In both scenarios described above, the query logic expressed by the first use case is equivalent to the query logic expressed by the union of the three reference use cases. The union of the three reference use cases can be represented by connecting them using "UNION ALL". The union result equivalent to the first use case can be expressed as: "SELECT sum(sub_0.c) FROM (SELECT count(*) AS c FROM table0 WHERE(c0>0) AND c1>100 UNION ALL SELECT count(*) AS c FROM table0 WHERE(c0>0) AND NOT c1>100 UNION ALL SELECT count(*) AS c FROM table0 WHERE(c0>0) AND c1>100 IS NULL) AS sub_0", where "AS sub_0" indicates the clause in the union result.

[0265] The advantage of this embodiment is that it takes into account the method of splitting the first test case based on the true, false, and is null condition fields, and performs query correctness (result consistency) verification on the database management system based on the principle that the first test case and the union logic of the three split SQL query statements are logically equivalent. It can meet the data query of various SQL query statements, thereby improving the test coverage.

[0266] In SQL, it is assumed that when the `projection` field of a SQL query does not contain aggregate functions, the `projection` field in the `SELECT DISTINCT` field and all `projection` fields loaded in the `GROUP BY` field of the SQL query should be equivalent. That is, the logic of the `SELECT DISTINCT` field and the `GROUP BY` field in the SQL query when defining the same query condition is equivalent, and the query results of both should be the same in the same dataset. Based on this, this disclosure provides a scheme for constructing reference test cases for SQL queries whose condition fields do not contain aggregate functions, based on the equivalence relationship between the `SELECT DISTINCT` field and the `GROUP BY` field, which can improve test coverage.

[0267] It should be noted that in this scheme, the target validation strategy for SQL queries whose projection fields do not contain aggregate functions is as follows: when the SQL query contains a SELECT DISTINCT field or a GROUP BY field, the query conditions remain unchanged, and either the SELECT DISTINCT field is replaced with a GROUP BY field, or vice versa, to obtain a reference use case. Simultaneously, the comparison condition is set so that the first and second query results should be identical; otherwise, logical flaws in the database management system are exposed.

[0268] For example, in the SQL query statement: select a form table1, the specific field projection selected in the query result refers to "a".

[0269] The construction rule is used to indicate that when the first use case has a first field and the target position of the first use case does not have a target function field, a reference use case is constructed using the second field corresponding to the first field.

[0270] The comparison condition is used to indicate that, given the logical equivalence of the first use case and the reference use case, the results of the first query and the second query should be the same; otherwise, the database management system has a logical flaw.

[0271] The first field refers to the SELECT DISTINCT field or the GROUP BY field.

[0272] When the first field is a SELECT DISTINCT field, the second field is a GROUP BY field; when the first field is a GROUP BY field, the second field is a SELECT DISTINCT field.

[0273] The target position refers to the position after the select field of the first use case. When the query conditions of the first use case are nested, the target position is used to load the outermost query conditions of the first use case.

[0274] The objective function field is used to indicate the aggregate function in the first use case.

[0275] Please refer to Figure 9 In one embodiment, step 330 specifically includes, but is not limited to, the following steps 910-920:

[0276] Step 910: Based on the construction rules, perform field detection on the first use case;

[0277] Step 920: If it is determined that the first use case has a first field and the target position of the first use case does not have a target function field, then based on the second field, the first segmentation method in the first use case is converted into the second segmentation method, while keeping the target query conditions in the first use case unchanged, to obtain the reference use case.

[0278] Steps 910-920 are described in detail below.

[0279] In step 910, based on the construction rules, the SQL fields included in the select field of the first use case are checked to determine whether the SQL fields included in the select field of the first use case have fields corresponding to aggregate functions, and to determine whether the first use case has a SELECT DISTINCT field or a GROUP BY field.

[0280] In step 920, if it is determined that the target position of the first use case does not contain a target function field (the field corresponding to any aggregate function), and the first use case contains a SELECT DISTINCT field or a GROUP BY field, then the existing SELECT DISTINCT field or GROUP BY field is used as the first field. Next, using the second field corresponding to the first field, the first splitting method in the first use case is converted into a second splitting method to change the query statement structure of the first use case, while keeping the target query condition in the first use case unchanged, thus obtaining a reference use case.

[0281] For example, for the first use case "SELECT DISTINCT c0,c1 FROM table0 WHERE c0>0 AND c1>0", replacing the SELECT DISTINCT field with the GROUP BY field in the first use case will result in the reference use case "SELECT c0,c1 FROM table0 WHERE c0>0 AND c1>0 GROUP BY c0,c1".

[0282] In this embodiment, step 350 includes:

[0283] If the first query result is the same as the second query result, then the singleton comparison result is determined to be positive.

[0284] If the results of the first query and the second query are different, then the singleton comparison result is determined to be negative.

[0285] Specifically, the first use case and the reference use case have different statement formats, but the query logic they represent is completely consistent. Therefore, their query results in the target corpus should be consistent. Based on this, the comparison condition is set to compare the first query result and the second query result. When the first query result and the second query result are consistent, it indicates that there is no logical defect in the database management system. When the first query result and the second query result are inconsistent, it indicates that there may be a logical defect in the database management system.

[0286] Therefore, in step 350, based on the comparison conditions, it is determined whether the first query result and the second query result are consistent. If the first query result and the second query result are the same, the singleton comparison result is determined to be a positive result. If the first query result and the second query result are not the same, the singleton comparison result is determined to be a negative result.

[0287] It should be noted that when the statement format of the first use case includes a nested structure, then for the above target validation strategy, it is necessary to ensure that the outermost condition field of the first use case does not contain aggregate functions in order to use the consistency validation method based on the mutual substitution of the SELECT DISTINCT field and the GROUP BY field. When the first query result is the same as the second query result, it meets the expectations.

[0288] The advantage of this embodiment is that, for SQL query statements where the projection condition field does not contain aggregate functions, it utilizes the equivalence relationship between the SELECT DISTINCT field and the GROUP BY field in the SQL language to design a method that substitutes the SELECT DISTINCT field and the GROUP BY field in the first test case while keeping the query condition fields unchanged. Then, it constructs a reference test case corresponding to the first test case and exposes potential logical defects in the database management system based on the consistency verification of the query results of the two, which can effectively improve test coverage.

[0289] Because database management system (DBMS) query engine filtering modules often contain optimization logic for certain first use cases (SQL query statements) to optimize the statement structure of the first use case before executing the query operation, this optimization logic can sometimes reduce the accuracy of the database management system's queries, hindering the acquisition of more accurate results for certain SQL queries. Therefore, this disclosure considers the impact of optimization logic on query accuracy. When testing the DBMS, a target verification strategy is provided that is not limited by aggregate functions or deduplication fields when optimization logic exists in the query engine filtering module.

[0290] Since the query conditions have not changed, the query results should be unaffected. When constructing reference test cases using the construction rules, the order of conditions in the first test case can be shuffled to eliminate the impact of the optimization logic.

[0291] Please refer to Figure 10 In one embodiment, step 330 specifically includes, but is not limited to, the following steps 1010-1020:

[0292] Step 1010: Detect the query engine filtering module of the database management system;

[0293] Step 1020: If it is determined that there is optimization logic for the first use case in the query engine filtering module, then use the construction rules to adjust the position of at least one condition field of the first use case to obtain a reference use case.

[0294] Steps 1010-1020 are described in detail below.

[0295] In step 1010, firstly, with authorization, the logical code segment of the database management system's query engine filtering module is invoked. Next, the logical code segment is examined to determine if it contains optimization logic for the SQL statement structure applied to the first use case.

[0296] In step 1020, if it is determined that there is optimization logic for the first use case in the query engine filtering module, then the condition fields existing in the first use case are first determined using the construction rules. Next, the positions of at least one condition field in the first use case are randomly changed to shuffle the order of the condition fields, thus obtaining a reference use case.

[0297] For example, for a first use case:

[0298] The query is: “SELECT c0,c1 FROM table0 WHERE c0>0 AND c1>0 ORDER BY c0,c1”. The first use case indicates that the query searches for results in the target corpus table0 where c0>0 and c1>0 are equal, and when outputting the query results, the results are sorted by the value of c0 in ascending order, and if the values ​​of c0 are the same, they are sorted by the value of c1 in ascending order.

[0299] When the query engine's filtering module has optimization logic for this SQL statement, it first determines that the WHERE clause in the first test case contains the condition fields "c0>0" and "c1>0". Then, it swaps the positions of these two condition fields to change their order, resulting in the reference test case "SELECT c0,c1FROM table0 WHERE c1>0AND c0>0ORDERBY c0,c1". At this point, the first test case and the reference test case have different statement formats, but the query logic they represent is completely consistent.

[0300] In this embodiment, step 350 includes:

[0301] If the first query result is the same as the second query result, then the singleton comparison result is determined to be positive.

[0302] If the results of the first query and the second query are different, then the singleton comparison result is determined to be negative.

[0303] Specifically, the first use case and the reference use case have different statement formats, but the query logic they represent is completely consistent. Therefore, their query results in the target corpus should be consistent. Based on this, the comparison condition is set to compare the first query result and the second query result. When the first query result and the second query result are consistent, it indicates that there is no logical defect in the database management system. When the first query result and the second query result are inconsistent, it indicates that there may be a logical defect in the database management system.

[0304] Therefore, in step 350, based on the comparison conditions, it is determined whether the first query result and the second query result are consistent. If the first query result and the second query result are the same, the singleton comparison result is determined to be a positive result. If the first query result and the second query result are not the same, the singleton comparison result is determined to be a negative result.

[0305] The advantage of this embodiment is that, when the query engine filtering module of a database management system contains optimization logic for SQL query statements, it considers a consistency verification strategy (target verification strategy) that is not restricted by aggregate functions and deduplication fields. First, the condition fields in the first test case are shuffled to obtain a reference test case with a different format but completely consistent query conditions. Then, the consistency of the query results of the first test case and the reference test case in the target corpus is verified to obtain a single-case comparison result. This method can realize query correctness testing when there is optimization logic for SQL query statements in the query engine filtering module of the database management system, without being restricted by aggregate functions and deduplication fields, thus improving test coverage.

[0306] It should be noted that steps 1010-1020 above are query correctness tests for SQL query statements in the query engine filtering module of the database management system. This reduces the statement restrictions on the first test case and covers more SQL verification scenarios.

[0307] Compared to the above methods, the embodiments of this disclosure also take into account the impact of not eliminating aggregate function restrictions and deduplication field restrictions in order to simplify the verification process. Based on this, this disclosure provides a test scheme when there are aggregate function restrictions and optimization logic for SQL query statements exists in the query engine filtering module of the database management system.

[0308] It should be noted that, in one embodiment, the target verification strategy with aggregation function constraints is:

[0309] Construction rule: When the query engine filtering module of the database management system contains optimization logic for SQL query statements, the condition field (condition expression) in the WHERE clause of the first test case is moved to the SELECT field to obtain the reference test case.

[0310] Comparison conditions: When the first use case is applied to the dataset, the first query result is the list of results that match the first use case, consisting of multiple retrieved data items. Since the reference use case, when applied to the same dataset as the first use case, will mark results that match the reference use case as true and those that do not as false, the second query result is the number of results that are True. Therefore, the number of True results in the second query result should be consistent with the number of data items in the first query result.

[0311] Please refer to Figure 11 In this embodiment, step 330 specifically includes, but is not limited to, the following steps 1110-1120:

[0312] Step 1110: Based on the construction rules, perform field detection on the first use case;

[0313] Step 1120: If it is determined that the first use case does not include an aggregate function field, then the conditional expression in the first use case is fronted to obtain the reference use case.

[0314] Steps 1110-1120 are described in detail below.

[0315] The specific implementation process of step 1110 is similar to that of step 910 above. The difference is that the field detection in step 910 is to detect whether the first field exists in the first use case, while the field detection in step 1110 is to detect whether the condition field of the first use case includes an aggregate function field. The field detection objectives of the two are different. To save space, they will not be elaborated further.

[0316] In step 1120, if it is determined that the first use case does not include an aggregate function field, then the conditional expression used to indicate the query conditions is first determined in the first use case. Next, the conditional expression is prepended to the select field of the first use case to obtain the reference use case.

[0317] The first query result is used to indicate the number of data items retrieved, and the first query result is used to indicate the total number of data items retrieved.

[0318] In this embodiment, step 350 includes:

[0319] Based on the comparison conditions, the data items in the first query result are statistically analyzed to obtain the data item statistical results;

[0320] If the statistical results of the data items are the same as the results of the second query, then the singleton comparison result is determined to be positive.

[0321] Specifically, firstly, based on the comparison conditions, the data items in the first query result are statistically analyzed to determine the total number of data items included in the first query result, resulting in a statistical result of the data items. Next, the statistical result of the data items is numerically compared with the second query result. If the statistical result of the data items is the same as the second query result, the single-case comparison result is determined to be a positive result. If the statistical result of the data items is different from the second query result, the single-case comparison result is determined to be a negative result.

[0322] For example, considering a first use case "SELECT c0 FROM table0 WHERE c0>0", the first query result in the target corpus table0 is c0=1 and c0=2. Moving the condition field of the WHERE clause of this first use case to the SELECT field, we get the reference use case "SELECT c0>0 FROM table0". Applying the reference use case to the target corpus table0, we obtain the matching status of each corpus in table0 with the reference use case. When a corpus matches the query condition of the reference use case, it is marked as true; when it does not, it is marked as false. Based on this, the total number of true rows in the second query result is 2. Since the first query result is c0=1 and c0=2, with two data items, and the data item count is 2, which is the same as the second query result rows=2, the singleton comparison result is determined to be positive.

[0323] It should be noted that when the query conditions in the first use case consist of multiple expressions, all expressions are treated as a single condition expression, and the entire condition expression is placed at the beginning. For example, for a first use case "SELECT c0 FROM table0 WHERE c0>0 and c1>1", the corresponding reference use case is "SELECT c0>0 and c1>1 FROM table0".

[0324] The advantage of this embodiment is that, when the query engine filtering module of the database management system contains optimization logic for SQL query statements, the impact of aggregate function restrictions is not eliminated. This ensures that even when the first test case is subject to aggregate function restrictions and deduplication field restrictions, there are corresponding construction rules to construct a reference test case based on the first test case. Specifically, all conditional expressions of the first test case are directly prepended to the select field to obtain the reference test case. Furthermore, when the first and second query results have different meanings, comparison conditions are set to compare the statistical results of the data items in the first query result with the second query result to determine whether the first and second query results satisfy an equivalence relationship. This approach can cover query correctness verification scenarios where the first test case is affected by both aggregate function restrictions and the optimization logic of the query engine filtering module, thus improving test coverage.

[0325] In SQL, if a single record is selected from the dataset as a pivoted query and added to the WHERE clause of the SQL query as a condition field, the query result will necessarily contain the pivoted query result. Therefore, this disclosure proposes a scheme to verify the query correctness of a database management system based on determining whether the query result contains a pivoted query result.

[0326] It should be noted that in this scheme, the target validation strategy for SQL queries whose projection condition field does not contain aggregate functions is as follows: First, randomly select one data point from the target corpus as a pivoted query result. Add this pivoted query result to the WHERE clause of the first test case as a condition field to obtain a reference test case. Simultaneously, check the second query result for the existence of a pivoted query result. If it does not exist, a logical flaw in the database management system is exposed.

[0327] Please refer to Figure 12 In one embodiment, step 330 specifically includes, but is not limited to, the following steps 1210-1220:

[0328] Step 1210: Based on the construction rules, extract the target data items from the target corpus;

[0329] Step 1220: Embed the target data item into the first use case to obtain the reference use case.

[0330] Steps 1210-1220 are described in detail below.

[0331] In step 1210, based on the construction rules, one or more data items are randomly selected from the target corpus as target data items.

[0332] In step 1220, the selected target database is embedded into the first use case according to the syntax rules of the SQL statement to obtain the reference use case.

[0333] In this embodiment, step 350 includes, but is not limited to:

[0334] Find the target data item in the second query results;

[0335] If it is determined that the target data item is present in the second query result, then the singleton comparison result is affirmative.

[0336] If it is determined that the target data item is not found in the second query result, then the singleton comparison result is negative.

[0337] The advantage of this embodiment is that, considering that if a certain result exists in the query statement, the query statement will inevitably find this result, it is possible to effectively expand multiple reference use cases corresponding to the first use case based on the corpus data contained in the target corpus, and obtain the verification result (single case comparison result) based on the detection of the second query result, thereby improving the test coverage.

[0338] Detailed description of step 360

[0339] In step 360, the first use case is mutated, and the steps of constructing a reference use case based on the first use case and using the construction rules are returned until the number of mutations of the first use case reaches a predetermined number.

[0340] In this specific implementation, the mutation methods for the first use case include adding, deleting, and modifying operators, conditional constraint numbers, and conditional expressions, so as to generate more SQL query statements based on the mutation of the first use case. For each mutated first use case, the steps of constructing a reference use case based on the first use case and using the construction rules are returned, and steps 330-350 are repeated until the number of mutations of the first use case reaches a predetermined number, at which point further mutation of the first use case stops.

[0341] In this embodiment, fuzz testing principles are incorporated, providing various SQL mutation strategies for the first test case. This allows for convenient and effective mutation of existing test cases (the first test case), generating new first test cases while ensuring the syntactic correctness of the mutated SQL statements. This achieves automated expansion of the test case set, improving test coverage and efficiency. Furthermore, the entire testing method of this disclosure is executed automatically, significantly reducing labor costs.

[0342] Please refer to Figure 13 In one embodiment, when the condition field in the first use case contains a number, the specific process of mutating the first use case includes, but is not limited to, the following steps 1310-1320:

[0343] Step 1310: For the first use case, determine the target number within the first use case;

[0344] Step 1320: In the first use case, the target number is replaced with a number to obtain the mutated first use case.

[0345] Steps 1310-1320 are described in detail below.

[0346] In step 1310, for the first use case, the target number is determined in the first use case.

[0347] The target number refers to the number in the condition field of the first use case.

[0348] For example, in the first use case "select * from table where a > 1", the target number refers to the number "1" in the condition field "where a > 1".

[0349] In the specific implementation of this embodiment, firstly, for the first use case, the condition field is located in the first use case according to the statement form of the first use case. Next, numbers are extracted from the condition field, and a target number is determined from multiple numbers. The target number can be one or more numbers randomly selected from the multiple numbers in the condition field.

[0350] In step 1320, in the first use case, the target number is replaced with a number to obtain the mutated first use case.

[0351] In a specific implementation of this embodiment, the methods for replacing the target number include, but are not limited to:

[0352] Method 1: Increase or decrease the target number according to a predetermined rule, and replace the target number with the increased or decreased target number.

[0353] In the specific implementation of this embodiment, the target number is increased or decreased according to the increase or decrease range defined by the predetermined rules, and the target number in the condition field of the first use case is replaced with the increased or decreased target number.

[0354] The predetermined rules include probability values ​​corresponding to different increases or decreases, which can be used to determine the increase or decrease of the target number.

[0355] For example, the predetermined rules include different increments and decrements and their corresponding probability values: (+1, 30%), (-1, 20%), (+2, 15%), (-2, 20%), (+3, 5%), (-3, 10%). Based on this, a mutation method of adding 2 to the target number is randomly selected according to the probability, and the target number is replaced with the result of adding 2 to the target number.

[0356] Method 2: Take the first value from the preset first set and replace the target number with the first value.

[0357] The first set is used to store multiple special values ​​that are preset to replace the target numbers in the first use case. For example, the special values ​​included in the first set can be "positive numbers, negative numbers, 0, int32_max, int32_min, int32_max-1, int32_min+1", etc.

[0358] In this specific implementation, firstly, a value is randomly selected from a preset first set as the first value. Then, the target number is replaced with the first value.

[0359] For example, for the first use case "select * from table where a > 1", if the number in the condition field of the first use case is reduced by 1, the resulting first use case becomes "select * from table where a > 0", changing the query content of the first use case from "find the result corresponding to a greater than 1 from the target corpus table" to "find the result corresponding to s greater than 0 from the target corpus table".

[0360] The advantage of this embodiment is that when there are numbers in the condition field of the first test case, the method of replacing the numbers is taken into consideration. The variation of the first test case is achieved by modifying the value range required by the condition field. Multiple new first test cases for testing can be easily and quickly extended based on the first test case, which can effectively improve the efficiency and scope of test case extension.

[0361] Please refer to Figure 14 In one embodiment, when an operator exists in the condition field of the first use case, the specific process of mutating the first use case includes, but is not limited to, the following steps 1410-1430:

[0362] Step 1410: For the first use case, determine the target operator in the first use case;

[0363] Step 1420: Take a reference operator that is different from the target operator from the preset second set;

[0364] Step 1430: In the first use case, the target number is replaced with a number to obtain the mutated first use case.

[0365] Steps 1410-1430 are described in detail below.

[0366] In step 1410, for the first use case, the target operator is determined in the first use case.

[0367] The target operator is used to indicate the operator to be modified selected from the operators available in the condition field of the first use case.

[0368] In the specific implementation of this embodiment, firstly, for the first use case, the condition field is located in the first use case according to the statement form of the first use case. Next, operators are extracted from the condition field, and the target operator is determined from multiple operators. The target operator can be one or more operators randomly selected from the multiple operators in the condition field.

[0369] In step 1420, a reference operator that is different from the target operator is selected from the preset second set.

[0370] The second set is used to store multiple preset reference operators for selection. For example, the reference operators in the second set include "<, ≤, >, ≥, =, !=", etc.

[0371] Reference operators are used to perform logical operations.

[0372] In the specific implementation of this embodiment, firstly, in a preset second set, each reference operator is compared with the target operator, and one of the reference operators that is different from the target operator is randomly selected as the reference operator to replace the target operator. Next, in the first use case, the target operator is replaced using the reference operator used to replace the target operator, resulting in the mutated first use case.

[0373] In step 1430, in the first use case, the target operator is replaced with the reference operator to obtain the mutated first use case.

[0374] In a specific implementation of this embodiment, in the first use case, the target operator is replaced with a reference operator, and the logical operation rules indicated by the target operator are changed to obtain the mutated first use case.

[0375] For example, for the first use case "select * from table where a > 1", if the operator ">" in the condition field of the first use case is changed to the operator "<", the resulting first use case will be "select * from table where a < 1". This changes the query content of the first use case from "find the result corresponding to a greater than 1 from the target corpus table" to "find the result corresponding to a less than 1 from the target corpus table".

[0376] The advantage of this embodiment is that when there is an operator in the condition field of the first test case, it takes into account the way to replace the operator and modify the logical operation rules required by the condition field to realize the variation of the first test case. It can easily and quickly extend multiple new first test cases for testing based on the first test case, which can effectively improve the efficiency of test case extension.

[0377] Please refer to Figure 15 In one embodiment, the specific process of modifying the first use case includes, but is not limited to, the following steps 1510-1530:

[0378] Step 1510: Perform a non-empty check on the first test case;

[0379] Step 1520: If it is determined that the first use case contains a non-empty logical field, then replace the non-empty logical field of the first use case with an empty logical field to obtain the mutated first use case;

[0380] Step 1530: If it is determined that the first use case contains a non-empty logical field, then replace the non-empty logical field of the first use case with an empty logical field to obtain the mutated first use case.

[0381] Steps 1510-1530 are described in detail below.

[0382] In step 1510, a non-empty check is performed on the first use case.

[0383] In the specific implementation of this embodiment, the condition field of the first use case is subjected to non-empty validation to verify whether the data in the condition field is empty, so as to realize random modification of the empty and non-empty logic in the condition field of the first use case.

[0384] In step 1520, if it is determined that the first use case contains a non-empty logical field, then the non-empty logical field of the first use case is replaced with an empty logical field to obtain the mutated first use case.

[0385] A NOT NULL logical field refers to "not null" in SQL.

[0386] An empty logical field refers to "null" in SQL.

[0387] In the specific implementation of this embodiment, if it is determined that the first use case contains a non-empty logical field according to the non-empty test, then in the first use case, the "not" in the non-empty logical field is deleted, so as to replace the non-empty logical field of the first use case with an empty logical field, and obtain the mutated first use case.

[0388] In step 1530, if it is determined that the first use case contains a non-empty logical field, then the non-empty logical field of the first use case is replaced with an empty logical field to obtain the mutated first use case.

[0389] In the specific implementation of this embodiment, if it is determined that the first use case contains an empty logical field according to the non-empty test, then in the first use case, a "not" field is added at the position of the byte before the empty logical field, so as to replace the empty logical field of the first use case with a non-empty logical field and obtain the mutated first use case.

[0390] For example, for the first use case "select*from table where a is null", the first use case contains the empty logical field "null". Based on this, the empty logical field "null" is replaced with the non-empty logical field "not null", resulting in the modified first use case "select*from table where a is not null". This changes the first use case from "finding the result corresponding to a being empty in the target corpus table" to "finding the result corresponding to a not being empty in the target corpus table".

[0391] The advantage of this embodiment is that when there are empty or non-empty logical fields in the condition fields of the first use case, the empty or non-empty logical fields can be directly added or deleted to change the query conditions required by the condition fields to the opposite conditions, thereby realizing the variation of the first use case. This method can easily and quickly extend the first use case to create a new use case with the opposite meaning of the logical fields of the first use case, which can effectively improve the efficiency of use case extension.

[0392] Please refer to Figure 16 In one embodiment, when the condition field for the first use case includes at least two expressions, the specific process of mutating the first use case includes, but is not limited to, the following steps 1610-1620:

[0393] Step 1610: For the first use case, determine the target expression from multiple candidate expressions of the first use case;

[0394] Step 1620: In the first use case, delete the target expression to obtain the mutated first use case.

[0395] Steps 1610-1620 are described in detail below.

[0396] In step 1610, for the first use case, the target expression is determined from multiple candidate expressions of the first use case.

[0397] One of the candidate expressions is used to indicate a query condition in the first use case.

[0398] For example, in the first use case "select * from table where expr1 and expr2 and expr3", "expr1", "expr2", and "expr3" are each a candidate expression.

[0399] In the specific implementation of this embodiment, firstly, for the first use case, the condition field is located in the first use case according to its statement form. Next, all expressions are extracted from the condition field to obtain candidate expressions. Further, one or more candidate expressions are randomly selected as the target expression.

[0400] In step 1620, the target expression is deleted in the first use case to obtain the mutated first use case.

[0401] In a specific implementation of this embodiment, in the first use case, the target expression and the connector between the target expression and its preceding candidate expression are directly deleted to obtain the mutated first use case. The connector between the target expression and its preceding candidate expression includes, but is not limited to, "and", "or", etc.

[0402] For example, for the first use case "select * from table where expr1 and expr2 and expr3", "expr2" is identified as the target expression. Based on this, the target expression "expr2" is deleted from the first use case, resulting in the mutated first use case "select * from table where expr1 and expr3". This changes the first use case from "finding the result in the target corpus table that simultaneously satisfies the three expressions expr1, expr2, and expr3" to "finding the result in the target corpus table that simultaneously satisfies the three expressions expr1 and expr3".

[0403] The advantage of this embodiment is that when there are multiple expressions in the condition field of the first use case, one or more expressions can be randomly deleted to obtain a new first use case, which can improve the expansion efficiency of the first use case.

[0404] Please refer to Figure 17 In one embodiment, the specific process of modifying the first use case includes, but is not limited to, the following steps 1710-1720:

[0405] Step 1710: Construct the new expression;

[0406] Step 1720: In the first use case, the target number is replaced with a number to obtain the mutated first use case.

[0407] Steps 1710-1720 are described in detail below.

[0408] In step 1710, the new expression is constructed.

[0409] The newly added expression refers to the expression written by the test terminal based on the SQL language.

[0410] In this specific implementation, when the test terminal mutates the test case by adding an expression to the first test case, the test terminal provides the written new expression to the server, along with information such as the location where the new expression should be added. Based on this, the server can obtain the new expression constructed by the test terminal.

[0411] In step 1720, the new expression is embedded into the first use case to obtain the mutated first use case.

[0412] In the specific implementation of this embodiment, based on the received information indicating the position where the new expression should be added, the new expression is embedded into the first use case to obtain the mutated first use case.

[0413] In order to ensure that the modified first use case can run normally after the addition of an expression, this disclosure provides a scheme for checking the syntax of the expression, which enables the statement structure of the first use case with the added expression to be correct.

[0414] In one embodiment, the new expression can be embedded into the first use case in ways including but not limited to the following two:

[0415] Method 1: Perform expression verification on the new expression, and after the expression verification is passed, embed the new expression into the first use case.

[0416] In the specific implementation of this embodiment, firstly, the syntax structure of the newly added expression is checked. If the syntax structure of the newly added expression conforms to the basic syntax, the expression check is deemed successful. Further, after the expression check is deemed successful, the newly added expression is embedded into the first use case according to the expression insertion rules indicated by the test terminal, resulting in the mutated first use case.

[0417] Method 2: Embed the new expression into the first use case to obtain the intermediate query statement. Perform syntax verification on the intermediate query statement, and after the syntax verification passes, determine the intermediate query statement as the mutated first use case.

[0418] In the specific implementation of this embodiment, firstly, according to the expression insertion rules indicated by the test terminal, the new expression is embedded into the first use case to obtain the intermediate query statement. Next, the intermediate query statement undergoes syntax verification; if the syntax structure of the intermediate query statement conforms to basic syntax, the syntax verification is deemed successful. Finally, after passing the syntax verification, the intermediate query statement is identified as the modified first use case.

[0419] For example, for the first use case "select * from table where a > 1", the new expression "a < 0" is embedded into the first use case using the "or" operator. Based on this, the mutated first use case becomes "select * from table where a > 1 or a < 0", changing the query content of the first use case from "finding the results corresponding to a > 1 in the target corpus table" to "finding the results corresponding to a > 1 and the results corresponding to a < 0 in the target corpus table".

[0420] The advantage of this embodiment is that it takes into account the method of adding expressions to expand the first use case. At the same time, it also takes into account the statement correctness of the modified first use case, and introduces two methods: expression verification before adding the new expression to the first use case and full statement verification after adding the new expression to the first use case. This can increase the number of use case expansions while ensuring the syntactic correctness of the expanded first use case.

[0421] Please refer to Figure 18 In one embodiment, when the condition field in the first use case contains a query result sorting restriction field, the specific process of mutating the first use case includes, but is not limited to, the following steps 1810-1830:

[0422] Step 1810: In the first use case, determine the target logical field used to indicate the sorting method of the first query results;

[0423] Step 1820: Take out a sorting logical field that is different from the target logical field from the preset third set;

[0424] Step 1830: In the first use case, replace the target logical field with the sorting logical field to obtain the mutated first use case.

[0425] Steps 1810-1830 are described in detail below.

[0426] In step 1810, a target logical field for indicating the sorting method of the first query result is determined in the first use case.

[0427] The sorting method is used to limit the order in which the first query results are arranged.

[0428] The target logical field refers to the code segment in the first use case that represents the sorting method.

[0429] In the specific implementation of this embodiment, firstly, for the first use case, the condition fields are located in the first use case according to the statement form of the first use case. Then, the target logical field used to indicate the sorting method of the first query results is extracted from the condition fields.

[0430] In step 1820, a sorting logical field that is different from the target logical field is taken from the preset third set.

[0431] The third set is used to store multiple sorting logic fields that represent the arrangement. For example, the sorting logic fields of the third set include, but are not limited to: asc, asc_nulls_first, asc_nulls_last, desc_nulls_first, desc_nulls_last, and desc.

[0432] The sorting logic field is used to indicate how the query results are arranged.

[0433] In the specific implementation of this embodiment, firstly, in the preset third set, each sorting logical field is compared with the target logical field, and one of the sorting logical fields that is different from the target logical field is randomly selected as the sorting logical field to replace the target logical field.

[0434] In step 1830, in the first use case, the target logical field is replaced with the sorting logical field to obtain the mutated first use case.

[0435] In a specific implementation of this embodiment, in the first use case, the target logical field is replaced by a sorting logical field used to replace the target logical field, resulting in the mutated first use case.

[0436] For example, for the first use case "select * from table where a > 1 order by b desc", replace the target logical field "order by b desc" with the sorting logical field "order by b asc nulls first". Based on this, the modified first use case becomes "select * from table where a > 1 order by b asc nulls first", changing the query content of the first use case from "find the results corresponding to a > 1 in the target corpus table and sort the results according to b desc" to "find the results corresponding to a > 1 in the target corpus table and sort the results according to b asc nulls first".

[0437] The advantage of this embodiment is that it takes into account the change of the sorting method of the query results defined in the first use case, replaces the original logical fields with the logical fields corresponding to other sorting methods, and randomly switches the order by direction in the first use case to expand the first use case. This method greatly improves the sorting flexibility of the query results without changing the query conditions of the first use case, and also improves the flexibility and diversity of use case expansion.

[0438] Detailed description of determining the predetermined number of times and updating the predetermined number of times in embodiments of this disclosure.

[0439] Different first use cases often contribute differently to the logical defect testing (correctness testing) of a database management system. For example, a first use case with a positive singleton comparison result is often not very effective in exposing logical defects in the database management system, while a first use case with a negative singleton comparison result is more likely to produce a negative singleton comparison result again after modification, and this type of first use case is of great value for logical defect testing of the database management system. In addition, the degree to which different forms of first use cases can expose logical defects will also vary. If the predetermined number of times for all first use cases is set to the same default value, it will often lead to poor test results. Based on this, embodiments of this disclosure provide a scheme for determining the predetermined number of times for each first use case based on the expression form of the first use case, which can improve the accuracy of the predetermined number of times setting, and is conducive to improving test coverage and test results.

[0440] Please refer to Figure 19 In one embodiment, the specific process for determining the predetermined number of times includes, but is not limited to, the following steps 1910-1920:

[0441] Step 1910: Determine the query statement structure of the first use case and determine the total amount of data in the database management system;

[0442] Step 1920: Determine the predetermined number of times based on the query statement structure and the total amount of data.

[0443] Steps 1910-1920 are described in detail below.

[0444] In step 1910, the query statement structure of the first use case is determined, and the total amount of data in the database management system is determined.

[0445] In the specific implementation of this embodiment, the process of determining the query statement structure of the first use case in step 1910 is similar to that in step 410 above. To save space, it will not be described again.

[0446] Furthermore, when determining the total amount of data in the database management system, statistics are performed on the data contained in the target corpus to obtain the total amount of data in the database management system.

[0447] In step 1920, the predetermined number of times is determined based on the query statement structure and the total amount of data.

[0448] In the specific implementation of this embodiment, the process of determining the predetermined number of times based on the query statement structure and the total amount of data includes, but is not limited to, the following steps:

[0449] Determine the first score based on the query statement structure;

[0450] Determine the second score based on the total amount of data;

[0451] The predetermined number of times is determined based on the first score and the second score.

[0452] In this specific implementation, firstly, in a table of correspondences between candidate statement structures and scores, the score corresponding to a candidate statement structure that matches the query statement structure is selected as the first score. Next, the total data volume is input into a preset function, and the output of the preset function is used as the second score. The preset function is an increasing function with the total data volume as the independent variable and the second score as the dependent variable. Finally, the first and second scores are weighted and summed according to predetermined weights to obtain the total score. Based on the frequency corresponding to the total score in different intervals, the frequency corresponding to the interval containing the total score is used as the predetermined frequency of the first use case.

[0453] like Figure 21AThe diagram illustrates the correspondence between the weighted sum of the first and second scores and the predetermined number of times. Specifically, when the weighted sum of the first and second scores is between 0 and 20, the predetermined number of times for the first use case is 10; when the weighted sum of the first and second scores is between 21 and 40, the predetermined number of times for the first use case is 20; when the weighted sum of the first and second scores is between 41 and 60, the predetermined number of times for the first use case is 30; when the weighted sum of the first and second scores is between 61 and 80, the predetermined number of times for the first use case is 40; and when the weighted sum of the first and second scores is between 81 and 100, the predetermined number of times for the first use case is 50.

[0454] The advantage of this embodiment is that, based on the expression form (query statement structure) of the first use case and the total amount of data in the database management system, the predetermined number of times each first use case is determined can improve the accuracy of the predetermined number of times setting, which is conducive to improving test coverage and test results.

[0455] Since first test cases with a negative singleton comparison result are more likely to produce another negative singleton comparison result when mutated, these types of first test cases are of great value for testing logical defects in database management systems. Setting a fixed number of times each first test case is not conducive to maximizing the effectiveness of these valuable first test cases, affecting test coverage and the breadth of test cases. Therefore, this disclosure provides a scheme for updating the predetermined number of times based on the singleton comparison result, allowing the predetermined number of times to be dynamically adjusted to improve test coverage and test effectiveness.

[0456] Please refer to Figure 20 In one embodiment, the specific process for updating the predetermined number of times includes, but is not limited to, the following steps 2010-2020:

[0457] Step 2010: When the singleton comparison result is negative, determine the mutation count update strategy for the first use case;

[0458] Step 2020: Update the predetermined number of times based on the mutation number update strategy to increase the predetermined number of times, and ensure that the updated predetermined number of times is not greater than the first value.

[0459] The following is a detailed description of steps 2010-2020.

[0460] In step 2010, when the singleton comparison result is negative, the mutation count update strategy for the first use case is determined.

[0461] The mutation count update strategy is used to indicate how the mutation count is increased.

[0462] In the specific implementation of this embodiment, when the singleton comparison result is negative, firstly, based on the statement query structure of the first use case, the mutation count update strategy corresponding to this type of statement query structure is searched in a predetermined correspondence table. The predetermined correspondence table contains multiple mutation count update strategies and the mutation count update strategies applicable to each statement query structure.

[0463] In step 2020, the predetermined number of mutations is updated based on the mutation number update strategy to increase the predetermined number of mutations and ensure that the updated predetermined number of mutations is not greater than the first value.

[0464] The first value is used to limit the maximum number of predetermined attempts.

[0465] In this specific implementation, the predetermined number is incremented according to the found mutation count update strategy. If the predetermined number is greater than a first value after a certain update, the first value is used as the updated predetermined number.

[0466] For example, if the number of predetermined mutations is 5 and the first value is 8, when updating the number of predetermined mutations by a factor of 2, if the result of a singleton comparison in the first use case is negative, then the number of predetermined mutations needs to be updated to 10. However, 10 is greater than 8, so the number of predetermined mutations for the first use case is set to 8 to ensure that the first use case can only undergo a maximum of 8 mutations.

[0467] like Figure 21B As shown, the first numerical limit restricts the maximum number of predetermined times to no more than k, where k is a positive integer. There are n mutation number update strategies: Mutation number update strategy 1 indicates a doubling update of the predetermined number; Mutation number update strategy 2 indicates an exponential update of the predetermined number; ...; Mutation number update strategy n indicates an exponential update followed by a doubling update of the predetermined number. Based on this, for the first test case with an aggregate function and a predetermined number of 5, when the singleton comparison result of the first query result of the first test case and the second query result of the reference test case constructed based on the first test case is invalid, mutation number update strategy 1 is selected to update the predetermined number, multiplying it by 2, making the predetermined number increase from 5 to 10, so as to mutate more test cases based on the first test case.

[0468] The advantage of this embodiment is that by selecting different mutation number update strategies for different first use cases, the degree of mutation number update for each first use case can be flexibly controlled. At the same time, by setting a maximum upper limit for the predetermined number of mutations, the test duration can be effectively controlled while making the predetermined number of mutations more suitable for each first use case. This avoids some first use cases from getting stuck in an endless mutation loop, keeps the expansion of the first use cases within a reasonable range, and thus achieves the goal of improving test coverage and test effectiveness.

[0469] Detailed implementation diagrams of the testing method for the database management system according to embodiments of this disclosure.

[0470] The following reference Figure 22 The following provides a detailed and exemplary description of the implementation details of the testing method for the database management system according to embodiments of the present disclosure.

[0471] like Figure 22 As shown, when executing the test method for the database management system, firstly, the first test case SQLA is obtained, and the target verification strategy corresponding to the first test case is determined. The specific implementation process is similar to steps 310-320 above. Next, based on the construction rules of the target verification strategy, an equivalent reference test case SQLB is generated using the first test case SQLA. The first test case SQLA is then executed in the database management system to apply the first test case to the target corpus, obtaining the first query result a. The reference test case SQLB is then executed in the database management system to apply the reference test case to the target corpus, obtaining the second query result b. The comparison conditions determine whether the first query result a and the second query result b are equivalent. The specific implementation process is similar to the various verification scenarios in steps 330-350 above. Furthermore, when the first query result and the second query result are not equivalent, the singleton comparison result is determined to be negative, exposing a logical flaw in the database management system. At this time, an update is performed a predetermined number of times. The specific implementation process is similar to steps 2010-2020 above. After updating the system a predetermined number of times, or after determining that the first query result and the second query result are equivalent, it is determined whether the mutation count of the first use case has reached the predetermined number. If it has not reached the predetermined number, the first use case SQLA is mutated to obtain the mutated first use case SQL A', and the process returns to the construction rules based on the target verification strategy described above, using the first use case to generate an equivalent reference use case SQL B. The specific implementation process is similar to step 360 above. When it is determined that the mutation count of the first use case has reached the predetermined number, the database management system test ends, and the test result of the database management system is determined based on the comparison results of multiple singletons to verify the query correctness of the database management system. The specific implementation process is similar to step 370 above. To save space, it will not be elaborated here.

[0472] Description of apparatus and devices according to embodiments of this disclosure

[0473] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0474] It should be noted that in various specific embodiments of this application, when processing is required based on data related to the characteristics of the target object, such as target object attribute information or a set of attribute information, the permission or consent of the target object will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require obtaining target object attribute information, separate permission or consent from the target object will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the target object's separate permission or consent will the necessary target object-related data for the normal operation of the embodiments of this application be obtained.

[0475] Figure 23 A schematic diagram of the structure of a testing apparatus 2300 for a database management system provided in an embodiment of this disclosure. The testing apparatus 2300 for the database management system includes:

[0476] Acquisition unit 2310 is used to acquire the first use case;

[0477] The first determining unit 2320 is used to determine the target verification strategy corresponding to the first use case from multiple candidate verification strategies. The target verification strategy includes construction rules for constructing a reference use case based on the first use case, and comparison conditions for the first query result corresponding to the first use case and the second query result corresponding to the reference use case.

[0478] Construction unit 2330 is used to construct a reference use case based on the first use case and using construction rules;

[0479] The query unit 2340 is used to apply a first use case to the target corpus to obtain a first query result, and to apply a reference use case to the target corpus to obtain a second query result;

[0480] The comparison unit 2350 is used to compare the first query result and the second query result based on the comparison conditions to obtain a single-instance comparison result;

[0481] The mutation unit 2360 is used to mutate the first use case and return the steps of constructing a reference use case based on the first use case and using the construction rules, until the number of mutations of the first use case reaches a predetermined number.

[0482] The second determining unit 2370 is used to determine the test results of the database management system based on the comparison results of multiple singletons.

[0483] Optionally, the first determining unit 2320 is used for:

[0484] Determine the query statement structure for the first use case;

[0485] The query field of the first use case is validated based on the predetermined validation rules to obtain the validation result, which is used to indicate whether the target field exists in the first use case.

[0486] Based on the query structure and the test results, the target validation strategy is determined from multiple candidate validation strategies.

[0487] Optionally, the target verification strategy includes multiple construction rules for constructing multiple reference use cases based on the first use case, and comparison conditions for the first query result corresponding to the first use case and multiple second query results corresponding to the multiple reference use cases; the comparison conditions include a first comparison sub-condition between multiple second query results and a second comparison sub-condition between the first query result and each second query result;

[0488] Comparator unit 2350 is used for:

[0489] Based on the first comparison sub-condition, the first comparison sub-result is obtained by comparing each pair of multiple second query results;

[0490] Based on the second comparison sub-condition, the first query result and each second query result are compared to obtain the second comparison sub-result;

[0491] Based on the results of the first and second comparison sub-comparisons, a singleton comparison result is generated.

[0492] Optionally, the first use case includes a first segmentation method and a target query condition, and the reference use case includes a second segmentation method and a target query condition. The construction rule is to convert the first segmentation method in the first use case into the second segmentation method while keeping the target query condition unchanged.

[0493] Query unit 2340 is used for:

[0494] Apply the first segmentation method to the target corpus to obtain the first segmented corpus segment;

[0495] Apply the target query conditions to each first segment of the corpus to obtain the first subquery result corresponding to each first segment of the corpus;

[0496] The results of the first subquery corresponding to each first segment of the corpus are integrated into the first query result.

[0497] The second segmentation method is applied to the target corpus to obtain the second segmented corpus segments;

[0498] Apply the target query conditions to each second segment of the corpus to obtain the second subquery result corresponding to each second segment of the corpus;

[0499] The results of the second subquery corresponding to each second segment of the corpus are integrated into the second query result.

[0500] Optionally, the target verification strategy includes multiple construction rules for constructing multiple reference use cases based on the first use case, as well as the comparison conditions for the first query result corresponding to the first use case and the multiple second query results corresponding to the multiple reference use cases;

[0501] Comparator unit 2350 is used for:

[0502] Based on the comparison conditions, the second query results corresponding to multiple reference use cases are merged to obtain the merged second query result, and the first query result and the merged second query result are compared.

[0503] If the result of the first query is the same as the result of the merged second query, then the singleton comparison result is determined to be positive.

[0504] If the result of the first query is different from the result of the merged second query, then the singleton comparison result is determined to be negative.

[0505] Optionally, the construction rule is used to indicate that when the first use case has a first field and the target position of the first use case does not have a target function field, a reference use case is constructed using the second field corresponding to the first field;

[0506] Construction unit 2330 is used for:

[0507] Based on the constructed rules, field detection is performed on the first use case;

[0508] If it is determined that the first use case has a first field and the target position of the first use case does not have a target function field, then based on the second field, the first segmentation method in the first use case is converted into the second segmentation method, while keeping the target query conditions in the first use case unchanged, to obtain the reference use case.

[0509] Optionally, the construction unit 2330 is used for:

[0510] Test the query engine filtering module of the database management system;

[0511] If it is determined that there is optimization logic for the first use case in the query engine filtering module, then the position of at least one condition field of the first use case is adjusted using the construction rules to obtain the reference use case.

[0512] Optionally, the variant unit 2360 is used for:

[0513] For the first use case, the target number is determined within the first use case;

[0514] In the first use case, the target number is replaced with a number to obtain the mutated first use case.

[0515] Optionally, the target number may be replaced with a number, including:

[0516] The target number is increased or decreased according to a predetermined rule, and the increased or decreased target number is used to replace the target number.

[0517] or;

[0518] Take the first value from the preset first set and replace the target number with the first value.

[0519] Optionally, the variant unit 2360 is used for:

[0520] For the first use case, determine the target operator within the first use case;

[0521] Take a reference operator that is different from the target operator from the predefined second set;

[0522] In the first use case, the target operator is replaced with the reference operator to obtain the mutated first use case.

[0523] Optionally, the variant unit 2360 is used for:

[0524] Perform a non-empty check on the first test case;

[0525] If it is determined that the first use case contains a non-empty logical field, then replace the non-empty logical field of the first use case with an empty logical field to obtain the mutated first use case;

[0526] If it is determined that the first use case contains an empty logical field, then the empty logical field of the first use case is replaced with a non-empty logical field to obtain the mutated first use case.

[0527] Optionally, the variant unit 2360 is used for:

[0528] For the first use case, the target expression is determined from multiple candidate expressions for the first use case;

[0529] In the first use case, the target expression is deleted to obtain the mutated first use case.

[0530] Optionally, the variant unit 2360 is used for:

[0531] Construct new expressions;

[0532] Embed the new expression into the first use case to obtain the mutated first use case.

[0533] Optionally, the new expression is embedded into the first use case, including:

[0534] The newly added expression is checked, and after the expression check is passed, the new expression is embedded into the first use case;

[0535] or;

[0536] The new expression is embedded into the first use case to obtain the intermediate query statement. The intermediate query statement is then subjected to syntax validation. After the syntax validation is passed, the intermediate query statement is determined as the mutated first use case.

[0537] Optionally, the variant unit 2360 is used for:

[0538] In the first use case, determine the target logical field used to indicate the sorting method of the first query results;

[0539] Take out a sorting logical field that is different from the target logical field from the preset third set;

[0540] In the first use case, the target logical field is replaced with the sorting logical field to obtain the mutated first use case.

[0541] Optionally, the number of reservations is determined in the following way:

[0542] Determine the query structure for the first use case and the total amount of data in the database management system;

[0543] The predetermined number of attempts is determined based on the query structure and the total amount of data.

[0544] Optionally, the predetermined number of times can be determined based on the query structure and the total amount of data, including:

[0545] Determine the first score based on the query statement structure;

[0546] Determine the second score based on the total amount of data;

[0547] The predetermined number of times is determined based on the first score and the second score.

[0548] Optionally, the database management system test apparatus 2300 further includes an update unit (not shown), which is used for:

[0549] When the singleton comparison result is negative, determine the mutation count update strategy for the first use case;

[0550] The predetermined number of times is updated based on the mutation number update strategy to increase the predetermined number of times, and the updated predetermined number of times is not greater than the first value.

[0551] Optionally, the construction unit 2330 is used for:

[0552] Based on the construction rules, field detection is performed on the first use case; if it is determined that the first use case does not include aggregate function fields, the conditional expression in the first use case is pre-expressed to obtain the reference use case;

[0553] The first query result indicates the number of data items retrieved, and the second query result indicates the total number of data items retrieved; the comparison unit 2350 is used for:

[0554] Based on the comparison conditions, the data items in the first query result are statistically analyzed to obtain the data item statistical results;

[0555] If the statistical results of the data items are the same as the results of the second query, then the singleton comparison result is determined to be positive.

[0556] Optionally, the second determining unit 2370 is used for:

[0557] Determine the first proportion of singleton comparison results that are negative among multiple singleton comparison results;

[0558] Based on the first proportion, the test results of the database management system are determined, wherein the test results are used to indicate the degree of logical defects in the database management system.

[0559] Optionally, the construction unit 2330 is used for:

[0560] Based on the construction rules, target data items are extracted from the target corpus; the target data items are then embedded into the first use case to obtain the reference use case.

[0561] Comparator unit 2350 is used for:

[0562] If it is determined that the target data item is not found in the second query result, then the singleton comparison result is negative.

[0563] Reference Figure 24 , Figure 24To implement the structural block diagram of the terminal portion of the database management system testing method according to the embodiments of this disclosure, the terminal includes: a radio frequency (RF) circuit 2410, a memory 2415, an input unit 2430, a display unit 2440, a sensor 2450, an audio circuit 2460, a wireless fidelity (WiFi) module 2470, a processor 2480, and a power supply 2490, etc. Those skilled in the art will understand that... Figure 24 The terminal structure shown does not constitute a limitation on mobile phones or computers and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0564] The RF circuit 2410 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 2480; in addition, it transmits uplink data to the base station.

[0565] The memory 2415 can be used to store software programs and modules. The processor 2480 executes various functional applications and data processing of the target terminal by running the software programs and modules stored in the memory 2415.

[0566] The input unit 2430 can be used to receive input numeric or character information, and to generate key signal inputs related to the settings and function control of the target terminal. Specifically, the input unit 2430 may include a touch panel 2431 and other input devices 2432.

[0567] Display unit 2440 can be used to display input or provided information, as well as various menus of the target terminal. Display unit 2440 may include display panel 2441.

[0568] Audio circuitry 2460, speaker 2461, and microphone 2462 provide an audio interface.

[0569] In this embodiment, the processor 2480 included in the terminal can execute the database management system testing method of the previous embodiment.

[0570] The terminals disclosed in this embodiment include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The embodiments of this invention can be applied to various scenarios, including but not limited to data security, blockchain, data storage, and information technology.

[0571] Figure 25This is a structural block diagram of a server portion for implementing the testing method of the database management system according to embodiments of the present disclosure. The server can vary considerably due to differences in configuration or performance, and may include one or more Central Processing Units (CPUs) 2522 (e.g., one or more processors) and memory 2532, and one or more storage media 2530 (e.g., one or more mass storage devices) for storing application programs 2542 or data 2544. The memory 2532 and storage media 2530 may be temporary or persistent storage. The program stored in the storage media 2530 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 2522 may be configured to communicate with the storage media 2530 and execute the series of instruction operations in the storage media 2530 on the server.

[0572] The server may also include one or more power supplies 2526, one or more wired or wireless network interfaces 2550, one or more input / output interfaces 2558, and / or one or more operating systems 2541, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0573] The central processing unit 2522 in the server can be used to execute the test method of the database management system of the present disclosure embodiments.

[0574] This disclosure also provides a computer-readable storage medium for storing program code for executing the test methods of the database management system of the foregoing embodiments.

[0575] This disclosure also provides a computer program product comprising a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the test method for implementing the database management system described above.

[0576] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0577] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0578] It should be understood that in the description of the embodiments disclosed herein, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0579] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0580] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0581] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0582] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0583] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0584] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A testing method for a database management system, characterized in that, The method includes: Obtain the first use case; Among multiple candidate verification strategies, a target verification strategy corresponding to the first use case is determined, wherein the target verification strategy includes a construction rule for constructing a reference use case based on the first use case, and a comparison condition between a first query result corresponding to the first use case and a second query result corresponding to the reference use case; Based on the first use case, the reference use case is constructed using the construction rules; The first use case is applied to the target corpus to obtain the first query result, and the reference use case is applied to the target corpus to obtain the second query result; Based on the comparison conditions, the first query result and the second query result are compared to obtain a single-instance comparison result; The first use case is mutated, and the step of constructing the reference use case based on the first use case and using the construction rules is returned, until the number of mutations of the first use case reaches a predetermined number; Based on the comparison results of multiple single instances, the test results of the database management system are determined.

2. The test method according to claim 1, characterized in that, The target verification strategy includes multiple construction rules for constructing multiple reference use cases based on the first use case, and comparison conditions for the first query result corresponding to the first use case and multiple second query results corresponding to the multiple reference use cases; The comparison conditions include first comparison sub-conditions between multiple second query results and second comparison sub-conditions between the first query result and each of the second query results; The comparison of the first query result and the second query result based on the comparison conditions to obtain a single-instance comparison result includes: Based on the first comparison sub-condition, the first comparison sub-result is obtained by comparing each pair of multiple second query results; Based on the second comparison sub-condition, a comparison is made between the first query result and each of the second query results to obtain the second comparison sub-result; The singleton comparison result is generated based on the first comparison result and the second comparison result.

3. The test method according to claim 1, characterized in that, The first use case includes a first segmentation method and a target query condition, the reference use case includes a second segmentation method and the target query condition, and the construction rule is to convert the first segmentation method in the first use case into the second segmentation method while keeping the target query condition unchanged; The step of applying the first use case to the target corpus to obtain the first query result, and applying the reference use case to the target corpus to obtain the second query result, includes: The first segmentation method is applied to the target corpus to obtain the first segmented corpus segment; Apply the target query condition to each of the first segmented corpus segments to obtain the first subquery result corresponding to each of the first segmented corpus segments; The first subquery results corresponding to each of the first segmented corpus segments are integrated into the first query result; The second segmentation method is applied to the target corpus to obtain the second segmented corpus segment; Apply the target query condition to each of the second segmented corpus segments to obtain the second subquery result corresponding to each of the second segmented corpus segments; The results of the second subquery corresponding to each of the second segmented corpus segments are integrated into the second query result.

4. The test method according to claim 1, characterized in that, The target verification strategy includes multiple construction rules for constructing multiple reference use cases based on the first use case, and comparison conditions for the first query result corresponding to the first use case and multiple second query results corresponding to the multiple reference use cases; The comparison of the first query result and the second query result based on the comparison conditions to obtain a single-instance comparison result includes: Based on the comparison conditions, the second query results corresponding to the multiple reference use cases are merged to obtain the merged second query result, and the first query result and the merged second query result are compared. If the first query result is the same as the merged second query result, then the single-instance comparison result is determined to be a positive result. If the first query result is different from the merged second query result, then the singleton comparison result is determined to be a negative result.

5. The test method according to claim 1, characterized in that, The construction rule is used to indicate that when the first use case has a first field and the target position of the first use case does not have a target function field, the reference use case is constructed using the second field corresponding to the first field; The step of constructing the reference use case based on the first use case and using the construction rules includes: Based on the construction rules, field detection is performed on the first use case; If it is determined that the first use case has a first field and the target position of the first use case does not have a target function field, then based on the second field, the first segmentation method in the first use case is converted into a second segmentation method, while keeping the target query conditions in the first use case unchanged, to obtain the reference use case.

6. The test method according to claim 1, characterized in that, The step of constructing the reference use case based on the first use case and using the construction rules includes: Detect the query engine filtering module of the database management system; If it is determined that there is optimization logic for the first use case in the query engine filtering module, then the position of at least one condition field of the first use case is adjusted using the construction rules to obtain the reference use case.

7. The test method according to claim 1, characterized in that, The mutation of the first use case includes: For the first use case, the target number is determined within the first use case; In the first use case, the target number is replaced with a number to obtain the modified first use case.

8. The test method according to claim 1, characterized in that, The mutation of the first use case includes: For the first use case, the target operator is determined in the first use case; Take a reference operator that is different from the target operator from the preset second set; In the first use case, the target operator is replaced with the reference operator to obtain the mutated first use case.

9. The test method according to claim 1, characterized in that, The mutation of the first use case includes: Perform a non-empty check on the first use case; If it is determined that the first use case contains a non-empty logical field, then the non-empty logical field of the first use case is replaced with an empty logical field to obtain the mutated first use case; If it is determined that the first use case contains an empty logical field, then the empty logical field of the first use case is replaced with a non-empty logical field to obtain the mutated first use case.

10. The test method according to claim 1, characterized in that, The mutation of the first use case includes: For the first use case, the target expression is determined from multiple candidate expressions of the first use case; In the first use case, the target expression is deleted to obtain the mutated first use case.

11. The test method according to claim 1, characterized in that, The mutation of the first use case includes: Construct new expressions; The newly added expression is embedded into the first use case to obtain the mutated first use case.

12. The test method according to claim 1, characterized in that, The mutation of the first use case includes: In the first use case, determine the target logical field used to indicate the sorting method of the first query results; Take out a sorting logical field that is different from the target logical field from the preset third set; In the first use case, the target logical field is replaced with the sorting logical field to obtain the modified first use case.

13. The test method according to claim 1, characterized in that, The predetermined number of times is determined in the following way: Determine the query statement structure of the first use case and determine the total amount of data in the database management system; The predetermined number of times is determined based on the structure of the query statement and the total amount of data.

14. The test method according to claim 1, characterized in that, After comparing the first query result and the second query result based on the comparison conditions to obtain a singleton comparison result, the method further includes: When the singleton comparison result is negative, determine the mutation count update strategy for the first use case; The predetermined number of mutations is updated based on the mutation number update strategy to increase the predetermined number of mutations, and the updated predetermined number of mutations is not greater than the first value.

15. The test method according to claim 1, characterized in that, The step of determining the target verification strategy corresponding to the first use case from multiple candidate verification strategies includes: Determine the query statement structure for the first use case; Based on predetermined verification rules, the query field of the first use case is verified to obtain the verification result, which is used to indicate whether the target field exists in the first use case. Based on the query statement structure and the verification results, the target verification strategy is determined from multiple candidate verification strategies.

16. The test method according to any one of claims 1 to 15, characterized in that, The step of constructing the reference use case based on the first use case and using the construction rules includes: performing field detection on the first use case based on the construction rules; if it is determined that the first use case does not include an aggregate function field, then prepending the conditional expression in the first use case to obtain the reference use case; The first query result indicates the number of data items retrieved, and the second query result indicates the total number of data items retrieved; the comparison of the first query result and the second query result based on the comparison condition to obtain a single-instance comparison result includes: Based on the comparison conditions, the data items in the first query result are statistically analyzed to obtain the data item statistical results; If it is determined that the statistical result of the data item is the same as the second query result, then the single-case comparison result is determined to be a positive result.

17. A testing apparatus for a database management system, characterized in that, The device includes: The acquisition unit is used to acquire the first use case; The first determining unit is configured to determine the target verification strategy corresponding to the first use case from among multiple candidate verification strategies, wherein the target verification strategy includes a construction rule for constructing a reference use case based on the first use case, and a comparison condition between a first query result corresponding to the first use case and a second query result corresponding to the reference use case. A construction unit is configured to construct the reference use case based on the first use case and using the construction rules. The query unit is configured to apply the first use case to the target corpus to obtain the first query result, and apply the reference use case to the target corpus to obtain the second query result; The comparison unit is used to compare the first query result and the second query result based on the comparison conditions to obtain a single-instance comparison result; The mutation unit is used to mutate the first use case and return the step of constructing the reference use case based on the first use case and using the construction rules, until the number of mutations of the first use case reaches a predetermined number. The second determining unit is used to determine the test results of the database management system based on multiple single-instance comparison results.

18. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the testing method of the database management system according to any one of claims 1 to 16.

19. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the testing method of the database management system according to any one of claims 1 to 16.

20. A computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform a test method for a database management system as described in any one of claims 1 to 16.

Images