Database test case extension method and device based on large language model and readable medium

Abstract

The invention discloses a database test case extension method and device based on a large language model and a readable medium. The method comprises the following steps: constructing a feature knowledge base corresponding to each database management system; identifying an original dialect feature in an original query SQL statement of the initial database management system through an error recovery mechanism, and performing dialect mapping on the original dialect feature in a feature knowledge base corresponding to the target database management system to obtain a target dialect feature and a description and an example corresponding to the target dialect feature; inputting an original SQL (Structured Query Language) statement of the initial database management system into the conversion model, constructing a first cue word, and guiding the conversion model to generate an original SQL statement of the target database management system through the first cue word; and inputting the original query SQL statement of the target database management system into the fine-tuned variation model corresponding to the selected variation strategy, and generating the variation query SQL statement of the corresponding target database management system, thereby effectively improving the universality and reliability.

Description

Technical Field

[0001] The present invention relates to the field of database testing, and in particular to a method, a device and a readable medium for expanding database test cases based on a large language model. Background Art

[0002] Database Management Systems (DBMS) are crucial in applications such as online banking and e-commerce. However, the complex code logic and diverse functions of DBMS make it prone to errors, especially logical errors, which often result in incorrect result sets being returned without obvious symptoms. Structured Query Language (SQL) is a domain-specific language used to manage data in DBMS. SQL syntax describes the syntax and semantics of the language. Generally, most DBMS products follow the basic features of the ANSI SQL standard to support common operations and extend support for advanced functions through unique SQL dialects. However, the syntax of these SQL dialects is not compatible because similar functions may be implemented by different syntax variants in different DBMS. For example, the character set comparison and sorting functions in MySQL are implemented as COLLATION, while in PostgreSQL they are implemented as COLLATE FOR. There are about 423 popular DBMS products on the market, and each product differs in the supported features, such as data types, functions, operators, etc.

[0003] Automated testing is a common method for detecting DBMS logical errors. Automated testing methods use oracle to verify the correctness of DBMS results. It is worth noting that metamorphic oracle is a widely used method that tests by constructing SQL statements that are exactly or approximately equivalent to the original query. This technology detects whether the DBMS has logical errors by comparing the execution results of these mutated queries with the execution results of the original query. Unlike traditional fuzz testing, metamorphic testing requires ensuring the correctness of syntax and semantics, and strictly maintaining the metamorphic relationship between queries. This method is called MOLT (Metamorphic-Oracle based Logical Bug Detection Technique).

[0004] In order to generate SQL statement pairs suitable for MOLT, including original queries and mutated queries, a general idea is to model the SQL grammar as an abstract syntax tree (AST). This AST model forms the basis for generating original queries, ensuring grammatical correctness according to the specific syntax of the target DBMS. Subsequently, these statements are mutated according to a specific pattern to ensure that the mutated queries strictly adhere to the defined metamorphosis relationship with the original queries. For example, Pinolo customized its parser for MySQL syntax. It randomly generates an original query based on the AST, such as "SELECT(COLLATION(`f`))from t", and then mutates the query to "SELECT DISTINCT(COLLATION(`f`))from t" according to a specific pattern, ensuring that the result of the original query contains the result of the mutated query.

[0005] Currently, the most advanced MOLT methods mainly rely on the syntax of a specific DBMS to generate valid SQL statement pairs. However, this approach requires a lot of manual work, which limits the existing MOLT to only support a few popular DBMSs, such as PostgreSQL, MySQL, and MariaDB. For example, in order to integrate the syntax of PostgreSQL, SQLRight needs to write an additional 93,671 lines of code. In addition, there are currently 423 different DBMS products on the market, most of which have unique dialects and syntaxes, and it is difficult for existing MOLT tools to cover the testing of these DBMSs. Therefore, there is an urgent need for a method that can easily extend the existing MOLT to a diverse database environment with minimal workload.

[0006] In recent years, software engineering applications based on large language models (LLM) have increased rapidly, from which we can observe the excellent performance of LLM in SQL generation. Based on this, a potential solution is to regard LLM as an expert in the field of DBMS SQL conversion, guide LLM by designing appropriate prompt words, and directly translate existing SQL statement pairs into a form that meets the syntax of the target database while ensuring the transformation relationship. This can effectively reduce the additional workload required to expand database testing tools. However, using LLM to directly translate SQL statement pairs between different DBMSs faces three major challenges:

[0007] 1. Limited understanding of dialect differences between DBMSs. LLMs lack a deep understanding of dialect differences between DBMSs, which can lead to a large number of grammatical and semantic errors, which in turn can generate semantically inequivalent queries. For example, when translating the MySQL query "SELECT(COLLATION(`f`))from t" to PostgreSQL, an LLM lacking domain knowledge may mistakenly simplify it to "SELECT f from t", omitting dialect-specific functions for character set comparison and sorting. This results in a functionally inequivalent conversion because the LLM fails to recognize that in PostgreSQL, character set comparison and sorting can be achieved through the COLLATE FOR function. In order to improve the conversion of semantically equivalent queries, it is very necessary to enhance LLM's understanding of DBMS-specific dialects.

[0008] 2. Limited understanding of the transformation mechanism. Secondly, LLM has a limited understanding of the transformation mechanism, which may result in the converted SQL statement pairs not satisfying the expected transformation relationship, thus affecting the efficiency of logical error detection. For example, Figure 1 As shown in the figure, when both the original MySQL query and the mutated query are translated into PostgreSQL, the mutated query loses the "DISTINCT" keyword, resulting in the failure to maintain the predefined lower approximation relationship between the original query and the mutated query. Without this relationship, the ability to detect errors by comparing query results is lost, which seriously weakens the effectiveness of metamorphic testing in discovering logical errors.

[0009] 3. Dependence on syntax limits the adaptability of MOLT. Specifically, whether it is a generative model, SQL parser or metamorphosis mutator, they are all strongly dependent on the syntax of the DBMS, and adapting a new DBMS requires a lot of additional code, which is time-consuming and labor-intensive. Table 1 shows the number of additional lines of code required for SQLancer, SQLRight, Pinolo, and DQE when adapting SQLite, PostgreSQL, MySQL, and MariaDB. It can be seen that due to their dependence on syntax, these tools require significant code expansion to adapt to different DBMS. For example, SQLRight uses more than 90,000 lines of code to build a generative model for PostgreSQL, while Pinolo simplifies its implementation by only supporting the common syntax shared between MySQL and MariaDB.

[0010] Table 1. The number of lines of code for SQLancer, SQRight, Pinolo, and DQE to adapt to SQLite, PostgreSQL, MySQL, and MariaDB

[0011] Tool Code Lines SQLancer SQLRight Pinolo DQ SQLite 10020 17120 - 7606 PostgreSQL 8012 93671 - - MySQL 6512 76067 8055 4225 MarinaDB 2085 - 8055 2092

[0012] More importantly, there are 423 different DBMSs on the market, most of which use different syntaxes. Existing MOLTs are mainly targeted at well-known DBMSs such as MySQL, PostgreSQL, and MariaDB. If you want to adapt them to these different DBMSs, the time and labor costs will be too high and economically unfeasible. Therefore, there is an urgent need for a technology that can easily extend different MOLTs to multiple DBMS environments. Summary of the invention

[0013] The purpose of this application is to propose a database test case expansion method, device and readable medium based on a large language model to address the above-mentioned technical problems.

[0014] In a first aspect, the present invention provides a method for expanding database test cases based on a large language model, comprising the following steps:

[0015] Construct a feature knowledge base corresponding to each database management system, the feature knowledge base storing all dialect features, descriptions and examples in the corresponding database management system;

[0016] Obtaining the original query SQL statement of the initial database management system, identifying the original dialect features in the original query SQL statement of the initial database management system through an error recovery mechanism, performing dialect mapping on the original dialect features in a feature knowledge base corresponding to the target database management system, and obtaining the target dialect features and their corresponding descriptions and examples;

[0017] Constructing a conversion model based on a pre-trained large language model, inputting the original query SQL statement of the initial database management system into the conversion model, constructing a first prompt word using the original database management system, the target database management system, the original query SQL statement of the initial database management system, target dialect features and their corresponding descriptions and examples, and guiding the conversion model to generate the original query SQL statement of the target database management system through the first prompt word;

[0018] A mutation model based on a pre-trained large language model is constructed, and the mutation model is fine-tuned for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy. The original query SQL statement of the target database management system is input into the fine-tuned mutation model corresponding to the selected mutation strategy to generate the corresponding mutated query SQL statement of the target database management system. The original query SQL statement and the mutated query SQL statement of the target database management system are used as test cases for metamorphosis testing.

[0019] Preferably, the dialect features in the feature knowledge base are divided into three types: data types, functions and operators, and dialect mapping is performed in the feature knowledge base based on each type of dialect features.

[0020] Preferably, identifying the original dialect features in the original query SQL statement of the initial database management system through the error recovery mechanism specifically includes:

[0021] The original query SQL statement of the initial database management system is disassembled into a series of lexical symbols, and the lexical symbols are passed to the standard parser. When the standard parser cannot recognize or is specific to a certain dialect symbol, the error recovery mechanism allows the standard parser to skip the dialect symbol and continue to parse the remaining symbols, and use the dialect symbol as the original dialect feature in the original query SQL statement of the initial database management system.

[0022] Preferably, the original dialect features are dialect-mapped in the feature knowledge base corresponding to the target database management system to obtain the target dialect features and their corresponding descriptions and examples, specifically including:

[0023] It is judged by keywords whether a dialect feature similar to the original dialect feature can be retrieved in the feature knowledge base corresponding to the target database management system. If so, the dialect feature similar to the original dialect feature is used as the target dialect feature and the corresponding description and example are obtained; otherwise, the similarity between the vector embedding of the original dialect feature and the vector embedding of each dialect feature in the feature knowledge base corresponding to the target database management system is calculated respectively, and the dialect feature in the feature knowledge base corresponding to the target database management system with the highest similarity is selected as the target dialect feature and the corresponding description and example are obtained.

[0024] Preferably, the first prompt word includes a task description, an original query SQL statement of an initial database management system, and step-by-step instructions. The task description defines the task and provides context for a conversion model, and clarifies the original database management system and the target database management system. The original query SQL statement of the initial database management system is displayed in the form of code blocks, and the step-by-step instructions use chain thinking reasoning and acquire knowledge from target dialect features and their corresponding descriptions and examples.

[0025] Preferably, the mutation model is fine-tuned for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy, specifically including:

[0026] Obtain the SQL statement pairs corresponding to each mutation strategy processed by the MOLT tool, adopt a role-playing approach and determine the answer format, construct a second prompt word based on the SQL statement pairs corresponding to each mutation strategy and the explanation of the mutation strategy, and use the second prompt word to guide the mutation model to fine-tune, and obtain the fine-tuned mutation model corresponding to each mutation strategy.

[0027] In a second aspect, the present invention provides a database test case expansion device based on a large language model, comprising:

[0028] A knowledge base building module is configured to build a feature knowledge base corresponding to each database management system, the feature knowledge base storing all dialect features, descriptions and examples in the corresponding database management system;

[0029] A retrieval module is configured to obtain an original query SQL statement of an initial database management system, identify original dialect features in the original query SQL statement of the initial database management system through an error recovery mechanism, perform dialect mapping on the original dialect features in a feature knowledge base corresponding to a target database management system, and obtain target dialect features and their corresponding descriptions and examples;

[0030] a conversion module configured to construct a conversion model based on a pre-trained large language model, input an original query SQL statement of an initial database management system into the conversion model, construct a first prompt word using the original database management system, the target database management system, the original query SQL statement of the initial database management system, target dialect features and their corresponding descriptions and examples, and guide the conversion model to generate the original query SQL statement of the target database management system through the first prompt word;

[0031] The mutation module is configured to build a mutation model based on a pre-trained large language model, fine-tune the mutation model for each mutation strategy, obtain the fine-tuned mutation model corresponding to each mutation strategy, input the original query SQL statement of the target database management system into the fine-tuned mutation model corresponding to the selected mutation strategy, generate the corresponding mutated query SQL statement of the target database management system, and use the original query SQL statement and the mutated query SQL statement of the target database management system as test cases for metamorphosis testing.

[0032] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0033] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0034] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) The database test case extension method based on large language model proposed in the present invention can automatically extend the existing MOLT technology to different DBMS. The core idea is to use LLM to translate the SQL statement pairs generated in the existing MOLT into a form suitable for different DBMS, so as to realize the direct extension of the existing MOLT technology to the new DBMS for metamorphosis testing.

[0037] (2) The database test case expansion method based on the large language model proposed in the present invention includes two stages: the conversion stage and the mutation stage. The conversion stage aims to identify potential dialect differences and use the information in the SQL document to improve the query retrieval capability, so that the conversion model can accurately translate the original query SQL statement into different DBMS. In the mutation stage, SQL statement pairs are collected from the existing MOLT, and these data are used to fine-tune the mutation model based on the pre-trained large language model to make it specifically adapted to the mutation task. The fine-tuned mutation model is then used to mutate the original query SQL statement of the target database management system, maintaining the necessary metamorphosis relationship in order to perform metamorphosis testing, thereby addressing the limitations of LLM in understanding dialect differences and metamorphosis mechanisms.

[0038] (3) The versatility of the database test case expansion method based on the large language model proposed in the present invention can be significantly improved, and the reliability of the database management system can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 This is a failure example diagram of using LLM to directly convert SQL statement pairs in Pinolo;

[0041] Figure 2 A flowchart of a method for expanding database test cases based on a large language model according to an embodiment of the present application;

[0042] Figure 3 This is an example diagram of a database test case expansion method based on a large language model according to an embodiment of the present application;

[0043] Figure 4A schematic diagram of the workflow of the conversion phase and the mutation phase of the database test case expansion method based on a large language model according to an embodiment of the present application;

[0044] Figure 5 A schematic diagram of a template of a first prompt word in a conversion phase of a database test case expansion method based on a large language model according to an embodiment of the present application;

[0045] Figure 6 The result diagram of conversion efficiency and false positive analysis of the database test case expansion method based on a large language model in an embodiment of the present application; wherein, Figure 6 (a) is a diagram of the manual analysis results of SQL statement pairs whose query results do not meet the expected transformation relationship (a true positive indicates that the abnormality of the execution result of the SQL statement pair is caused by a logical error, otherwise it is a false positive). Figure 6 (b) A graph showing the distribution of the time cost of each benchmark program in the conversion phase and the mutation phase;

[0046] Figure 7 This is an example result diagram of the execution of the database test case expansion method based on a large language model in an embodiment of the present application; wherein, Figure 7 (a) shows a pair of SQL statements generated by the NoREC tool in SQLite that satisfy the equivalence relationship; Figure 7 (b) shows a target SQL statement pair converted into DuckDB by the method of the embodiment, and a result graph of a logical error from DuckDB is detected through inconsistent results generated after the statement pair is executed;

[0047] Figure 7 (c) shows the result of directly using an unprocessed large language model to convert the target SQL statement pair without using the method of the embodiment;

[0048] Figure 8 A result diagram of a logic error report caused by implicit type conversion found by a database test case expansion method based on a large language model according to an embodiment of the present application;

[0049] Fig. 9 A schematic diagram of a database test case expansion device based on a large language model according to an embodiment of the present application;

[0050] Fig.10 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Figure 2 A method for expanding database test cases based on a large language model provided by an embodiment of the present application is shown, comprising the following steps:

[0053] S1, build a feature knowledge base corresponding to each database management system, and the feature knowledge base stores all dialect features, descriptions and examples in the corresponding database management system;

[0054] In a specific embodiment, the dialect features in the feature knowledge base are divided into three types: data types, functions, and operators, and dialect mapping is performed in the feature knowledge base based on each type of dialect features.

[0055] Specifically, refer to Figure 3 , the embodiment of the present application proposes a database test case extension method (QTRAN) based on a large language model, which aims to automatically extend the existing MOLTs to various database management systems (DBMS). Inspired by the DBMS metamorphosis test process, the embodiment of the present application divides the extension process into two stages: the conversion stage and the mutation stage. QTRAN starts with the SQL statement pairs of the existing MOLTs and extends these statement pairs to the new DBMS through these two stages.

[0056] Specifically, in order to address the problem that large language models (LLMs) lack a deep understanding of the dialects of database management systems, which in turn leads to grammatical and semantic errors, an embodiment of the present application constructs a feature knowledge base from structured SQL documents to enhance the large language model's understanding of dialect feature differences. The reason for extracting features from SQL documents is that these documents are highly structured and describe the SQL syntax of each database management system in detail. Most databases provide comprehensive SQL documents, which makes them an ideal source for building a powerful and accurate feature knowledge base.

[0057] Next, the process of building a feature knowledge base will be explained. The dialect features are classified into three major categories: data types, functions, and operators. It is important to note that these elements can usually be mapped across different database management systems because they usually perform similar operations, such as string processing, mathematical calculations, or date processing, which are common requirements in all database systems. To ensure consistency and accuracy, web crawling technology is used to extract relevant dialect features from SQL documents. Each dialect feature goes through a data cleaning process and is standardized into three key components: feature rules, descriptions, and examples. Through this unified structure, the knowledge base is able to provide rich and well-organized dialect-specific knowledge to the large language model, improving its ability to accurately translate queries between different database management systems.

[0058] The embodiments of the present application have constructed feature knowledge bases for eight popular database management systems. Table 2 provides an overview of the feature knowledge bases constructed for eight popular DBMSs, detailing the distribution of standard features and dialect features in the three major categories of data types, functions, and operators. To determine whether a feature is a dialect feature, a systematic approach is adopted. First, a simple query statement is constructed for each feature using a large language model. Then, the query statement is parsed using the ANSI standard parser. If the parsing fails, it indicates that the feature does not conform to the standard SQL syntax and is identified as a dialect feature. It is worth noting that dialect feature differences in functions and data types account for the majority of features in many database management systems, and these differences deviate significantly from the ANSI SQL standard. This observation emphasizes the importance of dialect mapping to ensure compatibility and accurate interpretation between different database management systems.

[0059] Table 2 Statistics of feature knowledge base

[0060]

[0061] S2, obtain the original query SQL statement of the initial database management system, identify the original dialect features in the original query SQL statement of the initial database management system through the error recovery mechanism, perform dialect mapping on the original dialect features of the original database management system in the feature knowledge base corresponding to the target database management system, and obtain the target dialect features and their corresponding descriptions and examples.

[0062] In a specific embodiment, identifying the original dialect features in the original query SQL statement of the initial database management system through the error recovery mechanism specifically includes:

[0063] The original query SQL statement of the initial database management system is disassembled into a series of lexical symbols, and the lexical symbols are passed to the standard parser. When the standard parser cannot recognize or is specific to a certain dialect symbol, the error recovery mechanism allows the standard parser to skip the dialect symbol and continue to parse the remaining symbols, and use the dialect symbol as the original dialect feature in the original query SQL statement of the initial database management system.

[0064] First, the original dialect features in the original query SQL statements of the original DBMS, such as function names, are identified through an error recovery mechanism. The identified original dialect features are then mapped to their equivalents in the target DBMS through a pre-established feature knowledge base with the help of RAG. This rich information and the original query SQL statements of the original DBMS are input into the conversion model, which converts the original query SQL statements of the original DBMS into the grammatical form of the target DBMS, ensuring accurate translation and semantic correctness. The goal of the conversion phase is to convert the original SQL queries from popular DBMS into queries that match the syntax of the target DBMS. The original dialect features in the original query SQL statements of the original DBMS are identified, mapped to the target DBMS, and adapted to the syntax of the target DBMS through the feature knowledge base retrieved by the enhanced RAG. The syntax includes standard features and dialect features.

[0065] Dialect identification is a key step when transferring the original query SQL statement from one DBMS to another DBMS. In this process, the embodiment of the present application utilizes the error recovery mechanism provided by ANTLR to process dialect symbols. Specifically, the content of the original query SQL statement is first disassembled into a series of lexical symbols (tokens), and then these lexical symbols are passed to the standard parser in turn. When the standard parser encounters dialect symbols that cannot be recognized or are specific to a certain DBMS, such as certain specific functions or data types, the error recovery mechanism allows the standard parser to skip the problem part and continue to parse the remaining symbols. These problematic symbols are exactly the objects of the next dialect mapping. This method enables the system to identify the query part that needs to be processed by a specific dialect without terminating the parsing process prematurely.

[0066] In a specific embodiment, the original dialect features are dialect-mapped in the feature knowledge base corresponding to the target database management system to obtain the target dialect features and their corresponding descriptions and examples, specifically including:

[0067] It is judged by keywords whether a dialect feature similar to the original dialect feature can be retrieved in the feature knowledge base corresponding to the target database management system. If so, the dialect feature similar to the original dialect feature is used as the target dialect feature and the corresponding description and example are obtained; otherwise, the similarity between the vector embedding of the original dialect feature and the vector embedding of each dialect feature in the feature knowledge base corresponding to the target database management system is calculated respectively, and the dialect feature in the feature knowledge base corresponding to the target database management system with the highest similarity is selected as the target dialect feature and the corresponding description and example are obtained.

[0068] Specifically, dialect mapping is the process of converting the original dialect features in the original DBMS into the corresponding target dialect features in the target DBMS through the feature knowledge base. Since dialect features (such as functions or data types) may vary greatly between different database management systems, the embodiment of the present application adopts the RAG method to accurately perform this mapping process. First, for each original dialect feature identified in the original DBMS, the corresponding dialect feature is searched in the feature knowledge base of the target DBMS. This can be achieved through keyword-based retrieval, finding similar dialect features from the feature knowledge base of the target DBMS and using them as target dialect features. If a match cannot be found directly, fall back to a vector embedding-based similarity search to retrieve semantically related dialect features and use them as target dialect features. For example, if the "COLLATION()" function in MySQL needs to be converted, a similar function, such as "COLLATE FOR()" in PostgreSQL, will be retrieved by performing a vector embedding similarity search in the target DBMS, and then replaced with it.

[0069] S3, construct a conversion model based on the pre-trained large language model, input the original query SQL statement of the initial database management system into the conversion model, use the original database management system, the target database management system, the original query SQL statement of the initial database management system, the target dialect features and their corresponding descriptions and examples to construct a first prompt word, and guide the conversion model to generate the original query SQL statement of the target database management system through the first prompt word.

[0070] In a specific embodiment, the first prompt word includes a task description, an original query SQL statement of an initial database management system, and step-by-step instructions. The task description defines the task and provides context for the conversion model, and clarifies the original database management system and the target database management system. The original query SQL statement of the initial database management system is displayed in the form of code blocks, and the step-by-step instructions use chain thinking reasoning and acquire knowledge from the target dialect features and their corresponding descriptions and examples.

[0071] Specifically, a conversion model based on a pre-trained large language model is constructed, and the pre-trained large language model can use a GPT series model. In an embodiment of the present application, the pre-trained large language model is implemented using the GPT-4o-mini model in the GPT series.

[0072] Once the target dialect features are identified, dialect mapping is used to enhance the retrieved knowledge, thereby helping LLM complete the conversion process. The first prompt word set in the conversion model is highly structured, aiming to guide LLM to gradually perform accurate SQL query statement conversion to adapt to different DBMS. Figure 4 As shown, the first prompt word contains three main components:

[0073] (1) Task description: Define the task and provide context for the LLM to help it understand the purpose and requirements of the transformation.

[0074] (2) Original query SQL statements of the initial database management system: The original query SQL statements of the initial database management system are displayed in the form of code blocks so that LLM can clearly see the statements that need to be converted.

[0075] (3) Step-by-step instructions: Use Chain-of-Thought (CoT) reasoning to ensure that the semantics remain consistent during the conversion process, column names remain equal, and can adapt to dialect-specific features.

[0076] By using such a structured first hint word, it can be ensured that when executing SQL statement transfer, the conversion model can not only correctly handle grammatical differences, but also ensure that the semantics of the original query SQL statement is not lost, while adapting to the specific dialect of the target DBMS.

[0077] S4, construct a mutation model based on the pre-trained large language model, fine-tune the mutation model for each mutation strategy, obtain the fine-tuned mutation model corresponding to each mutation strategy, input the original query SQL statement of the target database management system into the fine-tuned mutation model corresponding to the selected mutation strategy, generate the corresponding mutation query SQL statement of the target database management system, and use the original query SQL statement and the mutation query SQL statement of the target database management system as test cases for metamorphosis testing.

[0078] In a specific embodiment, fine-tuning the mutation model for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy specifically includes:

[0079] Obtain the SQL statement pairs corresponding to each mutation strategy processed by the MOLT tool, adopt a role-playing approach and determine the answer format, construct a second prompt word based on the SQL statement pairs corresponding to each mutation strategy and the explanation of the mutation strategy, and use the second prompt word to guide the mutation model to fine-tune, and obtain the fine-tuned mutation model corresponding to each mutation strategy.

[0080] Specifically, in the mutation stage, a mutation model based on a pre-trained large language model is first constructed, and then SQL statement pairs in the existing MOLTs are collected, and the role information and the explanation of the mutation strategy are added in the second prompt word to enrich these SQL statement pairs, and then the mutation model is fine-tuned. The fine-tuned mutation model is then used to mutate the original query SQL statement of the target database management system to generate a mutated query SQL statement that satisfies the metamorphosis relationship, which is crucial for effectively detecting logical errors. In addition, the embodiment of the present application also introduces a model validator to evaluate the validity of the fine-tuned mutation model to ensure that the mutated query SQL statement maintains its correctness and accurately reflects the expected logical transformation.

[0081] The goal of the mutation phase is to mutate the original query SQL statements of the target database management system from the conversion phase according to a specific mutation strategy, which is a key step in detecting logical errors in different DBMSs. Next, two key components of the mutation phase will be introduced: model fine-tuning and model validator.

[0082] Model fine-tuning is an important process that adapts pre-trained LLMs to specific tasks by further training them on domain-specific datasets to improve their performance in target applications. This process customizes the general language capabilities of the LLM to perform better in a specific domain. It usually starts with a pre-trained GPT-4o-mini, which has learned a wide range of language patterns and semantics from a large-scale text corpus. The model is then fine-tuned on a smaller, domain-specific dataset to transfer its general language knowledge to the variation task.

[0083] The data for model fine-tuning comes from existing MOLTs, which provide SQL statement pairs. Figure 5The fine-tuning process using the "FixMDistinctL" mutation strategy is demonstrated. The fine-tuning process adopts a role-playing format, in which the LLM is prompted to be an expert in the field of SQL mutation. The mutation model is input with SQL statement pairs from tools such as Pinolo, and the original query SQL statements of the target database management system are mutated according to mutation strategies such as "FixMDistinctL". By showing these structured statement pairs to the mutation model, it learns the predetermined logical transformation, thereby realizing the ability to generate SQL statements that meet the specified mutation strategy and metamorphosis relationship for different DBMS. For each mutation strategy, the embodiment of the present application collects 20 groups of SQL statement pairs, which have been confirmed to be effective by the model validator, ensuring that the mutation model is exposed to comprehensive mutation operations during the training process. It is worth noting that each MOLT may include multiple mutation strategies. For example, Pinolo includes 25 mutation strategies, such as "FixMDistinctL". In order to ensure the uniqueness of each mutation strategy, 500 groups (20*25) of SQL statement pairs need to be collected for fine-tuning. This refined approach ensures that each mutation strategy is carefully designed and customized for a specific logical transformation.

[0084] The purpose of the model validator is to ensure the effectiveness of the mutation model after fine-tuning. The collected SQL statements are divided into 80% for training set and 20% for validation set. During the training process, the model is evaluated by multiple indicators, including training loss, training labeling accuracy, validation loss, and validation labeling accuracy. These indicators are calculated on a small batch of data at each step and on the entire validation set at the end of each epoch. Once the training loss drops to a stable point and the validation labeling accuracy reaches a sufficiently high threshold (over 95%), the fine-tuned mutation model can be used to mutate the original query SQL statement of the target database management system to ensure that the generated mutated query SQL statement maintains the predetermined logical transformation, thereby effectively detecting logical errors.

[0085] The effectiveness of QTRAN is fully evaluated below. The purpose of the experiment is to answer the following research questions:

[0086] (1)Q1: Can QTRAN effectively and efficiently convert the SQL statement pairs of existing MOLTs into different DBMS environments for metamorphosis testing?

[0087] (2)Q2: Can QTRAN discover errors that have not been discovered before?

[0088] (3)Q3: How does dialect mapping help SQL statement translation?

[0089] (4)Q4: How does fine-tuning help mutation operations in metamorphosis testing?

[0090] The tested database management systems include: MySQL, MariaDB, TiDB, PostgreSQL, SQLite, MonetDB, DuckDB, and ClickHouse. Table 3 describes the detailed information of these databases.

[0091] Table 3 Basic information of the tested DBMS

[0092] DBMS Version Number Number of GitHub Stars Number of engine implementations First published time MySQL 8.0.39 8.6K 2 1995 MariaDB 11.5.2 4.6K 15 2009 TiDB 8.3.0 33.1K 77 2017 PostgreSQL 16.3 16.1K 4 1989 SQLite 3.45.1 6.5K 10 2000 MonetDB 11.5.3 375 135 2004 DuckDB 1.1.2 23.6K 57 2018 ClickHouse 24.9.2.42 37.2K 37 2018

[0093] Benchmark comparison: In this evaluation, four state-of-the-art MOLTs were selected for extension: NoRec, TLP, Pinolo, and DQE. The following is a detailed introduction to each tool:

[0094] (1) NoRec: This technique transfers the predicate in the WHERE clause to the SELECT clause. If the result is different after this conversion, it is recognized as a logical error.

[0095] (2) TLP: This method decomposes a query into three separate queries, each with its own predicate. If the results of these queries are different from those of the original query, a logical error is determined.

[0096] (3) Pinolo: Modify query predicates to adjust constraints, expecting a superset or subset of the original results to confirm the correctness of the logic.

[0097] (4) DQE: By changing the type of SQL query—such as SELECT, UPDATE, and DELETE—while keeping the predicate unchanged, if the results of the operation on the row are inconsistent, it is considered a potential error.

[0098] These MOLTs need to understand the SQL syntax of different DBMS. Table 4 describes the databases supported by these MOLTs. In order to avoid duplication of tests, QTRAN is used to extend these MOLTs and test the databases that are not natively supported by them.

[0099] Table 4 Applicability of existing MOLTs on selected DBMS

[0100]

[0101] Experimental environment: The embodiments of this application are performed on a 104-core Intel(R) Xeon(R) Gold 6230R CPU.

[0102] The experiments were conducted on a server with 1000 MHz @2.10 GHz and 500 GB of memory. The server ran the Ubuntu 20.04 operating system and used the 5.4.0-135-generic version of the Linux kernel. Unless otherwise specified, all experiments were performed by calling the OpenAI API (version GPT-4o-mini-2024-07-18) and using the first prompt word mentioned above. In order to take advantage of the deterministic nature of LLM and ensure more predictable and stable results, the temperature was set to 0.0.

[0103] In Q1, the goal is to evaluate the effectiveness and efficiency of QTRAN in converting SQL query statements in existing MOLT to different DBMS environments, which is crucial for the successful implementation of transformation testing.

[0104] Experimental setup: Four state-of-the-art MOLTs are extended with QTRAN, namely QTRAN+NoREC, QTRAN+TLP, QTRAN+Pinolo and QTRAN+DQE. These extended MOLTs are used to generate SQL statement pairs as converted raw query SQL statements. This application only focuses on databases not supported by existing MOLTs. In the experiment, if the successfully converted raw query SQL statements can maintain syntactic and semantic correctness when executed on the target DBMS and conform to the predefined metamorphosis relation, they are considered to be valid query statement pairs, which are called valid query statement pairs. For raw query SQL statement pairs that fail in the conversion phase, they are immediately discarded and the subsequent mutation phase is skipped. This strategy mainly affects the efficiency of logical error detection, but does not produce false positives. In each evaluation, QTRAN+MOLT (e.g., QTRAN+NoREC) is run on each target DBMS for 5 hours. For query statement pairs that do not conform to the metamorphosis relation, they are manually checked to verify whether they are potential logical errors.

[0105] Results: Table 5 shows the overall results of QTRAN in converting SQL query pairs to unsupported DBMS under extended MOLT. Overall, the proportion of valid query pairs in all benchmarks is high, and most of them exceed 99%, which shows that QTRAN is effective in supporting metamorphic testing through efficient SQL conversion. In particular, QTRAN+DQE successfully converted SQL query pairs on multiple DBMSs (including PostgreSQL, MonetDB, DuckDB, and ClickHouse), and the ratio of valid query pairs reached 100%. The consistent results obtained on different benchmarks and DBMSs show that QTRAN is able to generate valid SQL query pairs for metamorphic testing. For those SQL pairs that fail to satisfy the metamorphic relation, false positives are excluded by manually analyzing the potential logical errors. Figure 6 (a) shows the results of this analysis, indicating that the false positive rate remains low for all benchmarks. The true positive rate for all benchmarks exceeds 70%, with QTRAN+TLP reaching a high of 91%. Among the benchmarks, QTRAN+Pinolo has a slightly higher number of false positives, with 11 cases. This slightly higher rate may be due to the model occasionally experiencing slight hallucinations when handling more complex nested SQL queries.

[0106] Table 5 Performance of QTRAN extended baseline on unsupported DBMS

[0107]

[0108] Figure 6 (b) shows the average time allocation for the conversion phase and the mutation phase for each query SQL statement pair in the four benchmarks. On average, it takes about 6.4 seconds for QTRAN to generate a query SQL statement pair. In particular, QTRAN+Pinolo has a higher time allocation in the conversion phase, with an average of 4.78 seconds for each query SQL statement pair, which indicates that more complex query conversions require more time. In contrast, QTRAN+NoREC, QTRAN+TLP and QTRAN+DQE have a more balanced time allocation between the conversion phase and the mutation phase, with the conversion phase times being 2.98 seconds, 2.87 seconds and 2.57 seconds, respectively. In summary, the method proposed in the embodiments of the present application can efficiently generate valid query SQL statement pairs on different DBMSs in a relatively short time, thereby supporting effective mutation testing.

[0109] Conclusion of Q1: QTRAN can efficiently and accurately convert query SQL statement pairs in different DBMS environments, proving that it has practical application value in extending MOLT to support mutation testing of diverse database systems.

[0110] In Q2, we aim to check whether QTRAN can detect actual errors in DBMS that are not supported by existing MOLT, which also serves as an important indicator for evaluating the effectiveness of QTRAN.

[0111] Experimental setup: This application uses the same experimental setup as Q1. QTRAN+MOLT is run on each target DBMS for 5 hours. Potential errors are manually checked to exclude false positives and duplicate errors, and then the error reports are submitted to the developers.

[0112] Results: As shown in Table 6, a total of 24 logical errors were reported in different DBMSs. Among them, 16 have been confirmed by developers to be errors, and 6 were identified as non-errors. Specifically, 9 errors were confirmed in MariaDB and 3 errors were confirmed in TiDB. In addition, there was 1 confirmed error each in MySQL, PostgreSQL, DuckDB, and ClickHouse. In addition, there are 2 errors in ClickHouse waiting for developer confirmation. Real positive errors will continue to be recorded and error reports will be submitted to developers, waiting for confirmation and feedback, which usually takes a considerable amount of time. In order to track the status of reported errors, the error list is published in a public GitHub repository.

[0113] Table 6 Summary of logical errors found by QTRAN

[0114]

[0115] Conclusion of Q2: QTRAN detected a total of 24 logic errors across different DBMSs, 16 of which have been confirmed by the developers, indicating the effectiveness of QTRAN in revealing actual errors in DBMSs.

[0116] In Q3, we aim to explore whether dialect mapping can help successfully translate SQL statements.

[0117] Experimental setup: In order to understand the advantages of dialect mapping in the conversion stage, this application implements QTRAN- without dialect mapping. 100 original query SQL statements are collected from each of the four state-of-the-art MOLTs, denoted as QTRAN+NoREC, QTRAN+TLP, QTRAN+Pinolo, and QTRAN+DQE, and conversion experiments are conducted on DBMSs not supported by these baselines. The converted original query SQL statements are evaluated from two aspects: syntactic correctness and semantic correctness.

[0118] Results: As shown in Table 7, the semantic correctness ratios of QTRAN on various DBMS are as follows: MySQL is 0.80, MariaDB is 0.81, TiDB is 0.79, PostgreSQL is improved to 0.74, SQLite is 0.75, MonetDB is 0.56, DuckDB is 0.50, and ClickHouse is 0.53. Compared with the baseline QTRAN- without dialect mapping, these values ​​increase by +23%, +42%, +22%, +72%, +34%, +133%, +56%, and +141%, respectively. These results show that the addition of dialect mapping significantly improves the semantic correctness of SQL translation, highlighting the effectiveness and practicality of dialect mapping in improving the quality of LLM translation.

[0119] Table 7 Syntax correctness and semantic correctness under two configurations on different DBMS

[0120]

[0121] Conclusion of Q3: Dialect mapping significantly improves the semantic correctness of SQL translation across different DBMSs by adjusting SQL query statements to conform to the specific syntax and functional behavior of the target database.

[0122] In Q4, the aim is to explore whether fine-tuning can help mutate the original query SQL statements of the target database management system to support metamorphic testing.

[0123] Experimental setup: In order to understand the advantages of fine-tuning in the mutation stage, QTRAN- without using a pre-fine-tuned model was implemented. To ensure fairness, QTRAN- also used the same base model as QTRAN - GPT-4o-mini. 100 successfully converted original query SQL statements of the target database management system were selected from each of the four baselines, namely QTRAN+NoREC, QTRAN+TLP, QTRAN+Pinolo, and QTRAN+DQE. Then, these queries were mutated using QTRAN- and QTRAN. The same evaluation method as Q1 was also used. Valid SQL statement pairs must ensure syntactic and semantic correctness and satisfy the predefined metamorphosis relation. Manual inspection was performed on statement pairs that did not meet the metamorphosis relation.

[0124] Results: Table 8 shows the percentage of valid SQL statement pairs for QTRAN- and QTRAN in four baselines. QTRAN-, using the un-fine-tuned GPT-4o-mini model, fails to achieve 50% valid statement pair rate in all baselines. In contrast, benefiting from the fine-tuned mutation model, QTRAN shows significant improvements: the valid statement pair rate of QTRAN+NoREC is improved to 0.98 (+206%), QTRAN+TLP reaches 1.00 (+376%), QTRAN+Pinolo is 0.99 (+120%), and QTRAN+DQE is 1.00 (+156%). These improvements indicate that the fine-tuned mutation model can help QTRAN perform metamorphic testing effectively. By fine-tuning a small number of SQL statement pairs in the existing MOLT, the mutation model is able to better understand the metamorphic mechanism, thereby significantly improving its performance in performing SQL statement mutation in different DBMS environments.

[0125] Table 8 Comparison of the percentage of valid sentence pairs between QTRAN and QTRAN in four baselines

[0126] Baseline QTRAN+NoREC QTRAN+TLP QTRAN+Pinolo QTRAN+DQE QTRAN- 0.32 0.21 0.45 0.39 QTRAN 0.98(+206％) 1.00(+376％) 0.99(+120％) 1.00(+156％)

[0127] Conclusion of Q4: Fine-tuning facilitates metamorphic testing by helping the mutation model better understand the metamorphic mechanism and ensuring that accurate metamorphic relationships are maintained.

[0128] True positive errors: Figure 7 (b) shows a logic error identified by QTRAN in DuckDB, which is triggered by converting the "DATE" function in SQLite to the "CURRENT_DATE" function in DuckDB. The original query SQL statement pair comes from the NoREC oracle, and the error is detected by moving the predicate from the "WHERE" clause to the "SUM" function, thereby ensuring the semantic consistency between the original query SQL statement and the mutated query SQL statement. However, after QTRAN conversion, the results of the original query SQL statement and the mutated query SQL statement in DuckDB deviate, indicating a logic error.

[0129] Figure 7 (c) shows the result of directly using GPT-4o-mini to convert the original query SQL statement, which results in the conversion result not running smoothly. Due to LLM's lack of dialect-specific knowledge, the "DATE" function of SQLite is incorrectly translated into the "CURRENT_DATE" function in DuckDB, resulting in the error message "Scalar Function with name datedoes not exist! (There is no scalar function named date!)". This highlights the advantages of QTRAN in translating SQL statement pairs and can better understand dialect differences. In the list of logical errors found in this application, 18 of the 24 errors cannot be found by the method of directly using GPT-4o-mini for raw query conversion, which further highlights QTRAN's contribution to improving LLM SQL conversion capabilities and enhancing DBMS reliability.

[0130] False Positive Errors: Figure 8A representative error classified as non-error is listed, which was found in TiDB. The error is marked as MariaDB BUG#30299 and is confirmed to be a bug in MariaDB, but it is not considered as a bug by TiDB developers. This inconsistency stems from the differences between MariaDB and TiDB in processing SQL queries, especially in implicit type conversion and conditional expression parsing. Specifically, the condition "≥ANY" in sql2 is more relaxed than the condition "=ANY" in sql1. In theory, the result of sql1 should be a subset of the result of sql2, which would indicate an error due to inconsistent execution results. However, TiDB developers stated that "there is a problem with the form of the query"; this inconsistency is due to the use of different implicit type conversions in the execution plan for =ANY and ≥ANY. The developer also pointed out: "This is an uncommon syntax and is not recommended." In addition, 6 instances classified as "non-errors" were found in the error list of this application. These classifications are mainly due to the fact that the existing MOLT uses uncommon syntax designed to trigger edge case errors, and some DBMS developers choose not to fix these problems.

[0131] Impact of SQL length: This application also refines the impact of SQL length on translation efficiency through experimental analysis. The SQL statements used in the experiment are divided into four ranges according to their length. As shown in Table 9, the translation of SQL statements performs best at shorter token lengths. Specifically, SQL statements with less than 100 tokens achieve the highest grammatical and semantic correctness. For statements with more than 200 tokens, the semantic correctness begins to drop significantly. Therefore, it is recommended that for MOLTs like Pinolo, which generate longer SQL statements, using simplification tools such as SQLess can greatly improve translation efficiency.

[0132] Table 9 Impact of SQL length on SQL translation

[0133]

[0134] Validity threats: In the above implementation and evaluation process, the present application completely relies on the GPT-4o-mini model in the conversion and mutation stages, which may pose a threat to external validity due to model-specific behaviors. Another potential threat is related to the evaluation method of effective mutations in mutation testing. Although the results of the original query SQL statement and the mutated query SQL statement may conform to the mutation relationship, there is still some possibility that these mutations do not actually meet the mutation criteria. However, the observations of the present application show that this situation is extremely rare and will not significantly affect the ability of the present application to effectively test the DBMS. Despite these potential threats, the present application has taken measures to mitigate them and believes that the research results of the present application provide valuable insights for extending MOLT with LLM for logical error detection.

[0135] Further references Fig. 9 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a database test case expansion device based on a large language model, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0136] The embodiment of the present application provides a database test case expansion device based on a large language model, comprising:

[0137] A knowledge base building module 1 is configured to build a feature knowledge base corresponding to each database management system, and the feature knowledge base stores all dialect features, descriptions and examples in the corresponding database management system;

[0138] Retrieval module 2 is configured to obtain the original query SQL statement of the initial database management system, identify the original dialect features in the original query SQL statement of the initial database management system through an error recovery mechanism, perform dialect mapping on the original dialect features in a feature knowledge base corresponding to the target database management system, and obtain the target dialect features and their corresponding descriptions and examples;

[0139] The conversion module 3 is configured to construct a conversion model based on the pre-trained large language model; input the original query SQL statement into the conversion model, construct a first prompt word using the original database management system, the target database management system, the original query SQL statement, the target dialect features and their corresponding descriptions and examples, and guide the conversion model to generate the original query SQL statement of the target database management system through the first prompt word;

[0140] Mutation module 4 is configured to build a mutation model based on a pre-trained large language model, fine-tune the mutation model for each mutation strategy, obtain a fine-tuned mutation model corresponding to each mutation strategy, input the original query SQL statement of the target database management system into the fine-tuned mutation model corresponding to the selected mutation strategy, generate the corresponding mutated query SQL statement of the target database management system, and use the original query SQL statement and the mutated query SQL statement of the target database management system as test cases for metamorphosis testing.

[0141] Fig.10 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Fig.10 As shown, the electronic device of this embodiment includes: a processor 1001 and a memory 1002; wherein the memory 1002 is used to store computer-executable instructions; the processor 1001 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0142] Optionally, the memory 1002 may be independent or integrated with the processor 1001 .

[0143] When the memory 1002 is independently provided, the electronic device further includes a bus 1003 for connecting the memory 1002 and the processor 1001 .

[0144] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 1001 executes the computer execution instructions, the above method is implemented.

[0145] An embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 1001, the above method is implemented.

[0146] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0147] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0148] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit formed by the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0149] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 1001 to perform some steps of the methods of various embodiments of the present application.

[0150] It should be understood that the processor 1001 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 1001 may be any conventional processor 1001, etc. The steps of the method disclosed in the invention may be directly embodied in the hardware processor 1001 for execution, or may be executed by a combination of hardware and software modules in the processor 1001.

[0151] The memory 1002 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0152] The bus 1003 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 1003 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 1003 in the drawings of the present application is not limited to only one bus 1003 or one type of bus 1003.

[0153] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0154] An exemplary storage medium is coupled to the processor 1001, so that the processor 1001 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 1001. The processor 1001 and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor 1001 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0155] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for expanding database test cases based on a large language model, characterized in that: The following steps are involved: Constructing a feature knowledge base corresponding to each database management system, wherein the feature knowledge base stores all dialect features, descriptions and examples in the corresponding database management system; Obtaining an original query SQL statement of an initial database management system, identifying original dialect features in the original query SQL statement of the initial database management system through an error recovery mechanism, performing dialect mapping on the original dialect features in a feature knowledge base corresponding to a target database management system, and obtaining target dialect features and their corresponding descriptions and examples; Constructing a conversion model based on a pre-trained large language model, inputting the original query SQL statement of the initial database management system into the conversion model, constructing a first prompt word using the original database management system, the target database management system, the original query SQL statement of the initial database management system, target dialect features and their corresponding descriptions and examples, and guiding the conversion model to generate the original query SQL statement of the target database management system through the first prompt word; A mutation model based on a pre-trained large language model is constructed, and the mutation model is fine-tuned for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy. The original query SQL statement of the target database management system is input into the fine-tuned mutation model corresponding to the selected mutation strategy to generate a corresponding mutated query SQL statement of the target database management system. The original query SQL statement and the mutated query SQL statement of the target database management system are used as test cases for metamorphosis testing.

2. The method for expanding database test cases based on a large language model according to claim 1, characterized in that: The dialect features in the feature knowledge base are divided into three types: data types, functions and operators, and dialect mapping is performed in the feature knowledge base based on each type of dialect features.

3. The method for expanding database test cases based on a large language model according to claim 1, characterized in that: The original dialect features in the original query SQL statements of the initial database management system are identified through the error recovery mechanism, including: The original query SQL statement of the initial database management system is disassembled into a series of lexical symbols, and the lexical symbols are passed to the standard parser. When the standard parser cannot recognize or is specific to a certain dialect symbol, an error recovery mechanism allows the standard parser to skip the dialect symbol and continue to parse the remaining symbols, and use the dialect symbol as the original dialect feature in the original query SQL statement of the initial database management system.

4. The method for expanding database test cases based on a large language model according to claim 1, characterized in that: The original dialect features are mapped to a dialect in a feature knowledge base corresponding to the target database management system to obtain target dialect features and their corresponding descriptions and examples, specifically including: It is judged by keywords whether a dialect feature similar to the original dialect feature can be retrieved in the feature knowledge base corresponding to the target database management system. If so, the dialect feature similar to the original dialect feature is used as the target dialect feature and the corresponding description and example are obtained; otherwise, the similarity between the vector embedding of the original dialect feature and the vector embedding of each dialect feature in the feature knowledge base corresponding to the target database management system is calculated respectively, and the dialect feature in the feature knowledge base corresponding to the target database management system corresponding to the highest similarity is selected as the target dialect feature and the corresponding description and example are obtained.

5. The method for expanding database test cases based on a large language model according to claim 1, characterized in that: The first prompt word includes a task description, an original query SQL statement of an initial database management system, and step-by-step instructions. The task description defines the task and provides context for the conversion model, and clarifies the original database management system and the target database management system. The original query SQL statement of the initial database management system is displayed in the form of code blocks, and the step-by-step instructions use chain thinking reasoning and acquire knowledge from the target dialect features and their corresponding descriptions and examples.

6. The method for expanding database test cases based on a large language model according to claim 1, characterized in that: The mutation model is fine-tuned for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy, specifically including: The SQL statement pairs corresponding to each mutation strategy obtained by the MOLT tool are obtained, a role-playing method is used and an answer format is determined, a second prompt word is constructed in combination with the SQL statement pairs corresponding to each mutation strategy and the explanation of the mutation strategy, and the mutation model is fine-tuned by guiding the second prompt word to obtain a fine-tuned mutation model corresponding to each mutation strategy.

7. A database test case expansion device based on a large language model, characterized in that: include: A knowledge base building module is configured to build a feature knowledge base corresponding to each database management system, wherein the feature knowledge base stores all dialect features, descriptions and examples in the corresponding database management system; A retrieval module is configured to obtain an original query SQL statement of an initial database management system, identify original dialect features in the original query SQL statement of the initial database management system through an error recovery mechanism, perform dialect mapping on the original dialect features in a feature knowledge base corresponding to a target database management system, and obtain target dialect features and their corresponding descriptions and examples; A conversion module is configured to construct a conversion model and a variation model based on a pre-trained large language model, input the original query SQL statement of the initial database management system into the conversion model, construct a first prompt word using the original database management system, the target database management system, the original query SQL statement of the initial database management system, target dialect features and their corresponding descriptions and examples, and guide the conversion model to generate the original query SQL statement of the target database management system through the first prompt word; The mutation module is configured to build a mutation model based on a pre-trained large language model, fine-tune the mutation model for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy, input the original query SQL statement of the target database management system into the fine-tuned mutation model corresponding to the selected mutation strategy, generate a corresponding mutated query SQL statement of the target database management system, and use the original query SQL statement and the mutated query SQL statement of the target database management system as test cases for metamorphosis testing.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images