Database test case extension method and device based on large language model and readable medium

Abstract

This invention discloses a method, apparatus, and readable medium for expanding database test cases based on a large language model. The method includes: constructing a feature knowledge base corresponding to each database management system; identifying original dialect features in the original query SQL statement of the initial database management system through an error recovery mechanism; mapping the original dialect features to the feature knowledge base corresponding to the target database management system to obtain target dialect features and their corresponding descriptions and examples; inputting the original query SQL statement of the initial database management system into a transformation model to construct a first prompt word; using the first prompt word to guide the transformation model to generate the original query SQL statement of the target database management system; and inputting the original query SQL statement of the target database management system into a finely tuned mutation model corresponding to the selected mutation strategy to generate the corresponding mutated query SQL statement of the target database management system, effectively improving universality and reliability.

Description

Technical Field

[0001] This invention relates to the field of database testing, and specifically to a method, apparatus, and readable medium for expanding database test cases based on a large language model. Background Technology

[0002] Database Management Systems (DBMS) are crucial in applications such as online banking and e-commerce. However, the complex code logic and diverse functionality of DBMSs make them prone to errors, especially logical errors, which often result in incorrect result sets without obvious symptoms. Structured Query Language (SQL) is a domain-specific language used to manage data within a DBMS. SQL syntax describes the syntax and semantics of this language. Typically, most DBMS products adhere to the basic features of the ANSI SQL standard to support common operations and extend support for advanced features through unique SQL dialects. However, the syntax of these SQL dialects is not compatible, as similar functionality may be implemented with different syntax variants in different DBMSs. For example, character set comparison and sorting functions are implemented as COLLATION in MySQL, while in PostgreSQL they are implemented as COLLATE FOR. Currently, there are approximately 423 popular DBMS products on the market, each differing in the features it supports, such as data types, functions, and operators.

[0003] Automated testing is a common method for detecting logical errors in DBMSs. Automated testing methods utilize Oracle to verify the correctness of DBMS results. Notably, metamorphic Oracle is a widely used approach that constructs SQL statements that are exactly or nearly equivalent to the original queries for testing. This technique detects logical errors in the DBMS by comparing the execution results of these modified queries with those of the original queries. Unlike traditional fuzz testing, metamorphic testing requires ensuring syntactic and semantic correctness and strictly maintaining the metamorphic relationships between queries. This method is called MOLT (Metamorphic-Oracle based Logical Bug Detection Technique).

[0004] To generate SQL statement pairs suitable for MOLT, including original and mutated queries, a common approach is to model the SQL syntax as an Abstract Syntax Tree (AST). This AST model forms the basis for generating the original queries, ensuring syntactic correctness according to the target DBMS's specific syntax. These statements are then mutated according to a specific pattern, ensuring that the mutated queries strictly adhere to the predefined metamorphic relationships between them and the original queries. For example, Pinolo has customized its parser for MySQL syntax. It randomly generates an original query based on the AST, such as "SELECT (COLLATION(`f`)) from t", and then mutates it according to a specific pattern to "SELECT DISTINCT (COLLATION(`f`)) from t", ensuring that the results of the original query include the results of the mutated query.

[0005] Currently, state-of-the-art MOLT methods primarily rely on the syntax of specific DBMSs to generate valid SQL statement pairs. However, this approach requires significant human effort, limiting existing MOLT tools to supporting only a few popular DBMSs, such as PostgreSQL, MySQL, and MariaDB. For example, SQLRight requires an additional 93,671 lines of code to integrate PostgreSQL's syntax. Furthermore, there are currently 423 different DBMS products on the market, most of which have unique dialects and syntaxes, making it difficult for existing MOLT tools to cover testing with these DBMSs. Therefore, there is an urgent need for a method that can easily extend existing MOLT to diverse database environments with minimal effort.

[0006] In recent years, the application of software engineering based on Large Language Models (LLMs) has increased rapidly, demonstrating LLM's superior performance in SQL generation. Based on this, a potential solution is to treat LLM as an expert in DBMS SQL translation, guiding it with appropriate prompts to directly translate existing SQL statement pairs into a form that conforms to the target database's syntax while maintaining the transformation relationship. This can effectively reduce the additional workload required when expanding database testing tools. However, using LLM to directly translate SQL statement pairs between different DBMSs faces three major challenges:

[0007] 1. Limited understanding of dialect differences between DBMSs. LLMs lack a deep understanding of these dialect differences, which can lead to numerous syntactic and semantic errors, potentially resulting in semantically inequivalent queries. For example, when translating the MySQL query "SELECT (COLLATION(`f`)) from t" into PostgreSQL, an LLM lacking domain knowledge might incorrectly simplify it to "SELECT f from t," omitting the dialect-specific functions used for character set comparison and sorting. This results in a functionally inequivalent conversion because the LLM fails to recognize that in PostgreSQL, character set comparison and sorting can be achieved using the COLLATE FOR function. To improve the translation of semantically equivalent queries, enhancing the LLM's understanding of DBMS-specific dialects is essential.

[0008] 2. Limited understanding of metamorphosis mechanisms. Secondly, LLM's limited understanding of metamorphosis mechanisms may lead to the resulting SQL statements not meeting the expected metamorphic relationships, thus affecting the efficiency of logical error detection. For example, ... Figure 1 As shown, when both the original MySQL query and the mutated query are translated into PostgreSQL, the mutated query loses the "DISTINCT" keyword, making it impossible to maintain the predefined approximation relationship between the original and mutated queries. Without this relationship, the ability to detect errors by comparing query results is lost, severely weakening the effectiveness of metamorphosis testing in finding logical errors.

[0009] 3. The dependence on syntax limits MOLT's adaptability. Specifically, whether it's the generative model, the SQL parser, or the metamorphic mutant, they all heavily rely on the DBMS's syntax. Adapting to a new DBMS requires a significant amount of additional code, which is both time-consuming and labor-intensive. Table 1 shows the number of additional lines of code required by SQLancer, SQLRight, Pinolo, and DQE when adapting to SQLite, PostgreSQL, MySQL, and MariaDB. It is evident that due to their dependence on syntax, these tools all require significant code extensions to adapt to different DBMSs. For example, SQLRight uses over 90,000 lines of code to build a generative model for PostgreSQL, while Pinolo simplifies its implementation by supporting only a common syntax shared between MySQL and MariaDB.

[0010] Table 1. Lines of code for SQLancer, SQRight, Pinolo, and DQE to adapt SQLite, PostgreSQL, MySQL, and MariaDB.

[0011]

[0012] More importantly, there are 423 different DBMSs on the market, most of which use different syntaxes. Existing MOLT primarily targets well-known DBMSs like MySQL, PostgreSQL, and MariaDB. Adapting MOLT to these different DBMSs would be too time-consuming and costly, making it economically impractical. Therefore, a technology is urgently needed that can easily extend MOLT to multiple DBMS environments. Summary of the Invention

[0013] The purpose of this application is to provide a method, apparatus, and readable medium for extending database test cases based on a large language model to address the aforementioned technical problems.

[0014] In a first aspect, the present invention provides a method for extending database test cases based on a large language model, comprising the following steps:

[0015] Construct a feature knowledge base for each database management system. The feature knowledge base stores all dialect features, descriptions, and examples in the corresponding database management system.

[0016] Obtain the original query SQL statement of the initial database management system, identify the original dialect features in the original query SQL statement of the initial database management system through the error recovery mechanism, and perform dialect mapping on the original dialect features in the feature knowledge base corresponding to the target database management system to obtain the target dialect features and their corresponding descriptions and examples.

[0017] A conversion model based on a pre-trained large language model is constructed. The original query SQL statement of the initial database management system is input into the conversion model. The first prompt word is constructed using the original query SQL statements of the initial database management system and the target database management system, the target dialect features and their corresponding descriptions and examples. The first prompt word guides the conversion model to generate the original query SQL statement of the target database management system.

[0018] A mutation model based on a pre-trained large language model is constructed. The mutation model is fine-tuned for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy. The original query SQL statement of the target database management system is input into the fine-tuned mutation model corresponding to the selected mutation strategy to generate the corresponding mutated query SQL statement of the target database management system. The original query SQL statement and the mutated query SQL statement of the target database management system are used as test cases for metamorphosis testing.

[0019] As a preferred approach, dialect features in the feature knowledge base are divided into three types: data types, functions, and operators. Dialect mapping is performed in the feature knowledge base based on each type of dialect feature.

[0020] Preferably, the original dialect features in the original query SQL statements of the initial database management system are identified through an error recovery mechanism, specifically including:

[0021] The initial database management system's original query SQL statement is decomposed into a series of lexical symbols, which are then passed to the standard parser. When the standard parser cannot recognize or is specific to a certain dialect symbol, the error recovery mechanism allows the standard parser to skip the dialect symbol and continue parsing the remaining symbols, using the dialect symbol as the original dialect feature in the initial database management system's original query SQL statement.

[0022] Preferably, the original dialect features are mapped to the feature knowledge base corresponding to the target database management system to obtain the target dialect features and their corresponding descriptions and examples, specifically including:

[0023] The system determines whether a dialect feature similar to the original dialect feature can be retrieved in the feature knowledge base corresponding to the target database management system using keywords. If so, the dialect feature similar to the original dialect feature is used as the target dialect feature, and the corresponding description and example are obtained. Otherwise, the similarity between the vector embedding of the original dialect feature and the vector embedding of each dialect feature in the feature knowledge base corresponding to the target database management system is calculated, and the dialect feature with the highest similarity is selected as the target dialect feature, and the corresponding description and example are obtained.

[0024] As a preferred approach, the first prompt includes a task description, the initial query SQL statement of the initial database management system, and step-by-step instructions. The task description defines the task and provides context for the transformation model, clearly distinguishing between the initial and target database management systems. The initial query SQL statement of the initial database management system is presented in the form of code blocks, and the step-by-step instructions use chain-like reasoning and acquire knowledge from the target dialect features and their corresponding descriptions and examples.

[0025] Preferably, the mutation model is fine-tuned for each mutation strategy to obtain the fine-tuned mutation model corresponding to each mutation strategy, specifically including:

[0026] Obtain the SQL statement pairs corresponding to each mutation strategy obtained through the MOLT tool, adopt a role-playing approach and determine the response format, construct a second cue word by combining the SQL statement pairs corresponding to each mutation strategy and the explanation of the mutation strategy, and use the second cue word to guide the fine-tuning of the mutation model to obtain the fine-tuned mutation model corresponding to each mutation strategy.

[0027] Secondly, the present invention provides a database test case extension device based on a large language model, comprising:

[0028] The knowledge base building module is configured to build a feature knowledge base for each database management system. The feature knowledge base stores all dialect features, descriptions and examples in the corresponding database management system.

[0029] The retrieval module is configured to obtain the original query SQL statement of the initial database management system, identify the original dialect features in the original query SQL statement of the initial database management system through the error recovery mechanism, and perform dialect mapping on the original dialect features in the feature knowledge base corresponding to the target database management system to obtain the target dialect features and their corresponding descriptions and examples.

[0030] The conversion module is configured to build a conversion model based on a pre-trained large language model. The original query SQL statement of the initial database management system is input into the conversion model. The first prompt word is constructed using the original query SQL statements of the initial database management system and the target database management system, the target dialect features and their corresponding descriptions and examples. The first prompt word guides the conversion model to generate the original query SQL statement of the target database management system.

[0031] The mutation module is configured to build a mutation model based on a pre-trained large language model. It fine-tunes the mutation model for each mutation strategy to obtain a fine-tuned mutation model for each mutation strategy. The original query SQL statement of the target database management system is input into the fine-tuned mutation model corresponding to the selected mutation strategy to generate the corresponding mutated query SQL statement of the target database management system. The original query SQL statement and the mutated query SQL statement of the target database management system are used as test cases for metamorphosis testing.

[0032] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0033] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0034] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) The database test case extension method based on large language model proposed in this invention can automatically extend the existing MOLT technology to different DBMS. Its core idea is to use LLM to translate the SQL statements generated in the existing MOLT into a form suitable for different DBMS, thereby realizing the direct extension of the existing MOLT technology to the new DBMS for metamorphic testing.

[0037] (2) The database test case extension method based on the large language model proposed in this invention includes two stages: a conversion stage and a mutation stage. The conversion stage aims to identify potential dialect differences and improve query retrieval capabilities by utilizing information in SQL documents, thereby enabling the conversion model to accurately translate the original query SQL statements into different DBMSs. In the mutation stage, SQL statement pairs are collected from existing MOLTs, and these data are used to fine-tune the mutation model based on the pre-trained large language model to make it specifically adapted to the mutation task. Then, the fine-tuned mutation model is used to mutate the original query SQL statements of the target database management system, maintaining the necessary metamorphic relationships for metamorphic testing, thus overcoming the limitations of LLM in understanding dialect differences and metamorphic mechanisms.

[0038] (3) The generality of the database test case extension method based on the large language model proposed in this invention can be significantly improved, and the reliability of the database management system can be improved. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A failed example diagram of using LLM to directly transform SQL statement pairs in Pinolo;

[0041] Figure 2 This is a flowchart illustrating the database test case expansion method based on a large language model, as an embodiment of this application.

[0042] Figure 3 Example diagram of the database test case expansion method based on a large language model, as an embodiment of this application;

[0043] Figure 4This is a schematic diagram illustrating the workflow of the two stages of the database test case extension method based on a large language model, namely the conversion stage and the mutation stage, as shown in the embodiments of this application.

[0044] Figure 5 This is a template diagram of the first prompt word in the conversion stage of the database test case expansion method based on a large language model, as described in an embodiment of this application.

[0045] Figure 6 The figure shows the conversion efficiency and false positive analysis results of the database test case expansion method based on a large language model according to an embodiment of this application; wherein, Figure 6 (a) is a chart showing the results of manual analysis of SQL statement pairs whose query results do not meet the expected metamorphosis relationship (a true positive indicates that the abnormality of the execution result of the SQL statement pair is caused by a logical error, and the opposite is a false positive). Figure 6 (b) A graph showing the time cost distribution for each baseline procedure during the conversion and mutation phases;

[0046] Figure 7 The image shows an example execution result of the database test case expansion method based on a large language model, as described in an embodiment of this application; wherein, Figure 7 (a) shows a pair of SQL statements that satisfy equivalence relations generated in SQLite by the NoREC tool; Figure 7 (b) shows a result graph of the target SQL statement pair converted from the method of the embodiment into DuckDB, through which an inconsistent result was detected from DuckDB; Figure 7 (c) A graph showing the result of transforming the target SQL statement pair directly using the unprocessed large language model instead of the method described in the example;

[0047] Figure 8 The result diagram of the logical error report caused by implicit type conversion found by the database test case extension method based on the large language model in the embodiment of this application;

[0048] Figure 9 This is a schematic diagram of a database test case expansion device based on a large language model, as an embodiment of this application.

[0049] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0051] Figure 2 The present application illustrates an embodiment of a database test case expansion method based on a large language model, comprising the following steps:

[0052] S1. Construct a feature knowledge base for each database management system. The feature knowledge base stores all dialect features, descriptions, and examples in the corresponding database management system.

[0053] In a specific embodiment, the dialect features in the feature knowledge base are divided into three types: data type, function, and operator. Dialect mapping is performed in the feature knowledge base based on each type of dialect feature.

[0054] For details, please refer to Figure 3 This application proposes a Database Test Case Extension Method Based on Large Language Models (QTRAN) to automatically extend existing MOLTs to various database management systems (DBMS). Inspired by the DBMS metamorphosis testing process, this application divides the extension process into two phases: a transformation phase and a mutation phase. QTRAN starts with the SQL statement pairs of existing MOLTs and extends these statement pairs to the new DBMS through these two phases.

[0055] Specifically, to address the problem of Large Language Models (LLMs) lacking a deep understanding of the dialects of database management systems, leading to grammatical and semantic errors, embodiments of this application construct a feature knowledge base from structured SQL documents to enhance the understanding of dialect feature differences within the LLMs. Features are extracted from SQL documents because these documents are highly structured and detail the SQL syntax of each database management system. Most databases provide comprehensive SQL documents, making them an ideal source for building a robust and accurate feature knowledge base.

[0056] Next, we will explain the process of building the feature knowledge base. Dialect features are categorized into three main types: data types, functions, and operators. It's important to note that these elements can often be mapped across different database management systems because they typically perform similar operations, such as string manipulation, mathematical calculations, or date processing—operations that are common requirements across all database systems. To ensure consistency and accuracy, web scraping techniques are used to extract relevant dialect features from SQL documents. Each dialect feature undergoes a data cleaning process and is standardized into three key components: feature rules, descriptions, and examples. Through this unified structure, the knowledge base provides rich, well-organized dialect-specific knowledge for large language models, improving their ability to accurately translate queries across different database management systems.

[0057] The embodiments of this application have constructed feature knowledge bases for eight popular database management systems. Table 2 provides an overview of the feature knowledge bases constructed for the eight popular DBMSs, detailing the distribution of standard and dialect features across three main categories: data types, functions, and operators. A systematic approach is used to determine whether a feature is a dialect feature. First, a simple query statement is constructed for each feature using a large language model. Then, the query statement is parsed using an ANSI standard parser. If parsing fails, it indicates that the feature does not conform to standard SQL syntax and is identified as a dialect feature. Notably, dialect feature differences in functions and data types account for a significant proportion of features in many database management systems, and these differences deviate significantly from the ANSI SQL standard. This observation underscores the importance of dialect mapping to ensure compatibility and accurate interpretation across different database management systems.

[0058] Table 2 Statistical information of the feature knowledge base

[0059]

[0060] S2, obtain the original query SQL statement of the initial database management system, identify the original dialect features in the original query SQL statement of the initial database management system through the error recovery mechanism, and perform dialect mapping on the original dialect features of the initial database management system in the feature knowledge base corresponding to the target database management system to obtain the target dialect features and their corresponding descriptions and examples.

[0061] In a specific embodiment, the original dialect features in the original query SQL statements of the initial database management system are identified through an error recovery mechanism, specifically including:

[0062] The initial database management system's original query SQL statement is decomposed into a series of lexical symbols, which are then passed to the standard parser. When the standard parser cannot recognize or is specific to a certain dialect symbol, the error recovery mechanism allows the standard parser to skip the dialect symbol and continue parsing the remaining symbols, using the dialect symbol as the original dialect feature in the initial database management system's original query SQL statement.

[0063] First, an error recovery mechanism identifies original dialect features, such as function names, in the original SQL query statements of the original DBMS. These identified dialect features are then mapped to equivalents in the target DBMS using a pre-built feature knowledge base and the aid of RAG (Rich Artificial Intelligence). This rich information, along with the original SQL query statements from the original DBMS, is input into a transformation model that converts the original SQL query statements from the original DBMS into the grammatical form of the target DBMS, ensuring accurate translation and semantic correctness. The goal of the transformation phase is to convert original SQL queries from popular DBMSs into queries that match the syntax of the target DBMS. This is achieved by identifying and mapping original dialect features in the original SQL query statements from the original DBMS to the target DBMS, and then adapting the queries to the syntax of the target DBMS using an enhanced feature knowledge base retrieved via RAG. The syntax includes both standard and dialect features.

[0064] Dialect recognition is a crucial step when transferring raw SQL queries from one DBMS to another. In this process, embodiments of this application utilize the error recovery mechanism provided by ANTLR to handle dialect symbols. Specifically, the original SQL query is first broken down into a series of lexical tokens, which are then passed sequentially to the standard parser. When the standard parser encounters unrecognizable or DBMS-specific dialect symbols, such as certain functions or data types, the error recovery mechanism allows the standard parser to skip the problematic portion and continue parsing the remaining symbols. These problematic symbols are precisely the objects of the subsequent dialect mapping. This method enables the system to identify query portions requiring specific dialect processing without prematurely terminating the parsing process.

[0065] In a specific embodiment, the original dialect features are mapped to the feature knowledge base corresponding to the target database management system to obtain the target dialect features and their corresponding descriptions and examples, specifically including:

[0066] The system determines whether a dialect feature similar to the original dialect feature can be retrieved in the feature knowledge base corresponding to the target database management system using keywords. If so, the dialect feature similar to the original dialect feature is used as the target dialect feature, and the corresponding description and example are obtained. Otherwise, the similarity between the vector embedding of the original dialect feature and the vector embedding of each dialect feature in the feature knowledge base corresponding to the target database management system is calculated, and the dialect feature with the highest similarity is selected as the target dialect feature, and the corresponding description and example are obtained.

[0067] Specifically, dialect mapping is the process of converting original dialect features in the original DBMS into corresponding target dialect features in the target DBMS through a feature knowledge base. Since dialect features (such as functions or data types) can vary significantly between different database management systems, embodiments of this application employ the RAG method to accurately perform this mapping process. First, for each original dialect feature identified in the original DBMS, a corresponding dialect feature is searched in the feature knowledge base of the target DBMS. This can be achieved through keyword-based retrieval, finding similar dialect features from the target DBMS's feature knowledge base and using them as target dialect features. If no direct match is found, a fallback to vector embedding-based similarity search is performed to retrieve semantically related dialect features and use them as target dialect features. For example, if the "COLLATION()" function in MySQL needs to be converted, a similar function, such as "COLLATE FOR()" in PostgreSQL, will be retrieved by performing a vector embedding similarity search in the target DBMS and then used as the replacement.

[0068] S3. Construct a conversion model based on a pre-trained large language model. Input the original query SQL statement of the initial database management system into the conversion model. Construct first prompt words using the original query SQL statements of the initial database management system and the target database management system, target dialect features and their corresponding descriptions and examples. Guide the conversion model to generate the original query SQL statement of the target database management system using the first prompt words.

[0069] In a specific embodiment, the first prompt includes a task description, the original query SQL statement of the initial database management system, and step-by-step instructions. The task description defines the task and provides context for the transformation model, clarifying the initial database management system and the target database management system. The original query SQL statement of the initial database management system is displayed in the form of code blocks, and the step-by-step instructions use chain-like reasoning and acquire knowledge from the target dialect features and their corresponding descriptions and examples.

[0070] Specifically, a conversion model based on a pre-trained large language model is constructed, which can use the GPT series of models. In the embodiments of this application, the pre-trained large language model adopts the GPT-4o-mini model from the GPT series as the implementation.

[0071] Once the target dialect features are identified, dialect mapping is used to enhance the retrieved knowledge, thereby assisting the LLM in completing the transformation process. The first cue words in the transformation model are highly structured, designed to guide the LLM to progressively execute accurate SQL query transformations to adapt to different DBMSs. For example... Figure 4 As shown, the first cue word contains three main components:

[0072] (1) Task description: Define the task and provide context for LLM to help it understand the purpose and requirements of the transformation.

[0073] (2) The original query SQL statement of the initial database management system: The original query SQL statement of the initial database management system is displayed in the form of code blocks so that LLM can clearly see the statement that needs to be transformed.

[0074] (3) Step-by-step instructions: Use chain-of-thought (CoT) reasoning to ensure that the semantics remain consistent during the conversion process, the column names remain equal, and it can adapt to the specific characteristics of the dialect.

[0075] By using such structured first cue words, it can be ensured that the transformation model can correctly handle syntactic differences when performing SQL statement transfer, and that the semantics of the original query SQL statement are not lost, while adapting to the specific dialect of the target DBMS.

[0076] S4. Construct a mutation model based on a pre-trained large language model. Fine-tune the mutation model for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy. Input the original query SQL statement of the target database management system into the fine-tuned mutation model corresponding to the selected mutation strategy to generate the corresponding mutated query SQL statement of the target database management system. Use the original query SQL statement and the mutated query SQL statement of the target database management system as test cases for metamorphosis testing.

[0077] In a specific embodiment, the mutation model is fine-tuned for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy, specifically including:

[0078] Obtain the SQL statement pairs corresponding to each mutation strategy obtained through the MOLT tool, adopt a role-playing approach and determine the response format, construct a second cue word by combining the SQL statement pairs corresponding to each mutation strategy and the explanation of the mutation strategy, and use the second cue word to guide the fine-tuning of the mutation model to obtain the fine-tuned mutation model corresponding to each mutation strategy.

[0079] Specifically, in the mutation phase, a mutation model based on a pre-trained large language model is first constructed. Then, SQL statement pairs from existing MOLTs are collected, and role information and explanations of mutation strategies are added to the second cue words to enrich these SQL statement pairs, thereby fine-tuning the mutation model. This fine-tuned mutation model is then used to mutate the original query SQL statements of the target database management system to generate mutated query SQL statements that satisfy the metamorphic relation, which is crucial for effectively detecting logical errors. Furthermore, embodiments of this application introduce a model validator to evaluate the effectiveness of the fine-tuned mutation model, ensuring that the mutated query SQL statements maintain their correctness and accurately reflect the expected logical transformations.

[0080] The goal of the mutation phase is to mutate the original query SQL statements from the target database management system (DBMS) in the transformation phase according to a specific mutation strategy. This is a crucial step in detecting logical errors across different DBMSs. Next, we will introduce two key components of the mutation phase: model fine-tuning and the model validator.

[0081] Model fine-tuning is a crucial process that adapts pre-trained LLMs to specific tasks by further training them on domain-specific datasets, thereby improving their performance in the target application. This process customizes the general language capabilities of the LLM to perform better in a particular domain. It typically begins with a pre-trained GPT-4o-mini model that has already learned extensive language patterns and semantics from a large-scale text corpus. The model is then fine-tuned on a smaller, domain-specific dataset, transferring its general language knowledge to mutation tasks.

[0082] The data for model fine-tuning comes from existing MOLTs, which provide SQL statement pairs. Figure 5This paper demonstrates the fine-tuning process using the "FixMDistinctL" mutation strategy. The fine-tuning process employs a role-playing format, where the LLM is prompted to act as an expert in the SQL mutation domain. The mutation model is fed SQL statement pairs from tools such as Pinolo, and the original query SQL statements of the target database management system are mutated according to mutation strategies such as "FixMDistinctL". By exposing these structured statement pairs to the mutation model, it learns predetermined logical transformations, thereby achieving the ability to generate SQL statements for different DBMSs that satisfy specified mutation strategies and metamorphic relationships. For each mutation strategy, embodiments of this application collect 20 sets of SQL statement pairs, whose effectiveness has been verified by a model validator, ensuring that the mutation model is exposed to comprehensive mutation operations during training. Notably, each MOLT may include multiple mutation strategies. For example, Pinolo includes 25 mutation strategies, such as "FixMDistinctL". To ensure the uniqueness of each mutation strategy, 500 sets (20*25) of SQL statement pairs need to be collected for fine-tuning. This refined approach ensures that each mutation strategy is carefully designed and customized for specific logical transformations.

[0083] The purpose of the model validator is to ensure the effectiveness of the mutated model after fine-tuning. The collected SQL statement pairs are divided into 80% for the training set and 20% for the validation set. During training, the model is evaluated using multiple metrics, including training loss, training label accuracy, validation loss, and validation label accuracy. These metrics are calculated on a mini-batch of data at each step and on the entire validation set at the end of each epoch. Once the training loss stabilizes and the validation label accuracy reaches a sufficiently high threshold (above 95%), the fine-tuned mutated model can be used to mutate the original query SQL statements of the target database management system, ensuring that the generated mutated query SQL statements maintain the predetermined logical transformation, thereby effectively detecting logical errors.

[0084] The effectiveness of QTRAN was comprehensively evaluated below. The purpose of the experiments was to answer the following research questions:

[0085] (1) Q1: Can QTRAN effectively and efficiently convert existing MOLTs' SQL statement pairs to different DBMS environments for metamorphosis testing?

[0086] (2) Q2: Can QTRAN detect previously undetected errors?

[0087] (3) Q3: How does dialect mapping help translate SQL statements?

[0088] (4) Q4: How does fine-tuning help with mutation operations in metamorphosis testing?

[0089] The database management systems tested included MySQL, MariaDB, TiDB, PostgreSQL, SQLite, MonetDB, DuckDB, and ClickHouse. Table 3 describes the details of these databases.

[0090] Table 3 Basic Information of the DBMS Tested

[0091]

[0092] Benchmark Comparison: In this evaluation, four state-of-the-art MOLT tools were selected for extension: NoRec, TLP, Pinolo, and DQE. The following is a detailed introduction to each tool:

[0093] (1) NoRec: This technique moves the predicate in the WHERE clause to the SELECT clause. If the result is different after this transformation, it will be identified as a logical error.

[0094] (2) TLP: This method decomposes a query into three independent queries, each with isolated predicates. If the results of these queries differ from the results of the original query, a logical error is determined to exist.

[0095] (3) Pinolo: Modify the query predicate to adjust the constraints, expecting a superset or subset of the original result, used to confirm the correctness of the logic.

[0096] (4) DQE: By changing the type of the SQL query—such as SELECT, UPDATE, and DELETE—while keeping the predicate unchanged. If the results of operations on rows are inconsistent, it is considered a potential error.

[0097] These MOLTs need to understand the SQL syntax of different DBMSs. Table 4 describes the databases supported by these MOLTs. To avoid redundant testing, these MOLTs are extended using QTRAN, and they are tested on databases that are not natively supported.

[0098] Table 4. Applicability of Existing MOLTs to Selected DBMSs

[0099]

[0100] Experimental Environment: The embodiments of this application were conducted on a server configured with a 104-core Intel(R) Xeon(R) Gold 6230R CPU @ 2.10GHz and 500GB of memory. The server ran Ubuntu 20.04 operating system and used Linux kernel version 5.4.0-135-generic. Unless otherwise specified, all experiments were conducted by calling the OpenAI API (version GPT-4o-mini-2024-07-18) and using the aforementioned first prompt words. To leverage the deterministic nature of LLM and ensure more predictable and stable results, the temperature was set to 0.0.

[0101] In Q1, the aim was to evaluate the effectiveness and efficiency of QTRAN in transforming existing SQL query pairs from MOLT to different DBMS environments. This is crucial for the successful implementation of the transformation test.

[0102] Experimental Setup: Four state-of-the-art MOLTs were extended using QTRAN: QTRAN + NoREC, QTRAN + TLP, QTRAN + Pinolo, and QTRAN + DQE. These extended MOLTs were used to generate SQL statement pairs as the original query SQL statements for transformation. This application only focuses on databases not supported by existing MOLTs. In the experiments, if the successfully transformed original query SQL statements maintained syntactic and semantic correctness when executed on the target DBMS and conformed to predefined metamorphic relations, they were considered valid query statement pairs. Original query SQL statement pairs that failed during the transformation phase were immediately discarded, skipping subsequent mutation phases. This strategy primarily affects the efficiency of logical error detection but does not produce false positives. In each evaluation, QTRAN + MOLT (e.g., QTRAN + NoREC) was run for 5 hours on each target DBMS. Query statement pairs that did not conform to metamorphic relations were manually verified to be potential logical errors.

[0103] Results: Table 5 shows the overall results of QTRAN transforming SQL query pairs to unsupported DBMSs under extended MOLT. Overall, the proportion of valid query pairs was high across all benchmarks, exceeding 99% in most cases, demonstrating QTRAN's effectiveness in supporting metamorphosis testing through efficient SQL transformation. In particular, QTRAN + DQE successfully transformed SQL query pairs on multiple DBMSs (including PostgreSQL, MonetDB, DuckDB, and ClickHouse), achieving a 100% valid query pair ratio. The consistent results across different benchmarks and DBMSs demonstrate that QTRAN can generate valid SQL query pairs for metamorphosis testing. For SQL pairs that failed to satisfy the metamorphic relation, false positives were eliminated through manual analysis of potential logical errors. Figure 6 (a) presents the results of this analysis, showing that the false positive rate remained low across all benchmarks. The true positive rate exceeded 70% for all benchmarks, with QTRAN + TLP achieving a true positive rate as high as 91%. Among these benchmarks, QTRAN + Pinolo had a slightly higher number of false positives, with 11 cases. This slightly higher rate may be due to the model occasionally exhibiting mild hallucinations when handling more complex nested SQL queries.

[0104] Table 5 Performance of QTRAN on Extending Baselines on Unsupported DBMSs

[0105]

[0106] Figure 6 (b) shows the average time allocation for the transformation and mutation phases for each query SQL statement pair across four benchmarks. On average, QTRAN takes approximately 6.4 seconds to generate a query SQL statement pair. In particular, QTRAN + Pinolo exhibits a higher time allocation for the transformation phase, averaging 4.78 seconds per query SQL statement pair, indicating that more complex query transformations require more time. In contrast, QTRAN + NoREC, QTRAN + TLP, and QTRAN + DQE show a more balanced time allocation between the transformation and mutation phases, with transformation phase times of 2.98 seconds, 2.87 seconds, and 2.57 seconds, respectively. In summary, the method proposed in the embodiments of this application can efficiently generate valid query SQL statement pairs on different DBMSs in a relatively short time, thereby supporting effective metamorphosis testing.

[0107] Q1's conclusion: QTRAN can efficiently and accurately transform query SQL statement pairs in different DBMS environments, proving its practical application value in extending MOLT to support variation testing of diverse database systems.

[0108] In Q2, the aim is to check whether QTRAN can detect actual errors in DBMSs that are not supported by existing MOLT, which is also an important metric for evaluating the effectiveness of QTRAN.

[0109] Experimental Setup: This application uses the same experimental setup as Q1. QTRAN+MOLT is run for 5 hours on each target DBMS. Potential errors are manually checked to eliminate false positives and duplicate errors, and then error reports are submitted to the developers.

[0110] Results: As shown in Table 6, a total of 24 logical errors were reported across different DBMSs. Of these, 16 have been confirmed as errors by developers, and 6 have been identified as non-errors. Specifically, 9 errors were confirmed in MariaDB and 3 in TiDB. Additionally, 1 error was confirmed each in MySQL, PostgreSQL, DuckDB, and ClickHouse. Furthermore, 2 errors in ClickHouse are awaiting developer confirmation. We will continue to record genuine positive errors and submit error reports to developers, awaiting confirmation and feedback; this process typically takes a considerable amount of time. To track the status of reported errors, the error list is published in a public GitHub repository.

[0111] Table 6 Summary of logic errors detected by QTRAN

[0112]

[0113] Q2's conclusion: QTRAN detected a total of 24 logical errors across different DBMSs, 16 of which have been confirmed by the developers, demonstrating QTRAN's effectiveness in revealing actual errors in DBMSs.

[0114] In Q3, the aim was to explore whether dialect mapping could help in the successful translation of SQL statements.

[0115] Experimental Setup: To understand the advantages of dialect mapping in the conversion stage, this application implements QTRAN− without dialect mapping. One hundred original query SQL statements were collected from each of four state-of-the-art MOLT datasets, denoted as QTRAN+NoREC, QTRAN+TLP, QTRAN+Pinolo, and QTRAN+DQE, respectively, and conversion experiments were conducted on DBMSs not supported by these baselines. The converted original query SQL statements were evaluated in terms of both syntactic and semantic correctness.

[0116] Results: As shown in Table 7, the semantic correctness ratios of QTRAN on various DBMSs are as follows: MySQL 0.80, MariaDB 0.81, TiDB 0.79, PostgreSQL 0.74, SQLite 0.75, MonetDB 0.56, DuckDB 0.50, and ClickHouse 0.53. Compared with the baseline QTRAN− without dialect mapping, these values ​​increased by +23%, +42%, +22%, +72%, +34%, +133%, +56%, and +141%, respectively. These results demonstrate that adding dialect mapping significantly improves the semantic correctness of SQL translation, highlighting the effectiveness and practicality of dialect mapping in improving the quality of LLM translation.

[0117] Table 7. Syntax and semantic accuracy under two configurations on different DBMSs.

[0118]

[0119] Q3's conclusion: Dialect mapping significantly improves the semantic accuracy of SQL translation across different DBMSs by adjusting SQL query statements to conform to the specific syntax and functional behavior of the target database.

[0120] In Q4, the aim was to explore whether fine-tuning could help mutate the target database management system's original query SQL statements to support metamorphosis testing.

[0121] Experimental Setup: To understand the advantages of fine-tuning during the mutation phase, QTRAN− was implemented without using a pre-tuned model. To ensure fairness, QTRAN− also used the same base model as QTRAN—GPT-4o-mini. One hundred original query SQL statements from target database management systems that were successfully transformed were selected from each of the four baselines: QTRAN+NoREC, QTRAN+TLP, QTRAN+Pinolo, and QTRAN+DQE. These queries were then mutated using QTRAN− and QTRAN. The same evaluation method as Q1 was also used. Valid SQL statement pairs had to ensure syntactic and semantic correctness and satisfy predefined metamorphic relations. Statement pairs that did not conform to the metamorphic relations were manually checked.

[0122] Results: Table 8 shows the percentage of valid SQL statement pairs for QTRAN− and QTRAN across the four baselines. QTRAN−, using the untuned GPT-4o-mini model, failed to achieve a 50% valid statement pair rate across all baselines. In contrast, QTRAN showed significant improvements thanks to the fine-tuned mutation model: QTRAN+NoREC achieved a valid statement pair rate of 0.98 (+206%), QTRAN+TLP reached 1.00 (+376%), QTRAN+Pinolo 0.99 (+120%), and QTRAN+DQE 1.00 (+156%). These improvements demonstrate that the fine-tuned mutation model can help QTRAN perform metamorphosis testing more effectively. By fine-tuning a small number of SQL statement pairs in the existing MOLT, the mutation model can better understand the metamorphosis mechanism, thus significantly improving its performance in SQL statement mutation across different DBMS environments.

[0123] Table 8 Comparison of the percentage of valid statement pairs between QTRAN− and QTRAN in the four baselines.

[0124]

[0125] Q4's conclusion: Fine-tuning facilitates metamorphosis testing by helping mutation models better understand metamorphosis mechanisms and ensuring that accurate metamorphic relationships are maintained.

[0126] True positive error: Figure 7 (b) demonstrates a logical error identified by QTRAN in DuckDB, triggered by the conversion of the SQLite "DATE" function to the DuckDB "CURRENT_DATE" function. The original query SQL statement pair originated from NoREC Oracle, and error detection was performed by moving the predicate from the "WHERE" clause to the "SUM" function, ensuring semantic consistency between the original and mutated query SQL statements. However, after the QTRAN conversion, the results of the original and mutated query SQL statements in DuckDB deviated, indicating the presence of a logical error.

[0127] Figure 7(c) demonstrates the result of directly using GPT-4o-mini to transform the original query SQL statement, which resulted in the transformation failing. Due to the lack of dialect-specific knowledge in LLM, the SQLite "DATE" function was incorrectly translated as the DuckDB "CURRENT_DATE" function, leading to the error message "Scalar Function with name date does not exist!". This highlights the advantage of QTRAN in translating SQL statement pairs, as it better understands dialect differences. Of the 24 logical errors found in this application, 18 could not have been found using the method of directly using GPT-4o-mini to transform the original query, further highlighting QTRAN's contribution to improving LLM SQL transformation capabilities and enhancing DBMS reliability.

[0128] False positive error: Figure 8 A representative bug classified as non-error is listed here, found in TiDB. This bug, marked MariaDB BUG #30299, is confirmed as a bug in MariaDB, but TiDB developers did not recognize it as such. This inconsistency stems from differences between MariaDB and TiDB in handling SQL queries, particularly regarding implicit type conversions and condition expression parsing. Specifically, the condition "≥ ANY" in SQL2 is more lenient than the condition "= ANY" in SQL1. Theoretically, the result of SQL1 should be a subset of the result of SQL2, indicating an error due to inconsistent execution results. However, TiDB developers stated that "the form of the query is problematic"; this inconsistency is due to the different implicit type conversions used in the execution plans for =ANY and ≥ANY. The developers also noted, "This is uncommon syntax and is not recommended." Furthermore, six other instances classified as "non-error" were found in this application's bug list. These classifications are primarily due to the use of uncommon syntax in existing MOLTs designed to trigger edge case errors, which some DBMS developers have chosen not to fix.

[0129] The Impact of SQL Length: This application also refined the impact of SQL length on translation efficiency through experimental analysis. The SQL statements used in the experiments were divided into four ranges based on length. As shown in Table 9, the translation of SQL statements performed best with shorter tag lengths. Specifically, SQL statements with fewer than 100 tags achieved the highest syntactic and semantic accuracy. For statements with more than 200 tags, semantic accuracy began to decline significantly. Therefore, it is recommended that for MOLT programs like Pinolo, which generate longer SQL statements, using simplification tools such as SQLess can greatly improve translation efficiency.

[0130] Table 9. The impact of SQL length on SQL translation

[0131]

[0132] Validity Threat: In the above implementation and evaluation process, this application relies entirely on the GPT-4o-mini model during the transformation and mutation stages, which may pose a threat to external validity due to model-specific behavior. Another potential threat relates to the evaluation method of valid mutations in mutation testing. While the results of the original query SQL statement and the mutated query SQL statement may conform to mutation relationships, there are still some possibilities that these mutations do not truly meet the mutation criteria. However, our observations indicate that this situation is extremely rare and does not significantly affect our ability to effectively test DBMSs. Despite these potential threats, we have taken steps to mitigate them and believe that our research results provide valuable insights for utilizing LLM-extended MOLT for logical error detection.

[0133] Further reference Figure 9 As an implementation of the methods shown in the above figures, this application provides an embodiment of a database test case extension device based on a large language model. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0134] This application provides a database test case extension device based on a large language model, including:

[0135] Knowledge base building module 1 is configured to build a feature knowledge base for each database management system. The feature knowledge base stores all dialect features, descriptions and examples in the corresponding database management system.

[0136] The retrieval module 2 is configured to obtain the original query SQL statement of the initial database management system, identify the original dialect features in the original query SQL statement of the initial database management system through the error recovery mechanism, and perform dialect mapping on the original dialect features in the feature knowledge base corresponding to the target database management system to obtain the target dialect features and their corresponding descriptions and examples.

[0137] The conversion module 3 is configured to build a conversion model based on a pre-trained large language model. The original query SQL statement is input into the conversion model, and the first prompt word is constructed using the original query SQL statements of the initial database management system and the target database management system, the target dialect features and their corresponding descriptions and examples. The first prompt word guides the conversion model to generate the original query SQL statement of the target database management system.

[0138] Mutation module 4 is configured to build a mutation model based on a pre-trained large language model. It fine-tunes the mutation model for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy. The original query SQL statement of the target database management system is input into the fine-tuned mutation model corresponding to the selected mutation strategy to generate the corresponding mutated query SQL statement of the target database management system. The original query SQL statement and the mutated query SQL statement of the target database management system are used as test cases for metamorphosis testing.

[0139] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 10 As shown, the electronic device of this embodiment includes a processor 1001 and a memory 1002; wherein the memory 1002 is used to store computer execution instructions; and the processor 1001 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0140] Alternatively, the memory 1002 can be either standalone or integrated with the processor 1001.

[0141] When the memory 1002 is set up independently, the electronic device also includes a bus 1003 for connecting the memory 1002 and the processor 1001.

[0142] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by the processor 1001, implement the above method.

[0143] This invention also provides a computer program product, including a computer program, which, when executed by a processor 1001, implements the above-described method.

[0144] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0145] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0146] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0147] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 1001 to execute certain steps of the methods of the various embodiments of this application.

[0148] It should be understood that the processor 1001 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 1001 can be any conventional processor 1001. The steps of the method disclosed in this invention can be directly manifested as execution by the hardware processor 1001, or execution by a combination of hardware and software modules within the processor 1001.

[0149] The memory 1002 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0150] Bus 1003 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 1003 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 1003 in the accompanying drawings of this application is not limited to only one bus 1003 or one type of bus 1003.

[0151] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0152] An exemplary storage medium is coupled to a processor 1001, enabling the processor 1001 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor 1001. The processor 1001 and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor 1001 and the storage medium can exist as discrete components in an electronic device or a main control device.

[0153] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for expanding database test cases based on a large language model, characterized in that, Includes the following steps: Construct a feature knowledge base corresponding to each database management system, wherein the feature knowledge base stores all dialect features, descriptions and examples in the corresponding database management system; The process involves obtaining the original query SQL statement from the initial database management system, identifying the original dialect features in the original query SQL statement through an error recovery mechanism, mapping the original dialect features to the feature knowledge base corresponding to the target database management system, and obtaining the target dialect features and their corresponding descriptions and examples. Specifically, this includes: The system determines whether a dialect feature similar to the original dialect feature can be retrieved in the feature knowledge base corresponding to the target database management system using keywords. If so, the dialect feature similar to the original dialect feature is selected as the target dialect feature, and its corresponding description and example are obtained. Otherwise, the similarity between the vector embedding of the original dialect feature and the vector embedding of each dialect feature in the feature knowledge base corresponding to the target database management system is calculated, and the dialect feature with the highest similarity is selected as the target dialect feature, and its corresponding description and example are obtained. A conversion model based on a pre-trained large language model is constructed. The original query SQL statement of the initial database management system is input into the conversion model. A first prompt word is constructed using the original query SQL statements of the initial and target database management systems, target dialect features, and their corresponding descriptions and examples. The first prompt word includes a task description, the original query SQL statement of the initial database management system, and step-by-step instructions. The task description defines the task and provides context for the conversion model, clearly identifying the initial and target database management systems. The original query SQL statement of the initial database management system is displayed in code blocks. The step-by-step instructions use chain-like reasoning and acquire knowledge from the target dialect features and their corresponding descriptions and examples. The first prompt word guides the conversion model to generate the original query SQL statement of the target database management system. A mutation model based on a pre-trained large language model is constructed. The mutation model is fine-tuned for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy. The original query SQL statement of the target database management system is input into the fine-tuned mutation model corresponding to the selected mutation strategy to generate the corresponding mutated query SQL statement of the target database management system. The original query SQL statement and the mutated query SQL statement of the target database management system are used as test cases for metamorphosis testing.

2. The database test case expansion method based on a large language model according to claim 1, characterized in that, The dialect features in the feature knowledge base are divided into three types: data types, functions, and operators. Dialect mapping is performed in the feature knowledge base based on each type of dialect feature.

3. The database test case expansion method based on a large language model according to claim 1, characterized in that, The error recovery mechanism identifies the original dialect features in the original query SQL statements of the initial database management system, specifically including: The original query SQL statement of the initial database management system is decomposed into a series of lexical symbols, which are then passed to the standard parser. When the standard parser cannot recognize or is specific to a certain dialect symbol, the error recovery mechanism allows the standard parser to skip the dialect symbol and continue parsing the remaining symbols, using the dialect symbol as the original dialect feature in the original query SQL statement of the initial database management system.

4. The database test case expansion method based on a large language model according to claim 1, characterized in that, For each mutation strategy, the mutation model is fine-tuned to obtain the fine-tuned mutation model corresponding to each mutation strategy, specifically including: Obtain the SQL statement pairs corresponding to each mutation strategy obtained through the MOLT tool, adopt a role-playing approach and determine the answer format, construct a second cue word by combining the SQL statement pairs corresponding to each mutation strategy and the explanation of the mutation strategy, and guide the mutation model to fine-tune the mutation model to obtain the fine-tuned mutation model corresponding to each mutation strategy.

5. A database test case extension device based on a large language model, characterized in that, include: The knowledge base building module is configured to build a feature knowledge base corresponding to each database management system, wherein the feature knowledge base stores all dialect features, descriptions and examples in the corresponding database management system. The retrieval module is configured to obtain the original query SQL statement of the initial database management system, identify the original dialect features in the original query SQL statement of the initial database management system through an error recovery mechanism, and perform dialect mapping on the original dialect features in the feature knowledge base corresponding to the target database management system to obtain the target dialect features and their corresponding descriptions and examples, specifically including: The system determines whether a dialect feature similar to the original dialect feature can be retrieved in the feature knowledge base corresponding to the target database management system using keywords. If so, the dialect feature similar to the original dialect feature is selected as the target dialect feature, and its corresponding description and example are obtained. Otherwise, the similarity between the vector embedding of the original dialect feature and the vector embedding of each dialect feature in the feature knowledge base corresponding to the target database management system is calculated, and the dialect feature with the highest similarity is selected as the target dialect feature, and its corresponding description and example are obtained. The conversion module is configured to construct a conversion model and a mutation model based on a pre-trained large language model. It inputs the original query SQL statement from the initial database management system into the conversion model. Using the original query SQL statements from both the initial and target database management systems, target dialect features, and their corresponding descriptions and examples, it constructs a first prompt word. This first prompt word includes a task description, the original query SQL statement from the initial database management system, and step-by-step instructions. The task description defines the task and provides context for the conversion model, clearly identifying the initial and target database management systems. The original query SQL statement from the initial database management system is displayed as a code block. The step-by-step instructions use chain-like reasoning and acquire knowledge from the target dialect features and their corresponding descriptions and examples. The first prompt word guides the conversion model to generate the original query SQL statement from the target database management system. The mutation module is configured to construct a mutation model based on a pre-trained large language model, fine-tune the mutation model for each mutation strategy to obtain a fine-tuned mutation model corresponding to each mutation strategy, input the original query SQL statement of the target database management system into the fine-tuned mutation model corresponding to the selected mutation strategy to generate the corresponding mutated query SQL statement of the target database management system, and use the original query SQL statement and mutated query SQL statement of the target database management system as test cases for metamorphosis testing.

6. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-4.

Images