A SQL statement generation method based on template reconstruction and semantic instantiation

Abstract

The disclosure provides a SQL statement generation method based on template reconstruction and semantic instantiation, which is oriented to different structure database management systems. First, input the SQL seed, generate the desemanticized structure template corpus through the SQL structure template corpus construction module; then, use the structure template corpus, and obtain the SQL skeleton through the general variation module to perform structure-level reconstruction variation on the input SQL; finally, input the SQL skeleton into the semantic instantiation module, perform semantic instantiation on the SQL skeleton in combination with the runtime mode information of the target database management system, generate the SQL test input executable by the target database management system, execute the SQL test input by the target database management system, and output the corresponding test vulnerability. In this way, the problem of improving the DBMS test coverage and vulnerability discovery efficiency while reducing manual syntax adaptation is solved.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and more specifically, to a variation-based general fuzz testing method for different database management systems. Background Technology

[0002] Database Management Systems (DBMS) are fundamental components of modern software systems. As database functionality continues to expand, DBMS implementations become increasingly complex, making them prone to defects such as crashes, memory errors, logical errors, and performance anomalies. DBMS fuzzing, by automatically generating a large number of SQL test cases and executing them on the target database, can effectively uncover hidden defects and security vulnerabilities. Existing DBMS fuzzing techniques mainly include two methods: mutation-based or syntax rule-based generation. These methods are effective at uncovering vulnerabilities on mainstream DBMSs, but they still have the following shortcomings in cross-DBMS and cross-SQL dialect scenarios: Different DBMSs exhibit significant differences in SQL dialects, with substantial implementation variations in pagination, functions, types, DDL / DML syntax, expressions, and constraints. This necessitates testers typically writing extensive syntax adaptation code for each DBMS. However, dialect adaptation code is costly to maintain and often only covers a subset of the target DBMS's syntax features, resulting in insufficient diversity in generated SQL structures and limited coverage and vulnerability discovery capabilities. While existing general-purpose fuzzing methods can reduce some dialect adaptation work, they still fall short in balancing syntax diversity and semantic validity. Some semantic instantiation methods rely on the static context of the preceding SQL statement; if the preceding statement fails to execute successfully, subsequent semantic relationships such as tables, columns, types, foreign keys, and nested query results will exhibit errors and conflicts, leading to a large amount of invalid input.

[0003] Therefore, there is an urgent need for a general DBMS fuzzing method that can generate diverse and semantically valid SQL test inputs while reducing manual syntax adaptation, thereby improving DBMS test coverage and vulnerability discovery efficiency. Summary of the Invention

[0004] The purpose of this disclosure is to provide a method for generating SQL statements based on template reconstruction and semantic instantiation, which aims to solve the problem of improving DBMS test coverage and vulnerability discovery efficiency while reducing manual syntax adaptation.

[0005] In general, this paper provides a method for generating SQL statements based on template reconstruction and semantic instantiation, targeting database management systems with different structures. First, an SQL seed is input, and a desemanticized structure template corpus is generated through an SQL structure template corpus construction module. Then, using the structure template corpus, the input SQL is structurally reconstructed and mutated through a general mutation module to obtain an SQL skeleton. Finally, the SQL skeleton is input into a semantic instantiation module, and semantic instantiation is performed on the SQL skeleton in conjunction with the runtime mode information of the target database management system to generate SQL test input that can be executed by the target database management system. When a crash, memory access exception, or assertion failure is detected in the target database management system, the SQL test input that triggers the corresponding exception is recorded, the exception type and call stack information are extracted, and a database defect detection result is generated.

[0006] The specific implementation method of the SQL structure template corpus construction module is as follows: Step 1: Collect valid SQL seeds and preprocess them, filtering out SQL statements that cannot be parsed or are not suitable as a source of structural templates; Step 2: Use a general SQL parser to parse the SQL seed and convert the statement forms in different SQL dialects into a unified abstract syntax tree representation, so that SQL statements with the same logical structure but different syntax forms can be expressed uniformly. For SQL seeds and its corresponding SQL dialect Utilizing support for dialects The general-purpose SQL parser performs the parsing and obtains the parse tree. The general-purpose SQL parser normalizes nodes expressing the same syntactic function in different dialects into a unified node type, and the normalization operation is denoted as... To obtain the unified abstract syntax tree Traversal Each node in The table name, column name, constant value, and alias are replaced with table placeholder nodes, column placeholder nodes, constant placeholder nodes, and alias placeholder nodes, respectively, forming the semantic removal operation E, and resulting in a semantic removal unified abstract syntax tree. .

[0007] For nodes ,by This indicates its unified structure, in which To standardize node types, This refers to the syntactic role of the node within its parent node. This represents the number of child nodes. to The nodes are arranged according to the original syntactic order; the parent-child relationships, syntactic roles, and order constraints between nodes are preserved during the conversion. For structures with different written forms but the same syntactic function in different SQL dialects, they are mapped to the same unified node structure by the general SQL parser; for dialect-specific nodes that can be parsed, the node type and its subtree structure output by the parser are preserved, so that they can still be used as objects for template extraction and mutation of nodes of the same type. Step 3: Traverse the unified abstract syntax tree, remove the specific semantic information, and retain the syntactic structure relationship of the SQL statement. The specific semantic information is information related to a specific database instance. Step 4: Use the complete tree and subtrees at each level in the abstract syntax tree as candidate structure templates, and perform equivalence judgment and deduplication on the candidate structure templates according to node type, parent-child relationship, child node order and structural features; The equivalence judgment includes: traversing the semantically removed unified abstract syntax tree. Extract the complete tree and all subtrees rooted at each node into a set of candidate structure templates. For any candidate structure template , with its root node Recursively generate normalized structure codes starting from the given point. :when When it is a leaf node, When r has subtrees in sequence to hour, The subtree encoding is arranged according to the syntactic role and original order of each child node within its parent node. Two templates. and The structural equivalence criterion function is ,in This is an indicator function that takes 1 if the condition is true and 0 otherwise; when When two templates have the same desemantic syntax structure, it is determined that they have the same desemantic syntax structure. To improve deduplication efficiency, [the following is done]: Calculate the structure hash value First, the nodes are grouped according to their root node type and structural hash value. Then, a recursive structural comparison is performed on templates with the same hash value to eliminate hash collisions. A representative template is retained for each equivalence class, and the frequency of its occurrence and its source dialect are recorded. Finally, a node type index is built. This enables the mutation module to retrieve templates of the same type based on the uniform node type of the node to be mutated.

[0008] Step 5: Create a node type index for the deduplicated structure template so that the subsequent mutation module can quickly retrieve templates of the same or compatible types based on the type of the node to be mutated; Step 6: Output an SQL structure template corpus, which includes multiple desemanticized SQL structure templates and corresponding node type index information.

[0009] The specific semantic information includes table name, column name, constant value, and alias.

[0010] The specific implementation method of the general mutation module is as follows: Step ①: Parse the SQL statement to be mutated into a unified abstract syntax tree and determine the candidate nodes that can participate in the mutation. Determining candidate nodes involves parsing and desemanticizing the SQL statement to be mutated to obtain a unified abstract syntax tree. ;right The set of nodes in Perform traversal and based on the nodes Unified node type Query node type index The candidate node set is determined as For the selected candidate nodes ,from Select template subtrees with the same root node type and with Obtain the original subtree for the root The node address is determined by the child node's role path originating from the root node of the subtree. ,right and Perform structural alignment: Step 2: Traverse the candidate nodes and, based on the node type of the current node, retrieve the same or compatible structure templates from the SQL structure template corpus. Step 3: Modify the nodes to be mutated in the original SQL abstract syntax tree. The corresponding subtree is denoted as The retrieved structural template is denoted as .right and Perform structural alignment and denote the set of aligned node positions as follows: For any alignment position , respectively and This represents the node corresponding to the original subtree and the structural template at that position; when there is no node at the corresponding position, it is represented by the symbol... Indicate; with Indicates the node type. The common node set, the original unique node set, and the template unique node set are represented as follows:

[0011] Step 4: For Using random variables Choose to keep Or adopt ;for Using random variables Choose to keep or delete; for Using random variables Choose to insert or discard. Reconstruct the subtree in the order of parent nodes before child nodes, ensuring the structural consistency condition that the parent node must exist when a child node is retained or inserted; replace the reconstructed subtree with the child node. This yields a new abstract syntax tree. Step 5: Reconstruct the abstract syntax tree based on the above node retention, deletion, and insertion results to generate a new SQL structure; Step 6: Perform a validity check on the reconstructed SQL structure. If the structure cannot be parsed or does not meet the basic syntax constraints, select a new structure template or fall back to the previous valid structure. If the check passes, output the desemanticized SQL skeleton.

[0012] The specific implementation method of the semantic instantiation module based on dynamic pattern information is as follows: Step 1: Connect to the target database management system. Before each round of SQL skeleton instantiation, dynamically read the database name, schema name, table name, view name, column name, column type, primary key, foreign key, unique constraint, index information, and parameter types and return types of callable functions from the current database instance through the system directory, metadata query interface, or database connection interface provided by the target database management system. Then, organize the read metadata information into a current schema information set. When tables, columns, or constraints in the target database instance change, the current schema information set is reread and updated so that the semantic instantiation process uses schema information consistent with the actual operating state of the target database. Step 2: Parse the statement structure and scope relationships in the SQL skeleton, identify semantic fragments such as main query, subquery, nested clauses, expressions, join conditions, grouping conditions, and sorting conditions, and construct a scope tree based on the inclusion relationships between these semantic fragments: in, Represents a set of semantic fragment nodes. This represents the set of direct nested relationships between semantic segments; for any semantic segment ,by This indicates the nesting depth of the semantic fragment in the scope tree, and generates an instantiation sequence in descending order of nesting depth: The instantiation sequence satisfies: This allows for the priority instantiation of the innermost semantic fragment, followed by the instantiation of the corresponding outer semantic fragments layer by layer. Step 3: For the semantic fragment to be instantiated in the instantiation sequence Determine the set of direct inner semantic fragments of this semantic fragment in the scope tree: From the current pattern information set Extracting semantic fragments The accessible tables, columns, views, types, functions, and constraint information within the scope form a set of schema candidates. Simultaneously, obtain the output patterns formed after each direct inner semantic fragment has been instantiated. And construct a set of candidate semantic objects for the current semantic fragment: in, This includes the output column name, output alias, and data type that can be referenced by the outer semantic fragment for the corresponding inner semantic fragment; when the semantic fragment When it does not contain direct inner semantic fragments, its candidate semantic object set is composed of constitute; Step 4: For the current semantic fragment Placeholder objects such as tables, columns, constants, expressions, and function parameters in the text. From the candidate semantic object set Select candidate semantic objects that satisfy visibility constraints, type compatibility constraints, reference relationship constraints, and syntactic position constraints for filling; for any candidate semantic object Establish the following decision functions respectively:

[0013] in, This is a visibility determination function used to determine candidate semantic objects. Is it a placeholder object? Output database objects or inner semantic fragments that are accessible within the scope; This is a type compatibility determination function used to determine whether the data type of the candidate semantic object is the same as the data type required by the expression containing the placeholder object, or whether it belongs to an implicitly compatible type allowed by the target database management system. This is a function for determining reference relationships, used to determine whether the dependency relationship between candidate columns and selected tables, join columns, grouping columns, function parameters, and inner and outer reference objects is valid; This is a grammatical position determination function used to determine whether the object category, number of parameters, and grammatical role of a candidate semantic object are allowed to appear in the current placeholder position.

[0014] Construct a comprehensive constraint function based on the above decision function: And create placeholder objects Available candidate set: when When this occurs, candidate semantic objects are selected from the available candidate set to populate the placeholder objects. ;when When this happens, determine the type of unmet constraint and reselect a placeholder object. A populated object with a reference relationship or type dependency relationship, or the current semantic fragment is marked as a fragment to be rolled back and corrected; Step 5: Complete the current semantic segment After instantiating each placeholder object, obtain the set of projection expressions for that semantic fragment. And determine each projection expression respectively. Output name and output data types Construct the output mode of the current semantic fragment: When a projection expression has an alias, the corresponding alias is used as the output name; when a projection expression does not have an alias, the output name is determined based on its corresponding column name, function name, or expression type; the output pattern is associated with and stored with the scope identifier of the current semantic fragment, and when the direct outer semantic fragment of the current semantic fragment is instantiated, the output pattern is incorporated into the candidate semantic object set of the outer semantic fragment, so that the outer semantic fragment can reference the columns and corresponding data types actually output by the inner semantic fragment; Step 6: Instantiate each segment step by step in a bottom-up order to generate complete SQL test input, and submit it to the target database management system for parsing or execution verification; if a semantic error occurs, reselect candidate semantic objects or roll back and correct the corresponding segment until an executable SQL test input is generated.

[0015] The schema information includes database name, table name, column name, column type, key constraints, and index information.

[0016] The semantic fragments include main queries, subqueries, nested clauses, expressions, join conditions, sorting conditions, and grouping conditions.

[0017] The placeholder objects include tables, columns, constants, expressions, and function parameters.

[0018] The technical effects to be achieved by the embodiments of the present invention are as follows: This invention separates the generation of SQL syntax structures from the selection of semantic objects in specific database instances: the former is handled by a structure template corpus and a general variation module to improve cross-dialect structure adaptation capability and SQL structure diversity; the latter is handled by a dynamic semantic instantiation module to improve the semantic correctness of the generated SQL in the target database and the actual execution probability. Attached Figure Description

[0019] The above and other objects and features of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings.

[0020] Figure 1 This is a schematic diagram illustrating the architecture of a variation-based general fuzz testing method according to an embodiment of the present disclosure; Figure 2 This is a schematic diagram illustrating a template-based generic variation flowchart according to an embodiment of the present disclosure. Detailed Implementation

[0021] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0022] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become clear upon understanding the disclosure of this application.

[0023] As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more.

[0024] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0025] In the specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, the element may be directly "on" another element, directly "connected to," or "bonded to" the other element, or one or more other elements may be present in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, no other elements may be present in between.

[0026] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0027] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.

[0028] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.

[0029] Figure 1 This is a schematic diagram illustrating an SQL statement generation method based on template reconstruction and semantic instantiation according to an embodiment of the present disclosure.

[0030] To achieve the aforementioned objectives, the present invention employs the following technical framework: Figure 1 As shown.

[0031] The technical solution of this invention comprises three core modules: an SQL structure template corpus construction module, a template-based general mutation module, and a semantic instantiation module based on dynamic pattern information. These three modules work in sequence: first, a desemanticized structure template corpus is generated from the SQL seed; then, the input SQL is structurally reconstructed and mutated using this corpus to obtain the SQL skeleton; finally, the SQL skeleton is semantically instantiated by combining the runtime pattern information of the target database management system, generating SQL test input that can be executed by the target database management system.

[0032] An SQL seed is a single SQL statement or a sequence of SQL statements with execution dependencies that can be successfully parsed by at least one database management system or SQL parser. Sources of SQL seeds include regression test sets of the target database management system, historical query logs, database benchmark test cases, public SQL corpora, valid statements from other database management systems, and manually constructed statements. Based on statement function, SQL seeds include data query language statements, data manipulation language statements, data definition language statements, transaction control statements, and access control statements, specifically one or more of SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, DROP, BEGIN, COMMIT, ROLLBACK, GRANT, and REVOKE. Based on structural characteristics, SQL seeds include single-table queries, multi-table joins, subqueries, common table expressions, set operations, window functions, aggregation grouping, sorting and pagination, constraint definitions, and index definitions. The SQL seeds undergo character encoding standardization, comment removal, statement boundary identification, unparseable statement filtering, and duplicate statement elimination. The source dialect, statement type, node size, and parsing results of the seed are recorded.

[0033] (I) SQL Structure Template Corpus Construction Module This module serves as the foundational data construction module of the system. It receives valid SQL seeds from the target database management system or other database management systems, converts SQL statements in different SQL dialects into a unified abstract syntax tree based on a general SQL parser, performs semantic removal processing and extracts structural templates from the abstract syntax tree, and outputs an SQL structural template corpus.

[0034] The specific execution steps are as follows: Step 1: Collect valid SQL seeds and preprocess them, filtering out SQL statements that cannot be parsed or are not suitable as a source of structural templates; Step 2: Use a general SQL parser to parse the SQL seed and convert the statement forms in different SQL dialects into a unified abstract syntax tree representation, so that SQL statements with the same logical structure but different syntax forms can be expressed uniformly. Step 3: Traverse the unified abstract syntax tree and remove the specific semantic information, which includes information related to a specific database instance such as table names, column names, constant values, and aliases, while preserving the syntactic structure of the SQL statement; Step 4: Use the complete tree and subtrees at each level in the abstract syntax tree as candidate structure templates, and perform equivalence judgment and deduplication on the candidate structure templates according to node type, parent-child relationship, child node order and structural features; Step 5: Create a node type index for the deduplicated structure template so that the subsequent mutation module can quickly retrieve templates of the same or compatible types based on the type of the node to be mutated; Step 6: Output an SQL structure template corpus, which includes multiple desemanticized SQL structure templates and corresponding node type index information.

[0035] (ii) Template-based general mutation module This module, as the core structure generation module of the system, receives the SQL statement to be mutated and a corpus of SQL structure templates. It performs reconstructive mutation on the SQL statement at the unified abstract syntax tree level, outputting a desemanticized SQL skeleton. This module does not rely on handwritten dialect grammar rules from a specific database management system; instead, it uses valid SQL structure templates from the corpus to guide the mutation process, thereby generating SQL skeletons with high grammatical rationality and structural diversity.

[0036] The specific execution steps are as follows: Step ①: Parse the SQL statement to be mutated into a unified abstract syntax tree and determine the candidate nodes that can participate in the mutation. Step 2: Traverse the candidate nodes and, based on the node type of the current node, retrieve the same or compatible structure templates from the SQL structure template corpus. Step 3: Align the retrieved structure template with the corresponding subtree in the current SQL abstract syntax tree to determine the nodes that are common to both, the nodes that exist only in the original subtree, and the nodes that exist only in the structure template. Step 4: For common nodes, randomly select to retain nodes from the original subtree or use nodes from the structure template; for nodes that exist only in the original subtree, randomly select to retain or delete them; for nodes that exist only in the structure template, randomly select whether to insert them into the current SQL structure. Step 5: Reconstruct the abstract syntax tree based on the above node retention, deletion, and insertion results to generate a new SQL structure; Step 6: Perform a validity check on the reconstructed SQL structure. If the structure cannot be parsed or does not meet the basic syntax constraints, select a new structure template or fall back to the previous valid structure. If the check passes, output the desemanticized SQL skeleton.

[0037] (III) Semantic Instantiation Module Based on Dynamic Pattern Information This module, serving as the system's semantic completion module, receives the SQL skeleton generated by the template-based general mutation module. Taking the target database management system as the execution object, it dynamically obtains the schema information from the target database management system, instantiates the SQL skeleton based on the scope relationship and semantic constraints of the SQL statements, and outputs the SQL test input that can be executed by the target database management system.

[0038] The specific execution steps are as follows: Step ①: Connect to the target database management system and dynamically obtain the schema information of the current database instance. The schema information includes the database name, table name, column name, column type, key constraints, index information, and other metadata information that can be used for semantic instantiation. Step 2: Parse the statement structure and scope relationships in the SQL skeleton, identify semantic fragments such as main query, subquery, nested clauses, expressions, join conditions, sorting conditions, and grouping conditions, and determine the bottom-up instantiation order; Step 3: For each fragment to be instantiated, construct a set of candidate semantic objects based on the scope of the current fragment, from the schema information of the target database management system and the output schema of the inner query that has been instantiated. Step 4: For placeholder objects such as tables, columns, constants, expressions, and function parameters in the SQL skeleton, select objects that meet the conditions from the candidate semantic object set to fill them based on visibility constraints, type compatibility constraints, reference relationship constraints, and syntax position constraints. Step 5: After completing the instantiation of the current segment, record the output mode information of the segment and use it as candidate semantic information that can be referenced when instantiating the outer segment; Step 6: Instantiate each segment step by step in a bottom-up order to generate complete SQL test input, and submit it to the target database management system for parsing or execution verification; if a semantic error occurs, reselect candidate semantic objects or roll back and correct the corresponding segment until an executable SQL test input is generated.

[0039] Through the cooperation of the three modules described above, this invention achieves a complete generation process from a valid SQL seed to an SQL structure template, from the structure template to an SQL skeleton, and then from the SQL skeleton to executable SQL test input. Specifically, the SQL structure template corpus construction module provides a syntactic structure source for general variations; the template-based general variation module is responsible for expanding the structural space of the SQL input; and the semantic instantiation module based on dynamic pattern information is responsible for improving the executability of the generated SQL in the target database management system. The combined effect of these three modules enables this invention to improve the coverage and defect detection capabilities of fuzz testing in database management systems while reducing dialect adaptation costs.

[0040] This invention proposes a cross-database SQL dialect variant fuzzing method for database management systems. This method targets database management systems with different SQL dialects, such as MySQL, PostgreSQL, SQLite, DuckDB, MariaDB, and Percona. Through three core modules—an SQL structure template corpus construction module, a template-based general mutation method module, and a semantic instantiation module based on dynamic pattern information—it generates SQL test inputs that possess both syntactic structural diversity and high semantic executability. These SQL test inputs are then submitted to the target database management system for execution to trigger defects such as crashes, memory errors, and assertion failures.

[0041] The core idea of ​​this invention is that a large number of correct SQL syntax structures are implicitly contained in the set of valid SQL statements. Therefore, it is not necessary to manually write complete syntax generation rules for each database dialect. Instead, the valid SQL seeds are first parsed into a unified abstract syntax tree to form a reusable SQL structure template corpus. During testing, the abstract syntax tree of the input SQL is reconstructed and mutated based on the template corpus to generate a variety of SQL skeletons. Then, the target database management system is connected to dynamically obtain the current database's table, column, type, constraint, index, view, function, foreign key, and nested clause output patterns, and the SQL skeleton is instantiated according to the SQL scope and semantic constraints to obtain executable SQL test input.

[0042] The working principle and process are as follows: The working principle of this invention comprises three levels. First, the structural template level: a unified abstract syntax tree represents the same or similar grammatical structures in different SQL dialects, enabling SQL statements from different database systems to form searchable and reusable structural templates. Second, the reconstruction and mutation level: based on the abstract syntax tree nodes of the SQL to be mutated and similar nodes in the template corpus, different retention, deletion, insertion, or replacement strategies are applied to common nodes, original unique nodes, and template unique nodes, thereby generating new SQL structures without relying on specific dialect handwriting rules. Third, the dynamic semantic level: after the SQL skeleton is generated, instead of directly filling semantic information with static context, the actual schema information of the target database is read during execution, and tables, columns, types, expressions, functions, and constants that satisfy constraints are selected layer by layer from inner clauses to outer clauses according to the SQL scope, making the generated statement more likely to be successfully executed by the target database.

[0043] The working process of this invention includes an offline preparation process and an online testing process. During the offline preparation process, the system collects valid SQL seeds from the target database management system or other database systems, performs unified parsing, semantic information removal, subtree template extraction, template deduplication, and node type indexing on the SQL seeds to obtain an SQL structure template corpus. During the online testing process, the system selects the SQL to be mutated from the initial seed or the input provided by the coverage-guided fuzzing framework, and calls a template-based general mutation method to generate the SQL skeleton; then, it dynamically extracts the current pattern information from the target database management system and performs bottom-up semantic instantiation of the SQL skeleton; finally, it submits the generated SQL test input to the target database management system for execution, and performs defect detection, test scheduling, or subsequent seed selection based on feedback information such as coverage, anomalies, and crashes.

[0044] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

Claims

1. A method for generating SQL statements based on template reconstruction and semantic instantiation, characterized in that, For database management systems with different structures, the following steps are taken: First, an SQL seed is input, and a desemanticized structure template corpus is generated through the SQL structure template corpus construction module. Then, using the structure template corpus, the input SQL is structurally reconstructed and mutated through a general mutation module to obtain the SQL skeleton. Finally, the SQL skeleton is input into the semantic instantiation module, and semantic instantiation is performed on the SQL skeleton in conjunction with the runtime mode information of the target database management system to generate SQL test input that can be executed by the target database management system. When a crash, memory access exception, or assertion failure is detected in the target database management system, the SQL test input that triggers the corresponding exception is recorded, the exception type and call stack information are extracted, and a database defect detection result is generated.

2. The SQL statement generation method based on template reconstruction and semantic instantiation as described in claim 1, characterized in that, The specific implementation method of the SQL structure template corpus construction module is as follows: Step 1: Collect valid SQL seeds and preprocess them, filtering out SQL statements that cannot be parsed or are not suitable as a source of structural templates; Step 2: Use a general SQL parser to parse the SQL seed and convert the statement forms in different SQL dialects into a unified abstract syntax tree representation, so that SQL statements with the same logical structure but different syntax forms can be expressed uniformly. The specific method for converting statement forms from different SQL dialects into a unified abstract syntax tree representation is as follows: For SQL seeds... and its corresponding SQL dialect Utilizing support for dialects The general-purpose SQL parser performs the parsing and obtains the parse tree. ; The general-purpose SQL parser normalizes nodes expressing the same syntactic function in different dialects into a unified node type, and the normalization operation is denoted as... To obtain the unified abstract syntax tree ; Traversal Each node in The table name, column name, constant value, and alias are replaced with table placeholder nodes, column placeholder nodes, constant placeholder nodes, and alias placeholder nodes, respectively, forming the semantic removal operation E, and resulting in a semantic removal unified abstract syntax tree. ; For nodes ,by This indicates its unified structure, in which To standardize node types, This refers to the syntactic role of the node within its parent node. This represents the number of child nodes. to The nodes are arranged according to the original syntactic order; the parent-child relationship, syntactic role, and order constraints between nodes are preserved during the conversion process. For structures with different writing forms but the same syntactic function in different SQL dialects, they are mapped to the same unified node structure by the general SQL parser; for dialect-specific nodes that can be parsed, the node type and its subtree structure output by the parser are preserved, so that they can still be used as objects for template extraction and mutation of nodes of the same type; Step 3: Traverse the unified abstract syntax tree, remove the specific semantic information, and retain the syntactic structure relationship of the SQL statement. The specific semantic information is information related to a specific database instance. Step 4: Use the complete tree and subtrees at each level in the abstract syntax tree as candidate structure templates, and perform equivalence judgment and deduplication on the candidate structure templates according to node type, parent-child relationship, child node order and structural features; Step 5: Create a node type index for the deduplicated structure template so that the subsequent mutation module can quickly retrieve templates of the same or compatible types based on the type of the node to be mutated; Step 6: Output an SQL structure template corpus, which includes multiple desemanticized SQL structure templates and corresponding node type index information.

3. The SQL statement generation method based on template reconstruction and semantic instantiation as described in claim 2, characterized in that, The specific methods for equivalence judgment and deduplication are as follows: The equivalence judgment includes: traversing the semantically removed unified abstract syntax tree. Extract the complete tree and all subtrees rooted at each node into a set of candidate structure templates. ; For any candidate structure template , with its root node Recursively generate normalized structure codes starting from the given point. :when When it is a leaf node, When r has subtrees in sequence to hour, The subtree encoding is arranged according to the syntactic role and original order of each child node in the parent node; Two templates and The structural equivalence criterion function is ,in This is an indicator function that takes 1 if the condition is true and 0 otherwise; when When two templates have the same desemantic syntax structure, it is determined that they have the same semantic de-semantic syntax structure. To improve deduplication efficiency, Calculate the structure hash value First, group the data according to the root node type and the structure hash value. Then, perform a recursive structure comparison on templates with the same hash value to eliminate hash collisions. For each equivalence class, retain a representative template and record the number of times the structure appears and the dialect from which it originates. Finally, create a node type index. This enables the mutation module to retrieve templates of the same type based on the uniform node type of the node to be mutated.

4. The SQL statement generation method based on template reconstruction and semantic instantiation as described in claim 2, characterized in that, The specific semantic information includes table name, column name, constant value, and alias.

5. The SQL statement generation method based on template reconstruction and semantic instantiation as described in claim 1, characterized in that, The specific implementation method of the general mutation module is as follows: Step ①: Parse the SQL statement to be mutated into a unified abstract syntax tree and determine the candidate nodes that can participate in the mutation. Step 2: Traverse the candidate nodes and, based on the node type of the current node, retrieve the same or compatible structure templates from the SQL structure template corpus. Step 3: Modify the nodes to be mutated in the original SQL abstract syntax tree. The corresponding subtree is denoted as The retrieved structural template is denoted as ;right and Perform structural alignment and denote the set of aligned node positions as follows: For any alignment position , respectively and This represents the node corresponding to the original subtree and the structural template at that position; when there is no node at the corresponding position, it is represented by the symbol... Indicate; with Indicates the node type; The common node set, the original unique node set, and the template unique node set are represented as follows: Step 4: For Using random variables Choose to keep Or adopt ;for Using random variables Choose to keep or delete; for Using random variables Choose to insert or discard; rebuild the subtree in the order of parent node before child node, and satisfy the structural consistency condition that the parent node must exist when a child node is retained or inserted; replace the rebuilt subtree with the child node. A new abstract syntax tree is obtained; Step 5: Reconstruct the abstract syntax tree based on the above node retention, deletion, and insertion results to generate a new SQL structure; Step 6: Perform a validity check on the reconstructed SQL structure. If the structure cannot be parsed or does not meet the basic syntax constraints, select a new structure template or fall back to the previous valid structure. If the check passes, output the desemanticized SQL skeleton.

6. The SQL statement generation method based on template reconstruction and semantic instantiation as described in claim 5, characterized in that, The specific method for parsing the SQL statement to be mutated into a unified abstract syntax tree and determining the candidate nodes that can participate in the mutation is as follows: Determining candidate nodes involves parsing and desemanticizing the SQL statement to be mutated to obtain a unified abstract syntax tree. ;right The set of nodes in Perform traversal and based on the nodes Unified node type Query node type index The candidate node set is determined as For the selected candidate nodes ,from Select template subtrees with the same root node type and with Obtain the original subtree for the root The node address is the path of the child node originating from the root node of the subtree. ,right and Perform structural alignment.

7. The SQL statement generation method based on template reconstruction and semantic instantiation as described in claim 1, characterized in that, The specific implementation method of the semantic instantiation module based on dynamic pattern information is as follows: Step 1: Connect to the target database management system. Before each round of SQL skeleton instantiation, dynamically read the database name, schema name, table name, view name, column name, column type, primary key, foreign key, unique constraint, index information, and parameter types and return types of callable functions from the current database instance through the system directory, metadata query interface, or database connection interface provided by the target database management system. Then, organize the read metadata information into a current schema information set. ; When the tables, columns or constraints in the target database instance change, the current schema information set is reread and updated so that the semantic instantiation process uses schema information consistent with the actual running state of the target database. Step 2: Parse the statement structure and scope relationships in the SQL skeleton, identify semantic fragments such as main query, subquery, nested clauses, expressions, join conditions, grouping conditions, and sorting conditions, and construct a scope tree based on the inclusion relationships between these semantic fragments: in, Represents a set of semantic fragment nodes. This represents the set of direct nested relationships between semantic segments; for any semantic segment ,by This indicates the nesting depth of the semantic fragment in the scope tree, and generates an instantiation sequence in descending order of nesting depth: The instantiation sequence satisfies: This allows for the priority instantiation of the innermost semantic fragment, followed by the instantiation of the corresponding outer semantic fragments layer by layer. Step 3: For the semantic fragment to be instantiated in the instantiation sequence Determine the set of direct inner semantic fragments of this semantic fragment in the scope tree: From the current pattern information set Extracting semantic fragments The accessible tables, columns, views, types, functions, and constraint information within the scope form a set of schema candidates. Simultaneously, obtain the output patterns formed after each direct inner semantic fragment has been instantiated. And construct a set of candidate semantic objects for the current semantic fragment: in, This includes the output column name, output alias, and data type that can be referenced by the outer semantic fragment for the corresponding inner semantic fragment; when the semantic fragment When it does not contain direct inner semantic fragments, its candidate semantic object set is composed of constitute; Step 4: For the current semantic fragment Placeholder objects such as tables, columns, constants, expressions, and function parameters in the text. From the candidate semantic object set Select candidate semantic objects that satisfy visibility constraints, type compatibility constraints, reference relationship constraints, and syntactic position constraints for filling; for any candidate semantic object Establish the following decision functions respectively: in, This is a visibility determination function used to determine candidate semantic objects. Is it a placeholder object? Output database objects or inner semantic fragments that are accessible within the scope; This is a type compatibility determination function used to determine whether the data type of the candidate semantic object is the same as the data type required by the expression containing the placeholder object, or whether it belongs to an implicitly compatible type allowed by the target database management system. This is a function for determining reference relationships, used to determine whether the dependency relationship between candidate columns and selected tables, join columns, grouping columns, function parameters, and inner and outer reference objects is valid; This is a grammatical position determination function used to determine whether the object category, number of parameters, and grammatical role of a candidate semantic object are allowed to appear in the current placeholder position; Construct a comprehensive constraint function based on the above decision function: And create placeholder objects Available candidate set: when When this occurs, candidate semantic objects are selected from the available candidate set to populate the placeholder objects. ;when When this happens, determine the type of unmet constraint and reselect a placeholder object. A populated object with a reference relationship or type dependency relationship, or the current semantic fragment is marked as a fragment to be rolled back and corrected; Step 5: Complete the current semantic segment After instantiating each placeholder object, obtain the set of projection expressions for that semantic fragment. And determine each projection expression respectively. Output name and output data types Construct the output mode of the current semantic fragment: When a projection expression has an alias, the corresponding alias is used as the output name; when a projection expression does not have an alias, the output name is determined based on its corresponding column name, function name, or expression type; the output pattern is associated with and stored with the scope identifier of the current semantic fragment, and when the direct outer semantic fragment of the current semantic fragment is instantiated, the output pattern is incorporated into the candidate semantic object set of the outer semantic fragment, so that the outer semantic fragment can reference the columns and corresponding data types actually output by the inner semantic fragment; Step 6: Instantiate each segment step by step in a bottom-up order to generate complete SQL test input, and submit it to the target database management system for parsing or execution verification; if a semantic error occurs, reselect candidate semantic objects or roll back and correct the corresponding segment until an executable SQL test input is generated.

8. The SQL statement generation method based on template reconstruction and semantic instantiation as described in claim 7, characterized in that, The schema information includes database name, table name, column name, column type, key constraints, and index information.

9. The SQL statement generation method based on template reconstruction and semantic instantiation as described in claim 7, characterized in that, The semantic fragments include main queries, subqueries, nested clauses, expressions, join conditions, sorting conditions, and grouping conditions.

10. The SQL statement generation method based on template reconstruction and semantic instantiation as described in claim 7, characterized in that, The placeholder objects include tables, columns, constants, expressions, and function parameters.

Images