Data processing method and device, electronic equipment, storage medium and program product

Abstract

This application provides a data processing method, apparatus, electronic device, storage medium, and program product. The method includes: determining the database type corresponding to an initial SQL statement to be converted based on a received SQL statement conversion request, wherein the SQL statement conversion request includes the initial SQL statement and the target database type of the target SQL statement to which the initial SQL statement needs to be converted; based on the database type, calling a target parser instance corresponding to the database type to parse the initial SQL statement, obtaining an abstract syntax tree corresponding to the initial SQL statement; and obtaining the target SQL statement based on the abstract syntax tree and the target database type. This method aims to improve the efficiency and accuracy of database statement conversion and reduce the maintenance cost of the parsing method.

Description

Technical Field

[0001] This application relates to the field of databases, and more particularly to a data processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] In the field of database technology, Structured Query Language (SQL) statements, as the standard language for manipulating relational databases, are widely used in scenarios such as data querying, transaction processing, and data migration. However, due to differences in the implementation of the SQL standard by different database vendors (such as MySQL, Oracle, and PostgreSQL), their respective SQL dialects have emerged. This difference is particularly prominent in scenarios such as database migration, cross-database application development, and SQL compatibility assessment. Currently, traditional SQL parsing methods suffer from low parsing efficiency, poor parsing accuracy, and high maintenance costs in multi-SQL dialect scenarios.

[0003] Therefore, improving the efficiency and accuracy of database statement conversion and reducing the maintenance cost of parsing methods are urgent problems to be solved. Summary of the Invention

[0004] This application provides data processing methods, apparatus, electronic devices, storage media, and program products to improve the efficiency and accuracy of database statement conversion and reduce the maintenance cost of parsing methods.

[0005] In a first aspect, embodiments of this application provide a data processing method, including:

[0006] Based on the received SQL statement conversion request, determine the database type corresponding to the initial SQL statement to be converted. The SQL statement conversion request includes the initial SQL statement and the target database type of the target SQL statement to which the initial SQL statement needs to be converted.

[0007] Based on the database type, the target parser instance corresponding to the database type is invoked to parse the initial SQL statement, thereby obtaining the abstract syntax tree corresponding to the initial SQL statement;

[0008] The target SQL statement is obtained based on the abstract syntax tree and the target database type.

[0009] Optionally, the step of calling a target parser instance corresponding to the database type to parse the initial SQL statement based on the database type, and obtaining the abstract syntax tree corresponding to the initial SQL statement, includes:

[0010] Call the target parser instance corresponding to the database type from the pre-built parser instance pool;

[0011] The implementation class of the target parser instance implements the abstract layer interface and calls the language parsing file corresponding to the target parser instance to parse the initial SQL statement to obtain the abstract syntax tree corresponding to the initial SQL statement. The abstract layer interface can be implemented by the implementation classes of multiple parser instances in the parser instance pool.

[0012] Optionally, the implementation class that controls the target parser instance implements the abstract layer interface and calls the language parsing file corresponding to the target parser instance to parse the initial SQL statement, obtaining the abstract syntax tree corresponding to the initial SQL statement, including:

[0013] The implementation class that controls the target parser instance implements the abstract layer interface and calls the language parsing file corresponding to the target parser instance to parse the initial SQL statement, thereby obtaining the initial syntax tree of the initial SQL statement. The initial syntax tree is the syntax tree of the initial database corresponding to the initial SQL statement.

[0014] The initial syntax tree is traversed using the visitor implementation class of the target parser instance to obtain the syntax tree feature information of the initial syntax tree, which includes at least one of the following: node information, structural information, and semantic information.

[0015] Based on the syntax tree feature information, the abstract syntax tree corresponding to the initial SQL statement is obtained.

[0016] Optional, also includes:

[0017] Based on the initial SQL statement in the SQL statement conversion request, obtain the initial statement fingerprint corresponding to the initial SQL statement;

[0018] If a target statement fingerprint that is identical to the initial statement fingerprint exists in the historical parsing results, then the parsing result of the target statement fingerprint is extracted from the historical parsing results;

[0019] Based on the parsing results, the abstract syntax tree corresponding to the initial SQL statement is obtained.

[0020] Optional, also includes:

[0021] Based on the received database type update request, update the parser instances included in the parser instance pool and the language parsing files corresponding to the parser instances. The database type update request includes at least one of adding, deleting, or modifying.

[0022] Optional, also includes:

[0023] Based on the received SQL statement comparison request, determine the first database type corresponding to the first SQL statement and the second database type corresponding to the second SQL statement in the SQL statement comparison request;

[0024] The parser instances corresponding to the first database type and the second database type are called respectively to parse the first SQL statement and the second SQL statement to obtain the first abstract syntax tree of the first SQL statement and the second abstract syntax tree of the second SQL statement;

[0025] If the similarity between the first abstract syntax tree and the second abstract syntax tree is greater than or equal to a preset similarity threshold, a prompt message is output, which indicates that the first SQL statement and the second SQL statement are semantically equivalent.

[0026] Optional, also includes:

[0027] During the process of calling the target parser instance corresponding to the database type to parse the initial SQL statement, anomaly detection is performed on the syntax and semantics of the initial SQL statement;

[0028] If at least one of the syntax and semantics is incorrect, then an error message for the initial SQL statement is generated based on the error, and the error message includes at least one of the following: error information and repair instructions.

[0029] Secondly, embodiments of this application provide a data processing apparatus, including:

[0030] The first processing module is used to determine the database type corresponding to the initial SQL statement to be converted based on the received SQL statement conversion request. The SQL statement conversion request includes the initial SQL statement and the target database type of the target SQL statement to which the initial SQL statement needs to be converted.

[0031] The control module is used to call the target parser instance corresponding to the database type to parse the initial SQL statement based on the database type, and obtain the abstract syntax tree corresponding to the initial SQL statement;

[0032] The second processing module is used to obtain the target SQL statement based on the abstract syntax tree and the target database type.

[0033] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0034] The memory stores computer-executed instructions;

[0035] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0036] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0037] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0038] The data processing method, apparatus, electronic device, storage medium, and program product provided in this application dynamically identify the source database type of the initial SQL statement when receiving an SQL statement conversion request, and automatically instantiate a dedicated parser matching the type to perform syntax parsing, generating a standardized abstract syntax tree independent of the database as an intermediate representation layer. Then, according to the target database type, the corresponding syntax reconstruction module is called to convert the abstract syntax tree into an SQL statement that conforms to the target dialect syntax rules. Compared with the prior art, this solution decouples dialect recognition and conversion logic through a dynamic parsing mechanism, which not only ensures the accuracy of source statement parsing, but also ensures the efficiency of target statement generation by utilizing the dialect adaptation capability of syntax reconstruction. Moreover, only the parser needs to be maintained, without modifying the underlying code when adding new database types, thus reducing system maintenance costs. This achieves the effect of improving the efficiency and accuracy of database statement conversion and reducing the maintenance cost of parsing methods. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0040] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0041] Figure 2 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0042] Figure 3 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0043] Figure 4 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0044] Figure 5 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0045] Figure 6 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0046] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0047] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0048] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0049] Currently, existing technologies mainly employ the following two approaches to handle multi-SQL dialect parsing:

[0050] Option 1: SQL parsing based on a single database

[0051] This approach designs a dedicated parser for a specific database (such as Oracle or MySQL). It parses SQL statements into a syntax tree structure specific to that database by writing independent lexical and parsing units (e.g., based on regular expressions or finite state machines). For example, Oracle uses the PL / SQL parser, and MySQL uses the MySQL parser. However, this approach relies heavily on deep customization of the database's syntax, resulting in high code coupling. When new databases need support, a new parser must be developed from scratch, leading to long development cycles and significant code redundancy.

[0052] Option 2: SQL Adaptation and Parsing Based on Hard-coded Rules

[0053] This solution primarily relies on manually adapting to the syntax differences between different databases through numerous conditional statements. For example, to address differences in date functions (such as Oracle's TO_DATE and MySQL's STR_TO_DATE), it's necessary to determine the database type and perform string replacement or refactoring. While this solution depends on comprehensive coverage of syntax differences, the code complexity increases exponentially with the number of database types and syntax rules, leading to high maintenance costs and susceptibility to errors.

[0054] In summary, existing technologies suffer from high code duplication, high maintenance costs, and poor scalability when dealing with parsing scenarios involving multiple SQL dialects. Adding support for new databases requires modifications to existing code. Furthermore, current solutions cannot automatically identify SQL dialects, requiring users to manually specify the database type, making it difficult to achieve semantic analysis and conversion across SQL dialects.

[0055] In view of this, this application provides a data processing method that constructs a unified SQL dialect parsing framework through modular design and dynamic expansion mechanism. Upon receiving an SQL statement conversion request, the method automatically determines the database type corresponding to the initial SQL statement to be converted. Based on the database type, it calls the target parser instance corresponding to the database type to parse the initial SQL statement, obtaining the abstract syntax tree (AST) corresponding to the initial SQL statement. Then, based on the AST and the target database type, the target SQL statement is obtained. Compared to existing technologies, this solution decouples dialect recognition and conversion logic through a dynamic parsing mechanism. This ensures the accuracy of source statement parsing while leveraging the dialect adaptation capability of syntax reconstruction to ensure the efficiency of target statement generation. Furthermore, it only requires maintenance of the parser, eliminating the need to modify the underlying code when adding new database types, thus reducing system maintenance costs. This achieves the effect of improving the efficiency and accuracy of database statement conversion and reducing the maintenance cost of the parsing method.

[0056] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0057] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 As shown, the method includes:

[0058] S101. Based on the received SQL statement conversion request, determine the database type corresponding to the initial SQL statement to be converted.

[0059] The SQL statement conversion request includes the initial SQL statement and the target database type to which the initial SQL statement needs to be converted.

[0060] In this embodiment, after receiving an SQL statement conversion request, the initial SQL statement and the target database type are extracted from the request. The extraction can be achieved by parsing the request's data packet structure, for example, by reading the corresponding content from the request's message body according to preset field identifiers. The database type corresponding to the initial SQL statement can be determined by analyzing the syntactic elements in the initial SQL statement. Specifically, syntactic elements include, but are not limited to, specific keywords, function call formats, and data type declaration methods. This embodiment employs an automatic SQL dialect recognition function, which determines the corresponding database type by analyzing the syntactic features in the initial SQL statement and matching them with preset database dialect feature libraries.

[0061] S102. Based on the database type, call the target parser instance corresponding to the database type to parse the initial SQL statement and obtain the abstract syntax tree corresponding to the initial SQL statement.

[0062] In this embodiment, after determining the database type, the corresponding target parser instance is called based on that type.

[0063] Specifically, the invocation operation can be implemented by querying a predefined parser instance mapping table, which stores the correspondence between database types and parser instances. In this embodiment, a factory pattern creation mechanism can be used, where the factory class selects the corresponding parser implementation class based on the database type in the configuration information and creates an instance. This allows for centralized management of the parser creation logic. When new database support needs to be added, only the factory class needs to be modified, without modifying the invocation code, thus improving the system's scalability.

[0064] After the corresponding parser instance is found, it is started to parse the initial SQL statement. The parsing process may include two stages: lexical analysis and syntax analysis. In this embodiment, ANTLR (Another Tool for Language Recognition) can be used as the syntax parser generation tool to define independent language parsing files for each SQL dialect, such as lexical files (Lexer) and syntax files (Parser). The language parsing files can be stored and managed uniformly in a resource directory.

[0065] During the lexical analysis phase, the initial SQL statement can be decomposed into a series of lexical units, such as keywords, identifiers, constants, and operators. During the syntax analysis phase, these lexical units are combined into a grammatical structure based on the syntax rules corresponding to the database type, ultimately generating an abstract syntax tree.

[0066] S103. Based on the abstract syntax tree and the target database type, obtain the target SQL statement.

[0067] In this embodiment, after obtaining the abstract syntax tree and the target database type, the abstract syntax tree is traversed. The traversal process can be performed, for example, using a depth-first or breadth-first approach.

[0068] In this embodiment, a unified visitor pattern can be used. Each database dialect has a corresponding Visitor implementation class, which inherits from the BaseVisitor base class generated by ANTLR. By overriding specific visit methods, different nodes in the syntax tree are accessed to extract SQL information. All Visitor implementation classes return a unified result object, ensuring consistency in the result format.

[0069] During the traversal, the nodes of the abstract syntax tree can be transformed according to the syntax rules of the target database type. Transformation operations include, but are not limited to, replacing node types, modifying node attributes, and adjusting node structures.

[0070] This embodiment provides an Intermediate Representation (IR) layer, which can convert syntax trees from different databases into a unified Abstract Syntax Tree (AST) structure. The IR layer defines unified node types and attributes, including SELECT, INSERT, UPDATE, and DELETE nodes, as well as attributes such as table names, column names, and conditional expressions.

[0071] The conversion process can transform the grammatical differences between different dialects into a unified representation in the IR layer. Furthermore, the IR layer can convert the unified AST into SQL statements for the target database, achieving cross-database SQL conversion. After completing the conversion of the abstract syntax tree, the converted abstract syntax tree can be reassembled into the target SQL statement according to the syntax rules of the target database type.

[0072] The method provided in this application dynamically identifies the source database type of the initial SQL statement when receiving an SQL statement conversion request, and automatically instantiates a dedicated parser that matches the type to perform syntax parsing, generating a standardized abstract syntax tree (AST) independent of the database as an intermediate representation layer. Then, according to the target database type, the corresponding syntax reconstruction module is called to convert the AST into an SQL statement that conforms to the target dialect syntax rules. Compared with the prior art, this solution decouples dialect recognition and conversion logic through a dynamic parsing mechanism, ensuring the accuracy of source statement parsing and ensuring the efficiency of target statement generation by utilizing the dialect adaptation capability of syntax reconstruction. Moreover, it only requires maintenance of the parser and does not require modification of the underlying code when adding new database types, thus reducing system maintenance costs. This achieves the effect of improving the efficiency and accuracy of database statement conversion and reducing the maintenance cost of the parsing method.

[0073] The following section provides a detailed explanation of how, in step S102, the target parser instance corresponding to the database type is called to parse the initial SQL statement based on the database type, thereby obtaining the abstract syntax tree corresponding to the initial SQL statement. Figure 2 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Figure 2 As shown, the aforementioned step S102 may specifically include:

[0074] S201. Call the target parser instance corresponding to the database type from the pre-built parser instance pool.

[0075] In this embodiment, the pre-built parser instance pool is a collection that stores parser instances corresponding to various database types. Invocation operations can be implemented using index queries, for example, by searching the instance pool based on the database type identifier, retrieving the corresponding parser instance, and loading it into memory.

[0076] In this embodiment, a parser instance pooling mechanism is used to maintain the parser instance pool, which can avoid frequent creation and destruction of parser objects and reduce object creation overhead.

[0077] S202. The implementation class of the target parser instance implements the abstract layer interface and calls the language parsing file corresponding to the target parser instance to parse the initial SQL statement and obtain the abstract syntax tree corresponding to the initial SQL statement.

[0078] The abstract layer interface can be implemented by the implementation classes of multiple parser instances in the parser instance pool.

[0079] In this embodiment, the implementation class of the target parser instance can be controlled to implement the abstract layer interface through an interface implementation check mechanism. For example, it can be verified whether the implementation class contains all the methods defined in the abstract layer interface; if not, a prompt or termination of the parsing process can be given.

[0080] Specifically, this embodiment defines a unified abstraction layer interface. This interface specifies the methods that all SQL parsers must implement, including parsing methods, resource release methods, and cache cleanup methods. Through this abstraction layer interface, the implementation details of parsers for different databases can be hidden, providing a unified calling interface. All concrete parser implementation classes implement this interface to ensure interface consistency.

[0081] Calling the language parsing file corresponding to the target parser instance can be achieved by looking up the file path. For example, the corresponding language parsing file can be found based on the parser instance's configuration information, and then the file can be loaded and its parsing logic executed.

[0082] One possible implementation is to use dynamic proxies to control the implementation class's implementation of the abstract interface. Specifically, a dynamic proxy class can be created that implements the abstract interface, and the corresponding methods of the target resolver instance's implementation class can be called within the proxy class's methods.

[0083] Another possible implementation involves dynamically modifying the implementation class of the target parser instance using bytecode enhancement techniques, forcing it to implement the abstract layer interface, and then performing the entire parsing process—from initial syntax tree generation and syntax tree feature extraction to abstract syntax tree transformation—based on this modified implementation class. Specifically, this implementation can be achieved through the following sub-steps:

[0084] S2021. The implementation class of the target parser instance implements the abstract layer interface and calls the language parsing file corresponding to the target parser instance to parse the initial SQL statement and obtain the initial syntax tree of the initial SQL statement.

[0085] The initial syntax tree is the syntax tree of the initial database corresponding to the initial SQL statement.

[0086] In this embodiment, when the initial SQL statement is parsed using a language parsing file, the language parsing file contains lexical rules and grammatical rules specific to the initial database type. The lexical rules define how to decompose the initial SQL statement into lexical units, and the grammatical rules define how these lexical units form a valid grammatical structure. After parsing according to these rules, an initial syntax tree is generated.

[0087] S2022. Use the visitor implementation class of the target parser instance to access and traverse the initial syntax tree to obtain the syntax tree feature information of the initial syntax tree.

[0088] The syntax tree feature information includes at least one of the following: node information, structural information, and semantic information.

[0089] In this embodiment, the visitor implementation class defines access methods for different types of nodes. When traversing the initial syntax tree, each node can be accessed sequentially according to the node hierarchy of the tree, and the corresponding access method can be called to obtain the relevant information of the node.

[0090] For example, for a node representing a query operation, you can obtain information such as the query fields and query conditions of that node; for a node representing a table join, you can obtain information such as the join type and the table being joined.

[0091] S2023. Based on the syntax tree feature information, obtain the abstract syntax tree corresponding to the initial SQL statement.

[0092] In this embodiment, when generating an abstract syntax tree based on syntax tree feature information, it can be constructed according to a preset abstract syntax tree structure template.

[0093] For example, the node information in the syntax tree feature information can be classified according to the node type of the abstract syntax tree, and then the corresponding abstract syntax tree nodes can be created according to the classification results, and the relationship between the nodes can be established according to the structure information of the syntax tree.

[0094] The method provided in this application implements an abstract layer interface by controlling the implementation class, which ensures the interface consistency of the parser instance. At the same time, it uses the visitor pattern to traverse the initial syntax tree to obtain feature information, providing a data foundation for generating the abstract syntax tree, thereby improving the flexibility and scalability of the parsing process.

[0095] Figure 3 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Figure 3 As shown, the method may further include:

[0096] S301. Obtain the initial statement fingerprint corresponding to the initial SQL statement based on the initial SQL statement in the SQL statement conversion request.

[0097] In this embodiment, the initial statement fingerprint can be obtained using a hash algorithm (such as MD5, SHA256, etc.).

[0098] Specifically, the initial SQL statement can be preprocessed. Preprocessing operations may include removing irrelevant characters such as spaces and newlines. Then, the preprocessed string is hashed, and the resulting hash value is the fingerprint of the initial statement.

[0099] In this embodiment, a parsing result caching mechanism can be adopted. The cache is established based on the fingerprint of the SQL statement. The parsing result of the same SQL statement is directly obtained from the cache to avoid duplicate parsing.

[0100] S302. If a target statement fingerprint that is identical to the initial statement fingerprint exists in the historical parsing results, then extract the parsing results of the target statement fingerprint from the historical parsing results.

[0101] In this embodiment, finding the fingerprint of a target statement in historical parsing results can be achieved by establishing a fingerprint index table. The fingerprint index table stores the mapping relationship between historical statement fingerprints and their corresponding parsing results.

[0102] During a query, the fingerprint of the initial statement can be compared with the fingerprint in the index table. If the same fingerprint is found, the corresponding parsing result is extracted.

[0103] S303. Based on the parsing results, obtain the abstract syntax tree corresponding to the initial SQL statement.

[0104] In this embodiment, the abstract syntax tree (AST) generated from the parsing results can directly utilize the AST data stored in the parsing results. For example, if the AST is stored in serialized form in the parsing results, it can be restored to an AST object through deserialization.

[0105] The method provided in this application generates a statement fingerprint and compares it with historical parsing results. If the same fingerprint exists, the parsing result is directly reused, thus avoiding the process of repeated parsing and improving the efficiency of SQL statement conversion.

[0106] In one possible implementation, the method may further include: updating the parser instances included in the parser instance pool and the language parsing files corresponding to the parser instances based on the received database type update request. The database type update request may include at least one of adding, deleting, or modifying.

[0107] Specifically, for new database types, a new parser instance can be added to the parser instance pool, and the corresponding language parsing file can be stored in the specified directory. Adding an instance can be done by calling the instance pool's add interface, while storing can be done by copying or moving files.

[0108] Optionally, this application may also include a dialect extension mechanism to support the dynamic addition of new SQL dialects, such as including pluggable dialect grammar file loading, where new dialect grammar files can be dynamically loaded as plug-ins without modifying the core code.

[0109] Optionally, this application may also include hot updates of dialect grammar rules, supporting updates of dialect grammar rules without restarting the system, and adapting to grammatical changes brought about by database version upgrades.

[0110] Optionally, this application may also include support for custom dialect syntax, allowing users to define custom SQL dialect syntax to meet their custom scenario needs.

[0111] For database-type deletions, the corresponding parser instance can be removed from the parser instance pool, and the corresponding language parsing file can be deleted. The removal operation can be implemented by calling the instance pool's delete interface, while the deletion operation can be completed using a file deletion command.

[0112] When modifying the database type, the parser instance's configuration information can be updated, and the corresponding language parsing file can be replaced. Update operations can be performed by calling the instance pool's update interface, while replacement operations can be completed by overwriting files.

[0113] The method provided in this application improves the adaptability and maintainability of the method by receiving database type update requests and performing corresponding update operations on the parser instance pool and language parsing files.

[0114] Figure 4 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Figure 4 As shown, the method may further include:

[0115] S401. Based on the received SQL statement comparison request, determine the first database type corresponding to the first SQL statement and the second database type corresponding to the second SQL statement in the SQL statement comparison request.

[0116] In this embodiment, the method for determining the first database type and the second database type is the same as the method for determining the database type corresponding to the initial SQL statement in step S101. That is, the corresponding database type is determined by analyzing the syntactic elements in the SQL statement, such as keywords and function call formats.

[0117] S402. Call the parser instances corresponding to the first database type and the second database type respectively to parse the first SQL statement and the second SQL statement to obtain the first abstract syntax tree of the first SQL statement and the second abstract syntax tree of the second SQL statement.

[0118] In this embodiment, the method for calling the parser instance is the same as the method for calling the target parser instance in step S201. That is, by querying the parser instance mapping table, the corresponding parser instance is found and its parsing method is called to generate the corresponding abstract syntax tree.

[0119] Optionally, parallel parsing optimization can be adopted. For batch SQL statements, multi-threaded parallel parsing can be used to make full use of multi-core CPU resources and improve the overall parsing throughput.

[0120] S403. If the similarity between the first abstract syntax tree and the second abstract syntax tree is greater than or equal to the preset similarity threshold, then output a prompt message.

[0121] The prompt information indicates that the first SQL statement and the second SQL statement are semantically equivalent.

[0122] In this embodiment, the similarity between the first abstract syntax tree and the second abstract syntax tree can be calculated using a tree structure comparison algorithm. Specifically, the comparison algorithm can calculate indicators such as node matching degree and structural matching degree of the two trees, and then calculate the similarity value by weighting these indicators. The preset similarity threshold can be set according to actual needs.

[0123] If the similarity between the first abstract syntax tree and the second abstract syntax tree is greater than or equal to a preset similarity threshold, it indicates that the semantic similarity between the first SQL statement and the second SQL statement is high, and it can be determined that the first SQL statement and the second SQL statement are semantically equivalent; otherwise, it indicates that the semantic similarity between the first SQL statement and the second SQL statement is low, and it can be determined that the first SQL statement and the second SQL statement are semantically inequivalent.

[0124] When the first SQL statement and the second SQL statement are semantically equivalent, a prompt message can be output to indicate to the user that the first SQL statement and the second SQL statement are semantically equivalent. This prompt message can be implemented, for example, by generating a message with a specific format, such as a text message containing a semantic equivalence identifier.

[0125] The method provided in this application provides an effective method for semantic verification of SQL statements by parsing two SQL statements to obtain abstract syntax trees and then calculating the similarity of the abstract syntax trees to determine whether the SQL statements are semantically equivalent, thereby improving the accuracy of SQL statement usage.

[0126] Figure 5 This is a flowchart illustrating another data processing method provided in an embodiment of this application. Figure 5 As shown, the method may further include:

[0127] S501. During the process of calling the target parser instance corresponding to the database type to parse the initial SQL statement, anomaly detection is performed on the syntax and semantics of the initial SQL statement.

[0128] In this embodiment, syntax anomaly detection can be achieved by checking whether the initial SQL statement conforms to the syntax rules corresponding to the database type. For example, checking whether the keywords are spelled correctly and whether the statement structure is complete.

[0129] Semantic anomaly detection can be achieved by analyzing the logical rationality of the initial SQL statement. For example, checking whether the queried fields exist in the corresponding tables, or whether the parameter types of functions match.

[0130] Optionally, a unified error handling mechanism can be employed, using a unified error listener to collect error information during the syntax parsing process. Syntax errors are automatically captured during parsing, recording the error location (line number, column number) and error description. All error information is encapsulated into a unified object to ensure consistency in error message format.

[0131] S502. If at least one of the syntax and semantics is incorrect, then generate error information for the initial SQL statement based on the error.

[0132] The error message includes at least one of the following: error message, repair instructions.

[0133] In this embodiment, when checking for syntax errors, the initial SQL statement can be traversed character by character, and compared with the lexical rules (such as the legal spelling of keywords, the naming conventions of identifiers, and the usage format of operators) and grammatical rules (such as the structural order of clauses, the matching requirements of parentheses, and the number of parameters of functions) in the syntax rule base corresponding to the target database type. If a character or combination of characters is found to be inconsistent with any rule in the rule base, a syntax error can be determined.

[0134] When identifying semantic errors, you can first extract objects such as table names, column names, and function names involved in the initial SQL statement, and then match them with the metadata information corresponding to the target database type (such as the actual table structure in the database, the data type of the columns, and the parameter type requirements of the functions). If there are cases such as objects not existing, data types not matching, or incorrect number or type of function parameters, it can be determined that there is a semantic error.

[0135] When an error is detected, the error message can be generated by first recording the location information of the error (such as the starting row number, column number, and character offset), and then combining the error type to generate the corresponding error description.

[0136] If it is a syntax error, the error description can include the error type (such as "keyword misspelling", "parentheses mismatch", "missing clause"), the character content at the error location (such as "the character 'SELEC' in row 3, column 5"), and the syntax rule violation (such as "does not conform to the legal spelling rules for the keyword 'SELECT'").

[0137] If it is a semantic error, the error description can include the error type (such as "table does not exist", "column does not exist", "data type mismatch"), the name of the object involved in the error (such as table name, column name, etc.), and the error reason (such as "the table named 'A' was not found in the database", "column B does not exist in table C", etc.).

[0138] When generating repair instructions, if the error is a syntax error, the system can match the preset repair rule base according to the error type. For example, for a misspelled keyword, the system can retrieve the correct keyword corresponding to the misspelled character from the rule base (e.g., "SELEC" corresponds to "SELECT"); for mismatched parentheses, the system can determine the position of the parentheses that need to be added or deleted based on the number and position of the parentheses.

[0139] If the error is semantic, corresponding verification suggestions can be provided based on the error type. For example, for the "table does not exist" error type, it can be suggested to check the actual table name in the database; for the "column does not exist" error type, it can be suggested to check the column structure of the corresponding table; for the "data type mismatch" error type, it can be suggested to check the data type requirements of function parameters or columns, etc.

[0140] Optionally, it can also automatically repair errors in the initial SQL statement based on error messages and repair instructions, reducing manual repair and improving repair efficiency and accuracy. Specifically, for example, it can locate the erroneous segment in the initial SQL statement based on the error location (row number, column number) in the error message; then, based on the correction suggestions in the repair instructions, it can replace, supplement, or delete the erroneous segment; finally, it can perform a second syntax and semantic check on the repaired SQL statement. If the check passes, the repaired SQL statement is output; if errors still exist, the error message and repair instructions are regenerated.

[0141] The method provided in this application provides timely error feedback to users by performing anomaly detection on the syntax and semantics of the initial SQL statement during the parsing process and generating error information when errors are detected. This helps users quickly locate and correct errors in the SQL statement, thereby improving the efficiency and accuracy of SQL statement writing.

[0142] Optionally, this application may also provide a unified resource management method, that is, each parser implementation class implements a resource release method to release the resources of the lexer and parser. This could include resetting the lexer state, clearing the parser's token stream, and releasing the Deterministic Finite Automaton (DFA) cache. The parser can employ a counter mechanism to automatically clean up the DFA cache after parsing a certain number of SQL statements, thus balancing memory usage and performance.

[0143] Figure 6 A schematic diagram of the structure of a data processing apparatus provided in an embodiment of this application is shown below. Figure 6 As shown, the data processing device provided in this embodiment includes: a first processing module 11, a control module 12, and a second processing module 13.

[0144] The first processing module 11 is used to determine the database type corresponding to the initial SQL statement to be converted based on the received SQL statement conversion request. The SQL statement conversion request includes the initial SQL statement and the target database type of the target SQL statement to which the initial SQL statement needs to be converted.

[0145] The control module 12 is used to call the target parser instance corresponding to the database type to parse the initial SQL statement based on the database type, and obtain the abstract syntax tree corresponding to the initial SQL statement.

[0146] The second processing module 13 is used to obtain the target SQL statement based on the abstract syntax tree and the target database type.

[0147] Optionally, control module 12 is specifically used to call the target parser instance corresponding to the database type from the pre-built parser instance pool. The implementation class of the target parser instance implements the abstract layer interface and calls the language parsing file corresponding to the target parser instance to parse the initial SQL statement, obtaining the abstract syntax tree corresponding to the initial SQL statement. The abstract layer interface can be implemented by the implementation classes of multiple parser instances in the parser instance pool.

[0148] Optionally, control module 12 is specifically used to control the implementation class of the target parser instance to implement the abstract layer interface and call the language parsing file corresponding to the target parser instance to parse the initial SQL statement, obtaining the initial syntax tree of the initial SQL statement. The visitor implementation class of the target parser instance is used to access and traverse the initial syntax tree to obtain its syntax tree feature information. Based on the syntax tree feature information, the abstract syntax tree corresponding to the initial SQL statement is obtained. The initial syntax tree is the syntax tree of the initial database corresponding to the initial SQL statement, and the syntax tree feature information includes at least one of the following: node information, structural information, and semantic information.

[0149] Optionally, the first processing module 11 is further configured to obtain the initial statement fingerprint corresponding to the initial SQL statement based on the initial SQL statement in the SQL statement conversion request. If a target statement fingerprint identical to the initial statement fingerprint exists in the historical parsing results, the parsing result of the target statement fingerprint is extracted from the historical parsing results. Based on the parsing result, the abstract syntax tree corresponding to the initial SQL statement is obtained.

[0150] Optionally, the control module 12 is also used to update the parser instances included in the parser instance pool and the language parsing files corresponding to the parser instances according to the received database type update request. The database type update request includes at least one of adding, deleting, or modifying.

[0151] Optionally, the first processing module 11 is further configured to determine the first database type corresponding to the first SQL statement and the second database type corresponding to the second SQL statement in the received SQL statement comparison request. The control module 12 is further configured to call the parser instances corresponding to the first database type and the second database type respectively to parse the first SQL statement and the second SQL statement, and obtain the first abstract syntax tree of the first SQL statement and the second abstract syntax tree of the second SQL statement. If the similarity between the first abstract syntax tree and the second abstract syntax tree is greater than or equal to a preset similarity threshold, a prompt message is output, which indicates that the first SQL statement and the second SQL statement are semantically equivalent.

[0152] Optionally, the first processing module 11 is further configured to perform anomaly detection on the syntax and semantics of the initial SQL statement during the process of parsing the initial SQL statement by calling the target parser instance corresponding to the database type. If at least one of the syntax and semantics is incorrect, error information of the initial SQL statement is generated based on the error, and the error information includes at least one of the following: error message and repair instruction.

[0153] The data processing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0154] Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device is used to execute the aforementioned data processing method. Figure 7 As shown, the electronic device 700 may include at least one processor 701, a memory 702, and a communication interface 703.

[0155] The memory 702 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions.

[0156] The memory 702 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0157] The processor 701 is used to execute computer execution instructions stored in the memory 702 to implement the method described in the foregoing method embodiments. The processor 701 may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0158] The processor 701 can communicate and interact with external devices through the communication interface 703. In specific implementations, if the communication interface 703, memory 702, and processor 701 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0159] Optionally, in a specific implementation, if the communication interface 703, memory 702, and processor 701 are integrated on a single chip, then the communication interface 703, memory 702, and processor 701 can communicate through an internal interface.

[0160] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0161] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0162] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0163] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0164] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0165] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0166] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0167] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0168] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0169] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A data processing method, characterized in that, include: Based on the received SQL statement conversion request, determine the database type corresponding to the initial SQL statement to be converted. The SQL statement conversion request includes the initial SQL statement and the target database type of the target SQL statement to which the initial SQL statement needs to be converted. Based on the database type, the target parser instance corresponding to the database type is invoked to parse the initial SQL statement, thereby obtaining the abstract syntax tree corresponding to the initial SQL statement; The target SQL statement is obtained based on the abstract syntax tree and the target database type.

2. The method according to claim 1, characterized in that, The step of calling the target parser instance corresponding to the database type to parse the initial SQL statement based on the database type, and obtaining the abstract syntax tree corresponding to the initial SQL statement, includes: Call the target parser instance corresponding to the database type from the pre-built parser instance pool; The implementation class of the target parser instance implements the abstract layer interface and calls the language parsing file corresponding to the target parser instance to parse the initial SQL statement to obtain the abstract syntax tree corresponding to the initial SQL statement. The abstract layer interface can be implemented by the implementation classes of multiple parser instances in the parser instance pool.

3. The method according to claim 2, characterized in that, The implementation class that controls the target parser instance implements the abstract layer interface and calls the language parsing file corresponding to the target parser instance to parse the initial SQL statement, obtaining the abstract syntax tree corresponding to the initial SQL statement, including: The implementation class that controls the target parser instance implements the abstract layer interface and calls the language parsing file corresponding to the target parser instance to parse the initial SQL statement, thereby obtaining the initial syntax tree of the initial SQL statement. The initial syntax tree is the syntax tree of the initial database corresponding to the initial SQL statement. The initial syntax tree is traversed using the visitor implementation class of the target parser instance to obtain the syntax tree feature information of the initial syntax tree, which includes at least one of the following: node information, structural information, and semantic information. Based on the syntax tree feature information, the abstract syntax tree corresponding to the initial SQL statement is obtained.

4. The method according to claim 1, characterized in that, Also includes: Based on the initial SQL statement in the SQL statement conversion request, obtain the initial statement fingerprint corresponding to the initial SQL statement; If a target statement fingerprint that is identical to the initial statement fingerprint exists in the historical parsing results, then the parsing result of the target statement fingerprint is extracted from the historical parsing results; Based on the parsing results, the abstract syntax tree corresponding to the initial SQL statement is obtained.

5. The method according to claim 2, characterized in that, Also includes: Based on the received database type update request, update the parser instances included in the parser instance pool and the language parsing files corresponding to the parser instances. The database type update request includes at least one of adding, deleting, or modifying.

6. The method according to claim 1, characterized in that, Also includes: Based on the received SQL statement comparison request, determine the first database type corresponding to the first SQL statement and the second database type corresponding to the second SQL statement in the SQL statement comparison request; The parser instances corresponding to the first database type and the second database type are called respectively to parse the first SQL statement and the second SQL statement to obtain the first abstract syntax tree of the first SQL statement and the second abstract syntax tree of the second SQL statement; If the similarity between the first abstract syntax tree and the second abstract syntax tree is greater than or equal to a preset similarity threshold, a prompt message is output, which indicates that the first SQL statement and the second SQL statement are semantically equivalent.

7. The method according to claim 1, characterized in that, Also includes: During the process of calling the target parser instance corresponding to the database type to parse the initial SQL statement, anomaly detection is performed on the syntax and semantics of the initial SQL statement; If at least one of the syntax and semantics is incorrect, then an error message for the initial SQL statement is generated based on the error, and the error message includes at least one of the following: error information and repair instructions.

8. A data processing apparatus, characterized in that, include: The first processing module is used to determine the database type corresponding to the initial SQL statement to be converted based on the received SQL statement conversion request. The SQL statement conversion request includes the initial SQL statement and the target database type of the target SQL statement to which the initial SQL statement needs to be converted. The control module is used to call the target parser instance corresponding to the database type to parse the initial SQL statement based on the database type, and obtain the abstract syntax tree corresponding to the initial SQL statement; The second processing module is used to obtain the target SQL statement based on the abstract syntax tree and the target database type.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images