A SQL language visual conversion system in iPaaS platform
By designing a SQL language visual conversion system on the iPaaS platform, the problems of inefficiency and error proneness in the face of multiple database SQL statements are solved, and the automatic recognition, parsing and visualization of SQL statements are realized, which improves processing efficiency and user understanding capabilities.
Patent Information
- Application Number
- CN202411891768.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-20
- Publication Date
- 2025-04-04
- Estimated Expiration
- 2044-12-20
AI Technical Summary
The prior art is inefficient and error-prone when facing SQL statements of multiple different databases, especially when dealing with complex SQL statements or unfamiliar databases.
A SQL language visual conversion system in the iPaaS platform is designed, and the automatic recognition, parsing and visualization of SQL statements is realized through data collection and storage, feature extraction, model training, syntax analysis and visualization conversion modules.
The system can quickly and accurately identify the database type to which the SQL statement belongs, generate syntax trees or internal data structures, and convert them into intuitive visual representations, improving users' understanding and operational efficiency of SQL statements.
Smart Images

Figure CN119759939A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information processing technology, and in particular to a SQL language visual conversion system in an iPaaS platform. Background Art
[0002] In today's era of rapid development of information technology, enterprises and organizations widely use various database systems to store, manage and process data. SQL, as a standard language for interacting with databases, has dialect differences in different database systems. For example, Oracle and MySQL differ in function usage, syntax structure, keywords, etc.
[0003] In their daily work, developers and database administrators often need to deal with SQL statements involving multiple different databases. However, existing tools and methods have many shortcomings when facing this diversity. The traditional method often requires manual experience to determine the database type to which the SQL statement belongs, and then use the corresponding grammar rules to process it. This is not only inefficient but also error-prone. Especially when dealing with complex SQL statements or unfamiliar databases, syntax differences may lead to query writing errors, performance problems, and even system failures. Therefore, a SQL language visual conversion system in the iPaaS platform is proposed. Summary of the invention
[0004] The purpose of the present invention is to solve the shortcomings of low efficiency and easy errors in the prior art SQL statements facing multiple different databases, and to propose a SQL language visual conversion system in the iPaaS platform.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A SQL language visual conversion system in an iPaaS platform, comprising:
[0007] Data collection and storage module: collects SQL statement samples from different databases and stores them in a suitable storage structure;
[0008] Feature extraction module: extracts information that can reflect the characteristics of the database dialect from SQL statements and converts it into a feature vector representation;
[0009] Model training module: Use the extracted feature vectors to train a classification model to predict the database type to which the SQL statement belongs;
[0010] Syntax parsing module: according to the predicted database type, use the corresponding syntax parser to parse the SQL statement;
[0011] Visual conversion module: converts the parsed SQL statement structure into a visual representation, and handles the visual display of special syntax and functions of different databases;
[0012] The data collection and storage module collects SQL statement samples from multiple channels, labels each SQL statement, and determines the database type to which it belongs. The feature extraction module counts the number of occurrences of keywords in SQL statements, counts the number of occurrences of functions in SQL statements, extracts the subquery depth and the number of connection operations of SQL statements, counts the number of occurrences of data types in SQL statements, counts the number of occurrences of operators in SQL statements, and combines various feature vectors into a comprehensive feature vector. The model training module trains and evaluates a random forest classification model to predict the database type to which the SQL statement belongs. The syntax parsing module creates a syntax parser for each database type, predicts the database type to which the SQL statement belongs based on the trained classification model, and then calls the corresponding syntax parser to parse the SQL statement to generate a syntax tree or other internal data structure. The visualization conversion module defines a mapping rule from syntax tree nodes to visualization elements, generates a basic visualization representation, combines the basic visualization elements with visualization elements of special syntax, generates a final visualization representation, and displays it on a user interface to provide an intuitive SQL statement visualization and operation interface.
[0013] The process of collecting SQL statement samples for different databases is as follows:
[0014] SQL statements are collected from various sources, including open source database projects, enterprise database application systems, database tutorials, and online code libraries. Suppose the collected SQL statement set is S = {s1, s2, ..., s n}, where s i Represents the i-th SQL statement. For each SQL statement s i , based on the database-specific identification information, determine the database type to which it belongs i ;
[0015] The database type set is T = {t1, t2, ..., t m}, the annotated SQL statements are stored as a data set D = {(s1, t1), (s2, t2), ..., (s n , t n )}.
[0016] Furthermore, an open source database project is selected, such as MySQL, and the official repository of the open source database project is found through a code hosting platform. The contents of these files are read and parsed using a text editor or programming language, and the SQL statements in the files are identified. According to the selected open source database project, the database type to which each SQL statement belongs is recorded, and the extracted SQL statements and corresponding database types are organized into tuples.
[0017] The procedures stored in the appropriate storage structure are:
[0018] Create a table to store data set D, which contains two fields: SQL statement content and database type. Insert the collected SQL statements and their corresponding database types into the created table.
[0019] In one embodiment, a data set including SQL statements and their corresponding database types is prepared.
[0020] Use SQL INSERT statements to insert each record into the table, for example, insert a MySQL-type SQL statement and an Oracle-type SQL statement, use SQL query statements to retrieve SQL statements of a specific type from the table, and use database management tools to perform operations such as adding, deleting, modifying, and querying data;
[0021] The process of extracting information that can reflect the characteristics of the database dialect and converting it into a feature vector representation is as follows:
[0022] Construct keyword dictionary K;
[0023] Collect keywords of all common databases, build a keyword dictionary K, count the number of keyword occurrences, and for each SQL statement s i , count the number of times each keyword appears in it, let keyword k j In s i The number of occurrences in is f k (k j ,s i ), generate keyword feature vector k i =[f k (k1, s i ), f k (k1, s i ), …, f k (k1, s i )];
[0024] Collect all common database functions and build a function library F;
[0025] Collect functions: extract commonly used function names from various database documents, official manuals, and community resources;
[0026] Arrange the list: Arrange the collected function names into a list to ensure that common SQL functions are covered;
[0027] Standardization: unify the format of function names, such as converting all to uppercase or lowercase to ensure consistency;
[0028] For each SQL statement i , count the number of occurrences of each function, convert the SQL statement to a standard format, use regular expressions or a lexical analyzer to identify the functions in the SQL statement, and combine the number of occurrences of all functions into a vector f i ;
[0029] Combining the above-mentioned feature vectors, we get the SQL statement s i The final eigenvector v of i =[k i ,f i ].
[0030] In one embodiment, data is collected from multiple sources to ensure the comprehensiveness and accuracy of keyword and function libraries, and the formats of keywords and functions are unified. Regular expressions are used to automatically identify keywords and functions in SQL statements, and SQL statements are converted into feature vector form to facilitate further processing and analysis.
[0031] The implementation process of training a classification model is:
[0032] Select a random forest model, collect a dataset D containing SQL statements and their database type labels, and then use the final feature vector v i , denote the database type label of each SQL statement as Y, forming a data set;
[0033] Divide the data set into a training set and a test set with a division ratio of r, then the size of the training set is |D train |=r·|D|, the test set size is |D test |=(1-r)·|D|;
[0034] Using the training set D train Train the random forest model to learn the mapping relationship between SQL statements and database types. Suppose the random forest model contains n decision trees. Create a random forest model M. For the jth decision tree, according to the final feature vector v of the training data i and the corresponding database type label Y to learn the decision rule, traverse each possible value of each feature in the feature vector, divide the split point by the information gain ratio, and recursively repeat the above feature selection and data set partitioning process for each subset based on the selected split point until the stopping condition is met;
[0035] In one embodiment, the stopping condition may include that the depth of the tree reaches a preset maximum value, the number of samples in the subset is less than a certain threshold, or the purity of the sample category in the subset reaches a certain standard. For example, the maximum depth of the tree is set to 10, and the tree stops growing when the depth reaches 10; or the splitting stops when the number of samples in the subset is less than 5; or when the proportion of samples of a certain category in the subset exceeds 95%, the subset is considered to be pure enough and the splitting stops;
[0036] Model ensemble repeats the above decision tree training process. After each decision tree is trained, they are integrated to form a random forest model.
[0037] The process of predicting the database type to which a SQL statement belongs is:
[0038] In the prediction stage, for a new SQL statement feature vector, the prediction result of the random forest model is obtained by voting on the prediction results of each decision tree, that is, each decision tree predicts the statement feature vector to obtain a database type prediction result, and then the prediction results of all decision trees are counted. The database type that appears most times is the final prediction result of the random forest model.
[0039] The process of parsing SQL statements using a syntax parser is as follows:
[0040] Create syntax parsers that are precisely adapted to different database types;
[0041] When the SQL statement to be parsed is received, the trained classification model is first used to predict the database type to which it belongs. The predicted database types include Oracle database and MySQL database. According to the predicted database type, the corresponding syntax parser is called to parse the SQL statement. During the parsing process, the parser gradually decomposes the SQL statement into syntax elements (keywords, identifiers, expressions, clauses, etc.) according to the syntax rules of the database, and constructs the corresponding internal data structure, and constructs an abstract syntax tree that reflects the syntax structure of the statement, where nodes represent syntax elements and the relationship between nodes represents the syntax structure.
[0042] In one embodiment, for an Oracle database, its syntax parser can accurately identify and process Oracle-specific syntax structures (such as ROWNUM to limit the number of returned rows, CONNECTBY for hierarchical queries), keywords (such as PRIOR for specifying hierarchical relationships, etc.), and functions (such as NVL for processing null values); for a MySQL database, the parser can correctly parse its specific syntax and functions.
[0043] The process of converting the parsed SQL statement structure into a visual representation is:
[0044] Special visual representation rules are formulated for different parsed SQL statement structures. For Oracle's ROWNUM keyword, an input box or drop-down menu specifically for setting the limit on the number of returned rows is added in the query area of the visual interface, and it is clearly marked as related to ROWNUM. When the user enters a value in the input box, the system automatically converts it into the corresponding ROWNUM syntax structure. For MySQL's LIMIT keyword, an area specifically for setting the LIMIT parameter is added at the end of the query area, which contains two input boxes, one for specifying the starting position and the other for specifying the number of returned rows.
[0045] The process of handling the special syntax and visual display of functions for different databases is as follows:
[0046] For the parsed SQL statement, the database type is d, and the parsing result is the internal data structure G. According to the visual representation rules, the grammatical structure is converted into a basic visual element set M, and the special grammatical structures and functions contained in the statement are identified. For each special grammatical structure or function, a visual element of the special grammar is generated through the visualization function SPV.
[0047] For ROWNUM in Oracle statements and LIMIT in MySQL, the corresponding visual input box or drop-down menu is generated through SPV, and then all the visual elements of special syntax are combined to obtain a set of special syntax visual elements.
[0048] In one embodiment, the visualization function SPV is a visualization conversion function specifically used to process special syntax and functions in a specific database type;
[0049] Finally, the basic visualization element set and the special syntax visualization element set are combined to obtain the final visualization representation. These visualization elements are integrated into the visualization interface according to a reasonable layout and interaction logic, providing users with an intuitive, easy-to-understand and easy-to-operate SQL statement visualization display and operation interface.
[0050] The present invention has the following beneficial effects:
[0051] 1. In the present invention, a large number of SQL statement samples from different databases are collected from various channels to construct a comprehensive data set, and feature vectors containing multiple dimensions such as keywords, functions, structural features, data types, operators, etc. are extracted. The random forest model is used for training, and the database type to which the SQL statement belongs can be accurately predicted. This helps to quickly determine the applicable grammatical rules and processing methods when facing SQL statements from different databases, and improve the accuracy and efficiency of subsequent processing.
[0052] 2. In the present invention, a precisely adapted syntax parser is created for different database types, and the corresponding parser is called based on the predicted database type to parse the SQL statement, which can generate an accurate syntax tree or internal data structure. This enables the system to deeply understand the syntax structure of the SQL statement and provide a reliable foundation for subsequent visualization, query optimization and other operations.
[0053] 3. In the present invention, by defining the mapping rules from syntax tree nodes to visualization elements, the parsed SQL statement structure can be converted into an intuitive visualization representation. At the same time, special syntax and functions of different databases are processed through special visualization functions to generate visualization elements that are easy to understand and operate. This greatly improves the user's ability to understand SQL statements, especially for complex queries or unfamiliar database syntax. Users can quickly grasp the logic and function of the statement through the visualization interface. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 This is a system block diagram of the SQL language visual conversion system in the iPaaS platform proposed by the present invention. DETAILED DESCRIPTION
[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0056] Embodiment 1
[0057] like Figure 1 As shown, the present invention proposes a SQL language visual conversion system in an iPaaS platform, including:
[0058] Data collection and storage module: collects SQL statement samples from different databases and stores them in a suitable storage structure;
[0059] Feature extraction module: extracts information that can reflect the characteristics of the database dialect from SQL statements and converts it into a feature vector representation;
[0060] Model training module: Use the extracted feature vectors to train a classification model to predict the database type to which the SQL statement belongs;
[0061] Syntax parsing module: according to the predicted database type, use the corresponding syntax parser to parse the SQL statement;
[0062] Visual conversion module: converts the parsed SQL statement structure into a visual representation, and handles the visual display of special syntax and functions of different databases;
[0063] The data collection and storage module collects SQL statement samples from various channels (such as open source database projects, enterprise database application systems, database tutorials, online code libraries, etc.), annotates each SQL statement, and determines the database type to which it belongs. The feature extraction module counts the number of occurrences of keywords in SQL statements, counts the number of occurrences of functions in SQL statements, extracts the subquery depth and the number of connection operations of SQL statements, counts the number of occurrences of data types in SQL statements, counts the number of occurrences of operators in SQL statements, and combines various feature vectors into a comprehensive feature vector. The model training module trains and evaluates a random forest classification model to predict the database type to which the SQL statement belongs. The syntax parsing module creates a syntax parser for each database type, predicts the database type to which the SQL statement belongs based on the trained classification model, and then calls the corresponding syntax parser to parse the SQL statement to generate a syntax tree or other internal data structure. The visualization conversion module defines the mapping rules from syntax tree nodes to visualization elements, generates basic visualization representations, combines basic visualization elements with visualization elements of special syntax, generates the final visualization representation, and displays it on the user interface, providing an intuitive SQL statement visualization and operation interface.
[0064] The process of collecting SQL statement samples for different databases is as follows:
[0065] SQL statements are collected from various sources, including open source database projects, enterprise database application systems, database tutorials, and online code libraries. Suppose the collected SQL statement set is S = {s1, s2, ..., s n}, where s i Represents the i-th SQL statement. For each SQL statement s i , based on the database-specific identification information, determine the database type to which it belongs i ;
[0066] The database type set is T = {t1, t2, ..., t m}, the annotated SQL statements are stored as a data set D = {(s1, t1), (s2, t2), ..., (s n , t n )};
[0067] Furthermore, an open source database project is selected, such as MySQL, and the official repository of the open source database project is found through a code hosting platform. The contents of these files are read and parsed using a text editor or programming language, and the SQL statements in the files are identified. According to the selected open source database project, the database type to which each SQL statement belongs is recorded, and the extracted SQL statements and corresponding database types are organized into tuples.
[0068] The procedures stored in the appropriate storage structure are:
[0069] Create a table to store data set D, which contains two fields: SQL statement content and database type. Insert the collected SQL statements and their corresponding database types into the created table.
[0070] In one embodiment, a data set including SQL statements and their corresponding database types is prepared.
[0071] Use SQL INSERT statements to insert each record into the table, for example, insert a MySQL-type SQL statement and an Oracle-type SQL statement, use SQL query statements to retrieve SQL statements of a specific type from the table, and use database management tools to perform operations such as adding, deleting, modifying, and querying data;
[0072] The process of extracting information that can reflect the characteristics of the database dialect and converting it into a feature vector representation is as follows:
[0073] Construct keyword dictionary K;
[0074] Collect keywords of all common databases, build a keyword dictionary K, count the number of keyword occurrences, and for each SQL statement s i , count the number of times each keyword appears in it, let keyword k j In s i The number of occurrences in is f k (k j ,s i ), generate keyword feature vector k i =[f k (k1, s i ), f k (k1, s i ), …, f k (k1, s i )];
[0075] Collect all common database functions and build a function library F;
[0076] Collect functions: extract commonly used function names from various database documents, official manuals, and community resources;
[0077] Arrange the list: Arrange the collected function names into a list to ensure that common SQL functions are covered;
[0078] Standardization: unify the format of function names, such as converting all to uppercase or lowercase to ensure consistency;
[0079] For each SQL statement i , count the number of occurrences of each function, convert the SQL statement to a standard format, use regular expressions or a lexical analyzer to identify the functions in the SQL statement, and combine the number of occurrences of all functions into a vector f i ;
[0080] Combining the above-mentioned feature vectors, we get the SQL statement s i The final eigenvector v of i =[k i ,f i ].
[0081] In one embodiment, data is collected from multiple sources to ensure the comprehensiveness and accuracy of keyword and function libraries, and the formats of keywords and functions are unified. Regular expressions are used to automatically identify keywords and functions in SQL statements, and SQL statements are converted into feature vector form to facilitate further processing and analysis.
[0082] The implementation process of training a classification model is:
[0083] Select a random forest model, collect a dataset D containing SQL statements and their database type labels, and then use the final feature vector v i , denote the database type label of each SQL statement as Y, forming a data set;
[0084] Divide the data set into a training set and a test set with a division ratio of r, then the size of the training set is |D train |=r·|D|, the test set size is |D test |=(1-r)·|D|;
[0085] Using the training set D train Train the random forest model to learn the mapping relationship between SQL statements and database types. Suppose the random forest model contains n decision trees. Create a random forest model M. For the jth decision tree, according to the final feature vector v of the training data i and the corresponding database type label Y to learn the decision rule, traverse each possible value of each feature in the feature vector, divide the split point by the information gain ratio, and recursively repeat the above feature selection and data set partitioning process for each subset based on the selected split point until the stopping condition is met;
[0086] In one embodiment, the stopping condition may include that the depth of the tree reaches a preset maximum value, the number of samples in the subset is less than a certain threshold, or the purity of the sample category in the subset reaches a certain standard. For example, the maximum depth of the tree is set to 10, and the tree stops growing when the depth reaches 10; or the splitting stops when the number of samples in the subset is less than 5; or when the proportion of samples of a certain category in the subset exceeds 95%, the subset is considered to be pure enough and the splitting stops;
[0087] Model ensemble repeats the above decision tree training process. After each decision tree is trained, they are integrated to form a random forest model.
[0088] The process of predicting the database type to which a SQL statement belongs is:
[0089] In the prediction stage, for a new SQL statement feature vector, the prediction result of the random forest model is obtained by voting on the prediction results of each decision tree, that is, each decision tree predicts the statement feature vector to obtain a database type prediction result, and then the prediction results of all decision trees are counted. The database type that appears most times is the final prediction result of the random forest model.
[0090] The process of parsing SQL statements using a syntax parser is as follows:
[0091] Create syntax parsers that are precisely adapted to different database types;
[0092] When the SQL statement to be parsed is received, the trained classification model is first used to predict the database type to which it belongs. The predicted database types include Oracle database and MySQL database. According to the predicted database type, the corresponding syntax parser is called to parse the SQL statement. During the parsing process, the parser gradually decomposes the SQL statement into syntax elements (keywords, identifiers, expressions, clauses, etc.) according to the syntax rules of the database, and constructs the corresponding internal data structure, and constructs an abstract syntax tree that reflects the syntax structure of the statement, where nodes represent syntax elements and the relationship between nodes represents the syntax structure.
[0093] In one embodiment, for an Oracle database, its syntax parser can accurately identify and process Oracle-specific syntax structures (such as ROWNUM to limit the number of returned rows, CONNECTBY for hierarchical queries), keywords (such as PRIOR for specifying hierarchical relationships, etc.), and functions (such as NVL for processing null values); for a MySQL database, the parser can correctly parse its specific syntax and functions.
[0094] The process of converting the parsed SQL statement structure into a visual representation is:
[0095] Special visual representation rules are formulated for different parsed SQL statement structures. For Oracle's ROWNUM keyword, an input box or drop-down menu specifically for setting the limit on the number of returned rows is added in the query area of the visual interface, and it is clearly marked as related to ROWNUM. When the user enters a value in the input box, the system automatically converts it into the corresponding ROWNUM syntax structure. For MySQL's LIMIT keyword, an area specifically for setting the LIMIT parameter is added at the end of the query area, which contains two input boxes, one for specifying the starting position and the other for specifying the number of returned rows.
[0096] The process of handling the special syntax and visual display of functions for different databases is as follows:
[0097] For the parsed SQL statement, the database type is d, and the parsing result is the internal data structure G. According to the visual representation rules, the grammatical structure is converted into a basic visual element set M, and the special grammatical structures and functions contained in the statement are identified. For each special grammatical structure or function, a visual element of the special grammar is generated through the visualization function SPV.
[0098] For ROWNUM in Oracle statements and LIMIT in MySQL, the corresponding visual input box or drop-down menu is generated through SPV, and then all the visual elements of special syntax are combined to obtain a set of special syntax visual elements.
[0099] In one embodiment, the visualization function SPV is a visualization conversion function specifically used to process special syntax and functions in a specific database type;
[0100] Finally, the basic visualization element set and the special syntax visualization element set are combined to obtain the final visualization representation. These visualization elements are integrated into the visualization interface according to a reasonable layout and interaction logic, providing users with an intuitive, easy-to-understand and easy-to-operate SQL statement visualization display and operation interface.
[0101] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A SQL language visual conversion system in an iPaaS platform, characterized in that: include: Data collection and storage module: collects SQL statement samples from different databases and stores them in a suitable storage structure; Feature extraction module: extracts information that can reflect the characteristics of the database dialect from SQL statements and converts it into a feature vector representation; Model training module: Use the extracted feature vectors to train a classification model to predict the database type to which the SQL statement belongs; Syntax parsing module: according to the predicted database type, use the corresponding syntax parser to parse the SQL statement; Visual conversion module: converts the parsed SQL statement structure into a visual representation, and handles the visual display of special syntax and functions of different databases; The data collection and storage module collects SQL statement samples from multiple channels, labels each SQL statement, and determines the database type to which it belongs. The feature extraction module counts the number of occurrences of keywords in SQL statements, counts the number of occurrences of functions in SQL statements, extracts the subquery depth and the number of connection operations of SQL statements, counts the number of occurrences of data types in SQL statements, counts the number of occurrences of operators in SQL statements, and combines various feature vectors into a comprehensive feature vector. The model training module trains and evaluates a random forest classification model to predict the database type to which the SQL statement belongs. The syntax parsing module creates a syntax parser for each database type, predicts the database type to which the SQL statement belongs based on the trained classification model, and then calls the corresponding syntax parser to parse the SQL statement to generate a syntax tree or other internal data structure. The visualization conversion module defines a mapping rule from syntax tree nodes to visualization elements, generates a basic visualization representation, combines the basic visualization elements with visualization elements of special syntax, generates a final visualization representation, and displays it on a user interface to provide an intuitive SQL statement visualization and operation interface.
2. The SQL language visual conversion system in the iPaaS platform according to claim 1, characterized in that: The process of collecting SQL statement samples from different databases is as follows: SQL statements are collected from various sources, including open source database projects, enterprise database application systems, database tutorials, and online code libraries. Suppose the collected SQL statement set is S = {s1, s2, ..., s n }, where s i Represents the i-th SQL statement. For each SQL statement s i , based on the database-specific identification information, determine the database type to which it belongs i ; The database type set is T = {t1, t2, ..., t m }, the annotated SQL statements are stored as a data set D = {(s1, t1), (s2, t2), ..., (s n , t n )}.
3. The SQL language visual conversion system in the iPaaS platform according to claim 1, characterized in that: The process of storing in a suitable storage structure is: Create a table storage data set D that contains two fields: storing SQL statement content and storing database type, and insert the collected SQL statements and their corresponding database types into the created table.
4. The SQL language visual conversion system in the iPaaS platform according to claim 1, characterized in that: The process of extracting information that can reflect the database dialect characteristics and converting it into a feature vector representation is as follows: Construct keyword dictionary K; Collect keywords of all common databases, build a keyword dictionary K, count the number of keyword occurrences, and for each SQL statement s i , count the number of times each keyword appears in it, let keyword k j In s i The number of occurrences in is f k (k j ,s i ), generate keyword feature vector k i =[f k (k1, s i ), f k (k1, s i ), …, f k (k1, s i )]; Collect all common database functions and build a function library F; Collect functions: extract commonly used function names from various database documents, official manuals, and community resources; Arrange the list: Arrange the collected function names into a list; Standardization: unify the format of function names and convert them all to uppercase or lowercase; For each SQL statement i , count the number of occurrences of each function, convert the SQL statement to a standard format, use regular expressions or a lexical analyzer to identify the functions in the SQL statement, and combine the number of occurrences of all functions into a vector f i ; Combining the above-mentioned feature vectors, we get the SQL statement s i The final eigenvector v of i =[k i ,f i ].
5. The SQL language visual conversion system in the iPaaS platform according to claim 1, characterized in that: The implementation process of training a classification model is as follows: Select a random forest model, collect a dataset D containing SQL statements and their database type labels, and then use the final feature vector v i , denote the database type label of each SQL statement as Y, forming a data set; Divide the data set into a training set and a test set with a division ratio of r, then the size of the training set is |D train |=r·|D|, the test set size is |D test |=(1-r)·|D|; Using the training set D train Train the random forest model to learn the mapping relationship between SQL statements and database types. Suppose the random forest model contains n decision trees. Create a random forest model M. For the jth decision tree, according to the final feature vector v of the training data i and the corresponding database type label Y to learn the decision rule, traverse each possible value of each feature in the feature vector, divide the split point by the information gain ratio, and recursively repeat the above feature selection and data set partitioning process for each subset based on the selected split point until the stopping condition is met; Repeat the above decision tree training process. After each decision tree is trained, they are integrated to form a random forest model.
6. The SQL language visual conversion system in the iPaaS platform according to claim 1, characterized in that: The process of predicting the database type to which the SQL statement belongs is as follows: In the prediction stage, for a new SQL statement feature vector, the prediction result of the random forest model is obtained by voting on the prediction results of each decision tree, that is, each decision tree predicts the statement feature vector to obtain a database type prediction result, and then the prediction results of all decision trees are counted. The database type that appears most times is the final prediction result of the random forest model.
7. The SQL language visual conversion system in the iPaaS platform according to claim 1, characterized in that: The process of using the syntax analyzer to parse the SQL statement is as follows: Create syntax parsers that are precisely adapted to different database types; When the SQL statement to be parsed is received, the trained classification model is first used to predict the database type to which it belongs. The predicted database types include Oracle database and MySQL database. According to the predicted database type, the corresponding syntax parser is called to parse the SQL statement. During the parsing process, the parser gradually decomposes the SQL statement into syntax elements (keywords, identifiers, expressions, clauses) according to the syntax rules of the database, and constructs the corresponding internal data structure, and constructs an abstract syntax tree that reflects the syntax structure of the statement, where nodes represent syntax elements and the relationship between nodes represents the syntax structure.
8. The SQL language visual conversion system in the iPaaS platform according to claim 1, characterized in that: The process of converting the parsed SQL statement structure into a visual representation is as follows: Special visual representation rules are formulated for different parsed SQL statement structures. For Oracle's ROWNUM keyword, an input box or drop-down menu specifically for setting the limit on the number of returned rows is added in the query area of the visual interface, and it is clearly marked as related to ROWNUM. When the user enters a value in the input box, the system automatically converts it into the corresponding ROWNUM syntax structure. For MySQL's LIMIT keyword, an area specifically for setting the LIMIT parameter is added at the end of the query area, which contains two input boxes, one for specifying the starting position and the other for specifying the number of returned rows.
9. The SQL language visual conversion system in the iPaaS platform according to claim 1, characterized in that: The process of processing the special syntax and visual display of functions of different databases is as follows: For the parsed SQL statement, the database type is d, and the parsing result is the internal data structure G. According to the visual representation rules, the grammatical structure is converted into a basic visual element set M, and the special grammatical structures and functions contained in the statement are identified. For each special grammatical structure or function, a visual element of the special grammar is generated through the visualization function SPV. For ROWNUM in Oracle statements and LIMIT in MySQL, the corresponding visual input box or drop-down menu is generated through SPV, and then all the visual elements of special syntax are combined to obtain a set of special syntax visual elements; The basic visualization element set and the special syntax visualization element set are combined to obtain the final visualization representation, and these visualization elements are integrated into the visualization interface according to a reasonable layout and interaction logic, providing users with an intuitive, easy-to-understand and easy-to-operate SQL statement visualization display and operation interface.
Citation Information
Patent Citations
SQL statement grammar visual analysis method and system and computer readable storage medium
CN113177057A
SQL (Structured Query Language) statement forwarding method and device, electronic equipment and storage medium
CN117407413A
Large language model Text2SQL (Structured Query Language) chart generation method and device
CN117851445A
System and method for data warehouse migration
US20230185781A1