ICL large language model data query generation method and system based on diversity SQL reinforcement

Abstract

The invention discloses an ICL large language model data query generation method and system based on diversity SQL enhancement, and the method comprises the steps: S1, determining a target query database, and obtaining a database Schema; s2, vectorizing a database Schema, and storing the vectorized database Schema into a database meta-information vector library; s3, vectorizing SQL statements in the SQL sample set, and storing the vectorized SQL statements in an SQL sample vector library; s4, performing clustering diversity sample recall on the information stored in the SQL sample vector library; s5, according to the database summary and the natural language question of the user, combining the recalled sample question and the corresponding SQL query statement as model reference information, constructing comprehensive prompt information, and guiding the large language model to generate SQL query conforming to semantics and grammar; and S6, inputting the large language model, and generating SQL statements by the large language model. According to the method, the Prompt is formed by recalling the reasonable related information, the model is guided to correctly generate the SQL, and the accuracy of the SQL statement generated by the large language model can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular to a method and system for generating ICL large language model data queries based on diversity SQL enhancement. Background Art

[0002] With the advent of the big data era, data infrastructure construction is constantly improving, and the query of various historical and real-time data has become a high-frequency demand, while existing business systems or BI tools are difficult to meet fragmented query needs. At the same time, the increasingly complex database structure also makes it difficult for non-database designers or non-technical personnel to conduct accurate data queries.

[0003] The ability of the large language model in semantic understanding and recognition makes it possible to query databases based on natural language. The implementation methods of natural language database query sentence generation based on the large language model can be roughly divided into: context learning and model fine-tuning.

[0004] The principle of the In-context Learning (ICL) method based on a large language model is to directly combine the input and output examples of a task into a prompt without any fine-tuning of the model itself. This method is different from fine-tuning, and there is no need to adjust the parameters of the model for a specific task. In this way, the model can directly generate prediction results for the task based on the examples in the prompt. The ICL method is more efficient than unsupervised learning and avoids the tedious problem of training models separately for different tasks. In this way, there is no need to re-update the model parameters for each new task, simplifying the process of model application.

[0005] Early implementations of in-context learning (ICL) typically used randomly selected examples or manually crafted examples, while modern methods focus on learning in context using domain-specific data. Although these methods have achieved good results, they still face challenges when dealing with large databases and complex queries. Subsequent studies have shown that even if they are all examples from the same domain, the selection and order of these examples still have a significant impact on performance. Therefore, technical development focuses on optimizing example selection, aiming to maximize the semantic coverage of the task.

[0006] Existing ICL methods such as random selection and Nearest Neighbor Search still have a lot of room for optimization. For example, the random selection method may make the recalled samples irrelevant to the current task, making the relevant prompts too unhelpful for the current task. The Nearest Neighbor Search method may make the recalled samples highly similar, resulting in information redundancy, wasting model input tokens, and making it difficult to recall useful information-filled samples. Summary of the invention

[0007] In order to solve the above-mentioned technical problems existing in the prior art, the present invention provides an ICL large language model data query generation method and system based on diversity SQL enhancement, a clustering-first-recall method, and a natural language to SQL (NL2SQL) technology based on this method. It is an innovative method that relies on a powerful pre-trained large language model, forms prompts by recalling reasonable relevant information, and guides the model to correctly generate SQL, which can effectively improve the accuracy of SQL statements generated by the large language model.

[0008] The technical solution adopted by the present invention is:

[0009] A method for generating ICL large language model data query based on diversity SQL enhancement, characterized in that the method comprises the following steps:

[0010] S1. Determine the target query database and obtain the database schema;

[0011] S2, vectorize the database schema and store it in the database metadata vector library;

[0012] S3, vectorizing the SQL statements in the SQL sample set and storing them in the SQL sample vector library;

[0013] S4, clustering the information stored in the SQL sample vector library to retrieve diverse samples;

[0014] S5. Based on the database summary and the user's natural language questions, combined with the recalled sample questions and their corresponding SQL query statements, a comprehensive prompt information is constructed as model reference information to guide the large language model to generate SQL queries that conform to semantics and syntax;

[0015] S6. Input the large language model, and the large language model generates SQL statements.

[0016] Furthermore, in step S2, the specific steps of vectorizing the database schema include:

[0017] S21, obtain the table name and column name information in the database Schema;

[0018] S22. Use word embedding technology to vectorize table names and column names respectively;

[0019] S23. Store the above vectorized information into a vector library and map it with the original table name and column name.

[0020] Furthermore, in step S23, the mapping method is specifically completed by inserting the table name, column name, and vector into the database as a same piece of data.

[0021] Furthermore, in step S3, the specific steps of vectorizing the SQL statements in the SQL sample set include:

[0022] S31, disassemble the query questions and SQL statements in the SQL sample set;

[0023] S32, vectorizing the decomposed query questions and SQL statements respectively by sentence embedding;

[0024] Using pre-trained BERT or other pre-trained models, SQL samples are converted into vector representations rich in semantic information. For database schema, a structure-aware embedding method is developed to convert table names, fields, and their relationships into vectors and fuse them with semantic vectors. Through joint training, the semantic and structure-aware vector representations are optimized to make them more consistent and effective in the same vector space.

[0025] S33, mapping the above vectorized query questions and SQL statements one by one and storing them in a vector library.

[0026] Furthermore, in step S33, the mapping method is specifically as follows: the question character string, the question vector, and the SQL statement are mapped by inserting the same piece of data into the vector library.

[0027] Furthermore, in step S4, the specific steps of clustering diversity sample recall include:

[0028] Cluster the samples in the SQL sample vector library offline, and recall and integrate samples from different categories.

[0029] Furthermore, the database information is first encoded and vectorized using a vectorized model and stored in a vectorized database. Similarly, a certain set of questions and SQL sample sets are prepared, and then the samples are divided into different categories using a clustering algorithm and labeled. The vectorized model is then used to encode and vectorize the information and store it in a vectorized database, waiting for the system to be online and relevant information to be retrieved from the database. Then, keyword matching, BM25, and vector similarity recall methods are used to recall a large number of relevant samples from each type of sample. The dynamic weighting method is used to automatically adjust the weights of various recall strategies according to the specific query and context, and the recall results are reordered to ensure that the most relevant samples of each sample are obtained.

[0030] An ICL large language model data query generation system based on diversity SQL enhancement, characterized by comprising:

[0031] The target query database is used to obtain the database schema;

[0032] SQL sample set, used to store questions and the verified correct SQL statements corresponding to the questions, to provide effective auxiliary information for subsequent models to generate new question SQL;

[0033] Vector database, a database system for efficient storage, retrieval and management of high-dimensional vector data;

[0034] Vectorization model: In the field of natural language processing (NLP), vectorization models are used to convert text data into numerical vectors, providing a basis for subsequent text classification, sentiment analysis, machine translation, and question-answering system tasks.

[0035] A large language model refers to a complex neural network model that is trained on massive amounts of text data and has hundreds of millions or even tens of billions of parameters. It is used to generate coherent text and can demonstrate amazing generalization and comprehension capabilities in various natural language processing tasks.

[0036] Furthermore, the database Schema includes table names, fields, field types and their relationship information in the database.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] 1. The present invention can significantly improve the efficiency and accuracy of data query, and can quickly adapt to new tasks and fields with only a small number of examples or instructions.

[0039] 2. The present invention adopts a multi-way recall method, which can simultaneously take advantage of the advantages of multiple recall mechanisms, thereby covering a wider range of sample types and improving the comprehensiveness of the recall. Different recall strategies may perform differently in different scenarios. The dynamic weighting method can automatically adjust the weights of various recall strategies according to the specific query and context to optimize the quality and relevance of the recall results.

[0040] 3. The present invention not only considers the semantic information of the text, but also incorporates the structural information of the database schema, making the vector representation more comprehensive and specific. The method first uses pre-trained BERT or other pre-trained models to convert SQL samples into vector representations rich in semantic information. Afterwards, a structure-aware embedding method is developed for the database schema to convert table names, fields and their relationships into vectors and fuse them with semantic vectors. Through joint training, the semantic and structure-aware vector representations are optimized to make them more consistent and effective in the same vector space. This method can significantly improve the accuracy and richness of vector representations, thereby improving the accuracy of sample recall.

[0041] 4. First, cluster the samples into different categories. When new questions need to be generated, the most relevant samples can be recalled from each category to balance the relevance and diversity of the recalled samples. The balance between relevance and diversity can be adjusted by controlling the number of clusters. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is an example of a random recall method.

[0043] Figure 2 is an example of a k-nearest neighbor recall method.

[0044] Figure 3 is an ideal recall example.

[0045] Figure 4 It is the technical principle and system framework of the present invention.

[0046] Figure 5 It clusters the samples in the sample library offline. Points of different colors (SQL samples) represent different clustering categories.

[0047] Figure 6 It is to recall examples from different categories separately and integrate the examples.

[0048] Figure 7 This is a schematic diagram of Schema information vectorization and storage.

[0049] Figure 8 This is a schematic diagram of SQL question and answer vectorization and storage of sample information.

[0050] Fig. 9 This is a diagram showing the recall of schema and SQL examples.

[0051] Fig.10 This is a schematic diagram of the clustering-based recall method. DETAILED DESCRIPTION

[0052] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0053] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0054] The present invention will be described in detail below in conjunction with exemplary embodiments with reference to the accompanying drawings.

[0055] Example 1

[0056] refer to Figures 1 to 10 The present invention provides a method for generating ICL large language model data query based on diversity SQL enhancement, the method comprising the following steps:

[0057] S1. Determine the target query database and obtain the database schema;

[0058] S2, vectorize the database schema and store it in the database metadata vector library;

[0059] S3, vectorizing the SQL statements in the SQL sample set and storing them in the SQL sample vector library;

[0060] Specifically, in step S3, the specific steps of vectorizing the SQL statements in the SQL sample set include:

[0061] S31, disassemble the query questions and SQL statements in the SQL sample set;

[0062] S32, vectorizing the decomposed query questions and SQL statements respectively by sentence embedding;

[0063] S33, mapping the above vectorized query questions and SQL statements one by one and storing them in a vector library.

[0064] More specifically, in step S33, the mapping method is as follows: the question character string, the question vector, and the SQL statement are mapped by inserting the same piece of data into the vector library.

[0065] S4, clustering the information stored in the SQL sample vector library to retrieve diverse samples;

[0066] Specifically, in step S4, the specific steps of clustering diversity sample recall include:

[0067] Cluster the samples in the SQL sample vector library offline, and recall and integrate samples from different categories.

[0068] S5. Based on the database summary and the user's natural language questions, combined with the recalled sample questions and their corresponding SQL query statements, a comprehensive prompt information is constructed as model reference information to guide the large language model to generate SQL queries that conform to semantics and syntax;

[0069] S6. Input the large language model, and the large language model generates SQL statements.

[0070] Furthermore, in step S2, the specific steps of vectorizing the database schema include:

[0071] S21, obtain the table name and column name information in the database Schema;

[0072] S22. Use word embedding technology to vectorize table names and column names respectively;

[0073] S23. Store the above vectorized information into a vector library and map it with the original table name and column name.

[0074] Specifically, in step S23, the mapping method is specifically completed by inserting the table name, column name, and vector into the database in the same data.

[0075] In the present invention, the database information is first encoded and vectorized using a vectorization model and stored in a vectorized database. Similarly, a certain set of questions and SQL sample examples are prepared. Then, a clustering algorithm is used to divide the samples into different categories and label them. Then, the vectorization model is used to encode and vectorize the information and store it in a vectorized database. The relevant information is retrieved from the database when the system is running online.

[0076] Among them, the technical implementation type of clustering algorithm, due to the use of deep learning framework for clustering, there is no specific and clear formula here, clustering algorithm, (clustering algorithm is a type of unsupervised learning method, used to group objects in a data set into different clusters, so that the similarity of objects in the same cluster is high, while the similarity of objects in different clusters is low. Common clustering algorithms include: K-means clustering, which iteratively assigns data points to the nearest cluster center and updates the position of the cluster center until convergence; Gaussian mixture model, assuming that the data is a mixture of multiple Gaussian distributions, and the expectation maximization (EM) algorithm is used to estimate the parameters of each Gaussian distribution.)

[0077] Example 2

[0078] The present invention provides an ICL large language model data query generation system based on diversified SQL enhancement, comprising:

[0079] The target query database is used to obtain the database schema;

[0080] SQL sample set, used to store questions and the verified correct SQL statements corresponding to the questions, to provide effective auxiliary information for subsequent models to generate new question SQL;

[0081] Vector database, a database system for efficient storage, retrieval and management of high-dimensional vector data;

[0082] Vectorization model: In the field of natural language processing (NLP), vectorization models are used to convert text data into numerical vectors, providing a basis for subsequent text classification, sentiment analysis, machine translation, and question-answering system tasks.

[0083] A large language model refers to a complex neural network model that is trained on massive amounts of text data and has hundreds of millions or even tens of billions of parameters. It is used to generate coherent text and can demonstrate amazing generalization and comprehension capabilities in various natural language processing tasks.

[0084] The database Schema includes table names, fields, field types and their relationship information in the database.

[0085] The general recall method only considers the relevance of samples, which is likely to lead to highly consistent recall samples, resulting in information redundancy, and cannot fully guide the model to correctly generate SQL. For example, the current question is: What is the year-on-year increase in Liqun's sales in 2023? The most similar recall sample is: What is Liqun's sales in 2023? What is Liqun's sales in 2024? What is Liqun's commercial sales in 2024? ...What is Liqun's sales in January 2024? Too much redundant information, but lack of year-on-year increase calculation logic samples, will inevitably affect the model's better generation of SQL.

[0086] Figure 5 In the example library, the samples are clustered offline. The points of different colors (SQL samples) in the figure below are different clustering categories. Samples from different categories are recalled and integrated to avoid information redundancy caused by too many similar samples. At the same time, diverse samples provide more necessary information for the model.

[0087] The specific implementation of the present invention is as follows:

[0088] Working Offline

[0089] First, the database information is encoded and vectorized using the vectorization model and stored in the vectorized database. A certain set of questions and SQL sample sets are also prepared. Then, the samples are divided into different categories using the clustering algorithm and labeled. The vectorization model is then used to encode and vectorize the information and store it in the vectorized database. The relevant information is retrieved from the database when the system is running online.

[0090] Online operation

[0091] The user asks a query in natural language, for example, "Show the names of all users and the number of their orders". This natural language query is encoded using a vectorized model, and the encoded vector recalls the relevant database schema information from the vectorized database through similarity and other methods. Then, relevant QA sample pairs are recalled from the sample vector database. Different recall methods recall different samples, which will have a direct impact on the model generation effect. Therefore, this part is the core link of the generation link. In this system, the question vector is used to extract the most relevant SQL samples from various samples in the vector database by multi-route retrieval and result re-ranking methods. Then, based on the database overview and the user's natural language questions, the system combines the recalled sample questions and their corresponding SQL query statements to provide the model with reference information such as similar conversion examples, and constructs a comprehensive prompt information (Prompt). This prompt contains relevant database structure information and past similar query examples, so as to guide the large language model to generate SQL queries that conform to semantics and syntax. Input the large language model, and the large language model generates SQL statements.

[0092] Core Features

[0093] As mentioned above, recalling appropriate task examples is crucial to the final SQL generation effect. Therefore, an innovative search method is proposed in the present invention to improve the quality of recalled examples and thus improve the system generation effect.

[0094] Importance of recall example relevance

[0095] Improve precision: If the retrieved examples are highly relevant to the current question, the generation module can extract more useful information from them and generate more accurate answers.

[0096] Reduce error rate: Highly relevant examples can reduce the risk of introducing noise and erroneous information during the generation process, ensuring that the answers focus on user needs.

[0097] Importance of recall example richness

[0098] Comprehensiveness of information: Rich recall examples provide diverse sources of information, enabling the generation module to integrate various viewpoints and data in its responses and provide comprehensive and balanced answers.

[0099] Improve generation quality: Rich sample recall enables the generation module to cover a wider range of SQL and business knowledge points, especially when dealing with complex or multi-level problems, helping to build more accurate answers.

[0100] Text vectorization

[0101] In order to improve the accuracy and richness of sample representation in vector databases, the present invention proposes a vector representation method that integrates semantic understanding and structure perception. This method not only considers the semantic information of the text, but also incorporates the structural information of the database schema, making the vector representation more comprehensive and specific. The method first uses pre-trained BERT or other pre-trained models to convert SQL samples into vector representations rich in semantic information. Afterwards, a structure-aware embedding method is developed for the database schema to convert table names, fields and their relationships into vectors and fuse them with semantic vectors. By means of joint training, the semantic and structure-aware vector representations are optimized to make them more consistent and effective in the same vector space. This method can significantly improve the accuracy and richness of vector representations, thereby improving the accuracy of sample recall.

[0102] Ideal retrieval method requirements

[0103] In the ICL technical framework, the relevance and richness of the samples work together to determine the quality of the final generated SQL. This two-pronged approach can achieve the best response to user queries and provide high-quality natural language processing solutions. Therefore, it is crucial to have a reasonable recall method to balance the relevance and richness of the recalled samples.

[0104] Cluster recall principle

[0105] In order to better balance the relevance and diversity of recalled samples, this method proposes a clustering-based recall method. The method first clusters the samples into different categories. When new questions need to be generated, the most relevant samples can be recalled from each category of samples to balance the relevance and diversity of recalled samples. By controlling the number of clusters, the balance between relevance and diversity can be adjusted.

[0106] In order to better balance the relevance and diversity of recalled samples, this method proposes a clustering-based recall method. The method first clusters the samples into different categories. When new questions need to be generated, the most relevant samples can be recalled from each category of samples to balance the relevance and diversity of recalled samples. By controlling the number of clusters, the balance between relevance and diversity can be adjusted.

[0107] 1. Construct a feature vector based on the query conditions and query target related parts in the natural language question, and cluster the above feature vectors through a clustering algorithm.

[0108] 2. According to the new natural language problem, a sorting and sampling algorithm is added to the original clustering algorithm for multi-classification recall content to realize the recall of relevant samples in multi-classification content.

[0109] Dynamically weighted multi-way recall

[0110] In order to better recall the most relevant examples from each type of examples, this method proposes a dynamic weighted multi-way recall method. The method first uses recall methods such as keyword matching, BM25, and vector similarity to recall a large number of relevant examples from each type of examples. Through the dynamic weighting method, the weights of various recall strategies are automatically adjusted according to the specific query and context, and the recall results are re-sorted to ensure that the most relevant examples are obtained.

[0111] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for generating ICL large language model data queries based on diversified SQL enhancement, characterized in that: The method comprises the following steps: S1. Determine the target query database and obtain the database schema; S2, vectorize the database schema and store it in the database metadata vector library; S3, vectorizing the SQL statements in the SQL sample set and storing them in the SQL sample vector library; S4, clustering the information stored in the SQL sample vector library to retrieve diverse samples; S5. Based on the database summary and the user's natural language questions, combined with the recalled sample questions and their corresponding SQL query statements, a comprehensive prompt information is constructed as model reference information to guide the large language model to generate SQL queries that conform to semantics and syntax; S6. Input the large language model, and the large language model generates SQL statements.

2. The ICL large language model data query generation method based on diversity SQL enhancement as claimed in claim 1, characterized in that: In step S2, the specific steps of vectorizing the database schema include: S21, obtain the table name and column name information in the database Schema; S22. Use word embedding technology to vectorize table names and column names respectively; S23. Store the above vectorized information into a vector library and map it with the original table name and column name.

3. The ICL large language model data query generation method based on diversity SQL enhancement as claimed in claim 2 is characterized in that: In step S23, the mapping method is specifically completed by inserting the table name, column name, and vector into the database as a single piece of data.

4. The ICL large language model data query generation method based on diversity SQL enhancement as claimed in claim 1, characterized in that: In step S3, the specific steps of vectorizing the SQL statements in the SQL sample set include: S31, disassemble the query questions and SQL statements in the SQL sample set; S32, vectorizing the decomposed query questions and SQL statements respectively by sentence embedding; S33, mapping the above vectorized query questions and SQL statements one by one and storing them in a vector library.

5. The ICL large language model data query generation method based on diversity SQL enhancement as claimed in claim 4 is characterized in that: In step S33, the mapping method is specifically as follows: the question character string, the question vector, and the SQL statement are inserted into the vector library as a same piece of data to complete the mapping.

6. The ICL large language model data query generation method based on diversity SQL enhancement as claimed in claim 1, characterized in that: In step S4, the specific steps of clustering diversity sample recall include: Cluster the samples in the SQL sample vector library offline, and recall and integrate samples from different categories.

7. The ICL large language model data query generation method based on diversity SQL enhancement as claimed in claim 6, characterized in that: First, the database information is encoded and vectorized using the vectorization model and stored in the vectorized database. A certain set of questions and SQL sample sets are also prepared. Then, the samples are divided into different categories using the clustering algorithm and labeled. Then, the vectorization model is used to encode and vectorize the information and store it in the vectorized database. When the system is online, the relevant information is retrieved from the database.

8. An ICL large language model data query generation system based on diversity SQL enhancement, characterized in that: include: The target query database is used to obtain the database schema; SQL sample set, used to store questions and the verified correct SQL statements corresponding to the questions, to provide effective auxiliary information for subsequent models to generate new question SQL; Vector database, a database system for efficient storage, retrieval and management of high-dimensional vector data; Vectorization model: In the field of natural language processing (NLP), vectorization models are used to convert text data into numerical vectors, providing a basis for subsequent text classification, sentiment analysis, machine translation, and question-answering system tasks. A large language model refers to a complex neural network model that is trained on massive amounts of text data and has hundreds of millions or even tens of billions of parameters. It is used to generate coherent text and can demonstrate amazing generalization and comprehension capabilities in various natural language processing tasks.

9. The ICL large language model data query generation system based on diversity SQL enhancement as claimed in claim 8, characterized in that: The database Schema includes table names, fields, field types and their relationship information in the database.

Images