LLM and vector model-based Text2SQL intelligent question and answer query method and system

Abstract

The invention discloses a Text2SQL intelligent question and answer query method and system based on an LLM and a vector model, and aims to construct an interactive joint model with the LLM as a core and the vector model as an auxiliary, and realize an intelligent Text2SQL question and answer query chain through multi-dimensional data representation, task decoupling, model multi-task synchronous fine tuning and link construction. The method includes collaboration of multiple stages of SQL generation and execution, answer generation and the like, so that query requirements expressed by a user in a natural language are converted into professional MySQL query statements. According to the method, a complex Text2SQL task is disassembled into a plurality of relatively simple tasks through task decoupling design and link construction, task node scheduling is optimized through link construction, and fault-tolerant processing is carried out for model generation abnormity; meanwhile, in order to assist the user in checking, each subtask result is displayed at the front end, and the subtask results are editable, so that the final result is traceable.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a Text2SQL intelligent query question-answering method and system based on LLM and vector model. Background Art

[0002] With the rapid development of digitalization in various industries, how to efficiently and intelligently query, understand and analyze increasingly complex data tables has become an urgent problem to be solved. Traditional manual queries or keyword-based query systems are no longer sufficient to meet the needs of efficient and complex data retrieval. Early Seq2Seq models cannot accurately understand the context and model the implicit logic and alignment relationship between natural language questions and databases. As a breakthrough technology in the field of natural language processing, LLM (Large Language Models) has the ability of context learning, logical reasoning and text generation, which provides a new solution for the effective use of complex data information. Some scholars have specially developed or fine-tuned LLMs for the field of Text2SQL, such as the sqlcoder series, Llama-3-SQLCoder-8B and other models, aiming to improve the accuracy of Text2SQL. The input of these models consists of questions and data table descriptions, but there are many data tables in actual scenarios. The input restrictions of the model make it impossible for the model to obtain all the data table information. Even if the input length is increased, the input noise of the model will increase accordingly. Moreover, this kind of one-step method has extremely high requirements for the model and is very difficult to apply in practice.

[0003] At present, research on Text2SQL question-answering systems is mainly focused on improving the accuracy of natural language to SQL statements. However, natural language is highly complex and ambiguous. The model needs to accurately understand the database model, including the meaning of tables and columns and the relationship between them, while accurately parsing user intentions, in order to convert them into correct SQL statements. Often, due to the deviation between training input and actual application data, it is difficult for the model to directly convert natural language into query statements. In addition, there are some open source systems, such as DB-GPT and Vanna-ai. Although they have built question-answering systems based on LLM data tables, such systems lack optimization of the model itself and cannot guarantee good query accuracy through simple link implementation. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a Text2SQL intelligent query question-answering method and system based on LLM and vector model, aiming to build an interactive joint model with LLM as the core and vector model as the auxiliary, and realize intelligent Text2SQL query question-answering chain through multi-dimensional data representation, task decoupling, model multi-task synchronous fine-tuning and link construction, including user data table recall, time extraction, SQL generation and execution, answer generation and other stages of collaboration, so as to convert the user's query requirements expressed in natural language into professional MySQL query statements.

[0005] The technical solution provided by the present invention is as follows: A Text2SQL intelligent query and answer method based on LLM and vector model, comprising the following steps: S1. Construct a vector model fine-tuning dataset and use the dataset to fine-tune the vector model, where the vector model is used to map any input text into a low-dimensional dense vector; S2. Construct a multi-task LLM fine-tuning dataset and use the dataset to fine-tune the LLM, wherein the LLM is used to generate corresponding output content according to the input prompt, and the multi-task includes a time extraction task, a Text2SQL task, and an answer generation task; S3. Build the Text2SQL question-answer template vector library and data table vector database; S4. Input the user's query question into the fine-tuned vector model for vectorization, and retrieve the Text2SQL question-answering template most similar to the query question as the reference question-answering template and the top 3 data tables related to the query question as the reference data table based on the vector similarity. At the same time, extract the time-related expression from the query question based on the fine-tuned LLM, which is called the time expression. S5. Obtain description information of the reference data table from the SQL database according to the data table name, construct a query prompt by combining the description information, the user's query question, the extracted time expression and the reference example, and input the query prompt into the fine-tuned LLM to infer and generate a corresponding SQL statement, execute the SQL statement and obtain the execution result; S6. Based on the user's query question, the query data table, the generated SQL statement and the execution result, an answer prompt is constructed, and the answer prompt is input into the fine-tuned LLM to generate the answer content and feedback it to the user.

[0006] Furthermore, in step S1, the vector model fine-tuning data set includes a number of sample data, each sample data includes a query question, a text list related to the query question, and a text list unrelated to the query question; the text list related to the query question is called a positive sample, and the text list unrelated to the query question is called a negative sample. The difficult samples contained in the negative samples are obtained by using a difficult example mining method: all samples are vectorized using an unfine-tuned vector model and then entered into a vector database, and the questions in the question set are vectorized separately and similar samples are recalled in the vector database. Samples whose recall values ​​exceed a set threshold are difficult samples after removing positive samples.

[0007] Furthermore, in step S2, the LLM fine-tuning dataset is created according to the task type: the dataset corresponding to the time extraction task comes from the model generation, forming a question-answer pair for extracting time from the query question; the dataset corresponding to the Text2SQL task is data enhanced based on the large language model; the dataset corresponding to the answer generation task is labeled and generated according to user preferences; it also includes a mixed open source dataset to alleviate the model forgetting problem, and a dataset about model cognition generated to achieve model self-cognition; all datasets are merged to obtain the LLM fine-tuning dataset, and different system prompts are constructed according to each task as a distinction between different tasks, and then the LLM model is fine-tuned using the LLM fine-tuning dataset.

[0008] Furthermore, in step S2, the fine-tuning model is evaluated according to the task type: for the time extraction task, the consistency between the extraction result and the annotation is calculated. Acc time For the Text2SQL task, the correct matching rate is used Acc m and execution accuracy Acc e The weighted Acc sql , for answer generation tasks, manual scoring is used Acc answer ; Then the evaluation results of each task are weighted and summed to get the final model score Acc , select Acc Top fine-tuned models.

[0009] Furthermore, in step S3, representative data sets are selected from the fine-tuned data sets and the self-constructed data sets, and questions, data table information, time information, and SQL statements are recorded, and vectorized using the fine-tuned vector model to construct a Text2SQL question and answer template vector library; a data table information-related knowledge base is constructed, and the data table name, data table column interpretation, and data table overall description are recorded, and vectorized using the fine-tuned vector model to construct a data table vector database.

[0010] Furthermore, in step S5, for the string text generated by the fine-tuned LLM reasoning, select is used as a keyword to extract SQL statements; after the SQL statements are extracted, the SQL database is connected, the SQL statements are executed to perform SQL queries, and the returned execution results are parsed into a list form.

[0011] Furthermore, in step S5, if an SQL statement is executed incorrectly, the current SQL statement and the execution error are used as reference information, and the prompt is reconstructed and input into the fine-tuned LLM to generate the SQL statement. After the SQL statement is generated, the SQL statement is executed again. If there is still an execution error, the error information is returned as the SQL statement execution result.

[0012] Furthermore, in step S6, if the SQL statement cannot be generated, it means that the user's query question does not require SQL query, and LLM only needs to answer the query question; if the SQL statement is generated, the SQL query is executed. When the SQL statement execution error and it is the first error, the SQL statement is regenerated. If it is not the first time, the user needs to be informed that the SQL generation failed and provide feasible suggestions; when the SQL statement is executed correctly, it is determined whether the execution result is empty. If it is empty, it is necessary to combine the user's question and the SQL statement to inform the user which queries have been made but there is no relevant data in the data table. If it is not empty, it is necessary to further determine whether the number of rows and columns of the result exceeds the threshold. Data exceeding the threshold cannot be fully displayed, and the user needs to be informed of the actual amount of data queried. If it does not exceed the threshold, the user's query question, SQL statement, SQL statement execution result and data table information are constructed into prompts and input into the fine-tuned LLM to obtain the final answer.

[0013] Furthermore, it also includes building a Web front-end service: completing the process in steps S4~S6 through task scheduling, and visualizing the process subtask results by the front-end, and the user verifies the accuracy of each process subtask result through the front-end; the subtask results support editing, and the subsequent process will be re-executed after the subtask results are edited.

[0014] A Text2SQL intelligent query and answer system based on the above method includes: A vector model module, comprising a vector model data set construction module and a vector model fine-tuning module, wherein the vector model data set construction module is used to construct a vector model data set, and the vector model fine-tuning module is used to perform fine-tuning training on the vector model according to the vector model data set; The LLM module includes a multi-task dataset construction module and an LLM model fine-tuning module. The multi-task dataset construction module is used to construct a multi-task dataset suitable for LLM, and the LLM model fine-tuning module is used to fine-tune the LLM according to the multi-task dataset; Vector database building module, including Text2SQL question-answer template vector library and data table vector database; The intention construction module includes a query vectorization module, a retrieval matching module and a time expression extraction module. The query vectorization module is used to input the user's query into the fine-tuned vector model for vectorization; the retrieval matching module retrieves the Text2SQL question and answer template most similar to the query as a reference question and answer template, and the data table with the top 3 relevance to the query as a reference data table in the constructed Text2SQL question and answer template vector library and data table vector database according to vector similarity; the time expression extraction module is used to input the query into the fine-tuned LLM to extract the time expression; the query prompt construction module is used to obtain the description information of the reference data table from the index in the SQL database, and construct the query prompt in combination with the description information, the user's query, the extracted time expression and the reference example; The SQL statement generation, execution and feedback module is used to input the query prompt into the fine-tuned LLM to infer and generate the corresponding SQL statement, and is also used to execute the SQL statement to obtain the execution result, and construct the answer prompt based on the user's query question, the queried data table, the generated SQL statement and the execution result; it is also used to input the answer prompt into the fine-tuned LLM to generate the answer content and feedback it to the user.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects: (1) Multi-dimensional characterization of data, using high-quality data and combined with difficult sample mining to fine-tune the vector model, improves the accuracy of the vector model for data table positioning and the most relevant few-shot recall; (2) Through task decoupling design and link construction, the complex Text2SQL task is decomposed into multiple relatively simple tasks. Through link construction, the task node scheduling is optimized and fault tolerance is performed for model generation exceptions. (3) For the multi-task synchronous fine-tuning of LLM, the fine-tuning dataset is enhanced and optimized to improve the knowledge intake effect of the model's instruction learning during the fine-tuning process. Dora fine-tuning, which is closer to the full fine-tuning effect than LoRA, is used to fine-tune the Qwen2-72B-Chat model to improve the performance of the model on various tasks. (4) According to the different types of query data tables, MySQL statements and query results, prompt templates are designed to convert query results into natural language, so that users' natural language questions can be answered in more accurate and controllable natural language. At the same time, to assist users in verification, the results of each subtask are displayed on the front end, making the subtask results editable and the final results traceable. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0017] Figure 1 It is a flowchart of a Text2SQL intelligent query and answer method provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of the Text2SQL query, execution and optimization process provided by an embodiment of the present invention; Figure 3 The present invention is a flowchart of SQL generation and execution result prompt generation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, other embodiments obtained by ordinary technicians in this field without making creative work are all within the scope of protection of the present invention.

[0019] Embodiment 1 This embodiment provides a Text2SQL intelligent query and answer method based on LLM and vector model. Figure 1 As shown in the figure, the main process is as follows: first, the data table is represented in multiple dimensions, the data table information is characterized from multiple angles, the data table information and the question-answering dataset are stored in a relational database (MySQL) and a vector database (Milvus) respectively, and two types of fine-tuning datasets are initially formed based on the collected data, which are used for fine-tuning the LLM and vector model respectively; the full-scale fine-tuning vector model bge-large-zh-noinstrct and Dora fine-tuning LLM Qwen2-72B-Chat are fine-tuned, and the fine-tuned vector model and LLM are deployed using xinference and vLLM respectively; based on the fine-tuned model, the question-answering chain is constructed, the Text2SQL task is decomposed into the collaboration of multiple simple tasks, the scheduling between task nodes is optimized, and the delay is reduced while optimizing the effect; finally, according to the user's questions, the query data table information, the SQL statement and the execution result, and the classification of their types, differentiated prompts are designed to provide users with diversified answers. The subtask results are visualized and editable, so as to realize the verification and fine-tuning of the results.

[0020] The specific steps are as follows: S1. Construct a problem set and characterize the data table information in the database in multiple dimensions to form a fine-tuning dataset for the vector model. The dataset consists of about 20,000 data items in the form of {"query": str, "pos": List[str], "neg":List[str]}, where query is the query question, pos is a list of texts related to the question, and neg is a list of negative texts (i.e., a collection of samples that are not related to the question). The formation of negative samples adopts the method of difficult example mining. Specifically, all samples are quantized using the unfine-tuned vector model and then entered into the vector database. The questions in the problem set are vectorized separately and similar samples are recalled. After recalling samples that exceed the set threshold, the positive samples are removed, and the resulting samples are difficult samples. The difficult samples are mixed with other negative samples to obtain the final negative samples. The fine-tuning model selected is bge-large-zh-noinstruct from the BEG (BAAI General Embedding) series (a large model open sourced by the Beijing Academy of Artificial Intelligence that can map any text into a low-dimensional dense vector). The model performs better than other models in the current task before and after fine-tuning. The evaluation indicator is mainly MRR (Mean Reciprocal Rank).

[0021]

[0022] in, is the total number of queries, is the number of content sets, It is The correct ranking of queries. This embodiment uses the MRR@Top3 indicator, focusing on the model's ability to hit the most relevant sample among the top three recalled samples. The effect of the fine-tuned vector model was verified on the test set, and the effect was compared with the unfine-tuned model and the mainstream third-party vector model. In the data table positioning task, the MRR@Top3 of the fine-tuned bge-large-zh-noinstruct was 92.2%, and the MRR@Top3 of the model without fine-tuning was 43.6%. The MRR@Top3 on Baidu's vector model Embedding-V1 and Zhipu's Embedding-2 were 48.2% and 49.3%, respectively. In the similar text recall task, the MRR@Top3 of the fine-tuned bge-large-zh-noinstruct was 96.8%, and the MRR@Top3 of the model without fine-tuning was 57.7%. The MRR@Top3 on Baidu's vector model Embedding-V1 and Zhipu's Embedding-2 were 69% and 72.4%, respectively. It can be seen that the fine-tuned model has greatly improved in semantic retrieval and related data tables, providing reliable input for subsequent SQL generation tasks, thereby improving the overall accuracy. The fine-tuned model was deployed on the distributed reasoning framework Xinference.

[0023] S2. Fine-tune the base model Tongyi Qianwen Qwen2-72B-Chat in multiple tasks simultaneously (Tongyi Qianwen Qwen-72B-Chat is a model with 72 billion parameters developed by Alibaba Cloud). The fine-tuning dataset is created according to the task type. The time extraction task comes from model generation, forming a question-answer pair that extracts time from the question. To enhance data diversity, the time types of questions include year, quarter, month, week, day, etc. The dimensions include past, current, future, and composite dimensions. The extraction difficulty is divided into direct extraction and inferred extraction. The dataset collection for the Text2SQL task is relatively difficult. Therefore, with the help of data table information and existing questions, data enhancement can be performed based on LLM. After enhancement, data cleaning is performed to filter out low-quality data and build a fine-tuning dataset for Text2SQL. The dataset for the answer generation task is annotated and generated according to user preferences. Similarly, based on a small number of examples, it can be enhanced using LLM. In addition to the datasets for the task, in order to alleviate the problem of model forgetting during the fine-tuning process, open source datasets are mixed. At the same time, in order to achieve new self-cognition of the model, a dataset about model cognition is generated. The distribution ratio of the dataset is roughly as follows: time extraction accounts for 30%, Text2SQL accounts for 35%, answer generation accounts for 22%, open source data accounts for 10%, and self-awareness accounts for 3%. All datasets are merged, and different system prompts are constructed according to each task as a distinction between different tasks. The model is fine-tuned using the fine-tuning dataset.

[0024] The Text2SQL task requires a high level of reasoning ability of the model, and the ability of the model is often positively correlated with the number of model parameters. This embodiment uses Qwen2-72B-Chat for vector fine-tuning. Low-Rank Adaptation (LoRA) is the most common fine-tuning method currently. It only needs to adjust a small number of model parameters and can quickly adapt to new tasks while maintaining the model fine-tuning performance. However, it lacks the ability to make subtle changes in direction, which may lead to suboptimal fine-tuning results. Therefore, this embodiment uses Dora fine-tuning (Weight-Decomposed Low-Rank Adaptation), which is done by adjusting the weights To decompose, ,in, is the magnitude vector, is the direction matrix, represents the vector norm of the matrix on each column, m The corresponding scalar in defines the magnitude of each vector. During fine-tuning, the direction matrix is ​​first frozen. is a trainable vector, and then the direction matrix is ​​updated through LoRA.

[0025] , in, is the fine-tuned weight, are the pre-trained weights, is to transform two low-rank matrices into and The direction update increment learned by multiplication, the overline represents the trainable parameters. DoRA can be merged with the pre-trained weights before inference, so it does not introduce any additional overhead.

[0026] This embodiment designs an evaluation scheme for multi-task parallel models to select the best model in the fine-tuning process. The specific scheme is to first evaluate different tasks separately, and for the time extraction task, calculate the consistency between the extraction result and the annotation. Acc time , for SQL generation, the correct matching rate is used Acc m and execution accuracy Acc e The weighted Acc sql ,Right now ,in, , . Acc mIn order to ensure that the generated SQL statements can accurately match the proportion of annotated SQL statements, the generated SQL statements and standard SQL statements are split into multiple clauses according to keywords. If the clause sets corresponding to the two clauses are the same, it means that the two clauses are the same. If all clauses are consistent, it means that the generated SQL statement is correct. Acc e The percentage of correct SQL statement execution results. The combination of the two can more accurately evaluate the SQL accuracy. The answer generation is evaluated by manual scoring, which is recorded as Acc answer ,right Acc answer After obtaining the above results, weighting is performed, that is, the final , in, , , .Will The top model was deployed using vLLM (Vectorized LargeLanguage Model Serving System, an open source LLM reasoning and serving framework from the LMSYS organization at the University of Berkeley).

[0027] S3. Build a Text2SQL query chain to convert the query requirements expressed in natural language into professional query statements and automatically perform SQL query optimization. The specific steps are as follows: S3.1 Build a Text2SQL question-answering template library. Select representative data sets from the fine-tuned data sets and self-built data, and record questions, data table information, time information, and SQL statements. In addition, build a knowledge base related to data table information to record data table names, data table column definitions, and overall data table descriptions. The above data are vectorized using the fine-tuned vector model and stored in the Milvus vector database, denoted as M q and M t , M q A collection library for vectorizing question and answer templates. M t A collection library for vectorizing data tables.

[0028] S3.2 Input the user question into the fine-tuned vector model and LLM simultaneously, use LLM to extract the time in the question, and use the vector model to vectorize the question. The result is recorded as E q .Will E q Respectively M qThe vector and M t The vectors in the similarity calculation are used to recall M t TOP3 and M q This step simultaneously completes the tasks of time extraction, data table location, and example recall.

[0029] S3.3 After deduplication, the recalled TOP3 data tables are indexed to their schema information in the SQL database by table name (schema includes table definition, field name and type, and relationship between data. It defines elements such as tables, views, indexes, columns, data types, constraints, and relationships in the database). The recalled TOP2 question-answer template will be used as a few shot example for subsequent SQL generation. The user question, time extraction results, schema information, and recalled few shots are combined to form prompts for SQL generation, which are input into the fine-tuned LLM. With the help of LLM's contextual learning ability, RAG (Retrieval Augmented Generation) is used to generate SQL statements. Because the output of the model is in string format and noisy text may appear, it is necessary to extract query statements from the text generated by the model, using select as the keyword to extract SQL statements.

[0030] S3.4 After the SQL statement is extracted, connect to the SQL database, execute the SQL statement to perform SQL query, and parse the returned results and package them into a list format to facilitate subsequent unified processing. Errors may occur during SQL execution. If an error occurs during the initial execution, the model can be called again to generate SQL, and the current error SQL and execution error can be used as reference information to rebuild the prompts and input them into the fine-tuned LLM for SQL generation. After SQL generation, extract and execute the SQL statement again. If an error occurs again, the error will be returned as the SQL execution result. The above process is as follows Figure 2 shown.

[0031] S4. Users ask questions in natural language, and the question-answering system converts the query results into natural language for replies. Since users’ questions may not require table lookup, no results may be found, query errors may occur, or the dimensions of the query results may be different, we have designed the following: Figure 3The prompts shown are strongly related to SQL generation and execution results, which can make users feel the difference in answers in different situations. If SQL cannot be extracted, it means that there is no need for SQL query for this question, and LLM only needs to answer according to the question. If SQL can be extracted, SQL query is executed. When SQL execution error occurs and it is the first error, it is regenerated. If it is not the first time, the user needs to be informed that SQL generation failed and some feasible suggestions are provided, such as retrying. When SQL can be executed correctly, it is determined whether the SQL execution result is empty. If it is empty, it is necessary to combine the question and SQL to inform the user which queries have been made but there is no relevant data in the data table. If it is not empty, it is necessary to further determine whether the number of rows and columns of the result exceeds the threshold. The threshold is 50. Data exceeding the threshold cannot be fully displayed on the front end. The user needs to be informed of the actual amount of data queried and the part of the data actually displayed due to restrictions. If it does not exceed the threshold, the user's question, SQL statement, SQL query result and table information are constructed into prompts and input into the fine-tuned LLM to obtain the final answer. In this way, the user's natural language query is answered in a more accurate and controllable natural language.

[0032] S5. A web service is built around data table question and answer, which presents the model to users in a more intuitive way in the form of a question and answer system. Users input natural language, and the background completes the process described in S4 through task scheduling. The front end visualizes the subtask results, that is, users can verify the accuracy of each subtask. In addition, subtask results support editing, and subsequent process results can be regenerated after editing. For example, if there is an error in time extraction or table positioning, the user only needs to reselect the time period or reselect the data table related to the question to re-execute the SQL generation, execution, and answer generation tasks to obtain an answer that better meets the user's expectations. Users can achieve natural language data table query results by interacting with the designed system.

[0033] Embodiment 2 Based on the above method, this embodiment provides a Text2SQL intelligent query question-answering system, which mainly includes: The vector model module includes a vector model data set construction module and a vector model fine-tuning module. The vector model data set construction module is used to construct a vector model data set, and the vector model fine-tuning module is used to fine-tune the vector model according to the vector model data set.

[0034] The LLM module includes a multi-task dataset construction module and an LLM model fine-tuning module. The multi-task dataset construction module is used to construct a multi-task dataset suitable for LLM, and the LLM model fine-tuning module is used to fine-tune the LLM according to the multi-task dataset.

[0035] Vector database building module, including Text2SQL question and answer template vector library and data table vector database.

[0036] The intention construction module includes a query question vectorization module, a retrieval matching module and a time expression extraction module. The query question vectorization module is used to input the user's query question into the fine-tuned vector model for vectorization; the retrieval matching module retrieves the Text2SQL question and answer template that is most similar to the query question as a reference question and answer template, and the data table with the top 3 relevance to the query question as a reference data table in the constructed Text2SQL question and answer template vector library and data table vector database according to vector similarity; the time expression extraction module is used to input the query question into the fine-tuned LLM to extract the time expression; the query prompt construction module is used to index the description information of the reference data table from the SQL database, and construct the query prompt in combination with the description information, the user's query question, the extracted time expression and the reference example.

[0037] The SQL statement generation, execution and feedback module is used to input the query prompt into the fine-tuned LLM to infer and generate the corresponding SQL statement, and is also used to execute the SQL statement to obtain the execution result, and construct the answer prompt based on the user's query question, the queried data table, the generated SQL statement and the execution result; it is also used to input the answer prompt into the fine-tuned LLM to generate the answer content and feedback it to the user.

[0038] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A Text2SQL intelligent query and answering method based on LLM and vector model, characterized in that: Includes steps: S1. Construct a vector model fine-tuning dataset and use the dataset to fine-tune the vector model, where the vector model is used to map any input text into a low-dimensional dense vector; S2. Construct a multi-task LLM fine-tuning dataset and use the dataset to fine-tune the LLM, wherein the LLM is used to generate corresponding output content according to the input prompt, and the multi-task includes a time extraction task, a Text2SQL task, and an answer generation task; S3. Build the Text2SQL question-answer template vector library and data table vector database; S4. Input the user's query question into the fine-tuned vector model for vectorization, and retrieve the Text2SQL question-answering template most similar to the query question as the reference question-answering template and the top 3 data tables related to the query question as the reference data table based on the vector similarity. At the same time, extract the time-related expression from the query question based on the fine-tuned LLM, which is called the time expression. S5. Obtain description information of the reference data table from the SQL database according to the data table name, construct a query prompt by combining the description information, the user's query question, the extracted time expression and the reference example, and input the query prompt into the fine-tuned LLM to infer and generate a corresponding SQL statement, execute the SQL statement and obtain the execution result; S6. Based on the user's query question, the query data table, the generated SQL statement and the execution result, an answer prompt is constructed, and the answer prompt is input into the fine-tuned LLM to generate the answer content and feedback it to the user.

2. The Text2SQL intelligent query and answer method according to claim 1, characterized in that: In step S1, the vector model fine-tuning data set includes several sample data, each sample data includes a query question, a text list related to the query question, and a text list unrelated to the query question; the text list related to the query question is called a positive sample, and the text list unrelated to the query question is called a negative sample. The difficult samples contained in the negative samples are obtained by using a difficult example mining method: all samples are vectorized using an unfine-tuned vector model and then entered into a vector database, and the questions in the question set are vectorized separately and similar samples are recalled in the vector database. Samples whose recall values ​​exceed a set threshold are difficult samples after removing positive samples.

3. The Text2SQL intelligent query and answer method according to claim 1, characterized in that: In step S2, the LLM fine-tuning dataset is created according to the task type: the dataset corresponding to the time extraction task comes from model generation, forming a question-answer pair for extracting time from the query question; The dataset corresponding to the Text2SQL task is enhanced based on a large language model. The dataset corresponding to the answer generation task is annotated and generated according to user preferences. It also includes a mixed open source dataset to alleviate the model forgetting problem, and a dataset about model cognition generated to achieve model self-cognition. All datasets are merged to obtain the LLM fine-tuning dataset, and different system prompts are constructed according to each task to distinguish different tasks, and then the LLM model is fine-tuned using the LLM fine-tuning dataset.

4. The Text2SQL intelligent query and answer method according to claim 1, characterized in that: In step S2, the fine-tuning model is first evaluated according to the task type: for the time extraction task, the consistency between the extraction result and the annotation is calculated. Acc time For the Text2SQL task, the correct matching rate is used Acc m and execution accuracy Acc e The weighted Acc sql , for answer generation tasks, manual scoring is used Acc answer ; Then the evaluation results of each task are weighted and summed to get the final model score Acc , select Acc Top fine-tuned models.

5. The Text2SQL intelligent query and answer method according to claim 1, characterized in that: In step S3, representative data sets are selected from the fine-tuned data sets and self-built data, and questions, data table information, time information and SQL statements are recorded, and vectorized using the fine-tuned vector model to build a Text2SQL question and answer template vector library; a data table information-related knowledge base is built to record the data table name, data table column interpretation, and overall description of the data table, and vectorized using the fine-tuned vector model to build a data table vector database.

6. The Text2SQL intelligent query and answer method according to claim 1, characterized in that: In step S5, for the string text generated by the fine-tuned LLM reasoning, select is used as the keyword to extract the SQL statement; after the SQL statement is extracted, the SQL database is connected, the SQL statement is executed to perform SQL query, and the returned execution result is parsed into a list form.

7. The Text2SQL intelligent query and answer method according to claim 1, characterized in that: In step S5, if an SQL statement is executed incorrectly, the current SQL statement and the execution error are used as reference information, and the prompt is reconstructed and input into the fine-tuned LLM to generate the SQL statement. After the SQL statement is generated, the SQL statement is executed again. If there is still an execution error, the error information is returned as the SQL statement execution result.

8. The Text2SQL intelligent query and answer method according to claim 1, characterized in that: In step S6, if the SQL statement cannot be generated, it means that the user's query question does not require SQL query, and only LLM needs to answer the query question; if the SQL statement is generated, the SQL query is executed. If the SQL statement execution error is made and it is the first error, the SQL statement is regenerated. If it is not the first time, the user needs to be informed that the SQL generation failed and feasible suggestions are provided; When the SQL statement is executed correctly, determine whether the execution result is empty. If it is empty, it is necessary to combine the user's question and SQL statement to inform the user which queries have been made but there is no relevant data in the data table. If it is not empty, it is necessary to further determine whether the number of rows and columns of the result exceeds the threshold. Data exceeding the threshold cannot be fully displayed, and the user needs to be informed of the actual amount of data queried. If it does not exceed the threshold, the user's query question, SQL statement, SQL statement execution result and data table information are constructed into a prompt and entered into the fine-tuned LLM to obtain the final answer.

9. The Text2SQL intelligent query and answer method according to claim 1, characterized in that: It also includes building a Web front-end service: completing the process in steps S4~S6 through task scheduling, and visualizing the process subtask results by the front-end, and the user verifies the accuracy of each process subtask result through the front-end; the subtask results support editing, and the subsequent process will be re-executed after the subtask results are edited.

10. A Text2SQL intelligent query and answer system based on the method according to any one of claims 1 to 9, characterized in that: include: A vector model module, comprising a vector model data set construction module and a vector model fine-tuning module, wherein the vector model data set construction module is used to construct a vector model data set, and the vector model fine-tuning module is used to perform fine-tuning training on the vector model according to the vector model data set; The LLM module includes a multi-task dataset construction module and an LLM model fine-tuning module. The multi-task dataset construction module is used to construct a multi-task dataset suitable for LLM, and the LLM model fine-tuning module is used to fine-tune the LLM according to the multi-task dataset; Vector database building module, including Text2SQL question-answer template vector library and data table vector database; The intention construction module includes a query vectorization module, a retrieval matching module and a time expression extraction module. The query vectorization module is used to input the user's query into the fine-tuned vector model for vectorization; the retrieval matching module retrieves the Text2SQL question-answering template most similar to the query as a reference question-answering template and the data table with the top 3 relevance to the query as a reference data table from the constructed Text2SQL question-answering template vector library and data table vector database according to vector similarity; the time expression extraction module is used to input the query into the fine-tuned LLM to extract the time expression; The query prompt construction module is used to obtain the description information of the reference data table from the SQL database, and to construct the query prompt in combination with the description information, the user's query question, the extracted time expression and the reference example; The SQL statement generation, execution and feedback module is used to input the query prompt into the fine-tuned LLM to infer and generate the corresponding SQL statement, and is also used to execute the SQL statement to obtain the execution result, and construct the answer prompt based on the user's query question, the queried data table, the generated SQL statement and the execution result; it is also used to input the answer prompt into the fine-tuned LLM to generate the answer content and feedback it to the user.

Images