A text-to-sql system based on historical follow-up questions and fuzzy time rewriting

Abstract

This invention discloses a text-to-SQL system based on historical questioning and fuzzy time rewriting; it includes: a fuzzy time processing module and a historical questioning module; the fuzzy time processing module includes: obtaining the current system time as the base time; the system analyzes the user-input question pair using a predefined time template, which can identify various common fuzzy time expressions and extract relative time expression words; intelligently converting the identified fuzzy time expressions using a large language model, converting relative time into precise date and time expressions; verification and rewriting; the historical questioning module includes: after inputting a question, first calling the large model to query historical dialogues; generating rewritten questions using historical questioning nodes; and conducting follow-up questions, with the system generating appropriate follow-up content. This invention improves the accuracy and reliability of the system when handling time and date questions and multi-turn dialogues.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically a system for text-to-SQL (Structured Query Language) based on historical inquiry and fuzzy time rewriting. Background Technology

[0002] With the rapid development of artificial intelligence technology, Text-to-SQL technology has become an important research direction in the field of natural language processing. Text-to-SQL is a task designed to convert natural language questions into corresponding SQL queries that can be executed in relational databases. Formally, given a user question Q (also known as a user query, natural language question, etc.) and a database schema S, the goal of the task is to generate an SQL query Y that retrieves the desired content from the database to answer the user question. Text-to-SQL allows users to interact with databases using natural language without requiring expertise in SQL programming, thus potentially democratizing data access. By enabling less skilled users to easily retrieve target content from databases and facilitating more effective data analysis, this can benefit various fields such as business intelligence, customer support, and scientific research.

[0003] The core goal of Text-to-SQL technology is to transform users' natural language questions into executable SQL queries, thereby enabling efficient database queries. However, existing Text-to-SQL systems still face numerous challenges in practical applications, particularly in handling date and time questions and the inheritance of intent in multi-turn dialogues. On one hand, while date and time questions typically require only simple multi-step reasoning, existing large models still cannot reliably handle such problems, occasionally resulting in errors. In practice, to ensure accuracy and reliability, it is usually necessary to combine relevant tools (such as datetime libraries) to handle date and time questions, rather than relying entirely on large models for reasoning. On the other hand, in multi-turn dialogues, user intent may change or persist. Existing Text-to-SQL systems often fail to inherit and understand user intent well when handling multi-turn dialogues, leading to inaccurate generated SQL queries. These problems severely limit the widespread adoption and effective application of Text-to-SQL technology in real-world scenarios. Summary of the Invention

[0004] To address the problems existing in the background technology, this invention provides a text-to-SQL system based on historical questioning and fuzzy time rewriting. It implements a multi-turn dialogue system based on Text-to-SQL and its fuzzy time rewriting and historical questioning methods using the LangGraph architecture. This system significantly improves the accuracy and reliability of Text-to-SQL systems when handling date and time issues and multi-turn dialogues through fuzzy time rewriting and historical questioning methods. The technical solution includes: a fuzzy time processing module and a historical questioning module.

[0005] The fuzzy time processing module includes:

[0006] After the user enters the question, the system obtains the current system time as the base time to provide a reference point for subsequent time conversion;

[0007] The system uses a predefined time template to analyze the user-input questions. This template can identify various common ambiguous time expressions and extract relative time expression words.

[0008] The system uses a large language model to intelligently convert the identified fuzzy time expressions into precise date and time expressions.

[0009] Verify the accuracy of the time representation;

[0010] After verification, the ambiguous time in the original question will be rewritten as the precise time.

[0011] The historical questioning module includes:

[0012] After the input question is used, the large model is first called to query the history of the dialogue;

[0013] Use historical question nodes to generate rewritten questions;

[0014] Ask follow-up questions, and the system will generate appropriate follow-up questions.

[0015] This improved the accuracy of queries and the user experience.

[0016] During the conversion process from relative time to precise date and time, the system considers the current time as a reference point to ensure the accuracy of the conversion.

[0017] The historical dialogue queries are stored in a structured manner using entity-based software.

[0018] The historical inquiry nodes are analyzed to determine the correlation between the current question and the historical dialogue. By designing prompt word templates, the question is rewritten in a context-dependent manner using a large model.

[0019] The generation of follow-up questions includes: first, the system analyzes the completeness of the question by combining the database table structure information; second, the system determines whether follow-up questions are needed to obtain more information; and then, the system generates appropriate follow-up questions to ensure the relevance and effectiveness of the follow-up questions.

[0020] After the system generates appropriate follow-up questions, it decides whether to continue the SQL generation process based on the follow-up questions.

[0021] This improved the accuracy of queries and the user experience.

[0022] It can better understand users' query needs and provide more accurate query results.

[0023] After successful verification, the fuzzy time in the original question is rewritten as a precise time. Through the fuzzy time processing module, the system can accurately convert the user's natural language time expression into a precise time format that the database can recognize, thereby ensuring the accuracy and reliability of the query results.

[0024] The specific implementation steps for the proposed method are as follows:

[0025] Step B1: Retrieve the user's question and the original question's session_id from the state;

[0026] Step B2: If session_id does not exist, there is no need to follow up with previous questions; simply return to the original question.

[0027] Step B3: If session_id exists, retrieve the historical dialogue record from the database based on session_id. The historical dialogue format is "user:question\nAI:answer\n".

[0028] Step B4: Transfer the historical dialogue records into the designed prompt word template;

[0029] Step B5: Determine the relevance between the user's question and the historical dialogue. If there is a relevance, rewrite the question based on the historical dialogue; if there is no relevance, keep the original question unchanged.

[0030] The beneficial effects of this invention are as follows:

[0031] 1. This invention designs a system for converting natural language to SQL based on Large Language Models (LLMs), utilizing the extensive knowledge reserves and excellent generation capabilities of LLMs to explore the potential of text-to-SQL.

[0032] 2. This invention implements a multi-turn dialogue system based on Text-to-SQL and its fuzzy time rewriting and historical question follow-up methods based on the LangGraph architecture. The system significantly improves the accuracy and reliability of the Text-to-SQL system when handling time and date issues and multi-turn dialogues through fuzzy time rewriting and historical question follow-up methods.

[0033] 3. More accurate date and time processing: The fuzzy time rewriting module rewrites fuzzy date and time expressions into a clear time range. Combined with relevant tools (such as the datetime library), this greatly improves the accuracy and reliability of date and time processing.

[0034] 4. By using the historical question follow-up module and combining historical dialogue information to rewrite user questions, the system can better inherit and understand user intent in multi-turn dialogues, and generate more accurate SQL query statements.

[0035] 5. By breaking down complex Text-to-SQL problems into multiple subtasks, the system's scalability and maintainability are improved. Simultaneously, the follow-up query mechanism reduces erroneous queries caused by incomplete information, thus improving overall system performance. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the architecture of a system embodiment of the present invention that converts text to SQL based on historical inquiry and fuzzy time rewriting.

[0037] Figure 2 This is a flowchart illustrating the fuzzy time processing module in an embodiment of the present invention.

[0038] Figure 3 This is a flowchart illustrating the historical question inquiry module in an embodiment of the present invention. Detailed Implementation

[0039] The present invention will be further described in detail below with reference to the accompanying drawings.

[0040] like Figure 1 The embodiment of the present invention shown includes: using a fuzzy time processing module, a historical questioning module, and a LangGraph workflow architecture in a large language model;

[0041] Among them, such as Figure 2The fuzzy time processing module shown uses a time rewriting method to handle fuzzy time expressions in user queries. This is achieved through the following steps: First, after the user inputs the question, the system obtains the current system time as a base time, providing a reference point for subsequent time conversion. Second, the system analyzes the user-input question using a predefined time template. This template can identify and capture various common fuzzy time expressions, including but not limited to relative time expressions such as "today," "yesterday," "this Monday," and "last week." Then, a large language model is used to intelligently convert the identified fuzzy time expressions, transforming relative times into precise date and time expressions. During the conversion process, the system considers the current time as a base point to ensure accuracy. Finally, the accuracy of the time expression is verified by validating the rewritten question to ensure its accuracy and reasonableness. This design not only improves the system's understanding of time-related queries but also significantly enhances the accuracy and efficiency of user queries. After successful verification, the fuzzy time in the original question is rewritten into a precise time. Through this fuzzy time processing module, the system can accurately convert the user's natural language time expression into a precise time format recognizable by the database, thereby ensuring the accuracy and reliability of the query results.

[0042] like Figure 3 The historical question follow-up module shown implements an intelligent historical question follow-up method. This method achieves a deep understanding of the user's query intent through multi-level intelligent analysis. After inputting a question, it first calls the large model to query historical dialogues, thus constructing a complete dialogue history management mechanism. This mechanism not only records the user's questions and the system's answers but also uses Message entity class software to structurally store the dialogue content, supporting the continuity of multi-turn dialogues. Then, it generates a rewritten question. In the question correlation analysis, it uses historical follow-up node, which can analyze the correlation between the current question and historical dialogues. By designing prompt word templates, it uses the large model to rewrite the question in a context-sensitive manner, ensuring the completeness and accuracy of the question. Finally, it performs follow-up questions. In terms of intelligent follow-up question function, the system implements multi-level intelligent analysis: First, the system analyzes the completeness of the question by combining database table structure information; second, the system determines whether follow-up questions are needed to obtain more information; then, the system generates appropriate follow-up questions to ensure the relevance and effectiveness of the follow-up questions; finally, the system decides whether to continue the SQL generation process based on the follow-up question results. This design not only improves the system's ability to understand user intent but also significantly enhances query accuracy and user experience. Through this mechanism, the system can better understand user query needs and provide more accurate query results.

[0043] Both the fuzzy time processing module and the historical question follow-up module utilize the LangGraph workflow architecture to implement a complete workflow management system. This system achieves efficient conversion from natural language to SQL through modular design and intelligent control. In terms of node design, the system decomposes the entire conversion process into multiple functional nodes, including question rewriting, SQL generation, SQL execution, and error repair. Each node is responsible for a specific function, and the State class maintains the state information of the entire conversion process, ensuring correct data transmission between nodes. Regarding conditional flow, conditional edges include: determining whether SQL execution needs repair and whether follow-up questions are needed. These conditional edges determine the next execution path based on the current state. In terms of error handling, the error handling process implements a complete SQL execution error detection and automatic repair mechanism. When an SQL execution error occurs, it enters the SQL repair node, analyzing the error information to generate a repaired SQL statement. This LangGraph-based architecture not only improves the system's maintainability and scalability but also significantly enhances its intelligence and reliability. Through this design, the system can flexibly handle various complex query scenarios, providing users with accurate and efficient query services.

[0044] In this embodiment, the specific implementation steps of the fuzzy time processing module are as follows:

[0045] Step A1: Retrieve the user's question and the current system time from the state;

[0046] Step A2: Input the obtained information into the designed prompt word template;

[0047] Step A3: Determine whether there is fuzzy time using the large model;

[0048] Step A4: If it exists, rewrite the ambiguous time in the original problem into precise time, and return the rewritten problem;

[0049] Step A5: If it does not exist, return directly to the original question.

[0050] In this embodiment, the specific implementation steps of the application's historical question follow-up method are as follows:

[0051] Step B1: Retrieve the user's question and the original question's session_id from the state.

[0052] Step B2: If session_id does not exist, there is no need to ask the previous question; simply return to the original question.

[0053] Step B3: If session_id exists, retrieve the historical conversation record from the database based on session_id. The historical conversation format is "user:question\nAI:answer\n".

[0054] Step B4: Transfer the historical dialogue records into the designed prompt word template.

[0055] Step B5: Determine the relevance between the user's question and the historical dialogue. If there is a relevance, rewrite the question based on the historical dialogue; if there is no relevance, keep the original question unchanged.

Claims

1. A text-to-SQL system based on historical inquiry and fuzzy time rewriting, characterized in that, include: The fuzzy time processing module and the historical question follow-up module, The fuzzy time processing module includes: After the user enters the question, the system obtains the current system time as the base time to provide a reference point for subsequent time conversion; The system uses a predefined time template to analyze the user-input questions. This template can identify various common ambiguous time expressions and extract relative time expression words. The system uses a large language model to intelligently convert the identified fuzzy time expressions into precise date and time expressions. Verify the accuracy of the time representation; After verification, the ambiguous time in the original question will be rewritten as the precise time. The historical questioning module includes: After the input question is used, the large model is first called to query the history of the dialogue; Use historical question nodes to generate rewritten questions; Ask follow-up questions, and the system will generate appropriate follow-up questions.

2. The text-to-SQL system based on historical querying and fuzzy time rewriting according to claim 1, characterized in that, During the conversion process from relative time to precise date and time, the system considers the current time as a reference point to ensure the accuracy of the conversion.

3. The text-to-SQL system based on historical querying and fuzzy time rewriting according to claim 1, characterized in that, The historical dialogue queries are stored in a structured manner using entity-based software.

4. The text-to-SQL system based on historical querying and fuzzy time rewriting according to claim 1, characterized in that, The historical inquiry nodes are analyzed to determine the correlation between the current question and the historical dialogue. By designing prompt word templates, the question is rewritten in a context-dependent manner using a large model.

5. The text-to-SQL system based on historical querying and fuzzy time rewriting according to claim 1, characterized in that, The generation of follow-up questions includes: first, the system analyzes the completeness of the question by combining the database table structure information; second, the system determines whether follow-up questions are needed to obtain more information; and then, the system generates appropriate follow-up questions to ensure the relevance and effectiveness of the follow-up questions.

6. A text-to-SQL system based on historical inquiry and fuzzy time rewriting according to claim 1 or 5, characterized in that, After the system generates appropriate follow-up questions, it decides whether to continue the SQL generation process based on the follow-up questions.

7. A text-to-SQL system based on historical querying and fuzzy time rewriting according to claim 6, characterized in that, This improved the accuracy of queries and the user experience.

8. A text-to-SQL system based on historical querying and fuzzy time rewriting according to claim 6, characterized in that, It can better understand users' query needs and provide more accurate query results.

9. A text-to-SQL system based on historical querying and fuzzy time rewriting according to claim 1, characterized in that, After successful verification, the fuzzy time in the original question is rewritten as a precise time. Through the fuzzy time processing module, the system can accurately convert the user's natural language time expression into a precise time format that the database can recognize, thereby ensuring the accuracy and reliability of the query results.

10. A text-to-SQL system based on historical querying and fuzzy time rewriting according to claim 6, characterized in that, The specific implementation steps for the proposed method are as follows: Step B1: Retrieve the user's question and the original question's session_id from the state; Step B2: If session_id does not exist, there is no need to follow up with previous questions; simply return to the original question. Step B3: If session_id exists, retrieve the historical dialogue record from the database based on session_id. The historical dialogue format is "user:question\nAI:answer\n". Step B4: Transfer the historical dialogue records into the designed prompt word template; Step B5: Determine the relevance between the user's question and the historical dialogue. If there is a relevance, rewrite the question based on the historical dialogue; if there is no relevance, keep the original question unchanged.

Images