Method for automatically converting context-aware multi-round dialogue text into SQL (Structured Query Language)

Abstract

The invention relates to a context-aware method for automatically converting a multi-round dialogue text into an SQL (Structured Query Language), which comprises the following steps of: acquiring a to-be-retrieved multi-round dialogue text of a user, randomly sampling dialogue examples from a CoSQL data set, and generating an example set; inputting the to-be-retrieved multi-round dialogue text and the example set into a large language model to analyze a user intention, and obtaining a corresponding SQL statement; wherein the large language model performs low-rank adaptation fine tuning by using a training set, and optimizes model parameters in combination with a context-aware dynamic loss function; the training set comprises historical user intentions and historical SQL statements; and verifying the SQL statement, querying information corresponding to the SQL statement after verification is passed, and generating information required by the user. According to the method, intention analysis and SQL suggestion generation are performed by utilizing the large language model, and the quality and diversity of multi-round text-to-SQL task training data are improved.

Description

Technical Field

[0001] The present invention relates to the field of text query technology, and in particular to a context-aware multi-round conversation text to SQL automatic conversion method. Background Art

[0002] The prevalence of data-driven decision making in organizations has led to an increased demand for efficient and intuitive ways to interact with databases. Natural language database interfaces, especially text-to-SQL systems, have emerged as a promising solution to democratize data access and enable non-technical users to gain insights from complex datasets. While significant progress has been made on single-turn text-to-SQL tasks, where a single user utterance is converted into a SQL query, the challenge of handling multi-turn interactions remains a key area of ​​research.

[0003] The multi-turn text-to-SQL task presents unique challenges that go beyond those of a single-turn interaction. These include maintaining and updating context across multiple turns, handling ambiguous references, and generating SQL queries that accurately reflect the user's changing intent throughout the conversation. The complexity stems from the need to understand and integrate conversational context, handle various linguistic phenomena (such as anaphora and ellipsis), and generate increasingly complex SQL queries as the conversation progresses. To illustrate this process, Figure 1 A flowchart depicting the steps in a typical multi-turn text-to-SQL interaction is presented. The diagram demonstrates how the system identifies unique template IDs from the database, determines which templates are used multiple times, and retrieves the document IDs associated with those templates. This visual representation emphasizes the iterative nature of the multi-turn text-to-SQL task and highlights the need for the system to maintain accurate context across successive user inputs in order to generate correct SQL queries.

[0004] Single-round text-to-SQL: Text-to-SQL translation has evolved significantly over the past few decades, with approaches ranging from rule-based systems to advanced deep learning models. Early approaches relied on rule-based techniques using predefined templates, which performed well in specific scenarios but lacked adaptability. Subsequent statistical machine learning methods, such as semantic parsing, provided improved flexibility but still faced challenges with complex queries and domain adaptation. The advent of deep learning introduced sequence-to-sequence (seq2seq) models based on encoder-decoder architectures, which significantly enhanced the ability to generate SQL queries from natural language. However, these models still have difficulties in complex query translation and generalization to unseen database patterns.

[0005] To overcome these limitations, researchers have developed fine-tuning methods using pre-trained language models (PLMs), such as BERT-based text-to-SQL models, T5-based methods, and domain-specific pre-trained models such as GraPPa, which have demonstrated impressive performance on benchmarks such as the Spider dataset. These results benefit from the rich language knowledge encoded in PLMs. Recently, in-context learning (ICL) methods with large language models such as ChatGPT and GPT-4 have further advanced the field, achieving state-of-the-art results on datasets such as Spider and Bird, often surpassing fine-tuned models. Despite these advances, challenges remain in consistency, interpretability, and handling complex database patterns.

[0006] Multi-turn text to SQL: Multi-turn text to SQL systems address the challenge of handling complex, context-dependent queries across multiple interactions, and this area has made significant progress in recent years. Researchers have explored a variety of strategies to improve performance, including improved context modeling techniques, dialogue state tracking mechanisms, schema chaining methods, and intermediate representations. For example, IGSQL introduces an interaction-level graph representation to capture cross-turn dependencies, while EditSQL proposes an interaction history-based editing mechanism to optimize queries. RAT-SQL introduces a relation-aware Transformer architecture to better align query tokens with schema elements; DELTA proposes a decoupled framework that separates dialogue modeling from SQL generation. In addition, CQR-SQL introduces a context-dependent query rewriting step before generating SQL, which helps resolve ambiguities and maintain context consistency across rounds.

[0007] The application of large pre-trained language models has also been extended to multi-turn dialogue scenarios. COE-SQL leverages the capabilities of these models to handle single-turn and multi-turn text-to-SQL tasks, demonstrating their potential in dealing with complex, context-dependent queries. However, it remains challenging to efficiently fine-tune these large models for specific multi-turn text-to-SQL tasks and ensure consistency in performance across different domains and interaction modes.

[0008] Defects and shortcomings:

[0009] Although there has been progress in the multi-round text-to-SQL task, it still faces many challenges. Existing systems are unable to manage context, have difficulty maintaining coherence in long conversations, and cannot accurately capture subtle changes in user intent. SQL generation capabilities also need to be improved, especially when dealing with increasingly complex query requirements. Problems such as insufficient cross-domain generalization capabilities and poor model efficiency are also prominent, and the limitations of graph encoders in utilizing pattern information also restrict system performance. It is worth noting that existing evaluation systems may not fully reflect the performance of the system in actual applications, especially when dealing with complex and continuous conversation scenarios. Although these systems are designed to make data easily accessible to non-technical users, there is still much room for improvement in ease of use and user experience. In addition, although real-time response capabilities are not explicitly mentioned, they are undoubtedly a key factor affecting user experience. Summary of the invention

[0010] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a context-aware multi-round conversation text to SQL automatic conversion method, which uses a large language model to perform intent parsing and SQL suggestion generation, thereby improving the quality and diversity of multi-round text to SQL task training data.

[0011] To achieve the above object, the present invention provides the following solutions:

[0012] A context-aware multi-turn conversation text to SQL automatic conversion method, comprising:

[0013] Obtain the user's multi-round conversation texts to be retrieved, randomly sample conversation examples from the CoSQL dataset, and generate an example set;

[0014] Inputting the multi-round dialogue texts to be retrieved and the example set into a large language model to analyze user intent and obtain corresponding SQL statements; wherein the large language model uses a training set for low-rank adaptation fine-tuning and optimizes model parameters in combination with a context-aware dynamic loss function; the training set includes: historical user intent and historical SQL statements;

[0015] The SQL statement is verified, and after the verification is passed, the information corresponding to the SQL statement is queried to generate the information required by the user.

[0016] Optionally, the method to generate a sample collection is:

[0017] E={(S i , C i , Q i )|i∈{1,2},(S i , C i , Q i )~D train}

[0018] Among them, S i , C i and Q i denote the database schema, conversation context, and corresponding SQL query of the ith example, respectively. train Represents the training set of the CoSQL dataset.

[0019] Optionally, obtaining the SQL statement includes:

[0020] Input the multi-round conversation text to be retrieved and the example set into a large language model to analyze the user intention, and output it in JSON format to obtain the SQL statement:

[0021] f LLM :(u,h,s,E)→(I,G)

[0022] Among them, f LLM Features of a large language model, u represents user queries, h represents conversation history, s represents database schema, I and G represent the generated intent analysis and SQL statements respectively.

[0023] Optionally, the context-aware dynamic loss function includes: a cross entropy loss function, a piecewise cumulative loss, and a batch loss.

[0024] Optionally, obtaining the cross entropy loss function includes:

[0025]

[0026] Among them, y represents the true label, Represents the predicted probability.

[0027] Optionally, the method for obtaining the segmented cumulative loss is:

[0028]

[0029] Where B represents the batch size L b Represents the loss for each batch.

[0030] Optionally, the method for obtaining the batch loss is:

[0031]

[0032] Among them, T represents the number of dialogue rounds, represents the cumulative losses of the first T-1 rotations, Represents the cross entropy loss of the T-th rotation.

[0033] Optionally, obtaining the accumulated loss includes:

[0034]

[0035] in, represents the cumulative loss of the first t-1 rotations, represents the cross entropy loss of the current rotation t, a t Represents the dynamic smoothing parameter.

[0036] Optionally, the method for obtaining the dynamic smoothing parameter is:

[0037] α t =max(0.1,α-0.1t)

[0038] Among them, a represents the initial smoothing parameter and t represents the current round.

[0039] Optionally, the context-aware dynamic loss function:

[0040]

[0041] Among them, a represents the initial smoothing parameter, represents the cumulative losses of the first T-1 rotations, Represents the cross entropy loss of the T-th rotation.

[0042] The beneficial effects of the present invention are:

[0043] This paper proposes Context-Aware Intent-Driven Adaptive Learning SQL (CIDAL-SQL), which uses a large language model to perform intent parsing and SQL suggestion generation in multi-round text-to-SQL tasks, integrates the parsed user query intent and SQL generation suggestions into the original dataset, and creates an enhanced dataset for model fine-tuning. In order to improve context management in multi-round conversations, a context-aware dynamic loss function (SCALD Loss) is introduced to enhance the model's ability to retain previously generated SQL and maintain context information between consecutive conversation rounds.

[0044] The main contributions of the present invention are as follows:

[0045] The present invention utilizes a large language model for intent parsing and SQL suggestion generation, improving the quality and diversity of training data for multiple rounds of text-to-SQL tasks.

[0046] The present invention introduces a context-based dynamic loss function SCALD Loss, which improves the model's ability to maintain consistency and context information in multiple dialogue turns.

[0047] We conduct extensive experiments on benchmark datasets and demonstrate significant improvements over state-of-the-art baselines in both single-round and multi-round scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A flowchart of a multi-round text-to-SQL task for background technology;

[0050] Figure 2 A schematic diagram for describing an intent-driven SQL generation algorithm according to an embodiment of the present invention;

[0051] Figure 3 A workflow diagram for generating SQL driven by intent according to an embodiment of the present invention;

[0052] Figure 4 Schematic diagram of error type distribution in a SparC data set according to an embodiment of the present invention;

[0053] Figure 5 It is a schematic diagram of the performance of CIDAL-SQL of an embodiment of the present invention on a Sparc dataset with different LoRA ranks;

[0054] Figure 6 This is a schematic diagram of the performance of CIDAL-SQL in an embodiment of the present invention at different LoRA ranks on the CoSQL dataset. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] In the context-dependent text-to-SQL task, this embodiment considers a series of interactions between the user and the system. An interaction consists of multiple rounds, each of which represents a one-way communication between the user and the system. The interaction ends when the system generates an SQL query to retrieve the information required by the user from the database. Given a database mathcal{D}, the current user's statement U in round t t , the conversation context C t Usually includes the previous user statement U{<t} ={U1,...,U {t-1}}, and the total number of rounds in the interaction T. Formally, this embodiment defines the multi-round text-to-SQL task as follows:

[0058]

[0059] Where p(Q t |U t ,C t ,D}) indicates that given the current user statement U t 、Conversation context C t and the database mathcal{D} generates the SQL query Q t probability.

[0060] The present embodiment discloses a context-aware multi-round conversation text to SQL automatic conversion method, including: obtaining a user's multi-round conversation text to be retrieved, randomly sampling conversation examples from a training set of a CoSQL data set, and generating an example set; inputting both the multi-round conversation text to be retrieved and the example set into a large language model to analyze user intent and obtain corresponding SQL statements; wherein the large language model uses the training set for low-rank adaptation fine-tuning, and optimizes model parameters through back propagation in combination with a context-aware dynamic loss function; the training set includes: historical user intent and historical SQL statements, that is, user intent and SQL generation suggestions are added to each piece of data in the original data set; the SQL statement is verified, and after the verification is passed, the information corresponding to the SQL statement is queried to generate the information required by the user;

[0061] Verification refers to executing SQL statements in the database. If the statement can be executed successfully, the verification is successful.

[0062] Furthermore, the method for generating a sample set is:

[0063] E={(S i , C i , Q i )|i∈{1,2},(S i , C i , Q i )~D train}

[0064] Among them, S i , C i and Q i denote the database schema, conversation context, and corresponding SQL query of the ith example, respectively. train Represents the training set of the CoSQL dataset.

[0065] Specifically, the implementation process and theoretical basis of this embodiment. Initially, this embodiment customizes a specific system prompt for a large language model (LLM). From the training set D of the CoSQL dataset train In this embodiment, the conversation examples are randomly sampled to construct the example set E:

[0066] E={(S i , C i , Q i )|i∈{1,2},(S i , C i , Q i )~D train}

[0067] Among them, S i , C i and Q i denote the database schema, conversation context, and corresponding SQL query of the ith example respectively. The symbol sim denotes sampling from a distribution.

[0068] Furthermore, obtaining the SQL statement includes: inputting the multi-round conversation texts and the example set to be retrieved into a large language model to analyze the user intention, and outputting the result in JSON format to obtain the SQL statement:

[0069] f LLM :(u,h,s,E)→(I,G)

[0070] Among them, f LLM Features of a large language model, u represents user queries, h represents conversation history, s represents database schema, I and G represent the generated intent analysis and SQL statements respectively.

[0071] Specifically, in order to facilitate subsequent processing, this embodiment stipulates that the model output should be in JSON format. LLM The definition is as follows:

[0072] f LLM :(u,h,s,E)→(I,G)

[0073] Among them, u represents the user query, h represents the conversation history, s is the database schema, and I and G represent the generated intent analysis and SQL statement suggestions respectively.

[0074] For a given data set D = (u j ,h j ,s j ,q j )∈{1,...,N}, this embodiment applies f LLM Function to obtain the enhanced dataset D aug :

[0075]

[0076] Among them, N represents the size of the data set, and q j represents the original SQL query. Then, this embodiment uses the enhanced training set Perform low-rank adaptation (LoRA) fine-tuning.

[0077] Furthermore, the context-aware dynamic loss functions include: cross entropy loss function, piecewise cumulative loss, and batch loss.

[0078] Furthermore, obtaining the cross entropy loss function includes:

[0079]

[0080] Among them, y represents the true label, Represents the predicted probability.

[0081] Furthermore, the method for obtaining the segmented cumulative loss is:

[0082]

[0083] Where B represents the batch size L b Represents the loss for each batch.

[0084] Furthermore, the method to obtain batch loss is:

[0085]

[0086] Among them, T represents the number of dialogue rounds, represents the cumulative losses of the first T-1 rotations, Represents the cross entropy loss of the T-th rotation.

[0087] Furthermore, obtaining the accumulated losses includes:

[0088]

[0089] in, represents the cumulative loss of the first t-1 rotations, represents the cross entropy loss of the current rotation t, a t Represents the dynamic smoothing parameter.

[0090] Furthermore, the method for obtaining the dynamic smoothing parameters is:

[0091] α t =max(0.1,α-0.1t)

[0092] Among them, a represents the initial smoothing parameter and t represents the current round.

[0093] Furthermore, the method for obtaining the context-aware dynamic loss function is:

[0094]

[0095] Among them, a represents the initial smoothing parameter, represents the cumulative losses of the first T-1 rotations, Represents the cross entropy loss of the T-th rotation.

[0096] Specifically, Figure 2 and 3 As shown, the intent-driven SQL generation process includes multiple steps, including user query processing, intent and SQL generation, SQL validation and execution, and response generation. Each step is designed to ensure that the system accurately interprets user queries and generates correct SQL statements. The intent-driven SQL generation process begins with user query processing, during which natural language queries are received and processed by the LLM, which uses the provided conversation history and database schema. The model then analyzes the query to determine the user's intent and generates a corresponding SQL statement. This SQL statement is validated against the database structure to ensure its correctness. Once the validation is passed, the SQL query is executed to retrieve the required data. Finally, the retrieved data is formatted into a friendly response and presented to the user.

[0097] In addition to the above steps, this system also incorporates several key implementation details. Specific prompts are customized to guide the model to understand and generate SQL statements. In addition, sampling is performed based on diverse conversation examples to train the model to handle various natural language inputs. The model outputs results in JSON format for easy integration and processing, and LoRA is fine-tuned for the enhanced dataset, which improves model accuracy and performance.

[0098] Context-aware dynamic loss function:

[0099] The loss function consists of multiple components, which will be introduced step by step in this embodiment. First, let this embodiment define the basic cross entropy loss function:

[0100]

[0101] Among them, y represents the true label, Represents the predicted probability.

[0102] For the segmented cumulative loss, this embodiment defines:

[0103]

[0104] Where B represents the batch size L b Represents the loss for each batch.

[0105] For each batch b, the resulting loss is calculated as follows:

[0106]

[0107] Among them, T represents the number of dialogue rounds, represents the cumulative losses of the first T-1 rotations, Represents the cross entropy loss of the T-th rotation.

[0108] The cumulative loss is calculated as:

[0109]

[0110] Among them, the dynamic smoothing parameter is defined as:

[0111] α t =max(0.1,α-0.1t)

[0112] Among them, a represents the initial smoothing parameter and t represents the current round.

[0113] Finally, the complete loss function can be expressed as:

[0114]

[0115] Among them, a represents the initial smoothing parameter, represents the cumulative losses of the first T-1 rotations, Represents the cross entropy loss of the T-th rotation.

[0116] Effect of the context-aware multi-round conversation text to SQL automatic conversion method in this embodiment:

[0117] Model and hardware: This embodiment uses CIDAL-SQL as the base model for fine-tuning. CIDAL-SQL is a large language model optimized for SQL code generation tasks, containing 8 billion parameters. The experiments in this embodiment were conducted on a server equipped with 8 NVIDIA GeForce RTX 3090 GPUs, each with 24GB of video memory. Due to the memory limitations of a single GPU, this embodiment uses LoRA (low rank adaptation) technology for model fine-tuning. LoRA is a parameter-efficient fine-tuning method that significantly reduces computational and memory requirements by adding a small number of trainable parameters to adapt to downstream tasks while keeping most of the pre-trained weights unchanged.

[0118] Hyperparameters: This example carefully configured hyperparameters to optimize the experimental settings. This example set the random seed to 42 to ensure repeatability, and used a learning rate of 1e-4, with a cosine annealing scheduler and a warm-up factor of 0.1. The context length was set to 2048 tokens. To manage computing resources, this example used a batch size of 1 per GPU, 8 steps of gradient accumulation, and trained at float16 precision to reduce memory overhead. During fine-tuning, this example implemented low rank adaptation (LoRA), with a rank of 32 and a target of up j ,q j 、gate j , o j 、v j , k j and down j . Meanwhile, LoRA alpha is set to 1024. These settings are designed to balance model performance, computational efficiency, and adaptability.

[0119] Datasets and baseline models: To verify the effectiveness of the method proposed in this embodiment, this embodiment conducts experiments on two widely used datasets for context-dependent text-to-SQL tasks: Sparc and CoSQL. These datasets are specifically designed to evaluate the performance of models on multi-round interactions in the field of text-to-SQL. This embodiment compares the proposed CIDAL-SQL model with a variety of state-of-the-art baselines, including GAZP+BERT, HIE-SQL+G-raPPa, RASAT+PICARD, ACT-SQL, and CoE-SQL, to evaluate its performance in multi-round text-to-SQL tasks. Although GAZP+BERT works well in single-round scenarios, it has difficulty maintaining consistency in multi-round conversations. HIE-SQL+GraPPa uses a hierarchical pattern perception mechanism combined with a specific domain pre-training model to show strong performance, but it is weak for long-distance dependencies. RASAT+PICARD performs well in capturing pattern relationships and generating context-aware SQL through relationship-aware transformers and interactive retrieval methods. ACT-SQL introduces an adaptive control mechanism, but performs poorly in complex multi-round environments. CoE-SQL manages schema linking well with its context-aware encoder, but faces challenges brought by ambiguous dialogues.

[0120] Evaluation indicators: This embodiment uses a variety of evaluation indicators to comprehensively evaluate the performance of the method of this embodiment. For single-round evaluation, this embodiment uses exact matching (EM) and execution accuracy (EX). EM is used to evaluate structural correctness, while EX measures the correctness of the execution results. In multi-round scenarios, this embodiment introduces question matching (QM) and interaction matching (IM). QM evaluates the accuracy of a single question, while IM evaluates the correctness in the context of the entire conversation. The formulas for QM and IM within the execution accuracy (EX) framework are as follows:

[0121]

[0122] Where N is the number of individual questions in the evaluation set, M is the number of multi-round interactions in the evaluation set, and I is an indicator function that returns 1 if the condition is true and 0 otherwise. represents the predicted SQL query for the ith question. i Represents the actual SQL query for the ith question. represents the predicted SQL query for the jth interaction considering the entire conversation context. j represents the real SQL query for the jth interaction considering the entire conversation context.

[0123] Model hints: Hints used during training and inference are carefully designed to guide the model to generate accurate SQLite queries based on the task and database schema provided by the user. The following is a detailed description of the hint structure for the training and inference phases. During the training and inference phases, two main hint templates are used: Llama3 and Llama3_with_uqr. Llama3 hints focus on generating SQLite queries based only on the task description provided by the user. The interaction starts with the system introducing the database schema:

[0124] System: Database Schema: All SQLite queries will run on a database with the following schema: {{content}}, then the user specifies the task:

[0125] User: Task:

[0126] Generates a SQLite query to answer {{content}};

[0127] Finally, the helper generates the appropriate query:

[0128] Assistant: Answer:

[0129] After analyzing the user's question, here is the SQLite query that best answers the question, using only the appropriate tables and columns from your database schema: sql {{ content}};

[0130] The llama3_with_uqr prompt expands on this by incorporating a reference to the user's underlying intent. After the user asks for a task, the assistant generates a query taking into account the task and the user intent generated by LLM:

[0131] Assistant: Answer:

[0132] After analyzing the user's question and referring to the contents of [USER_INTENTION] and [ / USER_INTENTION], here is the SQLite query that best answers the question, using only the appropriate tables and columns from the database schema: sql{{content}}.

[0133] Table 1

[0134]

[0135] Table 2

[0136]

[0137] Main Results: This example conducts comprehensive experiments on the Sparc and CoSQL datasets, which are widely used benchmarks for evaluating text-to-SQL models. The experiments in this example compare the performance of the proposed model with multiple state-of-the-art baselines. This example uses a variety of evaluation metrics to evaluate the performance in single-round and multi-round scenarios. Table 1 (model performance comparison in the Sparc development set) and Table 2 (model performance comparison in the CoSQL development set) show detailed results in the Sparc and CoSQL development sets, respectively. In the single-round evaluation, this example uses execution accuracy (EX) and exact match (EM). EX measures the percentage of generated SQL queries that produce correct results when executed, while EM evaluates the correctness of the generated SQL structure. In the multi-round scenario, this example uses question matching (QM) and interaction matching (IM) metrics. QM evaluates the accuracy of each question in the conversation, while IM evaluates the consistency of the entire conversation context. On the Sparc dataset (Table 1), the CIDAL-SQL model of this example achieves 82.0% EX and 75.6% EM in the single-round evaluation (QM). These results surpass the previous best model RASAT+PICARD, which recorded 73.3% EX and 67.7% EM. The enhanced version of the model in this embodiment further improves the IM performance, achieving 61.4% EX and 51.2% EM in the multi-round scenario, which is also better than RASAT+PICARD, which has an EX of 54.0% and an EM of 49.1%.

[0138] For the CoSQL dataset (Table 2), CIDAL-SQL records 73.2\% EX and 64.3\% EM on QM, outperforming previous models such as CoE-SQL (69.6\% EX, 52.4\% EM) and RASAT+PICARD (67.0\% EX, 58.8\% EM). However, in multi-round evaluation (IM), the model of this embodiment achieved 39.9\% EX and 27.3\% EM, which, while comparable, still shows room for further improvement.

[0139] In these two datasets, the model of this embodiment performs better in most indicators than previous methods such as RASAT+PICARD, HIE-SQL+GraPPa, and CoE-SQL. This consistent performance shows that the method of this embodiment effectively captures the subtle differences between single-round and multi-round text-to-SQL tasks. However, this embodiment notes that the performance gap between single-round (QM) and multi-round (IM) scenarios, including the model of this embodiment, is still significant. For example, in the Sparc dataset, the EX of the model of this embodiment drops from 82.0\% in QM to 61.4\% in IM, which shows that processing context in multi-round interactions is still a challenging aspect of the text-to-SQL task.

[0140] The performance variations between different model versions highlight the importance of specific training strategies and loss functions on model performance. These differences emphasize the importance of carefully designing the model and training process to achieve the best results for text-to-SQL tasks.

[0141] Ablation Experiments. To evaluate the components proposed in this embodiment, this embodiment performs ablation studies on the Sparc and CoSQL datasets. This embodiment examines three model variants: CIDAL-SQL with only modified loss functions (-w / o DataAug), intent-driven SQL generation only (-w / o SCALD Loss), and a model combining the two (CIDAL-SQL). Tables ~\ref{tab:ablation_sparc} and ~\ref{tab:ablation_cosql} show the results. On Sparc, the CIDAL-SQL model achieves the highest performance with 82.0% EX(QM) and 75.6% EM(QM), while the loss-only model is 82.0% and 75.1%, and the w / o SCALD Loss model is 80.2% and 73.9%. For the multi-round scenario, CIDAL-SQL shows improvements over the loss-only model in both EX(IM) and EM(IM), from 60.9% to 61.4% (EX(IM)) and from 49.3% to 51.2% (EM(IM)). On CoSQL, the CIDAL-SQL model also performs best, achieving 73.2% EX(QM) and 64.3% EM(QM), compared to 73.0% and 63.4% for the loss-only model and 72.9% and 62.9% for the w / o SCALD Loss model. In terms of multi-round metrics, CIDAL-SQL obtains 39.9% EX(IM) and 27.3% EM(IM), slightly higher than other variants. These results show that both the loss function modification and the unified problem representation contribute to the performance improvement, and their combination works best on both datasets, as shown in Table 3 (ablation study results on the Sparc dataset) and Table 4 (ablation study results on the CoSQL dataset).

[0142] Table 3

[0143]

[0144] Table 4

[0145]

[0146] Error Analysis: To gain a deeper understanding of the performance of the multi-round text-to-SQL model of this embodiment, this embodiment performs a comprehensive error analysis using a large language model (LLM) approach. This approach leverages the advanced capabilities of LLM in natural language understanding to identify and classify various types of errors in generating SQL queries. This embodiment focuses on the Sparc dataset, noting that the error distribution in the CoSQL dataset exhibits a similar pattern.

[0147] Figure 4 The error distribution of different categories in the Sparc dataset is shown. This analysis reveals some key insights into the challenges faced by the model in this embodiment in the multi-round Text-to-SQL task.

[0148] Schema linking errors emerged as the most prevalent problem, accounting for 24.75% of all errors. The dominance of this error type can be attributed to several factors. First, the inherent uncertainty of natural language makes it challenging to accurately map user utterances to specific database elements. Users often use colloquial terms or domain-specific jargon that may not directly correspond to formal database schema elements. Second, models must cope with a large and diverse set of schemas across different databases, each with its own unique structure and naming conventions. This diversity makes it difficult for models to generalize effectively. Finally, in multi-round scenarios, models must maintain their understanding of the schema across multiple interactions, increasing the possibility of misunderstanding or loss of context.

[0149] The proportion of "other" errors is as high as 21.29%, highlighting the complexity and unpredictability of user queries in conversational environments. This category may cover a wide range of problems, from logical errors in query construction to misunderstandings of complex user intent. The prevalence of these errors indicates that the current error classification of this embodiment may not be sufficient to capture all challenges in multi-round Text-to-SQL tasks. At the same time, this also shows that users often construct queries in ways that deviate from standard patterns, which challenges the model's ability to generalize from training data.

[0150] JOIN-related errors (14.23%) represent a significant challenge, primarily due to the complexity of understanding and representing relationships between multiple tables. These errors often occur because the model has difficulty inferring implicit relationships that are not explicitly stated in the user's query. In addition, as the conversation evolves and queries become more complex, the number of potential JOIN operations grows exponentially, making it more difficult for the model to determine the correct data table relationships.

[0151] The presence of conversation-specific errors, such as query modification (10.15%), session consistency (9.03%), and incremental query construction (5.94%), highlights the unique challenges of multi-turn interactions. Query modification errors typically occur when the model fails to correctly interpret how the user’s new request should change the previous query. This can be due to subtle changes in user intent or unclear references to previous query elements. Session consistency errors occur when the model loses its grasp of the conversation context across multiple turns, perhaps due to limitations in its memory mechanisms or attention span. Incremental query construction errors stem from the model’s difficulty accumulating and integrating information from multiple turns into a coherent and complex query.

[0152] Context preservation (3.96%) and reference resolution (1.61%) errors, while less frequent, are critical to maintaining consistency across multiple turns of dialogue. These errors typically occur when the model fails to correctly link pronouns or ambiguously refers to their antecedents in previous turns. The relatively low frequency of these errors suggests that the model in this example has some ability to handle dialogue context, but there is still room for improvement, especially in resolving ambiguous references across turns.

[0153] GROUP BY (5.45%) and nested query (3.59%) errors, while less common, point to the challenges of handling more complex SQL constructs. These errors tend to occur when the model struggles to translate complex analytical requirements expressed in natural language into appropriate SQL constructs. The model may fail to recognize when aggregation is required (resulting in a GROUP BY error) or when a subquery is necessary to answer a complex, multi-part question (resulting in a nested query error).

[0154] The importance of schema linking and JOIN errors highlights the fundamental difficulty of bridging the gap between natural language understanding and database structure. The specific errors related to conversation highlight the additional complexity that comes with multi-turn interactions, where the model must not only understand a single query, but also maintain and evolve its understanding over time. The large proportion of "other" errors suggests that there is still much to be discovered about the nature of user interactions in this area.

[0155] It is worth noting that the error distribution observed in the CoSQL dataset is very close to these findings, indicating that these challenges are consistent across different multi-round text-to-SQL scenarios. This consistency emphasizes the fundamental nature of these challenges in the field of human-computer interaction and text-to-SQL systems.

[0156] In order to investigate the impact of LoRA rank on the performance of the CIDAL-SQL model of this embodiment, this embodiment conducted experiments on the Sparc and CoSQL datasets using different rank values ​​(8, 16, and 32). Figure 5 The Sparc data is shown in , and the CoSQL data is shown in \ref{fig:cosql_lora_rank}.

[0157] For the Sparc dataset, this example observed that higher LoRA ranks generally lead to better performance on all metrics. The model with 32-order LoRA achieved the best results, with an execution accuracy (EX) of 82.0% and an exact match (EM) of 75.6% for question matching (QM). In terms of interactive matching (IM), it achieved 61.4% EX and 51.2% EM. When this example reduced the LoRA rank to 16 and 8, the performance decreased slightly, with the most obvious decrease in the IM metric, such as Figure 6 shown.

[0158] On the CoSQL dataset, the effect of LoRA level is less pronounced. The model using LoRA level 32 performs best on the QM metric, achieving 73.2% EX and 64.3% EM. However, on the IM metric, the performance is more diverse between different levels. Interestingly, the model with LoRA level 16 shows a slight improvement in EX(IM) at 40.3%, while 32 and 8 both achieve 39.9%.

[0159] These results show that higher LoRA levels (32 in this example experiment) generally lead to better performance, especially on the Sparc dataset. However, in some cases, this improvement is small, especially on the CoSQL dataset. This suggests that while increasing the LoRA level may enhance the model's ability to capture complex patterns, the benefits may vary depending on the specific dataset and task complexity.

[0160] Considering the trade-off between performance and computational efficiency, this embodiment recommends using LoRA level 32 for best results. However, in the case of limited computing resources, using level 16 is also a reasonable compromise because it can maintain similar performance while potentially reducing the number of model parameters and training time.

[0161] The above embodiments are only descriptions of the preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the protection scope determined by the claims of the present invention.

Claims

1. A context-aware multi-round conversation text to SQL automatic conversion method, characterized in that: include: Obtain the user's multi-round conversation texts to be retrieved, randomly sample conversation examples from the CoSQL dataset, and generate an example set; Inputting the multi-round dialogue texts to be retrieved and the example set into a large language model to analyze user intent and obtain corresponding SQL statements; wherein the large language model uses a training set for low-rank adaptation fine-tuning and optimizes model parameters in combination with a context-aware dynamic loss function; the training set includes: historical user intent and historical SQL statements; The SQL statement is verified, and after the verification is passed, the information corresponding to the SQL statement is queried to generate the information required by the user.

2. The context-aware multi-round conversation text to SQL automatic conversion method according to claim 1 is characterized in that: The method to generate the sample set is: E={(S i ,C i ,Q i )|i∈{1,2},(S i ,C i ,Q i )~D train } Among them, S i , C i and Q i denote the database schema, conversation context, and corresponding SQL query of the ith example, respectively. train Represents the training set of the CoSQL dataset.

3. The context-aware multi-round conversation text to SQL automatic conversion method according to claim 1, characterized in that: Obtaining the SQL statement includes: Input the multi-round conversation text to be retrieved and the example set into a large language model to analyze the user intention, and output it in JSON format to obtain the SQL statement: f LLM :(u,h,s,E)→(l,G) Among them, f LLM Features of a large language model, u represents user queries, h represents conversation history, s represents database schema, I and G represent the generated intent analysis and SQL statements respectively.

4. The context-aware multi-round conversation text to SQL automatic conversion method according to claim 1, characterized in that: The context-aware dynamic loss function includes: cross entropy loss function, piecewise cumulative loss, and batch loss.

5. The context-aware multi-round conversation text to SQL automatic conversion method according to claim 4, characterized in that: Obtaining the cross entropy loss function includes: Among them, y represents the true label, Represents the predicted probability.

6. The context-aware multi-round conversation text to SQL automatic conversion method according to claim 4, characterized in that: The method for obtaining the segmented cumulative loss is: Where B represents the batch size L b Represents the loss for each batch.

7. The context-aware multi-round conversation text to SQL automatic conversion method according to claim 4, characterized in that: The method to obtain the batch loss is: Among them, T represents the number of dialogue rounds, represents the cumulative losses of the first T-1 rotations, Represents the cross entropy loss of the T-th rotation.

8. The context-aware multi-round conversation text to SQL automatic conversion method according to claim 7, characterized in that: Obtaining the accumulated losses includes: in, represents the cumulative loss of the first t-1 rotations, represents the cross entropy loss of the current rotation t, a t Represents the dynamic smoothing parameter.

9. The context-aware multi-round conversation text to SQL automatic conversion method according to claim 8, characterized in that: The method for obtaining the dynamic smoothing parameter is: a t =max(0.1,α-0.1t) Among them, a represents the initial smoothing parameter and t represents the current round.

10. The context-aware multi-round conversation text to SQL automatic conversion method according to claim 1, characterized in that: The context-aware dynamic loss function: Among them, a represents the initial smoothing parameter, represents the cumulative losses of the first T-1 rotations, Represents the cross entropy loss of the T-th rotation.

Images