SQL (Structured Query Language) generation and error correction method based on multi-stage feedback

Abstract

The invention provides an SQL (Structured Query Language) generation and error correction method based on multi-stage feedback, which comprises the following steps: S1, collecting user demands, and greatly improving the accuracy of generating SQL while keeping quick response by introducing an expert thinking model and by means of a mode of demand disassembly and task gradual combination, so that the user experience is improved; and it is ensured that effective query statements can be quickly generated in the face of simple or medium-complexity tasks. Common requirements can be efficiently processed, so that the query requirements of the user can be quickly met. And secondly, aiming at a task which is complicated or has a high-precision requirement, through a multi-round interaction and error correction mechanism, the accuracy of a generated result is ensured. When the SQL execution statement is generated for the first time, if an error occurs or a user is not satisfied, multiple rounds of error correction processes can be started, the SQL execution statement is corrected step by step, and finally high-precision output meeting the requirement is achieved. Through the double-link design, the speed and the accuracy are both considered, and the requirements in different scenes are met.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a SQL generation and error correction method based on multi-stage feedback.

[0002] Although traditional SQL generation tools can meet basic needs, they often perform poorly when dealing with complex and changing query tasks due to lack of deep learning and adaptive capabilities. Current technologies, especially in automated SQL generation and error correction, often fail to fully utilize advanced large model technologies.

[0003] Therefore, developing a SQL generation and error correction method based on multi-stage feedback has become an important technical requirement today. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a SQL generation and error correction method based on multi-stage feedback, which performs semantic analysis and structural optimization on SQL queries and can dynamically adapt to different database architectures and query requirements, in view of the shortcomings of the above technical solutions.

[0005] The present invention provides a SQL generation and error correction method based on multi-stage feedback, the method comprising the following steps:

[0006] S1, collect user requirements, optimize new user requirements and output the final table description, and store the table description in the database;

[0007] S2, splicing the user requirements, table descriptions and prompt words of the Prompt large model and inputting them into the requirement decomposition large model, wherein the requirement decomposition large model normalizes and reconstructs the user requirements to obtain standardized user requirements, decomposes the standardized user requirements into multiple user requirement execution steps, and generates corresponding basic SQL statements;

[0008] S3, establish an SQL vector knowledge base, convert the basic SQL statements into SQL statement vectors through the embedding model and store them in the SQL vector knowledge base, use RAG retrieval enhancement generation technology to calculate the similarity between the standard user requirements and the relevant knowledge in the SQL vector knowledge base; select the most similar user requirements and corresponding SQL execution statements and combine them into standard user requirements and similar basic SQL statements, and recall similar basic SQL statements, re-queue the recalled basic SQL statements, and select the top-ranked basic SQL statements;

[0009] S4, inputting the table description, the prompt word of the Prompt model, the user demand execution steps, the basic SQL statement, the standardized user demand and the similar SQL statement into the basic SQL execution statement fusion model; the basic SQL execution statement fusion model fuses the table description, the prompt word of the Prompt model, the user demand execution steps, the basic SQL statement, the standardized user demand and the similar SQL statement to generate the final output SQL execution statement;

[0010] S5, establishing an error correction vector library, executing SQL execution statements and automatically generating corresponding charts, and asking the user whether he is satisfied; if the execution fails, converting the outputted erroneous SQL execution statements and error information into vectors, and recalling relevant error correction examples in the error correction vector library; if the user is satisfied, the SQL execution statements that the user is satisfied with are included in the database; if the execution is successful but the user is not satisfied, the SQL execution statements that the user is not satisfied with are included in the database;

[0011] S6, transmitting the output SQL execution statements, error correction examples, and SQL execution statements that the user is dissatisfied with to the error correction guidance big model for determination to confirm whether there are erroneous SQL execution statements; if it is determined that there are erroneous SQL execution statements, the error correction guidance big model generates corresponding error correction guidance, and transmits the error correction guidance to the error correction big model; the error correction big model corrects the erroneous SQL execution statements according to the error correction guidance, regenerates the final SQL execution statements, and feeds the SQL execution statements back to the error correction guidance big model for re-checking; if it is determined that there are no erroneous SQL execution statements, execute and collect user feedback, reject unqualified SQL execution statements to experts for correction, and output unsatisfactory results of the corrected SQL statements;

[0012] S7, collecting and storing the intermediate results and operation result information of steps S2 to S6, and performing feedback optimization on the intermediate results and operation result information.

[0013] In the SQL generation and error correction method based on multi-stage feedback described in the present invention, step S1 includes the following steps:

[0014] S11, basic information D of maintenance table description base , where the basic information described in the table is D base Including table name T i 、List name C i and data type DT i , denoted as D base ={(T i ,C i ,DT i ,…)}; and for the D base ={(Ti ,C i ,DT i ,…)} to supplement and edit to obtain f ML (D base ); By integrating a proprietary large model to f ML (D base ) to supplement and improve again to get f U (D base ); ML (D base ) and f U (D base ) are added to obtain the final table description, where the final table description is expressed as D = f ML (D base )+f U (D base ).

[0015] In the SQL generation and error correction method based on multi-stage feedback of the present invention; in the step S2, the user demand, table description and Prompt large model prompt word are spliced ​​to express as X=Q+D+Prompt_spilit, wherein Q represents the user demand, D represents the table description, and Prompt_spilit represents the Prompt large model prompt word; X=Q+D+Prompt_spilit is input into the demand decomposition large model f DS (X;θ)=(Q standard ,S,SQL) to standardize and reconstruct to obtain standardized user requirements Q standard , will standardize user demand Q standard Decompose into multiple user demand execution steps S = (s1, s2, s3, s4, ...), and generate the corresponding basic SQL statements SQL = (sql1, sql2, sql3, sql4, ...); where θ is the model parameter, f DS Indicates the need to disassemble the large model.

[0016] In the SQL generation and error correction method based on multi-stage feedback of the present invention, step S3 includes the following steps:

[0017] S31, convert the basic SQL statement SQL = (sql1, sql2, sql3, sql4, ...) into a SQL statement vector V through the embedding model Q =f embedding (Q standard ).

[0018] In the SQL generation and error correction method based on multi-stage feedback of the present invention, the step S3 further includes the following steps:

[0019] S32, calculate the similarity Sim(V) between the standardized user requirements and the relevant knowledge in the SQL vector knowledge base by using the BM25 and cosine similarity weighting method Q ,V i )=α·BM25(V Q ,V i )+β·cosine(V Q ,V i ), select the most similar user requirements and corresponding SQL execution statements and combine them into standard user requirements and similar basic SQL statements, and recall similar basic SQL statements, requeue the recalled basic SQL statements, and select the top five basic SQL statements; among them, V i is represented by the vector in the SQL vector knowledge base, α and β represent the weights of the BM25 and cosine similarity methods preset according to experience, Sim(V Q ,V i ) is represented as the weighted final vector similarity.

[0020] In the SQL generation and error correction method based on multi-stage feedback described in the present invention; in the step S4, the table description D, the Prompt large model prompt word Prompt_spilit, the user demand execution step S=(s1, s2, s3, s4, ...), the basic SQL statement, the standardized user demand Q standard The similar SQL statements are input into the basic SQL execution statement fusion model to generate the final output SQL execution statement represented as SQL final_1 =f merge (Q standard ,S,SQL,D,Prompt merge ),in,

[0021] Represents the corresponding basic SQL statement, Prompt merge Indicates the prompt word for the basic SQL statement fusion model, f merge It is represented as a large model of basic SQL execution statements. SQL execution statements are represented as SQL final_1 .

[0022] In the SQL generation and error correction method based on multi-stage feedback of the present invention; in step S5, SQL is executed final_1 And automatically generate the corresponding chart Chart = f viz (SQL final_1 ), if the execution fails, the wrong SQL execution statement SQL will be output final_1 And the error message error will be f through the model embedding () is transformed into a vector V e, and recall the j correction examples CorrectExample with the highest similarity Sim in the correction vector library, where f viz () Executes and outputs the final SQL for the chart automatic display module final_1 .

[0023] The SQL generation and error correction method based on multi-stage feedback of the present invention greatly improves the accuracy of SQL generation while maintaining rapid response by introducing an expert thinking model, and by means of demand decomposition and gradual task merging, ensuring that the present application can quickly generate effective query statements when facing simple or medium-complexity tasks. It can efficiently handle common needs to quickly meet the query needs of users. Secondly, for complex or high-precision tasks, the accuracy of the generated results is guaranteed through multiple rounds of interaction and error correction mechanisms. When executing the initial SQL execution statement generation, if an error or user dissatisfaction occurs, multiple rounds of error correction processes will be started, and the SQL execution statement will be gradually corrected, ultimately achieving high-precision output that meets the needs. Through this dual-link design, the present application takes into account both speed and accuracy, and adapts to the needs of different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of the SQL generation and error correction method based on multi-stage feedback of the present invention;

[0025] Figure 2 It is a flow chart of an embodiment of the SQL generation and error correction method based on multi-stage feedback of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, application, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] like Figure 1-2As shown, Figure 1 The present invention provides a method for generating and correcting SQL based on multi-stage feedback, which includes the following steps:

[0029] In step S1, user requirements are collected, new user requirements are optimized and the final table description is output, and the table description is stored in the database;

[0030] Specifically, for example, the database mainly stores the company's sales data. It contains a sales data table, table name T1 = sales_data; the table contains three columns, namely product type, sales date and sales quantity, that is, C1 = (product_type, sale_date, amount). That is, the table description D base ={(sales_data,(product_type, sale_date, amount),…).

[0031] For the "sale_date" column in the table, users can manually add the description of the column to "the date a single sales record was generated", or they can use a proprietary large model to analyze and supplement it to improve the accuracy of the table description. Users can enter their needs on this platform, and the subsequent steps will automatically generate SQL execution statements and eventually generate corresponding charts.

[0032] It should be noted that users are allowed to manually supplement and edit table descriptions to better reflect business semantics. They can also use proprietary large models that have been fine-tuned with the company's internal data to intelligently supplement and improve the table's description information.

[0033] In step S2, the user needs, table descriptions and prompt words of the Prompt large model are spliced ​​and input into the demand decomposition large model. The demand decomposition large model normalizes and reconstructs the user needs to obtain standardized user needs, decomposes the standardized user needs into multiple user need execution steps, and generates corresponding basic SQL statements; for example, the user input demand "I want to see the sales of mobile phones in the past six months" will be normalized to "Statistics of mobile phone sales in the past 6 months" to ensure that this application can be understood and processed more accurately.

[0034] In step S3, an SQL vector knowledge base is established, the basic SQL statements are converted into SQL statement vectors through an embedding model and stored in the SQL vector knowledge base, and the similarity between the standard user requirements and the relevant knowledge in the SQL vector knowledge base is calculated using the RAG retrieval enhancement generation technology; the most similar user requirements and corresponding SQL execution statements are selected and combined into standard user requirements and similar basic SQL statements, and similar basic SQL statements are recalled, and the recalled basic SQL statements are requeued to select the top-ranked basic SQL statements;

[0035] In step S4, the table description, Prompt big model prompt words, user demand execution steps, basic SQL statements, standard user demands and similar SQL statements are input into the basic SQL execution statement fusion big model; the basic SQL execution statement fusion big model fuses the table description, Prompt big model prompt words, user demand execution steps, basic SQL statements, standard user demands and similar SQL statements to generate the final output SQL execution statement; for example, the user inputs "I want to view the total sales of all electronic products in 2023.", that is, Q = "I want to view the total sales of all electronic products in 2023", and the standard user demand Q will be obtained. standard = "Query the total sales of electronic products in 2023"; Steps to implement user requirements S = (select the sales date and amount fields in the sales data table, filter out records with a product type of 'Electronics', extract the year from the sales date, filter out records with a sales year of 2023, and sum the filtered records by amount); Basic SQL corresponding to the steps to implement user requirements = (SELECT sale_date, amount FROM sales_data, WHERE product_type = 'Electronics', SELECT YEAR(sale_date), SELECT YEAR(sale_date) = 2023).

[0036] In step S5, an error correction vector library is established, an SQL execution statement is executed and a corresponding chart is automatically generated, and the user is asked whether he is satisfied; if the execution fails, the output erroneous SQL execution statement and error message are converted into a vector, and relevant error correction examples are recalled in the error correction vector library; if the user is satisfied, the SQL execution statement that the user is satisfied with is included in the database; if the execution is successful but the user is not satisfied, the SQL execution statement that the user is not satisfied with is included in the database;

[0037] In step S6, the output SQL execution statements, error correction examples, and SQL execution statements that the user is dissatisfied with are transmitted to the error correction guidance large model for determination to confirm whether there are erroneous SQL execution statements; if it is determined that there are erroneous SQL execution statements, the error correction guidance large model generates corresponding error correction guidance, and transmits the error correction guidance to the error correction large model; the error correction large model corrects the erroneous SQL execution statements according to the error correction guidance, regenerates the final SQL execution statements, and feeds the SQL execution statements back to the error correction guidance large model for re-checking; if it is determined that there are no erroneous SQL execution statements, execute and collect user feedback, reject unqualified SQL execution statements to experts for correction, and output unsatisfactory results of the corrected SQL statements;

[0038] In step S7, the intermediate results and operation result information of steps S2 to S6 are collected and stored, and feedback optimization is performed on the intermediate results and operation result information, in order to improve the overall performance.

[0039] In this embodiment, step S1 includes the following steps:

[0040] S11, basic information D of maintenance table description base , where the basic information described in the table is D base Including table name T i 、List name C i and data type DT i , denoted as D base ={(T i ,C i ,DT i ,…)}; and for the D base ={(T i ,C i ,DT i ,…)} to supplement and edit to obtain f ML (D base ); By integrating a proprietary large model to f ML (D base ) to supplement and improve again to get f U (D base ); ML (D base ) and f U (D base ) are added to obtain the final table description, where the final table description is expressed as D = f ML (D base )+f U (D base ).

[0041] In this embodiment, in step S2, the user demand, table description and Prompt large model prompt word are spliced ​​and expressed as X=Q+D+Prompt_spilit, where Q represents the user demand, D represents the table description, and Prompt_spilit represents the Prompt large model prompt word; X=Q+D+Prompt_spilit is input into the demand decomposition large model f DS (X;θ)=(Q standard ,S,SQL) to standardize and reconstruct to obtain standardized user requirements Q standard , will standardize user demand Q standard Decompose into multiple user demand execution steps S = (s1, s2, s3, s4, ...), and generate the corresponding basic SQL statements SQL = (sql1, sql2, sql3, sql4, ...); where θ is the model parameter, f DS Indicates the need to disassemble the large model.

[0042] In this embodiment, step S3 includes the following steps:

[0043] S31, convert the basic SQL statement SQL = (sql1, sql2, sql3, sql4, ...) into a SQL statement vector V through the embedding model Q =f embedding (Q standard ).

[0044] In this embodiment, the step S3 further includes the following steps:

[0045] S32, calculate the similarity Sim(V) between the standardized user requirements and the relevant knowledge in the SQL vector knowledge base by using the BM25 and cosine similarity weighting method Q ,V i )=α·BM25(V Q ,V i )+β·cosine(V Q ,V i ), select the most similar user requirements and corresponding SQL execution statements and combine them into standard user requirements and similar basic SQL statements, and recall similar basic SQL statements, requeue the recalled basic SQL statements, and select the top five basic SQL statements; among them, V i is represented by the vector in the SQL vector knowledge base, α and β represent the weights of the BM25 and cosine similarity methods preset according to experience, Sim(V Q ,V i ) is represented as the weighted final vector similarity.

[0046] In this embodiment, in step S4, table description D, Prompt large model prompt word Prompt_spilit, user demand execution step S = (s1, s2, s3, s4, ...), basic SQL statement, standard user demand Q standard The similar SQL statements are input into the basic SQL execution statement fusion model to generate the final output SQL execution statement represented as SQL final_1 =f merge (Q standard ,S,SQL,D,Prompt merge ), where is the corresponding basic SQL statement, Prompt merge Indicates the prompt word for the basic SQL statement fusion model, f merge It is represented as a large model of basic SQL execution statements. SQL execution statements are represented as SQL final_1 .

[0047] In this embodiment, in step S5, SQL fimal_1 And automatically generate the corresponding chart Chart = f viz (SQL final_1 ), if the execution fails, the wrong SQL execution statement SQL will be output final_1 And the error message error will be f through the model embedding () is transformed into a vector V e , and recall the j correction examples CorrectExample with the highest similarity Sim in the correction vector library, where f viz () Executes and outputs the final SQL for the chart automatic display module final_1 .

[0048] Specifically, the present invention further improves the overall performance of the present application by optimizing vector retrieval, enhancing the processing capability of the generated model, and improving vector similarity calculation. Finally, the present application also introduces a feedback fine-tuning mechanism to accumulate weak point data and correctly generate results, and regularly fine-tune the model to ensure that the present application can self-optimize during continuous use, avoid model degradation, and maintain comprehensiveness and robustness.

[0049] By using a variety of methods to manage table description information to ensure the accuracy and comprehensiveness of table descriptions, when users describe their requirements, the subsequent RAG (enhanced retrieval generation) process is often disturbed due to the lack of professional language or redundant information. Therefore, in the process of requirement decomposition, we will also standardize and reconstruct the original requirements of users. In order to imitate the thinking process of human experts writing complex SQL, the model needs to have a deep understanding of the table structure first. Therefore, in the fine-tuning training of the large model for requirement decomposition, we added a small number of question-answer pairs involving understanding of table structure. This method helps the model to be more accurate in generating basic SQL statements. In addition, when facing complex SQL, human experts usually think about the problem step by step and solve it step by step. Therefore, when dealing with similar requirements, the model will also gradually generate basic SQL statements according to this step-by-step thinking method, thereby improving the accuracy of the final output.

[0050] The demand decomposition model has been fine-tuned and trained with a data set centered on "user demand-step-basic SQL" to enable it to accurately convert complex demands into multiple execution steps and generate corresponding SQL execution statements. The fine-tuning formula is as follows: S i =(s1,s2,s3,s4,…),SQL i =(sql1,sql2,sql3,sql4,…), where L DS is the optimization objective function of the model fine-tuning process, θ is the model parameter, X i Corresponding to the i-th user demand in the training set, f DS Indicates the need to disassemble the large model. DS The calculation here only needs to take the user demand implementation steps and the basic SQL corresponding to the user demand implementation steps, that is, f DS (X i ; The generated results of θ) will abandon the standard user needs. i and SQL i It is the calibrated steps to implement user requirements and the basic SQL statements corresponding to the steps to implement user requirements. DS The main meaning of is that the result generated by the demand decomposition model can be as close as possible to the value calibrated by the training set to achieve the purpose of fine-tuning.

[0051] In actual applications, the error correction guidance model is used to batch analyze the basic SQL statements that generate errors, find out the causes of the errors, and modify the prompts in a targeted manner. After processing n pieces of erroneous data, the model will correct the prompts based on the analysis results and put them into actual use after human review. This can continuously improve the generation capabilities of the model, making it more intelligent and accurate when processing similar requirements.

[0052] In practical applications, it is found that the recall results of normalized user requirements in the SQL vector knowledge base are unstable. Therefore, the normalized user requirements are converted into SQL statement vectors through the embedding model, and then the similarity is calculated with the user requirements in the SQL vector knowledge base. However, the embedding model has limitations in its understanding of specific domain knowledge, resulting in the vectors generated by sentences with similar semantics in the domain being dissimilar. To this end, we fine-tune the embedding model using the dataset in the domain to improve its performance in the specific domain. The embedding model is fine-tuned by L(a,p,n)=max(d(f(a),f(n))-d(f(a),f(p))+α,0), where a represents the anchor sample, p represents the positive sample similar to the anchor, and n represents the negative sample different from the anchor; f() represents the embedding model; d() represents the distance between samples; α is a hyperparameter representing the minimum interval between positive and negative samples. The overall meaning is that the vector representation of the anchor sample should be as close as possible to the vector representation of the positive sample, and as far away from the vector representation of the negative sample as possible, so that the subtraction between the two can be maximized. L(a,p,n) ensures that the generated embedding can effectively distinguish between similar data in the domain and data outside the domain by minimizing the distance between the anchor and the positive sample and maximizing the distance between the anchor and the negative sample.

[0053] It should be noted that a single similarity calculation method may not be accurate enough, so we use BM25 and cosine similarity weighted method to calculate similarity and re-rank the recalled knowledge. After standardizing user requirements and calculating the vector similarity of each knowledge in the knowledge base, we select the 20 records with the highest similarity. Then, use an additional model to re-rank these 20 records (i.e., reranker) and select the top five knowledge items as the final reference knowledge for the large model. This can effectively improve the accuracy of the retrieval results and provide a more reliable foundation for subsequent SQL generation.

[0054] like Figure 2 As shown in the figure, in order to improve the effect of the basic SQL execution statement fusion model in actual use, the following optimizations are made: (1) The basic SQL statement fusion prompt is continuously optimized through the feedback information generated by the basic SQL execution statement fusion model. (2) With reference to the above input content, a training set is created and the basic SQL execution statement fusion model is fine-tuned. (3) Based on the feedback data of the generated results, it is regularly fine-tuned.

[0055] Chain 1 performs well in terms of response time and accuracy, and is particularly suitable for time-sensitive scenarios such as real-time interactions. However, for some application scenarios that require higher accuracy, the SQL execution statements output by Chain 1 still have room for improvement. In addition, its generated results can also be used as high-quality training data after processing. Therefore, we propose Chain 2, which outputs the unsatisfactory results of the corrected SQL statements, in order to further optimize the performance of the entire chain by utilizing the generated results of Chain 1. Chain 2 uses the output of Chain 1 as a basis for more in-depth analysis and adjustments to meet the needs of applications with higher accuracy requirements.

[0056] In order to improve the effect of the error correction guidance model in actual use, the following optimizations are made: (1) Compare the effects of different models and select the most suitable error correction guidance model for error correction guidance. (2) Continuously optimize the error correction guidance prompt through the feedback information generated by the error correction guidance model to improve the accuracy of judgment and guidance. (3) Pre-fine-tune using the (error case, error correction guidance) data set. (4) Add table descriptions, standardize user requirements and other information in the previous steps as input to provide more comprehensive context to help model judgment. (5) Regularly fine-tune it based on the feedback data of the generated results.

[0057] It should be noted that three methods are provided to activate the chain error correction guidance model, including: (1) forced activation: all output SQL execution statements must be submitted to the error correction guidance model for judgment. (2) failed activation: examples of SQL execution statement execution failure must be submitted to the error correction guidance model for judgment. (3) user dissatisfaction activation: examples of SQL execution statement execution success but user dissatisfaction must be submitted to the error correction guidance model for judgment.

[0058] Regularly use the error correction guidance model to analyze the failed and dissatisfied results of the SQL execution statements, find out the causes of the fusion errors, and optimize Prompt in a targeted manner. Regularly use the error correction guidance model to analyze the dissatisfied examples of the SQL statements and the experts' error correction results, explore the reasons for the failure to identify errors and the failure of error correction, and optimize Prompt accordingly. User dissatisfaction with the output indicates that there is a problem with the generated SQL statement, which may be because the error correction guidance model failed to identify the error, or the error correction process failed to successfully correct it, or the number of corrections exceeded the set threshold.

[0059] Since each prompt indicates that the error correction guidance model performs different tasks, feedback optimization for different prompts requires analysis of different outputs or intermediate results. Usually, these failed tasks are collectively referred to as fail_example. During the link execution process, the corrected form can be collected, which is called correct_example. For example, the generated SQL execution statement fails to run or the result that the user is not satisfied with is recorded as fail_example. These failed results will be corrected, and the corrected examples are regarded as correct_example. These failed examples and the corresponding corrected examples are batched and submitted to the error correction guidance model for analysis. This process is called Analysis&Promote. The error correction guidance model will return an optimized prompt based on the analysis. promote .

[0060] Regularly use the large model to analyze the run failures and user-unsatisfactory results in the output of Chain 1, find out the causes of fusion errors, and optimize Prompt in a targeted manner.

[0061] Regularly use the big model to analyze the examples of user dissatisfaction and expert error correction results in Chain 2 output, explore the reasons for failure to identify errors and error correction failures, and optimize Prompt accordingly. User dissatisfaction with Chain 2 output indicates that there are problems with the generated SQL statements, which may be because the error correction guidance big model failed to identify the error, or the error correction process failed to successfully correct it, or the number of corrections exceeded the set threshold.

[0062] Prompt promote

[0063] =Analysis&Promote(fail_example,correct_example,Prompt) Since each Prompt instructs the large model to perform different tasks, feedback optimization for different Prompts requires analysis of different outputs or intermediate results. Usually, these failed tasks are collectively referred to as fail_example. During the execution of the link, the corrected form can be collected, which is called correct_example. For example, the results of the failed operation or user dissatisfaction generated in chain 1 are recorded as fail_example, and chain 2 will correct these failed results, and the corrected samples are regarded as correct_example. These failed examples and the corresponding corrected examples are batched and submitted to the large model for analysis. This process is Analysis&Promote, and the model will return an optimized Prompt based on the analysis. promote .

[0064] Examples of failed operations and user dissatisfaction in the output of chain 1 are usually corrected in chain 2. Comparing these data (wrong SQL and corrected SQL) is the weak part of the basic SQL execution statement fusion model. By transferring these comparison data to the generated SQL knowledge base in real time, real-time optimization can be achieved.

[0065] For those examples that are judged as error-free by the error correction guidance model but fail in actual operation or are not satisfactory to users, they are usually corrected in the expert error correction stage. This part of data (wrong SQL, error correction guidance, correct SQL) is exactly the weak part of the SQL error correction guidance model. By transferring these comparison data to the error correction knowledge base in real time, real-time optimization can be achieved.

[0066] Vector_Store_Append(fail_example,correct_example)

[0067] Since the content provided by each knowledge base is different, the supplement for different knowledge bases needs to be determined according to the large model tasks they serve, usually focusing on cases where large model tasks fail. These failed tasks are collectively referred to as fail_example. During the link execution process, the corrected version can be collected, called correct_example. For example, the run failure or user-unsatisfied results generated in chain 1 are recorded as fail_example, while chain 2 will correct these failures, and the corrected samples are considered correct_example. Subsequently, these failed cases and the corresponding corrected cases are processed into a format consistent with the knowledge vector library through the embedding model and stored in the vector knowledge library in real time. This process is called Vector_Store_Append.

[0068] (1) Fine-tuning the basic SQL execution statement fusion model feedback: Examples of failed operations and user dissatisfaction in the output of chain 1 are usually corrected in chain 2. Comparing these data (wrong SQL and corrected SQL) is exactly the weak point of the basic SQL execution statement fusion model. Combining them with some successfully generated SQL cases to create a training set, fine-tuning the basic SQL execution statement fusion model can improve the accuracy of chain 1.

[0069] (2) Fine-tuning the error correction guidance model with feedback: For those examples that are judged as error-free by the error correction guidance model but fail in actual operation or are not satisfactory to users, they are usually corrected in the expert error correction phase. This data (error SQL and corrected SQL) is combined with some successful error identification cases to create a training set, and the error correction guidance model is fine-tuned to improve its accuracy.

[0070] (3) Fine-tuning the error correction model with feedback: SQL statements that require multiple corrections to correct and expert corrections (wrong SQL statements, correction guidance, and correct SQL statements) reflect the weaknesses of the error correction model. By combining them with some cases where one correction was successful, we can create a training set and fine-tune the error correction model to improve its error correction capability.

[0071] The above large models each undertake different tasks, but when performing feedback fine-tuning, they all need to combine the generated correct examples and the corrected erroneous SQL into the training set in proportion. This strategy avoids the "capability degradation" problem that may occur when the model is trained only with weak point data, that is, forgetting the tasks that it was able to handle correctly.

[0072]

[0073] Among them, L represents the loss function in the fine-tuning process, and N is the total number of samples in the fine-tuning dataset. The role of the loss function is to optimize the performance of the model by minimizing the distance between the answer generated by the model and the standard answer (correct_example). In this process, in order to more specifically improve the performance of the model in the weak links, the cases where the model task failed (fail_example) and its corrected results (correct_example) were specially collected and added to the fine-tuning dataset. At the same time, in order to prevent the model from forgetting the previously learned capabilities during the fine-tuning process, the fine-tuning dataset is still dominated by successful execution cases, and fail_example and correct_example are added in appropriate proportions. This method ensures that the model maintains its original capabilities without degrading while strengthening its weaknesses.

[0074] The present invention enhances the error correction link through the innovative Text2SQL application architecture, realizes the automation of SQL generation and error correction, and significantly improves the generation efficiency and accuracy of complex queries. To meet the needs of different scenarios, this application adopts a dual-link architecture: Link 1 focuses on response speed, while Link 2 prioritizes accuracy. At the same time, Link 1 optimizes the path by imitating expert thinking to further improve the response effect.

[0075] The present invention further improves the performance of this application through a number of optimization methods. First, multiple RAG (retrieval enhanced generation) modules are set up, and their shortcomings are optimized. Through embedding fine-tuning and similarity calculation optimization, this application is more accurate in semantic matching, and real-time optimization is achieved through feedback supplementation. Secondly, this application automatically adjusts Prompt through a feedback mechanism to improve the accuracy and consistency of the generated results. In addition, this application compares and selects large models, and fine-tunes large models, while optimizing the fine-tuning strategy to ensure that it performs better in different tasks. By integrating and applying technology, not only the cost of manual intervention and maintenance is significantly reduced, but also the intelligence, robustness and user experience of this application are improved, and it is widely applicable to multiple fields such as database management and data analysis.

[0076] The beneficial effects of the SQL generation and error correction method based on multi-stage feedback provided by the embodiment of the present invention are at least:

[0077] 1. The present invention optimizes the management and maintenance of table descriptions in a variety of ways to ensure the accuracy and comprehensiveness of description information, including automatic maintenance of basic expressions, support for manual annotation of table descriptions, and supplementation of table descriptions through proprietary large models, to avoid affecting the final effect due to inaccurate basic table descriptions.

[0078] 2. When user requirements are complex, the present invention simulates the thinking mode of human experts, standardizes the requirements and breaks them down into multiple steps, gradually generates and combines them into complete SQL statements, and significantly improves the ability of the present application to handle complex tasks.

[0079] 3. Select the most suitable base model for each task by comparing different large language models. Secondly, create a data set for each task to fine-tune the model first, and insert some data that understands the table structure itself into the model fine-tuning data set.

[0080] 4. The present invention designs a self-improvement mechanism to build a fine-tuning data set by accumulating weak points and correctly generating results. When the model generates incorrect content, the present application will make corrections in subsequent links to obtain data for strengthening the weak points of the model. In order to avoid the degradation of the present application due to over-strengthening a certain problem and to ensure the comprehensiveness and robustness of the present application, the fine-tuning data set is mainly composed of correct generation results, and the proportion of data for weak points is appropriately increased to achieve effective optimization.

[0081] 5. The present invention introduces enhanced retrieval generation (RAG) technology in multiple key links, organizes existing databases and supplements with real-time feedback, and maintains multiple vector knowledge bases, such as an example SQL vector knowledge base containing user requirements and their corresponding SQL execution statements. Before the big model generates results, this application will convert the input into a vector through the embedding model, and calculate the similarity with the examples in the vector knowledge base, and recall examples with high similarity for reference by the big model. This process significantly improves the accuracy and effect of the big model in executing tasks.

[0082] 6. The present invention further fine-tunes the embedding model that converts text into vectors, and adopts targeted optimization strategies for different problems encountered in different scenarios in different links, ensuring that texts of different lengths and similar semantics can also be accurately matched.

[0083] 7. The present invention adopts a dual-link design to meet the needs of multiple scenarios: This application adopts a dual-link design. Link 1 focuses on fast response to ensure timeliness; Link 2 improves accuracy through multiple rounds of error correction and inspection, which is suitable for complex demand scenarios. The dual-link design combines speed and accuracy to meet the needs of multiple scenarios.

[0084] 8. The intermediate results and final results generated by the present invention during the operation of this application are recycled, screened and processed to become high-quality fine-tuning data sets for feedback optimization of various links and models.

[0085] 9. By collecting feedback results from each link, the present invention realizes targeted fine-tuning of Prompt, and combines real-time updating of the knowledge base with fine-tuning of the model to ensure continuous optimization and self-improvement of the performance of this application.

[0086] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0087] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0088] Therefore, the above is only a preferred specific implementation mode of the present invention, and the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A SQL generation and error correction method based on multi-stage feedback, characterized in that: The method comprises the following steps: S1, collect user requirements, optimize new user requirements and output the final table description, and store the table description in the database; S2, splicing the user requirements, table descriptions and prompt words of the Prompt large model and inputting them into the requirement decomposition large model, wherein the requirement decomposition large model normalizes and reconstructs the user requirements to obtain standardized user requirements, decomposes the standardized user requirements into multiple user requirement execution steps, and generates corresponding basic SQL statements; S3, establish an SQL vector knowledge base, convert the basic SQL statements into SQL statement vectors through the embedding model and store them in the SQL vector knowledge base, use RAG retrieval enhancement generation technology to calculate the similarity between the standard user requirements and the relevant knowledge in the SQL vector knowledge base; select the most similar user requirements and corresponding SQL execution statements and combine them into standard user requirements and similar basic SQL statements, and recall similar basic SQL statements, re-queue the recalled basic SQL statements, and select the top-ranked basic SQL statements; S4, inputting the table description, the prompt word of the Prompt model, the user demand execution steps, the basic SQL statement, the standardized user demand and the similar SQL statement into the basic SQL execution statement fusion model; the basic SQL execution statement fusion model fuses the table description, the prompt word of the Prompt model, the user demand execution steps, the basic SQL statement, the standardized user demand and the similar SQL statement to generate the final output SQL execution statement; S5, establishing an error correction vector library, executing SQL execution statements and automatically generating corresponding charts, and asking the user whether he is satisfied; if the execution fails, converting the outputted erroneous SQL execution statements and error information into vectors, and recalling relevant error correction examples in the error correction vector library; if the user is satisfied, the SQL execution statements that the user is satisfied with are included in the database; if the execution is successful but the user is not satisfied, the SQL execution statements that the user is not satisfied with are included in the database; S6, transmitting the output SQL execution statements, error correction examples, and SQL execution statements that the user is dissatisfied with to the error correction guidance big model for determination to confirm whether there are erroneous SQL execution statements; if it is determined that there are erroneous SQL execution statements, the error correction guidance big model generates corresponding error correction guidance, and transmits the error correction guidance to the error correction big model; the error correction big model corrects the erroneous SQL execution statements according to the error correction guidance, regenerates the final SQL execution statements, and feeds the SQL execution statements back to the error correction guidance big model for re-checking; if it is determined that there are no erroneous SQL execution statements, execute and collect user feedback, reject unqualified SQL execution statements to experts for correction, and output unsatisfactory results of the corrected SQL statements; S7, collecting and storing the intermediate results and operation result information of steps S2 to S6, and performing feedback optimization on the intermediate results and operation result information.

2. The SQL generation and error correction method based on multi-stage feedback according to claim 1 is characterized in that: The step S1 comprises the following steps: S11, basic information D of maintenance table description base , where the basic information described in the table is D base Including table name T i 、List name C i and data type DT i , denoted as D base ={(T i ,C i ,DT i ,…)}; and for the D base ={(T i ,C i ,DT i ,…)} to supplement and edit to obtain f ML (D base ); By integrating a proprietary large model to f ML (D base ) to supplement and improve again to get f U (D base ); ML (D base ) and f U (D base ) are added to obtain the final table description, where the final table description is expressed as D = f ML (D base )+f U (D base ).

3. The SQL generation and error correction method based on multi-stage feedback according to claim 2 is characterized in that: In step S2, the user demand, table description and prompt word of the Prompt large model are spliced ​​and expressed as X=Q+D+Prompt_spilit, where Q represents the user demand, D represents the table description, and Prompt_spilit represents the prompt word of the Prompt large model; X=Q+D+Prompt_spilit is input into the demand decomposition large model f DS (X;θ)=(Q standard ,S,SQL) to standardize and reconstruct to obtain standardized user requirements Q standard , will standardize user demand Q standard Decompose into multiple user demand execution steps S = (s1, s2, s3, s4, ...), and generate the corresponding basic SQL statements SQL = (sql1, sql2, sql3, sql4, ...); where θ is the model parameter, f DS Indicates the need to disassemble the large model.

4. The SQL generation and error correction method based on multi-stage feedback according to claim 3 is characterized in that: The step S3 comprises the following steps: S31, convert the basic SQL statement SQL = (sql1, sql2, sql3, sql4, ...) into a SQL statement vector V through the embedding model Q =f embedding (Q standard ).

5. The SQL generation and error correction method based on multi-stage feedback according to claim 4 is characterized in that: The step S3 further comprises the following steps: S32, calculate the similarity Sim(V) between the standardized user requirements and the relevant knowledge in the SQL vector knowledge base by using the BM25 and cosine similarity weighting method Q ,V i )=α·BM25(V Q ,V i )+β·cosine(V Q ,V i ), select the most similar user requirements and corresponding SQL execution statements and combine them into standard user requirements and similar basic SQL statements, and recall similar basic SQL statements, requeue the recalled basic SQL statements, and select the top five basic SQL statements; among them, V i is represented by the vector in the SQL vector knowledge base, α and β represent the weights of the BM25 and cosine similarity methods preset according to experience, Sim(V Q ,V i ) is represented as the weighted final vector similarity.

6. The SQL generation and error correction method based on multi-stage feedback according to claim 5 is characterized in that: In step S4, table description D, Prompt large model prompt word Prompt_spilit, user demand execution step S = (s1, s2, s3, s4, ...), basic SQL statements, standard user requirements Q standard The similar SQL statements are input into the basic SQL execution statement fusion model to generate the final output SQL execution statement represented as SQL final_1 =f merge (Q standard ,S,SQL,D,Prompt merge ), where SQL represents the corresponding basic SQL statement, Prompt merge Indicates the prompt word for the basic SQL statement fusion model, f merge It is represented as a large model of basic SQL execution statements. SQL execution statements are represented as SQL final_1 .

7. The SQL generation and error correction method based on multi-stage feedback according to claim 6 is characterized in that: In step S5, SQL is executed final_1 And automatically generate the corresponding chart Chart = f viz (SQL final_1 ), if the execution fails, the wrong SQL execution statement SQL will be output final_1 And the error message error uses the embedding model to transform f embedding () is transformed into a vector V e , and recall the j correction examples CorrectExample with the highest similarity Sim in the correction vector library, where f viz () Executes and outputs the final SQL for the chart automatic display module final_1 .

Images