Method and system for converting natural language to SQL (Structured Query Language) statement and electronic equipment

Abstract

The invention provides a method and a system for converting a natural language into an SQL (Structured Query Language) statement and electronic equipment, which are characterized in that difficulty level prediction and adaptive database mode prediction are respectively carried out on question information to obtain difficulty classification corresponding to the question information and a matched database mode; and according to the difficulty classification and the database mode corresponding to the problem information, inputting the problem information and the database mode corresponding to the problem information into the multi-agent system to obtain an SQL statement conversion result output by the multi-agent system. According to the method and the system disclosed by the invention, the problem information is classified through the mode selector, then the problem category of the problem information is input into the multi-agent system, and accurate conversion from a natural language to an SQL statement is realized by utilizing cooperation among multiple agents in the multi-agent system; the execution accuracy of the conversion task is improved, and the application cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular, to a method, system, and electronic device for converting natural language to SQL statements. Background Art

[0002] The conversion of natural language to SQL statements (Text-to-SQL) is a technology that converts natural language descriptions into corresponding SQL query statements, which can effectively assist in querying massive databases. It can generate corresponding SQL query statements from user questions on the premise of a given relational database.

[0003] In the prior art, the conversion methods of natural language to SQL statements include the method based on a pre-trained model and the method based on a large language model. However, due to the limitation of the model parameter size, the method based on the pre-trained model has a worse effect than the method based on the large language model. But for the method based on the large language model, when generating SQL each time, all table information of the database schema needs to be input as a prompt into the large language model, and the large language model needs to be called multiple times. As a result, a large number of tokens are consumed for each generated SQL statement, and the usage cost is high and the computing efficiency is low.

[0004] Therefore, the prior art needs to be further improved. Summary of the Invention

[0005] In view of the above deficiencies in the prior art, the purpose of the present invention is to provide a method, system, and device for converting natural language to SQL statements, so as to solve the defects of high cost and low computing efficiency in the conversion of natural language to SQL statements in the prior art.

[0006] In the first aspect, the present embodiment discloses a method for converting natural language to SQL statements, which includes:

[0007] Predict the difficulty level and the corresponding database schema of the problem information respectively, and obtain the difficulty classification corresponding to the problem information and the matching database schema;

[0008] According to the difficulty classification and the database schema corresponding to the problem information, input the problem information and the database schema corresponding to the problem information into a multi-agent system, and obtain the SQL statement conversion result output by the multi-agent system. Among them, there are multiple agents in the multi-agent system, and the prompts corresponding to each agent are different. One or more agents are respectively input based on the difficulty classification and the database schema.

[0009] Optionally, the step of respectively predicting the difficulty level of the question information and predicting the database schema adaptation, and obtaining the difficulty classification and the matching database schema corresponding to the question information includes:

[0010] Input the question information, database information, and pre-constructed difficulty level information into a preset pattern selector. The pattern selector obtains the difficulty level probability and the database schema selection probability according to the question information, database information, and pre-constructed difficulty level information, and determines the difficulty classification and database schema corresponding to the question information according to the difficulty level probability and the database schema selection probability.

[0011] Optionally, before the step of inputting the question information, database information, and pre-constructed difficulty level information into the constructed pattern selector, it further includes:

[0012] According to the number of keywords contained in the SQL statement, the existence of nested subqueries, and the feature arrangement and aggregation information, divide the difficulty of the SQL statements in the training set to obtain the difficulty level information.

[0013] Optionally, the pattern selector includes an encoder module and a probability prediction module;

[0014] The step that the pattern selector obtains the difficulty level probability and the database schema selection probability according to the question information, database information, and pre-constructed difficulty level information includes:

[0015] After splicing the question information, difficulty level, and database information, input them into the encoder module to obtain the question encoding information output by the encoder module;

[0016] Input the question encoding information into the probability prediction module to obtain the difficulty level probability and the database schema selection probability output by the probability prediction module.

[0017] Optionally, the pattern selector further includes: a normalization module;

[0018] The step of determining the difficulty classification and database schema corresponding to the question information according to the difficulty level probability and the database schema selection probability includes:

[0019] Input the difficulty level probability and the database schema selection probability into the normalization module in sequence to obtain the difficulty classification and database schema corresponding to the question information.

[0020] Optionally, the probability prediction module includes a long short-term memory network unit, a multi-cross attention unit, and a multi-layer perceptron unit;

[0021] The steps of inputting the problem encoding information into the probability prediction module to obtain the difficulty level probability and database mode selection probability output by the probability prediction module include:

[0022] The problem encoding information is synchronously input into the long short-term memory network unit to obtain the difficulty context information, problem context information, table name context information, and column name information output by the long short-term memory network unit;

[0023] The difficulty context information, problem context information, table name context information, and column name information are input into the multi-cross attention unit and the multi-layer perceptron unit to obtain the difficulty level probability and database mode selection probability output by the multi-layer perceptron unit.

[0024] Optionally, when the difficulty classification of the problem information is a simple type, the specific steps of inputting the problem information and the database mode corresponding to the problem information into the multi-agent system according to the difficulty classification and database mode corresponding to the problem information to obtain the SQL statement conversion result output by the multi-agent system include:

[0025] The problem information of the simple type and the database mode corresponding to the problem information are input into the first agent to obtain the first initial SQL statement output by the first agent; wherein, the first agent is driven by the first large language model and retrieves the context information corresponding to the problem information from the preset vector database as the first initial SQL statement;

[0026] The first initial SQL statement is input into the second agent to obtain the corrected SQL statement output by the second agent; wherein, the second agent is driven by the second large language model and is used to identify whether there are syntax errors in the first initial SQL statement and correct the SQL statement with syntax errors to obtain the modified SQL statement;

[0027] The corrected SQL statement is input into the third agent to obtain the final SQL statement with successful execution output by the third agent; wherein, the third agent is driven by the third large language model and is used to perform an SQL query on the corrected SQL statement, obtain the execution result of the corrected SQL statement, and send the SQL statement with an error execution result to the second agent so that the second agent corrects the SQL statement again.

[0028] Optionally, when the difficulty classification of the problem information is a complex type, the specific steps of inputting the problem information and the database mode corresponding to the problem information into the multi-agent system according to the difficulty classification and database mode corresponding to the problem information to obtain the SQL statement conversion result output by the multi-agent system include:

[0029] Input the problem information of complex types and the database schema corresponding to the problem information into the fourth intelligent agent, and obtain the second initial SQL statement output by the fourth intelligent agent; wherein, the fourth intelligent agent is driven by the fourth large language model and generates the second initial SQL statement based on task decomposition and chain of thought.

[0030] Input the second initial SQL statement into the second intelligent agent to obtain the corrected SQL statement output by the second intelligent agent.

[0031] Input the corrected SQL statement into the third intelligent agent to obtain the final SQL statement with successful execution output by the third intelligent agent.

[0032] In a second aspect, this embodiment discloses a system for converting natural language to SQL statements, which includes:

[0033] A problem classification module, configured to respectively predict the difficulty level and the corresponding database schema of the problem information, and obtain the difficulty classification corresponding to the problem information and the matching database schema.

[0034] An intelligent agent collaboration module, configured to input the problem information and the database schema corresponding to the problem information into a multi-intelligent agent system according to the difficulty classification and database schema corresponding to the problem information, and obtain the SQL statement conversion result output by the multi-intelligent agent system. Among them, there are multiple intelligent agents in the multi-intelligent agent system, and the prompt words corresponding to each intelligent agent are different. One or more intelligent agents are respectively input based on the difficulty classification and database schema.

[0035] In a third aspect, this embodiment discloses an electronic device, where the electronic device includes a processor and a memory; a computer-readable program executable by the processor is stored on the memory; when the processor executes the computer-readable program, it implements the method for converting natural language to SQL statements as described above.

[0036] Beneficial effects: The present invention provides a method, system and device for converting natural language to SQL statements. By respectively predicting the difficulty level and the corresponding database schema of the problem information, the difficulty classification corresponding to the problem information and the matching database schema are obtained; according to the difficulty classification and database schema corresponding to the problem information, the problem information and the database schema corresponding to the problem information are input into a multi-intelligent agent system, and the SQL statement conversion result output by the multi-intelligent agent system is obtained. The method and system disclosed in this application will first classify the problem information through a pattern selector, and then input it into a multi-intelligent agent system based on the problem category where the problem information is located, and utilize the collaboration between multiple intelligent agents in the multi-intelligent agent system to achieve accurate conversion between natural language and SQL statements, improve the execution accuracy of the conversion task, and reduce the application cost. Brief Description of the Drawings

[0037] Figure 1 is a flowchart of the steps of the method for converting natural language to SQL statements provided by the present invention;

[0038] Figure 2 is a schematic diagram of the principle of the method for converting natural language to SQL statements provided by the present invention;

[0039] Figure 3 is a flowchart of the steps of a specific application embodiment of the method provided by the present invention;

[0040] Figure 4 is a block diagram of the principle of the system for converting natural language to SQL statements provided by the present invention;

[0041] Figure 5 is a comparison diagram of the system provided by the present invention at different difficulty levels of Spider. Detailed Embodiments

[0042] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not used to limit the present invention.

[0043] In the prior art, there are mainly two different ways of the method for converting natural language to SQL statements. The first is the Text-to-SQL method based on a pre-trained model. This method learns the common features of the task through some pre-specified methods, then transfers the learned common features to a specific task model, and then uses the labeled data of the specific task to fine-tune the model to solve the specific downstream task. However, due to the need for high-quality labeled data and the limitation of the model parameter size, this method cannot convert to accurate SQL statements. The second is the Text-to-SQL method based on a large language model. Due to the strong generalization ability and learning ability of the large language model, significant effects can be achieved when applied to zero-shot, few-shot or in-context learning. However, since the method based on the large language model needs to input all table information of the database schema as a prompt into the large language model every time SQL is generated, and the large language model needs to be called multiple times, the usage cost is high and the calculation efficiency is low. Therefore, neither of the two commonly used methods in the prior art can meet the requirements of low cost, high efficiency and high accuracy.

[0044] To overcome the above-mentioned deficiencies, this embodiment provides a method, a system, and an electronic device for converting natural language into SQL statements. By inputting problem information and database information into a pattern selector, a difficulty classification and a database pattern corresponding to the problem information are obtained. Then, according to the determined classification, the problem information is input into a multi-agent system to obtain the SQL statement conversion result output by the multi-agent system. Since this method is based on the importance of problem classification, it first classifies the problem information using a pattern selector, and then inputs it into the corresponding agent according to the difficulty classification of the problem information, so as to realize the conversion of SQL statements based on different agents. The collaboration among multiple agents in the multi-agent system is used to achieve accurate conversion between natural language and SQL statements, improving the execution accuracy of the conversion task.

[0045] The following further describes in more detail a method, a system, and a device for converting natural language into SQL statements provided in this embodiment with reference to the accompanying drawings.

[0046] In the first aspect, this embodiment discloses a method for converting natural language into SQL statements, as Figure 1 shown, including:

[0047] Step S1: Predict the difficulty level and the corresponding database pattern of the problem information respectively to obtain the difficulty classification and the matching database pattern corresponding to the problem information.

[0048] When it is necessary to convert the problem information of a certain natural language into an SQL statement, this step is mainly used to analyze the problem information, identify the difficulty level of the problem information, classify the problem information according to the predicted difficulty level, and predict the database pattern suitable for the problem information to obtain the most suitable database pattern.

[0049] Specifically, in this step, the difficulty level of the problem information can be divided into simple type or complex type, and the database pattern most relevant to the problem information is selected from the complete database. When predicting the difficulty level, the problem information can be compared with the standard problems corresponding to the pre-set difficulty levels to predict the probabilities corresponding to different problem difficulty levels, so as to select the problem level with the highest probability value. For example: the probability values corresponding to different difficulty levels can be arranged from high to low, and the difficulty level with the highest probability value is selected.

[0050] When predicting the database pattern, the probability predictions of the table name and the column name can be carried out simultaneously, and the top several table names and the top several column names of each of these tables are selected as the database information most relevant to the problem information.

[0051] In one implementation, a preset mode selector can be used to classify the difficulty of the question and select the database mode. Then, the steps of respectively predicting the difficulty level of the question information and the corresponding database mode prediction to obtain the difficulty classification corresponding to the question information and the matching database mode include:

[0052] Input the question information, database information, and pre-built difficulty level information into a preset mode selector. The mode selector obtains the difficulty level probability and database mode selection probability based on the question information, database information, and pre-built difficulty level information, and determines the difficulty classification and database mode corresponding to the question information according to the difficulty level probability and database mode selection probability.

[0053] Combined Figure 2 and Figure 3 As shown, the preset mode selector includes multiple connected modules for processing the input information. Each module processes the received information respectively to obtain the final difficulty level and database mode (table name and column name). The mode selector includes: an encoder module, a long short-term memory network module, a cross-attention mechanism module, a multi-layer perceptron unit, and a normalization module.

[0054] Specifically, the mode selector includes an encoder module and a probability prediction module;

[0055] The steps of inputting the question information, database information, and pre-built difficulty level information into the mode selector to obtain the difficulty level probability and database mode selection probability include:

[0056] Step S11: After splicing the question information, difficulty level, and database information, input it into the encoder module to obtain the question encoding information output by the encoder module.

[0057] To achieve more accurate recognition of the question information, in this step, the question information, difficulty level, and data information are spliced, and the spliced information is input into the encoder module.

[0058] For example: Use D = {d 1 , d 2} to represent the set of difficulty levels, T = {t 1 , t 2 ,..., t N} to represent the set of tables, represents the set of columns, N represents the number of tables, represents the i-th column of the j-th table.

[0059] In one embodiment, the DeBERTa model is selected as the encoder. Compared with BERT, the DeBERTa model uses a larger dataset, a longer training time, and more refined parameter tuning, improving the training process of the BERT model and enhancing performance. The problem information, difficulty level, and database information are concatenated as the input to the encoder.

[0060] After the encoder module encodes the above concatenated information, problem encoding information is obtained, which is the vector representation corresponding to the above concatenated information.

[0061] Step S12: Input the problem encoding information into the probability prediction module to obtain the difficulty level probability and database schema selection probability output by the probability prediction module.

[0062] Input the problem encoding information into the probability prediction module to obtain the difficulty level probability and database schema selection probability output by the probability prediction module.

[0063] Specifically, the probability prediction module includes long short-term memory network units, multiple cross-attention units, and multi-layer perceptron units. The long short-term memory network units include: a difficulty bidirectional long short-term memory network, a problem bidirectional long short-term memory network, a table name bidirectional long short-term memory network, and a column name bidirectional long short-term memory network; the multiple cross-attention units include: a difficulty and problem cross-attention layer, a problem and table cross-attention layer, and a table and column cross-attention layer.

[0064] The steps of inputting the problem encoding information into the probability prediction module to obtain the difficulty level probability and database schema selection probability output by the probability prediction module include:

[0065] The problem encoding information is synchronously input into the long short-term memory network units to obtain the difficulty context information, problem context information, table name context information, and column name information output by the long short-term memory network units;

[0066] Input the difficulty context information, problem context information, table name context information, and column name information into the multiple cross-attention units and multi-layer perceptron units to obtain the difficulty level probability and database schema selection probability output by the multi-layer perceptron units.

[0067] Combined Figure 3 As shown, the bidirectional long short-term memory networks respectively obtain information in four aspects: difficulty, problem, surface, and column name. Using the difficulty and problem cross-attention layer, problem and table cross-attention layer, and table and column cross-attention layer, the difficulty, table, and column information are respectively embedded into the context of the problem to obtain more accurate difficulty level probability and database schema selection probability.

[0068] In practical applications, there may be some problem information that only contains column names and does not involve table names, which may affect the accuracy of schema selection. Therefore, in order to improve the accuracy, a method with a multi-cross attention mechanism is proposed in this step to assist the model in difficulty classification and schema selection. Specifically, the difficulty-question cross-attention layer embeds the difficulty information into the semantic context of the question. The question-table cross-attention layer embeds the question information into the semantic context of the question, and the table-column cross-attention layer embeds the table information and column information into the semantic context of the question.

[0069] Q D = MultiHeadAttn(Q, D i , D i , h)

[0070] Q T = MultiHeadAttn(Q, T i , T i , h)

[0071]

[0072] Here, D i represents the difficulty level, T i represents the table embedding, represents the column embedding, h represents the number of heads of the multi-head attention, and Q represents the question.

[0073] Furthermore, the schema selector further includes: a normalization module;

[0074] The steps of determining the difficulty classification and database schema corresponding to the question information according to the difficulty level probability and the database schema selection probability include:

[0075] Sequentially input the difficulty level probability and the database schema selection probability into the normalization module to obtain the difficulty classification and database schema corresponding to the question information.

[0076] Furthermore, before the step of inputting the question information, the database information, and the pre-constructed difficulty level information into the pre-constructed schema selector, it further includes:

[0077] According to the number of keywords included in the SQL statement, whether there are nested subqueries, and the feature layout and aggregation information, the SQL statements in the training set are divided into difficulty levels to obtain the difficulty level information.

[0078] The normalization module is used to normalize multiple probability values output by the probability prediction module to obtain the most suitable probability value, and then obtain the difficulty classification and database mode corresponding to the probability value. Further, in order to implement the preprocessed training set, the SQL statements in the Spider training set are divided into different difficulties. For example, keywords: 'where', 'group', 'order', 'limit', 'join', 'or', 'and', 'like', 'except', 'union', 'intersect','max','min', 'count','sum', 'avg'.

[0079] In specific implementation, the difficulty level classification standard is as follows: simple category: the number of keywords is less than 4 and there is no nested query. Complex category: the number of keywords is greater than or equal to 4 or there is a nested query.

[0080] Step S2: According to the difficulty classification and database mode corresponding to the question information, input the question information and the database mode corresponding to the question information into the multi-agent system to obtain the SQL statement conversion result output by the multi-agent system. Among them, there are multiple agents in the multi-agent system, and the prompt words corresponding to each agent are different. One or more agents are input respectively based on the difficulty classification and database mode.

[0081] After obtaining the difficulty level and the corresponding database mode of the question information in the above step S1, input the question information, the determined corresponding difficulty level, and the database mode into the multi-agent system to obtain the SQL statement obtained by converting the question information output by the multi-agent system.

[0082] Specifically, combined with Figure 2 As shown, when the difficulty classification of the question information is of the simple type, the specific steps of inputting the question information and the database mode corresponding to the question information into the multi-agent system according to the difficulty classification and database mode corresponding to the question information to obtain the SQL statement conversion result output by the multi-agent system include:

[0083] Step S211: Input the question information of the simple type and the database mode corresponding to the question information into the first agent to obtain the first initial SQL statement output by the first agent; among them, the first agent is driven by the first large language model, and retrieves the context information corresponding to the question information from the preset vector database as the first initial SQL statement.

[0084] For simple problems, the question-SQL pairs in the training set can be stored in a vector database, and the question-SQL pair with the highest semantic similarity to the question can be retrieved from the vector database as an example in the prompt of the first agent to generate a first preliminary SQL statement.

[0085] Step S212: Input the first initial SQL statement into the second agent to obtain a corrected SQL statement output by the second agent; wherein, the second agent is driven by a second large language model and is used to identify whether there are syntax errors in the first initial SQL statement and correct the SQL statement with syntax errors to obtain a modified SQL statement.

[0086] Step S213: Input the corrected SQL statement into the third agent to obtain a final SQL statement with successful execution output by the third agent; wherein, the third agent is driven by a third large language model and is used to perform an SQL query on the corrected SQL statement, obtain the execution result of the corrected SQL statement, and send the SQL statement with an error in the execution result to the second agent so that the second agent can correct the SQL statement again.

[0087] Further, when the difficulty classification of the question information is a complex type, the specific steps of inputting the question information and the database schema corresponding to the question information into the multi-agent system according to the difficulty classification corresponding to the question information and the database schema to obtain the SQL statement conversion result output by the multi-agent system include:

[0088] Step S221: Input the complex type of question information and the database schema corresponding to the question information into the fourth agent to obtain a second initial SQL statement output by the fourth agent; wherein, the fourth agent is driven by a fourth large language model and generates the second initial SQL statement based on task decomposition and chain of thought.

[0089] For complex problems, use the fourth agent to perform preliminary processing on them. The fourth agent first decomposes the complex problem into multiple sub-problems with a progressive relationship, and then generates SQL queries for each sub-problem respectively to gradually obtain a second preliminary SQL statement for the complete problem.

[0090] Step S222: Input the second initial SQL statement into the second agent, use the second agent to check the syntax of the second initial SQL statement, correct the second initial SQL statement with syntax errors, and output it to obtain a corrected SQL statement.

[0091] Step S223: Input the corrected SQL statement into the third agent to obtain a final SQL statement with successful execution output by the third agent.

[0092] Since the initial SQL statement may fail to execute, in both step S212 and step S222, the initial SQL statement is input into the second intelligent agent. The second intelligent agent performs a syntax check on the initial SQL statement and corrects the syntax of the initial SQL statement with syntax errors. The corrected SQL statement is then input into the third intelligent agent. The third intelligent agent executes a query on the SQL statement that has no syntax errors, determines whether the execution is successful. If the execution is successful, the output is the final SQL statement. Otherwise, the SQL statement that fails to execute is input into the second intelligent agent to correct the SQL statement using the second intelligent agent. The second intelligent agent inputs the corrected SQL statement into the third intelligent agent again for query execution. After successful execution, it is confirmed as the final SQL statement.

[0093] In specific implementation, the first intelligent agent can be respectively called an SQL developer, the second intelligent agent can be called an SQL expert, the third intelligent agent can be called an SQL executor, and the fourth intelligent agent can be called an SQL researcher. For simple questions, the SQL developer uses retrieval augmentation technology to retrieve similar Q&A-SQLs from the vector database as context information. For complex questions, the SQL researcher generates an initial SQL query based on task decomposition and chain of thought (CoT). Then these initial SQLs are first syntax-checked by the SQL expert, and the SQL statements with syntax errors are corrected. The SQL statements without syntax errors are transmitted to the SQL executor for query execution. If the execution is successful, the output is the final SQL query. If not, the SQL expert is called to modify the SQL statement according to the error information and resubmitted to the SQL executor for execution and verification.

[0094] Generating an executable SQL query Y based on the LLM is represented as:

[0095] Y = LLM(Q, S, I|θ)

[0096] Where Q represents the question information. S is the database schema corresponding to (or aligned with) the question information. I represents the instruction for the Text-to-SQL task, which guides the large language model (LLMs) to generate accurate SQL queries. LLM(·|θ) is the large language model with parameters θ.

[0097] In a second aspect, the present embodiment discloses a system for converting natural language to SQL statements, as Figure 4 shown, including:

[0098] A question classification module 410, configured to respectively predict the difficulty level and the database schema adapted to the question information for the question information, so as to obtain a difficulty classification corresponding to the question information and a matching database schema; its function is as shown in step S1.

[0099] The agent collaboration module 420 is configured to input the question information and the database schema corresponding to the question information into at least a multi-agent system according to the difficulty classification corresponding to the question information and the database schema, so as to obtain the SQL statement conversion result output by the multi-agent system. Wherein, the multi-agent system contains multiple agents, and the prompt words corresponding to each agent are different. One or more agents are respectively input based on the difficulty classification and the database schema, and its functions are as shown in step S2.

[0100] In a third aspect, the present embodiment discloses an electronic device, wherein the electronic device includes a processor and a memory; a computer-readable program executable by the processor is stored on the memory; when the processor executes the computer-readable program, the natural language to SQL statement conversion method described above is implemented.

[0101] In order to further illustrate the effects achieved by the method and system provided in the present application, the following verification was carried out.

[0102] 1. Dataset

[0103] Experiments were conducted on the cross-domain large-scale Text-to-SQL benchmarks: Spider and CSpider. The robustness of the system provided in this embodiment in three more challenging benchmarks: Spider-DK, Spider-SYN, and Spider-Realistic. The Spider dataset contains 7,000 question-SQL pairs in the training set and 1,034 question-SQL pairs in the development set, covering 200 different databases and 138 domains. CSpider is a large and complex cross-domain semantic parsing and text-to-SQL dataset, which is a Chinese dataset translated from Spider. Spider-DK, Spider-Syn, and Spider-Realistic are variants derived from the original Spider dataset. They are designed to mimic the questions that users may ask in real-world scenarios.

[0104] 2. Evaluation metrics

[0105] Two evaluation metrics are considered - execution accuracy (EX) and exact match accuracy (EM). EX quantifies the percentage of generated SQL queries that have the same execution result as the true query in the evaluation questions. EM measures the accuracy of the predicted SQL clauses by treating each clause as an independent set and ensuring that it exactly corresponds to the clauses of the reference query. The SQL prediction is considered accurate only when all elements are exactly the same as the true query components.

[0106] 3. Experimental settings

[0107] All experiments were run on 4 NVIDIA V100 GPUs, each with 32GB of memory. In the first stage, DeBERTa-v3 was used as the encoder, and the number of heads h in the multi-head attention mechanism was set to 8. The optimization process used the AdamW optimizer with a batch size of 8 and a learning rate set to 2e-5. In the second stage, gpt-4-turbo was used as the agent, and specific prompts were designed for different agents. Specifically, a 3-shot setting was adopted in the prompt for SQL Developer. The temperature of each agent was set equal to 0 to actively utilize the deterministic features of the model. Additionally, Chroma was used as the vector database.

[0108] 4. Experimental Results

[0109] In Table 1, the performance of the pattern selector on different datasets is shown. The AUC metric was used to evaluate the classification accuracy of the model. According to the experimental results, the pattern selector achieved an AUC score of 0.9984 for table classification and 0.9973 for column classification on the Spider dataset, both exceeding the performance metrics of the CSpider dataset. Additionally, for question difficulty classification, the difficulty AUC score was 0.9893. This indicates that the pattern selector can accurately classify questions and link patterns.

[0110] Table 1 Pattern Selector Performance

[0111]

[0112] Table 2 shows the comparative performance of QCMA-SQL and other baseline methods on the Spider development set. The results show that our method outperforms all baseline methods in terms of execution accuracy. Specifically, the execution accuracy of QCMA-SQL reached 86.1%, which is 0.7% higher than the second-best method (Codes-7B).

[0113] Table 2 Results of QCMA-SQL on Spider

[0114]

[0115] Table 3 shows the performance comparison between the method proposed in this study and several baseline methods on the CSpider dataset. QCMA-SQL is 0.9% higher than the second-best method. The table names and column names in the CSpider dataset are in English, but the questions are in Chinese, which increases the challenge of mapping questions to the database structure. The results show that the method of this application performs best on the CSpider dataset. This fully verifies the effectiveness and robustness of the method disclosed in this application.

[0116] Table 3 Results of QCMA-SQL on CSpider

[0117]

[0118] Table 4 evaluates the robustness of QCMA-SQL on three Spider variants: Spider-DK, Spider-Syn, and Spider-Realistic. The experimental results show that QCMA-SQL performs excellently compared to the best baseline. It achieves a 1.2% gain on Spider-Syn (from 81.4% to 82.6%) and a 2.7% gain on Spider-Realistic (from 83.1% to 85.8%). These results demonstrate the good generalization ability of the model on challenging real-world scenario datasets.

[0119] Table 4 Robustness of QCMA-SQL on Spider Variants

[0120]

[0121] As Figure 5 shown, these four difficulty levels are based on the official test suite provided by Spider. The results show that the system provided in this embodiment outperforms other models at all levels. It achieves a 0.9% gain at the simple level, 1.1% at the medium level, 8.6% at the difficult level, and 2.4% at the extremely difficult level. This is attributed to the multi-agent collaboration framework's ability to explicitly handle problems of different difficulties, indicating that the method disclosed in this embodiment performs excellently in handling SQL queries of any difficulty.

[0122] The present invention provides a method, system, and device for converting natural language to SQL statements. By respectively predicting the difficulty level and the corresponding database schema of the problem information, the difficulty classification and the matching database schema corresponding to the problem information are obtained; according to the difficulty classification and the database schema corresponding to the problem information, the problem information and the database schema corresponding to the problem information are input into a multi-agent system to obtain the SQL statement conversion result output by the multi-agent system. The method and system disclosed in this application will first classify the problem information through a pattern selector, and then input it into the multi-agent system based on the problem category where the problem information is located. By utilizing the collaboration among multiple agents in the multi-agent system, the accurate conversion between natural language and SQL statements is realized, improving the execution accuracy of the conversion task and reducing the application cost.

[0123] Those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in this application. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present invention are pointed out by the following claims.

[0124] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0125] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or in combination with these instruction execution systems, apparatus, or devices.

[0126] It should be understood that each part of this application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following technologies well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0127] It can be understood that the above embodiments are exemplary and should not be construed as limitations on this application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for converting natural language to SQL statements, characterized in that: include: Predicting the difficulty level and the matching database model of the question information respectively, and obtaining the difficulty classification corresponding to the question information and the matching database model; According to the difficulty classification and database model corresponding to the problem information, the problem information and the database model corresponding to the problem information are input into the multi-agent system to obtain the SQL statement conversion result output by the multi-agent system, wherein the multi-agent system contains multiple agents, each of which has a different prompt, and one or more agents are input respectively based on the difficulty classification and the database model.

2. The method for converting natural language to SQL statements according to claim 1, characterized in that: The steps of respectively predicting the difficulty level and the matching database model of the question information to obtain the difficulty classification corresponding to the question information and the matching database model include: The problem information, database information and pre-constructed difficulty level information are input into a preset mode selector, and the mode selector obtains the difficulty level probability and the database mode selection probability according to the problem information, database information and the pre-constructed difficulty level information, and determines the difficulty classification and database mode corresponding to the problem information according to the difficulty level probability and the database mode selection probability.

3. The method for converting natural language to SQL statements according to claim 1, characterized in that: Before the step of inputting the question information, database information and pre-built difficulty level information into the constructed mode selector, the method further includes: According to the number of keywords contained in the SQL statement, whether there are nested subqueries, and feature arrangement and aggregation information, the difficulty of the SQL statements in the training set is divided to obtain difficulty level information.

4. The method for converting natural language to SQL statements according to claim 3, characterized in that: The mode selector includes an encoder module and a probability prediction module; The step of obtaining the difficulty level probability and the database mode selection probability according to the problem information, the database information and the pre-built difficulty level information by the mode selector comprises: After the question information, the difficulty level and the database information are spliced, the information is input into the encoder module to obtain the question encoding information output by the encoder module; The question coding information is input into a probability prediction module to obtain the difficulty level probability and database mode selection probability output by the probability prediction module.

5. The method for converting natural language to SQL statements according to claim 4, characterized in that: The mode selector further includes: a normalization module; The step of determining the difficulty classification and database mode corresponding to the question information according to the difficulty level probability and the database mode selection probability comprises: The difficulty level probability and the database mode selection probability are sequentially input into the normalization module to obtain the difficulty classification and database mode corresponding to the question information.

6. The method for converting natural language to SQL statements according to claim 4 or 5, characterized in that: The probability prediction module includes a long short-term memory network unit, a multiple cross attention unit and a multi-layer perception unit; The step of inputting the question coding information into the probability prediction module to obtain the difficulty level probability and the database mode selection probability output by the probability prediction module comprises: The question encoding information is synchronously input into the long short-term memory network unit to obtain the difficulty context information, question context information, table name context information and column name information output by the long short-term memory network unit; The difficulty context information, question context information, table name upper and lower information and column name information are input into the multiple cross attention units and the multi-layer perception unit to obtain the difficulty level probability and database mode selection probability output by the multi-layer perception unit.

7. The method for converting natural language to SQL statements according to claim 1, characterized in that: When the difficulty classification of the question information is a simple type, the specific steps of inputting the question information and the database schema corresponding to the question information into the multi-agent system according to the difficulty classification corresponding to the question information and the database schema, and obtaining the SQL statement conversion result output by the multi-agent system include: Inputting simple type question information and a database model corresponding to the question information into a first agent, obtaining a first initial SQL statement output by the first agent; wherein the first agent is driven by a first large language model, and retrieves context information corresponding to the question information from a preset vector database as the first initial SQL statement; Input the first initial SQL statement into the second agent, and obtain a modified SQL statement output by the second agent; wherein the second agent is driven by the second largest language model, and is used to identify whether the first initial SQL statement contains a grammatical error, and correct the SQL statement containing the grammatical error to obtain a modified SQL statement; The corrected SQL statement is input into the third agent, and the final SQL statement of successful execution is output by the third agent; wherein the third agent is driven by the third largest language model, and is used to perform SQL query on the corrected SQL statement, obtain the execution result of the corrected SQL statement and send the SQL statement with an erroneous execution result to the second agent, so that the second agent can correct the SQL statement again.

8. The method for converting natural language to SQL statements according to claim 7, characterized in that: When the difficulty classification of the question information is a complex type, the specific steps of inputting the question information and the database schema corresponding to the question information into the multi-agent system according to the difficulty classification corresponding to the question information and the database schema, and obtaining the SQL statement conversion result output by the multi-agent system include: Inputting complex type of question information and a database model corresponding to the question information into the fourth agent, and obtaining a second initial SQL statement output by the fourth agent; wherein the fourth agent is driven by the fourth language model and generates the second initial SQL statement based on task decomposition and thinking chain; Inputting the second initial SQL statement into the second agent to obtain a modified SQL statement output by the second agent; The modified SQL statement is input into the third agent, and the final SQL statement which is successfully executed and output by the third agent is obtained.

9. A natural language to SQL statement conversion system, characterized in that: include: A question classification module is used to predict the difficulty level and the matching database model of the question information, and obtain the difficulty classification corresponding to the question information and the matching database model; The intelligent agent collaboration module is used to input the problem information and the database model corresponding to the problem information into the multi-agent system according to the difficulty classification and database model corresponding to the problem information, and obtain the SQL statement conversion result output by the multi-agent system, wherein the multi-agent system contains multiple agents, and the prompts corresponding to each agent are different. One or more agents are input respectively based on the difficulty classification and database model.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory; the memory stores a computer-readable program that can be executed by the processor; when the processor executes the computer-readable program, the method for converting a natural language into an SQL statement as described in any one of claims 1-8 is implemented.

Images