Text-to-sql model fine tuning method and data query system oriented to power grid dispatching business field

Abstract

The invention discloses a Text-to-sql model fine tuning method and a data query system oriented to the field of power grid dispatching businesses, and the method comprises the following steps: collecting database and database table information from a power grid dispatching business database, and constructing a database Schema; constructing a question and answer pair set in the power grid dispatching service field for fine tuning the large model; expanding the question-answer pair set based on a large model, generating question-answer pair data, and forming a new question-answer pair set; performing batch processing on the formed new question and answer pair data, and constructing a large model instruction fine tuning data set; finely adjusting the model. According to the scheme, various query problems in the power grid dispatching service are effectively processed, and the accuracy of automatic generation of the power grid service SQL is improved; the accuracy and the processing capability of intelligent query of power grid data are improved, and powerful support is provided for intelligent analysis and quick response of a power grid system.

Description

Technical Field

[0001] The present invention relates to a Text-to-SQL model fine-tuning method and a data query system, and in particular to a Text-to-SQL model fine-tuning method and a data query system for a power grid dispatching business field. Background Art

[0002] A large model refers to a machine learning model with large-scale parameters and complex computing structures. The purpose of designing a large model is to improve the model's expressiveness and predictive performance, and to be able to handle more complex tasks and data. Nowadays, the use of advanced large model technology to handle database queries in the field of traditional power grid dispatching faces two problems: first, the large model does not have the expertise to understand the field of power grid dispatching; second, the query statement structure of the database in the field of power grid dispatching is complex, and cross-table and nested operations are common; these problems lead to a low accuracy rate in generating SQL statements using large models. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a Text-to-SQL model fine-tuning method for the power grid dispatching business field, which improves the accuracy of automatic generation of SQL statements for power grid business through model fine-tuning; it also provides a data query system for the power grid dispatching business field, which can accurately query power grid business professional knowledge and provide support for intelligent analysis and rapid response of power grid systems.

[0004] Technical solution: The present invention provides a Text-to-SQL model fine-tuning method for the power grid dispatching business field, comprising the following steps:

[0005] (1) Collect database and database table information from the power grid dispatching business database and build a database schema;

[0006] (2) Construct a set of question-answer pairs in the power grid dispatching business domain for fine-tuning large models;

[0007] (3) Expand the question-answer pair set in step (2) based on the large model to generate question-answer pair data and form a new question-answer pair set;

[0008] (4) Batch processing the new question-answer pair data generated in step (3) to build a Text-to-SQL large model instruction fine-tuning dataset;

[0009] (5) Fine-tune the Text-to-SQL model.

[0010] Preferably, the step (1) specifically includes: collecting database and database table information from the power grid dispatching business database, and constructing a database schema, wherein the database includes several tables, and each table defines fields, primary keys and foreign keys.

[0011] Preferably, the step (2) is specifically as follows: a number of problems for the power grid dispatching business scenario are formed into a problem set Q1, each problem q i are expressed in natural language form. For each question q i ∈Q1, write the corresponding SQL statement s that can be executed and correctly answer the question i , all sql statements i Constitute the set S1,q i and the corresponding s i Constitute a question-answer pair, and all question-answer pairs constitute the question-answer pair set QS set .

[0012] Preferably, the step (3) is specifically as follows: based on the large model, the question-answer pair set QS set By expanding the model and adopting the context learning method, examples are given in the prompt words, so that the large model can learn the characteristics of user problems in the field of power grid dispatching and generate problems with typical characteristics in the field of power grid dispatching.

[0013] Preferably, the question-answer pairs expanded in step (3) are added to the question-answer pair set in step (2) to form a new question-answer pair data set.

[0014] Preferably, the step (4) is specifically as follows: batch processing the new question-answer pair data set, constructing a large model instruction fine-tuning data set, and each data item in the obtained data set is a four-tuple, which is saved in a file in json format.

[0015] Preferably, the step (5) is specifically as follows: the data set obtained in step (4) is divided into a training set and a test set, the large model Qwen-72B is used as the base, and the parameter efficient fine-tuning LoRA is used on the training set to perform model fine-tuning, and the performance of the fine-tuned model is evaluated on the test set; if the evaluation effect reaches the expected effect, the fine-tuning is completed, otherwise, steps (2)(3)(4) are repeated and fine-tuning is performed again.

[0016] Preferably, in step (3), when expanding the question-answer pair set in step (2) based on the large model and generating question-answer pair data, a temperature parameter is set to improve the diversity of the generated data.

[0017] Preferably, the four-tuple data specifically includes: a database name, an instruction to create a table of a database, a question expressed in a natural language or empty, and a SQL statement.

[0018] The data query system for the power grid dispatching business field described in the present invention includes:

[0019] User side: used for users to input questions and display answers;

[0020] Storage module: including a power grid dispatching business database, used to store power grid dispatching business knowledge;

[0021] Conversion module: used for the application of the Text-to-SQL model fine-tuning method. The Text-to-SQL model is used to automatically generate SQL statements for questions expressed by users in natural language. The generated SQL statements are made more accurate through model fine-tuning.

[0022] Execution module: executes SQL statements in the power grid dispatching business database to query answers to questions.

[0023] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: by fine-tuning the large language base model, the ability of the large model to understand and process professional terms in power grid dispatching business is enhanced, the application scenarios in the field of power grid dispatching are adapted, and the accuracy of generating SQL statements is improved; the large model is used to generate high-quality data, the amount of data for fine-tuning the large model is expanded, and the quality of the fine-tuning model is improved; the accuracy and processing capabilities of intelligent query of power grid data are improved, providing strong support for intelligent analysis and rapid response of power grid systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The present invention is a flowchart of the method. DETAILED DESCRIPTION

[0025] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0026] like Figure 1 As shown, a Text-to-SQL model fine-tuning method for the power grid dispatching business field includes the following steps:

[0027] (1) Collect database and database table information from the power grid dispatching business database and build a database schema;

[0028] (2) Construct a set of question-answer pairs in the power grid dispatching business domain for fine-tuning large models;

[0029] (3) Expand the question-answer pair set in step (2) based on the large model to generate question-answer pair data and form a new question-answer pair set;

[0030] (4) Batch processing the new question-answer pair data generated in step (3) to build a Text-to-SQL large model instruction fine-tuning dataset;

[0031] (5) Fine-tune the Text-to-SQL model.

[0032] The following is a detailed description of each step in the Text-to-SQL model fine-tuning method for the power grid dispatching business field:

[0033] Step 1: Collect database and database table information from the power grid dispatching business database and build the database schema;

[0034] The following is an example of the Schema format:

[0035] Database DB_NAME contains tables such as DB1,DB2,DB3,...,DB n1 .

[0036] Table DB1 has columns such as DB1_Col1,DB1_Col2,DB1_Col3,...,DB1_Coln2.DB1_Col1 is the primary key.

[0037] Table DB2 has columns such as DB2_Col1,DB1_Col1,DB2_Col3,...,DB2_Coln3.DB2_Col1 is the primary key. ......

[0039] Table DB n1 has columns such as DBn1_Col1,DBn1_Col2,DB2_Col1,...,DB2_Col n3 .DB n1 _Col1 is the primary key.

[0040] The DB1_Col1 ofDB1 is the foreign key ofDB1_Col1 ofDB2.

[0041] The DB2_Col1 ofDB2 is the foreign key ofDB2_Col1 ofDB n1 .

[0042] Define the database name as: DB_NAME, containing n1 tables, the table names are: DB1, DB2, DB3, ..., DB n1, define the fields, primary keys, and foreign keys of each table.

[0043] Step 2: Construct a set of question-answer pairs in the field of power grid dispatching business for fine-tuning the large model. Several questions for the power grid dispatching business scenario form the question set Q1. Each question q i are expressed in natural language; for each q i ∈Q1, write the corresponding SQL statement s that can be executed and correctly answer the question i , all sql statements can form a set S1; i and the corresponding s i Constitute a question-answer pair, and the set of all question-answer pairs is recorded as QS set .

[0044] Step 3: Based on the big model, the QS collected in step 2 set The model is expanded so that the number of question-answer pairs reaches the order of magnitude required for fine-tuning the model. The context learning method is used to give examples in the prompt words so that the large model can learn the characteristics of user questions in the field of power grid dispatching, thereby generating questions with typical characteristics in the field of power grid dispatching and improving the quality of data generation. The method is described as follows:

[0045] Input preparation: Use the QS obtained in step 1 set , concatenating it with a predefined prompt word template to create a prompt that contains examples and context.

[0046] Prompt word template construction: QS set The question-answer pairs in are concatenated according to the following prompt word template format. As an example, a complete input prompt is generated:

[0047] Objective: Set the task background, provide a reference example for the large language model, generate a problem in the field of power grid dispatch described in natural language, and write a SQL statement as a SQL expert that can be executed and correctly answer the question.

[0048] Database Schema Description: Insert the database schema obtained in step 1.

[0049] Question-answer pair example: Take any k question-answer pairs in QSset and concatenate them in the prompt word.

[0050] Requirements: Guide the model to generate sentence pairs of "problems in the field of power grid dispatching - SQL statements", including detailed steps to ensure that the model fully understands the task.

[0051] The prompt word format is as follows:

[0052]

[0053]

[0054] Since user questions in the field of power grid dispatching can be divided into different categories according to business scope, and similar question sentences have high similarity, and some phrases are repeated in multiple sentences, a Temperature parameter is set when the large model generates question-answer data, and the Temperature value is set to > 1, so as to improve the diversity of generated data. All question-answer pairs generated by the above method are added to the QS set In the question-answer pair dataset QS set '.

[0055] Step 4: Batch process the data obtained in the above steps to build a large model instruction fine-tuning dataset. Each data in the dataset contains db_id, instruction, input, and output. Among them, instruction is an instruction to create each table of a database, input is a question expressed in natural language or is empty, and output is a SQL statement. When the input is empty, the output is a SQL statement; when the input is not empty, the output is a SQL statement that can be executed correctly and returns the answer to the input question.

[0056] The data format example is as follows:

[0057]

[0058]

[0059] A quadruple constitutes a piece of data, and all quadruple data are saved in a JSON file named data.json.

[0060] Step 5: The data set for model fine-tuning formed by the above steps 1-4 is divided into two files according to a certain ratio, namely data.json, as a training set (named train.json) and a test set (named: test.json). Model fine-tuning uses the large model Qwen-72B with powerful SQL reasoning capabilities as the base, and is performed on the training set train.json. The fine-tuning method uses parameters to efficiently fine-tune LoRA. The performance of the fine-tuned model is evaluated on the test set test.json. If the test effect meets the expected effect, the fine-tuning is completed, otherwise repeat steps 2-4 and fine-tune again. This process is iterative until the fine-tuned Text-to-sql model achieves satisfactory results after testing.

[0061] A data query system for power grid dispatching business field, comprising:

[0062] User side: used for users to input questions and display answers;

[0063] Storage module: including a power grid dispatching business database, used to store power grid dispatching business knowledge;

[0064] Conversion module: used for the application of the Text-to-SQL model fine-tuning method. The Text-to-SQL model is used to automatically generate SQL statements for questions expressed by users in natural language. The generated SQL statements are made more accurate through model fine-tuning.

[0065] Execution module: executes SQL statements in the power grid dispatching business database to query answers to questions.

Claims

1. A Text-to-SQL model fine-tuning method for the power grid dispatching business field, characterized in that: The method comprises the following steps: (1) Collect database and database table information from the power grid dispatching business database and build a database schema; (2) Construct a set of question-answer pairs in the power grid dispatching business domain for fine-tuning large models; (3) Expand the question-answer pair set in step (2) based on the large model to generate question-answer pair data and form a new question-answer pair set; (4) Batch processing the new question-answer pair data generated in step (3) to build a Text-to-SQL large model instruction fine-tuning dataset; (5) Fine-tune the Text-to-SQL model.

2. According to claim 1, a Text-to-SQL model fine-tuning method for the power grid dispatching business field is characterized in that: The step (1) specifically includes: collecting database and database table information from the power grid dispatching business database, and constructing a database schema, wherein the database includes several tables, and each table defines fields, primary keys, and foreign keys.

3. According to claim 1, a Text-to-SQL model fine-tuning method for the power grid dispatching business field is characterized in that: The step (2) is specifically as follows: a number of problems for the power grid dispatching business scenario are formed into a problem set Q1, each problem q i are expressed in natural language form. For each question q i ∈Q1, write the corresponding SQL statement s that can be executed and correctly answer the question i , all sql statements i Constitute the set S1,q i and the corresponding s i Constitute a question-answer pair, and all question-answer pairs constitute the question-answer pair set QS set .

4. According to claim 1, a Text-to-SQL model fine-tuning method for the power grid dispatching business field is characterized in that: The step (3) is specifically as follows: based on the large model, the question-answer pair set QS set By expanding the model and adopting the context learning method, examples are given in the prompt words, so that the large model can learn the characteristics of user problems in the field of power grid dispatching and generate problems with typical characteristics in the field of power grid dispatching.

5. According to claim 1, a Text-to-SQL model fine-tuning method for the power grid dispatching business field is characterized in that: The question-answer pairs expanded in step (3) are added to the question-answer pair set in step (2) to form a new question-answer pair dataset.

6. A Text-to-SQL model fine-tuning method for the power grid dispatching business field according to claim 1, characterized in that: The step (4) is specifically as follows: batch processing the new question-answer pair data set, constructing a large model instruction fine-tuning data set, and each data item in the obtained data set is a four-tuple, which is saved in a file in json format.

7. A Text-to-SQL model fine-tuning method for the power grid dispatching business field according to claim 1, characterized in that: The step (5) is specifically as follows: the data set obtained in step (4) is divided into a training set and a test set, the large model Qwen-72B is used as the base, and the parameter efficient fine-tuning LoRA is used on the training set to perform model fine-tuning, and the performance of the fine-tuned model is evaluated on the test set; if the evaluation effect reaches the expected effect, the fine-tuning is completed, otherwise, steps (2)(3)(4) are repeated and fine-tuning is performed again.

8. A Text-to-SQL model fine-tuning method for the power grid dispatching business field according to claim 1, characterized in that: In the step (3), when expanding the question-answer pair set in step (2) based on the large model and generating question-answer pair data, a temperature parameter is set to improve the diversity of the generated data.

9. A Text-to-SQL model fine-tuning method for the power grid dispatching business field according to claim 6, characterized in that: The four-tuple data specifically includes: a database name, an instruction to create a table of the database, a question expressed in a natural language or empty, and an SQL statement.

10. A data query system for the power grid dispatching business field, characterized in that: include: User side: used for users to input questions and display answers; Storage module: including a power grid dispatching business database, used to store power grid dispatching business knowledge; Conversion module: used for the application of the Text-to-SQL model fine-tuning method. The Text-to-SQL model is used to automatically generate SQL statements for questions expressed by users in natural language. The generated SQL statements are made more accurate through model fine-tuning. Execution module: executes SQL statements in the power grid dispatching business database to query answers to questions.

Images