A method and device for training an SQL statement generation model

Abstract

The application provides a SQL statement generation model training method and device, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a plurality of candidate questions in a target business scenario; for each candidate question, generating a plurality of candidate SQL statements corresponding to the candidate question based on business data and core words in the target business scenario; performing positive and negative example marking and division processing on the candidate SQL statements based on execution data of each candidate SQL statement to obtain a first sample set and a second sample set; performing supervised fine-tuning processing on a basic model by using the first sample set to obtain an intermediate model; calculating a target reward based on the format, execution data and content quality of a predicted SQL statement, and adjusting model parameters of the intermediate model based on the target reward to obtain a SQL statement generation model. The application designs a reward function from multiple dimensions to ensure that the SQL statement generation model can accurately generate high-quality SQL statements in complex scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a SQL statement generation model training method and device. BACKGROUND

[0002] Text2SQL refers to a technology of converting natural language queries into SQL query statements, aiming to enable non-programming professional users to interact with database systems through natural language without understanding complex SQL syntax. Through this technology, users only need to input simple Chinese query questions, and the system can automatically generate corresponding SQL query statements and extract corresponding results from the database.

[0003] With the rapid development of large language models, necessary data can be provided through prompt words, and then the user query is converted into a SQL statement through a large language model. However, in scenarios where complex SQL statements need to be generated, for example, in scenarios where the generated SQL statement is long, such models still cannot accurately generate high-quality SQL statements. SUMMARY

[0004] To solve the above problems, the purpose of the present application is to provide a SQL statement generation model training method and device, which can construct high-quality training data and design reward functions from multiple dimensions to improve the generation accuracy of the SQL statement generation model in complex scenarios.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] On the one hand, the present application provides a SQL statement generation model training method, comprising:

[0007] Obtaining a plurality of candidate questions in a target business scenario, the candidate question being a question solved by a SQL query statement;

[0008] For each candidate question, generating a plurality of candidate SQL statements corresponding to the candidate question based on business data in the target business scenario and core words;

[0009] Based on the execution data of each candidate SQL statement, performing positive and negative example labeling and division processing on the candidate SQL statement to obtain a first sample set and a second sample set;

[0010] Supervised fine-tuning the base model with the first sample set to obtain an intermediate model;

[0011] The target reward is calculated based on the format, execution data and content quality of the predicted SQL statement, and the model parameters of the intermediate model are adjusted based on the target reward to obtain a SQL statement generation model, the predicted SQL statement being obtained by the intermediate model reasoning on the candidate question in the second sample set.

[0012] In another aspect, the present application also provides a SQL statement generation model training device, comprising:

[0013] The obtaining module is configured to obtain a plurality of candidate questions in a target business scenario, the candidate questions being questions solved by SQL query statements.

[0014] The generating module is configured to generate, for each candidate question, a plurality of candidate SQL statements corresponding to the candidate question based on business data and core words in the target business scenario.

[0015] The sample dividing module is configured to divide and process the candidate SQL statements into positive examples and negative examples based on execution data of each candidate SQL statement to obtain a first sample set and a second sample set.

[0016] The fine-tuning module is configured to perform supervised fine-tuning processing on the base model with the first sample set to obtain an intermediate model.

[0017] The adjusting module is configured to calculate a target reward based on the format, execution data and content quality of the predicted SQL statement, and adjust the model parameters of the intermediate model based on the target reward to obtain a SQL statement generation model, the predicted SQL statement being obtained by the intermediate model reasoning on the candidate question in the second sample set.

[0018] In another aspect, the present application also provides an electronic device comprising a processor and a memory, the memory storing a plurality of instructions; the processor loads the instructions from the memory to perform the steps in any of the SQL statement generation model training methods provided by the present application.

[0019] In another aspect, the present application also provides a computer-readable storage medium storing a plurality of instructions, the instructions being adapted to be loaded by a processor to perform the steps in any of the SQL statement generation model training methods provided by the present application.

[0020] In another aspect, the present application also provides a computer program product comprising computer programs / instructions, the computer programs / instructions being executed by a processor to implement the steps in any of the SQL statement generation model training methods provided by the present application.

[0021] The technical solutions provided by the present application have at least the following beneficial effects:

[0022] In the embodiment of the present application, a plurality of candidate questions in a target business scenario can be obtained; for the candidate questions, a plurality of candidate SQL statements corresponding to the candidate questions are generated by using business data and core words in the target business scenario; the candidate SQL statements are marked and divided into positive examples and negative examples by using execution data of each candidate SQL statement, and a high-quality first sample set and a second sample set can be automatically constructed; the first sample set is used to fine-tune a basic model to obtain an intermediate model; the format, execution data and content quality of a predicted SQL statement predicted by the intermediate model are comprehensively calculated to obtain a target reward, and the model parameters of the intermediate model are adjusted based on the target reward to obtain a final SQL statement generation model. The reward function is designed from multiple dimensions, which can improve the generation accuracy of the SQL statement generation model in a complex scenario, and the SQL statement generation model obtained by training can accurately generate high-quality SQL statements in a complex scenario. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 is an application scenario diagram of the SQL statement generation model training method provided by the embodiment of the present application;

[0025] Figure 2 is a flowchart of the SQL statement generation model training method provided by the embodiment of the present application;

[0026] Figure 3 is a diagram for generating auxiliary data provided by the embodiment of the present application;

[0027] Figure 4 is a diagram for calculating a target reward provided by the embodiment of the present application;

[0028] Figure 5 is a structural diagram of the SQL statement generation model training device provided by the embodiment of the present application;

[0029] Figure 6 is a structural diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] This invention proposes a training method for an SQL statement generation model. Through a designed training method, an SQL statement generation model is trained, which supports accurate generation of SQL statements in complex scenarios.

[0032] It is understood that, in specific embodiments of the present invention, data involving user information and related data requires user permission or consent, and the collection, use and processing of such data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0033] See also Figure 1 This diagram illustrates an application scenario for the SQL statement generation model training method. The application scenario can include a terminal 101 and a server 102, which can exchange data via a network. The terminal 101 can be a mobile phone, tablet, smart Bluetooth device, computer, large screen, robot, or other similar device; the server 102 can be a single server or a server cluster consisting of multiple servers.

[0034] Multiple candidate questions in the target business scenario can be collected in advance or obtained by the server 102 after data processing. Specifically, the terminal 101 can send the target business scenario or candidate questions to the server 102. The server 102 can obtain the candidate questions in the target business scenario. For each candidate question, based on the business data and core words in the target business scenario, multiple candidate SQL statements are generated corresponding to the candidate question. Based on the execution data of each candidate SQL statement, the candidate SQL statements are marked and divided into positive and negative examples to obtain a first sample set and a second sample set. The basic model is subjected to supervised fine-tuning processing using the first sample set to obtain an intermediate model. The target reward is calculated based on the format, execution data, and content quality of the predicted SQL statement, and the model parameters of the intermediate model are adjusted based on the target reward to obtain an SQL statement generation model. The predicted SQL statement is obtained by the intermediate model through reasoning on the candidate questions in the second sample set.

[0035] After the SQL statement generation model is trained, the server 102 can retain the corresponding model parameters and send notification information to the terminal 101, so that the user knows that the model has been trained. Subsequently, the user can send a user query to the server 102 through the terminal 101, and the server 102 can generate a SQL statement corresponding to the user query through the SQL statement generation model and feed back to the terminal 101. In this embodiment, a SQL statement generation model training method is provided, as shown in Figure 2 The specific process of the SQL statement generation model training method can be as follows:

[0036] S110, obtaining a plurality of candidate questions in a target business scenario.

[0037] The target business scenario refers to a real business scenario. In different business scenarios, business logic and generated business data are different. The target business scenario can be set according to actual needs. For example, the real business scenario is an industrial scenario, and the target business scenario is an industrial scenario. For another example, the real business scenario is a pension service scenario, and the target business scenario is a pension service scenario.

[0038] The candidate question is a question solved by a SQL query statement, that is, the answer corresponding to the candidate question can be obtained by searching, analyzing and processing in the corresponding database through the SQL query statement. In some embodiments, the candidate question can be selected from a real question in the target business scenario, for example, obtaining a real question in the target business scenario in a historical time period; obtaining the answer data of the real question; if the answer data contains a SQL statement, the real question is taken as a candidate question.

[0039] In other embodiments, the candidate question can also be automatically generated after analyzing the data in the database. For example, business data under the target business scenario can be obtained, and the business data includes table building information, business logic data, and business document data; core words related to the target business scenario are extracted from the business data; for each target document data, the target document data and the table building information are filled into a conversion template to construct a conversion prompt word, and the target document data is business document data containing core words; the conversion prompt word is used to guide a large language model to generate a candidate question corresponding to the target document data.

[0040] Business data refers to data generated in a target business scenario, usually text data, which can include business logic data, business document data, and these business data is usually stored in a database. Business data can also include table creation information of the database. The table creation information refers to the detailed description of the table structure and related attributes defined when creating a table in the database. Business logic data refers to data that needs to be combined with business scenarios and business processing background knowledge when performing statistical calculations and data processing, such as total power consumption, which needs to be summarized based on production conditions, and personnel quantitative performance, which needs to be calculated based on data under different businesses. Business document data refers to other data in addition to business logic data and question and answer data.

[0041] The above table creation information, business logic data, and business document data are obtained by processing business data. Based on the original table creation information of the database, general formatted table creation information can be constructed. An example of the table creation information is as follows: CREATE TABLE TABLE_NAME(COLUMN_NAME,COL_TYPE,EXAMPLES / ALL VALUES / ENUM VALUES / CHECK IN, / *VALUE MAPPING* / ......).

[0042] TABLE_NAME refers to the table name, COLUMN_NAME refers to the column name, and COL_TYPE refers to the numerical type corresponding to the column. EXAMPLES / ALL VALUES / ENUM VALUES / CHECK IN, / *VALUE MAPPING* / can be configured according to the sampling strategy. If the data sample value is not an enumeration type and the degree of repetition is not high, n samples can be randomly sampled, and the value of n can be configured according to actual needs. If the data sample value is not an enumeration type and is highly repetitive, ALL VALUES can be used to mark, and CHECK IN(v1, v2,...) can be added after it, where v1 and v2 are sample values. If the data sample value is an enumeration type, ENUM VALUES can be used to mark, and a numerical mapping / *VALUE MAPPING* / can be added at the end of the table creation information to map the corresponding concise specific business meaning. The constructed table creation information can be saved in a general format such as txt or json. When constructing the table creation information, different sample value characteristics can be distinguished in the table creation information, and the use of CHECK IN, / ** / , and other pseudo-code styles can further improve the understanding and instruction following ability of large language models.

[0043] It should be noted that if there is a historical case library accumulated in the business data, i.e., historical cases from questions to SQL statements, this part of data can be extracted as QA data. If there is no historical case library accumulated in the business data, no processing is needed.

[0044] Core words related to the target business scenario are extracted from the business data to construct a core word library. The method of extracting core words can adopt mainstream technologies such as TF-IDF, TextRank, LDA, OCR, LLM, etc. Among them, the above business data can be pre-processed and stored, and the corresponding data can be read when used.

[0045] The business document data can be sliced data. For each business document data, it can be determined whether any core word in the core word library is contained therein; if so, the business document data can be used as target document data. For each target document data, a conversion prompt word can be constructed using the target document data, table building information, and conversion template for subsequent use.

[0046] Among them, the conversion template is a prompt word template pre-set for generating candidate questions. The conversion template contains target slots and slots of table building information. The target document data and table building information are filled into the corresponding slots to obtain the conversion prompt word. Among them, the conversion template can specifically include task objectives and requirements, which can be set according to actual needs. In the embodiments of the present application, the conversion template can be:

[0047] “#Task

[0048] As an SQL learning aid, you need to design problems that can be solved by SQL queries based on the database table structure information and business materials provided by the user.

[0049] #Requirements

[0050] 1. Table structure driven principle: all problems must be based on the fields, data types and constraint conditions contained in the given CREATE TABLE statement to design executable SQL practice questions.

[0051] 2. SQL solvability verification: ensure that each problem can be solved by writing a SELECT standard SQL statement, focusing on the following abilities:

[0052] 2.1, data query and condition filtering

[0053] 2.2, table join and aggregation statistics

[0054] 3. Problem restriction conditions:

[0055] 3.1, prohibit involving database management, database addition / deletion / change, etc.

[0056] 3.2, avoid open-ended questions that require external knowledge or subjective judgment;

[0057] 3.3, exclude graphical interface operation or performance optimization and other advanced topics;

[0058] 4, direct answer, no explanation.

[0059] # Database table structure information

[0060] {P_schema}

[0061] # Business information

[0062] {P_doc_filtered_i}”

[0063] Wherein, {P_schema} can be filled with table creation information, and {P_doc_filtered_i} can be filled with target document data to obtain a conversion prompt word. The conversion prompt word is input into a large language model, and a candidate question is output through analysis and reasoning of the large language model. The large language model is set to be creative, for example, temperature = 1.0.

[0064] As an implementation manner, candidate questions can be collected from real questions, and corresponding candidate questions can also be generated by using a large language model. Of course, both manners can be used to collect candidate questions, and the candidate questions collected by the two manners are processed for deduplication and then used.

[0065] S120, for each candidate question, based on the business data in the target business scenario and the core word, a plurality of candidate SQL statements corresponding to the candidate question are generated.

[0066] Based on the foregoing description, the business data can include table creation information, business logic data, and business document data, and the core word is extracted from the business data. By using the core word and the business data, a plurality of candidate SQL statements corresponding to the candidate question can be generated. The candidate SQL statement refers to the SQL statement corresponding to the candidate question.

[0067] As an implementation manner, when the candidate SQL statement is generated, a mapping relationship between the key data and the business logic data and the business document data respectively can be obtained to obtain key associated data, the key data being window content extracted from the business logic data and the business document data based on a core word and a preset window; for each candidate question, a candidate key question is extracted from the candidate question based on the core word and the preset window, the candidate key question being window content with the core word at the center of the preset window; auxiliary data including candidate logic data and candidate document data is determined according to the key associated data and the similarity of the candidate key question and the key data; the specified prompt word is constructed based on the auxiliary data and the table building information, and the specified prompt word is used to guide the large language model to generate a plurality of candidate SQL statements corresponding to the candidate question.

[0068] The key data is data extracted from the business logic data and the business document data, which can be extracted and stored in advance, and then directly obtained. Specifically, the key data is window content extracted by using a core word and a preset window. For ease of description, the business logic data and the business document data are collectively referred to as to-be-processed data, and these data are all data processed by slicing. The preset window is a window that can slide in the to-be-processed data. The preset window slides in the to-be-processed data to extract the key data. The length of the preset window can be set according to actual needs, and in the embodiment of the present application, the length of the preset window can be set as an odd number greater than 1.

[0069] For each to-be-processed data, a word segmentation process is performed to obtain a to-be-processed word sequence; the preset window is controlled to be at the starting position of the to-be-processed word sequence, and the content in the preset window is taken as to-be-judged content; it is determined whether the to-be-judged data contains a core word; if the to-be-judged data contains any core word, the preset window is moved so that the core word is at the center position of the preset window, an offset window is obtained, and the content in the offset window is taken as key data; if the to-be-judged content does not contain a core word, the preset window is controlled to move by a specified step, and the content in the moved preset window is taken as new to-be-judged content; the step of determining whether the to-be-judged content contains a core word and subsequent steps are returned to be executed until the preset window moves to the end position of the to-be-processed word sequence.

[0070] Thus, for each piece of to-be-processed data, the corresponding key data can be extracted, and there is an association relationship between the key data and the to-be-processed data. Thus, the association relationship between each key data and the to-be-processed data can be established as key association data. One key data can correspond to multiple to-be-processed data. In order to avoid duplication of key data, the key data can be deduplicated to obtain final key relationship data, which is stored in a database for subsequent use. Through the preset window and the core word, the original data can be greatly compressed, which is convenient for subsequent retrieval.

[0071] It should be noted that the business data can include key data, core words, table building information, business logic data, and business document data. The business data can be analyzed and processed in advance for the data in the target business scenario and stored in the corresponding database. When needed, it can be directly read from the database.

[0072] For each candidate question, the candidate key question can also be extracted from the candidate question in the manner of extracting the key data. The candidate key question is the window content with the core word at the center of the preset window. Specifically, when extracting the candidate key question from the candidate question, the preset window can be controlled to be at the starting position of the question tokenization sequence, and the content in the preset window can be taken as the to-be-judged content. The question tokenization sequence is obtained after the candidate question is tokenized. It is determined whether the to-be-judged content contains the core word. If the to-be-judged content contains any core word, the preset window is moved so that the core word is at the center position of the preset window, an offset window is obtained, and the content in the offset window is taken as the candidate key question. If the to-be-judged content does not contain the core word or after the offset window is obtained, the preset window is controlled to move according to a specified step size, and the content in the moved preset window is taken as new to-be-judged content. The step of determining whether the to-be-judged content contains the core word and the subsequent steps are returned to be executed until the preset window moves to the end position of the question tokenization sequence.

[0073] After the candidate question is tokenized, the corresponding question tokenization sequence can be obtained. The starting position of the question tokenization sequence is the first word therein, and the end position is the last word therein. The length of the preset window can be set according to actual needs. For example, the length of the preset window is set to 5. The preset window is set at the starting position of the question tokenization sequence, that is, the first word in the question tokenization sequence is contained in the preset window.

[0074] Then, the content in the preset window can be used as the content to be judged to determine whether it contains a core word. If the content contains any core word, for each core word, the preset window is moved so that the core word is in the center of the preset window to obtain an offset window. For example, if the length of the preset window is 5, then after moving, the position of the core word in the offset window should be at position 3. The content in the offset window is used as a candidate key question. For example, if the candidate question is: "What was the output of the factory in location A in the past week?", there is a content to be judged as: "output of the factory in location A", where output is the core word. After offsetting, we can obtain "how much output the factory produces", which is used as a candidate key question.

[0075] Then, the preset window is controlled to move according to a specified step size, and the content in the preset window after the movement is used as the new content to be judged. The specified step size represents the distance the preset window moves each time. This distance can be measured by the number of word segments, and can be set according to actual needs. In this embodiment of the invention, it can be set to 1.

[0076] If the content to be judged does not contain the core words, the preset window can be directly controlled to move according to the specified step size to obtain new content to be judged.

[0077] After obtaining new content to be judged, the process can return to the step of determining whether the content to be judged contains the core word and subsequent steps, until the preset window moves to the end position of the question segmentation sequence. The end position of the question segmentation sequence is the last word in the question segmentation sequence. When the preset window contains the last word of the question segmentation sequence, it is considered that the preset window has moved to the end position of the question analysis sequence.

[0078] Based on key correlation data and the similarity between candidate key questions and key data, auxiliary data is identified. Auxiliary data is information used by users to assist in generating SQL statements, providing more background information to enhance the model's understanding. This auxiliary data may include candidate logical data and candidate document data.

[0079] Specifically, when determining auxiliary data, candidate key data can be determined from the key data based on the similarity between the candidate key questions and the key data; business logic data corresponding to the candidate key data in the key related data can be determined as designated logic data, and business document data corresponding to the candidate key data can be determined as designated document data; a first number of candidate logic data and a second number of candidate document data can be determined based on the length of the candidate questions; the first number of designated logic data are filtered to obtain candidate logic data, and the second number of designated document data are filtered as candidate document data.

[0080] See alsoFigure 3 The schematic diagram of generating auxiliary data is shown. The candidate key question and the key data are obtained by presetting a window and extracting core words. For each candidate key question, the similarity between the candidate key question and each key data can be calculated. For example, the candidate key question and the key data can be converted into vectors, and then the cosine similarity between the vectors can be calculated. For each key data, the corresponding similarity can be obtained. After filtering the key data with a similarity less than a preset similarity, the remaining key data is sorted in descending order of similarity, and the top specified number of key data is taken as the candidate key data. By calculating the similarity between the candidate key question and the key data, the unstructured data and the question can be efficiently associated.

[0081] The aforementioned key association data has been stored, that is, each key data has its corresponding business logic data and business document data. Based on the key association data, the business logic data corresponding to the candidate key data can be determined as the specified logic data, and the business document data corresponding to the candidate key data can be determined as the specified document data.

[0082] The specified logic data and the specified document data can be understood as the business logic data and the business document data related to the candidate question filtered based on the core word dimension. In order to balance the relationship between data redundancy and data quality, the data to be retained can be dynamically determined based on the candidate question. Thus, the first number of candidate logic data and the second number of candidate document data can be determined based on the length of the candidate question, and only the corresponding number of specified logic data and specified document data is retained, which can provide necessary background data without causing data redundancy.

[0083] Optionally, a first mapping relationship between a preset length range and a preset number can be preset. The preset length range in which the candidate question is located is determined as the target length range, and the preset number corresponding to the target length range is determined as the first number. In the same way, a second mapping relationship can be additionally set, and the second number can be determined based on the second mapping relationship. The first mapping relationship and the second mapping relationship can be set to be the same or directly shared.

[0084] In the embodiment of the application, only the first mapping relationship can be set, and the first number and the second number are the same. The first mapping relationship can be seen from Table 1.

[0085] Table 1

[0086] preset length range [0,500] (500,2000] (2000,+∞) preset number 1 2 3

[0087] It should be noted that the length of the candidate question can be measured by the number of characters. For example, if the number of characters of the candidate question is 100, the length is 100.

[0088] In screening the first number of specified logical data, the similarity of the key data corresponding to the specified logical data can be determined, for example, if the first number is 1, one of the specified logical data corresponding to the key data with the highest similarity is taken as the candidate logical data. In screening the second number of specified document data, the similarity of the key data corresponding to the specified document data can be determined, for example, if the second number is 1, one of the specified document data corresponding to the key data with the highest similarity is taken as the candidate document data.

[0089] In some embodiments, in order to further improve the accuracy of retrieval, in determining the candidate logical data and the candidate document data, a third number of intermediate logical data can be determined based on the similarity between the candidate question and the business logical data, the third number being related to the first number; a fourth number of intermediate document data can be determined based on the similarity between the candidate question and the business document data, the fourth number being related to the second number; the intermediate logical data and the first number of specified logical data are merged to obtain the candidate logical data; and the intermediate document data and the second number of specified document data are merged to obtain the candidate document data.

[0090] The candidate question is converted into a vector, and the business logical data is converted into a vector, and then the cosine similarity between the vectors is calculated as the similarity between the candidate question and the business logical data. The third number of intermediate logical data is determined by using the similarity. The third number is related to the first number, for example, the third number can be the first specified number minus the product of the second specified number and the first number. The first specified number and the second specified number can be set according to actual needs or actual experience. In the embodiment of the present application, the first specified number is 10, and the second specified number is 2.

[0091] In determining the third number of intermediate logical data, the business logical data can be sorted in descending order of similarity, and the top third number of business logical data is extracted as the intermediate logical data.

[0092] Similarly, the fourth number is the number of intermediate document data, the fourth number is related to the second number, and the fourth number can be the third specified number minus the product of the fourth specified number and the second number. The third specified number and the fourth specified number can be set according to actual needs or actual experience. In the embodiment of the present application, the third specified number is 10, and the fourth specified number is 2.

[0093] Then the intermediate logical data and the first number of specified logical data can be de-duplicated and merged, that is, the candidate logical data can be obtained, and the intermediate document data and the second number of specified document data can be de-duplicated and merged, that is, the candidate document data can be obtained.

[0094] The obtained candidate document data and candidate logic data can be used as auxiliary data, and the specified prompt word is constructed by using the auxiliary data and the table building information, and the specified prompt word is used to guide the large language model to generate a plurality of candidate SQL sentences corresponding to the candidate question.

[0095] The specified prompt word is a prompt word for generating an SQL sentence corresponding to a question, and the specified prompt word can be obtained based on a specified template, and the specified template can be set according to actual needs. In the embodiment of the application, the specified template is as follows:

[0096] “

task target

[0097] Please generate an accurate and business logic-compliant SQL query statement according to the business question and related data information provided by the user.

[0098]

input information processing

[0099] 1、<question description>

[0100] {P_q_gen_i}

[0101] 2、<database structure>

[0102] {P_schema}

[0103] (including table name, field name and type, primary key and foreign key relationship, index description)

[0104] 3、<reference case>

[0105] {P_qa_n}

[0106] (historical similar questions and corresponding SQL cases, pay attention to field mapping relationship and calculation logic)

[0107] 4、<business logic>

[0108] {P_logic_n}

[0109] (including related calculation formula, statistical rule, special filtering condition and other key calculation logic, such as sales = SUM (sales * unit price))

[0110] 5、<supplementary document>

[0111] {P_doc_n}

[0112] (business terminology table, process description document, data dictionary and other reference materials)

[0113]

generation requirement

[0114] 1. Strictly follow the ANSI SQL standard, adapt to the database type declared in <database structure>

[0115] 2. Prioritize using semantically clear field aliases (e.g., total_sales instead of sum(amount)).

[0116] 3. When joining multiple tables, the join conditions must be clearly defined to avoid Cartesian products.

[0117] 4. Numerical calculations must strictly match the business rules in the <Business Logic>.

[0118] 5. Pay attention to handling null values ​​and deduplication requirements.

[0119] 6. Include clear comments and explanations (using / ** / to indicate the source of business logic)

[0120] 7. Output only the SQL that can be executed correctly, without any additional information.

[0121] Among them, {P_q_gen_i} can be filled with candidate questions; {P_schema} can be filled with table creation information; {P_logic_n} can be filled with candidate logical data in the auxiliary data; and {P_doc_n} can be filled with candidate document data in the auxiliary data.

[0122] The {P_qa_n} field can be filled with candidate question-answer pairs similar to the candidate question; it can be left blank if none exist. If historical question-answer pairs exist in the aforementioned business data, candidate question-answer pairs can be obtained by filtering them. For example, a general retrieval method can be used to calculate the similarity between historical questions and candidate questions in historical question-answer pairs, and a certain number of question-answer pairs with high similarity can be extracted as candidate question-answer pairs. The specific number can be set according to actual needs; for example, extracting 3 candidate question-answer pairs.

[0123] The constructed specified prompt words are then input into the large language model to guide it in generating multiple candidate SQL statements corresponding to the candidate questions. The number of candidate SQL statements generated by the large language model can be set according to actual needs, and its answer setting can be set to creative to ensure the diversity of the generated candidate SQL statements.

[0124] Optionally, multiple window contents can be extracted from candidate questions according to a preset window; if the window contents contain any core word, the preset window is moved so that the core word is in the center of the preset window to obtain an offset window; the contents of the offset window are used as candidate key questions.

[0125] S130. Based on the execution data of each candidate SQL statement, the candidate SQL statements are marked with positive and negative examples and divided to obtain a first sample set and a second sample set.

[0126] In the foregoing manner, for each candidate question, a plurality of corresponding candidate SQL statements can be obtained. For each candidate SQL statement, the candidate SQL statement can be executed to label and divide the candidate SQL statement with execution data of the candidate SQL statement to generate a corresponding data set for subsequent training of a SQL statement generation model. The execution data refers to data obtained by executing the candidate SQL statement, for example, can include data obtained by executing the SQL statement, whether the execution is correct, the time taken for correct execution, and if the execution is erroneous, the corresponding error information can also be recorded. Based on the execution data, the candidate SQL statement is labeled and divided into positive and negative examples to obtain a first sample set and a second sample set required for training.

[0127] In some embodiments, the execution data can include an execution state, an execution duration, and an execution result, wherein the execution state can represent whether the candidate SQL statement is successfully executed, the execution duration represents the specific duration consumed for successfully executing the candidate SQL statement, and the execution result refers to the specific return or operation content. When labeling and dividing the positive and negative examples according to the execution data of the candidate SQL statement, for each candidate SQL statement, the candidate SQL statement can be executed to obtain the execution state, the execution duration, and the execution result of the candidate SQL statement. For each candidate question, among the candidate SQL statements with successful execution state, the candidate SQL statement with the highest repetition rate of execution result and the shortest execution duration is marked as a first positive example, and the rest are marked as second positive examples. The candidate SQL statement with an execution failure state is marked as a negative example. The first positive examples are filtered according to the number to obtain a question to be used. The candidate SQL statements corresponding to all the questions to be used are summarized and sorted according to the number of second positive examples to obtain a positive and negative example sequence. The first sample set and the second sample set are determined from the positive and negative example sequence according to a specified proportion.

[0128] Each candidate SQL statement is sent to a SQL terminal for execution, and the corresponding execution state, execution duration, and execution result can be obtained. For each candidate question, the execution data of the corresponding candidate SQL statement is used to label the positive and negative examples, for example, among the candidate SQL statements with successful execution state, the candidate SQL statement with the highest repetition rate of execution result and the shortest execution duration is marked as a first positive example.

[0129] For each execution result, the repetition ratio of the execution result and each other execution result can be calculated, and the sum of all repetition ratios is the repetition degree. If there are multiple candidate SQL statements with the highest repetition degree, the one with the shortest execution time is selected as the first positive example. If there is no candidate SQL statement that meets the corresponding condition, the first positive example is empty. For the candidate SQL statement with the execution state of execution success, it is marked as the second positive example. The candidate SQL statement with the execution state of execution failure is marked as the negative example. In this way, the first positive example, the second positive example, and the negative example can be marked in the candidate SQL statement corresponding to the candidate problem.

[0130] According to the number of first positive examples, the candidate problems can be filtered to remove low-quality data to ensure the data quality of the subsequent sample set. For example, the candidate problems with empty first positive examples can be removed, and the remaining candidate problems are used as the problems to be used. That is, in the subsequent data used as samples, the first positive example corresponding to the candidate problem must exist.

[0131] For all problems to be used, the candidate SQL statements corresponding to all problems to be used can be sorted according to the number of second positive examples. As an implementation, the problems to be used can be arranged in descending order of the number of second positive examples. If there are problems to be used with the same number of second positive examples, these problems to be used can be randomly arranged, and then the candidate SQL statements corresponding to each problem to be used are filled into the corresponding positions to obtain the positive-negative example sequence. One data in the positive-negative example sequence contains a problem to be used, a candidate SQL statement corresponding to the problem to be used, and a positive-negative example label corresponding to the candidate SQL statement.

[0132] The first sample set, the second sample set, and the verification set are data used to train the SQL statement generation model, which can be used in different training stages. For example, the SQL statement generation model needs to be fine-tuned before reinforcement learning in training. The first sample set can be used in the fine-tuning stage, and the second sample set can be used in the reinforcement learning stage.

[0133] The specified proportion is a proportion set in advance according to actual needs, which can be used to determine the first sample set, the second sample set, and the verification set from the positive-negative example sequence. In the embodiment of the present application, the specified proportion can be 20:75:5, that is, the first sample set is the first 20% of the data in the positive-negative example sequence, the second sample set is the middle 75% of the data in the positive-negative example sequence, and the verification set is the last 5% of the data in the positive-negative example sequence.

[0134] The candidate problems are automatically extracted from the business data, and the candidate SQL statements of the candidate problems are automatically generated. The positive and negative examples are divided by executing the SQL statements, which can construct high-quality training data at low cost and effectively reduce the construction cost of training data.

[0135] S140. Supervise fine-tuning of the base model with the first sample set to obtain an intermediate model.

[0136] The base model can be selected according to actual needs, and the base model has basic semantic understanding ability. The base model is supervised and fine-tuned by the first sample set, so that the base model can learn specific patterns, rules and requirements related to the SQL statement generation task, and obtain an intermediate model that can better complete the SQL statement generation task. Alternatively, all second positive examples in the first sample set can be randomly sampled to fine-tune the base model to obtain the intermediate model. The specific processing of fine-tuning can refer to existing technologies, and will not be limited here.

[0137] S150. Calculate a target reward based on the format, execution data and content quality of the predicted SQL statement, and adjust the model parameters of the intermediate model based on the target reward to obtain a SQL statement generation model.

[0138] The predicted SQL statement is the SQL statement obtained by the intermediate model after reasoning and predicting the candidate questions in the second sample set. It should be noted that the intermediate model will analyze and reason the questions in the second sample set according to the current strategy, i.e. the behavior determined by the model parameters of the intermediate model, to generate multiple different predicted SQL statements.

[0139] For each predicted SQL statement, a target reward can be calculated based on the format, execution data and content quality of the predicted SQL statement to determine the optimization direction of the intermediate model. The target reward can be used to construct a loss function for the entire reinforcement learning in the subsequent process, so that the SQL statement generation model can be obtained when the loss function converges. Designing a reward function from the three dimensions of format, execution data and content quality can realize multi-dimensional reinforcement learning collaborative optimization, which can improve the processing capability of complex SQL statements and ensure that the SQL statement generation model can accurately generate high-quality SQL statements in complex scenarios.

[0140] As an implementation manner, when the target reward is calculated and the model parameters of the intermediate model are adjusted based on the target reward to obtain the SQL statement generation model, the intermediate model can be used to generate a plurality of predicted SQL statements and predicted reasoning process data corresponding to the candidate question in the second sample set; the format reward is calculated based on the inclusion relationship between the predicted reasoning process data and the specified reasoning label, and the correlation between the specified answer label and the specified core word in the predicted SQL statement, the specified core word being a core word in the candidate question; the execution reward is determined according to the execution data of the predicted SQL statement, the execution data of the first positive example and the second positive example corresponding to the candidate question; the content quality reward of the predicted SQL statement is calculated according to the specified business data and the first positive example and the second positive example corresponding to the candidate question, the specified business data being business data related to the candidate question; the target reward of the predicted SQL statement is obtained by fusing the format reward, the execution reward and the content quality reward; and the model parameters of the intermediate model are updated based on the target rewards of all predicted SQL statements to obtain the SQL statement generation model.

[0141] The second sample set includes a plurality of candidate questions. For the candidate question in the second sample set, the intermediate model can be used to analyze and reason the candidate question to generate a predicted SQL statement corresponding to the candidate question. It should be noted that, due to the randomness of sampling and the exploratory nature of the model strategy, the intermediate model can generate a plurality of predicted SQL statements for the same candidate model when generating the predicted SQL statement. The intermediate model can also output analysis and reasoning data when generating the predicted SQL statement, i.e., predicted reasoning process data.

[0142] For the predicted SQL statement and the predicted reasoning process data corresponding thereto, the target reward of the predicted SQL statement can be calculated. The target reward can include a format reward, an execution reward and a content quality reward. The format reward can be used to measure whether the reasoning logic of the model meets the demand, the execution reward can be used to measure whether the predicted SQL statement output by the model meets the basic execution demand, and the content quality reward can be used to measure the quality of the predicted SQL statement. From the three dimensions, the target reward is calculated to ensure that the SQL statement generation model finally trained can accurately generate high-quality SQL statements. For example, refer to Figure 4 , which shows a schematic diagram for calculating the target reward.

[0143] Optionally, the format reward can be calculated based on the containment relationship between the predicted inference process data and the specified inference label, and the correlation between the specified answer label in the predicted SQL statement and the specified core word. That is, the format reward in the embodiment of the application can include two parts, one part is the inference process format reward, and the other part is the result format reward. The final format reward can be obtained by adding the inference process format reward and the result format reward.

[0144] For the inference process format reward, the specified inference label is a text that is set in advance to detect whether the thinking process of the intermediate model meets the requirements. The specified inference label can be set according to actual needs. It is judged whether the predicted inference process data contains the specified inference label. If the predicted inference process data contains the specified inference label, it indicates that the inference or thinking process of the intermediate model meets the requirements, and it can be encouraged. Otherwise, if the specified inference label is not contained, it indicates that the inference or thinking process of the intermediate model does not meet the requirements, and it can be punished. The specific performance of encouragement and punishment can also be set according to actual needs, for example, the encouragement can be to set the inference process format reward to a first value, and the punishment can be to set the inference process format reward to a second value, wherein the second value is less than the first value.

[0145] For the result format reward, a specified answer label can be set in advance, which is a text set in advance to detect whether the format of the output of the intermediate model meets the requirements. The specified answer label can contain core words in the business data, which can be set according to actual needs. As mentioned above, when generating sample data, the candidate key questions are extracted from the candidate questions according to the preset window and the core words, which can be directly used as the specified core words. First, it is determined whether the predicted SQL statement contains the specified answer label. If it does not contain, the result format reward can be directly set to a third value. If the predicted SQL statement contains the specified answer label, the core word can be extracted from the specified answer label, and the correlation score between the core word and the specified core word is calculated. The correlation score is multiplied by a specified score value to obtain the result format reward. The correlation score can be calculated by Jaccard coefficient. The inference process format reward and the result format reward are summed to obtain the final format reward.

[0146] For each predicted SQL statement, the predicted SQL statement can be sent to a SQL terminal and executed to obtain its corresponding execution data, which can include execution status, execution duration, and execution result, etc. If the execution status is execution failure, the corresponding execution reward can be directly set as a fourth value. If the execution status is execution success, the execution data of the first positive example and the second positive example corresponding to the candidate question can be obtained. The execution data of the positive example is used as a standard to determine whether the execution data of the predicted SQL statement is correct. If it is correct, the corresponding execution reward can be set as a fifth value, wherein the fourth value is less than the fifth value.

[0147] The content quality of the predicted SQL statement can be evaluated using the specified business data, the first positive example and the second positive example in the second sample set to obtain the content quality reward of the predicted SQL statement. The specified business data is business data related to the candidate question.

[0148] Optionally, when calculating the content quality reward of the predicted SQL statement, an evaluation template can be obtained, which includes evaluation dimensions, evaluation rules, and output requirements. The specified business data is determined from the business data according to the similarity between the candidate question and the business data. The evaluation prompt words are generated based on the first positive example, the second positive example, the specified business data, the predicted SQL statement, and the evaluation template corresponding to the candidate question. The evaluation prompt words are used to guide the large language model to evaluate the predicted SQL statement in the evaluation dimensions according to the evaluation rules. The evaluation result of the predicted SQL statement is output according to the output requirement to obtain the content quality reward of the predicted SQL statement.

[0149] The evaluation template is a prompt word template pre-set for content quality evaluation of the predicted SQL statement. The prompt word template can pre-set evaluation dimensions, evaluation rules, and output requirements. The evaluation dimensions refer to the content that needs to be focused on when evaluating the predicted SQL statement. The evaluation dimensions can include syntax correctness, logical accuracy, execution efficiency, readability, and problem matching degree, etc. The evaluation rules can be specific scoring rules, such as 0 points for complete error, partial correct points according to the proportion, etc. The specific scoring rules can be set according to actual needs, which are not limited here.

[0150] In the embodiments of the present application, the evaluation template can be:

[0151] "

task

[0152] You are a SQL quality evaluation expert. Please score the "user question" and the corresponding "generated SQL" according to the following dimensions (0-10 integer), and the scoring standard:

[0153]

Scoring dimensions

[0154] 1. Syntactic correctness (2 points): Does it conform to the SQL syntax specification?

[0155] 2. Logical accuracy (4 points): Does it accurately reflect the problem requirements?

[0156] 3. Execution efficiency (2 points): Whether reasonable optimization methods are used.

[0157] 4. Readability (1 point): Is the format / naming clear and standardized?

[0158] 5. Problem Matching Degree (1 point): Whether the requirements are fully covered.

[0159] [Reference Materials]

[0160] {P_logic_n}

[0161] {P_doc_n}

[0162] [Scoring Rules]

[0163] - Completely wrong gets 0 points, partially correct gets points deducted proportionally.

[0164] -Refer to the SQL example:

[0165] {best_practice}

[0166]

Input Format

[0167] User issue:

[0168] {P_q_gen_i}

[0169] Generate SQL:

[0170] {generated_sql}

[0171] Output Requirements

[0172] Please return in JSON format:

[0173]

[0174] Example output:

[0175] {"score":7,"comment":"The syntax is correct but lacks index optimization. The WHERE condition does not cover boundary cases. It is recommended to add NULL value handling."}

[0176] Optionally, when generating candidate SQL statements corresponding to candidate questions, auxiliary data containing candidate document data and candidate logical data was determined. This auxiliary data can be directly used as the specified business data.

[0177] Optionally, the similarity between the candidate question and the business logic data and the business document data can be directly calculated, and the parts of the business logic data and the business document data with higher similarity can be used as the specified business data.

[0178] The reference in the evaluation template can be filled in the corresponding specified business data, {best_practice} can be filled in the first positive example and the second positive example corresponding to the candidate question as a reference, {P_q_gen_i} can be filled in the candidate question, and {generated_sql} can be filled in the predicted SQL statement. After filling, the evaluation prompt word can be obtained. The evaluation prompt word is input into the large language model to guide the large language model to evaluate the predicted SQL statement in the evaluation dimension according to the evaluation rule, and output the evaluation result of the predicted SQL statement according to the output requirement, and the evaluation result is used as the content quality reward.

[0179] Finally, the sum format reward, the execution reward and the content quality reward can be obtained to obtain the target reward of the predicted SQL statement. The target reward can be used as part of the loss function. Thus, by calculating the loss function, the model parameters of the intermediate model can be adjusted and updated. When the loss function converges, it is considered that the training is completed, and at this time, the SQL statement generation model can be obtained.

[0180] After the SQL statement generation model is trained, it can be deployed for subsequent SQL statement generation scenarios. In generating the SQL statement, the business data in the target business scenario can be obtained, the business data including table building information, business logic data, business document data, core words and key associated data, the key data being window content extracted from the business logic data and the business document data based on the core words and a preset window; the key query is extracted from the to-be-processed query based on the core words and the preset window, the to-be-processed query being a user query in the target business scenario, and the key query being window content with the core words at the center of the preset window; the to-be-used document data and the to-be-used logic data are determined according to the similarity between the key query and the key data and the key associated data, the key associated data including mapping relationships between the key data and the business logic data and the business document data respectively; the target prompt word is constructed based on the to-be-used logic data, the to-be-used document data and the table building information; and the SQL statement generation model is guided to generate the target SQL statement corresponding to the to-be-processed query based on the target prompt word, the SQL statement generation model being trained by the SQL statement generation model training method provided in the foregoing embodiments.

[0181] Wherein, the business data can refer to the corresponding content in the foregoing embodiments, the key query can be extracted from the to-be-processed query by referring to the corresponding content of the candidate key question extraction, and the determination of the to-be-used document data and the to-be-used logic data can refer to the determination of the auxiliary data part. That is, when using the SQL statement generation model, the to-be-processed query and other data can be filled into the specified template to construct the corresponding target prompt word, and then the target prompt word is input into the SQL statement generation model, and the target SQL statement corresponding to the to-be-processed query can be obtained.

[0182] The SQL statement generation model training scheme provided by the embodiments of the present application can be applied in various scenarios requiring SQL statement generation. For example, in the industrial field, the yield data needs to be queried, and in the medical field, the patient's illness data needs to be queried. The training method provided by the embodiments of the present application can train the SQL statement generation model, and can accurately generate high-quality SQL statements for data query.

[0183] Through the method provided by the embodiments of the present application, multiple candidate SQL statements can be automatically generated based on the candidate question in the scene without historical cases, high-quality training data can be constructed at a low cost, and when generating the candidate SQL statement, the auxiliary data related to the candidate question can be accurately retrieved through the core word window sliding and filtering, the training data can be filtered and divided by using the execution data of the SQL statement, and in subsequent training, the reward is designed from three dimensions of format, execution and content quality, to realize multi-dimensional collaborative optimization, improve the processing capability of the SQL statement generation model in complex scenarios, and ensure that the SQL statement generation model finally trained can accurately generate high-quality SQL statements in complex scenarios.

[0184] In order to better implement the above method, the embodiments of the present application also provide a SQL statement generation model training device, which can be integrated in an electronic device. The electronic device can be a terminal, a server, etc. Wherein, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a notebook computer, a personal computer, etc. The server can be a single server or a server cluster composed of multiple servers.

[0185] For example, in this embodiment, the SQL statement generation model training device is integrated in a server.

[0186] For example, as shown in the figure, Figure 5 The SQL statement generation model training device 200 can include an acquisition module 210, a generation module 220, a sample division module 230, a fine-tuning module 240, and an adjustment module 250.

[0187] The obtaining module 210 is configured to obtain a plurality of candidate questions in a target business scenario, the candidate question being a question solved by a SQL query statement.

[0188] The generating module 220 is configured to, for each candidate question, generate a plurality of candidate SQL statements corresponding to the candidate question based on business data in the target business scenario and a core word.

[0189] The sample dividing module 230 is configured to perform positive and negative example labeling and dividing processing on each candidate SQL statement based on execution data of the candidate SQL statement, to obtain a first sample set and a second sample set.

[0190] The fine-tuning module 240 is configured to perform supervised fine-tuning processing on the base model with the first sample set, to obtain an intermediate model.

[0191] The adjusting module 250 is configured to calculate a target reward based on a format, execution data and content quality of a predicted SQL statement, and adjust model parameters of the intermediate model based on the target reward, to obtain a SQL statement generation model, the predicted SQL statement being obtained by the intermediate model performing reasoning on a candidate question in the second sample set.

[0192] In some embodiments, the obtaining module 210 is specifically configured to:

[0193] Obtain business data in a target business scenario, the business data including table creation information, business logic data and business document data.

[0194] Extract a core word related to the target business scenario from the business data.

[0195] For each target document data, fill the target document data and the table creation information into a conversion template to construct a conversion prompt word, the target document data being business document data containing the core word.

[0196] Use the conversion prompt word to guide a large language model to generate a candidate question corresponding to the target document data.

[0197] In some embodiments, the generating module 220 is specifically configured to:

[0198] Obtain a mapping relationship between key data and the business logic data and the business document data, to obtain key association data, the key data being window content extracted from the business logic data and the business document data based on a core word and a preset window.

[0199] For each candidate question, a candidate key question is extracted from the candidate question based on a core word and a preset window, the candidate key question being a window content with the core word at the center of the preset window;

[0200] According to the key association data, the similarity between the candidate key question and the key data, auxiliary data is determined, the auxiliary data including candidate logic data and candidate document data;

[0201] Based on the auxiliary data and the table building information, a specified prompt word is constructed, and the specified prompt word is used to guide a large language model to generate a plurality of candidate SQL statements corresponding to the candidate question.

[0202] In some embodiments, the generating module 220 is specifically configured to:

[0203] The preset window is controlled to be at the starting position of a question tokenization sequence, and the content in the preset window is taken as the to-be-judged content, the question tokenization sequence being a sequence obtained after the candidate question is tokenized;

[0204] It is determined whether the to-be-judged content contains a core word;

[0205] If the to-be-judged content contains any core word, the preset window is moved so that the core word is at the center position of the preset window, an offset window is obtained, and the content in the offset window is taken as a candidate key question;

[0206] If the to-be-judged content does not contain a core word or after the offset window is obtained, the preset window is controlled to move according to a specified step size, and the content in the moved preset window is taken as new to-be-judged content;

[0207] The step of determining whether the to-be-judged content contains a core word and subsequent steps are returned to be executed until the preset window moves to the ending position of the question tokenization sequence.

[0208] In some embodiments, the generating module 220 is specifically configured to:

[0209] According to the similarity between the candidate key question and the key data, candidate key data is determined from the key data;

[0210] The business logic data corresponding to the candidate key data in the key association data is determined as specified logic data, and the business document data corresponding to the candidate key data is determined as specified document data;

[0211] According to the length of the candidate question, a first number of candidate logic data and a second number of candidate document data are determined;

[0212] Screening the first number of specified logical data obtains candidate logical data, and screening the second number of specified document data obtains candidate document data.

[0213] In some embodiments, the generating module 220 is specifically configured to:

[0214] Based on the similarity between the candidate question and the business logic data, determine a third number of intermediate logical data, the third number being related to the first number;

[0215] Based on the similarity between the candidate question and the business document data, determine a fourth number of intermediate document data, the fourth number being related to the second number;

[0216] Merge the intermediate logical data and the first number of specified logical data to obtain candidate logical data;

[0217] Merge the intermediate document data and the second number of specified document data to obtain candidate document data.

[0218] In some embodiments, the execution data includes execution status, execution duration, and execution result, and the sample division module 230 is specifically configured to:

[0219] For each candidate SQL statement, execute the candidate SQL statement to obtain the execution status, execution duration, and execution result of the candidate SQL statement;

[0220] For each candidate question, among the candidate SQL statements with execution success, mark the one with the highest execution result repetition rate and the shortest execution duration as the first positive example, and mark the rest as the second positive example;

[0221] Mark the candidate SQL statement with execution failure as a negative example;

[0222] Filter all the candidate questions according to the number of first positive examples to obtain questions to be used;

[0223] According to the number of second positive examples, all the candidate SQL statements corresponding to the questions to be used are summarized and sorted to obtain a positive and negative example sequence;

[0224] According to a specified proportion, determine a first sample set and a second sample set from the positive and negative example sequence.

[0225] In some embodiments, the adjusting module 250 is specifically configured to:

[0226] For the candidate questions in the second sample set, use the intermediate model to generate a plurality of predicted SQL statements and predicted reasoning process data corresponding to the candidate questions;

[0227] calculate a format reward based on a containment relationship between the predicted inference process data and the specified inference label, and a correlation between the specified answer label in the predicted SQL statement and a specified core word, the specified core word being a core word in the candidate question;

[0228] determine an execution reward based on execution data of the predicted SQL statement, execution data of the first positive example and the second positive example corresponding to the candidate question;

[0229] calculate a content quality reward of the predicted SQL statement based on specified business data, the first positive example and the second positive example corresponding to the candidate question, the specified business data being business data related to the candidate question;

[0230] fuse the format reward, the execution reward and the content quality reward to obtain a target reward of the predicted SQL statement;

[0231] update model parameters of the intermediate model based on target rewards of all predicted SQL statements to obtain a SQL statement generation model.

[0232] In some embodiments, the adjusting module 250 is specifically configured to:

[0233] obtain an evaluation template, the evaluation template including an evaluation dimension, an evaluation rule and an output requirement;

[0234] determine specified business data from the business data based on a similarity between the candidate question and the business data;

[0235] generate an evaluation prompt word based on the first positive example, the second positive example, the specified business data, the predicted SQL statement and the evaluation template corresponding to the candidate question;

[0236] guide the large language model to evaluate the predicted SQL statement in the evaluation dimension according to the evaluation rule by using the evaluation prompt word;

[0237] output an evaluation result of the predicted SQL statement according to the output requirement to obtain a content quality reward of the predicted SQL statement.

[0238] In specific implementation, each of the above modules can be implemented as an independent entity, or can be combined as the same or several entities. The specific implementation of each of the above modules can be referred to the method embodiments described above, and will not be described here.

[0239] As shown above, the SQL statement generation model training device in this embodiment can acquire multiple candidate questions in the target business scenario. For each candidate question, it uses business data and core keywords from the target business scenario to generate multiple candidate SQL statements. Using the execution data of each candidate SQL statement, it marks and classifies positive and negative examples, automatically constructing a high-quality first sample set and a second sample set. It then fine-tunes the basic model using the first sample set to obtain an intermediate model. Finally, it calculates the target reward based on the format, execution data, and content quality of the predicted SQL statements from the intermediate model, and adjusts the model parameters of the intermediate model based on the target reward to obtain the final SQL statement generation model. Designing a reward function from multiple dimensions can improve the accuracy of the SQL statement generation model in complex scenarios, thus enabling the trained SQL statement generation model to accurately generate high-quality SQL statements in complex scenarios.

[0240] This invention also provides an electronic device, which can be a terminal, a server, or other similar devices. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0241] In some embodiments, the SQL statement generation model training device can also be integrated into multiple electronic devices. For example, the SQL statement generation model training device can be integrated into multiple servers, and the SQL statement generation model training method of the present invention can be implemented by multiple servers.

[0242] In this embodiment, a server will be used as an example for detailed description. For example, ... Figure 6 As shown, it illustrates a structural schematic diagram of the electronic device involved in an embodiment of the present invention, specifically:

[0243] The electronic device may include components such as a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will understand that... Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0244] The processor 310 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and lines, executing various functions of the electronic device and processing data by running or executing software programs and / or modules stored in the memory 320 and calling data stored in the memory 320. In some embodiments, the processor 310 can include one or more processing cores; in some embodiments, the processor 310 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 310.

[0245] The memory 320 can be used to store software programs and modules, and the processor 310 executes various functions and data processing by running the software programs and modules stored in the memory 320. The memory 320 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 320 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 320 can also include a memory controller to provide access for the processor 310 to the memory 320.

[0246] The electronic device also includes a power supply 330 for powering various components, and in some embodiments, the power supply 330 can be logically connected to the processor 310 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 330 can also include one or more direct current or alternating current power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power state indicator, and any other components.

[0247] The electronic device can also include an input module 340, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0248] The electronic device can also include a communication module 350, which in some embodiments can include a wireless module, and the electronic device can perform short-range wireless transmission through the wireless module of the communication module 350, thereby providing the user with wireless broadband Internet access. For example, the communication module 350 can be used to help the user send and receive emails, browse web pages, and access streaming media, etc.

[0249] Although not shown, the electronic device can further include a display unit and the like, which will not be described herein. Specifically, in the present embodiment, the processor 310 in the electronic device will load the executable file corresponding to the process of one or more application programs into the memory 320 according to the following instructions, and run the application program stored in the memory 320 by the processor 310, thereby realizing the steps in the method of each embodiment of the application.

[0250] The specific implementation of each operation above can refer to the previous embodiments, which will not be described herein.

[0251] As can be seen from the above, the electronic device provided by the embodiments of the application can obtain a plurality of candidate questions in a target business scenario; for the candidate questions, a plurality of candidate SQL statements corresponding to the candidate questions are generated using business data and core words in the target business scenario; the candidate SQL statements are labeled and divided into positive and negative examples using the execution data of each candidate SQL statement, and a high-quality first sample set and a second sample set can be automatically constructed; the first sample set is used to fine-tune the base model to obtain an intermediate model; the format, execution data and content quality of the predicted SQL statement predicted by the intermediate model are comprehensively calculated to obtain a target reward, and the model parameters of the intermediate model are adjusted based on the target reward to obtain a final SQL statement generation model. The reward function is designed from multiple dimensions, which can improve the generation accuracy of the SQL statement generation model in a complex scenario, and the SQL statement generation model obtained by training can accurately generate high-quality SQL statements in a complex scenario.

[0252] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling related hardware, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0253] To this end, the embodiments of the application provide a computer readable storage medium, which stores a plurality of instructions capable of being loaded by a processor to execute the steps in any one of the SQL statement generation model training methods provided by the embodiments of the application.

[0254] The storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0255] According to an aspect of the present application, a computer program product or computer program is provided, which comprises computer programs / instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer programs / instructions from the computer readable storage medium, and the processor executes the computer programs / instructions, so that the electronic device executes the method provided in various optional implementation manners of the training data construction aspect or the SQL statement generation model training aspect or the SQL statement generation model use aspect provided in the above embodiments.

[0256] Due to the instructions stored in the storage medium, the steps in any of the SQL statement generation model training methods provided in the embodiments of the present application can be executed, thus the beneficial effects that can be achieved by any of the SQL statement generation model training methods provided in the embodiments of the present application can be achieved, which are described in detail in the foregoing embodiments and will not be repeated here.

[0257] The above describes in detail a SQL statement generation model training method and device provided in the embodiments of the present application, and the principles and implementation manners of the present application are described by applying specific examples; the above embodiment descriptions are only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will have changes; in summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A method for training an SQL statement generation model, characterized in that, The method includes: Obtain multiple candidate questions from the target business scenario, wherein the candidate questions are those that can be solved through SQL query statements; For each candidate question, based on the business data and core keywords in the target business scenario, multiple candidate SQL statements are generated corresponding to the candidate question; Based on the execution data of each candidate SQL statement, the candidate SQL statements are marked with positive and negative examples and divided to obtain a first sample set and a second sample set; The base model is subjected to supervised fine-tuning using the first sample set to obtain an intermediate model. The target reward is calculated based on the format, execution data, and content quality of the predicted SQL statement, and the model parameters of the intermediate model are adjusted based on the target reward to obtain the SQL statement generation model. The predicted SQL statement is obtained by the intermediate model through reasoning on the candidate questions in the second sample set. The acquisition of multiple candidate questions in the target business scenario includes: Acquire business data for the target business scenario, including table creation information, business logic data, and business document data; extract core words related to the target business scenario from the business data; for each target document data, fill the target document data and the table creation information into a conversion template to construct conversion prompt words, wherein the target document data is business document data containing the core words; use the conversion prompt words to guide a large language model to generate candidate questions corresponding to the target document data.

2. The method of claim 1, wherein, For each candidate question, based on business data and core keywords in the target business scenario, multiple candidate SQL statements are generated corresponding to the candidate question, including: Obtain the mapping relationship between key data and the business logic data and the business document data respectively to obtain key related data. The key data is window content extracted from the business logic data and business document data based on core words and preset windows. For each candidate question, based on the core words and a preset window, candidate key questions are extracted from the candidate questions. The candidate key questions are the window content where the core words are located in the center of the preset window. Based on the key correlation data and the similarity between the candidate key questions and the key data, auxiliary data is determined, which includes candidate logical data and candidate document data. Based on the auxiliary data and the table creation information, a specified prompt word is constructed, and the specified prompt word is used to guide the large language model to generate multiple candidate SQL statements corresponding to the candidate question.

3. The method of claim 2, wherein, The extraction of candidate key questions from candidate questions based on core words and a preset window includes: The preset window is positioned at the beginning of the question segmentation sequence, and the content in the preset window is used as the content to be judged. The question segmentation sequence is the sequence obtained after segmenting the candidate questions. Determine whether the content to be judged contains core words; If the to-be-judged content contains any core word, the preset window is moved so that the core word is at the center of the preset window, an offset window is obtained, and the content in the offset window is taken as a candidate key question; If the to-be-judged content does not contain a core word or after the offset window is obtained, the preset window is moved according to a specified step size, and the content in the moved preset window is taken as new to-be-judged content; The step of determining whether the to-be-judged content contains a core word and the subsequent steps are returned to be executed until the preset window moves to the end position of the question segmentation sequence.

4. The method of claim 2, wherein, The auxiliary data is determined according to the similarity between the key association data, the candidate key question and the key data, including: The candidate key data is determined from the key data according to the similarity between the candidate key question and the key data; The business logic data corresponding to the candidate key data in the key association data is determined as specified logic data, and the business document data corresponding to the candidate key data is determined as specified document data; The first number of candidate logic data and the second number of candidate document data are determined according to the length of the candidate question; The first number of specified logic data is screened to obtain candidate logic data, and the second number of specified document data is screened as candidate document data.

5. The method of claim 4, wherein, The method further includes: A third number of intermediate logic data is determined based on the similarity between the candidate question and the business logic data, and the third number is related to the first number; A fourth number of intermediate document data is determined based on the similarity between the candidate question and the business document data, and the fourth number is related to the second number; The intermediate logic data and the first number of specified logic data are merged to obtain candidate logic data; The intermediate document data and the second number of specified document data are merged to obtain candidate document data.

6. The method of claim 1, wherein, The execution data includes execution status, execution duration and execution result, and the candidate SQL statements are marked and divided into first and second sample sets based on the execution data of each candidate SQL statement, including: For each candidate SQL statement, the candidate SQL statement is executed to obtain the execution status, execution duration and execution result of the candidate SQL statement; For each candidate question, in the candidate SQL statement with execution success, the candidate SQL statement with the highest execution result repetition rate and the shortest execution duration is marked as the first positive example, and the rest is marked as the second positive example; The candidate SQL statement with execution failure is marked as a negative example; All the candidate questions are filtered according to the number of first positive examples to obtain to-be-used questions; All the to-be-used questions are sorted according to the number of second positive examples to obtain a positive and negative example sequence; The first and second sample sets are determined from the positive and negative example sequence according to a specified proportion.

7. The method of claim 1, wherein, The format of the predicted SQL statement, execution data, and content quality are used to calculate a target reward, and the model parameters of the intermediate model are adjusted based on the target reward to obtain a SQL statement generation model, including: For a candidate question in the second sample set, the intermediate model is used to generate a plurality of predicted SQL statements corresponding to the candidate question and predicted reasoning process data; Based on the inclusion relationship between the predicted reasoning process data and the specified reasoning label, and the correlation between the specified answer label and the specified core word in the predicted SQL statement, a format reward is calculated, the specified core word being a core word in the candidate question; According to the execution data of the predicted SQL statement, the execution data of the first positive example and the second positive example corresponding to the candidate question, an execution reward is determined; According to the specified business data, the first positive example and the second positive example corresponding to the candidate question, the content quality reward of the predicted SQL statement is calculated, the specified business data being business data related to the candidate question; The format reward, execution reward and content quality reward are fused to obtain the target reward of the predicted SQL statement; Based on the target rewards of all predicted SQL statements, the model parameters of the intermediate model are updated to obtain a SQL statement generation model.

8. The method of claim 7, wherein, The content quality reward of the predicted SQL statement is calculated according to the specified business data, the first positive example and the second positive example corresponding to the candidate question, including: An evaluation template is obtained, the evaluation template including evaluation dimensions, evaluation rules and output requirements; According to the similarity between the candidate question and the business data, the specified business data is determined from the business data; The first positive example, the second positive example, the specified business data, the predicted SQL statement and the evaluation template corresponding to the candidate question are used to generate evaluation prompt words; The evaluation prompt words are used to guide the large language model to evaluate the predicted SQL statement according to the evaluation rules in the evaluation dimensions; According to the output requirements, the evaluation results of the predicted SQL statement are output to obtain the content quality reward of the predicted SQL statement.

9. A SQL statement generation model training apparatus for implementing the method according to any one of claims 1 to 8, characterized in that, The device includes: An acquisition module is configured to acquire a plurality of candidate questions in a target business scenario, the candidate question being a question solved by a SQL query statement; A generation module is configured to generate, for each candidate question, a plurality of candidate SQL statements corresponding to the candidate question based on business data and core words in the target business scenario; A sample division module is configured to perform positive and negative example labeling and division processing on the candidate SQL statements based on execution data of each candidate SQL statement to obtain a first sample set and a second sample set; A fine-tuning module is configured to perform supervised fine-tuning processing on a base model using the first sample set to obtain an intermediate model; An adjustment module is configured to calculate a target reward based on the format of a predicted SQL statement, execution data, and content quality, and adjust the model parameters of the intermediate model based on the target reward to obtain a SQL statement generation model, the predicted SQL statement being obtained by reasoning on a candidate question in the second sample set by the intermediate model; The obtaining module is specifically configured to: obtain business data in a target business scenario, wherein the business data comprises table building information, business logic data, and business document data; extract core words related to the target business scenario from the business data; for each target document data, fill the target document data and the table building information into a conversion template to construct a conversion prompt word, wherein the target document data is business document data containing core words; and use the conversion prompt word to guide a large language model to generate a candidate question corresponding to the target document data.

Images