SQL statement generation model training method and device

Abstract

The invention provides an SQL statement generation model training method and device, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a plurality of candidate questions in a target service scene; for each candidate question, generating a plurality of candidate SQL statements corresponding to the candidate question based on the business data and the core word in the target business scene; based on the execution data of each candidate SQL statement, performing positive and negative example marking and division processing on the candidate SQL statements to obtain a first sample set and a second sample set; performing supervision fine tuning processing on the basic model by using the first sample set to obtain an intermediate model; and calculating a target reward based on the predicted format of the SQL statement, the execution data and the content quality, and adjusting model parameters of the intermediate model based on the target reward to obtain an SQL statement generation model. According to the method, the reward function is designed from multiple dimensions so as to ensure that the SQL statement generation model can accurately generate high-quality SQL statements in a complex scene.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for training an SQL statement generation model. Background Art

[0002] Text2SQL is a technology that converts natural language queries into SQL statements, enabling non-programming users to interact with database systems using natural language, without having to understand complex SQL syntax. With this technology, users simply enter a simple query in Chinese, and the system automatically generates the corresponding SQL query statement and retrieves the corresponding results from the database.

[0003] With the rapid development of large language models, necessary data can be provided through prompt words, and then the large language model can be used to convert user queries into SQL statements. However, such models still cannot accurately generate high-quality SQL statements in scenarios where complex SQL statements need to be generated, such as scenarios where the generated SQL statements are long. Summary of the Invention

[0004] In response to the above problems, the purpose of the present invention is to provide a SQL statement generation model training method and device, which can construct high-quality training data and design reward functions from multiple dimensions to improve the generation accuracy of the SQL statement generation model in sub-complex scenarios.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In one aspect, the present invention provides a method for training a SQL statement generation model, comprising:

[0007] Obtain multiple candidate questions in the target business scenario, where the candidate questions are questions that can be solved using SQL query statements;

[0008] For each candidate question, generate multiple candidate SQL statements corresponding to the candidate question based on the business data and core words in the target business scenario;

[0009] Based on the execution data of each candidate SQL statement, the candidate SQL statement is marked as positive and negative and divided to obtain a first sample set and a second sample set;

[0010] Performing supervised fine-tuning on the basic model using the first sample set to obtain an intermediate model;

[0011] The target reward is calculated based on the format, execution data and content quality of the predicted SQL statement, and the model parameters of the intermediate model are adjusted based on the target reward to obtain a SQL statement generation model, where the predicted SQL statement is obtained by the intermediate model inferring the candidate questions in the second sample set.

[0012] On the other hand, the present invention also provides a SQL statement generation model training device, comprising:

[0013] An acquisition module is used to acquire multiple candidate questions in a target business scenario, where the candidate questions are questions that can be solved using SQL query statements;

[0014] A generation module is used to generate, for each candidate question, multiple candidate SQL statements corresponding to the candidate question based on the business data and core words in the target business scenario;

[0015] A sample partitioning module, configured to perform positive and negative example partitioning and partitioning processing on the candidate SQL statements based on the execution data of each candidate SQL statement, so as to obtain a first sample set and a second sample set;

[0016] A fine-tuning module, configured to perform supervised fine-tuning on the base model using the first sample set to obtain an intermediate model;

[0017] An adjustment module is used to calculate a target reward based on the format, execution data, and content quality of the predicted SQL statement, and adjust the model parameters of the intermediate model based on the target reward to obtain a SQL statement generation model, where the predicted SQL statement is obtained by the intermediate model inferring the candidate questions in the second sample set.

[0018] On the other hand, the present invention also provides an electronic device comprising a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps in any one of the SQL statement generation model training methods provided by the present invention.

[0019] On the other hand, the present invention also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in any SQL statement generation model training method provided by the present invention.

[0020] On the other hand, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in any one of the SQL statement generation model training methods provided by the present invention.

[0021] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0022] In an embodiment of the present invention, multiple candidate questions in a target business scenario can be obtained; for the candidate questions, multiple candidate SQL statements corresponding to the candidate questions are generated using the business data and core words in the target business scenario; the execution data of each candidate SQL statement is used to mark and divide the candidate SQL statements into positive and negative examples, and a high-quality first sample set and a second sample set can be automatically constructed; the first sample set is used to fine-tune the basic model to obtain an intermediate model; the target reward is then calculated based on the format, execution data, and content quality of the predicted SQL statements predicted by the intermediate model, and the model parameters of the intermediate model are adjusted based on the target reward to obtain the final SQL statement generation model. The present invention designs reward functions from multiple dimensions, which can improve the generation accuracy of the SQL statement generation model in complex scenarios, and the trained SQL statement generation model can accurately generate high-quality SQL statements in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is a schematic diagram of an application scenario of the SQL statement generation model training method provided by an embodiment of the present invention;

[0025] Figure 2 This is a flow chart of a SQL statement generation model training method provided by an embodiment of the present invention;

[0026] Figure 3 is a schematic diagram of generating auxiliary data provided by an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of providing a calculation target reward according to an embodiment of the present invention;

[0028] Figure 5 It is a structural diagram of the SQL statement generation model training device provided by an embodiment of the present invention;

[0029] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0031] The present invention proposes a SQL statement generation model training method, which trains a SQL statement generation model through a designed training method and supports accurate generation of SQL statements in complex scenarios.

[0032] It is understandable that in the specific implementation of the present invention, data related to user information, etc., requires user permission or consent, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0033] See Figure 1 , shows a schematic diagram of an application scenario for the SQL statement generation model training method. The application scenario may include a terminal 101 and a server 102, which can exchange data over a network. Terminal 101 can be a mobile phone, tablet computer, smart Bluetooth device, computer, large screen, robot, or other device; server 102 can be a single server or a server cluster consisting of multiple servers.

[0034] Multiple candidate questions in the target business scenario can be collected in advance or obtained after data processing by the server 102. Specifically, the terminal 101 can send the target business scenario or candidate questions to the server 102, and the server 102 can obtain the candidate questions in the target business scenario. For each candidate question, based on the business data and core words in the target business scenario, multiple candidate SQL statements corresponding to the candidate questions are generated; based on the execution data of each candidate SQL statement, the candidate SQL statements are marked and divided into positive and negative examples to obtain a first sample set and a second sample set; the basic model is supervised and fine-tuned with the first sample set to obtain an intermediate model; the target reward is calculated based on the format, execution data and content quality of the predicted SQL statement, and the model parameters of the intermediate model are adjusted based on the target reward to obtain a SQL statement generation model, wherein the predicted SQL statement is obtained by the intermediate model inferring the candidate questions in the second sample set.

[0035] After the SQL statement generation model is trained, the server 102 can retain the corresponding model parameters and send a notification message to the terminal 101 so that the user knows that the model has been trained. Subsequently, the user can send a user query to the server 102 through the terminal 101, and the server 102 can generate an SQL statement corresponding to the user query through the SQL statement generation model and feed it back to the terminal 101. In this embodiment, a method for training a SQL statement generation model is provided, such as Figure 2 As shown, the specific process of the SQL statement generation model training method can be as follows:

[0036] S110: Acquire multiple candidate questions in the target business scenario.

[0037] The target business scenario refers to a real business scenario. Different business scenarios have different business logic and generated business data. You can set the target business scenario based on your actual needs. For example, if the real business scenario is an industrial scenario, then the target business scenario would be an industrial scenario. Another example is if the real business scenario is an elderly care service scenario, then the target business scenario would be an elderly care service scenario.

[0038] Candidate questions are questions that can be solved using SQL query statements. That is, answers to candidate questions can be obtained by searching, analyzing, and processing the corresponding database using SQL query statements. In some implementations, candidate questions can be screened from real questions in the target business scenario. For example, real questions in the target business scenario within a historical time period are obtained; data for solving these real questions is obtained; and if the data for solving these real questions contains SQL statements, the real question is selected as a candidate question.

[0039] In other implementations, candidate questions can also be automatically generated after analyzing data in a database. For example, business data for a target business scenario can be obtained, including table creation information, business logic data, and business document data; core words related to the target business scenario can be extracted from the business data; for each target document data, the target document data and the table creation information can be entered into a conversion template to construct conversion prompt words, where the target document data is business document data containing the core words; and the conversion prompt words can be used to guide the large language model to generate candidate questions corresponding to the target document data.

[0040] Business data refers to the data generated in the target business scenario, typically text data, which may include business logic data and business document data. This business data is typically stored in a database and may also include database table creation information. Table creation information refers to a detailed description of the table structure and related attributes defined when creating a table in the database. Business logic data refers to data that requires integration with business scenarios and business processing background knowledge when performing statistical calculations and data processing. For example, total power consumption needs to be summarized based on production conditions, and personnel quantitative performance needs to be calculated comprehensively based on data from different businesses. Business document data refers to data other than business logic data and question-and-answer data.

[0041] The table creation information, business logic data, and business document data described above are all obtained through processing business data. Based on the original database table creation information, a commonly formatted table creation information can be constructed. An example of this table creation information is as follows: CREATE TABLE TABLE_NAME(COLUMN_NAME,COL_TYPE,EXAMPLES / ALL VALUES / ENUM VALUES / CHECK IN, / *VALUE MAPPING* / ......).

[0042] TABLE_NAME refers to the table name, COLUMN_NAME refers to the column name, and COL_TYPE refers to the corresponding value type of the column. EXAMPLES / ALL VALUES / ENUM VALUES / CHECK IN, / *VALUE MAPPING* / is optionally configured based on the sampling strategy. If the data sample values ​​are not of an enumeration type and are not highly repetitive, randomly sample n values, and the value of n can be configured based on actual needs. If the data sample values ​​are not of an enumeration type and are highly repetitive, use the ALL VALUES annotation and follow it with CHECK IN(v1,v2,...), where v1 and v2 are the sample values. If the data sample values ​​are of an enumeration type, use the ENUMVALUES annotation and add the value mapping / *VALUE MAPPING* / to the final table creation information to map the corresponding concise and specific business meaning. The constructed table information can be directly saved in common formats such as txt and json. When constructing the table information, it can be distinguished based on the characteristics of different sample values. Using pseudo-code style expressions such as CHECK IN and / ** / can further improve the understanding of large language models and their instruction following capabilities.

[0043] It should be noted that if the business data contains a historical case library, that is, historical cases from problems to SQL statements, this data can be extracted and used as QA data. If the business data does not contain a historical case library, no processing is required.

[0044] Extract core words related to the target business scenario from business data to build a core vocabulary. This can be done using mainstream technologies such as TF-IDF, TextRank, LDA, OCR, and LLM. All of this business data can be pre-processed and stored, allowing you to access the corresponding data when needed.

[0045] Business document data can be sliced. For each business document, we can determine whether it contains any core words in the core vocabulary. If so, we can use that business document as the target document. For each target document, we can use the target document data, table creation information, and conversion templates to construct conversion prompt words for subsequent use.

[0046] The conversion template is a pre-set prompt word template used to generate candidate questions. The conversion template contains target slots and table creation information slots. By filling the target document data and table creation information into the corresponding slots, the conversion prompt word can be obtained. The conversion template can specifically include task objectives and requirements, which can be set according to actual needs. In an embodiment of the present invention, the conversion template can be:

[0047] "#Task

[0048] As an SQL learning aid, you need to design problems that can be solved through SQL queries based on the database table structure information and business data provided by the user.

[0049] #Require

[0050] 1. Table structure driven principle: All questions must be based on the fields, data types and constraints contained in the given CREATE TABLE statement, and executable SQL exercises must be designed

[0051] 2. SQL Solvability Verification: Ensure that each problem can be solved by writing standard SQL statements such as SELECT. Focus on the following abilities:

[0052] 2.1 Data query and conditional screening

[0053] 2.2. Table Joins and Aggregate Statistics

[0054] 3. Problem constraints:

[0055] 3.1. Operations involving database management, addition / deletion / modification of databases, etc. are prohibited;

[0056] 3.2. Avoid open-ended questions that require external knowledge or subjective judgment;

[0057] 3.3. Exclude advanced topics such as graphical interface operations or performance optimization;

[0058] 4. Answer directly without explanation.

[0059] #Database table structure information

[0060] {P_schema}

[0061] #Business Information

[0062] {P_doc_filtered_i}”

[0063] The {P_schema} field can be filled with table creation information, and the {P_doc_filtered_i} field can be filled with target document data to generate conversion prompts. These conversion prompts are then fed into the large language model, which analyzes and infers the results to generate candidate questions. The large language model is configured with a more creative setting, for example, temperature = 1.0.

[0064] As an implementation method, candidate questions can be collected from real questions, or corresponding candidate questions can be generated using a large language model. Of course, both methods can be used simultaneously to collect candidate questions, and the candidate questions collected from both methods can be deduplicated before use.

[0065] S120 . For each candidate question, generate multiple candidate SQL statements corresponding to the candidate question based on the business data and core words in the target business scenario.

[0066] Based on the previous description, business data can include table creation information, business logic data, and business document data. Core terms are extracted from business data. Using these core terms and business data, multiple candidate SQL statements corresponding to candidate questions can be generated. Candidate SQL statements refer to the SQL statements corresponding to the candidate questions.

[0067] As an implementation method, when generating candidate SQL statements, the mapping relationship between key data and the business logic data and the business document data can be obtained to obtain key association data, where the key data is the window content extracted from the business logic data and the business document data based on the core word and the preset window; for each candidate question, based on the core word and the preset window, a candidate key question is extracted from the candidate question, where the candidate key question is the window content with the core word at the center of the preset window; based on the key association data, the similarity between the candidate key question and the key data, auxiliary data is determined, where the auxiliary data includes candidate logic data and candidate document data; based on the auxiliary data and the table creation information, a specified prompt word is constructed, and the specified prompt word is used to guide the large language model to generate multiple candidate SQL statements corresponding to the candidate question.

[0068] Key data is data extracted from business logic data and business document data. The key data can be extracted and stored in advance, and can be directly obtained at this time. Among them, the key data is specifically the window content extracted from it using core words and preset windows. For the convenience of description, business logic data and business document data are collectively referred to as data to be processed, and these data are all sliced ​​data. The preset window is a sliding window in the data to be processed, and the preset window slides in the data to be processed to extract key data. Among them, the length of the preset window can be set according to actual needs. In the embodiment of the present invention, it can be set to an odd number greater than 1.

[0069] For each data to be processed, word segmentation processing can be performed on it to obtain a sequence of word segments to be processed; a preset window can be controlled at the starting position of the sequence of word segments to be processed, and the content in the preset window can be used as the content to be judged; it is determined whether the data to be judged contains a core word; if the data to be judged contains any core word, the preset window can be moved so that the core word is at the center position of the preset window to obtain an offset window, and the content in the offset window can be used as key data; if the content to be judged does not contain a core word, the preset window can be controlled to move according to a specified step size, and the content in the preset window after the move can be used as the new content to be judged; the step of determining whether the content to be judged contains a core word and subsequent steps can be returned to execute until the preset window moves to the end position of the sequence of word segments to be processed.

[0070] Thus, for each piece of data to be processed, the corresponding key data can be extracted. A correlation exists between the key data and the data to be processed, and thus, a correlation between each key data and the data to be processed can be established as key correlation data. A single key data may correspond to multiple pieces of data to be processed. To avoid duplication of key data, the key data can be deduplicated to obtain the final key relationship data and store it in the database for subsequent use. By using pre-set windows and core words, the original data can be greatly compressed, making it easier to use in subsequent retrieval.

[0071] It should be noted that business data may include key data, core words, table creation information, business logic data and business document data. Business data can be analyzed and processed in the target business scenario in advance and stored in the corresponding database. When needed, it can be read directly from the database.

[0072] For each candidate question, a candidate key question can also be extracted from it in the manner of extracting key data as described above. The candidate key question is the window content in which the core word is at the center of the preset window. Specifically, when extracting a candidate key question from a candidate question, the preset window can be controlled to be at the starting position of the question word segmentation sequence, and the content in the preset window is used as the content to be judged. The question word segmentation sequence is the sequence obtained after word segmentation processing of the candidate question; determine whether the content to be judged contains the core word; if the content to be judged contains any core word, move the preset window so that the core word is at the center of the preset window, obtain an offset window, and use the content in the offset window as the candidate key question; if the content to be judged does not contain the core word or after obtaining the offset window, control the preset window to move according to the specified step size, and use the content in the preset window after the move as the new content to be judged; return to execute the step of determining whether the content to be judged contains the core word and the subsequent steps until the preset window moves to the end position of the question word segmentation sequence.

[0073] After word segmentation of the candidate question, a corresponding question word segmentation sequence is obtained. The starting position of the question word segmentation sequence is the first word in the sequence, and the ending position is the last word in the sequence. The length of the preset window can be set according to actual needs. For example, the length of the preset window is set to 5. The preset window is set at the starting position of the question word segmentation sequence, that is, the preset window contains the first word in the question word segmentation sequence.

[0074] Then the content in the preset window can be used as the content to be judged to determine whether the content to be judged contains the core word; if the content to be judged contains any core word, for each core word, move the preset window so that the core word is in the center of the preset window to obtain the offset window. For example, if the length of the preset window is 5, then the position of the core word in the offset window should be at position 3 after the move, and the content in the offset window is used as a candidate key question. For example, the candidate question is: What was the output of the factory in place A in the past week? There is a content to be judged: The output of the factory in place A, where output is the core word. After offsetting, we can get the output of the factory as a candidate key question.

[0075] The preset window is then controlled to move according to a specified step size, and the content in the moved preset window is used as the new content to be judged. The specified step size represents the distance the preset window moves each time, which can be measured by the number of segmented words. The specific setting can be based on actual needs. In this embodiment of the present invention, it can be set to 1.

[0076] If the content to be judged does not contain the core word, the preset window can be directly controlled to move according to the specified step size to obtain new content to be judged.

[0077] After obtaining the new content to be judged, the process returns to the step of determining whether the content to be judged contains the core word and subsequent steps until the preset window moves to the end position of the question word sequence. The end position of the question word sequence is the last word in the question word sequence. When the preset window contains the last word in the question word sequence, it is considered that the preset window has moved to the end position of the question analysis sequence.

[0078] Based on the similarity between key association data, candidate key questions, and key data, auxiliary data is determined. Auxiliary data is information that assists users in generating SQL statements. It can provide more context for SQL statement generation and enhance the model's understanding. Auxiliary data can include candidate logical data and candidate document data.

[0079] Specifically, when determining the auxiliary data, the candidate key data can be determined from the key data based on the similarity between the candidate key question and the key data; the business logic data corresponding to the candidate key data in the key-related data is determined as the designated logic data, and the business document data corresponding to the candidate key data is determined as the designated document data; based on the length of the candidate question, the first number of candidate logic data and the second number of candidate document data are determined; the first number of designated logic data are screened to obtain the candidate logic data, and the second number of designated document data are screened as the candidate document data.

[0080] See Figure 3 , which shows a schematic diagram of generating auxiliary data. The candidate key questions and key data are obtained through preset windows and core word extraction. For each candidate key question, the similarity between the candidate key question and each key data can be calculated. For example, the candidate key question and key data can be converted into vectors, and then the cosine similarity between the vectors can be calculated. For each key data, its corresponding similarity can be obtained. After filtering out the key data with a similarity less than the preset similarity, the remaining key data are sorted in descending order according to the similarity, and the first specified number of key data with the highest ranking are taken as candidate key data. By calculating the similarity between the candidate key questions and the key data, unstructured data and questions can be efficiently associated.

[0081] The aforementioned key associated data has been stored, that is, each key data has its corresponding business logic data and business document data. Based on the key associated data, the business logic data corresponding to the candidate key data can be determined as the designated logic data, and the business document data corresponding to the candidate key data can be determined as the designated document data.

[0082] Designated logical data and designated document data can be understood as business logic data and business document data related to candidate questions, filtered based on the core word dimension. To balance data redundancy and data quality, the data to be retained can be dynamically determined based on the candidate questions. This allows for a first quantity of candidate logical data and a second quantity of candidate document data to be determined based on the length of the candidate question, and only the corresponding amount of designated logical data and designated document data is retained. This ensures that necessary context is provided without causing data redundancy.

[0083] Optionally, a first mapping relationship between a preset length range and a preset number can be pre-set, and the preset length range of the candidate question is determined as the target length range, and the preset number corresponding to the target length range is used as the first number. In the same manner, a second mapping relationship can be additionally set, and the second number is determined based on the second mapping relationship. The first mapping relationship and the second mapping relationship can be set to be the same or directly shared.

[0084] In the embodiment of the present invention, only the first mapping relationship may be set, and the first quantity and the second quantity are the same. The first mapping relationship may be specifically shown in Table 1.

[0085] Table 1

[0086] Preset length range [0,500] (500,2000] (2000,+∞) Preset quantity 1 2 3

[0087] It should be noted that the length of the candidate question can be measured by the number of characters. For example, if the number of characters in the candidate question is 100, then its length is 100.

[0088] When screening the first number of specified logical data, the candidate data may be determined based on the similarity of the key data corresponding to the specified logical data. If the first number is 1, one of the specified logical data corresponding to the key data with the highest similarity is selected as the candidate logical data. When screening the second number of specified document data, the candidate data may be determined based on the similarity of the key data corresponding to the specified document data. If the second number is 1, one of the specified document data corresponding to the key data with the highest similarity is selected as the candidate document data.

[0089] In some embodiments, in order to further improve the accuracy of retrieval, when determining candidate logical data and candidate document data, it is also possible to determine a third number of intermediate logical data based on the similarity between the candidate question and the business logic data, and the third number is related to the first number; determine a fourth number of intermediate document data based on the similarity between the candidate question and the business document data, and the fourth number is related to the second number; merge the intermediate logical data and the first number of specified logical data to obtain candidate logical data; merge the intermediate document data and the second number of specified document data to obtain candidate document data.

[0090] The candidate question is converted into a vector, and the business logic data is converted into a vector. The cosine similarity between the vectors is then calculated as the similarity between the candidate question and the business logic data. This similarity is then used to determine a third number of intermediate logic data. The third number is related to the first number. For example, the third number can be the product of the first number and the second number minus the first number. The first and second numbers can be set based on actual needs or experience. In this embodiment of the present invention, the first number is 10 and the second number is 2.

[0091] When determining the third number of intermediate logic data, the business logic data may be sorted in descending order of similarity, and the first third number of business logic data ranked high are extracted as the intermediate logic data.

[0092] Similarly, the fourth number is the number of intermediate document data. The fourth number is related to the second number. The fourth number can be the third specified number minus the product of the fourth specified number and the second number. The third specified number and the fourth specified number can be set according to actual needs or actual experience. In an embodiment of the present invention, the third specified number is 10 and the fourth specified number is 2.

[0093] Then, the intermediate logical data and the first number of specified logical data may be deduplicated and merged to obtain candidate logical data, and the intermediate document data and the second number of specified document data may be deduplicated and merged to obtain candidate document data.

[0094] The obtained candidate document data and candidate logical data can be used as auxiliary data. The auxiliary data and table construction information are used to construct specified prompt words, and the specified prompt words are used to guide the large language model to generate multiple candidate SQL statements corresponding to the candidate questions.

[0095] The designated prompt words are prompt words for generating SQL statements corresponding to the question. The designated prompt words can be obtained based on a pre-set designated template, which can be set according to actual needs. In an embodiment of the present invention, the designated template is as follows:

[0096] "[Mission Objective]

[0097] Please generate accurate SQL query statements that conform to business logic based on the business questions and related data information provided by the user.

[0098] Input information processing

[0099] 1. <Problem Description>

[0100] {P_q_gen_i}

[0101] 2. Database Structure

[0102] {P_schema}

[0103] (including table name, field name and type, primary and foreign key relationships, and index description)

[0104] 3. Reference Case

[0105] {P_qa_n}

[0106] (Historical similar problems and corresponding SQL cases, pay attention to field mapping relationships and calculation logic)

[0107] 4. <Business Logic>

[0108] {P_logic_n}

[0109] (Includes relevant calculation formulas, statistical rules, special filtering conditions and other key calculation logic, such as: Sales = SUM (sales volume * unit price))

[0110] 5. <Supplementary Documentation>

[0111] {P_doc_n}

[0112] (Reference materials such as business glossary, process description documents, data dictionary, etc.)

[0113]

Generation requirements

[0114] 1. Strictly follow the ANSI SQL standard and adapt to the database type declared in <Database Structure>

[0115] 2. Prefer semantically clear field aliases (such as total_sales instead of sum(amount))

[0116] 3. When joining multiple tables, the join conditions must be clearly defined to avoid Cartesian products

[0117] 4. Numerical calculations must strictly match the business rules in <Business Logic>

[0118] 5. Pay attention to null value processing (NULL value) and deduplication requirements

[0119] 6. Include clear comments (use / ** / to mark the source of business logic)

[0120] 7. Directly output SQL that can be executed correctly without any additional information.

[0121] Among them, {P_q_gen_i} can be filled with candidate questions; {P_schema} can be filled with table construction information; {P_logic_n} can be filled with candidate logical data in the auxiliary data; {P_doc_n} can be filled with candidate document data in the auxiliary data.

[0122] {P_qa_n} can be filled in with candidate question-answer pairs similar to the candidate question; if there are none, this field is optional. If historical question-answer pairs exist in the aforementioned business data, they can be filtered to obtain candidate question-answer pairs. For example, a general search method can be used to calculate the similarity between the historical questions and the candidate questions in the historical question-answer pairs. Based on this similarity, a certain number of question-answer pairs with high similarity can be extracted as candidate question-answer pairs. The specific number can be set based on actual needs; for example, three candidate question-answer pairs can be extracted.

[0123] The constructed designated prompt words are then input into the large language model to guide it in generating multiple candidate SQL statements corresponding to the candidate questions. The number of candidate SQL statements generated by the large language model can be set according to actual needs, and its answer setting can be set to creative to ensure the diversity of the generated candidate SQL statements.

[0124] Optionally, multiple window contents can be extracted from candidate questions according to a preset window; if the window content contains any core word, the preset window is moved so that the core word is in the center of the preset window to obtain an offset window; and the content in the offset window is used as a candidate key question.

[0125] S130 : Based on the execution data of each candidate SQL statement, perform positive and negative example labeling and classification processing on the candidate SQL statement to obtain a first sample set and a second sample set.

[0126] In the aforementioned manner, for each candidate question, a corresponding plurality of candidate SQL statements can be obtained. For each candidate SQL statement, the candidate SQL statement can be executed so that the execution data of the candidate SQL statement can be used to mark the positive and negative examples and divide them into different categories to generate a corresponding data set for use in the subsequent training of the SQL statement generation model. The execution data refers to the data obtained by executing the candidate SQL statement. For example, it may include the data obtained by executing the SQL statement, whether it is executed correctly, the time taken for correct execution, and if an error is reported during execution, the corresponding error message can also be recorded. Based on these execution data, the candidate SQL statement is marked, the positive and negative examples are marked, and the data set is divided to obtain the first sample set and the second sample set required for training.

[0127] In some embodiments, the execution data may include an execution status, an execution time, and an execution result, wherein the execution status may indicate whether the candidate SQL statement is successfully executed, the execution time indicates the specific time consumed by the successful execution of the candidate SQL statement, and the execution result refers to the specific return or operation content. When performing positive and negative example marking and division processing based on the execution data of the candidate SQL statement, it can be that for each candidate SQL statement, the candidate SQL statement is executed to obtain the execution status, execution time, and execution result of the candidate SQL statement; for each candidate question, among the candidate SQL statements with a successful execution status, the one with the highest execution result repetition rate and the shortest execution time is marked as the first positive example, and the rest are marked as the second positive examples; the candidate SQL statement with a failed execution status is marked as a negative example; all the candidate questions are filtered according to the number of first positive examples to obtain the questions to be used; according to the number of second positive examples, the candidate SQL statements corresponding to all the questions to be used are aggregated and sorted to obtain a positive and negative example sequence; and the first sample set and the second sample set are determined from the positive and negative example sequence according to a specified ratio.

[0128] Each candidate SQL statement is executed in the SQL terminal to obtain its corresponding execution status, execution duration, and results. For each candidate question, the execution data of the corresponding candidate SQL statement is used to mark positive and negative examples. For example, among the candidate SQL statements with a successful execution status, the one with the highest repeatability and shortest execution time can be marked as the first positive example.

[0129] For each execution result, the repetition ratio of the execution result to each other execution result can be calculated, and the sum of all repetition ratios is taken as the repetition degree. If there are multiple candidate SQL statements with the highest repetition degree, the one with the shortest execution time is selected as the first positive example. If the corresponding conditions are not met, the first positive example is empty. As for other candidate SQL statements with a successful execution status, they are marked as second positive examples. Candidate SQL statements with a failed execution status are marked as negative examples. In this way, the first positive example, the second positive example, and the negative example can be marked in the candidate SQL statements corresponding to a candidate question.

[0130] Based on the number of first positive examples, candidate questions can be filtered to remove low-quality data to ensure the quality of subsequent sample sets. For example, candidate questions with no first positive examples can be removed, and the remaining candidate questions can be used as questions. In other words, the first positive example corresponding to the candidate question must be present in the data used as samples.

[0131] For all pending questions, the candidate SQL statements corresponding to all pending questions can be sorted according to the number of second positive examples. As an implementation method, the pending questions can be arranged in descending order according to the number of second positive examples. If there are pending questions with the same number of second positive examples, these pending questions can be randomly arranged, and then the candidate SQL statements corresponding to each pending question can be filled in the corresponding position to obtain a positive and negative example sequence. One data in the positive and negative example sequence includes the pending question, the candidate SQL statement corresponding to the pending question, and the positive and negative example labels corresponding to the candidate SQL statement.

[0132] The first sample set, the second sample set, and the validation set are data used to train the SQL statement generation model, and can be used in different training stages. For example, the SQL statement generation model needs to be fine-tuned before reinforcement learning during training. The first sample set can be used in the fine-tuning stage, and the second sample set can be used in the reinforcement learning stage.

[0133] The designated ratio is a ratio pre-set based on actual needs and can be used to determine the first sample set, the second sample set, and the validation set from the positive and negative example sequences. In an embodiment of the present invention, the designated ratio can be 20:75:5, i.e., the first sample set is the first 20% of the data in the positive and negative example sequences, the second sample set is the middle 75% of the data in the positive and negative example sequences, and the validation set is the last 5% of the data in the positive and negative example sequences.

[0134] Automatically extract candidate questions from business data and automatically generate candidate SQL statements for the candidate questions. By executing the SQL statements to achieve the division of positive and negative examples, high-quality training data can be constructed at a low cost, effectively reducing the cost of constructing training data.

[0135] S140: Perform supervised fine-tuning on the basic model using the first sample set to obtain an intermediate model.

[0136] The base model can select an appropriate large language model based on actual needs. This base model should possess basic semantic understanding capabilities. The first sample set is used to perform supervised fine-tuning on the base model, enabling it to learn the specific patterns, rules, and requirements relevant to the SQL statement generation task, resulting in an intermediate model that can better complete the SQL statement generation task. Alternatively, all second positive examples in the first sample set can be randomly sampled to fine-tune the base model to obtain the intermediate model. The specific fine-tuning process can refer to existing technologies and is not specifically limited here.

[0137] S150 , calculating a target reward based on the format, execution data, and content quality of the predicted SQL statement, and adjusting the model parameters of the intermediate model based on the target reward to obtain a SQL statement generation model.

[0138] The predicted SQL statements are the SQL statements generated by the intermediate model after inferring and predicting the candidate questions in the second sample set. It should be noted that the intermediate model analyzes and infers the questions in the second sample set based on the current strategy, that is, the behavior determined by the intermediate model's model parameters, to generate multiple different predicted SQL statements.

[0139] For each predicted SQL statement, a target reward is calculated based on the format, execution data, and content quality of the predicted SQL statement to determine the optimization direction of the intermediate model. The target reward is then used to construct the loss function for the entire reinforcement learning process. When the loss function converges, the SQL statement generation model is obtained. Designing a reward function based on the three dimensions of format, execution data, and content quality enables multi-dimensional reinforcement learning collaborative optimization, improving its ability to handle complex SQL statements and ensuring that the SQL statement generation model can accurately generate high-quality SQL statements in complex scenarios.

[0140] As an implementation method, the target reward is calculated, and the model parameters of the intermediate model are adjusted based on the target reward. When the SQL statement generation model is obtained, it can be that for the candidate questions in the second sample set, the intermediate model is used to generate multiple predicted SQL statements and predicted reasoning process data corresponding to the candidate questions; based on the inclusion relationship between the predicted reasoning process data and the specified reasoning label, and the correlation between the specified answer label and the specified core word in the predicted SQL statement, the format reward is calculated, and the specified core word is the core word in the candidate question; according to the execution data of the predicted SQL statement, the first positive example corresponding to the candidate question, and the execution data of the second positive example, the execution reward is determined; according to the specified business data, the first positive example corresponding to the candidate question, and the second positive example, the content quality reward of the predicted SQL statement is calculated, and the specified business data is the business data related to the candidate question; the format reward, execution reward and content quality reward are integrated to obtain the target reward of the predicted SQL statement; based on the target rewards of all predicted SQL statements, the model parameters of the intermediate model are updated to obtain the SQL statement generation model.

[0141] The second sample set contains multiple candidate questions. The intermediate model can be used to analyze and reason about the candidate questions in the second sample set to generate corresponding predicted SQL statements. It should be noted that due to the random sampling and the exploratory nature of the model strategy, the intermediate model can generate multiple predicted SQL statements for the same candidate model when generating the predicted SQL statement. When generating the predicted SQL statement, the intermediate model can also output the analytical reasoning data used to generate the predicted SQL statement, i.e., the predicted reasoning process data.

[0142] The target reward for the predicted SQL statement and its corresponding prediction reasoning process data can be calculated. The target reward can include format reward, execution reward, and content quality reward. Among them, format reward can be used to measure whether the reasoning logic of the model meets the requirements, execution reward can be used to measure whether the predicted SQL statement output by the model meets the basic execution requirements, and content quality reward can be used to measure the quality of the predicted SQL statement. Calculating the target reward from these three dimensions can ensure that the SQL statement generation model obtained by training can accurately generate high-quality SQL statements. For example, see Figure 4 , shows a schematic diagram of calculating the target reward.

[0143] Optionally, a format reward can be calculated based on the predicted inclusion relationship between the inference process data and a specified inference tag, as well as the predicted correlation between a specified answer tag and a specified core word in the SQL statement. In other words, the format reward in this embodiment of the present invention can consist of two parts: a format reward for the inference process and a format reward for the result. The final format reward is obtained by adding the format reward for the inference process and the result.

[0144] For the reasoning process format reward, the specified reasoning label is a pre-set text that detects whether the thinking process of the intermediate model meets the requirements. The specific setting can be made according to actual needs. Determine whether the specified reasoning label is included in the predicted reasoning process data; if the specified reasoning label is included in the predicted reasoning process data, it indicates that the reasoning or thinking process of the intermediate model meets the requirements, and it can be encouraged; on the contrary, if the specified reasoning label is not included, it indicates that the reasoning or thinking process of the intermediate model does not meet the requirements, and it can be punished. The specific manifestations of encouragement and punishment can also be set according to actual needs. For example, encouragement can be setting the reasoning process format reward to a first value, and punishment can be setting the reasoning process format reward to a second value, wherein the second value is smaller than the first value.

[0145] For the result format reward, a designated answer tag can be pre-set. The designated answer tag is a text that is pre-set to detect whether the format of the output of the intermediate model meets the requirements. The designated answer tag can contain the core words in the business data, and can be set according to actual needs. When generating sample data, candidate key questions have been extracted from the candidate questions according to the preset window and core words. The candidate key questions extracted above can be directly used as designated core words. First, it can be determined whether the predicted SQL statement contains the designated answer tag; if not, the result format reward can be directly set to the third value. If the predicted SQL statement contains the designated answer tag, the core word can be extracted from the designated answer tag, and the correlation score between the core word and the designated core word can be calculated, and the correlation score can be multiplied by the specified score as the result format reward. Among them, the correlation score can be calculated by the Jaccard coefficient. The final format reward can be calculated by summing the reasoning process format reward and the result format reward.

[0146] For each predicted SQL statement, the predicted SQL statement can be sent to the SQL terminal and executed to obtain its corresponding execution data. Similarly, this execution data can include data such as execution status, execution duration, and execution results. If the execution status is failure, the corresponding execution reward can be directly set to the fourth value. If the execution status is success, the execution data of the first and second positive examples corresponding to the candidate problem can be obtained. Based on the execution data of the positive examples, the correctness of the execution data of the predicted SQL statement can be determined. If correct, the corresponding execution reward can be set to the fifth value, where the fourth value is less than the fifth value.

[0147] The content quality of the predicted SQL statement can be evaluated using the designated business data, the first positive example, and the second positive example in the second sample set to obtain a content quality reward for the predicted SQL statement. The designated business data is business data related to the candidate question.

[0148] Optionally, when calculating the content quality reward of a predicted SQL statement, an evaluation template may be obtained, the evaluation template including evaluation dimensions, evaluation rules, and output requirements; based on the similarity between the candidate question and the business data, designated business data is determined from the business data; evaluation prompt words are generated using the first positive example, the second positive example, the designated business data, the predicted SQL statement, and the evaluation template corresponding to the candidate question; the evaluation prompt words are used to guide the large language model to evaluate the predicted SQL statement on the evaluation dimension according to the evaluation rules; and according to the output requirements, the evaluation result of the predicted SQL statement is output to obtain the content quality reward of the predicted SQL statement.

[0149] The evaluation template is a pre-set prompt word template used to evaluate the content quality of predicted SQL statements. This prompt word template can be pre-set with evaluation dimensions, evaluation rules, and output requirements. The evaluation dimensions refer to the content that needs to be considered when evaluating the predicted SQL statement. These dimensions can include syntactic correctness, logical accuracy, execution efficiency, readability, and question matching. The evaluation rules can refer to specific scoring rules, such as completely incorrect results receiving 0 points and partially correct results receiving proportional points deductions. These can be set according to actual needs and are not specifically limited here.

[0150] In an embodiment of the present invention, the evaluation template may be:

[0151] "

Task

[0152] You are a SQL quality assessment expert. Please give a comprehensive score (0-10 integers) for the "user question" and the corresponding "generated SQL" based on the following dimensions. Scoring criteria:

[0153]

Rating Dimensions

[0154] 1. Syntactic Correctness (2 points): Whether it complies with SQL syntax specifications

[0155] 2. Logical accuracy (4 points): Does it accurately reflect the problem requirements?

[0156] 3. Execution efficiency (2 points): Whether reasonable optimization methods are used

[0157] 4. Readability (1 point): Is the format / naming clear and standardized?

[0158] 5. Question matching (1 point): Whether the requirements details are fully covered

[0159]

References

[0160] {P_logic_n}

[0161] {P_doc_n}

[0162]

Rating Rules

[0163] - Completely wrong will receive 0 points, and partially correct will receive proportional points deduction

[0164] -Reference SQL case:

[0165] {best_practice}

[0166]

Input format

[0167] User Question:

[0168] {P_q_gen_i}

[0169] Generate SQL:

[0170] {generated_sql}

[0171] Output requirements

[0172] Please return in JSON format:

[0173] {

[0174] "score":[integer score calculated according to the standard],

[0175] "comment": "A concise analysis of the pros and cons, including suggestions for improvement"

[0176] }

[0177] Sample output:

[0178] {"score":7,"comment":"The syntax is correct but lacks index optimization. The WHERE condition does not cover edge cases. It is recommended to add NULL value processing."}"

[0179] Optionally, when generating the candidate SQL statement corresponding to the candidate question, auxiliary data including the candidate document data and the candidate logical data is determined, and the auxiliary data can be directly used as the designated business data here.

[0180] Optionally, the similarity between the candidate question and the business logic data and business document data may be directly calculated, and a portion of the business logic data and business document data with a higher similarity may be used as the designated business data.

[0181] The reference material in the evaluation template can be filled with the corresponding business data. {best_practice} can be filled with the first and second positive examples corresponding to the candidate question as a reference. {P_q_gen_i} can be filled with the candidate question. {generated_sql} can be filled with the predicted SQL statement. After filling in, the evaluation prompt word is obtained. The evaluation prompt word is input into the large language model to guide the large language model to evaluate the predicted SQL statement according to the evaluation rules and the evaluation dimensions. The model then outputs the evaluation results of the predicted SQL statement according to the output requirements, and the evaluation results are used as content quality rewards.

[0182] Finally, the format reward, execution reward, and content quality reward are summed to obtain the target reward for predicting SQL statements. The target reward can be used as part of the loss function. By calculating the loss function, the model parameters of the intermediate model can be adjusted and updated. Training is considered complete when the loss function converges, and a SQL statement generation model is obtained.

[0183] After the SQL statement generation model is trained, it can be deployed for subsequent SQL statement generation scenarios. When generating SQL statements, business data in the target business scenario can be obtained, wherein the business data includes table construction information, business logic data, business document data, core words, and key association data, wherein the key data is the window content extracted from the business logic data and business document data based on the core words and a preset window; a key query is extracted from the query to be processed based on the core words and the preset window, wherein the query to be processed is a user query in the target business scenario, and the key query is the window content with the core word at the center of the preset window; based on the similarity between the key query and the key data and the key association data, the document data to be used and the logic data to be used are determined, wherein the key association data includes the mapping relationship between the key data and the business logic data and the business document data respectively; a target prompt word is constructed using the logic data to be used, the document data to be used, and the table construction information; and the target prompt word is used to guide the SQL statement generation model to generate a target SQL statement corresponding to the query to be processed, wherein the SQL statement generation model is trained based on the SQL statement generation model training method described in any one of claims 1 to 9.

[0184] The business data can refer to the corresponding content in the aforementioned embodiments. Extracting key queries from the pending query can refer to the corresponding content of extracting candidate key questions. Determining the document data and logical data to be used can refer to determining the auxiliary data section. That is, when using the SQL statement generation model, the pending query and other data can be populated into a designated template to construct the corresponding target prompt word. The target prompt word is then input into the SQL statement generation model to obtain the target SQL statement corresponding to the pending query.

[0185] The SQL statement generation model training solution provided by the embodiments of the present invention can be applied to various scenarios requiring SQL statement generation. For example, in the industrial field, queries on production data or in the medical field, queries on patient condition data can be performed. Using the training method provided by the embodiments of the present invention, a SQL statement generation model can be trained to accurately generate high-quality SQL statements for data query.

[0186] The method provided by the embodiment of the present invention can automatically generate multiple candidate SQL statements based on candidate questions in scenarios without historical cases, construct high-quality training data at a relatively low cost, and when generating candidate SQL statements, accurately retrieve auxiliary data related to the candidate questions through sliding and filtering the core word window, and filter and divide the training data using the execution data of the SQL statements. In subsequent training, rewards are comprehensively designed from the three dimensions of format, execution, and content quality to achieve multi-dimensional collaborative optimization, improve the processing capability of the SQL statement generation model in complex scenarios, and ensure that the SQL statement generation model finally trained can accurately generate high-quality SQL statements in complex scenarios.

[0187] To better implement the above method, an embodiment of the present invention further provides a SQL statement generation model training device. The SQL statement generation model training device can be integrated into an electronic device, such as a terminal or a server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc. The server can be a single server or a server cluster consisting of multiple servers.

[0188] For example, in this embodiment, the method of the embodiment of the present invention will be described in detail by taking the SQL statement generation model training device specifically integrated into the server as an example.

[0189] For example, Figure 5 As shown, the SQL statement generation model training device 200 may include an acquisition module 210 , a generation module 220 , a sample division module 230 , a fine-tuning module 240 and an adjustment module 250 .

[0190] An acquisition module 210 is configured to acquire multiple candidate questions in a target business scenario, where the candidate questions are questions that can be solved using SQL query statements.

[0191] A generation module 220 is configured to generate, for each candidate question, a plurality of candidate SQL statements corresponding to the candidate question based on the business data and core words in the target business scenario;

[0192] A sample partitioning module 230 is configured to perform positive and negative example labeling and partitioning processing on the candidate SQL statements based on the execution data of each candidate SQL statement to obtain a first sample set and a second sample set;

[0193] A fine-tuning module 240 is configured to perform supervised fine-tuning on the basic model using the first sample set to obtain an intermediate model;

[0194] The adjustment module 250 is used to calculate the target reward based on the format, execution data and content quality of the predicted SQL statement, and adjust the model parameters of the intermediate model based on the target reward to obtain a SQL statement generation model, where the predicted SQL statement is obtained by the intermediate model inferring the candidate questions in the second sample set.

[0195] In some embodiments, the acquisition module 210 is specifically configured to:

[0196] Obtain business data for the target business scenario, including table creation information, business logic data, and business document data;

[0197] Extracting core words related to the target business scenario from the business data;

[0198] For each target document data, the target document data and the table building information are filled into a conversion template to construct a conversion prompt word, wherein the target document data is business document data containing a core word;

[0199] The conversion prompt word is used to guide the large language model to generate candidate questions corresponding to the target document data.

[0200] In some embodiments, the generation module 220 is specifically configured to:

[0201] Obtaining mapping relationships between key data and the business logic data and the business document data, respectively, to obtain key associated data, wherein the key data is window content extracted from the business logic data and the business document data based on core words and preset windows;

[0202] For each candidate question, based on the core word and the preset window, extract candidate key questions from the candidate questions, wherein the candidate key questions are the window content in which the core word is at the center of the preset window;

[0203] Determining auxiliary data based on the key association data, the similarity between the candidate key questions and the key data, the auxiliary data including candidate logical data and candidate document data;

[0204] Designated prompt words are constructed based on the auxiliary data and the table building information, and the designated prompt words are used to guide the large language model to generate multiple candidate SQL statements corresponding to the candidate questions.

[0205] In some embodiments, the generation module 220 is specifically configured to:

[0206] Controlling a preset window at the starting position of a question word segmentation sequence, and using the content in the preset window as content to be judged, wherein the question word segmentation sequence is a sequence obtained after word segmentation processing of the candidate question;

[0207] Determining whether the content to be judged contains a core word;

[0208] If the content to be judged contains any core word, the preset window is moved so that the core word is at the center of the preset window to obtain an offset window, and the content in the offset window is used as a candidate key question;

[0209] If the content to be judged does not contain the core word or after the offset window is obtained, the preset window is controlled to move according to the specified step size, and the content in the moved preset window is used as the new content to be judged;

[0210] Return to the step of determining whether the content to be determined contains a core word and subsequent steps until the preset window moves to the end position of the question word segmentation sequence.

[0211] In some embodiments, the generation module 220 is specifically configured to:

[0212] determining candidate key data from the key data according to the similarity between the candidate key questions and the key data;

[0213] Determining the business logic data corresponding to the candidate key data in the key associated data as designated logic data, and determining the business document data corresponding to the candidate key data as designated document data;

[0214] determining a first quantity of candidate logical data and a second quantity of candidate document data according to a length of the candidate question;

[0215] A first number of designated logical data are screened to obtain candidate logical data, and a second number of designated document data are screened as candidate document data.

[0216] In some embodiments, the generation module 220 is specifically configured to:

[0217] determining a third number of intermediate logic data based on a similarity between the candidate question and the business logic data, where the third number is related to the first number;

[0218] determining a fourth number of intermediate document data based on a similarity between the candidate question and the business document data, where the fourth number is related to the second number;

[0219] Merging the intermediate logical data and the first number of designated logical data to obtain candidate logical data;

[0220] The intermediate document data and the second number of designated document data are merged to obtain candidate document data.

[0221] In some embodiments, the execution data includes execution status, execution duration, and execution results. The sample segmentation module 230 is specifically configured to:

[0222] For each candidate SQL statement, execute the candidate SQL statement to obtain the execution status, execution duration, and execution result of the candidate SQL statement;

[0223] For each candidate question, among the candidate SQL statements whose execution status is successfully executed, the one with the highest execution result repetition rate and the shortest execution time is marked as the first positive example, and the rest are marked as the second positive examples;

[0224] Mark candidate SQL statements with failed execution status as negative examples;

[0225] Filter all the candidate questions according to the number of the first positive examples to obtain questions to be used;

[0226] According to the number of second positive examples, all candidate SQL statements corresponding to the questions to be used are aggregated and sorted to obtain a sequence of positive and negative examples;

[0227] A first sample set and a second sample set are determined from the positive and negative example sequences according to a specified ratio.

[0228] In some embodiments, the adjustment module 250 is specifically configured to:

[0229] For the candidate questions in the second sample set, use the intermediate model to generate multiple prediction SQL statements and prediction reasoning process data corresponding to the candidate questions;

[0230] Calculating a format reward based on the inclusion relationship between the predicted reasoning process data and the specified reasoning label, and the correlation between the specified answer label and the specified core word in the predicted SQL statement, where the specified core word is the core word in the candidate question;

[0231] Determining an execution reward based on the execution data of the predicted SQL statement, the execution data of the first positive example and the second positive example corresponding to the candidate question;

[0232] Calculating a content quality reward for the predicted SQL statement based on specified business data, a first positive example, and a second positive example corresponding to the candidate question, wherein the specified business data is business data related to the candidate question;

[0233] Combining the format reward, execution reward, and content quality reward to obtain a target reward for the predicted SQL statement;

[0234] Based on the target rewards of all predicted SQL statements, the model parameters of the intermediate model are updated to obtain a SQL statement generation model.

[0235] In some embodiments, the adjustment module 250 is specifically configured to:

[0236] Obtaining an evaluation template, wherein the evaluation template includes evaluation dimensions, evaluation rules, and output requirements;

[0237] determining designated business data from the business data based on similarities between the candidate questions and the business data;

[0238] Generate evaluation prompt words based on the first positive example and the second positive example corresponding to the candidate question, the specified business data, the predicted SQL statement, and the evaluation template;

[0239] Using the evaluation prompt words to guide the large language model to evaluate the predicted SQL statement on the evaluation dimension according to the evaluation rules;

[0240] According to the output requirement, the evaluation result of the predicted SQL statement is output, and a content quality reward for the predicted SQL statement is obtained.

[0241] During specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments and will not be repeated here.

[0242] As can be seen from the above, the SQL statement generation model training device of this embodiment can obtain multiple candidate questions in the target business scenario; for the candidate questions, the business data and core words in the target business scenario are used to generate multiple candidate SQL statements corresponding to the candidate questions; the execution data of each candidate SQL statement is used to mark and divide the candidate SQL statements into positive and negative examples, and a high-quality first sample set and a second sample set can be automatically constructed; the basic model is fine-tuned using the first sample set to obtain an intermediate model; the target reward is then calculated based on the format, execution data, and content quality of the predicted SQL statements predicted by the intermediate model, and the model parameters of the intermediate model are adjusted based on the target reward to obtain the final SQL statement generation model. Designing reward functions from multiple dimensions can improve the generation accuracy of the SQL statement generation model in complex scenarios, and the trained SQL statement generation model can accurately generate high-quality SQL statements in complex scenarios.

[0243] An embodiment of the present invention further provides an electronic device, which may be a terminal, a server, or the like. The terminal may be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop computer, a personal computer, or the like; the server may be a single server or a server cluster consisting of multiple servers, or the like.

[0244] In some embodiments, the SQL statement generation model training device can also be integrated into multiple electronic devices. For example, the SQL statement generation model training device can be integrated into multiple servers, and the SQL statement generation model training method of the present invention can be implemented by multiple servers.

[0245] In this embodiment, the electronic device of this embodiment is a server as an example for detailed description, for example, Figure 6 , which shows a schematic structural diagram of an electronic device involved in an embodiment of the present invention, specifically:

[0246] The electronic device may include one or more processing core processors 310, one or more computer-readable storage media memories 320, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will appreciate that Figure 6 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0247] The processor 310 is the control center of the electronic device. It connects all parts of the electronic device using various interfaces and circuits. It executes software programs and / or modules stored in the memory 320 and accesses data stored in the memory 320 to perform various functions of the electronic device and process data. In some embodiments, the processor 310 may include one or more processing cores. In some embodiments, the processor 310 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 310.

[0248] The memory 320 can be used to store software programs and modules. The processor 310 executes various functional applications and data processing by running the software programs and modules stored in the memory 320. The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 320 may also include a memory controller to provide the processor 310 with access to the memory 320.

[0249] The electronic device also includes a power supply 330 for supplying power to various components. In some embodiments, the power supply 330 can be logically connected to the processor 310 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 330 can also include any components such as one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0250] The electronic device may further include an input module 340 , which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0251] The electronic device may further include a communication module 350. In some embodiments, the communication module 350 may include a wireless module. The electronic device may perform short-range wireless transmission via the wireless module of the communication module 350, thereby providing the user with wireless broadband Internet access. For example, the communication module 350 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0252] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically, in this embodiment, the processor 310 in the electronic device loads the executable files corresponding to one or more application processes into the memory 320 according to the following instructions, and the processor 310 runs the application stored in the memory 320, thereby implementing the steps of the method in each embodiment of the present invention.

[0253] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0254] As can be seen from the above, the electronic device provided by the embodiment of the present invention can obtain multiple candidate questions in the target business scenario; for the candidate questions, the business data and core words in the target business scenario are used to generate multiple candidate SQL statements corresponding to the candidate questions; the execution data of each candidate SQL statement is used to mark and divide the candidate SQL statements into positive and negative examples, and a high-quality first sample set and a second sample set can be automatically constructed; the basic model is fine-tuned using the first sample set to obtain an intermediate model; the target reward is then calculated based on the format, execution data, and content quality of the predicted SQL statements predicted by the intermediate model, and the model parameters of the intermediate model are adjusted based on the target reward to obtain the final SQL statement generation model. Designing reward functions from multiple dimensions can improve the generation accuracy of the SQL statement generation model in complex scenarios, and the trained SQL statement generation model can accurately generate high-quality SQL statements in complex scenarios.

[0255] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0256] To this end, an embodiment of the present invention provides a computer-readable storage medium, which stores multiple instructions that can be loaded by a processor to execute the steps in any SQL statement generation model training method provided by an embodiment of the present invention.

[0257] The storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0258] According to one aspect of the present invention, a computer program product or computer program is provided, comprising a computer program / instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instructions from the computer-readable storage medium and executes the computer program / instructions, causing the electronic device to perform the methods provided in various optional implementations of the aforementioned embodiments for constructing training data, training an SQL statement generation model, or using an SQL statement generation model.

[0259] Since the instructions stored in the storage medium can execute the steps in any SQL statement generation model training method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any SQL statement generation model training method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0260] The above is a detailed introduction to a SQL statement generation model training method and device provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A SQL statement generation model training method, characterized in that: The method comprises: Obtain multiple candidate questions in the target business scenario, where the candidate questions are questions that can be solved using SQL query statements; For each candidate question, generate multiple candidate SQL statements corresponding to the candidate question based on the business data and core words in the target business scenario; Based on the execution data of each candidate SQL statement, the candidate SQL statement is marked as positive and negative and divided to obtain a first sample set and a second sample set; Performing supervised fine-tuning on the basic model using the first sample set to obtain an intermediate model; The target reward is calculated based on the format, execution data and content quality of the predicted SQL statement, and the model parameters of the intermediate model are adjusted based on the target reward to obtain a SQL statement generation model, where the predicted SQL statement is obtained by the intermediate model inferring the candidate questions in the second sample set.

2. The method according to claim 1, characterized in that The step of obtaining multiple candidate questions in the target business scenario includes: Obtain business data for the target business scenario, including table creation information, business logic data, and business document data; Extracting core words related to the target business scenario from the business data; For each target document data, the target document data and the table building information are filled into a conversion template to construct a conversion prompt word, wherein the target document data is business document data containing a core word; The conversion prompt word is used to guide the large language model to generate candidate questions corresponding to the target document data.

3. The method according to claim 2, characterized in that For each candidate question, based on the business data and core words in the target business scenario, multiple candidate SQL statements corresponding to the candidate question are generated, including: Obtaining mapping relationships between key data and the business logic data and the business document data, respectively, to obtain key associated data, wherein the key data is window content extracted from the business logic data and the business document data based on core words and preset windows; For each candidate question, based on the core word and the preset window, extract candidate key questions from the candidate questions, wherein the candidate key questions are the window content in which the core word is at the center of the preset window; Determining auxiliary data based on the key association data, the similarity between the candidate key questions and the key data, the auxiliary data including candidate logical data and candidate document data; Designated prompt words are constructed based on the auxiliary data and the table building information, and the designated prompt words are used to guide the large language model to generate multiple candidate SQL statements corresponding to the candidate questions.

4. The method according to claim 3, characterized in that The step of extracting candidate key questions from candidate questions based on core words and preset windows includes: Controlling a preset window at the starting position of a question word segmentation sequence, and using the content in the preset window as content to be judged, wherein the question word segmentation sequence is a sequence obtained after word segmentation processing of the candidate question; Determining whether the content to be judged contains a core word; If the content to be judged contains any core word, the preset window is moved so that the core word is at the center of the preset window to obtain an offset window, and the content in the offset window is used as a candidate key question; If the content to be judged does not contain the core word or after the offset window is obtained, the preset window is controlled to move according to the specified step size, and the content in the moved preset window is used as the new content to be judged; Return to the step of determining whether the content to be determined contains a core word and subsequent steps until the preset window moves to the end position of the question word segmentation sequence.

5. The method according to claim 3, characterized in that The determining of auxiliary data according to the key association data, the similarity between the candidate key question and the key data includes: determining candidate key data from the key data according to the similarity between the candidate key questions and the key data; Determining the business logic data corresponding to the candidate key data in the key associated data as designated logic data, and determining the business document data corresponding to the candidate key data as designated document data; determining a first quantity of candidate logical data and a second quantity of candidate document data according to a length of the candidate question; A first number of designated logical data are screened to obtain candidate logical data, and a second number of designated document data are screened as candidate document data.

6. The method according to claim 5, characterized in that The method further comprises: determining a third number of intermediate logic data based on a similarity between the candidate question and the business logic data, where the third number is related to the first number; determining a fourth number of intermediate document data based on a similarity between the candidate question and the business document data, where the fourth number is related to the second number; Merging the intermediate logical data and the first number of designated logical data to obtain candidate logical data; The intermediate document data and the second number of designated document data are merged to obtain candidate document data.

7. The method according to claim 1, characterized in that The execution data includes an execution status, an execution duration, and an execution result. Based on the execution data of each candidate SQL statement, the candidate SQL statement is marked with positive and negative examples and divided into two groups to obtain a first sample set and a second sample set, including: For each candidate SQL statement, execute the candidate SQL statement to obtain the execution status, execution duration, and execution result of the candidate SQL statement; For each candidate question, among the candidate SQL statements whose execution status is successfully executed, the one with the highest execution result repetition rate and the shortest execution time is marked as the first positive example, and the rest are marked as the second positive examples; Mark candidate SQL statements with failed execution status as negative examples; Filter all the candidate questions according to the number of the first positive examples to obtain questions to be used; According to the number of second positive examples, all candidate SQL statements corresponding to the questions to be used are aggregated and sorted to obtain a sequence of positive and negative examples; A first sample set and a second sample set are determined from the positive and negative example sequences according to a specified ratio.

8. The method according to claim 1, characterized in that The calculating of the target reward based on the format, execution data, and content quality of the predicted SQL statement, and adjusting the model parameters of the intermediate model based on the target reward to obtain the SQL statement generation model includes: For the candidate questions in the second sample set, use the intermediate model to generate multiple prediction SQL statements and prediction reasoning process data corresponding to the candidate questions; Calculating a format reward based on the inclusion relationship between the predicted reasoning process data and the specified reasoning label, and the correlation between the specified answer label and the specified core word in the predicted SQL statement, where the specified core word is the core word in the candidate question; Determining an execution reward based on the execution data of the predicted SQL statement, the execution data of the first positive example and the second positive example corresponding to the candidate question; Calculating a content quality reward for the predicted SQL statement based on specified business data, a first positive example, and a second positive example corresponding to the candidate question, wherein the specified business data is business data related to the candidate question; Combining the format reward, execution reward, and content quality reward to obtain a target reward for the predicted SQL statement; Based on the target rewards of all predicted SQL statements, the model parameters of the intermediate model are updated to obtain a SQL statement generation model.

9. The method according to claim 8, characterized in that The calculating the content quality reward of the predicted SQL statement according to the specified business data, the first positive example corresponding to the candidate question, and the second positive example includes: Obtaining an evaluation template, wherein the evaluation template includes evaluation dimensions, evaluation rules, and output requirements; determining designated business data from the business data based on similarities between the candidate questions and the business data; Generate evaluation prompt words based on the first positive example and the second positive example corresponding to the candidate question, the specified business data, the predicted SQL statement, and the evaluation template; Using the evaluation prompt words to guide the large language model to evaluate the predicted SQL statement on the evaluation dimension according to the evaluation rules; According to the output requirement, the evaluation result of the predicted SQL statement is output, and a content quality reward for the predicted SQL statement is obtained.

10. A SQL statement generation model training device, used to implement the method according to any one of claims 1 to 9, characterized in that: The device comprises: An acquisition module is used to acquire multiple candidate questions in a target business scenario, where the candidate questions are questions that can be solved using SQL query statements; A generation module is used to generate, for each candidate question, multiple candidate SQL statements corresponding to the candidate question based on the business data and core words in the target business scenario; A sample division module, configured to perform positive and negative example marking and division processing on the candidate SQL statements based on the execution data of each candidate SQL statement, so as to obtain a first sample set and a second sample set; A fine-tuning module, configured to perform supervised fine-tuning on the base model using the first sample set to obtain an intermediate model; An adjustment module is used to calculate a target reward based on the format, execution data, and content quality of the predicted SQL statement, and adjust the model parameters of the intermediate model based on the target reward to obtain a SQL statement generation model, where the predicted SQL statement is obtained by the intermediate model inferring the candidate questions in the second sample set.

Images