Method and device for fine-tuning a large model to generate SQL statements

Abstract

The embodiment of the specification discloses a method and device for fine-tuning a large model, a method and device for generating an SQL statement. The fine-tuning method comprises the following steps: first, generating training samples, which include prompt texts with a predetermined format and answer texts corresponding to the prompt texts, wherein the prompt texts include a Chinese query statement, meta information of a database table queried by the Chinese query statement, and Chinese annotations of the meta information; the answer texts include SQL statements corresponding to the Chinese query statement; then, fine-tuning a pre-trained large model by using a plurality of training samples to obtain a fine-tuned large model. Based on the fine-tuned large model, a target SQL statement corresponding to a target query statement input by a user can be accurately generated.

Description

Technical Field

[0001] This specification relates to the field of natural language processing technology, and more particularly to a method and apparatus for fine-tuning large models, a method and apparatus for generating SQL statements, a computer-readable storage medium, and a computing device. Background Technology

[0002] Large Language Models (LLMs) generally refer to machine learning models trained on large amounts of text data from multiple different domains. They can generate natural language text or understand the semantics of natural language text. Pre-trained LLMs can be used directly or, after fine-tuning, for handling domain-specific natural language tasks, such as intelligent question answering. Specifically, this involves generating corresponding Structured Query Language (SQL) statements based on user queries to a database.

[0003] However, currently, the performance of LLM-based SQL statement responses is limited when users ask questions in Chinese. Therefore, an improved solution is needed to meet higher requirements in practical applications, including supporting users to ask SQL generation questions in pure Chinese and providing accurate SQL statements as answers. Summary of the Invention

[0004] This specification describes a method and apparatus for fine-tuning a large model, as well as a method and apparatus for generating SQL statements, which can meet higher requirements in practical applications.

[0005] Based on the first aspect, a method for fine-tuning large models is provided. This method includes:

[0006] Training samples are generated, including prompt text with a predetermined format and corresponding response text. The prompt text includes a Chinese query statement, metadata of the database table queried by the Chinese query statement, and Chinese annotations for the metadata. The response text includes the SQL statement corresponding to the Chinese query statement. A pre-trained large model is fine-tuned using multiple training samples to obtain a fine-tuned large model, which is used to generate the target SQL statement corresponding to the user's input target query statement.

[0007] In one embodiment, the metadata includes multiple information items, and the Chinese annotations include annotation items for each information item; the content in the predetermined format prompt text is arranged in the following order: the Chinese query statement, multiple pairs of information items-annotation items.

[0008] In one specific embodiment, the prompt text also includes the task name of the task that generates the SQL statement, and the predetermined format further indicates that the task name should be placed before the Chinese query statement.

[0009] In one embodiment, generating training samples includes: obtaining the Chinese query statement and the corresponding SQL statement; extracting the metadata of the database table corresponding to the table name contained in the SQL statement from the database system, as well as the Chinese annotations of the metadata; and constructing the training samples using the Chinese query statement, the SQL statement, and the extracted metadata and Chinese annotations.

[0010] In one embodiment, generating training samples includes: automatically populating a template configured for the training samples based on a database system to generate the training samples.

[0011] In one specific embodiment, the template includes a first sub-template configured for the Chinese query statement, a second sub-template configured for the metadata and Chinese annotations, and a third sub-template configured for the SQL statement; wherein, automatically filling the template configured for the training samples includes: automatically determining the data to be filled into the corresponding slots according to the filling rules configured for each slot of the first sub-template; and automatically filling the second and third sub-templates based on the data filled into the first sub-template and the database system.

[0012] In a more specific embodiment, the first sub-template includes a first slot; wherein, automatically determining data to be filled into the corresponding slot according to the filling rules configured for each slot of the first sub-template includes: determining data under the data category contained in the filling rules of the first slot based on the database system to fill the first slot.

[0013] Furthermore, in one example, determining data under the data category based on the database system to fill the first slot includes: determining multiple original data under the data category based on the database system; selecting one of the multiple original data to fill the first slot, or filling the first slot with an alias of one of the original data.

[0014] In a more specific example, the data category is a table name class, a column name class, or a field value class with a predefined data type.

[0015] In another example, the first sub-template includes a second slot, and the configuration information of the template includes multiple candidate slot values ​​configured for the second slot; wherein, automatically determining the data to be filled into the corresponding slot according to the filling rules configured for each slot of the first sub-template includes: selecting one of the multiple candidate slot values ​​to fill into the second slot according to the filling rules of the second slot.

[0016] In another example, the first sub-template includes a third slot; wherein, according to the filling rules configured for each slot of the first sub-template, automatically determining the data to be filled into the corresponding slot includes: randomly generating data of the data type contained in the filling rules of the third slot and filling it into the third slot.

[0017] In a more specific example, the data type is a numeric type or a date type.

[0018] In another example, the first sub-template includes a mandatory part and randomly includes an optional part; the constituent units of the optional part include a fourth slot, a fifth slot, and a sixth slot, and the filling rules of each slot sequentially indicate the Chinese description of the filling logical operator, the operator for the predetermined data type, and the operand for the predetermined data type.

[0019] In a more specific example, before automatically determining the data to be filled into the corresponding slot according to the filling rules configured for each slot of the first sub-template, the method further includes: randomly determining whether to add the optional part to the first sub-template; if it is determined that an addition is to be made, determining the number of constituent units in the added optional part, and adding template content corresponding to the spliced ​​optional part to the third sub-template.

[0020] In another example, based on the data to be filled into the first sub-template and the database system, the second and third sub-templates are automatically filled, including: based on the table name indicated by the data to be filled into the first sub-template, extracting the metadata of the database table corresponding to the table name and the Chinese annotation of the metadata from the database system, and filling it into the second sub-template; based on the slot mapping relationship between the first and third sub-templates, determining the data to be filled into the corresponding slots in the third sub-template according to the data to be filled into each slot of the first sub-template.

[0021] According to the second aspect, a method for generating SQL statements is provided. This method includes:

[0022] The system receives a target query statement input by the user; based on the target query statement, the metadata of the queried data table, and Chinese annotations of the metadata, it constructs a target prompt text with a predetermined format. The target prompt text is then processed using a large model fine-tuned using the method in the first aspect to generate a target SQL statement corresponding to the target query statement.

[0023] According to a third aspect, an apparatus for fine-tuning a large model is provided. The apparatus includes:

[0024] The training sample generation module is configured to generate training samples, which include prompt text with a predetermined format and corresponding response text. The prompt text includes a Chinese query statement, metadata of the database table queried by the Chinese query statement, and Chinese annotations for the metadata. The response text includes the SQL statement corresponding to the Chinese query statement. The large model fine-tuning module is configured to fine-tune a pre-trained large model using multiple training samples to obtain a fine-tuned large model, which is used to generate a target SQL statement corresponding to the user-input target query statement.

[0025] According to a fourth aspect, an apparatus for generating SQL statements is provided. The apparatus includes: a query statement receiving module configured to receive a target query statement input by a user; a prompt text construction module configured to construct a target prompt text with a predetermined format based on the target query statement, metadata of the queried data table, and Chinese annotations of the metadata; and an SQL statement generation module configured to process the target prompt text using a large model fine-tuned using the method provided in the first aspect or the apparatus provided in the third aspect, to generate a target SQL statement corresponding to the target query statement.

[0026] According to a fifth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method provided in the first or second aspect.

[0027] According to a sixth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method provided in the first or second aspect.

[0028] In summary, by using the methods or apparatus disclosed in the embodiments of this specification, the large model can learn from multiple generated training samples the relationship between the Chinese semantics hidden in the user's question and the meta-information such as the table name of the database table, as well as the Chinese annotations of the meta-information, thereby supporting users to raise SQL generation questions in pure Chinese form and being able to answer accurate SQL statements. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This example demonstrates a scenario where a generic question-and-answer model is used to generate erroneous SQL statements.

[0031] Figure 2 This diagram illustrates the implementation architecture of the scheme for fine-tuning a large model as disclosed in the embodiments of this specification.

[0032] Figure 3 This shows the use of a method based on Figure 2 The example shown illustrates a scenario where a large model with fine-tuned architecture generates correct SQL statements.

[0033] Figure 4 This diagram illustrates a method flow chart for fine-tuning a large model as disclosed in an embodiment of this specification.

[0034] Figure 5 This document shows a schematic flowchart of the method for generating SQL statements disclosed in the embodiments of this specification;

[0035] Figure 6 This diagram illustrates the structure of the apparatus for fine-tuning a large model as disclosed in the embodiments of this specification.

[0036] Figure 7 This diagram illustrates the structure of the apparatus for generating SQL statements disclosed in the embodiments of this specification. Detailed Implementation

[0037] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0038] As mentioned earlier, current Chinese SQL question answering based on large language models (or simply large models) is not performing well. For example, current general question answering models based on large models often cannot directly respond to users' text-to-SQL requests, requiring users to input more prompts to complete the task, and they will also guess field information on their own, such as... Figure 1 As shown, the information selected by the rectangle is incorrect and does not match the information in the database table.

[0039] Based on the above observations, the applicant concluded that the general question-answering model did not establish the relationship between Chinese semantics and data table fields when using open-source datasets for training, and therefore could only mechanically guess the answers to user questions based on learning experience.

[0040] Therefore, the applicant proposed a Chinese SQL question-answering scheme based on a large model, which allows the model to learn the relationship between Chinese semantics and data table fields, thereby enabling users to ask SQL generation questions in pure Chinese and to provide accurate SQL statements.

[0041] Specifically, a new prompt text format is designed, which includes extracting metadata (such as table name, column name, etc.) from the database table query for the user's question, along with Chinese annotations of the metadata, from the raw information of the database. This metadata is then concatenated with the user's question to form a prompt text with accompanying Chinese annotations, and the answer is provided based on the aforementioned metadata according to the user's question. See [link to relevant documentation] for more details. Figure 2 The example shown is a newly designed prompt text format (a feasible example, but not the only one), and training samples consisting of prompt text and corresponding answers can be used for fine-tuning of large models.

[0042] During the fine-tuning process based on the aforementioned training samples, the large model learns the mapping relationship between the implicit Chinese semantics in user questions and the metadata of database tables, as well as the Chinese annotations within that metadata. This fine-tuned large model then supports users submitting SQL generation questions in pure Chinese and can provide accurate SQL statements. See also... Figure 3 Compared to Figure 1 Error SQL statements obtained using a general question-and-answer model Figure 3 The image shows the correct SQL statement generated by the large model after fine-tuning using a graph-based architecture.

[0043] The following is combined Figure 4 This document provides a more detailed description of the implementation steps of the Chinese SQL question-answering scheme based on a large model disclosed in the embodiments of this specification. Figure 4 This diagram illustrates a method for fine-tuning a large model as disclosed in an embodiment of this specification. The entity executing this method can be any device, server, platform, or device cluster with computing and processing capabilities. Figure 4 As shown, the method includes the following steps:

[0044] Step S410: Generate training samples, which include prompt text with a predetermined format and corresponding answer text; wherein the prompt text includes a Chinese query statement (or user question), the metadata of the database table queried by the Chinese query statement, and Chinese annotations on the metadata; wherein the answer text includes the SQL statement corresponding to the Chinese query statement.

[0045] To facilitate understanding, we will first introduce the composition of the training samples, and then introduce how the training samples are generated.

[0046] Specifically, the above-mentioned content included in the training samples can be organized and illustrated in Table 1 below:

[0047] Table 1

[0048]

[0049] The comparison of each item in Table 1 is as follows:

[0050] 1) Chinese query statement

[0051] Chinese query statements correspond to user questions and are text statements in natural language form intended for data retrieval. The natural language type involved may include at least Chinese, or only Chinese. In one embodiment, a Chinese query statement includes only Chinese characters. For example, a Chinese query statement might be: "Query the list of users who currently have no credit default behavior." In another embodiment, a Chinese query statement primarily includes Chinese characters, but also includes a small number of English letters and / or numbers. For example, a Chinese query statement might be: "Query the list of students in Grade 3, Class 1 who have achieved an A in Chinese language."

[0052] 2) Meta-information of the database table queried by the Chinese query statement

[0053] Metadata, also known as metadata, is the information that defines the structure of a database table. In one example, this metadata includes the table name and column names; in another, it includes field values ​​and / or the rules governing those values. It's important to understand that while training samples may include some or all of the full metadata for a database table, the specific information items are pre-configured by the administrators.

[0054] 3) Chinese annotations of metadata

[0055] It should be understood that metadata is generally composed of English letters, numbers, special characters, etc., and Chinese annotations of metadata provide comments and explanations in Chinese.

[0056] 4) Prompt text with a pre-defined format

[0057] It should be understood that the content included in the pre-formatted instruction text, as well as the arrangement of the content, can be preset by staff.

[0058] The prompt text shall include at least items 1), 2), and 3) above. In one embodiment, the prompt text may also include the task name, for example, "Generate SQL," or, for example, Figure 2 The example shown is "Text to SQL". In another embodiment, the prompt text may also include tag information, for example, Figure 2 The labels “Issue,” “Task Name,” and “Mode” shown in the diagram are used to identify the content that follows.

[0059] The key to arranging the items is that the arrangement is the same across different training samples, not the arrangement itself. In one embodiment, the items can be arranged in the following order:

[0060] Task name, Chinese query statement, metadata, and Chinese annotations for the metadata.

[0061] In another embodiment, the metadata includes multiple information items, where each information item and its corresponding annotation item in the Chinese annotation form an information item-annotation item pair. Based on this, the arrangement of the items can be as follows:

[0062] Chinese query statements, multiple pairs of information items - annotation items.

[0063] Furthermore, in one example, the metadata includes the table name and column names of the database table, and the Chinese annotations include the Chinese names of the table name and column names. In this case, the arrangement of the items can be as follows:

[0064] {Chinese query text}, pattern information: [table name, Chinese name], (column name 1, Chinese name 1), (column name 2, Chinese name 2)......(column name N, Chinese name N)

[0065] 5) SQL statements corresponding to Chinese query statements

[0066] It is understandable that the correct SQL statement corresponding to the Chinese query statement can be directly used to query data in the database, thereby obtaining the correct query results.

[0067] 6) Answer the text

[0068] The response text must at least include the SQL statement corresponding to the Chinese query. For example, the response text may also include tag information indicating that it is followed by the answer, such as... Figure 1 The text indicates the "Answer" label, or the label information could be the English word "answer".

[0069] The above, combined with Table 1, describes the composition of the training samples. Next, we will introduce how the training samples are generated. It should be understood that, for clarity and conciseness, the following description mainly focuses on the generation of a single training sample; however, in practical applications, multiple training samples will be generated.

[0070] In one feasible implementation method A, training samples can be generated through the following steps a1, a2 and a3.

[0071] Step a1: Obtain the Chinese query statement and the corresponding SQL statement.

[0072] In one embodiment, given existing research on natural language to SQL statements, relevant corpora can be purchased directly. In another embodiment, Chinese query-SQL statement pairs can be manually constructed, which is also a mainstream approach in the industry.

[0073] Step a2: Based on the table name contained in the obtained SQL statement, extract the metadata of the database table corresponding to that table name and the Chinese annotations of the metadata from the database system.

[0074] This is understandable. The table name can be extracted from the FROM clause of an SQL statement, and then the corresponding metadata and Chinese annotations can be retrieved from the database system based on this table name. Furthermore, a database system typically includes the database and a database management system (DBMS). Generally, the database table's metadata and corresponding Chinese annotations can be obtained from the DBMS, and data within the database can be accessed through the DBMS.

[0075] Step a3: Using the Chinese query statements and corresponding SQL statements obtained above, as well as the metadata and corresponding Chinese annotations extracted from the database system, construct training samples.

[0076] The above describes the generation of training samples using Implementation Method A. Furthermore, considering the excessive cost of directly purchasing corpora or manually constructing Chinese query statements (SQL statements), Implementation Method B is proposed to automate data production, achieving low-cost and high-efficiency construction of a large amount of training data.

[0077] Specifically, in implementation method B, the template configured for the training samples is automatically populated based on the database system to generate training samples. It can be understood that this automatic population can be achieved by running a script.

[0078] To aid in intuitive understanding, we will first present a template example and a training sample example generated based on this template example.

[0079] Template Example ①:

[0080] / / Numerical condition, enumerated values: ["greater than", "less than", "lower than", "exceeding", "greater than or equal to", "less than or equal to", "not greater than", "not exceeding", "not lower than", "not less than"]

[0081] {"Prompt text":"%s%s%s%s (and %s%s%s) of %s, pattern information is %s",

[0082] "Answer":"SELECT%s FROM%s WHERE%s%s%s(AND%s%s%s)"}

[0083] In the template example above, each %s symbol represents a slot, and the parentheses () in the template indicate that the content within is optional.

[0084] Training sample example ②:

[0085] {"Prompt Text":"For users whose accidental insurance claim amount exceeds 100,000 and whose driving experience is less than 2 years, the mode information is [] "

[0086] [accident_insurance_claims_v5 Accidental Injury Insurance Claims Table], (insurance_amount (insured amount, recorded in yuan)), (user_system_id (user identifier, assigned by the system)), (claim_amount (claim amount, recorded in yuan)), (driving_experience (driving experience, recorded in years)), (user_occupation (user occupation, painter, writer, musician, scientist))",

[0087] "Answer":"SELECT user_system_id FROM accident_insurance_claims_v5 WHEREclaim_amount>100000AND driving_experience≤2)"}

[0088] The above first visually demonstrates template example ① and the training sample example ② obtained after automatic filling. The following details the composition of the template and the corresponding filling method. Specifically, the template content includes at least three sub-templates, denoted as the first sub-template, the second sub-template, and the third sub-template, which are used to generate the [Chinese query statement], [meta-information and Chinese annotations], and [SQL statement] in the training samples, respectively. See column 3 of Table 1 for details. It should be noted that the training samples described above can also include other content, such as label information. This content is relatively fixed and can be directly included in the template. Furthermore, the use of "first" in "first sub-template," "second" in "second sub-template," and similar terms elsewhere in the text are for distinguishing similar items and do not have any ranking or other limiting function.

[0089] It should be noted that each of the above three sub-templates includes several slots, and each slot has a pre-defined filling rule. These filling rules can be written into a script used to automatically generate training samples. The automatic filling of the above three sub-templates includes:

[0090] b1) First sub-template

[0091] Automatically determine the data to be filled into the corresponding slots according to the filling rules configured for each slot of the first sub-template. Exemplarily, the first sub-template is: %s%s%s%s (and %s%s%s)'s %s in Template Example ①.

[0092] It should be understood that the first sub-template is used for generating Chinese query statements, the second sub-template is used for generating meta information and its Chinese annotations, and the third sub-template is used for generating SQL statements. Since the meta information and Chinese annotations depend on the data table to be queried by the Chinese query statement, and the SQL statement is used to implement the query function indicated by the Chinese query statement, accordingly, it is designed that after the data filled into the slots of the first sub-template (also called slot values) are determined, the slot values of the second sub-template and the third sub-template are determined correspondingly, which also makes the filling of the first sub-template particularly important.

[0093] The data source for the slots in the first sub-template may include any one of the following three types: 1) database system, 2) enumeration values in the template, 3) random generation.

[0094] In one embodiment, the first sub-template includes a first slot, and data under the data category may be determined from the database system to fill the first slot according to the data category included in the filling rule of the first slot. In a specific embodiment, the data category is a table name category, a column name category, or a field value category with a predetermined data type. It should be understood that the predetermined data type may be a string type or the like, and exemplarily, the field value of the string type may be a city name, a product name or the like.

[0095] On the other hand, in a specific embodiment, a plurality of raw data under the data category may be determined based on the database system first, then one piece of raw data, or the Chinese name or Chinese alias of one piece of raw data is selected from the plurality of raw data to fill the first slot.

[0096] Taking the first sub-template "%s%s%s%s's %s" as an example, assuming that the data category included in the filling rule of the first slot is a table name, at this time, the Chinese name of a certain table name or a certain alias may be extracted from the database system to fill the first slot. For example, a certain table name is accident_insurance_claims_v5, its Chinese name is accidental injury insurance, and its aliases include accidental injury insurance and accident insurance. At this time, the accident insurance in the aliases may be selected to fill the first slot, to obtain:

[0097] Accident insurance %s%s%s's %s.

[0098] Further, assuming that the data category included in the filling rule for the second slot position is a column name, at this time, a certain column name or one of the aliases of a certain column name may be extracted from the database system and filled into the second slot position. For example, the Chinese name of a column named claim_amount is claim amount, and its aliases include amount, compensation amount, etc. At this time, the Chinese name "claim amount" may be selected and filled into the second slot position, obtaining:

[0099] Accident insurance claim amount %s %s's %s.

[0100] For another example, assuming that the data category included in the filling rule for the last slot position is also a column name, in this case, illustratively, filling the last slot position may obtain:

[0101] Users with accident insurance claim amount %s %s.

[0102] The above embodiments introduce extracting data from the database system to fill the slot positions in the first sub-template.

[0103] In another embodiment, the first sub-template includes a second slot position, and the configuration information of the template includes a plurality of candidate slot values configured for the second slot position. Correspondingly, one of the plurality of candidate slot values may be selected and filled into the second slot position according to the filling rule of the second slot position.

[0104] In a specific embodiment, the filling rule of the second slot position indicates filling an operator for a predetermined data type. Correspondingly, one of the plurality of candidate operators configured in the template may be randomly selected and filled into the second slot position. Still taking the first sub-template as "%s %s %s %s's %s" as an example, assuming that the third slot position indicates filling an operator for a numerical type, at this time, an enumeration value may be randomly selected from a plurality of enumeration values in the " / / numerical condition" part of template example ①, such as "greater than", and filled into the third slot position. Combining the data already filled into the first and second slot positions in the foregoing example, it is obtained that:

[0105] Users whose accident insurance claim amount is greater than %s.

[0106] The above embodiments introduce selecting a candidate enumeration value from the template configuration information to fill the slot positions in the first sub-template.

[0107] In yet another embodiment, the first sub-template includes a third slot position, whose filling rule indicates randomly generating data within a predetermined value range under a certain data type (such as a numerical type or a date type). Correspondingly, random data may be generated according to the filling rule and filled into the third slot position. Exemplarily, assuming that the data type is a numerical type, the predetermined value range is 0-1000, and the unit is ten thousand, at this time, a numerical value such as 100 thousand may be randomly generated and filled into the third slot position, for example:

[0108] Users whose accidental injury insurance claim amount exceeds 100,000 yuan.

[0109] The above mainly introduces how to obtain data from the corresponding data source and fill it according to the filling rules of the slots in the first sub-template.

[0110] According to another embodiment, optional parts other than mandatory parts can also be configured for the first sub-template. Whether the first sub-template includes optional parts is random, and the slots in the optional parts are also random. This can make the final constructed training dataset more diverse and effectively avoid the problem of a single data scenario.

[0111] Specifically, the constituent units of the optional portion may include multiple predetermined slots, and the number of constituent units can be one or more. It should be understood that before filling the first sub-template, it can be determined whether to add an optional portion to the first sub-template, as well as the number of constituent units of the optional portion and the number of slots in each constituent unit.

[0112] The following section introduces the slots and filling methods of optional components from the perspective of constituent units.

[0113] In one specific embodiment, for a unit with four slots, to distinguish it from the aforementioned slots, they are referred to as the fourth slot, the fifth slot, the sixth slot, and the seventh slot, respectively.

[0114] The filling rule for the fourth slot mentioned above indicates the Chinese description of the logical operator to be filled. Accordingly, a Chinese description for different logical operators, such as "and", "and", "or", "not", etc., can be randomly selected from the multiple Chinese descriptions for different logical operators included in the template's configuration information to fill the fourth slot. For example, see the optional part included in the aforementioned template example ①: (and %s %s %s).

[0115] The filling rule for the fifth slot mentioned above indicates that the column name, or the Chinese name or alias of the column name, should be filled. Accordingly, the corresponding data can be extracted from the database system for filling. For example, see the slot value "Driving Experience" in the aforementioned training sample example ②.

[0116] The filling rule for the sixth slot indicates that an operator or its Chinese description should be filled in, where the operator corresponds to the data type of the preceding fifth slot. Accordingly, Chinese descriptions of candidate operators for that data type can be extracted from the template's configuration information and filled into the sixth slot. For example, if the data type corresponding to the fifth slot is numeric, the operator filled into the sixth slot is "not exceeding" from the aforementioned training sample example ②.

[0117] The filling rule for the seventh slot indicates the operand to be filled. The operand has the data type corresponding to the fifth slot, which can be extracted from the database system or randomly generated. For example, the string value "Zhuhai" can be extracted from the database and filled into the seventh slot, or the numeric value 2 can be randomly generated and filled into the seventh slot.

[0118] For example, if the first sub-template includes an optional part, and includes one component unit, and the component unit includes four slots, after filling it, we can get: and the driving experience is no more than two years, or we can get: and the registered address is not in the capital city.

[0119] In another specific embodiment, for a unit with three slots, the filling rules for each of the three slots sequentially indicate the Chinese description of the logical operator, the operator for a predetermined data type, and the operand for the predetermined data type. It should be understood that the predetermined data type can be the data type corresponding to a slot in the mandatory part of the first sub-template.

[0120] For example, if the optional part in the first sub-template includes one component unit and the component unit includes three slots, after filling it, we can get: and less than 3000, or we can get: and not belonging to East China.

[0121] The above describes the filling rules for each slot in the first sub-template, as well as the optional parts that can be randomly included. By filling the first sub-template, we can obtain Chinese query statements used to construct training samples, such as "users with accident insurance claims exceeding 100,000 and driving experience not exceeding 2 years" in training sample example ②.

[0122] b2) Second sub-template

[0123] The second sub-template is used to generate metadata about the data table queried by the Chinese query and Chinese annotations for the metadata. For example, the second sub-template is as shown in template example ①: the schema information is %s.

[0124] During the generation of training samples, the metadata of the database table corresponding to the table name indicated by the data filled into the first sub-template can be automatically extracted from the database system, along with the Chinese annotation of the metadata, and then filled into the second sub-template.

[0125] For example, one or more slots in the first sub-template are filled with the Chinese name or alias of the table name. This allows the location of one or more corresponding table names, and the extraction of the table's metadata and corresponding Chinese annotations, which are then filled into the second sub-template.

[0126] By filling in the second sub-template, we can obtain the metadata of the database table used to construct the training samples and the corresponding Chinese annotations. For example, in training sample example ②: the pattern information is [accident_insurance_claims_v5 accidental injury insurance claim table], (insurance_amount (in yuan)), (user_system_id (assigned by the system)), (claim_amount (in yuan)), (driving_experience (in years)), (user_occupation (painter, writer, musician, scientist)).

[0127] b3) Automatically fill the third sub-template based on the data filled into the first sub-template.

[0128] The third sub-template is used to generate the SQL statement corresponding to the Chinese query statement. For example, the third sub-template is "SELECT %s FROM %s WHERE %s %s %s (AND %s %s %s)" in template example ①.

[0129] It is understandable that SQL statements are actually translating Chinese query statements into another language. Accordingly, the slots of the third sub-template and the slots of the first sub-template have a pre-established mapping relationship. The mapping relationship includes conversion rules between slot values, etc. For example, the slot after the keyword FROM in the third sub-template is filled with the table name indicated by the data filled in the first sub-template.

[0130] Based on the above, the data to be filled into the corresponding slot in the third sub-template can be determined according to the slot value of the first sub-template.

[0131] The above introduction describes how to quickly construct a large number of rich, usable, and diverse training samples by automatically filling in templates. It's important to understand that the filling of each sub-template within a template can be done sequentially, in parallel, or alternately, as long as the data flow is logical. Furthermore, there can be one or more templates, but generally multiple templates are used.

[0132] The above description uses implementation method B, which generates a large number of training samples in a short time by automatically filling templates.

[0133] In step S410 above, multiple training samples can be obtained by using either implementation method A or B.

[0134] Next, in step S420, the pre-trained large model is fine-tuned using multiple training samples to obtain a fine-tuned large model, which is used to generate the target SQL statement corresponding to the user's input target query statement. It can be understood that the large model in this step refers to the large language model.

[0135] Specifically, for each training sample, the prompt text is processed using a large model to generate corresponding predicted SQL statements. Then, the training loss is calculated based on the predicted answer and the SQL statement in the training sample (the SQL statement in the training sample is correct and used as a label in supervised learning). The training loss is then used to adjust the model parameters in the large model. It should be understood that fine-tuning can adjust the model parameters of local components or some network layers in the large model. For specific implementation details, please refer to relevant existing technologies, which will not be elaborated here.

[0136] During the fine-tuning process described above, the large model can learn from the multiple training samples generated above the relationship between the Chinese semantics hidden in the user's question and the meta-information such as the table name of the database table, as well as the Chinese annotations of the meta-information. This enables the model to support users in asking SQL generation questions in pure Chinese and to provide accurate SQL statements.

[0137] The fine-tuned large model is used to generate the target SQL statement corresponding to the target query statement entered by the user. Figure 5 This diagram illustrates a method for generating SQL statements as disclosed in an embodiment of this specification. The execution entity of this method can be any device, server, platform, or device cluster with computing and processing capabilities. Figure 5 As shown, the method includes the following steps:

[0138] First, in step S510, the target query statement input by the user is received.

[0139] It is understood that the target query is a Chinese query. For example, the target query could be... Figure 3 The document shows "Query payment accounts with a total credit limit of more than 3,000 for Credit Product No. 1".

[0140] Next, in step S520, based on the target query statement, the metadata of the data table it queries, and the Chinese annotations on the metadata, a target prompt text with a predetermined format is constructed.

[0141] Specifically, the target data table can be determined first based on the target query statement. In one embodiment, this can be achieved using matching rules. For example, the target query statement can be segmented into words, and then the resulting words can be matched against a pre-configured set of table information. The target data table to be queried can be located based on the successfully matched table information. In another embodiment, a machine learning model for classifying query statements can be pre-trained, and its output can be the category identifier of the data table (such as the table name), thereby determining the data table to be queried in this step.

[0142] Furthermore, after locating the target data table, the table's metadata and its Chinese annotations can be extracted from the database system based on the table name, thereby constructing a target prompt text with a predetermined format. It should be understood that the description of the predetermined format can be found in the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0143] Then, in step S530, the target prompt text is processed using the fine-tuned large model to generate the target SQL statement corresponding to the target query statement.

[0144] It should be noted that the large model used in this step adopts... Figure 4 The illustrated method is a fine-tuned version of the larger model. For example, the target SQL statement could be... Figure 3 The example shown is "SELECT pay_user_id FROM numberone_credit_v2 WHERE total_amt>3000".

[0145] Based on the above, we can accurately provide the corresponding SQL statement to answer the user's target Chinese query.

[0146] Figure 6 This diagram illustrates a structural schematic of the apparatus for fine-tuning a large model, as disclosed in an embodiment of this specification. Figure 6 As shown, the device 600 includes:

[0147] The training sample generation module 610 is configured to generate training samples, which include prompt text with a predetermined format and corresponding response text. The prompt text includes a Chinese query statement, metadata of the database table queried by the Chinese query statement, and Chinese annotations for the metadata. The response text includes the SQL statement corresponding to the Chinese query statement. The large model fine-tuning module 620 uses multiple training samples to fine-tune a pre-trained large model, obtaining a fine-tuned large model, which is used to generate a target SQL statement corresponding to the user-input target query statement.

[0148] In one embodiment, the metadata includes multiple information items, and the Chinese annotations include annotation items for each information item; the content in the predetermined format prompt text is arranged in the following order: the Chinese query statement, multiple pairs of information items-annotation items.

[0149] In one specific embodiment, the prompt text also includes the task name of the task that generates the SQL statement, and the predetermined format further indicates that the task name should be placed before the Chinese query statement.

[0150] In a more specific embodiment, the training sample generation module 610 is specifically configured to: obtain the Chinese query statement and the corresponding SQL statement; extract the metadata of the database table corresponding to the table name from the database system based on the table name contained in the SQL statement, as well as the Chinese annotation of the metadata; and construct the training sample using the Chinese query statement, the SQL statement, and the extracted metadata and Chinese annotation.

[0151] In one embodiment, the training sample generation module 610 is specifically configured to: automatically fill in the template configured for the training samples based on the database system to generate the training samples.

[0152] In one specific embodiment, the template includes a first sub-template configured for the Chinese query statement, a second sub-template configured for the metadata and Chinese annotations, and a third sub-template configured for the SQL statement; the training sample generation module 610 specifically includes: a first sub-template filling module 611, configured to automatically determine data to be filled into the corresponding slots according to the filling rules configured for each slot of the first sub-template; and other sub-template filling modules 612, configured to automatically fill the second and third sub-templates based on the data filled into the first sub-template and the database system.

[0153] In one example, the first sub-template contains a first slot; the first sub-template filling module 611 is specifically configured to: determine the data under the data category contained in the filling rule of the first slot based on the database system to fill the first slot.

[0154] In a specific example, the first sub-template filling module 611 is further configured to: determine multiple original data under the data category based on the database system; select one of the multiple original data to fill the first slot, or fill the first slot with an alias of one of the original data.

[0155] For example, the data category is a table name class, a column name class, or a field value class with a predetermined data type.

[0156] In another example, the first sub-template includes a second slot, and the configuration information of the template includes multiple candidate slot values ​​configured for the second slot; the first sub-template filling module 611 is specifically configured to: select one of the multiple candidate slot values ​​to fill the second slot according to the filling rules of the second slot.

[0157] In another example, the first sub-template includes a third slot; the first sub-template filling module 611 is specifically configured to: randomly generate data of the data type contained in the filling rules of the third slot, and fill the third slot.

[0158] In another example, the data type is a numeric type or a date type.

[0159] In another example, the first sub-template includes a mandatory part and randomly includes an optional part; the constituent units of the optional part include a fourth slot, a fifth slot, and a sixth slot, and the filling rules of each slot sequentially indicate the Chinese description of the filling logical operator, the operator for the predetermined data type, and the operand for the predetermined data type.

[0160] Furthermore, the device 600 also includes an optional part randomization module 630, configured to randomly determine whether to add the optional part to the first sub-template; if it is determined that an addition is to be made, determine the number of constituent units in the added optional part, and add template content corresponding to the spliced ​​optional part to the third sub-template.

[0161] In another example, the other sub-template filling module 612 is specifically configured as follows: based on the table name indicated by the data to be filled into the first sub-template, extract the metadata of the database table corresponding to the table name from the database system, as well as the Chinese annotation of the metadata, and fill it into the second sub-template; based on the slot mapping relationship between the first sub-template and the third sub-template, determine the data to be filled into the corresponding slot in the third sub-template according to the data to be filled into each slot of the first sub-template.

[0162] Figure 7 This diagram illustrates the structure of the apparatus for generating SQL statements disclosed in embodiments of this specification. Figure 7 As shown, the device 700 includes:

[0163] The query statement receiving module 710 is configured to receive the target query statement input by the user. The prompt text construction module 720 is configured to construct a target prompt text with a predetermined format based on the target query statement, the metadata of the queried data table, and Chinese annotations of the metadata. The SQL statement generation module 730 is configured to utilize... Figure 6The device shown, after fine-tuning, processes the target prompt text and generates a target SQL statement corresponding to the target query statement.

[0164] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 4 or Figure 5 The method described.

[0165] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein executable code is stored in the memory, and when the processor executes the executable code, it implements a combination... Figure 4 or Figure 5 The method described.

[0166] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0167] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for fine-tuning a large model, comprising: Training samples are generated, including prompt text with a predetermined format and corresponding response text. The prompt text includes a Chinese query statement, metadata of the database table queried by the Chinese query statement, and Chinese annotations on the metadata. The response text includes an SQL statement corresponding to the Chinese query statement. The metadata includes table names or column names, and each information item in the metadata and the corresponding annotation item in the Chinese annotation form an information item-annotation item pair. The SQL statement can be directly used to query data in the database. The pre-trained large model is fine-tuned using multiple training samples to obtain a fine-tuned large model, which is used to generate the target SQL statement corresponding to the target query statement input by the user.

2. The method according to claim 1, wherein, The metadata includes multiple information items; the content in the predefined format prompt text is arranged in the following order: The Chinese query statement and multiple pairs of information items - annotation items.

3. The method according to claim 2, wherein, The prompt text also includes the task name for generating the SQL statement, and the predetermined format further indicates that the task name should be placed before the Chinese query statement.

4. The method according to claim 1, wherein, Generate training samples, including: Obtain the Chinese query statement and its corresponding SQL statement; Based on the table name contained in the SQL statement, extract the metadata of the database table corresponding to the table name and the Chinese annotation of the metadata from the database system; The training samples are constructed using the Chinese query statements, SQL statements, extracted metadata, and Chinese annotations.

5. The method according to claim 1, wherein, Generate training samples, including: Based on the database system, the templates configured for the training samples are automatically populated to generate the training samples.

6. The method according to claim 5, wherein, The template includes a first sub-template configured for the Chinese query statement, a second sub-template configured for the metadata and Chinese annotations, and a third sub-template configured for the SQL statement; wherein, the automatic filling of the template configured for the training samples includes: Based on the filling rules configured for each slot of the first sub-template, the data is automatically determined and filled into the corresponding slot; Based on the data filled into the first sub-template and the database system, the second and third sub-templates are automatically filled.

7. The method according to claim 6, wherein, The first sub-template includes a first slot; wherein, based on the filling rules configured for each slot of the first sub-template, data is automatically determined to be filled into the corresponding slot, including: Based on the data category contained in the filling rules of the first slot, the database system determines the data under that data category to fill the first slot.

8. The method according to claim 7, wherein, Based on the database system, data under this data category is determined to fill the first slot, including: Based on the database system, multiple original data points under this data category are determined; Alternatively, one of the multiple original data can be selected to fill the first slot, or an alias of one of the original data can be filled into the first slot.

9. The method according to claim 7 or 8, wherein the data category is a table name class, a column name class, or a field value class with a predetermined data type.

10. The method according to claim 6, wherein, The first sub-template includes a second slot, and the template's configuration information includes multiple candidate slot values ​​configured for the second slot; wherein, based on the filling rules configured for each slot of the first sub-template, data is automatically determined to be filled into the corresponding slot, including: According to the filling rules of the second slot, one of the multiple candidate slot values ​​is selected and filled into the second slot.

11. The method according to claim 6, wherein, The first sub-template includes a third slot; wherein, based on the filling rules configured for each slot of the first sub-template, data is automatically determined to be filled into the corresponding slot, including: Based on the data type contained in the filling rules of the third slot, data of that data type is randomly generated and filled into the third slot.

12. The method according to claim 11, wherein, The data type is either numeric or date.

13. The method according to claim 6, wherein, The first sub-template includes a mandatory part and randomly includes an optional part; the constituent units of the optional part include a fourth slot, a fifth slot and a sixth slot, and their respective filling rules indicate the Chinese description of the filling logical operator, the operator for the predetermined data type, and the operand for the predetermined data type in sequence.

14. The method according to claim 13, wherein, Before automatically determining where to fill data into the corresponding slots based on the filling rules configured for each slot of the first sub-template, the method further includes: Randomly determine whether to add the optional part to the first sub-template; If an addition is determined, the number of constituent units in the optional portion to be added is determined, and template content corresponding to the optional portion to be added to the third sub-template is added.

15. The method according to claim 6, wherein, Based on the data populated into the first sub-template and the database system, the second and third sub-templates are automatically populated, including: Based on the table name indicated by the data filled into the first sub-template, extract the metadata of the database table corresponding to the table name and the Chinese annotation of the metadata from the database system, and fill it into the second sub-template; Based on the slot mapping relationship between the first sub-template and the third sub-template, the data to be filled into the corresponding slot in the third sub-template is determined according to the data filled into each slot of the first sub-template.

16. A method for generating SQL statements, comprising: Receive the target query statement input by the user; Based on the target query statement, the metadata of the data table it queries, and the Chinese annotations on the metadata, a target prompt text with a predetermined format is constructed. The target prompt text is processed using the large model finely tuned by the method described in claim 1 to generate a target SQL statement corresponding to the target query statement.

17. An apparatus for fine-tuning a large model, comprising: The training sample generation module is configured to generate training samples, which include prompt text with a predetermined format and corresponding response text. The prompt text includes a Chinese query statement, metadata of the database table queried by the Chinese query statement, and Chinese annotations on the metadata. The response text includes an SQL statement corresponding to the Chinese query statement. The metadata includes table names or column names, and each information item in the metadata and the corresponding annotation item in the Chinese annotation form an information item-annotation item pair. The SQL statement can be directly used to query data in the database. The large model fine-tuning module is configured to fine-tune the pre-trained large model using multiple training samples to obtain the fine-tuned large model, which is used to generate the target SQL statement corresponding to the target query statement input by the user.

18. An apparatus for generating SQL statements, comprising: The query statement receiving module is configured to receive the target query statement input by the user. The prompt text construction module is configured to construct a target prompt text with a predetermined format based on the target query statement, the metadata of the data table it queries, and the Chinese annotations on the metadata; The SQL statement generation module is configured to process the target prompt text using the large model fine-tuned by the method described in claim 1, and generate a target SQL statement corresponding to the target query statement.

19. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed in the computer, it causes the computer to perform the method of any one of claims 1-16.

20. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-16.

Images