A self-enhancing fine-tuning method and apparatus for NL2SQL large language model

Abstract

This invention relates to a self-enhancing fine-tuning method and apparatus for an NL2SQL large language model. The method includes: 1. confirming a first model and initializing an iteration counter; 2. initializing a seed dataset; 3. cleaning the seed dataset; 4. fine-tuning the first model based on the seed dataset; 5. using the first model to perform data augmentation and low-quality record filtering on the seed dataset, and merging the augmented dataset and the seed dataset to obtain a new seed dataset; 6. if the iteration counter exceeds a threshold, proceeding to step 7; otherwise, incrementing the counter by 1 and returning to step 3; 7. fixing the parameters of the first model after iterative fine-tuning and confirming the end of fine-tuning training. This invention can shorten the training cycle, reduce training costs, and improve training efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a self-enhancing fine-tuning method and apparatus for an NL2SQL large language model. Background Technology

[0002] Large language models (LLMs) can handle various complex language understanding tasks and multiple natural language (NL) instruction tasks. These NL instruction tasks can be further subdivided into two main categories: routine tasks and customized tasks. Routine tasks include text quality assessment, document retrieval, question answering, and text comparison based on NL instructions. Customized tasks include programming instructions / code synthesis based on NL instructions. The NL2SQL large language model is one such model capable of programming instructions / code synthesis based on NL instructions. Besides handling routine tasks, this type of model can also handle customized tasks such as programming instructions / code synthesis, specifically generating corresponding SQL instructions based on input NL instructions.

[0003] Before being deployed, the NL2SQL large language model needs to be pre-trained on a regular large language model dataset. The pre-trained NL2SQL large language model can effectively handle multiple regular tasks and also handle customized tasks (i.e., SQL instruction synthesis tasks). However, the processing quality for customized tasks is not optimal. To improve the processing quality for customized tasks, the model needs to be further fine-tuned after pre-training through instruction fine-tuning.

[0004] When fine-tuning the NL2SQL large language model, it is necessary to collect a large number of natural language command-SQL command pairs to construct a fine-tuning dataset, and then fine-tune the NL2SQL large language model based on the fine-tuning dataset. Currently, the collection / compilation of fine-tuning data is mainly done manually, which has some obvious drawbacks: 1) The collection cycle is long, which leads to a longer training cycle and increased training costs; 2) It is difficult to maintain a high quality level for manually collected / compiled fine-tuning data, which results in insufficient model training and slow performance improvement. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a self-enhancing fine-tuning method, apparatus, electronic device, and computer-readable storage medium for an NL2SQL large language model. This invention first initializes a seed dataset by collecting some natural language instructions—SQL instructions—from a publicly available dataset, and then performs seed cleaning on the seed dataset. Next, it fine-tunes the NL2SQL large language model based on the cleaned seed dataset. Then, it performs data augmentation on the seed dataset based on the fine-tuned NL2SQL large language model and filters out low-quality records from the augmented dataset. Finally, it merges the filtered augmented dataset and the seed dataset to obtain a new seed dataset. If the number of iterations does not reach a specified threshold, the seed dataset is cleaned again, the NL2SQL large language model is fine-tuned again, data augmentation and low-quality record filtering are performed again, and the dataset is reassembled to obtain an updated seed dataset. This process is repeated iteratively until the number of iterations reaches the specified threshold. Therefore, this invention provides a self-enhancing fine-tuning mechanism based on the NL2SQL large language model itself. This mechanism can reduce the workload of manual data collection, thereby shortening the training cycle and reducing training costs. On the other hand, it can improve the quality of fine-tuning data and increase the number of fine-tuning rounds, thereby improving the training level and accelerating the performance improvement.

[0006] To achieve the above objectives, a first aspect of the present invention provides a self-enhancing fine-tuning method for an NL2SQL large language model, the method comprising:

[0007] Step 1: Use the pre-trained NL2SQL large language model as the corresponding first model; and initialize the iteration counter to 1;

[0008] Step 2: Initialize the first seed dataset based on the preset public dataset; the first seed dataset includes multiple first seed records; the first seed record includes a first NL instruction, a first SQL instruction, a first database access interface, and a first query text; the first NL instruction is a natural language instruction, and the first SQL instruction is an SQL instruction.

[0009] Step 3: Perform seed cleaning on the first seed dataset;

[0010] Step 4: Fine-tune the first model based on the first seed dataset;

[0011] Step 5: After this round of fine-tuning, the first model is used to perform data augmentation on the first seed dataset to obtain the corresponding first augmented dataset; and the first model is used to perform low-quality record filtering on the first augmented dataset; and the filtered first augmented dataset and the first seed dataset are merged, and the merged dataset is used as the new first seed dataset; the first augmented dataset includes multiple first augmented records; the first augmented record includes a second NL instruction, a second SQL instruction, a second database access interface, and a second query text;

[0012] Step 6: Check whether the iteration counter exceeds the preset iteration number threshold; if it does, proceed to step 7; if it does not, increment the iteration counter by 1 and return to step 3.

[0013] Step 7: Solidify the parameters of the first model after iterative fine-tuning and confirm that the model fine-tuning training is complete.

[0014] Preferably, the public dataset includes at least some or all of the Spider dataset, CSpider dataset, SParC dataset, CoSQL dataset, and DuSQL dataset;

[0015] Each dataset in the publicly available dataset consists of multiple natural language instruction-SQL instruction pairs and multiple reference databases; the natural language instruction-SQL instruction includes source natural language instructions and source SQL instructions; each natural language instruction-SQL instruction pair corresponds to one of the reference databases; each source natural language instruction is an instruction text used to prompt the large language model to generate an SQL instruction based on a segment of natural language description text in the instruction, and the generated SQL instruction matches the corresponding source SQL instruction; each source SQL instruction is an SQL query instruction that can retrieve the corresponding query text from the corresponding reference database.

[0016] Preferably, the initialization of the first seed dataset based on a preset public dataset specifically includes:

[0017] Create a mirror database with the same structure and content for each of the reference databases of the public dataset, and denote it as the corresponding first mirror database;

[0018] Based on a preset instruction pair collection principle, multiple natural language instruction-SQL instruction pairs are collected from the public dataset; and each collected natural language instruction-SQL instruction pair is used as the corresponding first collection instruction pair.

[0019] A round of traversal is performed on all the obtained first acquisition instruction pairs; during the traversal, the currently traversed first acquisition instruction pair is taken as the corresponding current instruction pair, and the reference database and the first mirror database corresponding to the current instruction pair are taken as the corresponding first and second databases; the access interfaces of the first and second databases are taken as the corresponding first and second interfaces; the source natural language instruction and the source SQL instruction of the current instruction pair are taken as the corresponding first NL instruction and the first SQL instruction; the query text obtained by executing the first SQL instruction on the first database through the first interface is taken as the corresponding first text; the query text obtained by executing the first SQL instruction on the second database through the second interface is taken as the corresponding second text; when the first and second texts are matched, the second interface is taken as the corresponding first database access interface, and the second text is taken as the corresponding first query text, and the first NL instruction, the first SQL instruction, the first database access interface, and the first query text obtained in this round are combined to form a corresponding first seed record; and at the end of this round of traversal, all the obtained first seed records are combined to form the corresponding first seed dataset.

[0020] Preferably, the seed cleaning process for the first seed dataset specifically includes:

[0021] A traversal is performed on all first seed records in the first seed dataset. During the traversal, the currently traversed first seed record is taken as the corresponding current seed record. The first NL instruction and the first SQL instruction of the current seed record are taken as the corresponding current NL instruction and current SQL instruction. The syntax correctness of the current SQL instruction is identified to obtain the corresponding first identification result. The complexity of the current SQL instruction is identified to obtain the corresponding second identification result. The natural language semantic clarity of the current NL instruction is identified to obtain the corresponding third identification result. The first, second, and third identification results are then evaluated. If the first identification result is a syntax error, the second identification result is simple, or the third identification result is semantically ambiguous, the current seed record is marked as a preliminary screening failure record. After this round of traversal, all preliminary screening failure records in the first seed dataset are deleted. The first identification result includes syntax correctness and syntax error; the second identification result includes simple, moderate, and complex; and the third identification result includes semantically ambiguous and semantically clear.

[0022] Based on preset text embedding encoding rules, the first SQL instruction of each remaining first seed record in the first seed dataset is processed by text embedding encoding to obtain the corresponding first text encoding vector; the vector similarity between any two first text encoding vectors is calculated to obtain the corresponding first similarity; and all first text encoding vectors are clustered based on a preset similarity threshold to obtain multiple first class vector sets; wherein, each first class vector set consists of one or more first text encoding vectors; when the number of first text encoding vectors in the first class vector set is 1, the first similarity of the unique first text encoding vector in the current set with any other first text encoding vector is less than or equal to the similarity threshold; when the number of first text encoding vectors in the first class vector set is greater than 1, the first similarity between any two first text encoding vectors in the current set is greater than the similarity threshold; the text embedding encoding rules include at least BOW encoding rules, n-gram encoding rules, and Word2Vec encoding rules.

[0023] A traversal is performed on the first class vector set for all first text encoding vectors with a number greater than 1. During the traversal, the first class vector set currently being traversed is taken as the corresponding current vector set. One first text encoding vector is randomly selected from the current vector set as the corresponding retained vector, and all other first text encoding vectors except the retained vector are marked as duplicate vectors. At the end of this traversal, the first seed record corresponding to each duplicate vector in the first seed dataset is deleted.

[0024] Preferably, the step of using the first model to perform data augmentation processing on the first seed dataset to obtain the corresponding first augmented dataset specifically includes:

[0025] The first seed record in the first seed dataset is traversed; and during the traversal, the first seed record currently being traversed is taken as the corresponding current seed record; and the first SQL instruction of the current seed record is taken as the corresponding current SQL instruction; and a corresponding first augmented record is created based on the current SQL instruction and the first model; and at the end of the traversal, the first augmented dataset is composed of all the obtained first augmented records.

[0026] Furthermore, the step of creating a corresponding first enhanced record based on the current SQL command and the first model specifically includes:

[0027] The current SQL instruction is substituted into a preset concept analysis instruction template to replace the instruction text X in the template, thereby obtaining the corresponding SQL concept analysis instruction; the SQL concept analysis instruction is then input into the first model for processing, and the SQL concept analysis text generated by the first model is received as the corresponding current analysis text; the concept analysis instruction template is a natural language instruction template used to prompt the first model to analyze the SQL concept of the instruction text X; the instruction text X is a replaceable text variable;

[0028] The current analysis text is then input into a preset natural language description instruction template to replace the SQL concept analysis text Y in the template, thereby obtaining the corresponding natural language description instruction. This natural language description instruction is then input into the first model for processing, and the SQL instruction natural language description text generated by the first model is received as the corresponding current description text. The natural language description instruction template is a template used to prompt the first model to generate the corresponding SQL instruction natural language description text based on the SQL concept analysis text Y. The SQL concept analysis text Y is a replaceable text variable.

[0029] The current description text is then substituted into a preset NL2SQL instruction template to replace the natural language description text Z in the template, thereby obtaining the corresponding NL2SQL instruction. The NL2SQL instruction template is a natural language instruction template used to prompt the first model to generate a corresponding SQL instruction text based on the natural language description text Z. The natural language description text Z is a replaceable text variable.

[0030] The NL2SQL instruction corresponding to the current seed record is used as the corresponding second NL instruction, and the first SQL instruction, the first database access interface, and the first query text of the current seed record are used as the corresponding second SQL instruction, the second database access interface, and the second query text; and the second NL instruction, the second SQL instruction, the second database access interface, and the second query text corresponding to the current seed record are used to form a corresponding first enhanced record.

[0031] Preferably, the step of using the first model to filter low-quality records in the first augmented dataset specifically includes:

[0032] A traversal is performed on all first augmented records in the first augmented dataset. During the traversal, the currently traversed first augmented record is taken as the corresponding current augmented record. The second NL instruction, the second database access interface, and the second query text of the current augmented record are extracted as the corresponding current NL instruction, current access interface, and current label text. The first mirror database corresponding to the current access interface is taken as the corresponding current mirror database. The current NL instruction is input into the first model for processing, and the SQL instruction text generated by the first model is received as the corresponding current SQL instruction. The query text obtained by executing the current SQL instruction on the corresponding current mirror database through the current access interface is taken as the corresponding current query text. The current query text and the current label text are checked for matching. If they do not match, the current augmented record is marked as a low-quality record. At the end of this traversal, all low-quality records in the first augmented dataset are deleted.

[0033] A second aspect of the present invention provides an apparatus for implementing the self-enhancing fine-tuning method for the NL2SQL large language model described in the first aspect above. The apparatus includes: a first initialization module, a second initialization module, a data cleaning module, a model fine-tuning module, a data self-enhancing module, an iterative judgment module, and a parameter solidification module.

[0034] The first initialization module is used to take the pre-trained NL2SQL large language model as the corresponding first model; and initialize the iteration counter to 1;

[0035] The second initialization module is used to initialize a first seed dataset based on a preset public dataset; the first seed dataset includes multiple first seed records; the first seed record includes a first NL instruction, a first SQL instruction, a first database access interface, and a first query text; the first NL instruction is a natural language instruction, and the first SQL instruction is an SQL instruction;

[0036] The data cleaning module is used to perform seed cleaning processing on the first seed dataset.

[0037] The model fine-tuning module is used to perform one round of fine-tuning on the first model based on the first seed dataset;

[0038] The data self-enhancement module is used to perform data augmentation processing on the first seed dataset using the first model after this round of fine-tuning to obtain the corresponding first augmented dataset; and to perform low-quality record filtering processing on the first augmented dataset using the first model; and to perform dataset merging processing on the filtered first augmented dataset and the first seed dataset and use the resulting merged dataset as the new first seed dataset; the first augmented dataset includes multiple first augmented records; the first augmented record includes a second NL instruction, a second SQL instruction, a second database access interface, and a second query text;

[0039] The iteration judgment module is used to identify whether the iteration counter exceeds a preset iteration number threshold; if it exceeds, a training end command is sent to the parameter solidification module; if it does not exceed, the iteration counter is incremented by 1 and a new first seed dataset is sent to the data cleaning module.

[0040] The parameter fixing module is used to fix the parameters of the first model that has completed iterative fine-tuning when the training end instruction is received, and to confirm that the training of this model fine-tuning is over.

[0041] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0042] The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above;

[0043] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0044] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0045] This invention provides a self-enhancing fine-tuning method, apparatus, electronic device, and computer-readable storage medium for an NL2SQL large language model. As described above, this invention first initializes a seed dataset by collecting some natural language instructions (SQL instructions) from a public dataset and performs seed cleaning on the seed dataset. Then, based on the cleaned seed dataset, a round of fine-tuning of the NL2SQL large language model is performed. Next, based on the finely tuned NL2SQL large language model, data augmentation processing is performed on the seed dataset, and low-quality records are filtered from the augmented dataset. The filtered augmented dataset and the seed dataset are then merged to obtain a new seed dataset. If the number of iterations does not reach a specified threshold, the seed dataset is cleaned again, the NL2SQL large language model is fine-tuned again, data augmentation and low-quality record filtering are performed on the seed dataset again, and the dataset is reassembled to obtain an updated seed dataset. This process is repeated iteratively until the number of iterations reaches the specified threshold. Therefore, the embodiments of the present invention provide a self-enhancing fine-tuning processing mechanism based on the NL2SQL large language model itself. This processing mechanism reduces the workload of manual data collection, shortens the training cycle, and reduces training costs. On the other hand, it improves the quality of fine-tuning data, increases the number of fine-tuning rounds, improves the training level, and accelerates the performance improvement speed. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of a self-enhancing fine-tuning method for an NL2SQL large language model provided in Embodiment 1 of the present invention;

[0047] Figure 2 This is a module structure diagram of a self-enhancing fine-tuning device for an NL2SQL large language model provided in Embodiment 2 of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0050] Embodiment 1 of this invention provides a self-enhancing fine-tuning method for the NL2SQL large language model, such as... Figure 1 The diagram illustrates a self-enhancing fine-tuning method for an NL2SQL large language model provided in Embodiment 1 of the present invention. This method mainly includes the following steps:

[0051] Step 1: Use the pre-trained NL2SQL large language model as the corresponding first model; and initialize the iteration counter to 1.

[0052] Here, the NL2SQL large language model mentioned in this embodiment of the invention refers to any type of large language model that can generate corresponding SQL commands based on input natural language commands; common examples include Llama, ChatGLM, Wenxin Yiyan, Alpaca, Vicuna, WizardLM, Falcon, StarCoder2, etc. A pre-trained NL2SQL large language model can effectively handle various routine tasks, such as text quality assessment based on natural language commands, document retrieval based on natural language commands, question answering based on natural language commands, and text comparison based on natural language commands. Of course, it can also handle customized tasks (SQL command synthesis tasks), but the quality of customized task processing by the NL2SQL large language model in the pre-training stage can be further improved, i.e., through fine-tuning training.

[0053] Step 2: Initialize the first seed dataset based on the preset public dataset;

[0054] The first seed dataset includes multiple first seed records; each first seed record includes a first NL instruction, a first SQL instruction, a first database access interface, and a first query text; the first NL instruction is a natural language instruction, and the first SQL instruction is an SQL instruction.

[0055] Specifically, this includes: Step 21, creating a mirror database with the same structure and content for each reference database of the public dataset, denoted as the corresponding first mirror database;

[0056] Here, the publicly available datasets in this embodiment of the invention include at least some or all of the Spider dataset, CSpider dataset, SParC dataset, CoSQL dataset, and DuSQL dataset;

[0057] Each dataset in the disclosed datasets of this invention consists of multiple natural language instruction-SQL instruction pairs and multiple reference databases; wherein, the natural language instruction-SQL instruction includes source natural language instructions and source SQL instructions; each natural language instruction-SQL instruction pair corresponds to a reference database; each source natural language instruction is an instruction text used to prompt the large language model to generate an SQL instruction based on a segment of natural language description text in the instruction, and the generated SQL instruction matches the corresponding source SQL instruction; each source SQL instruction is an SQL query instruction that can retrieve the corresponding query text from the corresponding reference database;

[0058] Here, the reference databases in the above public datasets are all remote databases with low access efficiency. To improve the efficiency of fine-tuning training, a local mirror database is specifically created for each reference database. This way, the training efficiency will not be affected by the database access efficiency during the training process.

[0059] Step 22: Based on the preset instruction pair collection principle, collect multiple natural language instruction-SQL instruction pairs from the public dataset; and use each collected natural language instruction-SQL instruction pair as the corresponding first collection instruction pair;

[0060] Here, the instruction specifies a pre-defined data collection principle that can be customized based on application requirements. This data collection principle typically sets high requirements for the quality level of the collected data and also provides clear requirements for the data's diversity characteristics.

[0061] Step 23: Perform a round of traversal on all obtained first acquisition command pairs; during the traversal, the currently traversed first acquisition command pair is taken as the corresponding current command pair, and the reference database and the first mirror database corresponding to the current command pair are taken as the corresponding first and second databases; the access interfaces of the first and second databases are taken as the corresponding first and second interfaces; the source natural language command and source SQL command of the current command pair are taken as the corresponding first NL command and first SQL command; the query text obtained by executing the first SQL command on the first database through the first interface is taken as the corresponding first text; the query text obtained by executing the first SQL command on the second database through the second interface is taken as the corresponding second text; when matching the first and second texts, the second interface is taken as the corresponding first database access interface, and the second text is taken as the corresponding first query text, and the first NL command, first SQL command, first database access interface and first query text obtained in this round are combined to form a corresponding first seed record; and at the end of this round of traversal, all obtained first seed records are combined to form the corresponding first seed dataset.

[0062] It should be noted that the reason for matching and comparing the first and second texts in this step is to ensure a complete match between the mirror database and the corresponding reference database. If the first and second texts corresponding to any first collection command do not match during this round of traversal, the entire fine-tuning process will be stopped and a warning message indicating a mismatch between the mirror and reference databases will be given.

[0063] Step 3: Perform seed cleaning on the first seed dataset;

[0064] Specifically, this includes: Step 31, performing a round of traversal on all first seed records in the first seed dataset; during the traversal, the first seed record currently being traversed is taken as the corresponding current seed record; the first NL instruction and the first SQL instruction of the current seed record are taken as the corresponding current NL instruction and the current SQL instruction; the syntax correctness of the current SQL instruction is identified to obtain the corresponding first identification result; the instruction complexity of the current SQL instruction is identified to obtain the corresponding second identification result; the natural language semantic clarity of the current NL instruction is identified to obtain the corresponding third identification result; and the first, second, and third identification results are identified. If the first identification result is a syntax error, the second identification result is simple, or the third identification result is semantically ambiguous, the current seed record is marked as a record that fails the initial screening; and after this round of traversal, all records that fail the initial screening in the first seed dataset are deleted.

[0065] The first identification result includes grammatically correct and grammatically incorrect; the second identification result includes simple, moderate and complex; and the third identification result includes semantically ambiguous and semantically clear.

[0066] It should be noted that there are several methods for identifying the syntax correctness of the current SQL command in this step. One method is based on predetermined rules, which involves pre-setting a syntax checking rule and checking the syntax correctness of the current SQL command based on this rule. If the syntax check rule is met, the first identification result is set as syntax correct; otherwise, the first identification result is set as syntax incorrect. Another method is based on a classification prediction model, which involves pre-constructing a binary classification prediction model A based on a non-linear binary classifier model structure and setting... The binary classification prediction model A is set to input SQL command text and output a binary classification vector (composed of two classification probabilities: the probability of grammatically correct type prediction and the probability of grammatically incorrect type prediction). The binary classification prediction model A is trained using a supervised training method. Based on the trained binary classification prediction model A, the syntax of the current SQL command is binary classified to obtain the corresponding binary classification vector. If the probability value of the grammatically correct type prediction in the binary classification vector is too large, the first recognition result is set as grammatically correct; if the probability value of the grammatically incorrect type prediction in the binary classification vector is too large, the first recognition result is set as grammatically incorrect.

[0067] It should also be noted that there are several ways to identify the complexity of the current SQL command in this step. One method is based on predetermined rules, which involves pre-setting a statistical rule for distinguishing between three levels of complexity (simple, moderate, and complex), confirming the complexity type (simple, moderate, and complex) of the current SQL command based on this rule, and setting a corresponding second identification result based on the confirmed complexity type. Another method is based on a classification prediction model, which involves pre-constructing a three-classification prediction model B based on a non-linear three-classifier model structure, and setting the model input of the three-classification prediction model B as SQL. The instruction text is output as a three-class vector (composed of three classification probabilities: simple type prediction probability, moderate type prediction probability, and complex type prediction probability). The three-class prediction model B is trained using a supervised training method. Based on the trained three-class prediction model B, the instruction complexity of the current SQL instruction is predicted to obtain the corresponding three-class vector. If the probability value of the simple type prediction probability in the three-class vector is too large, the second recognition result is set to simple; if the probability value of the moderate type prediction probability in the three-class vector is too large, the second recognition result is set to moderate; if the probability value of the complex type prediction probability in the three-class vector is too large, the second recognition result is set to complex.

[0068] It should also be noted that there are multiple methods for identifying the natural language semantic clarity of the current NL instruction in this step. One method is based on predetermined semantic / syntactic templates, that is, a series of semantic / syntactic templates are pre-set, and the semantic clarity of the current NL instruction is evaluated based on all semantic / syntactic templates to obtain the corresponding evaluation score. If the evaluation score meets the preset semantic fuzziness score range, the corresponding third recognition result is set as semantically fuzzy; if the evaluation score meets the preset semantic clarity score range, the corresponding third recognition result is set as semantically clear. Another method is based on a classification prediction model, that is, a non-linear binary classifier model is pre-set... Construct a binary classification prediction model C, and set the input of the binary classification prediction model C to NL instruction text and the output to a binary classification vector (composed of two classification probabilities: the prediction probability of semantically ambiguous type and the prediction probability of semantically clear type). Use supervised training to train the binary classification prediction model C, and perform binary classification prediction on the semantic clarity of the current NL instruction based on the trained binary classification prediction model C to obtain the corresponding binary classification vector. If the probability value of the semantically ambiguous type prediction in the binary classification vector is too large, the third recognition result is set to semantically ambiguous; if the probability value of the semantically clear type prediction in the binary classification vector is too large, the third recognition result is set to semantically clear.

[0069] Step 32: Based on the preset text embedding encoding rules, perform text embedding encoding on the first SQL instruction of each remaining first seed record in the first seed dataset to obtain the corresponding first text encoding vector; calculate the vector similarity between any two first text encoding vectors to obtain the corresponding first similarity; and cluster all first text encoding vectors based on the preset similarity threshold to obtain multiple first class vector sets.

[0070] Each first-class vector set consists of one or more first-text encoded vectors; when the number of first-text encoded vectors in the first-class vector set is 1, the first similarity between the unique first-text encoded vector in the current set and any other first-text encoded vector is less than or equal to the similarity threshold; when the number of first-text encoded vectors in the first-class vector set is greater than 1, the first similarity between any two first-text encoded vectors in the current set is greater than the similarity threshold; the text embedding encoding rules include at least BOW encoding rules, n-gram encoding rules, and Word2Vec encoding rules;

[0071] Here, in this embodiment of the invention, the cosine similarity algorithm is used by default to calculate vector similarity. Of course, other similarity algorithms can also be used according to application requirements. The similarity threshold is a pre-set threshold parameter, which is a threshold with a relatively large value. The current step uses similarity as a clustering reference to cluster highly similar first text encoding vectors into one class so that subsequent steps can perform deduplication.

[0072] Step 33: Perform a traversal of the first class vector set where the number of first text encoding vectors is greater than 1. During the traversal, take the first class vector set currently being traversed as the corresponding current vector set. Select any first text encoding vector from the current vector set as the corresponding retained vector, and mark all other first text encoding vectors except the retained vector as duplicate vectors. At the end of this traversal, delete the first seed record corresponding to each duplicate vector in the first seed dataset.

[0073] Here, the current step is actually to deduplicate seed records that are highly similar to the SQL commands.

[0074] Step 4: Fine-tune the first model based on the first seed dataset.

[0075] Here, in this embodiment of the invention, the first seed dataset is used to perform conventional weakly supervised fine-tuning training on the first model; the weakly supervised fine-tuning training process for any type of NL2SQL large language model can be obtained by querying publicly available model literature and fine-tuning training literature, and will not be repeated here.

[0076] Step 5: After this round of fine-tuning, the first model is used to perform data augmentation on the first seed dataset to obtain the corresponding first augmented dataset; the first model is used to perform low-quality record filtering on the first augmented dataset; and the first augmented dataset and the first seed dataset after filtering are merged and the merged dataset is used as the new first seed dataset.

[0077] Here, step 5 is actually using the NL2SQL large language model that has completed one round of fine-tuning to generate a new batch of training datasets, namely the first augmented dataset, and using the first augmented dataset + the first seed dataset to increase the total amount of training data for the next round of fine-tuning training and improve the model's full training degree.

[0078] Specifically, it includes: Step 51, after this round of fine-tuning, using the first model to perform data augmentation on the first seed dataset to obtain the corresponding first augmented dataset;

[0079] The first augmented dataset includes multiple first augmented records; each first augmented record includes a second NL instruction, a second SQL instruction, a second database access interface, and a second query text.

[0080] Specifically, this includes: traversing all first seed records in the first seed dataset; during traversal, using the currently traversed first seed record as the corresponding current seed record; using the first SQL instruction of the current seed record as the corresponding current SQL instruction; creating a corresponding first augmented record based on the current SQL instruction and the first model; and at the end of the traversal, using all the obtained first augmented records to form the corresponding first augmented dataset.

[0081] Specifically, creating a corresponding first enhanced record based on the current SQL command and the first model includes:

[0082] Step A1: Substitute the current SQL instruction into the preset concept analysis instruction template to replace the instruction text X in the template to obtain the corresponding SQL concept analysis instruction; input the SQL concept analysis instruction into the first model for processing, and receive the SQL concept analysis text generated by the first model as the corresponding current analysis text;

[0083] Among them, the concept analysis instruction template is a natural language instruction template used to prompt the first model to analyze the SQL concept of instruction text X; instruction text X is a replaceable text variable;

[0084] For example, suppose the concept analysis instruction template is "Analyze the SQL instruction: instruction text X and give the core SQL concept";

[0085] It is known that the current SQL command is:

[0086] SELECT first_name, last_name, email";

[0087] Therefore, the SQL concept analysis instruction obtained by substituting the current SQL instruction into the concept analysis instruction template and replacing the instruction text X in the template is:

[0088] For SQL commands:

[0089] SELECT first_name, last_name, email

[0090] "Analyze and present the core SQL concepts";

[0091] Given that the first model is "Wenxin Yiyan", inputting the SQL concept analysis command into "Wenxin Yiyan" will result in the SQL concept analysis text, which is the current analysis text.

[0092] SELECT statement: This is one of the most basic statements in SQL, used to select data from a database; in this query, SELECT is followed by three column names: first_name, last_name, and email, which means that we only query data in these columns;

[0093] Step A2: Substitute the current analysis text into the preset natural language description instruction template to replace the SQL concept analysis text Y in the template to obtain the corresponding natural language description instruction; input the natural language description instruction into the first model for processing, and receive the SQL instruction natural language description text generated by the first model as the corresponding current description text;

[0094] Among them, the natural language description instruction template is a natural language instruction template used to prompt the first model to generate the corresponding SQL instruction natural language description text based on the SQL concept analysis text Y; the SQL concept analysis text Y is a replaceable text variable;

[0095] For example, suppose the natural language description instruction template is "Based on the following SQL concept analysis results: SQL concept analysis text Y, generate the corresponding SQL instruction natural language description text";

[0096] Therefore, by substituting the current analysis text into the natural language description instruction template and replacing the SQL concept analysis text Y in the template, the resulting natural language description instruction would be: "Based on the following SQL concept analysis results:

[0097] The SELECT statement: This is one of the most basic statements in SQL, used to select data from a database. In this query, SELECT is followed by three column names: first_name, last_name, and email, meaning we only query data from these columns.

[0098] Generate the corresponding natural language description text of the SQL commands.

[0099] Given that the first model is Wenxin Yiyan, inputting natural language description commands into Wenxin Yiyan yields the following SQL command natural language description text, which is the current description text:

[0100] We are using the SELECT statement in SQL to select specific data from a database. Specifically, we are only interested in three columns: first_name (first name), last_name (last name), and email (email address). Therefore, we explicitly specify these three column names in the SELECT statement to ensure that the query results contain only data from these columns.

[0101] Step A3, and substitute the current description text into the preset NL2SQL instruction template to replace the natural language description text Z in the template to obtain the corresponding NL2SQL instruction;

[0102] The NL2SQL instruction template is a natural language instruction template used to prompt the first model to generate a corresponding SQL instruction text based on the natural language description text Z; the natural language description text Z is a replaceable text variable.

[0103] For example, suppose the NL2SQL instruction template is "Generate a corresponding SQL instruction based on the following natural language description text: natural language description text Z";

[0104] Therefore, the NL2SQL instruction obtained by substituting the current description text into the NL2SQL instruction template and replacing the natural language description text Z in the template is: "Based on the following natural language description text:

[0105] We are using the SELECT statement in SQL to select specific data from a database. Specifically, we are only interested in three columns: first_name (first name), last_name (last name), and email (email address). Therefore, we explicitly specify these three column names in the SELECT statement to ensure that the query results contain only data from these columns.

[0106] Generate a corresponding SQL command;

[0107] Step A4, and take the NL2SQL instruction corresponding to the current seed record as the corresponding second NL instruction, and take the first SQL instruction, first database access interface and first query text of the current seed record as the corresponding second SQL instruction, second database access interface and second query text; and take the second NL instruction, second SQL instruction, second database access interface and second query text corresponding to the current seed record as a corresponding first enhanced record;

[0108] Step 52, and use the first model to filter low-quality records in the first augmented dataset;

[0109] Specifically, this includes: performing a round of traversal on all first augmentation records in the first augmentation dataset; during the traversal, taking the currently traversed first augmentation record as the corresponding current augmentation record; extracting the second NL instruction, second database access interface, and second query text of the current augmentation record as the corresponding current NL instruction, current access interface, and current label text; taking the first mirror database corresponding to the current access interface as the corresponding current mirror database; inputting the current NL instruction into the first model for processing, and receiving the SQL instruction text generated by the first model as the corresponding current SQL instruction; taking the query text obtained by executing the current SQL instruction on the corresponding current mirror database through the current access interface as the corresponding current query text; identifying whether the current query text and the current label text match, and if they do not match, marking the current augmentation record as a low-quality record; and deleting all low-quality records in the first augmentation dataset at the end of this round of traversal.

[0110] Here, the current step is actually to filter out the augmentation records with poor instruction quality of NL instructions in the first augmentation dataset, thereby improving the overall data quality of the first augmentation dataset;

[0111] Step 53 involves merging the first enhanced dataset and the first seed dataset after filtering, and using the resulting merged dataset as the new first seed dataset.

[0112] Here, the merged dataset, which is the new first seed dataset, is the training dataset needed for the next round of fine-tuning training.

[0113] Step 6: Check whether the iteration counter exceeds the preset iteration number threshold; if it does, proceed to step 7; if it does not, increment the iteration counter by 1 and return to step 3.

[0114] Here, the iteration count threshold is a pre-set threshold parameter. In this embodiment of the invention, when the iteration counter does not exceed the iteration count threshold, i.e., the number of iterations does not exceed the specified threshold, the default is to return to step 3 to start a new round of seed cleaning, fine-tuning, data augmentation and other operations. However, if you want to further shorten the processing time and improve the processing efficiency, you can also set it to return to step 4.

[0115] Step 7: Solidify the parameters of the first model that has completed iterative fine-tuning, and confirm that the training of this model fine-tuning is complete.

[0116] Figure 2 This is a module structure diagram of a self-enhancing fine-tuning device for an NL2SQL large language model provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiments, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiments. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 2 As shown, the device includes: a first initialization module 201, a second initialization module 202, a data cleaning module 203, a model fine-tuning module 204, a data self-enhancement module 205, an iterative judgment module 206, and a parameter solidification module 207.

[0117] The first initialization module 201 is used to take the pre-trained NL2SQL large language model as the corresponding first model and initialize the iteration counter to 1.

[0118] The second initialization module 202 is used to initialize a first seed dataset based on a preset public dataset; the first seed dataset includes multiple first seed records; the first seed record includes a first NL instruction, a first SQL instruction, a first database access interface, and a first query text; the first NL instruction is a natural language instruction, and the first SQL instruction is an SQL instruction.

[0119] The data cleaning module 203 is used to perform seed cleaning processing on the first seed dataset.

[0120] The model fine-tuning module 204 is used to perform one round of fine-tuning on the first model based on the first seed dataset.

[0121] The data self-enhancement module 205 is used to perform data augmentation processing on the first seed dataset using the first model after the current round of fine-tuning to obtain the corresponding first augmented dataset; and to perform low-quality record filtering processing on the first augmented dataset using the first model; and to perform dataset merging processing on the filtered first augmented dataset and the first seed dataset and use the resulting merged dataset as the new first seed dataset; the first augmented dataset includes multiple first augmented records; the first augmented record includes a second NL instruction, a second SQL instruction, a second database access interface, and a second query text.

[0122] The iteration judgment module 206 is used to identify whether the iteration counter exceeds the preset iteration number threshold; if it exceeds, it sends a training end command to the parameter solidification module 207; if it does not exceed, it increments the iteration counter by 1 and sends a new first seed dataset to the data cleaning module 203.

[0123] The parameter fixing module 207 is used to fix the parameters of the first model that has completed iterative fine-tuning when the training end instruction is received, and to confirm that the training of this model fine-tuning is over.

[0124] The self-enhancing fine-tuning device for the NL2SQL large language model provided in this embodiment of the invention can execute the method steps in the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0125] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the first initialization module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0126] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0127] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0128] Figure 3 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 3As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0129] exist Figure 3 The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include Non-Volatile Memory, such as at least one disk storage device.

[0130] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0131] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0132] This invention also provides a chip for executing instructions, which is used to perform the processing steps described in the foregoing method embodiments.

[0133] This invention provides a self-enhancing fine-tuning method, apparatus, electronic device, and computer-readable storage medium for an NL2SQL large language model. As described above, this invention first initializes a seed dataset by collecting some natural language instructions (SQL instructions) from a public dataset and performs seed cleaning on the seed dataset. Then, based on the cleaned seed dataset, a round of fine-tuning of the NL2SQL large language model is performed. Next, based on the finely tuned NL2SQL large language model, data augmentation processing is performed on the seed dataset, and low-quality records are filtered from the augmented dataset. The filtered augmented dataset and the seed dataset are then merged to obtain a new seed dataset. If the number of iterations does not reach a specified threshold, the seed dataset is cleaned again, the NL2SQL large language model is fine-tuned again, data augmentation and low-quality record filtering are performed on the seed dataset again, and the dataset is reassembled to obtain an updated seed dataset. This process is repeated iteratively until the number of iterations reaches the specified threshold. Therefore, the embodiments of the present invention provide a self-enhancing fine-tuning processing mechanism based on the NL2SQL large language model itself. This processing mechanism reduces the workload of manual data collection, shortens the training cycle, and reduces training costs. On the other hand, it improves the quality of fine-tuning data, increases the number of fine-tuning rounds, improves the training level, and accelerates the performance improvement speed.

[0134] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0135] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0136] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A self-enhancing fine-tuning method for an NL2SQL large language model, characterized in that, The method includes: Step 1: Use the pre-trained NL2SQL large language model as the corresponding first model; and initialize the iteration counter to 1; Step 2: Initialize the first seed dataset based on the preset public dataset; the first seed dataset includes multiple first seed records; the first seed record includes a first NL instruction, a first SQL instruction, a first database access interface, and a first query text; the first NL instruction is a natural language instruction, and the first SQL instruction is an SQL instruction. Step 3: Perform seed cleaning on the first seed dataset; Step 4: Fine-tune the first model based on the first seed dataset; Step 5: After this round of fine-tuning, the first model is used to perform data augmentation on the first seed dataset to obtain the corresponding first augmented dataset; and the first model is used to perform low-quality record filtering on the first augmented dataset; and the filtered first augmented dataset and the first seed dataset are merged, and the merged dataset is used as the new first seed dataset; the first augmented dataset includes multiple first augmented records; the first augmented record includes a second NL instruction, a second SQL instruction, a second database access interface, and a second query text; Step 6: Check whether the iteration counter exceeds the preset iteration number threshold; if it does, proceed to step 7; if it does not, increment the iteration counter by 1 and return to step 3. Step 7: Solidify the parameters of the first model after iterative fine-tuning and confirm the end of this model fine-tuning training. Specifically, the step of using the first model to perform data augmentation on the first seed dataset to obtain the corresponding first augmented dataset includes: The first seed record in the first seed dataset is traversed; and during the traversal, the first seed record currently being traversed is taken as the corresponding current seed record; and the first SQL instruction of the current seed record is taken as the corresponding current SQL instruction; and a corresponding first augmented record is created based on the current SQL instruction and the first model; and at the end of the traversal, the first augmented dataset is composed of all the obtained first augmented records. The step of creating a corresponding first enhanced record based on the current SQL command and the first model specifically includes: The current SQL instruction is substituted into a preset concept analysis instruction template to replace the instruction text X in the template, thereby obtaining the corresponding SQL concept analysis instruction; the SQL concept analysis instruction is then input into the first model for processing, and the SQL concept analysis text generated by the first model is received as the corresponding current analysis text; the concept analysis instruction template is a natural language instruction template used to prompt the first model to analyze the SQL concept of the instruction text X; the instruction text X is a replaceable text variable; The current analysis text is then input into a preset natural language description instruction template to replace the SQL concept analysis text Y in the template, thereby obtaining the corresponding natural language description instruction. This natural language description instruction is then input into the first model for processing, and the SQL instruction natural language description text generated by the first model is received as the corresponding current description text. The natural language description instruction template is a template used to prompt the first model to generate the corresponding SQL instruction natural language description text based on the SQL concept analysis text Y. The SQL concept analysis text Y is a replaceable text variable. The current description text is then substituted into a preset NL2SQL instruction template to replace the natural language description text Z in the template, thereby obtaining the corresponding NL2SQL instruction. The NL2SQL instruction template is a natural language instruction template used to prompt the first model to generate a corresponding SQL instruction text based on the natural language description text Z. The natural language description text Z is a replaceable text variable. The NL2SQL instruction corresponding to the current seed record is used as the corresponding second NL instruction, and the first SQL instruction, the first database access interface, and the first query text of the current seed record are used as the corresponding second SQL instruction, the second database access interface, and the second query text; and the second NL instruction, the second SQL instruction, the second database access interface, and the second query text corresponding to the current seed record are used to form a corresponding first enhanced record.

2. The self-enhancing fine-tuning method for the NL2SQL large language model according to claim 1, characterized in that, The publicly available datasets include at least some or all of the Spider dataset, CSpider dataset, SParC dataset, CoSQL dataset, and DuSQL dataset; Each dataset in the publicly available dataset consists of multiple natural language instruction-SQL instruction pairs and multiple reference databases; the natural language instruction-SQL instruction includes source natural language instructions and source SQL instructions; each natural language instruction-SQL instruction pair corresponds to one of the reference databases; each source natural language instruction is an instruction text used to prompt the large language model to generate an SQL instruction based on a segment of natural language description text in the instruction, and the generated SQL instruction matches the corresponding source SQL instruction; each source SQL instruction is an SQL query instruction that can retrieve the corresponding query text from the corresponding reference database.

3. The self-enhancing fine-tuning method for the NL2SQL large language model according to claim 2, characterized in that, The initialization of the first seed dataset based on a preset public dataset specifically includes: Create a mirror database with the same structure and content for each of the reference databases of the public dataset, and denote it as the corresponding first mirror database; Based on a preset instruction pair collection principle, multiple natural language instruction-SQL instruction pairs are collected from the public dataset; and each collected natural language instruction-SQL instruction pair is used as the corresponding first collection instruction pair. A round of traversal is performed on all the obtained first acquisition instruction pairs; during the traversal, the currently traversed first acquisition instruction pair is taken as the corresponding current instruction pair, and the reference database and the first mirror database corresponding to the current instruction pair are taken as the corresponding first and second databases; the access interfaces of the first and second databases are taken as the corresponding first and second interfaces; the source natural language instruction and the source SQL instruction of the current instruction pair are taken as the corresponding first NL instruction and the first SQL instruction; the query text obtained by executing the first SQL instruction on the first database through the first interface is taken as the corresponding first text; the query text obtained by executing the first SQL instruction on the second database through the second interface is taken as the corresponding second text; when the first and second texts are matched, the second interface is taken as the corresponding first database access interface, and the second text is taken as the corresponding first query text, and the first NL instruction, the first SQL instruction, the first database access interface, and the first query text obtained in this round are combined to form a corresponding first seed record; and at the end of this round of traversal, all the obtained first seed records are combined to form the corresponding first seed dataset.

4. The self-enhancing fine-tuning method for the NL2SQL large language model according to claim 2, characterized in that, The seed cleaning process for the first seed dataset specifically includes: A traversal is performed on all first seed records in the first seed dataset. During the traversal, the currently traversed first seed record is taken as the corresponding current seed record. The first NL instruction and the first SQL instruction of the current seed record are taken as the corresponding current NL instruction and current SQL instruction. The syntax correctness of the current SQL instruction is identified to obtain the corresponding first identification result. The complexity of the current SQL instruction is identified to obtain the corresponding second identification result. The natural language semantic clarity of the current NL instruction is identified to obtain the corresponding third identification result. The first, second, and third identification results are then evaluated. If the first identification result is a syntax error, the second identification result is simple, or the third identification result is semantically ambiguous, the current seed record is marked as a preliminary screening failure record. After this round of traversal, all preliminary screening failure records in the first seed dataset are deleted. The first identification result includes syntax correctness and syntax error; the second identification result includes simple, moderate, and complex; and the third identification result includes semantically ambiguous and semantically clear. Based on preset text embedding encoding rules, the first SQL instruction of each remaining first seed record in the first seed dataset is processed by text embedding encoding to obtain the corresponding first text encoding vector; the vector similarity between any two first text encoding vectors is calculated to obtain the corresponding first similarity; and all first text encoding vectors are clustered based on a preset similarity threshold to obtain multiple first class vector sets; wherein, each first class vector set consists of one or more first text encoding vectors; when the number of first text encoding vectors in the first class vector set is 1, the first similarity of the unique first text encoding vector in the current set with any other first text encoding vector is less than or equal to the similarity threshold; when the number of first text encoding vectors in the first class vector set is greater than 1, the first similarity between any two first text encoding vectors in the current set is greater than the similarity threshold; the text embedding encoding rules include at least BOW encoding rules, n-gram encoding rules, and Word2Vec encoding rules. A traversal is performed on the first class vector set for all first text encoding vectors with a number greater than 1. During the traversal, the first class vector set currently being traversed is taken as the corresponding current vector set. One first text encoding vector is randomly selected from the current vector set as the corresponding retained vector, and all other first text encoding vectors except the retained vector are marked as duplicate vectors. At the end of this traversal, the first seed record corresponding to each duplicate vector in the first seed dataset is deleted.

5. The self-enhancing fine-tuning method for the NL2SQL large language model according to claim 3, characterized in that, The step of using the first model to filter low-quality records in the first augmented dataset specifically includes: A traversal is performed on all first augmented records in the first augmented dataset. During the traversal, the currently traversed first augmented record is taken as the corresponding current augmented record. The second NL instruction, the second database access interface, and the second query text of the current augmented record are extracted as the corresponding current NL instruction, current access interface, and current label text. The first mirror database corresponding to the current access interface is taken as the corresponding current mirror database. The current NL instruction is input into the first model for processing, and the SQL instruction text generated by the first model is received as the corresponding current SQL instruction. The query text obtained by executing the current SQL instruction on the corresponding current mirror database through the current access interface is taken as the corresponding current query text. The current query text and the current label text are checked for matching. If they do not match, the current augmented record is marked as a low-quality record. At the end of this traversal, all low-quality records in the first augmented dataset are deleted.

6. An apparatus for executing the self-enhancing fine-tuning method for the NL2SQL large language model according to any one of claims 1-5, characterized in that, The device includes: a first initialization module, a second initialization module, a data cleaning module, a model fine-tuning module, a data self-enhancement module, an iterative judgment module, and a parameter solidification module; The first initialization module is used to take the pre-trained NL2SQL large language model as the corresponding first model; and initialize the iteration counter to 1; The second initialization module is used to initialize a first seed dataset based on a preset public dataset; the first seed dataset includes multiple first seed records; the first seed record includes a first NL instruction, a first SQL instruction, a first database access interface, and a first query text; the first NL instruction is a natural language instruction, and the first SQL instruction is an SQL instruction; The data cleaning module is used to perform seed cleaning processing on the first seed dataset. The model fine-tuning module is used to perform one round of fine-tuning on the first model based on the first seed dataset; The data self-enhancement module is used to perform data augmentation processing on the first seed dataset using the first model after this round of fine-tuning to obtain the corresponding first augmented dataset; and to perform low-quality record filtering processing on the first augmented dataset using the first model; and to perform dataset merging processing on the filtered first augmented dataset and the first seed dataset and use the resulting merged dataset as the new first seed dataset; the first augmented dataset includes multiple first augmented records; the first augmented record includes a second NL instruction, a second SQL instruction, a second database access interface, and a second query text; The iteration judgment module is used to identify whether the iteration counter exceeds a preset iteration number threshold; if it exceeds, a training end command is sent to the parameter solidification module; if it does not exceed, the iteration counter is incremented by 1 and a new first seed dataset is sent to the data cleaning module. The parameter fixing module is used to fix the parameters of the first model that has completed iterative fine-tuning when the training end instruction is received, and to confirm that the training of this model fine-tuning is over.

7. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-5; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1-5.

Images