Group relative strategy optimization-based Text2SQL (Structured Query Language) large model fine tuning method and device, equipment and medium

Abstract

The invention discloses a Text2SQL large model fine tuning method and device based on group relative strategy optimization, equipment and a medium, and relates to the technical field of interaction between natural languages and databases. The method comprises the following steps: S1, constructing a training data set, loading a pre-trained Text2SQL base model as a strategy network, initializing the strategy network, and obtaining an initial strategy; s2, performing grouping sampling on the training data set to obtain a candidate set; and S3, according to the candidate set, performing multi-dimensional comprehensive evaluation on the candidate SQL of the candidate set by adopting a weighted combination reward function to obtain relative advantages. And S4, loss is calculated according to the initial strategy and the relative advantage, and parameter updating is carried out. And S5, iterating through a performance monitoring system, an adaptive training strategy, a convergence judgment mechanism and an early stop and model selection strategy to obtain a trained model. According to the method, efficient learning and generation of the model on complex SQL query are realized, and the accuracy and the performability of a Text2SQL task are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language database interaction, in particular to a Text2SQL large model fine-tuning method and device based on group relative strategy optimization, equipment and medium. BACKGROUND

[0002] Text2SQL technology, as a bridge connecting natural language and structured data queries, plays an important role in enterprise data analysis, intelligent customer service, business intelligence and other fields. This technology enables non-technical users to query databases directly through natural language, significantly reducing the technical threshold of data access and improving the efficiency and convenience of data utilization.

[0003] Existing Text2SQL technologies mainly include supervised learning-based methods and optimization strategies combined with reinforcement learning. Supervised learning methods rely on labeled natural language-SQL sample pairs for training, but lack feedback mechanisms for execution results and actual performance. Although traditional reinforcement learning methods such as Proximal Policy Optimization (PPO) can introduce execution feedback, they rely on additional Critic networks to estimate the value function, which has problems such as unstable training, low sample utilization and high computational overhead, making it difficult to achieve efficient and stable training processes in complex tasks.

[0004] However, existing methods still have obvious limitations. On the one hand, traditional models perform poorly when dealing with complex queries involving multiple table joins, nested subqueries, window functions and other advanced SQL features, often generating SQL statements with semantic errors or structural deviations. On the other hand, the evaluation system focuses more on syntax matching and execution success rate, lacking comprehensive consideration of query efficiency, semantic integrity and structural rationality.

[0005] In addition, existing optimization strategies fail to fully utilize the relative quality differences between multiple candidate SQLs corresponding to the same query, resulting in low training efficiency and easy falling into local optimum. Group Relative Policy Optimization (GRPO) has shown potential in mathematical reasoning and other fields, but its application in Text2SQL tasks has not been systematized, especially in reward design, sampling strategy and convergence mechanism, which still lack mature solutions. SUMMARY

[0006] The present application provides a Text2SQL large model fine-tuning method, device, equipment and medium based on group relative strategy optimization to improve at least one of the above technical problems.

[0007] In a first aspect, the present application provides a Text2SQL large model fine-tuning method based on group relative strategy optimization, which includes steps S1 to S5.

[0008] S1, construct a training data set and load a pre-trained Text2SQL base model as a strategy network and initialize, and obtain an initial strategy.

[0009] S2, group sampling is performed on the training data set to obtain a candidate set.

[0010] S3, according to the candidate set, a weighted combination reward function is used to perform multi-dimensional comprehensive evaluation on the candidate SQL of the candidate set to obtain a relative advantage.

[0011] S4, calculate the loss according to the initial strategy and the relative advantage and perform parameter update.

[0012] S5, through the performance monitoring system, the adaptive training strategy, the convergence judgment mechanism and the early stopping and model selection strategy, the trained model is obtained.

[0013] In a second aspect, the present application provides a Text2SQL large model fine-tuning device based on group relative strategy optimization, which comprises an initial module, a sampling module, an evaluation module, an update module and an iteration module.

[0014] The initial module is used to construct a training data set and load a pre-trained Text2SQL base model as a strategy network and initialize, and obtain an initial strategy.

[0015] The sampling module is used to group sample the training data set to obtain a candidate set.

[0016] The evaluation module is used to perform multi-dimensional comprehensive evaluation on the candidate SQL of the candidate set according to the candidate set by using a weighted combination reward function to obtain a relative advantage.

[0017] The update module is used to calculate the loss according to the initial strategy and the relative advantage and perform parameter update.

[0018] The iteration module is used to perform iteration through the performance monitoring system, the adaptive training strategy, the convergence judgment mechanism and the early stopping and model selection strategy to obtain a trained model.

[0019] In a third aspect, the present application provides a Text2SQL large model fine-tuning device based on group relative strategy optimization, which comprises a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement the Text2SQL large model fine-tuning method based on group relative strategy optimization as described in the first aspect.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium comprises a stored computer program, wherein the computer program, when executed, controls a device in which the computer-readable storage medium is located to perform the Text2SQL large model fine-tuning method based on group relative strategy optimization according to the first aspect.

[0021] By adopting the above technical solution, the present application can achieve the following technical effects: The present application provides a large model fine-tuning scheme combining group relative strategy optimization (GRPO) and Text2SQL tasks. This scheme generates candidate SQL for each natural language query, performs multi-dimensional reward evaluation, and optimizes model strategies based on group relative advantage. At the same time, it combines difficulty progressive training, dynamic sample weight adjustment, and online difficult sample mining to realize efficient learning and generation of complex SQL queries by the model, and improve the accuracy and executability of the Text2SQL task. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the specific embodiments of the present application. It should be understood that the following drawings only show some specific embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0023] Fig. 1 is a training process schematic diagram of the Text2SQL large model fine-tuning method.

[0024] Fig. 2 is a loss function calculation schematic diagram of the Text2SQL large model fine-tuning method.

[0025] Fig. 3 is a task reward module working schematic diagram of the Text2SQL large model fine-tuning method. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0027] Embodiment one, please refer to Figs. 1 to 3 The first embodiment of the present application provides a Text2SQL large model fine-tuning method based on group relative strategy optimization. It can be executed by a large model fine-tuning device (hereinafter referred to as: large model fine-tuning device). In particular, it is executed by one or more processors in the large model fine-tuning device to realize steps S1 to S5. Wherein, Text is a natural language query. SQL is a structured query language.

[0028] S1, constructing a training data set and loading a pre-trained Text2SQL base model as a policy network and initializing, and obtaining an initial policy. Preferably, step S1 specifically comprises steps S11 to S14.

[0029] S11, constructing a training data set. Each training sample contains a natural language query, a corresponding database schema (i.e., a database schema) structure, and a standard SQL statement label. Specifically, the training data set needs to be preprocessed to splice the natural language query and the corresponding database schema to form a complete input sequence.

[0030] S12, loading a pre-trained Text2SQL base model as a policy network , and copying a copy of its parameters as an initial policy for importance sampling calculation.

[0031] S13, setting the key hyperparameters of the Group Relative Policy Optimization (GRPO for short) algorithm: group sampling size , algorithm clipping coefficient , KL divergence regularization coefficient , and learning rate . Specifically, the parameter configuration of step S13 directly affects the stability and convergence speed of subsequent training. In this embodiment, . According to the training stability requirement, the initial value of the learning rate is 1e-5.

[0032] S14, constructing a training data loader, setting the batch size, and initializing the Adam optimizer. The configured training environment includes the policy network, the reference policy, the formatted training data set, and the hyperparameter setting.

[0033] S2, grouping sampling of the training data set to obtain a candidate set.

[0034] Preferably, step S2 specifically comprises steps S21 to S24.

[0035] S21, for the sample set of the current training batch, splice the natural language query and the database schema and input the policy network .

[0036] S22, generate candidate SQL using the temperature adjustment sampling strategy, divide the logits output by the model by the temperature coefficient , and perform softmax normalization.

[0037] S23, a token subset with a cumulative probability reaching a preset top-p value is selected by using a core sampling strategy for random sampling, to ensure the diversity of the generated candidate SQLs.

[0038] S24, for each training sample, generate different candidate SQLs, record the log probability of generating each candidate SQL under the current strategy , and obtain a complete candidate set. The set contains candidate SQLs and their corresponding generation probability records.

[0039] where logits represents the original prediction score output by the neural network model. Top-p is the threshold of core sampling. is the strategy network. represents the th candidate SQL query statement generated by the current strategy model on the th sample. represents the th sample. represents the database schema structure information corresponding to the th sample. The value of .

[0040] Specifically, for the th sample in the training data set, generate candidate SQL query statements from the current strategy network, and record the generation probability of each candidate SQL. To ensure the diversity of candidate SQLs within the group, temperature adjustment sampling or core sampling (nucleus sampling) strategy is used to ensure that the generated candidate SQLs have sufficient differences in semantics and structure, providing an effective comparison basis for subsequent relative evaluation. The guarantee of diversity is a key prerequisite for the GRPO algorithm to take advantage of, which can reduce sampling bias and improve the accuracy of advantage estimation.

[0041] S3, according to the candidate set, a weighted combination reward function is used to comprehensively evaluate the candidate SQLs in the candidate set in multiple dimensions, and the relative advantage is obtained. Preferably, step S3 includes steps S31 to S37.

[0042] S31, execute each candidate SQL through a database connector, capture the execution state and calculate the execution success reward: .

[0043] S32, standardize the candidate SQLs and standard answers, including uniformizing the case of keywords, standardizing spaces, and replacing synonymous expressions, and calculate the complete match reward: .

[0044] Specifically, The calculation process includes: comparing each component of the candidate SQL with the standard SQL one by one, including table names, column names, query conditions, and constant values. Standardization is performed to unify the format of the SQL statements and eliminate the influence of superficial differences on the matching judgment. After standardizing the two SQL statements using a normalization function, their consistency is determined. When the standardized candidate SQL perfectly matches the standard answer, The value is 1 if it is 1, otherwise it is 0.

[0045] S33. Compare the execution results of the candidate SQL and the standard SQL, and calculate the consistency reward: .

[0046] S34. Verify syntax correctness using the SQL parser and calculate the syntax reward: .

[0047] S35. Calculate the comprehensive reward score based on the weighted combined reward function: .

[0048] S36. Calculate the group statistics based on the overall reward score. The statistics include the reward mean and standard deviation.

[0049] S37. Calculating the relative advantages of standardization based on group statistics: .

[0050] In the formula: Indicates the first On the nth sample, the nth sample generated by the current policy model Candidate SQL query statements. This is standard SQL. For the first The database instance corresponding to each sample. This is a function that indicates successful SQL execution. This is the standardized perfect match score. The consistency score is given for the execution results. Score for SQL syntax correctness. For the first The first sample The overall reward score for each candidate SQL query statement. for The coefficient. for The coefficient. for The coefficient. for The coefficient. This is a relative advantage. This is the mean. The standard deviation is denoted as . This is a numerically stable term used to prevent division by zero errors. This indicates an indicator function used to indicate whether a certain condition is true or false.

[0051] .

[0052] .

[0053] In this embodiment of the invention, a relative advantage estimation method is used to replace the Critic network in the traditional Actor-Critic architecture to avoid the problems of unstable training of the Critic network and bias in the estimation of the value function.

[0054] S4. Calculate the loss and update the parameters based on the initial strategy and the relative advantage. Preferably, step S4 includes steps S41 to S45.

[0055] S41. Calculate the generation probability of each candidate SQL under the current strategy. .

[0056] S42. Calculate the importance sampling ratio. This ratio represents the probability of generating candidate SQL statements using the current strategy versus the old strategy. The old strategy refers to the strategy network from the previous iteration.

[0057] S44. Calculate the truncated proxy loss for each candidate SQL statement based on the importance sampling ratio and relative advantage: .

[0058] S43. Calculate the KL divergence regularization term between the initial policy and the current policy: This regularization term is used to control the deviation between the current policy and the reference policy, preventing instability caused by excessive policy updates during training.

[0059] S45. Integrate the losses of all candidate SQL statements within the group to obtain the overall optimization objective for the group's relative strategy optimization: .

[0060] S46. Based on the overall optimization objective, calculate the gradient of the loss function with respect to the model parameters using an automatic differentiation framework, and update the parameters using the Adam optimizer: .

[0061] in, For policy networks. Indicates the first On the nth sample, the nth sample generated by the current policy model Candidate SQL query statements. a natural language query representing a th sample. a database schema structure information corresponding to a th sample. an importance sampling ratio of model parameters . an old policy. a truncated surrogate loss of model parameters . min represents taking the minimum value. a relative advantage. trunc represents truncation. a clipping coefficient. KL divergence regularization term. an initial policy. a total optimization objective of model parameters . a training data set. representing taking expectation on samples in . an expected value. a group sampling size. a coefficient of the KL divergence regularization term. updated policy network parameters. a learning rate. a gradient.

[0062] The term is used for truncation to prevent the policy from being updated too much. Through the optimization process of step S5, the iterative update of the Text2SQL base model parameters is realized, so that the model can generate higher quality SQL query statements.

[0063] S5, through the performance monitoring system, the adaptive training strategy, the convergence judgment mechanism and the early stopping and model selection strategy, a trained model is obtained. The multi-level performance monitoring system includes monitoring four core indicators: execution success rate, complete matching rate, execution accuracy and syntax correctness. The adaptive training strategy includes difficulty progressive training, dynamic sample weight adjustment and online difficult sample mining. The multi-dimensional convergence judgment mechanism sets the main convergence condition and the auxiliary convergence condition. The early stopping and model selection strategy includes three modes: conservative strategy, aggressive strategy and mixed strategy, realizing the automatic termination control of the training process.

[0064] Establishing a comprehensive multi-level performance monitoring system is the key link to ensure the controllability and effectiveness of the GRPO training process. This monitoring system needs to track the performance of the model in multiple dimensions in real time during the training process, providing reliable data support for subsequent training strategy adjustment and convergence judgment. The embodiments of the present application can comprehensively evaluate the training state and ability development of the model from different angles by constructing a multi-level performance monitoring system.

[0065] The execution success rate: The execution success rate reflects the basic executability of the generated SQL.

[0066] The complete matching rate: The complete matching rate measures the consistency of the generated SQL and the standard answer in the syntax structure, specifically considering the matching of each component in the SQL statement, such as table name, column name, query condition, and constant value, etc.

[0067] The execution accuracy rate: The execution accuracy rate directly reflects the correctness of the actual results of the generated SQL.

[0068] The syntax correctness rate: The syntax correctness rate is used to ensure that the generated SQL statement conforms to the standard syntax specification.

[0069] Wherein, is the execution success rate. is the complete matching rate. is the execution accuracy rate. is the syntax correctness rate. is the number of samples. is the group sampling size. is the serial number of the sample. is the serial number of the candidate SQL query statement. represents the th sample. th candidate SQL query statement generated by the current strategy model on the th sample. is the database instance corresponding to the th sample. is the standard SQL.

[0070] The execution success rate, the complete matching rate, the execution accuracy rate, and the syntax correctness rate, these four core indicators constitute a complete basic framework for Text2SQL task evaluation.

[0071] The multi-level performance monitoring system comprehensively evaluates the training state and ability development of the model from different angles by constructing a hierarchical index system. The execution success rate is calculated by counting the proportion of candidate SQLs that are successfully executed on the database, reflecting the basic usability of the SQL statements generated by the model. The complete matching rate considers the matching of each component in the SQL statement, including table name, column name, query condition and constant value, and compares them one by one, eliminating the influence of format differences through standardization processing. The execution accuracy directly compares the execution results of the candidate SQL and the standard SQL, ensuring that the generated SQL can return the correct query result. The syntax correctness is verified by the SQL parser, ensuring that the generated SQL statement conforms to the SQL standard syntax specification.

[0072] The adaptive training strategy is one of the core designs of the present embodiment, which is used to dynamically adjust the training process to adapt to the development stage of the model ability and the change of data characteristics. This strategy mainly adjusts the training focus and difficulty distribution intelligently according to the training progress and performance, thereby maximizing the training efficiency and final performance.

[0073] According to determine the current training stage. Wherein, the early stage processes the first difficulty query, the middle stage introduces the second difficulty query, the later stage processes the third difficulty query. Wherein, the first difficulty < the second difficulty < the third difficulty.

[0074] The difficulty progressive training adopts a phased ability construction strategy. In the early stage (1-30% training progress), the system focuses on the processing of the first difficulty query, allowing the model to establish basic SQL generation capabilities, including single table query, simple condition filtering and basic aggregation operation. In the middle stage (30-70% training progress), the second difficulty query is introduced to enhance the model's ability to handle multi-table JOIN operation and subquery processing, gradually increasing the complexity of logical reasoning. In the later stage (70-100% training progress), the third difficulty query is processed, including nested query, window function, complex aggregation and advanced SQL features, ensuring that the model can handle various complex scenarios in actual applications. The training focus and sample distribution of each stage are automatically adjusted according to the training progress, realizing the gradual improvement of the ability.

[0075] The dynamic sample weight adjustment mechanism intelligently adjusts the sampling weight of the training sample according to the sample difficulty and the current ability state of the model.

[0076] dynamically adjusts the current sample sampling weight. . In the formula, is the sample sampling weight. is the sample. The probability calculation function. The sample difficulty. The gap between the current ability of the model and the sample requirement. The historical error rate of the model on this type of sample is directly used for calculation, or is adjusted based on the historical error rate.

[0077] By dynamically adjusting the sampling weight of the current sample, it is ensured that the model always learns near its ability boundary, avoiding both the time waste of too simple samples and the training collapse caused by too difficult samples. By intelligently allocating weights, training efficiency is improved and the speed of model ability improvement is accelerated.

[0078] The online difficult sample mining mechanism identifies the sample types and query patterns where the model performs poorly in real time and increases their sampling frequency in subsequent training. Specifically, the error rates of different types of queries are automatically counted to identify the weaknesses of the model, including specific JOIN types, aggregation function usage, or conditional expression processing. By analyzing error patterns and performance bottlenecks, it is determined where the model performs poorly. In subsequent training, the identified weaknesses are focused on and the sampling frequency of the corresponding sample types is increased.

[0079] This mechanism achieves targeted reinforcement training by analyzing error patterns and performance bottlenecks. The system automatically counts the error rates of different types of queries to identify the weaknesses of the model, such as specific JOIN types, aggregation function usage, or conditional expression processing, and then focuses on strengthening these aspects in subsequent training. By dynamically adjusting the sample distribution, targeted ability improvement is achieved, improving the model's ability to handle complex SQL queries.

[0080] The multi-dimensional convergence judgment mechanism establishes a multi-level convergence judgment system, which comprehensively considers performance indicators, parameter stability, and advantage distribution, etc. to achieve comprehensive convergence monitoring of the GRPO training process. This mechanism sets two levels of main convergence conditions and auxiliary convergence conditions, providing a complete training termination judgment framework.

[0081] The main convergence conditions include performance saturation judgment, parameter stability judgment, and advantage distribution stability judgment as the three core judgment standards.

[0082] Performance saturation is determined when the validation set execution accuracy improves by less than 0.1% for five consecutive rounds and the perfect match rate improves by less than 0.05% for five consecutive rounds. Specifically, this performance saturation determination employs a continuous monitoring mechanism: a performance monitoring window is set for five consecutive training rounds, and the improvement in validation set execution accuracy and perfect match rate between rounds is calculated. When the improvement in validation set execution accuracy is less than 0.1% for five consecutive rounds, the execution accuracy is considered to have reached saturation. When the improvement in perfect match rate is less than 0.05% for five consecutive rounds, the perfect match rate is considered to have reached saturation. Only when both core metrics simultaneously meet the saturation conditions is the model performance considered saturated, triggering a convergence signal.

[0083] Parameter stability assessment, calculation of parameter variation norm An assessment was conducted, in which For the first Wheel parameters, To observe window size, L2 norm, This is the stability threshold. Specifically, the parameter stability judgment uses parameter change norm monitoring: setting the parameter observation window size k, and calculating the current parameter... With k-round parameters The difference in L2 norm between them. Set a stability threshold. When the parameter changes norm When the values ​​are below this threshold, the model parameters are considered to have reached a stable state. Parameter changes are continuously monitored over multiple window periods to ensure the continuity of parameter stability. Parameter stability is a crucial indicator for judging model convergence, reflecting whether the optimization process tends towards stability.

[0084] Determining the stability of the dominance distribution and the relative dominance variance The change has been less than 5% in the last 10 rounds. This condition is based on the relative advantage defined in step S3. Calculations are performed. Specifically, the stability assessment of the dominance distribution is based on monitoring changes in the group dominance variance: calculating the relative dominance in each training round. variance The monitoring window is set to the most recent 10 training epochs, and the rate of change of the advantage variance within this window is calculated. When the rate of change of the advantage variance is less than 5%, the advantage distribution is considered to have reached a stable state. The stability of the advantage distribution reflects the convergence state of the relative advantage estimation in the GRPO algorithm and is a key indicator for judging whether the training process is stable.

[0085] The auxiliary convergence conditions include overfitting detection mechanism, gradient vanishing detection mechanism, and loss oscillation detection mechanism, in order to construct a multi-layered safety guarantee mechanism.

[0086] Overfitting detection mechanism, monitor the performance gap between training dataset and validation set, trigger corresponding process when the gap exceeds preset threshold . .

[0087] Gradient vanishing detection mechanism, observe whether the gradient norm is less than threshold for consecutive rounds , realize the timely identification of abnormal state of training process.

[0088] Loss shock detection mechanism, monitor the short-term fluctuation variance of loss function, determine the training unstable state when the variance exceeds the threshold .

[0089] Early stopping and model selection strategy realizes the automatic termination control of training process and the systematic selection of optimal model version. The strategy is based on the convergence judgment mechanism defined in the multi-dimensional convergence judgment mechanism, combined with various stopping strategies and model selection methods, to ensure that the training process is terminated in time when the expected performance target is reached.

[0090] Early stopping and model selection strategy provides differentiated training termination selection.

[0091] Conservative strategy, execute training termination operation when any main convergence condition is met. Suitable for scenarios with strict requirements on training time or limited computing resources.

[0092] Aggressive strategy, requires all main convergence conditions to be met before executing training termination. Suitable for scenarios pursuing optimal performance.

[0093] Hybrid strategy, dynamically switches between conservative and aggressive strategies according to validation set performance trends, adopts aggressive strategy in the early training stage to fully explore model potential, and switches to conservative strategy when performance improvement slows down to avoid overtraining. The dynamic switching mechanism of the hybrid strategy includes: judging the current training stage according to the validation set performance trend, determining the strategy switching time by analyzing the change rate and trend of performance indicators. In the early training stage, when the validation set performance still has a significant improvement trend, the aggressive strategy is adopted to continue training to fully exploit the model potential. When the performance improvement slows down and the improvement amplitude is less than the preset threshold for consecutive rounds, switch to the conservative strategy. Specifically, dynamic strategy switching ensures that the training data can be fully utilized to improve model performance, and the training can be terminated in time to avoid overfitting and waste of computing resources. The stage before the performance improvement slows down is the early training stage.

[0094] Early stopping mechanism realizes the automatic management of training process by considering multiple convergence conditions, reduces the need for human intervention and improves the stability of training process.

[0095] On the basis of the above-mentioned embodiments, in an optional embodiment of the present application, the Text2SQL large model fine-tuning method based on group relative strategy optimization further comprises step S6. S6, inputting the natural language query to be queried into the trained model to obtain the SQL structured query language.

[0096] The Text2SQL large model fine-tuning method based on group relative strategy optimization of the present application, for the natural language to SQL conversion task, generates multiple candidate SQLs for each input query, and performs multi-dimensional comprehensive reward evaluation, including SQL execution success, standardized matching degree, execution result consistency and syntax correctness and the like.

[0097] The standardized relative advantage is calculated based on the group statistics, replacing the traditional Critic network, to realize the stability and efficiency of the strategy optimization process. Further combining the difficulty progressive training, dynamic sample weight adjustment and online difficult sample mining mechanism, the training progress and sample distribution are intelligently controlled to improve the generation ability of the model in complex query scenarios. Through multi-level performance monitoring, convergence judgment and early stopping strategy, the training process is controllable and automated, and finally the SQL query result with high accuracy and executability is output. The scheme is applicable to the Text2SQL fine-tuning training based on large language model, and significantly improves the performance of the model in complex database query tasks.

[0098] The scheme of the present embodiment will be further introduced in combination with specific cases.

[0099] The present case adopts QwenCoder7B as the basic model, and implements the group relative strategy optimization (GRPO for short) Text2SQL fine-tuning process on the BIRD-SQL training data set. The present case shows the complete technical process from model initialization to final convergence, and verifies the effectiveness and operability of the scheme of the present application in practical application. Among them, the BIRD-SQL training data set is an existing public data set.

[0100] S1, construct a training data set and load a pre-trained Text2SQL base model as a strategy network and initialize.

[0101] The input content of step S1 is: the pre-trained Qwen-Coder-7B-Instruct model weight file, the BIRD-SQL training data set, and the system configuration parameters.

[0102] The processing process of step S1 is: the system first loads the Qwen-Coder-7B-Instruct model, initializes the parameters of the model as the strategy network , and copies a copy as the initial strategy For importance sampling calculation. Preprocess the BIRD-SQL training dataset, concatenate the natural language query and the corresponding database schema to form a complete input sequence in the format "[question]: {question sentence} [schema]: {schema information} [SQL query sentence]:". Set the core hyperparameters of the GRPO algorithm: group sampling size , algorithm clipping coefficient , initially set to 0.2. KL divergence regularization coefficient . Learning rate , initially set to 1e-5. Construct the training data loader, set the batch size to , and initialize the Adam optimizer.

[0103] The output content of step S1 is: the initialized policy network and the initial policy , the formatted training dataset , and the configured training environment and hyperparameter settings . In the formula, is the training dataset, represents the natural language query of the th sample. represents the database schema structure information corresponding to the th sample. is the SQL query statement of the th sample, is the number of samples.

[0104] S2, group sampling on the training dataset to obtain the candidate set.

[0105] The input content of step S2 is: the sample set of the current training batch , the policy network , and the sampling configuration parameters (temperature coefficient , top-p value of kernel sampling).

[0106] In the formula, is the number of samples in .

[0107] The processing process of step S2 is: for each sample in the batch, the system concatenates the natural language query and the database schema and inputs it into the policy network, and generates different candidate SQL queries using the temperature adjustment sampling strategy. Specifically, the logits output by the model are divided by the temperature coefficient Softmax normalization is performed, and then core sampling is used to select a token subset with a cumulative probability of top-p for random sampling to ensure the diversity of generated candidate SQLs. For each candidate SQL, its generation log probability under the current policy is recorded.

[0108] The output of step S2 is the candidate SQL set corresponding to each training sample , and the corresponding generation probability record , forming a complete candidate SQL set .

[0109] S3, a weighted combination reward function is used to comprehensively evaluate the candidate SQLs in the candidate set in multiple dimensions.

[0110] The input of step S3 is the candidate SQL set , the corresponding database instance , the standard SQL answer , and the reward function weight configuration (the sum of weights is 1).

[0111] The processing procedure of step S3 is: the system comprehensively evaluates each candidate SQL in multiple dimensions. First, the candidate SQL is executed through the database connector, the execution state is captured, and the execution success reward is calculated. Then, the candidate SQL and the standard answer are standardized, including keyword case unification, space standardization, and synonymous expression replacement, and the complete match reward is calculated. Then, the execution results of the candidate SQL and the standard SQL are compared, and the result consistency reward is calculated. Finally, the syntax correctness is verified through the SQL parser, and the syntax reward is calculated. Then the comprehensive reward score is calculated. Then calculate the group statistics, including the reward mean and standard deviation. Based on the group statistics, the standardized relative advantage is calculated: The output of step S3 is the comprehensive reward score of each candidate SQL , the group statistics , and the standardized relative advantage , which constitute the complete reward evaluation result .

[0112] S4, calculate the loss according to the initial policy and the relative advantage and update the parameters.

[0113] The input of step S4 is the policy network , the reward evaluation result , the generation probability record of the old policy, and the training hyperparameter configuration.

[0114] The processing procedure of step S4 is: The system first calculates the generation probability of each candidate SQL under the current strategy .

[0115] Then the importance sampling ratio is calculated: .

[0116] Based on the importance sampling ratio and the relative advantage, the loss of each candidate SQL is calculated: .

[0117] The KL divergence regularization term between the initial strategy and the current strategy is calculated: .

[0118] Integrate the loss of all candidate SQLs in the group to get the overall optimization goal of GRPO: .

[0119] Calculate the gradient of the loss function with respect to the model parameters through the automatic differentiation framework , and use the Adam optimizer to update the parameters:

[0120] The output of step S4 is the updated strategy network parameters , the loss value of the current round , the gradient norm , and the detailed breakdown of the component loss .

[0121] S5, through the performance monitoring system, adaptive training strategy, convergence judgment mechanism and early stopping and model selection strategy, to obtain the trained model.

[0122] The input of step S5 is the updated model parameters, the validation dataset , the historical performance record , and the threshold configuration of convergence judgment .

[0123] The processing process of step S5 is: the system first evaluates the performance of the current model on the validation set, and calculates four core indicators.

[0124] The success rate is obtained by running the generated candidate SQL on the validation set and calculating the proportion of successful execution: .

[0125] The complete match rate is calculated by comparing the standardized generated SQL with the standard answer: .

[0126] The execution accuracy is obtained by comparing the consistency of SQL execution results: .

[0127] Syntax accuracy is calculated by verifying syntax compliance through the SQL parser. .

[0128] Based on performance monitoring results, the system implements adaptive training strategy adjustments.

[0129] according to Determine the current training phase and dynamically adjust the sample difficulty distribution. In the early stages... The main sampling is simple queries, in the mid-term stage. Introducing medium-complexity queries in later stages Focus on handling complex queries.

[0130] Sample sampling weights are based on the formula Dynamic adjustments are made, among which Indicates the sample difficulty score. This represents the gap between the model's current capabilities and the sample requirements, primarily calculated based on the model's historical error rate on this type of sample.

[0131] Online hard sample mining mechanism: Real-time identification of sample types and query patterns with poor model performance, and increase their sampling frequency in subsequent training.

[0132] The system continuously monitors multi-dimensional convergence metrics. Performance saturation detection is achieved by observing the performance improvement of the validation set execution accuracy over five consecutive rounds. A performance saturation signal is triggered when the improvement in validation set execution accuracy is continuously less than 0.001 and the improvement in perfect match rate is continuously less than 0.0005. Parameter stability is determined by calculating the parameter variation norm. With stability threshold Comparison, when consecutive The parameters are considered stable when the change in wheel parameters is less than a threshold. The stability of the dominance distribution is monitored by assessing the relative dominance variance. The rate of change over the most recent 10 rounds, when the rate of change is less than At that time, it was believed that the dominant distribution had stabilized.

[0133] An auxiliary convergence condition is used to construct an additional monitoring mechanism. Overfitting detection is performed by monitoring whether the performance difference between the training dataset and the validation set exceeds a threshold. Implement abnormal state identification. Gradient vanishing detection observes whether the gradient norm is continuously less than a threshold for multiple rounds. This enables the detection of gradient propagation anomalies. Loss oscillation detection monitors the short-term variance of the loss function; when the variance exceeds a threshold... Identify unstable training states in real time.

[0134] Early stopping and model selection strategy realizes the automatic termination control of the training process and the selection of the optimal model version. Early stopping and model selection strategy includes three modes of conservative strategy, aggressive strategy and mixed strategy. Conservative strategy: training termination is performed when any primary convergence condition is met. Aggressive strategy: training termination is performed only after all primary convergence conditions are met. Mixed strategy: conservative or aggressive strategy is dynamically selected according to the performance trend of the validation set, and aggressive strategy is adopted in the early stage of training, and conservative strategy is adopted after the performance improvement slows down.

[0135] The output content of step S5 is: performance evaluation report of the current round Convergence state judgment result and decision flag of whether to continue training. When the convergence condition is met, the final optimized model parameters and training summary report are output, including training rounds, final performance indicators, convergence types and other key information. is the success rate of the current round. is the complete match rate of the current round. is the execution accuracy of the current round. is the syntax accuracy of the current round.

[0136] Embodiment two, the application provides a Text2SQL large model fine-tuning device based on group relative strategy optimization, which comprises an initial module, a sampling module, an evaluation module, an updating module and an iteration module.

[0137] The initial module is used for constructing a training data set, loading a pre-trained Text2SQL base model as a strategy network and initializing, and obtaining an initial strategy.

[0138] The sampling module is used for grouping sampling of the training data set to obtain a candidate set.

[0139] The evaluation module is used for multi-dimensional comprehensive evaluation of candidate SQL in the candidate set by using a weighted combination reward function according to the candidate set to obtain a relative advantage.

[0140] The updating module is used for calculating a loss and updating parameters according to the initial strategy and the relative advantage.

[0141] The iteration module is used for iteration through a performance monitoring system, a self-adaptive training strategy, a convergence judgment mechanism and early stopping and model selection strategy to obtain a trained model.

[0142] Embodiment three, the application provides a Text2SQL large model fine-tuning device based on group relative strategy optimization, which comprises a processor, a memory and a computer program stored in the memory. The computer program can be executed by the processor to realize the Text2SQL large model fine-tuning method based on group relative strategy optimization as described in embodiment one.

[0143] In some embodiments, the computer readable storage medium further comprises a stored computer program, wherein the computer program is configured to control the device on which the computer readable storage medium is located to perform the method of any one of the preceding embodiments.

[0144] It should be understood that the large model fine-tuning device can be a portable notebook computer, a desktop computer, a server, a smart phone, a tablet computer or other electronic devices with computing performance.

[0145] Obviously, the above-described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0146] In the several embodiments provided by the embodiments of the present application, it should be understood that the disclosed apparatus and method can also be implemented in other manners. The embodiments described above are only schematic. For example, the flowcharts and block diagrams in the embodiments of the present application illustrate the possible implementation architecture, functions and operation of the apparatus, method and computer program product according to the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a segment or a portion of code which comprises one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions shown in the blocks can be performed in a different order from that shown in the flowcharts or block diagrams. For example, the flowcharts or block diagrams can show the functions in a sequential order, but in some implementations, these functions can actually be performed in parallel or in a reverse order. In addition, it should also be noted that each block in the flowcharts or block diagrams and combinations of blocks in the flowcharts or block diagrams can be implemented by dedicated hardware-based systems which perform the specified functions or actions, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0147] In addition, each functional module in the various embodiments of the present application can be integrated together to form a separate part, or each module can exist independently, or two or more modules can be integrated to form a separate part.

[0148] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts of the prior art that make contributions or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes. It should be noted that in this document, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements that are not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0149] The terms used in the embodiments of the present application are only for the purpose of describing particular embodiments and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0150] It should be understood that the term "and / or" used herein is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this document generally represents an "or" relationship between the front and rear associated objects.

[0151] Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted as "when it is determined" or "in response to determining" or "when (a stated condition or event) is detected" or "in response to detecting (a stated condition or event)".

[0152] The "first / second" mentioned in the embodiments only distinguishes similar objects, and does not represent a specific order for the objects. Understandably, the "first / second" can be interchanged in a specific order or sequence as appropriate. It should be understood that the objects distinguished by "first / second" can be interchanged as appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0153] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made to the present application by those skilled in the art. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for fine-tuning a large Text2SQL model based on population relative strategy optimization, characterized in that, Include: The training dataset is constructed and the pre-trained Text2SQL base model is loaded as the policy network and initialized, as well as the initial policy is obtained; The training dataset is divided into groups for sampling to obtain a candidate set; Based on the candidate set, a weighted combined reward function is used to comprehensively evaluate the candidate SQL in the candidate set from multiple dimensions to obtain relative advantages; Calculate the loss and update the parameters based on the initial strategy and the relative advantage; The trained model is obtained through iterative processes using a performance monitoring system, adaptive training strategy, convergence judgment mechanism, and early stopping and model selection strategy.

2. The method for fine-tuning a large Text2SQL model based on population relative strategy optimization according to claim 1, characterized in that, The training dataset is divided into groups for sampling to obtain a candidate set, specifically including: For the sample set of the current training batch, the natural language query and the database schema are concatenated and then input into the policy network; Candidate SQL queries are generated using a temperature-controlled sampling strategy, and the model output logits are divided by the temperature coefficient. Perform softmax normalization; A kernel sampling strategy is used to select a subset of tokens whose cumulative probability reaches a preset top-p value for random sampling, ensuring the diversity of generated candidate SQLs; Generate for each training sample There are 10 distinct candidate SQL statements, and the logarithmic probability of each candidate SQL statement under the current strategy is recorded. , obtain the complete candidate set; the set contains candidate SQL and its corresponding generation probability record; Where logits represents the raw prediction score output by the neural network model; top-p is the threshold for kernel sampling; For policy networks; Indicates the first On the nth sample, the nth sample generated by the current policy model One candidate SQL query statement; Indicates the first Natural language queries for each sample; Indicates the first Database schema structure information corresponding to each sample.

3. The method for fine-tuning a large Text2SQL model based on population relative strategy optimization according to claim 1, characterized in that, Based on the candidate set, a weighted combined reward function is used to comprehensively evaluate the candidate SQL statements from multiple dimensions to obtain relative advantages, specifically including: Execute each candidate SQL through the database connector, capture the execution status, and calculate the execution success reward: ; The candidate SQL and the standard answer are standardized, including consistent keyword case, whitespace standardization, and synonym replacement. The exact match reward is then calculated. ; Compare the execution results of the candidate SQL and the standard SQL, and calculate the consistency reward: ; The syntax is verified using the SQL parser, and a syntax reward is calculated. ; Calculate the overall reward score based on the weighted combined reward function: ; Group statistics are calculated based on the overall reward score; the statistics include the reward mean and standard deviation. The relative advantages of standardization based on group statistics: ; in: Indicates the first On the nth sample, the nth sample generated by the current policy model One candidate SQL query statement; Standard SQL; For the first The database instance corresponding to each sample; This is a function that indicates successful SQL execution. The standardized perfect match score; A score is awarded for consistency of execution results; Score the SQL syntax correctness; For the first The first sample The overall reward score for each candidate SQL query statement; for The coefficient; for The coefficient; for The coefficient; for The coefficient; It is a relative advantage; The mean; Standard deviation; It is a numerically stable term; Indicates an indicator function.

4. The method for fine-tuning a large Text2SQL model based on population relative strategy optimization according to claim 1, characterized in that, Calculate the loss and update the parameters based on the initial strategy and the relative advantage, specifically including: Calculate the generation probability of each candidate SQL under the current strategy. ; Calculate the importance sampling ratio ; Calculate the truncated proxy loss for each candidate SQL based on the importance sampling ratio and relative advantage: ; Calculate the KL divergence regularization term between the initial policy and the current policy: ; By integrating the losses of all candidate SQL queries within the group, we obtain the overall optimization objective for the group's relative strategy optimization: ; Based on the overall optimization objective, the gradient of the loss function with respect to the model parameters is calculated using an automatic differentiation framework, and the parameters are updated using the Adam optimizer. ; in, For policy networks; Indicates the first On the nth sample, the nth sample generated by the current policy model One candidate SQL query statement; Indicates the first Natural language queries for each sample; Indicates the first Database schema structure information corresponding to each sample; For model parameters Importance sampling ratio; This is the old strategy; Model parameters The loss due to truncated proxy; This indicates taking the minimum value; It is a relative advantage; Indicates truncation; The shear coefficient; Represents the KL divergence regularization term; This is the initial strategy; Represents model parameters The overall optimization objective; This represents the training dataset; Indicates to Samples in Take the expected value; Indicates the expected value; Group sampling size; The coefficient of the KL divergence regularization term; This represents the updated policy network parameters; The learning rate; This represents the gradient.

5. The method for fine-tuning a large Text2SQL model based on population relative strategy optimization according to claim 3, characterized in that, The performance monitoring system includes four core metrics: execution success rate, exact match rate, execution accuracy, and syntax correctness. Execution success rate ; Perfect match rate ; Execution accuracy ; Grammar accuracy ; in The number of samples; Group sampling size; The sample number; The sequence number of the candidate SQL query statement; Adaptive training strategies include: According to the current Determine the current training phase; among which, the early phase Handling the most difficult queries, mid-stage Introducing queries of second difficulty, in the later stages Handling queries of the third difficulty level; Dynamically adjust the sampling weight of the current sample; In the formula, For sample sampling weights; For the first One sample; This is a probability calculation function; For sample difficulty; This represents the gap between the model's current capabilities and the sample requirements. Real-time identification of sample types and query patterns that perform poorly in the model, and increase their sampling frequency in subsequent training.

6. The method for fine-tuning a large Text2SQL model based on population relative strategy optimization according to claim 1, characterized in that, The multi-dimensional convergence judgment mechanism is set with primary convergence conditions and auxiliary convergence conditions; The main convergence conditions include: Performance saturation is assessed when the accuracy of the validation set improves by less than 0.1% for five consecutive rounds and the perfect match rate improves by less than 0.05% for five consecutive rounds. Parameter stability assessment, calculation of parameter variation norm An assessment was conducted, in which For the first Wheel parameters, To observe window size, L2 norm, The preset stability threshold; Determining the stability of the dominance distribution and the variance of the group dominance. The change in the last 10 rounds was less than 5%; Auxiliary convergence conditions include: An overfitting detection mechanism monitors the performance gap between the training dataset and the validation set. When the gap exceeds a preset threshold, the mechanism detects the overfitting. The corresponding processing flow is triggered at that time; The gradient vanishing detection mechanism observes whether the gradient norm is less than a threshold for multiple consecutive rounds. This enables timely identification of abnormal states during the training process; The loss oscillation detection mechanism monitors the short-term volatility variance of the loss function. When the variance exceeds a threshold... This is considered an unstable training state. Early stopping and model selection strategies include: A conservative strategy is adopted, which terminates training when any major convergence condition is met. An aggressive strategy requires that training be terminated only after all major convergence conditions are met. A hybrid strategy is used to dynamically switch between conservative and aggressive strategies based on the performance trend of the validation set. An aggressive strategy is used in the early stage of training to fully explore the potential of the model, and a conservative strategy is used after the performance improvement slows down to avoid overtraining.

7. The method for fine-tuning a large Text2SQL model based on population relative strategy optimization according to claim 1, characterized in that, The process includes constructing the training dataset, loading and initializing the pre-trained Text2SQL base model as the policy network, and obtaining the initial policy, specifically including: Construct a training dataset; where each training sample contains a natural language query, the corresponding database schema structure, and standard SQL statement labels; Load the pre-trained Text2SQL base model as the policy network, and copy its parameters as the initial policy for importance sampling calculation; Set the key hyperparameters of the group relative policy optimization algorithm: group sampling size, algorithm shearing coefficient, KL divergence regularization coefficient, and learning rate; Build a training data loader, set the batch size, and initialize the Adam optimizer; the configured training environment includes the policy network, reference policy, formatted training dataset, and hyperparameter settings.

8. A Text2SQL large model fine-tuning device based on group relative strategy optimization, characterized in that, Include: The initialization module is used to build the training dataset, load the pre-trained Text2SQL base model as the policy network and initialize it, and obtain the initial policy; The sampling module is used to group and sample the training dataset to obtain a candidate set; The evaluation module is used to perform a multi-dimensional comprehensive evaluation of the candidate SQL in the candidate set using a weighted combination reward function to obtain relative advantages. The update module is used to calculate the loss and update the parameters based on the initial strategy and the relative advantage; The iteration module is used to iterate through a performance monitoring system, adaptive training strategy, convergence judgment mechanism, and early stopping and model selection strategy to obtain a trained model.

9. A Text2SQL large-scale model fine-tuning device based on group relative strategy optimization, characterized in that, It includes a processor, a memory, and a computer program stored in the memory; the computer program can be executed by the processor to implement a Text2SQL large model fine-tuning method based on population relative strategy optimization as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a Text2SQL large model fine-tuning method based on population relative strategy optimization as described in any one of claims 1 to 7.

Images