Database question and answer model training method and device, storage medium and computer equipment

Abstract

The invention discloses a database question and answer model training method and device, a storage medium and computer equipment, and the method comprises the steps: associating standard structured query language statements, standard execution result answers and standard natural language questions, and generating training annotation data; and collecting simulation derivation problems possibly proposed for the database to obtain non-labeled data for training. Based on a GRPO reinforcement learning framework and a scoring reward function provided by a double-tower model, training is carried out on the scoring reward function by utilizing training labeling data, supervised fine tuning training is carried out on a database question and answer model, and non-labeling data for training, format rewards, executable rewards and scoring rewards of the scoring reward function are combined, so that the scoring reward function of the database question and answer model is obtained. And continuing to train the database question and answer model after supervised fine tuning training. Preliminary training is carried out through a small amount of annotation data, then subsequent training is carried out through non-annotation data, the reasoning ability of the model can be stimulated, the annotation cost is reduced, and the training efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and apparatus for training a database question-answering model, a storage medium, and a computer device. Background Art

[0002] With the rapid development of natural language processing (NLP) technology, Text-to-SQL tasks (converting natural language questions into structured query language tasks) have become an important bridge connecting non-technical users and databases.

[0003] Traditional methods rely on rule generation, matching key information in user questions and applying a fixed format to generate SQL (Structured Query Language) statements. With the introduction of deep learning methods, these methods are generally based on seq2seq (Sequence-to-Sequence) generative models, relying on supervised learning and training on large amounts of labeled (natural language, SQL) paired data to adjust weights and achieve a one-step conversion. Summary of the Invention

[0004] In view of this, the present application provides a database question-answering model training method and device, storage medium, and computer equipment, which do not require a large amount of labeled test2sql data. Instead, only a small amount of high-quality labeled data needs to be collected, and then a large amount of unlabeled training data can be used to complete the entire training process. This can stimulate the model's reasoning ability and reduce labeling costs. At the same time, through the design of a reward function, the model is guided to complete the generation of SQL queries through reasoning without being limited to any database structure, thereby enhancing the generalization performance of the model.

[0005] According to one aspect of the present application, a method for training a database question-answering model is provided, the method comprising:

[0006] Collecting natural language questions raised by users with respect to a database, wherein the database corresponds to a database executor;

[0007] Converting the natural language question into a structured query language statement, and associating a qualified structured query language statement that meets a preset execution standard, a qualified execution result answer obtained after the qualified structured query language statement is executed by a database executor, and the qualified natural language question corresponding to the qualified structured query language statement to generate labeled data for training;

[0008] Collecting simulated derived questions that may be asked about the database, and using the simulated derived questions as unlabeled data for training, wherein the simulated derived questions are in natural language form;

[0009] Based on the GRPO reinforcement learning framework and the scoring reward function provided by the dual-tower model, the scoring reward function is trained using labeled training data, and the database question-answering model is supervised and fine-tuned using labeled training data. Combined with the training unlabeled data, format rewards, executable rewards, and scoring rewards corresponding to the scoring reward function, the database question-answering model after supervised fine-tuning training is continued to be trained. The database question-answering model is established based on a neural network. The trained database question-answering model inputs natural language questions raised by users and outputs structured query language statements. The output structured query language statements are executed by the database executor and the execution results are returned to the user.

[0010] According to another aspect of the present application, a database question-answering model training device is provided, the device comprising:

[0011] A natural language question collection module, configured to collect natural language questions raised by users with respect to a database, wherein the database corresponds to a database executor;

[0012] a training annotated data generation module configured to convert the natural language question into a structured query language statement, and to associate a qualified structured query language statement that meets a preset execution standard, a qualified execution result answer obtained after the qualified structured query language statement is executed by a database executor, and the qualified natural language question corresponding to the qualified structured query language statement, thereby generating training annotated data;

[0013] a training unlabeled data generation module, configured to collect simulated derivative questions that may be asked about the database and use the simulated derivative questions as training unlabeled data, wherein the simulated derivative questions are in natural language form;

[0014] The database question-answering model training module is used to train the scoring reward function based on the GRPO reinforcement learning framework and the scoring reward function provided by the dual-tower model using the training labeled data, and to perform supervised fine-tuning training on the database question-answering model using the training labeled data. The database question-answering model after supervised fine-tuning training is continued to be trained in combination with the training unlabeled data, format rewards, executable rewards and scoring rewards corresponding to the scoring reward function. The database question-answering model is established based on a neural network. The trained database question-answering model outputs structured query language statements by inputting natural language questions raised by the user. The output structured query language statements are executed by the database executor and the execution results are returned to the user.

[0015] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned database question-answering model training method is implemented.

[0016] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned database question-answering model training method when executing the program.

[0017] By means of the above technical solution, the present application provides a method and device for training a database question-answering model, a storage medium, and a computer device, which associate qualified structured query language statements, qualified execution result answers, and qualified natural language questions to generate labeled data for training; collect simulated derivative questions that may be raised for the database to obtain unlabeled data for training. Based on the GRPO reinforcement learning framework and the scoring reward function provided by the twin-tower model, the scoring reward function is trained using labeled training data, and the database question-answering model is supervised and fine-tuned. The database question-answering model after supervised fine-tuning training is continued to be trained by combining unlabeled training data, format rewards, executable rewards, and scoring rewards of the scoring reward function. Preliminary training with a small amount of labeled data and subsequent training with unlabeled training data can stimulate the model's reasoning ability, reduce labeling costs, and improve training efficiency.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A flowchart of a database question-answering model training method provided in an embodiment of the present application is shown;

[0021] Figure 2 A flow chart of a method for generating labeled data for training provided in an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of a process for generating labeled data for training provided in an embodiment of the present application is shown;

[0023] Figure 4 A schematic structural diagram of a database question-answering model training device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0025] In this embodiment, a database question answering model training method is provided. Figure 1 As shown, the method includes:

[0026] Step 101: Collect natural language questions raised by users with respect to a database, wherein the database corresponds to a database executor.

[0027] Step 102: Convert the natural language question into a structured query language statement, and associate the qualified structured query language statement that meets the preset execution standard, the qualified execution result answer obtained after the qualified structured query language statement is executed by the database executor, and the qualified natural language question corresponding to the qualified structured query language statement to generate labeled data for training.

[0028] Currently, the mainstream training approach for database question-answering models still requires extensive manual annotation and is highly dependent on database structure. Furthermore, because the text2sql task involves semantic transformation across two dimensions, it places high demands on model performance without reinforcement learning, and models with smaller parameters are generally inadequate. A fatal flaw of traditional approaches is their poor generalization. Whether based on rule matching or sequence-to-sequence (Seq2Seq) model training, both rely heavily on table structure. Changes to table structures or migration to other industries require rewriting rules or retraining models. Furthermore, model approaches rely on supervised learning, requiring large amounts of labeled (natural language, SQL) paired data. However, in real-world scenarios, high annotation costs, dynamic data distribution, and insufficient generalization for complex queries limit model performance. Furthermore, supervised learning models tend to overfit the training data and struggle to handle the diverse expressions and implicit logic in user questions. Therefore, reinforcement learning (RL) offers a new approach to addressing these issues.

[0029] Reinforcement learning (RL) is an important branch of machine learning. Its core concept is that a model learns optimal strategies through interaction with the environment to maximize cumulative rewards. Unlike supervised and unsupervised learning, RL emphasizes learning through trial and error in a dynamic environment. It has unique theoretical and applied value. Its core elements are as follows:

[0030] Agent: A learning decision-making entity that interacts with the environment through actions.

[0031] Environment: The external world in which the agent lives, which provides feedback on its actions.

[0032] State: A description of the current state of the environment.

[0033] Action: A behavior that an agent can perform.

[0034] Reward: The immediate feedback signal from the environment to the action.

[0035] Policy: The agent's mapping rule from state to action (e.g., deterministic or stochastic policy).

[0036] Value Function: Evaluates the long-term expected return of a state or action (such as Q value, V value).

[0037] Environment model (Model) (optional): a function that predicts environment state transitions and rewards;

[0038] By designing a reasonable reward function, reinforcement learning can enable the trained model to self-optimize in its interaction with the virtual environment, thereby reducing dependence on labeled data while enhancing robustness.

[0039] However, traditional RL algorithms (such as PPO) still face challenges in Text-to-SQL tasks, such as:

[0040] 1. Sparse reward problem: The correctness of generated SQL usually needs to be verified after execution. However, frequent SQL execution in real environments may bring security risks or computational overhead.

[0041] 2. Complex search space: SQL syntax is strict, and generating incorrect actions (such as syntax errors) may lead to invalid exploration;

[0042] 3. Low training efficiency: RL requires a lot of trial and error, while the Text-to-SQL task requires processing long sequence generation, which increases training costs.

[0043] Therefore, in the above-mentioned embodiments of the present application, GRPO (Generalized Reinforcement Learning with Policy Optimization) can be introduced. This efficient reinforcement learning framework aims to improve training stability and sample efficiency through policy constraint optimization and dynamic reward shaping. The core idea of ​​GRPO is to avoid policy collapse by constraining the amplitude of policy updates. At the same time, it combines the prior knowledge of imitation learning pre-training to accelerate model convergence, better handle the ambiguity of user intentions, and gradually correct the model output using interactive feedback in the synthetic environment. Ultimately, a strong inference model for Text-to-SQL tasks is achieved based on the GRPO reinforcement learning framework and using a small amount of labeled data.

[0044] Furthermore, the GRPO reinforcement learning framework, proposed by the DeepSeek team, is a reinforcement learning algorithm for large-scale language models (LLMs). It aims to address the limitations of traditional methods (such as PPO) in training efficiency, stability, and computing resource consumption. Specifically, it optimizes the learning process by comparing the relative performance of different strategies or actions within the same group, rather than relying on the traditional critic model to evaluate the value of each action. Its core idea is to allow the agent to compare multiple candidate actions generated by other agents or itself, and use the relative differences in group scores to update the strategy, thereby reducing computational costs and improving training efficiency. In the training of database question-answering models, the GRPO framework can combine the scoring reward function provided by the dual-tower model with other reward functions (such as format rewards and executable rewards) to guide the model's learning direction. At the same time, the GRPO framework can use labeled data to train the model. This data contains information such as natural language questions, corresponding SQL query statements, and execution results, providing the model with rich learning resources.

[0045] Furthermore, the core innovation of GRPO lies in replacing the traditional value network (Critic Model) by comparing actions within the group. Specifically, multiple actions (groups) are sampled for each input state, and the advantage function is calculated based on the relative rewards of the actions within the group, rather than relying on a separate value network. For example, in mathematical reasoning tasks, the model generates multiple candidate answers and quantifies the relative advantage of each answer through intra-group normalization (such as mean and standard deviation); in addition, the threshold (τ) of the gradient norm can be dynamically adjusted, and a gradient regularization term is introduced in the loss function. When the gradient norm exceeds the threshold, a penalty is imposed to avoid gradient explosion and reduce excessive constraints on policy updates. This mechanism significantly improves the stability of training; and adds a KL divergence constraint when updating the policy to limit the difference between the new and old policies, prevent drastic changes in policy distribution, and further ensure training stability.

[0046] Specifically, the Text-to-SQL task aims to automatically convert user questions posed in natural language (e.g., "Search for the product with the highest sales") into Structured Query Language (SQL) statements to retrieve accurate data from the database. SQL (Structured Query Language) is a standard programming language used to manage and operate relational databases. SQL queries are used to retrieve, insert, update, or delete data from a database. To this end, the core process of the Text-to-SQL task is as follows:

[0047] 1. User questions (natural language), such as "Query the products with the highest sales in the past month."

[0048] 2. Convert to SQL query;

[0049] 3. Execute SQL queries, for example, the database returns results based on SQL statements (such as "Product A, sales of 10,000 yuan").

[0050] 4. Return the results to the user, for example, "The product with the highest sales in the past month is Product A, with sales of 10,000 yuan."

[0051] To this end, the Text-to-SQL task allows users to query the database without having to learn SQL syntax. At the same time, automated conversion reduces the time and errors associated with manually writing SQL. It also supports complex queries such as multi-table joins and aggregate functions. By combining natural language with database operations, it provides non-technical personnel with a convenient data query method.

[0052] Specifically, first collect the natural language questions that users have asked about the database, that is, complete a small amount of high-quality data collection. For example, you can collect user questions and developer answers from historical information in the production environment, but it does not include specific SQL queries. For example, the customer service department of a financial company usually records various questions raised by customers through telephone, online chat, etc. and the responses of customer service personnel. This is also an important source for obtaining users' questions related to the database. In addition, in the internal operating system of financial business, employees may leave messages to communicate on certain business issues (such as database queries), such as business personnel consulting technical personnel for data information. At the same time, during the development of financial projects or business optimization, team members will also discuss issues on the platform, which may include users' questions about data needs and developers' responses.

[0053] Next, use keywords related to financial data, such as "account balance", "transaction record", "customer information", "risk assessment", "investment return", etc., to quickly locate relevant records in historical information.

[0054] In particular, you can also set a time range based on business needs, such as collecting data from the last six months or one year, to ensure that the data is timely and has reference value.

[0055] Next, use data extraction tools or write scripts to extract user questions (natural language questions) and developer answer data that meet the screening criteria from each system.

[0056] Next, the extracted data can be manually reviewed to remove irrelevant information (such as system failure reports, non-business communication content, etc.) to ensure that the collected data are valid questions and answers for the financial database.

[0057] Next, by identifying qualified structured query language statements that meet pre-set execution criteria and clearly linking "qualified natural language questions - qualified structured query language statements - qualified execution result answers," the database question-answering model can learn the complete mapping relationship from user intent to database operations during training. This relational data provides clear input-output pairs for supervised learning, enabling the database question-answering model to more accurately understand the semantic correspondence between natural language and SQL, reducing ambiguity during training, thereby accelerating the convergence of the database question-answering model and improving conversion accuracy.

[0058] While traditional annotation requires manual SQL writing and result verification, which is costly and error-prone, the aforementioned embodiment of this application uses historical production environment information to directly capture actual user questions and developer responses, then correlates the execution results to automatically generate some annotation data. For example, a user's query about an account balance is extracted from customer service records, correlated with the SQL actually executed by the system and the returned results, and formed a "user question - standard SQL - execution result" triple. This reduces the workload of manual annotation while ensuring data consistency with real business scenarios.

[0059] Alternatively, as Figure 2 As shown, in step 102, the step of "associating a qualified structured query language statement that meets a preset execution standard, a qualified execution result answer obtained after the qualified structured query language statement is executed by a database executor, and a qualified natural language question corresponding to the qualified structured query language statement to generate training labeled data" specifically includes:

[0060] Step 1021: Collect historical real answers corresponding to the natural language question.

[0061] In step 1022, the structured query language statement is sent to the database executor. When the sent structured query language statement can be executed by the database executor, an execution result answer is obtained based on the execution result of the executable structured query language statement after being executed by the database executor.

[0062] Step 1023 , compare the vector similarity between the execution result answer and the historical real answer, and determine the qualified execution result answer that meets the preset vector similarity comparison rules.

[0063] Step 1024 , associate the standard-compliant execution result answer, the standard-compliant structured query language statement corresponding to the standard-compliant execution result answer, and the standard-compliant natural language question to generate labeled data for training.

[0064] In the above embodiments of the present application, Figure 3As shown in the figure, during data collection, the Deepseek-R1 model can be used to initially annotate natural language questions posed by users (e.g., "Query the cumulative returns of a client's high-risk fund over the past year") and generate initial SQL. This process is based on the Deepseek-R1 model's understanding of financial terms (e.g., "high-risk fund," "cumulative returns") and database structure.

[0065] Next, the generated SQL is sent to the database executor. If the statement is executable (e.g., the syntax is correct and the table associations are valid), it is executed and the results are obtained, generating a response (such as specific profit values ​​and transaction records).

[0066] If an SQL statement fails to execute (e.g., due to incorrect field names or misused aggregate functions), the Deepseek-R1 model analyzes the error (e.g., checking for missing JOIN conditions or incorrect use of WHERE clauses) and regenerates a corrected SQL statement. For example, if a user's question involves joining multiple tables, the model may fail due to incorrectly joining the "Fund Information Table" and "Transaction Record Table," requiring adjustments to the JOIN logic.

[0067] For complex errors that the Deepseek-R1 model cannot automatically correct (such as business rule conflicts), manual intervention is required to correct them. For example, if the SQL violates financial compliance requirements (such as not filtering sensitive customer data), the query conditions must be manually adjusted.

[0068] Next, the SQL statement is compared with the actual historical answer (e.g., "Cumulative profit is 15%") for vector similarity. If the similarity falls below a threshold (e.g., the semantics differ significantly), the SQL statement may not accurately reflect the user's intent (e.g., it fails to consider dividend reinvestment).

[0069] The Deepseek-R1 model then rechecks the SQL query to ensure it matches the financial semantics of the user's question. For example, if a user asks about "risk assessment," the model might mistakenly look up the "Customer Assets Table" and ignore the "Risk Assessment Record Table." This requires re-association with the correct table and adjustment of the WHERE clause.

[0070] The verified qualified SQL (qualified structured query language statement), execution result answer (qualified execution result answer) and original question (qualified natural language question) are associated to form labeled data for training.

[0071] Step 103 : collect simulated derivative questions that may be raised for the database, and use the simulated derivative questions as unlabeled data for training, wherein the simulated derivative questions are in natural language form.

[0072] In the above embodiments of the present application, a large number of diverse natural language queries can be generated by simulating derivative questions, covering various query scenarios of the database (such as single-table query, multi-table association, conditional screening, aggregate calculation, etc.). These data do not require manual annotation of SQL or execution results, and can be used as non-labeled data for training for pre-training or semi-supervised learning, which can increase the amount of training data. For example, in the financial field, simulation questions such as "querying the returns of all high-risk financial products of a certain customer in the past year" can be generated, covering multiple types of assets such as funds, stocks, and bonds, so that the model can learn a wider range of financial query patterns, reduce dependence on specific table structures, and improve generalization capabilities.

[0073] Optionally, the database corresponds to database structure information. Regarding step 103 of "collecting simulation-derived questions that may be raised about the database", the steps specifically include:

[0074] Step 1031 , by parsing the open source data set corresponding to the open source database, obtain simulated derivative questions that may be raised for the database, wherein the open source data set includes database structure information of the open source database and natural language questions raised for the open source database.

[0075] Step 1032, and / or utilizing the text generation capability of the deep learning model, automatically generates simulated derivative questions that conform to the database semantics based on the database structure information of the database, wherein the database structure information includes at least one of a table name, a field name, and a data relationship, and the types of simulated derivative questions include at least one of a single-table query, a multi-table association query, and a conditional screening.

[0076] In the above embodiment of the present application, collecting simulation-derived questions that may be raised about the database can be achieved through the following ways:

[0077] Leverage open-source resources to extract questions. Specifically, you can obtain ready-made data from community-shared open-source datasets. For example, publicly available text2sql datasets typically contain database structure information (such as table names, field names, and data relationships) along with the corresponding natural language questions. By parsing such datasets, you can directly extract natural language questions related to a specific database, which can be used as a source for modeling derived questions.

[0078] And / or automatically generate questions based on the database structure. Specifically, for databases, simulated questions can be generated with the help of automated tools. For example, the text generation capabilities of deep learning models (such as deepseek-R1) can be used to automatically generate questions that conform to the database's semantics based on the database's structural information (such as table structure, field types, and associations). These generated questions must cover all dimensions of the database, including single-table queries, multi-table association queries, and conditional screening to ensure the diversity and comprehensiveness of the questions. The generated questions must be screened and cleaned to remove duplicate, invalid, or semantically ambiguous questions, ultimately forming a high-quality set of simulated derivative questions.

[0079] Step 104, based on the GRPO reinforcement learning framework and the scoring reward function provided by the dual-tower model, the scoring reward function is trained using the training labeled data, and the database question-answering model is supervised and fine-tuned using the training labeled data. The database question-answering model after supervised fine-tuning training is continued to be trained in combination with the training unlabeled data, format rewards, executable rewards and scoring rewards corresponding to the scoring reward function. The database question-answering model is established based on a neural network, and the trained database question-answering model outputs structured query language statements by inputting natural language questions raised by the user. The output structured query language statements are executed by the database executor and the execution result is returned to the user.

[0080] In the above embodiment of the present application, after preliminary data preparation, a small amount of high-quality labeled data (for example, about 1.8k) can be obtained. Then, a dual-tower model is first trained as part of the reward function. Its input is the user's question (natural original question) and the execution result answer obtained after the structured query language statement output by the database question-answering model is executed. At the end of the dual-tower model, the matching score of the low-dimensional dense vector (Embedding) encoded by the two towers is calculated by dot product, cosine similarity, etc. If the execution result answer finally returned to the user can perfectly match the user's question, a high score is recorded. The role of the dual-tower model is to serve as a scoring reward function, based on a positive reward for high-quality answers that can match the user's question, guiding the training model to converge in this direction.

[0081] Then, formal training of the model (a database question-answering model) began. First, one or two rounds of supervised fine-tuning (SFT) were performed as a cold start step to allow the model to learn to think about the output pattern of generated SQL statements. Subsequently, based on the large amount of unlabeled training data collected, two additional reward functions were constructed in addition to the scoring reward: a format reward and an executable reward. These functions guided the model to converge toward the "think-SQL" format and output executable SQL statements. Finally, GRPO reinforcement learning training was completed.

[0082] In particular, the Dual-Tower Model is a deep learning architecture widely used in fields such as recommendation systems, natural language processing (NLP), and search. Its core idea is to encode the features of different modalities or entities through two independent neural networks ("dual towers"), and then achieve matching or sorting tasks through similarity calculation. The model is trained by optimizing the similarity between users and positive samples while reducing the similarity with negative samples. The loss functions used in the above embodiments of the present application may include cross entropy loss (Softmax with Negative Sampling), Triplet Loss, and contrastive learning loss (such as InfoNCE).

[0083] Optionally, the dual-tower model includes a first encoding tower and a second encoding tower. Regarding step 104, "training the scoring reward function using the training labeled data" specifically includes:

[0084] Step 1041, when the scoring reward function is trained using the training labeled data, the first encoding tower is used to encode the qualified natural language question to obtain a low-dimensional dense vector of the question, and the second encoding tower is used to encode the qualified execution result answer to obtain a low-dimensional dense vector of the answer.

[0085] Step 1042 , calculate the degree of matching between the question low-dimensional dense vector and the answer low-dimensional dense vector, and obtain a matching score based on the matching degree calculation result.

[0086] Step 1043, construct a loss function with the goal of obtaining a high matching score for a correct question-answer pair and a low matching score for an incorrect question-answer pair, and incorporate the matching score into the calculation of the loss function so that the twin-tower model adjusts its own parameters according to the calculated matching score, wherein the question-answer pair includes a qualified natural language question and a qualified execution result answer corresponding to the qualified natural language question.

[0087] In the above embodiment of the present application, in the process of training the scoring reward function using the training labeled data, specifically, a natural language question, such as "What is the cumulative income of a customer from all financial products in the past year?"

[0088] At this time, the first encoding tower performs semantic encoding on the question to generate a low-dimensional dense vector of the question. The low-dimensional dense vector of the question needs to capture financial terms (such as "financial products" and "cumulative returns"), time range ("past year") and user intention (query returns).

[0089] Next, input the execution result answer corresponding to the question (such as "The customer's cumulative return is 15%, mainly from high-risk fund A and bond B"), and the second encoding tower encodes the answer to generate a low-dimensional dense vector of the answer. The low-dimensional dense vector of the answer must contain key financial information (such as yield, product name, risk level) and the answer structure (value + explanation).

[0090] Next, the cosine similarity or dot product between the question vector and the answer vector is calculated to measure the degree of semantic match. For example, if the question focuses on "cumulative returns" and the answer includes a specific value (15%) and a product breakdown (Fund A, Bond B), the match should be high. If the answer deviates from the topic (such as simply returning the account balance), the match is low.

[0091] The output matching score can be set between 0 and 1, for example, 0.85 for a correct question and answer and 0.20 for an incorrect question and answer.

[0092] Next, we construct a loss function. The goal is to maximize the matching score of the correct question-answer pairs and minimize the score of the incorrect question-answer pairs. For example, the loss function is:

[0093]

[0094] Among them, τ represents the loss function, s i Score the matching degree of the i-th question and answer pair, y i ∈{0,1} is the label (1 means correct, 0 means wrong), and N is the total number of question and answer pairs.

[0095] During model training, backpropagation is used to adjust the twin-tower model parameters, bringing the vectors of correct question-answer pairs closer together and incorrect pairs further apart. Specifically, when a user asks a natural language question about fund returns, such as "How has the fund I bought performed in the past three months?" a correct answer might be, "The fund's return over the past three months was 8.5%, outperforming the 6.2% average for its peers, primarily due to a rise in technology stocks." An incorrect answer might be, "Your account balance is 100,000 yuan." (This is off-topic). To this end, the model can learn through a loss function that correct answers must include return, a benchmark, and attribution analysis, while incorrect answers are penalized with low scores for not mentioning the keywords "return" or "fund."

[0096] In summary, ensuring that answers directly match user intent (e.g., differentiating between "return inquiry" and "balance inquiry") improves query relevance. Furthermore, combining the Twin Towers model with a scoring reward function enables a shift from "being able to answer" to "accurately answering." This improves user experience and trust, particularly when handling complex financial queries (e.g., comparing returns across multiple products and assessing risk).

[0097] Optionally, in step 104, “performing supervised fine-tuning training on the database question answering model using the training labeled data” specifically includes:

[0098] Step 1044: For the qualified natural language questions in the training labeled data, obtain the output structured query language statements output by the database question answering model for the qualified natural language questions.

[0099] Step 1045, by minimizing the difference between the output structured query language statement and the qualified structured query language statement corresponding to the qualified natural language question, update the parameters of the database question answering model until the database question answering model forms an input and output mode for converting the natural language question into a structured query language statement.

[0100] In the above embodiment of the present application, for a qualified natural language question in the training annotated data, an output structured query language statement output by the database question answering model for the qualified natural language question is obtained. The data in the training annotated data includes triples:

[0101] Qualified natural language questions: such as "Query the cumulative returns of a client's high-risk funds over the past year."

[0102] Comply with the standard Structured Query Language statement;

[0103] Answer to the results of the execution that meets the standards: For example, "The client's cumulative return is 15%, mainly from Fund A."

[0104] Next, supervised fine-tuning training is performed, where a qualified natural language question (e.g., "What was the return on a certain client's high-risk fund over the past year") is input and the expected output is set, i.e., the corresponding qualified structured query language statement (e.g., the SELECT statement above). The specific training process is as follows:

[0105] 1. Forward propagation: The model encodes the input question and generates prediction SQL.

[0106] 2. Loss calculation: Compare the differences between the predicted SQL and the expected SQL (such as field names and conditional logic). Cross entropy loss or tree structure loss (for SQL syntax trees) is commonly used.

[0107] 3. Backpropagation: Update model parameters (such as Transformer’s attention weights) to narrow the gap between prediction and expectation.

[0108] 4. Iterative optimization: Repeat training until the model can stably generate correct SQL (e.g., the validation set accuracy is > 95% for 5 consecutive epochs).

[0109] At the same time, multi-table association learning can also be performed. For example, the training data needs to cover multi-table queries (such as associating customer tables, fund tables, and transaction tables). The model needs to learn to generate JOIN statements, as well as aggregate functions and conditional logic, such as distinguishing between "cumulative return" (SUM) and "average return" (AVG), and handling conditions such as "risk level = 'high'".

[0110] In addition, compliance constraints can also be performed, such as adding regularization terms to the loss function to avoid generating SQL that violates financial regulations (such as unmasked customer ID queries).

[0111] Optionally, in step 104, "continuing to train the database question answering model after supervised fine-tuning training by combining the unlabeled training data and the format reward" specifically includes:

[0112] Step 1046 , input the simulated derived questions in the training unlabeled data into the database question answering model;

[0113] Step 1047, in the process of the database question-answering model generating output based on the input simulated derivative question, a format reward mechanism is introduced so that the database question-answering model, under the guidance of the format reward mechanism, first outputs the thinking process for the simulated derivative question and generates a structured query language statement based on the thinking process, wherein the thinking process includes at least one of the extraction of key information of the simulated derivative question and semantic understanding.

[0114] In the above embodiment of the present application, simulated derived questions from the unlabeled training data are input into the database question answering model. For example, the simulated derived questions are:

[0115] Simulation derivative question 1: "Query the changing trend of a company's debt-to-asset ratio over the past three years."

[0116] Simulation derivative question 2: "Calculate the interest rate distribution of all outstanding loans of a customer."

[0117] The above questions are generated based on financial business logic and do not require manual SQL annotation or execution results, but cover core scenarios such as financial statement analysis and customer credit management.

[0118] Next, irrelevant symbols or redundant information (such as colloquial expressions such as "Please help me check") can be removed, and financial terms (such as "asset-liability ratio" and "outstanding loans") can be mapped into representations that the model can understand (such as word vectors or subword units).

[0119] In the process of model-generated SQL, a format reward mechanism is introduced to encourage the model to output the thinking process first and then generate structured query language statements. The specific process is as follows:

[0120] 1. Thinking process generation:

[0121] Key information extraction: The model identifies the key information in the problem (such as the time range "past three years" and the indicator "liability-to-asset ratio").

[0122] Semantic understanding: The model interprets the intent of the question (e.g., analyzing trends, statistical distributions).

[0123] Example output is:

[0124] Thought Process:

[0125] 1. Extraction time range: 2021-2023;

[0126] 2. Extract indicators: debt-to-asset ratio;

[0127] 3. Understand the intention: analyze the trend of change.

[0128] Next, combined with the reward rules implemented by the format reward mechanism, the database question-answering model generates SQL based on the thinking process, for example:

[0129] If the thinking process is complete and accurate (such as correctly extracting time frames and indicators), high rewards will be given.

[0130] If the SQL statement is consistent with the thinking process (such as GROUP BY and ORDER BY are consistent with the thinking process), additional rewards will be given.

[0131] Next, adjust the loss function by adding a format reward term to the original supervised fine-tuning loss function.

[0132] To this end, by combining training with unlabeled data and a format reward mechanism, the practicality of the database question-answering model can be improved, so that the model can not only generate accurate SQL but also demonstrate reasoning logic through the thinking process.

[0133] Optionally, in step 104, "continuing to train the database question answering model after supervised fine-tuning training by combining unlabeled training data and executable rewards" specifically includes:

[0134] Step 1048: Input the simulated derived questions in the training unlabeled data into the database question answering model.

[0135] Step 1049, in the process of the database question-answering model generating output based on the input simulated derivative questions, an executable reward mechanism is introduced so that the database question-answering model generates executable structured query language statements for the simulated derivative questions under the guidance of the executable reward mechanism.

[0136] In the above embodiment of the present application, simulated derivative questions from the training unlabeled data are input into the database question-answering model. Then, colloquial expressions or irrelevant information (such as "Please help me check it") are removed. In the process of model generating SQL, an executable reward mechanism is introduced to encourage the model to generate SQL statements that can actually be executed in the database, specifically including:

[0137] 1. SQL syntax check:

[0138] The SQL generated by the model must conform to the syntax rules of the database (such as MySQL, Oracle, etc.). For example, avoid using functions or reserved words that are not supported by the database.

[0139] 2. Table and field existence verification:

[0140] Make sure that the tables (such as credit_card_records, loan_applications) and fields (such as overdue_count, approval_status) referenced in the SQL actually exist in the database.

[0141] 3. Reasonableness of execution results:

[0142] The generated SQL should be able to return reasonable results (such as the number of overdue payments is a non-negative integer, and the approval rate is a floating point number between 0 and 1).

[0143] The executable reward mechanism can be implemented by setting reward rules, specifically:

[0144] If the SQL syntax is correct, a basic reward will be given.

[0145] If the tables and fields referenced by the SQL exist, additional rewards will be given.

[0146] If the execution result complies with the business logic (such as the number of overdue payments is an integer), a higher reward will be given.

[0147] Next, adjust the loss function, that is, add an executable reward term to the original supervised fine-tuning loss function.

[0148] Finally, the model outputs SQL as follows:

[0149] 1. Problem understanding: The model first parses the input simulation derivative problem and extracts key information (such as time range, indicators, and conditions).

[0150] 2. SQL generation: The model generates SQL based on the executable reward mechanism, for example:

[0151] To this end, an executable reward mechanism makes model-generated SQL more likely to execute successfully directly in the database, reducing manual debugging costs. Combining training with unlabeled data and an executable reward mechanism significantly improves the practicality of database question-answering models. The model not only generates syntactically correct SQL but also ensures that the SQL is actually executable in the database, meeting the stringent accuracy, operability, and compliance requirements of the financial industry. This approach is particularly suitable for data-sensitive and expensive annotation scenarios in the financial sector, enabling efficient and cost-effective model optimization through an executable reward mechanism.

[0152] By applying the technical solution of this embodiment, since a single task is to be completed, model selection does not require too many parameters. After experiments, for example, a model with a volume of less than 7B can be selected to achieve the effect of a large-volume general model, which can reduce the training and deployment costs of developers.

[0153] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a database question answering model training device, such as Figure 4 As shown, the device includes:

[0154] A natural language question collection module 201 is used to collect natural language questions raised by users with respect to a database, wherein the database corresponds to a database executor;

[0155] The training labeled data generation module 202 is configured to convert the natural language question into a structured query language statement, and associate a qualified structured query language statement that meets a preset execution standard, a qualified execution result answer obtained after the qualified structured query language statement is executed by a database executor, with the qualified natural language question corresponding to the qualified structured query language statement, to generate training labeled data;

[0156] The training unlabeled data generation module 203 is used to collect simulated derived questions that may be raised for the database and use the simulated derived questions as training unlabeled data, wherein the simulated derived questions are in natural language form;

[0157] The database question-answering model training module 204 is used to train the scoring reward function based on the GRPO reinforcement learning framework and the scoring reward function provided by the twin-tower model using the training labeled data, and to perform supervised fine-tuning training on the database question-answering model using the training labeled data. The database question-answering model after supervised fine-tuning training is continued to be trained in combination with the training unlabeled data, format rewards, executable rewards and scoring rewards corresponding to the scoring reward function. The database question-answering model is established based on a neural network. The trained database question-answering model outputs structured query language statements by inputting natural language questions raised by the user. The output structured query language statements are executed by the database executor and the execution result is returned to the user.

[0158] Optionally, the training labeled data generating module 202 is further configured to:

[0159] Collecting historical real answers corresponding to the natural language questions;

[0160] Sending a structured query language statement to a database executor, and when the sent structured query language statement is executable by the database executor, obtaining an execution result answer based on an execution result of the executable structured query language statement after being executed by the database executor;

[0161] Comparing the vector similarity between the execution result answer and the historical real answer, and determining the qualified execution result answer that meets the preset vector similarity comparison rules;

[0162] The standard-reaching execution result answer, the standard-reaching structured query language statement corresponding to the standard-reaching execution result answer, and the standard-reaching natural language question are associated to generate labeled data for training.

[0163] Optionally, the dual-tower model includes a first encoding tower and a second encoding tower, and the database question-answering model training module 204 is further configured to:

[0164] When the scoring reward function is trained using the training labeled data, the first encoding tower encodes the qualified natural language question to obtain a low-dimensional dense vector of the question, and the second encoding tower encodes the qualified execution result answer to obtain a low-dimensional dense vector of the answer;

[0165] Calculate the degree of match between the low-dimensional dense vector of the question and the low-dimensional dense vector of the answer, and obtain a match score based on the result of the match calculation;

[0166] A loss function is constructed with the goal of obtaining a high matching score for a correct question-answer pair and a low matching score for an incorrect question-answer pair, and the matching score is incorporated into the calculation of the loss function so that the twin-tower model adjusts its own parameters according to the calculated matching score, wherein the question-answer pair includes a qualified natural language question and a qualified execution result answer corresponding to the qualified natural language question.

[0167] Optionally, the database question-answering model training module 204 is further configured to:

[0168] For qualified natural language questions in the training labeled data, obtaining output structured query language statements output by the database question answering model for the qualified natural language questions;

[0169] By minimizing the difference between the output structured query language statement and the qualified structured query language statement corresponding to the qualified natural language question, the parameters of the database question answering model are updated until the database question answering model forms an input-output mode for converting the natural language question into the structured query language statement.

[0170] Optionally, the database question-answering model training module 204 is further configured to:

[0171] Feed the database question answering model with simulated derived questions from the training unlabeled data;

[0172] In the process of the database question-answering model generating output based on the input simulated derivative questions, a format reward mechanism is introduced so that the database question-answering model, under the guidance of the format reward mechanism, first outputs the thinking process for the simulated derivative questions and generates a structured query language statement based on the thinking process, wherein the thinking process includes at least one of the extraction of key information of the simulated derivative questions and semantic understanding.

[0173] Optionally, the database question-answering model training module 204 is further configured to:

[0174] Feed the database question answering model with simulated derived questions from the training unlabeled data;

[0175] In the process of the database question-answering model generating output based on the input simulated derivative questions, an executable reward mechanism is introduced so that the database question-answering model can generate executable structured query language statements for the simulated derivative questions under the guidance of the executable reward mechanism.

[0176] Optionally, the database corresponds to database structure information, and the training non-labeled data generation module 203 is further used to:

[0177] By parsing the open source data set corresponding to the open source database, simulated derivative questions that may be raised about the database are obtained, wherein the open source data set includes database structure information of the open source database and natural language questions raised about the open source database;

[0178] And / or utilize the text generation capability of the deep learning model to automatically generate simulated derivative questions that conform to the database semantics based on the database structure information of the database, wherein the database structure information includes at least one of a table name, a field name, and a data relationship, and the types of simulated derivative questions include at least one of a single-table query, a multi-table association query, and a conditional screening.

[0179] It should be noted that for other corresponding descriptions of the functional units involved in the database question answering model training device provided in the embodiment of the present application, please refer to Figure 1 The corresponding description in the method will not be repeated here.

[0180] Based on the above Figure 1 The method shown in FIG. 1 is a method for performing the above-mentioned operation. Accordingly, the embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned operation is performed. Figure 1 The database question answering model training method shown.

[0181] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0182] Based on the above Figure 1 The method shown, and Figure 4 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a computer device, which can be a personal computer, a server, a network device, etc. The computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 The database question answering model training method shown.

[0183] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a Wi-Fi interface), etc.

[0184] Those skilled in the art will understand that the computer device structure provided in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0185] The storage medium may also include an operating system and a network communication module. An operating system is a program that manages and stores the hardware and software resources of a computer device, supporting the execution of information processing programs and other software and / or programs. The network communication module facilitates communication between components within the storage medium, as well as with other hardware and software within the physical device.

[0186] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by means of hardware to associate qualified structured query language statements, qualified execution result answers and qualified natural language questions to generate labeled data for training; collect simulated derivative questions that may be raised for the database to obtain unlabeled data for training. Based on the GRPO reinforcement learning framework, and taking the scoring reward function provided by the twin-tower model as the basis, the scoring reward function is trained using labeled training data, and the database question-answering model is supervised and fine-tuned, and the database question-answering model after supervised fine-tuning training is continued to be trained in combination with unlabeled training data, format rewards, executable rewards and scoring rewards of the scoring reward function. Preliminary training with a small amount of labeled data, followed by subsequent training with unlabeled training data, can stimulate the reasoning ability of the model, reduce labeling costs and improve training efficiency.

[0187] Those skilled in the art will understand that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application. Those skilled in the art will understand that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0188] The serial numbers of the above application are for descriptive purposes only and do not represent the advantages or disadvantages of the implementation scenarios. The above disclosures are only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any changes that can be made by those skilled in the art should fall within the scope of protection of the present application.

Claims

1. A database question answering model training method, characterized in that: The method comprises: Collecting natural language questions raised by users with respect to a database, wherein the database corresponds to a database executor; Converting the natural language question into a structured query language statement, and associating a qualified structured query language statement that meets a preset execution standard, a qualified execution result answer obtained after the qualified structured query language statement is executed by a database executor, and the qualified natural language question corresponding to the qualified structured query language statement to generate labeled data for training; Collecting simulated derived questions that may be asked about the database, and using the simulated derived questions as unlabeled data for training, wherein the simulated derived questions are in natural language form; Based on the GRPO reinforcement learning framework and the scoring reward function provided by the dual-tower model, the scoring reward function is trained using labeled training data, and the database question-answering model is supervised and fine-tuned using labeled training data. Combined with the training unlabeled data, format rewards, executable rewards, and scoring rewards corresponding to the scoring reward function, the database question-answering model after supervised fine-tuning training is continued to be trained. The database question-answering model is established based on a neural network. The trained database question-answering model inputs natural language questions raised by users and outputs structured query language statements. The output structured query language statements are executed by the database executor and the execution results are returned to the user.

2. The method according to claim 1, characterized in that The step of associating a qualified structured query language statement that meets a preset execution standard, a qualified execution result answer obtained after the qualified structured query language statement is executed by a database executor, and a qualified natural language question corresponding to the qualified structured query language statement to generate labeled data for training includes: Collecting historical real answers corresponding to the natural language questions; Sending a structured query language statement to a database executor, and when the sent structured query language statement is executable by the database executor, obtaining an execution result answer based on an execution result of the executable structured query language statement after being executed by the database executor; Comparing the vector similarity between the execution result answer and the historical real answer, and determining the qualified execution result answer that meets the preset vector similarity comparison rules; The standard-reaching execution result answer, the standard-reaching structured query language statement corresponding to the standard-reaching execution result answer, and the standard-reaching natural language question are associated to generate labeled data for training.

3. The method according to claim 1, characterized in that The dual-tower model includes a first encoding tower and a second encoding tower. The training of the scoring reward function using the training labeled data includes: When the scoring reward function is trained using the training labeled data, the first encoding tower encodes the qualified natural language question to obtain a low-dimensional dense vector of the question, and the second encoding tower encodes the qualified execution result answer to obtain a low-dimensional dense vector of the answer; Calculate the degree of match between the low-dimensional dense vector of the question and the low-dimensional dense vector of the answer, and obtain a match score based on the result of the match calculation; A loss function is constructed with the goal of obtaining a high matching score for a correct question-answer pair and a low matching score for an incorrect question-answer pair, and the matching score is incorporated into the calculation of the loss function so that the twin-tower model adjusts its own parameters according to the calculated matching score, wherein the question-answer pair includes a qualified natural language question and a qualified execution result answer corresponding to the qualified natural language question.

4. The method according to claim 1, wherein The supervised fine-tuning training of the database question answering model using the training labeled data includes: For qualified natural language questions in the training labeled data, obtaining output structured query language statements output by the database question answering model for the qualified natural language questions; By minimizing the difference between the output structured query language statement and the qualified structured query language statement corresponding to the qualified natural language question, the parameters of the database question answering model are updated until the database question answering model forms an input-output mode for converting the natural language question into the structured query language statement.

5. The method according to claim 1, wherein Combined with unlabeled training data and format rewards, the supervised fine-tuning database question answering model is further trained, including: Feed the database question answering model with simulated derived questions from the training unlabeled data; In the process of the database question-answering model generating output based on the input simulated derivative questions, a format reward mechanism is introduced so that the database question-answering model, under the guidance of the format reward mechanism, first outputs the thinking process for the simulated derivative questions and generates a structured query language statement based on the thinking process, wherein the thinking process includes at least one of the extraction of key information of the simulated derivative questions and semantic understanding.

6. The method according to claim 1, characterized in that Combined with unlabeled training data and actionable rewards, the supervised fine-tuning database question answering model is further trained, including: Feed the database question answering model with simulated derived questions from the training unlabeled data; In the process of the database question-answering model generating output based on the input simulated derivative questions, an executable reward mechanism is introduced so that the database question-answering model can generate executable structured query language statements for the simulated derivative questions under the guidance of the executable reward mechanism.

7. The method according to any one of claims 1 to 6, wherein the database corresponds to database structure information, characterized in that: The collection of simulation-derived questions that may be asked of the database includes: By parsing the open source data set corresponding to the open source database, simulated derivative questions that may be raised about the database are obtained, wherein the open source data set includes database structure information of the open source database and natural language questions raised about the open source database; And / or utilize the text generation capability of the deep learning model to automatically generate simulated derivative questions that conform to the database semantics based on the database structure information of the database, wherein the database structure information includes at least one of a table name, a field name, and a data relationship, and the types of simulated derivative questions include at least one of a single-table query, a multi-table association query, and a conditional screening.

8. A database question-answering model training device, characterized in that: The device comprises: A natural language question collection module, configured to collect natural language questions raised by users with respect to a database, wherein the database corresponds to a database executor; a training annotated data generation module configured to convert the natural language question into a structured query language statement, and to associate a qualified structured query language statement that meets a preset execution standard, a qualified execution result answer obtained after the qualified structured query language statement is executed by a database executor, and the qualified natural language question corresponding to the qualified structured query language statement, thereby generating training annotated data; a training unlabeled data generation module, configured to collect simulated derivative questions that may be asked about the database and use the simulated derivative questions as training unlabeled data, wherein the simulated derivative questions are in natural language form; The database question-answering model training module is used to train the scoring reward function based on the GRPO reinforcement learning framework and the scoring reward function provided by the dual-tower model using the training labeled data, and to perform supervised fine-tuning training on the database question-answering model using the training labeled data. The database question-answering model after supervised fine-tuning training is continued to be trained in combination with the training unlabeled data, format rewards, executable rewards and scoring rewards corresponding to the scoring reward function. The database question-answering model is established based on a neural network. The trained database question-answering model outputs structured query language statements by inputting natural language questions raised by the user. The output structured query language statements are executed by the database executor and the execution results are returned to the user.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for training a database question-answering model according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method for training a database question-answering model according to any one of claims 1 to 7 is implemented.

Images