A data expansion method based on hierarchical question generation and self-game

Abstract

The application discloses a data expansion method based on hierarchical question generation and self game, first designs a hierarchical question generation model, and predicts the next question through four steps of clause decomposition, predicted clause set generation, sub-question conversion and question combination by using a target query statement and a current query statement. On the basis of the existing multi-round medical Text-to-SQL data, new data is generated by means of the self game idea. The hierarchical question generation can better process complex SQL statements, generate more natural and accurate questions, and increase the diversity of the dialogue flow. The expansion data set generated by the self game can effectively alleviate the problem of the scarcity of multi-round Text-to-SQL data sets in the medical consultation scene, improve the generalization ability of the Text-to-SQL model after retraining, and further improve the model performance.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and relates to a data augmentation method based on hierarchical question generation and self-game theory. Background Technology

[0002] Medical databases are increasingly used in clinical practice due to their convenience. However, in most cases, querying and analyzing data requires interactive operations using specialized languages ​​(such as SQL) or tools, which presents a certain barrier for doctors. Against this backdrop, leveraging Text-to-SQL technology to assist doctors in querying databases using natural language in medical scenarios such as consultations becomes essential.

[0003] Due to the complexity of multi-turn dialogue queries, the annotation cost of multi-turn dialogue text and SQL pairs is very high, especially in professional fields such as medicine, where annotation requires trained experts. This results in a scarcity of medical-related multi-turn dialogue Text-to-SQL data. The only solution is to utilize pre-training to train a Text-to-SQL parser on existing multi-turn Text-to-SQL datasets, such as the classic SParC, CoSQL, and CHASE, and then generalize it to medical databases. However, models trained using this method exhibit poor generalization ability with limited training data. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a data augmentation method based on hierarchical question generation and self-game theory to augment the training text data of a multi-round medical Text-to-SQL model, thereby achieving more robust generalization.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] A data augmentation method based on hierarchical question generation and self-game theory includes the following steps:

[0007] Step 1. On the best available training data, pre-train an end-to-end Text-to-SQL model using the minimum negative log-likelihood estimation formula, as shown below:

[0008]

[0009] Where N represents the number of training data, and V is the sequence length, each Both represent a token in the sequence.

[0010] Step 2. Use standard SQL query statements from the dataset to create a rough slot template. Then, randomly select an unseen database, insert column names and values ​​into the corresponding slots of the template, and construct a new target SQL statement G.

[0011] Step 3. Decompose the target SQL statement into multiple clauses based on the keywords in the SQL statement. For single-level SQL statements without nesting, decompose them directly into clauses according to the keywords. For nested SQL statements, keywords such as WHERE, HAVING, GROUP_BY, LIMIT, and ORDER_BY are grouped into clauses according to their semantics. Thus, a valid SQL statement Q is decomposed into a clause sequence S.

[0012] Step 4. Enter the current query statement Q. t The query target G is decomposed into the current clause set S. t and the set of target clauses S G After comparing the differences, non-target sequences were obtained. and filling sequence Then, predict the clause sequence S of the next question according to the following rules. t+1 :

[0013] Differential sequence replacement: Selecting non-target sequences and filling sequence Replace the clause with one of the same type in the target clause; if no clause of the same type exists, delete the non-target clause.

[0014] Database conditional replacement: Selecting non-target sequences One of the clauses in the database selects column names or values ​​of the same type from the corresponding table in database D and replaces them;

[0015] Fill in the sequence insertion: Repeat the following steps until the next question's clause sequence S. t+1 Meets grammatical requirements:

[0016] 1) Take sequence S t+1 Let S be the current clause sequence. t ;

[0017] From the imputed sequence S tG A clause is randomly selected and inserted into the clause sequence S of the next question. t+1 Then, it is selected for insertion again with a 50% probability, then 25% after that, and so on.

[0018] 2) Termination condition: When the non-target sequence... and filling sequence All are empty, S t With S G If they are the same, then in the output predicted sequence St+1 Add a special stop interaction symbol.

[0019] Step 5. Segment the natural language question into sub-questions. Align the SQL clauses with the sub-questions based on string matching to obtain SQL clause-sub-question pairs. Train an end-to-end transformation model using minimizing negative log-likelihood estimation. The formula is as follows:

[0020]

[0021] Input the predicted sentence sequence S for the next question. t+1 This yields the corresponding natural language sub-question sequence for the next question. A special stop interaction symbol is retained to the end of the sequence.

[0022] Step 6. Concatenate the corresponding natural language sub-questions according to the execution order of the SQL statements to obtain the complete predicted next natural language question U. t+1 The next natural language question U t+1 The input is fed into the Text-to-SQL model trained in step 1 to generate the corresponding SQL query statement Q. t+1 .

[0023] Step 7. Convert the SQL query statement Q t+1 As input to step 3, repeat steps 3 through 6 until step 6 is concatenated into a sequence containing a special stop interaction symbol. At this point, a complete set of multi-turn dialogue question texts and SQL statement data is obtained.

[0024] Step 8. Retrieve the final natural language question from the corresponding SQL query statement Q. T The method in step 3 is used to decompose the query into a final set of query clauses S. T Compare with the target clause set S generated in step 4 G Calculate the average accuracy of the clauses and use it as the final similarity score (Q) between the question and the target query. T If the score is lower than the set accuracy threshold, the data generated in that round will be discarded.

[0025] Step 9. Repeat step 2 to construct different target SQL statements G and input them into the loop described in steps 3 to 8 to obtain a large amount of high-quality multi-round Text-to-SQL training data.

[0026] This invention has the following characteristics and beneficial effects:

[0027] By adopting the above technical solution and optimizing the hierarchical question generation technology, a hierarchical SQL-to-Question generation model is first established. Then, the self-game concept is integrated, and this model and the Text-to-SQL model are used to generate multi-turn dialogue text and SQL data through a self-game method. After filtering, high-quality augmented data can be obtained, thereby expanding the training text data of the multi-turn medical Text-to-SQL model to achieve more robust generalization. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of the data augmentation method for generating self-game based on hierarchical question generation according to the present invention.

[0030] Figure 2 To compare the performance test results of the model retrained with augmented data with the baseline model. Detailed Implementation

[0031] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings.

[0033] Conversely, this invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the invention as defined in the claims. Furthermore, to provide a better understanding of the invention, certain specific details are described in detail below. However, those skilled in the art will fully understand the invention even without these detailed descriptions.

[0034] This invention provides a data augmentation method based on hierarchical question generation of self-games, such as... Figure 1 As shown, the specific implementation steps include:

[0035] Step 1. On the best available training data, pre-train an end-to-end Text-to-SQL model using the minimum negative log-likelihood estimation formula, as shown below:

[0036]

[0037] Where N represents the number of training data, and V is the sequence length, each Both represent a token in the sequence.

[0038] Step 2. Use standard SQL query statements from the dataset to create a rough slot template. Then, randomly select an unseen database, insert column names and values ​​into the corresponding slots of the template, and construct a new target SQL statement G.

[0039] It should be noted that the method for creating a rough slot template is as follows: extract the standard SQL statements from the dataset, and select a portion of the table headers or conditions to replace with blank slots.

[0040] Step 3. Decompose the target SQL statement into multiple clauses based on the keywords in the SQL statement. For single-level SQL statements without nesting, decompose them directly into clauses according to the keywords. For nested SQL statements, keywords such as WHERE, HAVING, GROUP_BY, LIMIT, and ORDER_BY are grouped into clauses based on semantics. Thus, a valid SQL statement Q is decomposed into a set of clauses S.

[0041] Understandably, since WHERE and the SELECT clause within parentheses semantically represent a complete operation, they are grouped into the same clause. Based on this pattern, any valid SQL statement can always be composed of one or more common clause patterns from the training data.

[0042] Step 4. Enter the current query statement Q. t The query target G is decomposed into the current clause set S. t and the set of target clauses S G After comparing the differences, non-target sequences were obtained. and filling sequence Then, predict the clause sequence S of the next question according to the following rules. t+1 :

[0043] A. Differential sequence replacement: Selecting non-target sequences and filling sequence The system replaces a clause of the same type with the target clause. If no clause of the same type exists, the non-target clause is deleted. For example, if the current question is "What is the AFP value of patient Zhang 1?" and the target question is "What is the AFP value of patient 2?", the system will predict that the next sentence will replace "WHERE Name = patient 1" with "WHERE Name = patient 2".

[0044] B. Database Conditional Replacement: Selecting Non-Target Sequences A clause in the database D selects column names or values ​​of the same type from the corresponding table and replaces them; for example, in the sentence "What is the patient ID with the highest AFP?", if the clause "SELECT Id" is replaced with "SELECT Name", the next sentence will be "What is his name?".

[0045] C. Filling in the sequence: Repeat the following steps until the next question's clause sequence S. t+1 Meets grammatical requirements:

[0046] 1) Sequence S of clauses for the next question t+1 Let S be the current clause sequence. t ;

[0047] 2) From the imputed sequence Randomly select a clause and insert it into the prediction clause sequence S t+1 Then, it is selected for insertion again with a 50% probability, then 25% after that, and so on.

[0048] For example, if the current question is "How many patients are there in total?", and the target question is "How many patients have an AFP level higher than 400?", then the next sentence will be predicted to insert "WHERE AFP>400".

[0049] D. Termination condition: When the non-target sequence... and filling sequence All are empty, S t With S G If they are the same, then in the output predicted sequence S t+1 Add a special stop interaction symbol.

[0050] Understandably, rules A, B, and C above cover the four contextual dependencies in multi-turn dialogue text, including:

[0051] 1) Refinement, which means that the next question asks about the same entity as the current question, but the constraints are different;

[0052] 2) Theme-entity: This refers to the next question asking about other attributes of the entity mentioned in the current question.

[0053] 3) Theme-property, meaning the next question asks about the same property of another entity;

[0054] 4) Answer refinement, which means that the next question asks a subset of the current question.

[0055] Rule B uses information such as table names, column names, and values ​​from the database to generate clauses unrelated to the target and add them to the question to better mimic the questioning style in natural language. By adding new information, it helps the model improve its generalization ability.

[0056] Rule C will randomly add 1 to the current clause sequence from the difference sequence: Each clause, rule A, and rule C will ensure that each round of dialogue is closer to the query target G than the previous round, guaranteeing that the dialogue will extend towards the query target.

[0057] Thus, the clause sequence S for predicting the next question is obtained. t+1 .

[0058] Step 5. Segment the natural language question into sub-questions. Align the SQL clauses with the sub-questions based on string matching to obtain SQL clause-sub-question pairs. Train an end-to-end transformation model using minimizing negative log-likelihood estimation. The formula is as follows:

[0059]

[0060] Input the predicted sentence sequence S for the next question. t+1 This yields the corresponding natural language sub-question sequence for the next question. A special stop interaction symbol is retained to the end of the sequence.

[0061] Step 6. Concatenate the corresponding natural language sub-questions according to the execution order of the SQL statements to obtain the complete predicted next natural language question U. t+1 Example of the overall effect generated by layered questions: Figure 2 Then, the next natural language question U... t+1 The input is fed into the Text-to-SQL model trained in step 1 to generate the corresponding SQL query Q. t+1 .

[0062] Step 7. Convert the SQL query statement Q t+1 As input to step 3, repeat steps 3 through 6 until step 6 is concatenated into a sequence containing a special stop interaction symbol. At this point, a complete set of multi-turn dialogue question texts and SQL statement data is obtained.

[0063] Step 8. Calculate the SQL query Q corresponding to the final natural language question. T The method in step 3 is used to decompose the query into a final set of query clauses S. T Compare with the target clause set S generated in step 4 G Calculate the average accuracy of the clauses and use it as the final similarity score (Q) between the question and the target query. TIf the score is lower than the set accuracy threshold, the data generated in that round will be discarded.

[0064] Step 9. Repeat step 2 to construct different target SQL statements G and input them into the loop described in steps 3 to 8 to obtain a large amount of high-quality multi-round Text-to-SQL training data.

[0065] In this embodiment, a hierarchical question generation model is first designed. Utilizing the target query and the current query, it predicts the next question through four steps: clause decomposition, prediction clause set generation, sub-question transformation, and question combination. Based on existing multi-turn medical Text-to-SQL data, new data is generated cyclically using a self-game approach. This hierarchical question generation can better handle complex SQL statements, generate more natural and accurate questions, and increase the diversity of the dialogue flow. The expanded dataset generated through self-game effectively alleviates the scarcity of multi-turn Text-to-SQL datasets in medical consultation scenarios, improves the generalization ability of the retrained Text-to-SQL model, and thus improves model performance.

[0066] Furthermore, to demonstrate the validity of the data generated by this method, such as... Figure 2 As shown, the performance comparison of the model retrained using this method with the baseline model on different datasets is presented in two evaluation metrics: Question Match (QM) and Interaction Match (IM). The results demonstrate that retraining using data generated by the method of this invention can effectively improve the performance of multi-turn dialogue Text-to-SQL models and alleviate the data scarcity problem when applying Text-to-SQL technology to medical consultation scenarios.

[0067] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments, including components, without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A data augmentation method based on hierarchical question generation and self-game theory, characterized in that, Includes the following steps: Step 1: Obtain an existing multi-round medical Text-to-SQL dataset; Step 2: Construct new target SQL statements using standard SQL query queries from the dataset through slotting. ; Step 3: Enter the current query statement and the target SQL statement for querying Decomposed into the current clause set and target clause set After comparing the differences, non-target sequences were obtained. and filling sequence This allows us to predict the SQL clause sequence for the next question. ; In step 3, the SQL clause sequence of the next question is predicted. The method is as follows: Differential sequence replacement: Selecting non-target sequences and filling sequence Replace the clause with one of the same type in the target clause; if no clause of the same type exists, delete the non-target clause. Database conditional replacement: Selecting non-target sequences A clause from the database Replace the column names or values ​​of the same type in the corresponding table; Fill sequence insertion: Insert the next question SQL clause sequence First set it as the current clause sequence From the filling sequence Randomly select a clause to insert into the sequence Then, it is selected for insertion again with a 50% probability, then 25% probability, and so on. Termination condition: When the non-target sequence and filling sequence All are empty. and If they are the same, then output the SQL clause sequence for the next question. Add a special stop interaction symbol; Step 4: Split the natural language question into natural language sub-questions, and align the SQL clause with the natural language sub-question based on string matching to obtain SQL clause-sub-question pair data; Step 5: Using the SQL clause-subquestion pair data as input, and employing a pre-trained end-to-end transformation model, predict the SQL clause sequence for the next question. This yields the corresponding natural language sub-question sequence for the next question. ; Step 6: Based on the execution order of the SQL statements, concatenate the corresponding natural language sub-question sequences to obtain the complete predicted next natural language question. And through a pre-trained end-to-end Text-to-SQL model, the corresponding SQL query statement is generated. ; Step 7: Convert the SQL query statement As input to step 3, repeat steps 3 through 6 to obtain the final SQL query statement corresponding to the natural language question. As well as a complete set of multi-turn dialogue question texts and SQL statement data; Step 8: Calculate the SQL query statement corresponding to the final natural language question. Decompose the query into a final set of query clauses using the method in step 3. and compare with the target clause set Calculate the average accuracy of the clauses and use it as the similarity score between the final question and the target query. Based on the requirements, design an accuracy threshold and discard the data generated in that round when the similarity score is low; Step 9: Using the retained data as input, repeat the loop described in steps 2 to 8 to obtain a large amount of high-quality, multi-round Text-to-SQL training data.

2. The data augmentation method based on hierarchical question generation and self-game theory according to claim 1, characterized in that, In step 2, a new target SQL statement for querying is constructed. The method is as follows: Using standard SQL query statements from the dataset, select some table headers or conditions and replace them with blank slots to create a rough slot template. Then, randomly select an unseen database and insert the column names and values ​​from the SQL query statement into the corresponding slots of the template to obtain a new target SQL statement for the query. .

3. The data augmentation method based on hierarchical question generation and self-game theory according to claim 1, characterized in that, In step 3, the current query statement is respectively and the target SQL statement for querying As input, for single-level SQL statements without nesting, they are directly decomposed into clauses based on keywords; for nested SQL statements, they are semantically grouped into clauses based on keywords WHERE, HAVING, GROUP_BY, LIMIT, and ORDER_BY, and then decomposed into the current clause set. and target clause set .

4. The data augmentation method based on hierarchical question generation and self-game theory according to claim 1, characterized in that, Both the end-to-end conversion model and the end-to-end Text-to-SQL model are trained using the method of minimizing the negative log-likelihood estimation, as shown in the following expression: in, Indicates the amount of training data. It is the sequence length, each Both represent a token in the sequence.

5. The data augmentation method based on hierarchical question generation and self-game theory according to claim 1, characterized in that, In step 6, a special stop interaction symbol is retained at the end of the sequence.

6. The data augmentation method based on hierarchical question generation and self-game theory according to claim 1, characterized in that, In step 7, the operations of steps 3 to 6 are repeated until step 6 is assembled into a sequence containing a special stop interaction symbol.

Images