Data expansion method based on hierarchical question generation and self-gaming
Through the data expansion method based on layered question generation and self-game, the problem of poor generalization ability of multiple rounds of medical Text-to-SQL models is solved, and high-quality expanded data is generated, which improves the performance of the model in medical consultation scenarios. , solves the problem of data scarcity.
Patent Information
- Application Number
- CN202411054872.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-02
AI Technical Summary
The existing multi-round medical Text-to-SQL model has poor generalization ability in medical scenarios. It is mainly due to the complexity of multi-round dialogue query and the high annotation cost, resulting in scarce multi-round dialogue data for medical-related multi-round dialogue. The existing methods are under limited training data. Insufficient generalization ability.
Using a data expansion method based on hierarchical question generation and self-game, the Text-to-SQL model is pre-trained by minimizing negative log-likelihood estimation, and a new query target is constructed using SQL statement decomposition and slot templates, combining self-game Ideas generate multiple rounds of dialogue text and SQL data, optimize the hierarchical SQL-to-Question generation model, and generate high-quality expanded data.
Effectively expand the training data of multiple rounds of medical Text-to-SQL models, improve the robust generalization ability of the model, enhance the multi-round dialogue processing capabilities in medical consultation scenarios, improve model performance, and alleviate the problem of data scarcity.
Smart Images

Figure CN119025540A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of natural language processing and relates to a data expansion method based on hierarchical question generation and self-game. Background Art
[0002] Medical databases are increasingly used in clinical practice due to their convenience. However, in most cases, data query and analysis must be performed through professional languages (such as SQL) or tools for interactive operations, which poses a certain threshold for doctors. In this context, it is necessary to use Text-to-SQL technology to assist doctors in using natural language to query databases in medical scenarios such as consultations.
[0003] Due to the complexity of multi-round dialogue queries, the annotation cost of multi-round dialogue text and SQL pairs is very high, especially in professional fields such as medicine, which requires trained experts to annotate, which makes medical-related multi-round dialogue Text-to-SQL data very scarce. We can only use the idea of pre-training to train a Text-to-SQL parser on existing multi-round Text-to-SQL datasets, such as classic SParC, CoSQL, and CHASE, and then generalize it to the database of medical scenarios. However, the model trained by this method has poor generalization ability under limited training data. Summary of the invention
[0004] In view of the shortcomings of the prior art, the present invention proposes a data expansion method based on hierarchical question generation and self-game, which expands the training text data of multiple rounds of medical Text-to-SQL model training to achieve more robust generalization.
[0005] In order to solve the above technical problems, the technical solution of the present invention is:
[0006] A data expansion method based on hierarchical question generation and self-game, comprising the following steps:
[0007] Step 1. On the currently available best quality training data, pre-train an end-to-end Text-to-SQL model by minimizing the negative log-likelihood estimate. The formula is as follows:
[0008]
[0009] Where N is the number of training data, V is the sequence length, and each Both represent a token in a sequence.
[0010] Step 2. Use the standard SQL query statements in the dataset to create a rough slot template, then randomly select an unseen database, insert the column name and value information into the slot of the template, and construct a new query target SQL statement G.
[0011] Step 3. Decompose the target SQL statement into multiple clauses according to the keywords in the SQL statement. For a single-layer SQL statement without nesting, decompose it directly into clauses according to the keywords. For nested SQL statements, assign keywords such as WHERE, HAVING, GROUP_BY, LIMIT, and ORDER_BY into a clause according to semantics. Thus, a legal SQL statement Q is decomposed into a clause sequence S.
[0012] Step 4. Enter the current query statement Q t and the query target G are decomposed into the current clause set S t and the target clause set S G , and then compare the differences to get the non-target sequence and fill sequence After that, the clause sequence S of the next question is predicted according to the following rules: t+1 :
[0013] Differential sequence replacement: Selecting non-target sequences and fill sequence Replace it with a clause of the same type in the target clause. If there is no clause of the same type, delete the non-target clause.
[0014] Database conditional replacement: Select non-target sequences A clause in , selects the column name or value of the same type from the corresponding table of database D for replacement;
[0015] Fill in the sequence insertion: loop the following steps to the clause sequence S of the next question t+1 Meets grammatical requirements:
[0016] 1) Sequence S t+1 Set as the current clause sequence S t ;
[0017] From the filling sequence S tG Randomly select a clause and insert it into the clause sequence S of the next question t+1 ; Then the probability of selecting and inserting it again is 50%, and then 25%, and so on;
[0018] 2) End condition: When non-target sequence and fill sequence All are empty, S t With S G The same, then in the output prediction sequence St+1 Add a special stop interaction symbol to .
[0019] Step 5. Split the natural language question into sub-questions, align the SQL clause with the sub-questions based on string matching, and obtain the SQL clause-sub-question pair data. The end-to-end conversion model is trained by minimizing the negative log-likelihood estimation. The formula is as follows:
[0020]
[0021] Input the predicted next question sentence sequence S t+1 , get the corresponding natural language sub-question sequence of the next question The special stop interaction symbol is reserved until the end of the sequence.
[0022] Step 6. According to the order in which the SQL statements are executed, the corresponding natural language sub-questions are concatenated to obtain the complete predicted next natural language question U t+1 . The next natural language question U t+1 Input into the Text-to-SQL model trained in step 1 to generate the corresponding SQL query statement Q t+1 .
[0023] Step 7. SQL query statement Q t+1 As the input of step 3, the operations of step 3 to step 6 are repeated until step 6 is spliced into a sequence containing a special stop interaction symbol. At this point, a complete set of multi-round dialogue question texts and SQL statement data is obtained.
[0024] Step 8. Compose the SQL query statement Q corresponding to the final natural language question T Use the method in step 3 to decompose into the final query clause set S T , compared with the target clause set S generated in step 4 G Calculate the average accuracy of the clauses and use it as the similarity score between the final question and the target query (Q T ,G), when the score is lower than the set accuracy threshold, the data generated in this round is discarded.
[0025] Step 9. Repeat step 2 to construct different query target SQL statements G, input them into the loop described in steps 3 to 8, and obtain a large amount of high-quality multi-round Text-to-SQL training data.
[0026] The present invention has the following characteristics and beneficial effects:
[0027] By adopting the above technical solution, by optimizing the hierarchical question generation technology, a hierarchical SQL-to-Question generation model is first established. Then, the self-game idea is integrated. This model is used together with the Text-to-SQL model to generate multiple rounds of dialogue text and SQL data through the self-game method. After filtering, high-quality extended data can be obtained, thereby expanding the training text data of multiple rounds of medical Text-to-SQL model training to achieve more robust generalization. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0029] Figure 1 A schematic diagram of a data expansion method based on hierarchical question generation self-game according to the present invention;
[0030] Figure 2 To compare the performance of the model retrained with enhanced data with the baseline model. DETAILED DESCRIPTION
[0031] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0032] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings.
[0033] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.
[0034] The present invention provides a data expansion method based on hierarchical question generation self-game, such as Figure 1 As shown, the specific implementation steps include:
[0035] Step 1. On the currently available best quality training data, pre-train an end-to-end Text-to-SQL model by minimizing the negative log-likelihood estimate. The formula is as follows:
[0036]
[0037] Where N is the number of training data, V is the sequence length, and each Both represent a token in a sequence.
[0038] Step 2. Use the standard SQL query statements in the dataset to create a rough slot template, then randomly select an unseen database, insert the column name and value information into the slot of the template, and construct a new query target SQL statement G.
[0039] It should be noted that the method for establishing a rough slot template is: extract the standard SQL statement in the data set, select a part of the table header or condition and replace it with a blank slot.
[0040] Step 3. Decompose the target SQL statement into multiple clauses according to the keywords in the SQL statement. For a single-layer SQL statement without nesting, decompose it directly into clauses according to the keywords. For nested SQL statements, assign keywords such as WHERE, HAVING, GROUP_BY, LIMIT, and ORDER_BY into a clause according to semantics. Thus, a legal SQL statement Q is decomposed into a set of clauses S.
[0041] It is understandable that since WHERE and the SELECT inside the parentheses semantically represent a complete operation, they are grouped into the same clause. According to this pattern, any legal SQL statement can always be composed of one or several common clause patterns in the training data.
[0042] Step 4. Enter the current query statement Q t and the query target G are decomposed into the current clause set S t and the target clause set S G , and then compare the differences to get the non-target sequence and fill sequence After that, the clause sequence S of the next question is predicted according to the following rules: t+1 :
[0043] A. Differential sequence replacement: Select non-target sequences and fill sequence If there is no clause of the same type, the non-target clause will be replaced. For example, if the current question is "What is the AFP value of patient 1?" and the target question is "What is the AFP value of patient 2?", the next sentence will be predicted to replace "WHERE Name = patient 1" with "WHERE Name = patient 2".
[0044] B. Database condition replacement: Select non-target sequences In the sentence "What is the patient ID with the highest AFP?", the entity type clause "SELECT Id" is selected and replaced with "SELECT Name", and the predicted next sentence is "What is his name?".
[0045] C. Filling sequence insertion: loop the following steps to the next question clause sequence S t+1 Meets grammatical requirements:
[0046] 1) The clause sequence S of the next question t+1 Set as the current clause sequence S t ;
[0047] 2) From the filling sequence Randomly select a clause and insert it into the prediction clause sequence S t+1 ; Then the probability of selecting and inserting it again is 50%, and then 25%, and so on;
[0048] For example, if the current question is "How many patients are there in total?" and the target question is "How many patients have an AFP higher than 400", the next sentence will be predicted to insert "WHERE AFP>400";
[0049] D. End condition: When the non-target sequence and fill sequence All are empty, S t With S G The same, then in the output prediction sequence S t+1 Add a special stop interaction symbol to .
[0050] It can be understood that the above rules A, B, and C cover four context dependencies of multi-round dialogue texts, including:
[0051] 1) Refinement, that is, the next question and the current question ask the same entity, but the constraints are different;
[0052] 2) Theme-entity, that is, the next question asks about other attributes of the entity mentioned in the current question;
[0053] 3) Theme-property, that is, the next question asks about the same property of another entity;
[0054] 4) Answer refinement, that is, the next question is a subset of the current question.
[0055] Rule B uses information such as database table names, column names, and values to generate some clauses that are irrelevant to the target and add them to the question to better imitate the way of asking questions in natural language. By adding new information, the model can improve its generalization ability.
[0056] Rule C randomly adds 1 from the difference sequence to the current clause sequence: Clauses, rules A and C will promote each round of dialogue to be closer to the query target G than the previous round, ensuring that the dialogue will extend to the query target.
[0057] Thus, we obtain the clause sequence S for predicting the next question. t+1 .
[0058] Step 5. Split the natural language question into sub-questions, align the SQL clause with the sub-questions based on string matching, and obtain the SQL clause-sub-question pair data. The end-to-end conversion model is trained by minimizing the negative log-likelihood estimation. The formula is as follows:
[0059]
[0060] Input the predicted next question sentence sequence S t+1 , get the corresponding natural language sub-question sequence of the next question The special stop interaction symbol is reserved until the end of the sequence.
[0061] Step 6. According to the order in which the SQL statements are executed, the corresponding natural language sub-questions are concatenated to obtain the complete predicted next natural language question U t+1 , the overall effect of layered question generation is as follows Figure 2 . Then the next natural language question U t+1 Input into the Text-to-SQL model trained in step 1 to generate the corresponding SQL query Q t+1 .
[0062] Step 7. SQL query statement Q t+1 As the input of step 3, the operations of step 3 to step 6 are repeated until step 6 is spliced into a sequence containing a special stop interaction symbol. At this point, a complete set of multi-round dialogue question texts and SQL statement data is obtained.
[0063] Step 8. Compose the SQL query Q corresponding to the final natural language question T Use the method in step 3 to decompose into the final query clause set S T , compared with the target clause set S generated in step 4 G Calculate the average accuracy of the clauses and use it as the similarity score between the final question and the target query (Q T,G), when the score is lower than the set accuracy threshold, the data generated in this round is discarded.
[0064] Step 9. Repeat step 2 to construct different query target SQL statements G, input them into the loop described in steps 3 to 8, and obtain a large amount of high-quality multi-round Text-to-SQL training data.
[0065] In this embodiment, a hierarchical question generation model is first designed, and the target query statement and the current query statement are used to predict the next question through four steps: clause decomposition, predicted clause set generation, sub-question conversion, and question combination. Based on the existing multi-round medical Text-to-SQL data, new data is generated cyclically with the help of the self-game idea. This hierarchical question generation can better handle complex SQL statements, generate more natural and accurate questions, and increase the diversity of dialogue flows. The expanded data set generated by self-game can effectively alleviate the problem of scarcity of multi-round Text-to-SQL data sets in medical consultation scenarios, improve the generalization ability of the retrained Text-to-SQL model, and thus improve the model performance.
[0066] Furthermore, to prove the validity of the data generated by this method, Figure 2 As shown in the figure, the performance comparison between the model retrained using this method and the baseline model on different data sets in terms of two evaluation indicators, Question Match (QM) and Interaction Match (IM). From the results, it can be seen that retraining with the data generated by the method of the present invention can effectively improve the performance of the multi-round dialogue Text-to-SQL model and alleviate the data scarcity problem when the Text-to-SQL technology is applied in medical consultation scenarios.
[0067] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.
Claims
1. A data augmentation method based on hierarchical question generation and self-game, It is characterized in that The steps include: Step 1: Obtain existing multi-round medical Text-to-SQL datasets; Step 2: Use the standard SQL query in the data set to construct a new query target SQL statement G through the slot method; Step 3: Enter the current query statement Q t and query target SQL statement G are decomposed into current clause set S t and the target clause set S G , and then compare the differences to get the non-target sequence and fill sequence Then predict the SQL clause sequence S of the next question t+1 ; Step 4: Split the natural language question into natural language sub-questions, align the SQL clause with the natural language sub-questions based on string matching, and obtain SQL clause-sub-question pair data; Step 5: Take the SQL clause-sub-question pair data as input and use the pre-trained end-to-end conversion model to predict the SQL clause sequence S of the next question. t+1 , get the corresponding natural language sub-question sequence of the next question Step 6: According to the order in which the SQL statements are executed, the corresponding natural language sub-question sequences are concatenated to obtain the complete predicted next natural language question U t+1 , and generates the corresponding SQL query statement Q through the pre-trained end-to-end Text-to-SQL model t+1 ; Step 7: SQL query statement Q t+1 As the input of step 3, repeat the operations from step 3 to step 6 to obtain the SQL query statement Q corresponding to the final natural language question T , as well as a complete set of multi-round dialogue question texts and SQL statement data; Step 8: Compose the SQL query statement Q corresponding to the final natural language question T Use the method in step 3 to decompose into the final query clause set S T , and compare it with the target clause set S G Calculate the average accuracy of the clauses and use it as the similarity score between the final question and the target query (Q T ,G), design the accuracy threshold according to the requirements, and discard the data generated in this round when the similarity score is low; Step 9: Use the retained data as input and repeat the loop described in steps 2 to 8 to obtain a large amount of high-quality multi-round Text-to-SQL training data.
2. According to claim 1, a data expansion method based on hierarchical question generation and self-game, It is characterized in that In step 2, the method for constructing a new query target SQL statement G is: Using the standard SQL query statements in the data set, some table headers or conditions are selected to replace with blank slots to establish a rough slot template. Then, an unseen database is randomly selected, and the column names and values in the SQL query statements are correspondingly inserted into the slots of the template to obtain a new query target SQL statement G.
3. According to the data expansion method based on hierarchical question generation and self-game according to claim 1, It is characterized in that In step 3, the current query statement Q t The query target SQL statement G is taken as input. For a single-layer SQL statement without nesting, it is directly decomposed into clauses according to keywords. For a nested SQL statement, the keywords WHERE, HAVING, GROUP_BY, LIMIT, and ORDER_BY are grouped into a clause according to semantics and decomposed into the current clause set S. t and the target clause set S G .
4. According to claim 1, a data expansion method based on hierarchical question generation and self-game, It is characterized in that In step 3, the SQL clause sequence S of the next question is predicted. t+1 The method is: Differential sequence replacement: Select non-target sequence S tG and fill sequence S tG Replace it with a clause of the same type in the target clause. If there is no clause of the same type, delete the non-target clause. Database condition replacement: Select non-target sequence S tG A clause in , selects the column name or value of the same type from the corresponding table of database D for replacement; Fill sequence insertion: insert the next question SQL clause sequence S t+1 First set it to the current clause sequence S t , from the filling sequence Randomly select a clause and insert it into the sequence S t+1 ; Then the probability of selecting and inserting it again is 50%, and then 25%, and so on; End condition: When non-target sequence and fill sequence All are empty, S t With S G If the same, then output the SQL clause sequence S of the next question t+1 Add a special stop interaction symbol to .
5. According to claim 1, a data expansion method based on hierarchical question generation and self-game, It is characterized in that The end-to-end conversion model and the end-to-end Text-to-SQL model are both trained by minimizing the negative log-likelihood estimation method, as shown below: Where N is the number of training data, V is the sequence length, and each Both represent a token in a sequence.
6. The data expansion method based on hierarchical question generation and self-game according to claim 1, It is characterized in that In step 6, the special stop interaction symbol is retained to the end of the sequence.
7. The data expansion method based on hierarchical question generation and self-game according to claim 1, It is characterized in that In step 7, the operations from step 3 to step 6 are repeated until step 6 is spliced to a sequence containing a special stop interaction symbol.
Citation Information
Patent Citations
Intelligent air combat imitation learning training sample generation method based on self-game model
CN114528750A
Cited By
Construction method, generation method and device of Text-to-SQL model, equipment and medium
CN119311721A
