A difficulty perception and structure controllable data synthesis method and device

Abstract

The application discloses a difficulty perception and structure controllable data synthesis method and device. Abstract syntax tree analysis is performed on SQL statements in a data set, multi-dimensional structure frequency statistics are performed, complexity scores are calculated, difficulty levels are adaptively divided, and initial proportions of various structures are counted. The structure distribution of the data pool is counted in real time, the missing dimensions are screened by comparing the target distribution, the simple samples are removed, the corresponding level examples are matched to construct the prompt words, the large model is called to generate SQL question and answer candidate samples. The complexity of the candidate sentences is checked, and unqualified samples are corrected, and database executable verification and multi-model cross-reproduction verification are completed. Syntax skeleton deduplication and semantic similarity filtering are successively performed, qualified samples are stored in the database and the data pool distribution, skeleton library and semantic index are updated, and medium difficulty samples are returned as high difficulty generation materials. The application balances the difficulty and structure distribution of the data set, guarantees the syntactic and semantic diversity, and effectively improves the Text-to-SQL model training effect.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and database interaction, and in particular to a method and apparatus for data synthesis with difficulty perception and controllable structure. Background Technology

[0002] Text-to-SQL serves as a crucial bridge between natural language processing and database interaction, finding widespread application in fields such as intelligent question answering and data analysis. Related technologies utilize large-scale data synthesis, transformation-based perturbation methods, and the synergy of structure-aware evolutionary operators to construct diverse training corpora. Specifically, this technological system encompasses the entire process from natural language question parsing to structured query generation, including key stages such as semantic parsing, pattern linking, and SQL generation.

[0003] In Text-to-SQL tasks, large language models rely on extensively labeled "natural language problem-SQL statement" pairing data for training. However, existing data synthesis methods directly employ large-scale generation strategies without fundamental control over the synthesis direction. This can lead to the widespread reproduction of simple queries while complex structures become systematically scarce, or the marginal benefits of synthesized data diminish sharply, thus affecting the model's robustness in complex query and distribution shift scenarios. Specifically, existing methods lack quantitative measurements of SQL structural complexity, struggle to accurately fill structural gaps, and lack multi-dimensional quality verification mechanisms. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose a data synthesis method that is both difficulty-aware and structurally controllable.

[0006] Another objective of this invention is to provide a data synthesis device with difficulty perception and controllable structure.

[0007] The third objective of this invention is to provide a computer device.

[0008] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0009] To achieve the above objectives, a first aspect of the present invention proposes a data synthesis method with difficulty perception and controllable structure, comprising:

[0010] S1, for the dataset used for training the Text-to-SQL model, the dataset contains structured query language statements and their corresponding natural language questions. For each structured query language statement in the dataset, an abstract syntax tree is parsed. The occurrence frequency of the structured statements is counted on multiple independent structural dimensions. Based on the statistical results of each dimension, the complexity score of each statement is determined. An adaptive hierarchical algorithm is used to divide the dataset into different difficulty levels. At the same time, the initial proportion distribution of each structural dimension in the dataset is calculated. S2, recalculate the current proportion distribution of each structural dimension in the current data pool, compare it with the target proportion distribution to obtain the gap value of each dimension, select the multiple structural dimensions with positive and largest gap values ​​as the generation direction of this round, exclude simple level statements from the dataset, retrieve example statements that share with at least one target structural dimension according to the target difficulty level, construct a hint containing the target difficulty level, target structural dimension, example statements and underutilized data table information, input the large language model to generate candidate structured query language statements and their corresponding natural language question pairs; S3. Perform a complexity compliance check on the candidate structured query language statements to verify whether their complexity scores fall within the range of the target difficulty level. If they do not meet the requirements, perform structured correction. After passing the complexity check, execute the statement on the target database and verify that it returns a non-empty result. Then, use multiple different large language models to generate structured query language statements based on the same natural language problem. If the execution results of at least two models are consistent with the candidate statements, then the semantic reproducibility verification is passed. S4. For the candidate statements that have passed the verification, perform syntactic diversity filtering, extract their structural skeletons and match them with the skeleton library of accepted statements. If a match is found, discard them. Then, perform semantic diversity filtering on the candidate statements that have passed the syntactic filtering, use various different style instructions to translate the statements back into natural language expressions, select the most central and representative expression as the final natural language question, and calculate its semantic similarity with the accepted natural language questions. If it exceeds a preset threshold, discard it. Incorporate all the candidate statements that have passed the filtering and their natural language questions into the synthetic dataset, and synchronously update the component proportion distribution, skeleton library and semantic embedding index of the current data pool. Also, expand the accepted intermediate-level statements into the seed pool for subsequent example retrieval at the difficult level.

[0011] In one embodiment of the present invention, S1 includes: Join operations, conditional predicates, grouping and aggregation, subqueries / CTEs, sorting / limiting, window functions, set operations and deduplication selection; The complexity score is expressed by the formula:

[0012] Calculation, where For the construction count of the k-th dimension, .

[0013] In one embodiment of the present invention, it further includes: The adaptive stratification algorithm uses the Head / TailBreaks algorithm to adaptively stratify the complexity score sequence, filtering out the boundaries between the easy, medium and difficult layers. The upper bound of the difficult layer is determined by the interquartile range method, i.e., excluding those exceeding the limit. Extreme outliers.

[0014] In one embodiment of the present invention, S2 includes: The gap value is obtained through the formula Calculation, where For the target proportion distribution, This represents the current proportional distribution; Specifically, the three structural dimensions with the largest positive gap values ​​are selected as the generation directions for this round.

[0015] In one embodiment of the present invention, S3 includes: During the complexity compliance check, if the complexity score of the candidate structured query language statement does not fall within the target difficulty level range, a structured correction will be made, with a maximum of 3 corrections allowed. Multiple different large language models are used to generate structured query language statements based on the same natural language problem. Specifically, five different large language models are used. If the execution results of at least two models are consistent with the candidate statements, the semantic reproducibility is verified.

[0016] In one embodiment of the present invention, S4 includes: Extracting its structural skeleton specifically involves replacing table names with [TABLE], column names with [COLUMN], and literals with [VALUE] in the Structured Query Language statement; The sentence is back-translated into a natural language expression using a variety of different style instructions. Specifically, seven different large language models are used to back-translate the sentence with different style instructions. The final natural language problem is selected from eight candidates with the highest average similarity to the other seven.

[0017] In one embodiment of the present invention, it further includes: Calculate its semantic similarity with accepted natural language questions, specifically using the all-MiniLM-L6-v2 embedding model to calculate cosine similarity. If it exceeds a threshold... Then discard it.

[0018] To achieve the above objectives, a second aspect of the present invention provides a data synthesis device with difficulty perception and controllable structure, comprising: The complexity stratification module is used for the dataset used to train the Text-to-SQL model. The dataset contains structured query language statements and their corresponding natural language questions. It performs abstract syntax tree parsing on each structured query language statement in the dataset, counts the occurrence frequency of the constructs on multiple independent structural dimensions, determines the complexity score of each statement based on the statistical results of each dimension, and uses an adaptive stratification algorithm to divide the dataset into different difficulty levels. At the same time, it calculates the initial proportion distribution of each structural dimension in the dataset. The targeted sample generation module is used to recalculate the current proportion distribution of each structural dimension in the current data pool, compare it with the target proportion distribution to obtain the gap value of each dimension, select the multiple structural dimensions with positive and largest gap values ​​as the generation direction for this round, exclude simple level statements from the dataset, retrieve example statements that share at least one target structural dimension according to the target difficulty level, construct a hint containing the target difficulty level, target structural dimension, example statements and underutilized data table information, and input the large language model to generate candidate structured query language statements and their corresponding natural language question pairs. The multi-model joint verification module is used to perform complexity compliance checks on candidate structured query language statements, verifying whether their complexity scores fall within the range of the target difficulty level. If they do not meet the requirements, they are corrected in terms of structure. After passing the complexity check, the statement is executed on the target database and the return of a non-empty result is verified. Then, multiple different large language models are used to generate structured query language statements based on the same natural language problem. If the execution results of at least two models are consistent with the candidate statement, the semantic reproducibility verification is passed. The dual diversity filtering module is used to perform syntactic diversity filtering on the candidate statements that have passed the verification, extract their structural skeletons and match them with the skeleton library of accepted statements. If a match is found, the statements are discarded. Then, the candidate statements that have passed the syntactic filtering are subjected to semantic diversity filtering. The statements are translated into natural language expressions using various style instructions. The most representative expression is selected as the final natural language question, and its semantic similarity with the accepted natural language questions is calculated. If the similarity exceeds a preset threshold, the statements are discarded. All candidate statements that have passed the filtering and their natural language questions are included in the synthetic dataset. The component proportion distribution, skeleton library and semantic embedding index of the current data pool are updated synchronously. The accepted medium-level statements are expanded to the seed pool for subsequent example retrieval at the difficult level.

[0019] This invention discloses a difficulty-aware and structurally controllable data synthesis method and apparatus. Through an eight-dimensional complexity metric based on AST, adaptive difficulty hierarchies, a closed-loop synthesis pipeline driven by component gaps, and multi-stage verification and diversity filtering, it achieves accurate filling of structural gaps, provides high-quality training samples with balanced structure and adapted difficulty for Text-to-SQL models, thereby improving the model's generalization ability and execution accuracy in real database query scenarios, and fully meeting the domestic independent and controllable requirements of natural language database interaction scenarios.

[0020] To achieve the above objectives, a third aspect of this application provides a computer device, including a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for implementing the method described in the first aspect embodiment.

[0021] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect.

[0022] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0023] Figure 1 This is a flowchart of a data synthesis method with difficulty perception and controllable structure according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a data synthesis method with difficulty perception and controllable structure according to an embodiment of the present invention; Figure 3 This is a framework diagram of a data synthesis system with difficulty perception and controllable structure according to an embodiment of the present invention; Figure 4 This is a diagram showing the overall experimental results according to an embodiment of the present invention; Figure 5 This is an execution accuracy graph stratified by difficulty according to an embodiment of the present invention; Figure 6 This is a graph showing the ablation experiment results (BIRD Dev, %) according to an embodiment of the present invention; Figure 7 This is a graph showing the ablation experiment results (Spider Dev & Test, %) according to an embodiment of the present invention; Figure 8 This is a structural diagram of a data synthesis device with difficulty perception and controllable structure according to an embodiment of the present invention; Figure 9It is a computer device according to an embodiment of the present invention. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] The following description, with reference to the accompanying drawings, describes a data synthesis method and apparatus for difficulty perception and structural controllability according to an embodiment of the present invention.

[0027] Figure 1 This is a flowchart of a difficulty-aware and structure-controllable data synthesis method according to an embodiment of the present invention, such as... Figure 1 As shown, it includes: S1, for the dataset used for training the Text-to-SQL model, the dataset contains structured query language statements and their corresponding natural language questions. For each structured query language statement in the dataset, an abstract syntax tree is parsed. The occurrence frequency of the structured statements is counted on multiple independent structural dimensions. Based on the statistical results of each dimension, the complexity score of each statement is determined. An adaptive hierarchical algorithm is used to divide the dataset into different difficulty levels. At the same time, the initial proportion distribution of each structural dimension in the dataset is calculated. Furthermore, an abstract syntax tree is parsed for each Structured Query Language (SCL) statement in the dataset. The occurrence frequency of construct nodes within each independent structural dimension is counted. These dimensions cover various core syntactic components of SCL, such as join operations, conditional predicates, grouping and aggregation, subqueries and common table expressions, sorting and constraints, window functions, set operations, and deduplication selection. Based on the statistical results of each dimension, the counts of each dimension are weighted and summed using preset weights to determine a quantified complexity score for each statement. This score comprehensively reflects the complexity of the statement across multiple structural dimensions. Subsequently, an adaptive hierarchical algorithm is used to automatically partition the sequence composed of the complexity scores of all seed statements. Based on the inherent patterns of score distribution, the dataset is divided into multiple difficulty levels with different complexity ranges, such as easy, medium, and hard levels. The upper bound of the hard level can be determined using statistical methods such as the interquartile range (IMR) to exclude extreme outliers. Simultaneously, the initial proportion distribution of each structural dimension in the dataset is calculated, i.e., the ratio of the sum of construct counts for each dimension to the sum of the total counts for all dimensions. This distribution reflects the initial abundance of different structural components in the seed data. As one implementation method, after parsing the Structured Query Language statement into an abstract syntax tree, the occurrence frequency of constructs is recursively counted on eight independent structural dimensions, and a complexity score is obtained by weighting the sums with equal weights. The Head / TailBreaks algorithm is used to adaptively stratify the complexity score sequence, while simultaneously calculating the proportion distribution of each component in the seed data. The stratification results and component proportion distributions serve as the input benchmark for subsequent synthesis pipelines.

[0028] This step, through structural analysis and difficulty stratification of seed data, achieves deterministic and decomposable quantitative measurement of the complexity of Structured Query Language statements and adaptively establishes a difficulty level division. This provides a diagnostic basis for structural gaps and a difficulty control benchmark for the subsequent synthesis process, enabling data synthesis to shift from blindly pursuing scale to accurately filling structural gaps, and significantly improving the relevance and utility of the synthesized data.

[0029] S2, recalculate the current proportion distribution of each structural dimension in the current data pool, compare it with the target proportion distribution to obtain the gap value of each dimension, select the multiple structural dimensions with positive and largest gap values ​​as the generation direction of this round, exclude simple level statements from the dataset, retrieve example statements that share with at least one target structural dimension according to the target difficulty level, construct a hint containing the target difficulty level, target structural dimension, example statements and underutilized data table information, input the large language model to generate candidate structured query language statements and their corresponding natural language question pairs; Furthermore, within the current data pool, the current proportional distribution of each structural dimension is recalculated and compared dimension-by-dimensionally with the preset target proportional distribution to determine the gap value in each structural dimension. This gap value reflects the relative deficiency of the current data pool in that dimension. Based on this, multiple structural dimensions with positive and largest gap values ​​are selected as the generation directions for this round of synthesis, ensuring that the synthesis process can specifically fill structural deficiencies. In the dataset, statements already classified as simple are excluded. According to the requirements of the target difficulty level, example statements shared with at least one selected target structural dimension are retrieved from the remaining seed data. These example statements are used to provide structural references for the generation process. Subsequently, a structured hint is constructed, containing the target difficulty level and its corresponding numerical range, the selected target structural dimension, the retrieved example statements, and data table information in the database that has not been fully utilized by the seed query. This hint is input into the large language model to generate candidate structured query language statements and their corresponding natural language question pairs that meet the target structural features and difficulty requirements. As one implementation method, the current proportion distribution can be obtained by statistically counting the construction of each query in the current data pool on each structural dimension and normalizing it. The gap value is calculated by the difference between the target proportion and the current proportion, and the three dimensions with the largest difference are selected as the generation direction. The number of retrievals for the example statement can be set to two, and it is required to share at least one target structural dimension. The prompt content can further include the numerical range of the target difficulty level to enhance the accuracy of the generation.

[0030] This step achieves structural controllability in the synthesis process by dynamically assessing structural gaps and guiding the generation direction accordingly. It can effectively avoid the over-generation of simple queries, ensure balanced coverage of the synthetic data in the structural dimension, thereby improving the difficulty alignment and structural diversity of the synthetic data, and providing more targeted high-quality samples for subsequent model training.

[0031] S3. Perform a complexity compliance check on the candidate structured query language statements to verify whether their complexity scores fall within the range of the target difficulty level. If they do not meet the requirements, perform structured correction. After passing the complexity check, execute the statement on the target database and verify that it returns a non-empty result. Then, use multiple different large language models to generate structured query language statements based on the same natural language problem. If the execution results of at least two models are consistent with the candidate statements, then the semantic reproducibility verification is passed. Furthermore, the candidate structured query language statements undergo multi-stage cascading verification to ensure they meet preset complexity requirements and possess executability and semantic correctness. First, a complexity compliance check is performed, calculating the complexity score of the candidate statement based on a pre-determined complexity metric and verifying whether the score falls within the numerical range corresponding to the target difficulty level. If not, the statement is structurally modified to ensure its complexity score meets the range requirement. After passing the complexity check, the executability verification stage begins, executing the candidate statement on the target database and confirming that it returns a non-empty result set to exclude invalid statements with correct syntax but no actual query results. Subsequently, cross-model semantic reproducibility verification is performed, using multiple different large language models to independently generate structured query language statements based on the same natural language problem, and comparing the execution results of the statements generated by each model with the execution results of the candidate statement. If the execution results of at least two models are consistent with the execution results of the candidate statement, the candidate statement is deemed to have passed the semantic reproducibility verification. As one implementation method, the complexity score can be obtained by weighted summation of the construction counts of the abstract syntax tree across multiple independent structural dimensions. The target difficulty level range is predefined by an adaptive hierarchical algorithm. Structured correction can be achieved by adjusting the sentence structure through prompts in the large language model. In cross-model validation, five different large language models are used, and the condition for passing is that the execution results of at least two models are consistent with the candidate sentence, i.e., the condition is met. ,in This represents the execution result set. This two-stage cascaded verification mechanism uses lightweight complexity to filter out obviously unsuitable candidates, avoiding unnecessary multi-model inference calculations, thereby improving overall efficiency while ensuring verification effectiveness.

[0032] This step, through a cascaded design of complexity compliance checks, executability verification, and cross-model semantic reproducibility verification, effectively eliminates noisy samples with mismatched complexity, unexecutability, or semantic inconsistency, significantly improving the correctness and reliability of the synthetic data. This provides a high-quality data foundation for subsequent model training, thereby enhancing the generalization ability and robustness of the final model in complex query scenarios.

[0033] S4. For the candidate statements that have passed the verification, perform syntactic diversity filtering, extract their structural skeletons and match them with the skeleton library of accepted statements. If a match is found, discard them. Then, perform semantic diversity filtering on the candidate statements that have passed the syntactic filtering, use various different style instructions to translate the statements back into natural language expressions, select the most central and representative expression as the final natural language question, and calculate its semantic similarity with the accepted natural language questions. If it exceeds a preset threshold, discard it. Incorporate all the candidate statements that have passed the filtering and their natural language questions into the synthetic dataset, and synchronously update the component proportion distribution, skeleton library and semantic embedding index of the current data pool. Also, expand the accepted intermediate-level statements into the seed pool for subsequent example retrieval at the difficult level.

[0034] Furthermore, for the validated candidate structured query language statements, diversity filtering needs to be implemented at both the syntactic and semantic levels to prevent redundant samples in the synthetic dataset and ensure that each included statement and its corresponding natural language question can bring new structural or semantic information to the training data. At the syntactic level, an abstract structural representation of the candidate statement is extracted, that is, specific identifiers (such as table names, column names, literals, etc.) are replaced with type placeholders to obtain a feature skeleton reflecting the overall structural pattern of the statement. This skeleton is then matched with the skeleton library of accepted statements; if a match is found, it indicates that the candidate statement is structurally highly similar to existing samples and should be discarded. At the semantic level, candidate statements filtered by syntax are back-translated into natural language expressions using various styles of instructions. The statement with the highest average similarity to the remaining expressions is selected as the final natural language question to ensure its semantic central representativeness. Subsequently, a semantic embedding model is used to calculate the semantic similarity between this final natural language question and the accepted natural language questions. If the similarity exceeds a preset threshold, it indicates that the final question is too semantically close to existing samples and is discarded. Candidate statements filtered through all syntax and semantic diversity, along with their corresponding natural language questions, are formally incorporated into the synthetic dataset. Simultaneously, the component proportion distribution, skeleton library, and semantic embedding index of the current data pool are updated. Meanwhile, accepted intermediate-level statements are expanded into the seed pool for subsequent hard-level example retrieval. As one implementation method, the syntactic skeleton can be obtained by replacing table names with [TABLE], column names with [COLUMN], and literals with [VALUE]; semantic reverse translation can be completed by multiple independent large language models with different style instructions, and semantic similarity can be calculated using a Transformer-based sentence embedding model with a cosine similarity threshold of 0.9.

[0035] This step effectively avoids the repetitive accumulation of structural patterns and semantic content in synthetic data through a dual diversity filtering mechanism of syntax and semantics, ensuring the structural diversity and semantic richness of the synthetic dataset, thereby improving the data utility of subsequent model training and preventing the model from overfitting or declining generalization ability due to excessive exposure to similar samples.

[0036] Furthermore, this invention proposes a difficulty-aware and structure-controllable data synthesis pipeline. The basic idea is to achieve difficulty layering based on an eight-dimensional complexity metric using an abstract syntax tree, drive directional generation with component gap vectors, and ensure synthesis quality through multi-stage cascaded verification and dual diversity filtering. The method of this invention will be described step-by-step below; the overall computational flow and detailed framework are detailed in [link to relevant documentation]. Figure 2 and Figure 3 As shown, specifically: Step 1, Structural Analysis and Difficulty Stratification: This step is the preprocessing stage. For the dataset used to train the Text-to-SQL model (this dataset can be derived from publicly available Text-to-SQL benchmark datasets, such as Spider, BIRD, or user-accumulated historical query logs, where each data point consists of a natural language question and its corresponding executable SQL statement), this step performs a one-time structural analysis on the entire dataset. First, each SQL query is parsed into an AST. The occurrence counts are recursively counted across eight independent structural dimensions (JOIN, logical / arithmic / advanced predicates, GROUP BY / HAVING, subqueries / CTEs, ORDER BY / LIMIT, window functions, set operations, and SELECTDISTINCT). The results are then weighted equally and summed to obtain a deterministic complexity score. Simultaneously, the proportional distribution of each component in the seed data is calculated. Then, the Head / TailBreaks algorithm is used to adaptively stratify the complexity score sequence, filtering out the boundaries between Simple, Moderate, and Hard levels (the upper bound of the Hard level is determined by IQR to exclude extreme outliers). The stratification results and component proportion distribution will serve as the input benchmark for subsequent synthesis pipelines.

[0037] Further, this step is the preprocessing stage, where a one-time structural analysis is performed on all seed data, outputting the component proportion distribution and difficulty stratification results, which serve as the input benchmark for subsequent synthesis pipelines. First, each SQL query is parsed into an AST, and the occurrence frequency of the following eight structural dimensions is recursively counted: Table 1

[0038] The complexity score is defined as the weighted sum of the counts for the eight dimensions, as shown in Table 1:

[0039] in This is the construction count for the k-th dimension. Equal weights are used by default. Practitioners can adjust this according to domain requirements. This metric possesses three key properties: determinism (the same SQL always receives the same score), traceability (each score point corresponds to a specific AST node), and decomposability (the eight dimensions are independently accumulated). After calculating the complexity score of all seed data based on formula (1), the initial proportional distribution of each component in the seed set is also calculated. .

[0040] Furthermore, since the SQL complexity exhibits a heavy-tailed distribution, this invention employs Head / TailBreaks

[11] to adaptively stratify the complexity score sequence, filtering out the boundaries between the easy, medium, and difficult layers. The upper bound of the difficult layer is determined by the IQR method, i.e., excluding those exceeding the limit. Extreme outliers. Only queries from medium and hard layers are included in the seed pool. Stratification results: BIRD training set: Simple layer (5,718 entries, 61.7%), middle level (2,314 items, 25.0%), Difficult Level (1,234 entries, 13.3%), seed pool contains 3,548 entries; Spider training set: simple layers (4,173 entries, 59.9%), middle level (2,022 entries, 29.0%), Difficult Level (767 entries, 11.0%), with a total of 2,789 entries in the seed pool. The synthesis budget was set at 50% of the seed pool size, totaling 3,169 entries (1,774 BIRD entries + 1,395 Spider entries). For each synthesis request, two seed queries sharing at least one target component were retrieved from the target layer as Few-Shot examples. Accepted medium-layer synthesis samples also became Few-Shot example candidates for subsequent hard-layer synthesis, achieving cascading expansion.

[0041] Step 2, Seed Selection and Direction Control, is performed in each synthesis iteration. First, the proportions of each component are recalculated based on the current data pool (original seed data + accepted synthesis data). The gap vector is obtained by comparing it with the target distribution. Select The three components with the largest positive values ​​are selected as the generation direction for this round. Then, simple layer queries are excluded from the seed pool determined in step 1, and two Few-Shot examples shared with at least one target component are retrieved according to the target difficulty level. The LLM receives structured hints containing the target difficulty level and numerical range, the three target components, the Few-Shot examples, and a list of underutilized data tables: this leads to the generation of candidate SQL-NLQ pairs.

[0042] Furthermore, in each synthesis iteration, two operations are performed sequentially: direction control and seed selection. Direction control: for each component... The component proportions are recalculated based on the current data pool (raw seed data + accepted synthetic data):

[0043] Where N is the total number of queries in the current data pool. The construction count for the j-th query in the k-th component. The gap vector is defined as:

[0044] Target ratio Practitioners can customize this according to their deployment scenarios. (Select) The three components with the largest values ​​are selected as the generation direction for this round. Seed selection and hint construction: After excluding the simple layer queries defined in step 1, two Few-Shot examples shared with at least one target component are retrieved from the seed pool according to the target difficulty level (medium or hard). The LLM receives structured hints containing four types of constraints: (i) the target difficulty level and numerical range (e.g., medium level [5,7], hard level [8,14]); (ii) the three most lacking candidate components (the LLM evaluates whether each component is naturally applicable in a given database schema); (iii) two queries at the target difficulty level as Few-Shot examples; and (iv) a list of data tables in the database that are not fully utilized by the seed queries. The LLM generates candidate SQL-NLQ pairs based on these constraints. After the candidates are accepted... and Real-time updates create a closed-loop feedback loop. Accepted intermediate-layer synthesis samples are subsequently added to the seed pool, expanding the source of Few-Shot examples for difficult-layer synthesis.

[0045] Step 3, SQL correctness assurance, involves a two-stage cascaded verification of candidate SQL. The first stage is a complexity compliance check: verifying whether the complexity score of the generated SQL falls within the target level range. If it doesn't, structured corrections are performed (up to 3 times). Candidates that still don't meet the requirement after three attempts are discarded. The second stage is executability and semantic verification: the SQL must first be successfully executed on the target database and return a non-empty result; then, five different LLMs are used to generate SQL based on the same NLQ. At least two models must produce execution results consistent with the candidate SQL to pass.

[0046] Further, the first phase: complexity compliance check. Calculate candidate... fractions First, verify whether it falls within the target hierarchical range. If it does not, construct a structured correction suggestion, allowing a maximum of 3 corrections. Second stage: Executionability verification and cross-model semantic reproducibility verification. The execution must succeed and return a non-empty result set. Then, five independent LLMs each generate SQL based on the same NLQ, let E(SQL,DB) represent the execution result set, and the condition for success is:

[0047] This means that the execution results of at least two models must match the candidate SQL. Those that fail are returned for correction (up to three times). This two-phase design uses lightweight pre-filtering checks to avoid wasting expensive multi-model inference computations.

[0048] Step 4 ensures diversity by preventing redundancy at both syntactic and semantic levels. Syntactic level: Extract the SQL structural skeleton, replacing table names, column names, and literals with type placeholders [TABLE], [COLUMN], and [VALUE], respectively. If the skeleton matches any entry in the database, it is discarded. Semantic level: Reverse translate the validated SQL using seven different style instructions, selecting the most representative expression from eight variants as the final NLQ. Then, a sentence embedding model (all-MiniLM-L6-v2 in this invention) is used to calculate cosine similarity for semantic deduplication, with a threshold... .

[0049] Furthermore, at the syntactic level, deduplication must first be performed using the SQL skeleton. The SQL skeleton structure is defined as follows:

[0050] Table names are replaced with [TABLE], column names with [COLUMN], and literals with [VALUE]. If the skeleton matches any entry in the skeleton library, it is discarded. Then, semantic deduplication is performed: multi-style reverse translation and NLQ semantic deduplication. Seven independent LLMs use different style instructions to... The translation is reversed into natural language. The candidate with the highest average similarity to the remaining 7 out of 8 is selected as the final NLQ. The maximum cosine similarity with the accepted NLQ is calculated using the all-MiniLM-L6-v2 embedding model; if it exceeds a certain threshold... Then discard it.

[0051] Step 5, closed-loop output and feedback update: Candidates that pass all validations and diversity checks are formally included in the synthetic dataset, and three data items are updated simultaneously: NLQ embedding index (for subsequent semantic deduplication), SQL skeleton library (for subsequent syntactic deduplication), and difficulty distribution and component proportion vector. This closed-loop update mechanism ensures that each iteration is always based on the current true distribution state, sustainably guiding subsequent synthesis towards the remaining structural deficiencies. Accepted intermediate-layer synthesis samples are simultaneously included in the seed pool as candidates for Few-Shot examples of difficult-layer synthesis, achieving cascaded expansion.

[0052] Furthermore, after all candidates pass the verification, the NLQ embedded index, SQL skeleton library, and component scaling vector are updated synchronously. This ensures that the next round of gap analysis is always based on the current true distribution. Accepted intermediate-level synthetic samples are simultaneously used to expand the seed pool for Few-Shot example retrieval in difficult-level synthesis. The system operates in a closed-loop manner, with a total synthesis budget of 3,169 samples (1,774 BIRD samples + 1,395 Spider samples). To verify the effectiveness of this invention, the QLoRA fine-tuning technique was used to fine-tune the two basic models, Qwen2.5-Coder-7B-Instruct and Meta-Llama-3.1-8B-Instruct, configuring them with 4-bit NF4 quantization and low-rank dimension. Scaling parameters Training is performed for 3 epochs with a batch size of 2 and gradient accumulator steps of 8 steps (equivalent to a total batch size of 16). The learning rate is set to... It employs cosine annealing scheduling and greedy decoding during inference, running entirely on a single RTX 5090 (24GB VRAM). Comprehensive experiments were conducted on seven benchmarks, covering in-domain, robustness, and out-of-domain migration scenarios, using execution accuracy (EX) as the metric. Comparison methods included SQLFLOW, SynSQL-2.5M, and Gretel, all using the exact same QLoRA configuration and 3,169 synthetic datasets.

[0053] Furthermore, such as Figure 4As shown, the method of this invention achieves the best results on all seven benchmarks of the Qwen model and the best results on five out of seven benchmarks of the Llama model. It improves upon the best baseline by 6.0 percentage points (Qwen) on the robustness benchmark Spider-DK and outperforms SQLFLOW by 5.4 percentage points (Llama) on the out-of-domain migration benchmark ScienceBenchmark.

[0054] Furthermore, such as Figure 5 As shown, the performance improvements are systematically concentrated in the difficult levels (SpiderExtra+3.1, Hard+2.4, Medium+3.1), while the accuracy in the easy levels remains fully competitive, verifying the effect of the selective enhancement of this invention.

[0055] Furthermore, ablation experiments were conducted using Qwen2.5-Coder-7B-Instruct as the base model. Under the same data volume (3,169 synthetic samples) and QLoRA configuration, the three core modules were removed one by one for ablation experiments. The results are as follows: Figure 6 and Figure 7 As shown. All three components make significant and complementary contributions to performance: on BIRD Dev, removing the diversity guarantee (w / oDiversity) resulted in the largest performance drop ( The figure of 0.8 indicates that SQL skeleton deduplication and NLQ semantic filtering can effectively prevent template overfitting in real database scenarios. On SpiderTest, removing direction control (without DirectionControl) decreased by 3.5 percentage points, indicating that the lack of gap guidance will cause the model to regress to simple templates, and insufficient coverage of complex structures will lead to degradation of generalization ability. Removing correctness guarantee (without Correctness) decreased by 3.3 percentage points, verifying that the two-stage cascaded filtering (complexity check + cross-model semantic reproducibility voting) can effectively remove noisy samples that are "executable but semantically inconsistent". Removing diversity guarantee decreased by 4.1 percentage points (the largest among all ablation variants), further highlighting the key role of syntactic-semantic dual diversity filtering in cross-database scenarios. The three form a progressive and complementary relationship: direction control determines "what is generated", correctness guarantee determines "whether the generation is correct", and diversity guarantee determines "whether the generation is useful".

[0056] To achieve the above embodiments, such as Figure 8 As shown, this embodiment also provides a data synthesis device 10 with difficulty perception and controllable structure, including: The complexity layering module 100 is used for a dataset used for training a Text-to-SQL model. The dataset contains structured query language statements and their corresponding natural language questions. It performs abstract syntax tree parsing on each structured query language statement in the dataset, counts the occurrence frequency of the constructs on multiple independent structural dimensions, determines the complexity score of each statement based on the statistical results of each dimension, and uses an adaptive layering algorithm to divide the dataset into different difficulty levels. At the same time, it calculates the initial proportion distribution of each structural dimension in the dataset. The directional sample generation module 200 is used to recalculate the current proportion distribution of each structural dimension in the current data pool, compare it with the target proportion distribution to obtain the gap value of each dimension, select multiple structural dimensions with positive and largest gap values ​​as the generation direction for this round, exclude simple level statements from the dataset, retrieve example statements that share with at least one target structural dimension according to the target difficulty level, construct a prompt containing the target difficulty level, target structural dimension, example statements and underutilized data table information, and input the large language model to generate candidate structured query language statements and their corresponding natural language question pairs in a directional manner. The multi-model joint verification module 300 is used to perform complexity compliance checks on candidate structured query language statements, verifying whether their complexity scores fall within the range of the target difficulty level. If they do not meet the requirements, they are corrected in terms of structure. After passing the complexity check, the statement is executed on the target database and the return of a non-empty result is verified. Then, multiple different large language models are used to generate structured query language statements based on the same natural language problem. If the execution results of at least two models are consistent with the candidate statement, the semantic reproducibility verification is passed. The dual diversity filtering module 400 is used to perform syntactic diversity filtering on the candidate statements that have passed the verification, extract their structural skeletons and match them with the skeleton library of accepted statements. If a match is found, the statements are discarded. Then, the candidate statements that have passed the syntactic filtering are subjected to semantic diversity filtering. The statements are translated into natural language expressions using various style instructions. The most representative expression is selected as the final natural language question, and its semantic similarity with the accepted natural language questions is calculated. If the similarity exceeds a preset threshold, the statements are discarded. All candidate statements that have passed the filtering and their natural language questions are included in the synthetic dataset. The component proportion distribution, skeleton library and semantic embedding index of the current data pool are updated synchronously. The accepted intermediate-level statements are expanded to the seed pool for subsequent example retrieval at the difficult level.

[0057] An embodiment of the present invention provides a data synthesis device with difficulty perception and structural controllability. Through an eight-dimensional complexity measurement based on AST, adaptive difficulty layering, a closed-loop synthesis pipeline driven by component gaps, and multi-stage verification and diversity filtering, it achieves accurate filling of structural gaps and fully meets the domestic independent and controllable requirements of natural language database interaction scenarios.

[0058] To implement the methods of the above embodiments, the present invention also provides a computer device, such as... Figure 9 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein, the processor 602 reads executable program code stored in the memory 601 to run a program corresponding to the executable program code, so as to implement the various steps of the method described above.

[0059] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0060] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0061] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A data synthesis method with difficulty perception and controllable structure, characterized in that, include: S1, for the dataset used for training the Text-to-SQL model, the dataset contains structured query language statements and their corresponding natural language questions. For each structured query language statement in the dataset, an abstract syntax tree is parsed. The occurrence frequency of the structured statements is counted on multiple independent structural dimensions. Based on the statistical results of each dimension, the complexity score of each statement is determined. An adaptive hierarchical algorithm is used to divide the dataset into different difficulty levels. At the same time, the initial proportion distribution of each structural dimension in the dataset is calculated. S2, recalculate the current proportion distribution of each structural dimension in the current data pool, compare it with the target proportion distribution to obtain the gap value of each dimension, select the multiple structural dimensions with positive and largest gap values ​​as the generation direction of this round, exclude simple level statements from the dataset, retrieve example statements that share with at least one target structural dimension according to the target difficulty level, construct a hint containing the target difficulty level, target structural dimension, example statements and underutilized data table information, input the large language model to generate candidate structured query language statements and their corresponding natural language question pairs; S3. Perform a complexity compliance check on the candidate structured query language statements to verify whether their complexity scores fall within the range of the target difficulty level. If they do not meet the requirements, perform structured correction. After passing the complexity check, execute the statement on the target database and verify that it returns a non-empty result. Then, use multiple different large language models to generate structured query language statements based on the same natural language problem. If the execution results of at least two models are consistent with the candidate statements, then the semantic reproducibility verification is passed. S4. For the candidate statements that have passed the verification, perform syntactic diversity filtering, extract their structural skeletons and match them with the skeleton library of accepted statements. If a match is found, discard them. Then, perform semantic diversity filtering on the candidate statements that have passed the syntactic filtering, use various different style instructions to translate the statements back into natural language expressions, select the most central and representative expression as the final natural language question, and calculate its semantic similarity with the accepted natural language questions. If it exceeds a preset threshold, discard it. Incorporate all the candidate statements that have passed the filtering and their natural language questions into the synthetic dataset, and synchronously update the component proportion distribution, skeleton library and semantic embedding index of the current data pool. Also, expand the accepted intermediate-level statements into the seed pool for subsequent example retrieval at the difficult level.

2. The method as described in claim 1, characterized in that, S1 includes: Join operations, conditional predicates, grouping and aggregation, subqueries / CTEs, sorting / limiting, window functions, set operations and deduplication selection; The complexity score is expressed by the formula: Calculation, where For the construction count of the k-th dimension, .

3. The method as described in claim 2, characterized in that, The method further includes: The adaptive stratification algorithm uses the Head / TailBreaks algorithm to adaptively stratify the complexity score sequence, filtering out the boundaries between the easy, medium and difficult layers. The upper bound of the difficult layer is determined by the interquartile range method, i.e., excluding those exceeding the limit. Extreme outliers.

4. The method as described in claim 1, characterized in that, S2 includes: The gap value is obtained through the formula Calculation, where For the target proportion distribution, This represents the current proportional distribution; Specifically, the three structural dimensions with the largest positive gap values ​​are selected as the generation directions for this round.

5. The method as described in claim 1, characterized in that, The S3 includes: During the complexity compliance check, if the complexity score of the candidate structured query language statement does not fall within the target difficulty level range, a structured correction will be made, with a maximum of 3 corrections allowed. Multiple different large language models are used to generate structured query language statements based on the same natural language problem. Specifically, five different large language models are used. If the execution results of at least two models are consistent with the candidate statements, the semantic reproducibility is verified.

6. The method as described in claim 1, characterized in that, The S4 includes: Extracting its structural skeleton specifically involves replacing table names with [TABLE], column names with [COLUMN], and literals with [VALUE] in the Structured Query Language statement; The sentence is back-translated into a natural language expression using a variety of different style instructions. Specifically, seven different large language models are used to back-translate the sentence with different style instructions. The final natural language problem is selected from eight candidates with the highest average similarity to the other seven.

7. The method as described in claim 1, characterized in that, The method further includes: Calculate its semantic similarity with accepted natural language questions, specifically using the all-MiniLM-L6-v2 embedding model to calculate cosine similarity. If it exceeds a threshold... Then discard it.

8. A data synthesis device with difficulty perception and controllable structure, characterized in that, include: The complexity stratification module is used for the dataset used to train the Text-to-SQL model. The dataset contains structured query language statements and their corresponding natural language questions. It performs abstract syntax tree parsing on each structured query language statement in the dataset, counts the occurrence frequency of the constructs on multiple independent structural dimensions, determines the complexity score of each statement based on the statistical results of each dimension, and uses an adaptive stratification algorithm to divide the dataset into different difficulty levels. At the same time, it calculates the initial proportion distribution of each structural dimension in the dataset. The targeted sample generation module is used to recalculate the current proportion distribution of each structural dimension in the current data pool, compare it with the target proportion distribution to obtain the gap value of each dimension, select the multiple structural dimensions with positive and largest gap values ​​as the generation direction for this round, exclude simple level statements from the dataset, retrieve example statements that share at least one target structural dimension according to the target difficulty level, construct a hint containing the target difficulty level, target structural dimension, example statements and underutilized data table information, and input the large language model to generate candidate structured query language statements and their corresponding natural language question pairs. The multi-model joint verification module is used to perform complexity compliance checks on candidate structured query language statements, verifying whether their complexity scores fall within the range of the target difficulty level. If they do not meet the requirements, they are corrected in terms of structure. After passing the complexity check, the statement is executed on the target database and the return of a non-empty result is verified. Then, multiple different large language models are used to generate structured query language statements based on the same natural language problem. If the execution results of at least two models are consistent with the candidate statement, the semantic reproducibility verification is passed. The dual diversity filtering module is used to perform syntactic diversity filtering on the candidate statements that have passed the verification, extract their structural skeletons and match them with the skeleton library of accepted statements. If a match is found, the statements are discarded. Then, the candidate statements that have passed the syntactic filtering are subjected to semantic diversity filtering. The statements are translated into natural language expressions using various style instructions. The most representative expression is selected as the final natural language question, and its semantic similarity with the accepted natural language questions is calculated. If the similarity exceeds a preset threshold, the statements are discarded. All candidate statements that have passed the filtering and their natural language questions are included in the synthetic dataset. The component proportion distribution, skeleton library and semantic embedding index of the current data pool are updated synchronously. The accepted medium-level statements are expanded to the seed pool for subsequent example retrieval at the difficult level.

9. A computer device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images