A method and apparatus for generating SQL datasets

Abstract

The application discloses a SQL data set generation method and device, and relates to the technical field of artificial intelligence. The method comprises the following steps: performing information extraction processing on text data to obtain structured text data comprising complexity distribution, business rules and query modes; constructing a plurality of candidate question and answer pairs based on the structured text data; for each candidate question and answer pair, calculating a first quality score based on the dimension scores of the candidate SQL statement in the execution dimension, the matching dimension and the syntax dimension and the complexity level of the candidate question; calculating the confidence of the candidate SQL statement in the text unit and the clause level to obtain a second quality score; and fusing the first quality score and the second quality score to determine a target SQL data set. The application can ensure that a high-quality target SQL data set is quickly generated.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and apparatus for generating SQL datasets. Background Technology

[0002] With the rapid development of artificial intelligence and large language model technology, Text2sql technology has gained widespread attention and application in enterprise-level data analysis in industrial vertical industries. The successful implementation of Text2sql technology in industrial vertical industries requires the accumulation of large amounts of high-quality data to build and optimize algorithm strategies, among which question-and-answer data consisting of questions and their corresponding SQL statements is particularly crucial.

[0003] Traditional question-and-answer data acquisition methods heavily rely on manual annotation, which is not only costly and inefficient but also fails to meet the needs of large-scale applications. While automated methods based on large language models can generate question-and-answer data in batches, the inherent uncertainties of large language models may result in a large number of low-quality question-and-answer pairs in the generated data. Introducing this low-quality data into business processes can severely impact model performance and the final application effect. Therefore, there is an urgent need for a method that can quickly generate large amounts of question-and-answer data and also systematically evaluate it to achieve automated production of high-quality SQL question-and-answer data. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a method and apparatus for generating SQL datasets, which can quickly and automatically generate SQL question-and-answer data and automatically evaluate and filter the SQL question-and-answer data from multiple perspectives, thereby generating high-quality target SQL datasets.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] On one hand, embodiments of the present invention provide a method for generating SQL datasets, the method comprising:

[0007] Information extraction processing is performed on text data to obtain structured text data, which includes complexity distribution, business rules, and query patterns;

[0008] Based on the complexity distribution, business rules, and query patterns, multiple candidate question-answer pairs are constructed, each candidate question-answer pair including a candidate question and a corresponding candidate SQL statement.

[0009] For each candidate question-answer pair, a first quality score is calculated based on the dimensional scores of the candidate SQL statement across multiple dimensions and the complexity level of the candidate question. The multiple dimensions include execution dimension, matching dimension, and syntax dimension.

[0010] For each candidate question-answer pair, the confidence of the candidate SQL statement is calculated at multiple levels to obtain a second quality score. The multiple levels include text units and clauses.

[0011] By combining the first quality score and the second quality score, the target SQL dataset is determined from the candidate question-answer pairs.

[0012] On the other hand, embodiments of the present invention also provide an SQL dataset generation apparatus, the apparatus comprising:

[0013] The structuring module is used to extract information from text data to obtain structured text data, which includes complexity distribution, business rules, and query patterns.

[0014] The candidate module is used to construct multiple candidate question-answer pairs based on the complexity distribution, business rules, and query patterns. Each candidate question-answer pair includes a candidate question and a corresponding candidate SQL statement.

[0015] The first calculation module is used to calculate a first quality score for each candidate question-answer pair based on the dimensional scores of the candidate SQL statement on multiple dimensions and the complexity level of the candidate question. The multiple dimensions include execution dimension, matching dimension and syntax dimension.

[0016] The second calculation module is used to calculate the confidence of the candidate SQL statement at multiple levels for each candidate question-answer pair, and obtain a second quality score. The multiple levels include text units and clauses.

[0017] A generation module is used to fuse the first quality score and the second quality score to determine the target SQL dataset from the candidate Vidalin pairs.

[0018] On the other hand, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute steps in any of the SQL dataset generation methods provided in embodiments of the present invention.

[0019] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute steps in any of the SQL dataset generation methods provided in embodiments of the present invention.

[0020] On the other hand, embodiments of the present invention also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps in any of the SQL dataset generation methods provided in embodiments of the present invention.

[0021] The beneficial effects of the technical solution provided by this invention include at least the following:

[0022] In this embodiment of the invention, structured text data is extracted from text data, and multiple candidate question-answer pairs are automatically constructed using the complexity distribution, business rules, and query patterns within the structured text data. For each candidate question-answer pair, its dimensional scores across multiple dimensions are calculated to obtain a first quality score; its confidence scores across multiple levels are calculated to obtain a second quality score; finally, based on the first and second quality scores, an SQL dataset is selected from the candidate question-answer pairs. This invention automatically extracts information from text data and constructs question-answer pairs. The calculation of the first and second quality scores includes multiple dimensions and multiple levels, ensuring multi-faceted evaluation of candidate question-answer pairs, thereby ensuring the rapid generation of a high-quality target SQL dataset. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram illustrating an application scenario of the SQL dataset generation method provided in this embodiment of the invention;

[0025] Figure 2 This is a flowchart illustrating the SQL dataset generation method provided in an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram illustrating the determination of a specified text block provided in an embodiment of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of the SQL dataset generation device provided in an embodiment of the present invention;

[0028] Figure 5 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] It is understood that in specific embodiments of the present invention, data involving user information and related data requires user permission or consent, and the collection, use and processing of such data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0031] This invention proposes a method for generating SQL datasets, which can be found in the following embodiments. Figure 1 This diagram illustrates an application scenario for a method to generate SQL datasets. The application scenario may include a terminal 101 and a server 102, which can exchange data via a network. The terminal 101 may have a question-and-answer related application installed. The terminal 101 can be a mobile phone, tablet, smart Bluetooth device, computer, large screen, robot, etc.; the server 102 can be a single server or a server cluster consisting of multiple servers.

[0032] Users can send text data to server 102 via terminal 101, so that server 102 can extract information from the text data to obtain structured text data. The structured text data includes complexity distribution, business rules, and query patterns. Based on the complexity distribution, business rules, and query patterns, multiple candidate question-answer pairs are constructed. Each candidate question-answer pair includes candidate questions and corresponding candidate SQL statements. For each candidate question-answer pair, a first quality score is calculated based on the dimensional scores of the candidate SQL statement across multiple dimensions and the complexity level of the candidate question. The multiple dimensions include execution, matching, and syntax dimensions. For each candidate question-answer pair, the confidence level of the candidate SQL statement at multiple levels is calculated to obtain a second quality score. The multiple levels include text units and clauses. The first quality score and the second quality score are fused to determine the target SQL dataset from the candidate question-answer pairs.

[0033] After generating the target SQL dataset, server 102 can store the target SQL dataset in a fixed location and return data such as the successful creation of the target SQL dataset and the address information of the fixed location to terminal 101, so that users can view and use the target SQL dataset.

[0034] Then server 102 sends the reply text to terminal 101 so that the reply text can be displayed to the user.

[0035] In this embodiment, a method for generating SQL datasets is provided, such as... Figure 2 As shown, the specific process of this SQL dataset generation method can be as follows:

[0036] S110. Extract information from the text data to obtain structured text data.

[0037] Text data is textual information collected from business scenarios. Structured text data is data obtained by extracting and organizing specific data from text data, and it forms the basis for generating SQL datasets.

[0038] Optionally, when performing information extraction processing on text data to obtain structured text data, this can involve acquiring text data from a business scenario and the data type corresponding to each text data; dividing the text data into text blocks and performing vectorized storage processing on the text blocks to obtain a text knowledge base, the text knowledge base including text block vectors and database structure information; determining specified text blocks and specified structure information from text blocks of specified data types and database structure information according to a sampling strategy; analyzing the specified structure information and the specified text blocks to extract specified information to obtain structured text data.

[0039] Various textual materials are collected from actual business scenarios. These materials can be in formats that can be parsed into text, such as PDF, Text, Word, and Markdown. To facilitate subsequent processing, multiple data types corresponding to the textual data can be constructed based on the actual business scenario. For example, in this embodiment, data types may include business requirements, technical specifications, user manuals, historical query logs, data dictionaries, and business rules. For each piece of textual data, it can be divided into text blocks, and then the text blocks can be vectorized to generate corresponding text vectors.

[0040] For each text block, its content, text vector, metadata, and data type can be stored in a document vector library, which supports efficient semantic retrieval and similarity matching. Then, a structured meta-database is constructed, storing table structures, relationships, and statistical information. Table structures can include table names and columns; relationships can include foreign keys and associations; and statistical information can include the number of data samples and the null value rate. Furthermore, knowledge indexes can be established, such as mappings between text data and structural information, and mappings between business terms and database elements. The document vector library and the structured meta-database together form a text knowledge base. The document vector library can contain the text block vector corresponding to each text block, the structured meta-database can include all data structure information, and the knowledge index can be selectively included in the text knowledge base as needed.

[0041] The text data extracted from the business scenario is stored in the text knowledge base. The data type corresponding to each text block can be obtained through the text knowledge base. The specified text block and specified structure information can be determined from the text block of the specified data type and the database structure information according to the sampling strategy, so as to be used for subsequent extraction of structured data.

[0042] The specified data type can be any one or more data types, and can be set according to actual needs. When extracting structured data, a large language model can be used for processing. Since the input length of the large language model is limited, partial data can be extracted from text blocks of specified data types and database structure information for analysis.

[0043] Optionally, a portion of the data can be randomly sampled from the text block and the database structure information to obtain the specified text block and specified structure information. By analyzing the specified text block and specified structure information, structured text data can be extracted.

[0044] Optionally, to ensure the accuracy of the extracted structured text data, when determining the specified text block and specified structural information from the text blocks of the specified data type and the database structure information according to the sampling strategy, it can be done by randomly sampling a first proportion of first text blocks from the text blocks of the specified data type; for each first text block, retrieving the first structural information corresponding to the first text block and the number of times the first structural information appears in the database structure information; determining the specified structural information and second structural information from the database structure information based on the number of times the first structural information appears; collecting a second proportion of second text blocks from the text blocks corresponding to the second structural information; and using the first text block and the second text block as the specified text block.

[0045] A text block of a specified data type is obtained, and a first proportion of text blocks are randomly sampled from it as the first text block. The first proportion can be set according to actual needs; in this embodiment, the first proportion can be 10%. For the sampled first text block, a first text block vector corresponding to the first text block can be obtained. The first text block vector is then used to search the database structure information to recall the most relevant preset number of first structural information items. Since each first text block can recall a preset number of first structural information items, different first text blocks may recall the same first structural information items. Therefore, for each first structural information item, the frequency of occurrence of each first structural information item can be counted.

[0046] By utilizing the frequency of occurrence of each first structural information, specified structural information and second structural information can be determined from the database structural information. The specified structural information is the structural information related to text blocks of a specified data type. As one implementation, all first structural information can be directly used as the specified structural information. As another implementation, a preset number of first structural information entries with the highest frequency can be used as the specified structural information. The preset number is an integer and can be set according to actual needs.

[0047] The second structural information refers to structural information that is not involved or is involved less in the first text block. Optionally, it can be based on the number of times the first structural information appears, and the first structural information that appears less than a specified number of times can be used as the second structural information. Of course, it can also be that all other structural information besides the second structural information can be used as the second structural information.

[0048] To ensure that the sampled text blocks cover most of the database structure information, sampling can continue based on the second structure information. Optionally, a mapping relationship between structure information and text blocks can be obtained, and text blocks corresponding to the second structure information can be obtained based on this mapping relationship. Then, a second proportion of second text blocks can be randomly sampled from these blocks. The second proportion is a dynamic proportion, and a preset proportion range can be set in advance. The second proportion only needs to be within the preset proportion range; for example, the preset proportion range could be 5%-20%.

[0049] After collecting the second text block from the text block corresponding to the second structural information according to the second ratio, the first text block and the second text block can be used together as the specified text block.

[0050] Optional, see [link / reference] Figure 3 The diagram illustrates the determination of a specified text block. To ensure that the collected text blocks cover as much of the database structure information as possible, after determining the second text block, the first and second text blocks can be combined as a new first text block. The new first text block is then used to retrieve the new first structural information corresponding to the first text block in the database structure information, and the coverage rate of the first structural information in the database structure information is calculated. If the coverage rate does not reach the specified coverage rate, the second text block is sampled again based on the number of occurrences of the new first structural information in the original manner until the coverage rate of the first structural information of the first text block reaches the specified coverage rate. At this point, the first text block is used as the specified text block.

[0051] Alternatively, the number of times the second text block is sampled can be recorded. If the number of times the second text block is sampled reaches a preset number, even if the coverage does not reach the specified coverage, the new first text block obtained will be used as the specified text block and the process will end. The occurrence frequency of the first structural information can facilitate the adjustment of the result information that needs to be supplemented each time, ensuring that the specified text block obtained by the final sampling can cover most of the structural information.

[0052] By analyzing the identified text blocks and structural information, the specified information can be extracted to obtain structured text data. The specified information can be pre-defined content to be extracted, such as business entities, data relationships, query patterns, business rules, explanations of technical terms, query complexity, etc. Combining the extracted content according to a fixed format yields the structured text data.

[0053] As one implementation method, when analyzing the specified structural information and the specified text block to extract specified information to obtain structured text data, the specified data type, specified structural information, and the specified text block can be combined with an extraction template to obtain extraction prompts. The extraction prompts include data cleaning rules, data extraction rules, and an output format. The data extraction rules include query pattern mining rules, complexity analysis rules, and business extraction rules. Using the extraction prompts, a large language model is guided to clean the specified text block according to the data cleaning rules to obtain a cleaned text block. Using the extraction prompts, the large language model is guided to extract specified information from the cleaned text block according to the data extraction rules. The specified information is then organized according to the output format to obtain structured text data.

[0054] The extraction template is a pre-set prompt word template used to extract structured text data. The prompt word template may include multiple slots to be filled, such as type slots, content slots, and structure information slots. By filling the corresponding slots with the specified data type, specified structure information, and specified text block, the extraction prompt words can be obtained.

[0055] The extraction prompts may include relevant requirements for data processing, such as data cleaning rules, data extraction rules, and output formats. Data cleaning rules refer to the basic rules for cleaning the specified input text block; data extraction rules may include rules for extracting various specified information, such as query patterns, mining rules, complexity analysis rules, and business extraction rules; and the output format is the final organizational pattern of the extracted specified information, which becomes structured text data after organization.

[0056] The extraction template can be set according to actual needs. In this embodiment of the invention, the extraction template can be:

[0057] Task: Apply processing tasks to the following production materials and extract structured knowledge.

[0058] Input type: [document_type]

[0059] Input content: [document_content]

[0060] Related database schema (TopK):

[0061] [topk_schema_info]

[0062] Please complete the following processing tasks:

[0063] 1. Data cleaning and standardization:

[0064] Analyze the input content and perform the following cleaning operations:

[0065] - Identify and remove content irrelevant to database queries (such as headers and footers, copyright information, redundant punctuation, Markdown internal formatting codes, etc.).

[0066] - Standardize professional terminology: unify synonyms (e.g., unify "customer" / "user" / "Customer" into standard terms).

[0067] - Correct obvious spelling errors and formatting issues

[0068] - Extract the core business logic description and remove redundant expressions.

[0069] Output the core content after cleaning (within 200 words).

[0070] 2. Information Extraction and Structuring (In-depth analysis based on business documents and TopK Schema):

[0071] 2.1 Business Entity Identification:

[0072] - Identify all business entities (such as customers, orders, products, etc.) from documents.

[0073] - Traverse each entity and map it to a specific table in the TopK Schema if possible.

[0074] 2.2 Data Relationship Extraction:

[0075] - Analyze the business processes and data flows described in the documentation (if any, otherwise skip the relationship extraction process).

[0076] - Identify potential JOIN relationships and association conditions between tables in the schema.

[0077] - Extract primary and foreign key relationships, one-to-many / many-to-many relationships

[0078] 2.3 Query Pattern Mining:

[0079] - Infer possible query needs based on business descriptions (such as statistical analysis, data filtering, and report generation).

[0080] - Identify the complexity level of the query (simple / medium / complex)

[0081] - Generate typical query scenario examples

[0082] 2.4 Constraint Rule Extraction:

[0083] - Identify business constraints (such as VIP discount rules, order amount limits).

[0084] - Extract data validation rules (such as field not null, value range).

[0085] - Mark special business logic (such as tiered pricing, time window restrictions).

[0086] 2.5 Explanation of Technical Terms:

[0087] - Extract domain-specific terms and provide definitions

[0088] - Establish a mapping between terms and schema elements (table names, column names).

[0089] - Explain how the term is used in different scenarios

[0090] 2.6 Complexity Assessment:

[0091] - Evaluate query complexity based on schema structure

[0092] - Calculate table join density and nested query probability

[0093] Output format (strictly matches JSON format):

[0094] {

[0095] "cleaned_content": "Core content after cleaning",

[0096] "entities": [

[0097] {"name": "Entity name", "type": "Entity type", "table_mapping": "Corresponding table name", "description": "Description"}

[0098] ],

[0099] "relationships": [

[0100] {"source": "source table", "target": "target table", "type": "relationship type", "join_condition": "join condition", "description": "relationship description"}

[0101] ],

[0102] "query_patterns": [

[0103] {"pattern": "query pattern", "frequency": "frequency", "complexity": "complexity", "involved_tables": ["tables involved"], "example": "example"}

[0104] ],

[0105] "business_rules": [

[0106] {"rule": "rule description", "tables_involved": ["tables involved"], "columns_involved": ["columns involved"], "validation": "validation logic"}

[0107] ],

[0108] "domain_terms": [

[0109] {"term": "term", "definition": "definition", "table_column_mapping": "table column mapping", "context": "use case"}

[0110] ],

[0111] "complexity_metrics": {

[0112] "table_count": "Number of tables",

[0113] "avg_columns_per_table": "Average number of columns",

[0114] "relationship_density": "relationship density",

[0115] "query_complexity_distribution": {"simple": 0.3, "medium": 0.5, "complex": 0.2}

[0116] }

[0117] }”

[0118] The [document_type] field can be filled with a specified data type; the [document_content] field can be filled with a specified text block; and the [topk_schema_info] field can be filled with specified structural information. After filling in the information, the extraction prompt words will be obtained.

[0119] Based on this prompt template, the extraction prompts obtained after filling in the corresponding information can include specific data cleaning rules and specific rules for extracting specified information.

[0120] The structured text data in this embodiment of the invention may include complexity distribution, business rules, and query patterns. Therefore, the extracted specified information may include, in addition to these three items, business entities, data relationships, and explanations of technical terms. Extraction prompts are input into a large language model to guide it in processing the data according to its data cleaning and extraction rules. The extracted specified information is then assembled according to the output format to obtain the structured text data.

[0121] In its business entity identification, the large language model can identify all business entities, such as customers, orders, and products, from a specified text block. It then iterates through each identified entity and correlates it with specific tables within the specified structural information. For data relationship extraction, it analyzes the business processes and data flows described in the specified text block, identifies potential relationships and association conditions between database tables, and extracts primary and foreign key relationships. In query pattern mining, it infers possible query requirements based on the business descriptions in the specified text block, such as statistical analysis, data filtering, and report generation. It then identifies the complexity of these query requirements, determines the corresponding complexity level, and generates typical query examples. For business rule extraction, it identifies business constraints from the specified text block, such as discount rules and order amount limits. It can also annotate special business logic, such as tiered pricing and time window restrictions. Finally, for the explanation of technical terms, it extracts domain-specific terms from the specified text block, provides corresponding definitions, establishes a mapping between terms and table / column names, and explains the usage of these terms in different scenarios. When extracting complexity distribution, the query complexity can be evaluated based on specified structural information, the density of associations between tables, the possibility of nested queries, etc., to obtain the complexity distribution.

[0122] The extracted information is organized according to the output format to form structured text data. Through information extraction processing, useless information in the text data can be removed. This structured text data contains content related to the business scenario, such as explanations of professional terms and business rules, which can deepen the model's understanding of the business in subsequent processing and ensure the generation of high-quality SQL datasets.

[0123] S120. Based on the complexity distribution, business rules, and query patterns, construct multiple candidate question-answer pairs.

[0124] After extracting the structured text data, multiple candidate question-answer pairs can be constructed based on the complexity distribution, business rules, and query patterns. Each candidate question-answer pair contains a candidate question and a corresponding candidate SQL statement.

[0125] Optionally, when constructing multiple candidate question-answer pairs, the structured text data can be combined with a first template to obtain a first prompt word, wherein the first prompt word includes question generation requirements related to the complexity distribution, business rules, and query patterns; using the first prompt word, a large language model can be guided to process the structured text data according to the question generation requirements to generate multiple candidate questions of different complexity levels; for each candidate question, multiple candidate SQL statements corresponding to the candidate question can be generated using various different SQL generation strategies; and the candidate questions and the candidate SQL statements can be combined to obtain multiple candidate question-answer pairs.

[0126] The first template is a prompt word template used to generate candidate questions. This first template can be pre-configured according to actual needs. The first template may include corresponding structured text filling slots. The extracted structured text data is then filled into these slots to obtain the first prompt word. This first prompt word can guide the large language model to generate candidate questions. The first prompt word may contain the requirements for generating the candidate questions, which are related to the complexity distribution, business rules, and query patterns in the structured text data.

[0127] The first prompt word obtained can be input into the large language model, guiding it to process the structured text data according to the question generation requirements to generate multiple candidate questions of different complexity levels. These requirements may include generating candidate questions of different complexity levels according to complexity distribution, generating candidate questions that conform to business logic using business rules, and generating questions for common scenarios based on query patterns. The database table information involved in the candidate questions can be obtained from the table names mapped to entities in the structured text data and the extracted data relationships.

[0128] It should be noted that the first prompt may also include information such as the specific number of candidate questions to be generated. In this embodiment of the invention, the first template may be:

[0129] "Task: Based on the given database structure and business scenario, generate diverse natural language query questions."

[0130] The structured knowledge input:

[0131] - Entity information: [StructuredKnowledge.entities]

[0132] - Relationship information: [StructuredKnowledge.relationships]

[0133] - Business rules: [StructuredKnowledge.business_rules]

[0134] - Query pattern: [StructuredKnowledge.query_patterns]

[0135] - Complexity distribution: [StructuredKnowledge.complexity_metrics]

[0136] Generation requirements:

[0137] 1. Generate problems of different complexities based on the distribution in complexity_metrics.

[0138] 2. Make full use of business_rules to generate queries that conform to business logic.

[0139] 3. Generate common query scenarios based on query_patterns

[0140] 4. Obtain the relevant table information from the entities' table_mapping.

[0141] 5. The questions are expressed naturally and in a variety of ways.

[0142] Please generate 10 query questions, in the following format:

[0143] {

[0144] "question": "Query question text",

[0145] "complexity": "simple / medium / complex",

[0146] "involved_tables": [...], # Infer from entities.table_mapping and relationships

[0147] "query_type": "selection / aggregation / join / subquery",

[0148] "business_rules_applied": ["The business rules involved"]

[0149] }”

[0150] The input structured knowledge can be filled with corresponding structured text data to generate the first prompt word. This first prompt word is then input into the large language model, which in turn guides the model to generate multiple candidate questions of varying complexity according to the question generation requirements.

[0151] Guided by the first prompt word, each candidate question generated by the large language model can include the question itself, its complexity, the data tables involved, the question type, and the industry rules it depends on. For example, a candidate question could be:

[0152] {

[0153] "question": "Query the total order amount of all VIP customers and apply a 10% discount."

[0154] "complexity": "medium",

[0155] "involved_tables": ["customers", "orders", "order_items"],

[0156] "query_type": "aggregation",

[0157] "business_rules_applied": ["VIP customers enjoy a 10% discount"]

[0158] }

[0159] For each candidate question, various SQL statement generation strategies can be used to generate multiple candidate SQL statements corresponding to the candidate question. The candidate question can then be combined with each candidate SQL statement to form multiple candidate question-answer pairs.

[0160] Optionally, when generating candidate SQL statements corresponding to candidate questions, multiple existing SQL generation algorithms can be used to process the candidate questions to obtain multiple candidate SQL statements. For example, multiple specially trained SQL statement generation models can be used to generate multiple candidate SQL statements corresponding to candidate questions, wherein the different SQL statement generation models use different model architectures and training samples.

[0161] Optionally, prompt words can be constructed based on candidate questions, and these prompt words can be input into a large language model to generate candidate SQL statements. The prompt words can contain various contextual information such as database structure information, reference examples, and relevant complete documentation. By setting different prompt words or introducing different contexts into the prompt words, different methods can be considered as using to generate candidate SQL statements.

[0162] By combining each candidate question with its generated multiple candidate SQL statements, multiple candidate question-answer pairs can be obtained.

[0163] S130. For each candidate question-answer pair, calculate the first quality score based on the dimensional scores of the candidate SQL statement on multiple dimensions and the complexity level of the candidate question.

[0164] For each candidate question-answer pair, a first quality score can be calculated based on the dimensional scores of the candidate SQL statements across multiple dimensions and the complexity level of the candidate questions. These multiple dimensions can include execution, matching, and syntax dimensions. Dimensional scores for the candidate SQL statements are calculated on each of these dimensions, and then the first quality score is calculated using these dimensional scores and the complexity level of the candidate questions. The first quality score can be used to measure the quality of the candidate question-answer pair.

[0165] In one implementation, when calculating the first quality score, the execution score of the candidate question-answer pair can be determined based on the execution result of the candidate SQL statement; candidate question-answer pairs with execution scores greater than a specified value are designated question-answer pairs, which include candidate questions and designated SQL statements; the syntactic structure of the designated SQL statement is parsed, and a syntactic score is calculated based on the syntactic structure; a large language model is used to analyze the matching degree between the candidate questions and the designated SQL statements to obtain a matching score; for each designated SQL statement, the first quality score is calculated based on the complexity level of the candidate questions, the designated score corresponding to the designated SQL statement, the syntactic score, and the matching score.

[0166] For each candidate SQL statement in the candidate question-answer pair, the feasibility of execution of the candidate SQL statement can be evaluated. That is, the candidate SQL statement can be executed, and the execution result corresponding to the candidate SQL statement can be obtained. The execution result can include the execution status, execution time, return result, etc. The execution score can be calculated using the execution result.

[0167] Optionally, a pre-defined rule for calculating the execution score can be used to calculate the execution score. For example, in this embodiment of the invention, the execution status can include execution success or execution failure; the returned result can include empty or non-empty; if the execution status is execution success and the returned result is non-empty, the execution score can be set as the first score; if the execution status is execution success but the returned result is empty, the execution score can be set as the second score; if the execution status is execution failure, the execution score can be set as the third score. The first score is greater than the second score, and the second score is greater than the third score.

[0168] After calculating the execution score, candidate question-answer pairs with execution scores greater than a specified value can be selected and used as designated question-answer pairs in subsequent processes. The specified value can be set according to actual needs; in this embodiment, the specified value can be the same as the third score. That is, in this embodiment, the difference between designated question-answer pairs and candidate question-answer pairs is that candidate question-answer pairs contain candidate SQL statements that failed to execute, while designated question-answer pairs contain designated SQL statements that can all be executed successfully.

[0169] For a specific SQL statement in a given question-answer pair, the syntax structure of the specified SQL statement can be further parsed, and a syntax score can be calculated based on the parsed syntax structure. First, the specified SQL statement can be parsed to obtain the parsing result, which may include the parsing status and the syntax tree. Based on the parsing status and the syntax tree, the syntax score of the specified SQL statement can be calculated.

[0170] Optionally, a pre-defined rule for calculating the syntax score can be used, and the syntax score can be calculated according to the rule. For example, in this embodiment of the invention, the parsing status can include parsing success or parsing failure, and the parsed syntax tree can include select and group by; the parsing score can be calculated based on the parsing status; the syntax consistency score can be calculated based on the select and group by in the syntax tree; and the syntax score can be calculated by multiplying the parsing score and the syntax consistency score.

[0171] Specifically, if the parsing status is successful, the parsing score can be set to the fourth score; if the parsing status is unsuccessful, the parsing score can be set to the fifth score; determine whether all non-aggregate columns in the select are present in the group by; if yes, the syntax consistency score can be set to the sixth score; if no, the syntax consistency score can be set to the seventh score; finally, calculate the product of the parsing score and the syntax consistency score to obtain the syntax score.

[0172] The execution score and syntax score calculated above are evaluated from the dimension of SQL statement. To ensure the comprehensiveness of the evaluation, we can further analyze the matching degree between candidate questions and specified SQL statements in a specified question-answer pair.

[0173] Optionally, when analyzing the matching degree between candidate questions and specified SQL statements, the analysis can be performed from four aspects: query target matching degree, filter condition matching degree, aggregation operation matching degree, and business logic matching degree.

[0174] The query target matching score is calculated as follows: Query target matching score determines whether the SELECT clause in the specified SQL statement correctly returns the information required by the candidate question, and whether it includes the required fields, etc., and is assigned a score of 0-1. Filter condition matching score determines whether the WHERE clause in the specified SQL statement correctly implements the filter conditions in the candidate question, and whether its conditional logic is consistent with the description in the candidate question, etc., and is assigned a score of 0-1. Aggregate operation matching score determines whether the aggregate functions in the specified SQL statement, such as SUM, COUNT, AVG, etc., are consistent with the candidate question, and whether the grouping in GROUP BY is correct, and is assigned a score between 0-1. Business logic matching score determines whether the specified SQL statement implements business rules, and whether the JOIN relationship is reasonable, etc., and is assigned a score between 0-1. It should be noted that the higher the values ​​of the query target matching score, filter condition matching score, aggregate operation matching score, and business logic matching score, the more closely the specified SQL statement meets the corresponding requirements.

[0175] The matching score can be calculated by weighting the matching degree of the query target, the matching degree of the filtering conditions, the matching degree of the aggregation operation, and the matching degree of the business logic.

[0176] In one implementation, the matching degree of each aspect can be evaluated and scored using a large language model. Finally, a weighted average of the scores for each aspect is calculated to obtain the matching score. In another implementation, a large language model can be invoked, and corresponding prompt words can be designed. These prompt words can contain the calculation logic for the matching degree of each aspect and provide the calculation logic for the matching score, thus allowing the matching score to be calculated directly.

[0177] For example, the prompt words could be:

[0178] Task: Evaluate the semantic matching degree between natural language questions and SQL queries.

[0179] Question: [question]

[0180] SQL: [sql]

[0181] Relevant Schema: [schema_info]

[0182] Please analyze and evaluate:

[0183] 1. Query target matching score (0-1 point):

[0184] - Does the SQL SELECT clause correctly return the information requested in the question?

[0185] - Does it include all the required fields?

[0186] - Are there any extra or missing fields?

[0187] 2. Matching score of filter criteria (0-1 point):

[0188] Does the WHERE clause correctly implement the filtering conditions in the question?

[0189] - Is the conditional logic consistent with the problem description?

[0190] 3. Aggregation operation matching score (0-1 point):

[0191] - Are the aggregate functions (SUM / COUNT / AVG, etc.) consistent with the requirements of the question?

[0192] - Is the GROUP BY grouping correct?

[0193] 4. Business logic matching degree (0-1 point):

[0194] Does the SQL correctly implement the business rules?

[0195] Is the JOIN relationship reasonable?

[0196] Output format:

[0197] {

[0198] "target_match": 0.95,

[0199] "filter_match": 1.0,

[0200] "aggregation_match": 0.9,

[0201] "business_logic_match": 1.0,

[0202] "overall_semantic_match": 0.96, / / Weighted average

[0203] "explanation": Briefly explain the reasons for the scoring.

[0204] }

[0205] Example output:

[0206] {

[0207] "target_match": 1.0,

[0208] "filter_match": 1.0,

[0209] "aggregation_match": 1.0,

[0210] "business_logic_match": 0.95,

[0211] "overall_semantic_match": 0.99,

[0212] "explanation": "The SQL query correctly fulfills the requirement of querying the total order amount for VIP customers, and includes the correct aggregation and filtering conditions."

[0213] }

[0214] For each specified question-answer pair, candidate questions are filled into [question], specified SQL statements are filled into [sql], and database structure information retrieved based on candidate questions or specified SQL statements is filled into [schema]. These are then passed to the large language model, which can output the matching degree for each aspect and calculate the final matching score.

[0215] At this point, an execution score, a syntax score, and a matching score are calculated for each specified question-answer pair. Using these three scores and the complexity level of the candidate questions, a first quality score can be calculated.

[0216] As one implementation method, when calculating the first quality score, the following steps can be taken: for each candidate question, obtain multiple specified SQL statements corresponding to the candidate question; for each specified SQL statement, obtain the return content corresponding to the specified SQL statement; compare the consistency of the multiple specified SQL statements based on the return content to obtain a consistency score; determine the dimension weight corresponding to each dimension based on the complexity level of the candidate question; and based on the dimension weight, fuse the consistency score, execution score, syntax score, and matching score of the specified SQL statement to obtain the first quality score.

[0217] The aforementioned filtering based on execution scores identified specific question-answer pairs. Each pair includes candidate questions and corresponding specified SQL statements. For each candidate question, multiple specified SQL statements can be retrieved. For each of the multiple specified SQL statements corresponding to a candidate question, the execution result of executing that SQL statement can be obtained. As previously explained, the execution result includes data such as execution time and returned content.

[0218] For a candidate problem, consistency analysis can be performed by comparing the execution results of its corresponding specified SQL statements. For example, it can be done by comparing whether the returned content of all specified SQL statements corresponding to the candidate problem is semantically equivalent and whether the returned content is the same; if they are semantically equivalent or the returned content is the same, they can be considered consistent. The consistency score can be set as the eighth score when consistency is confirmed, and as the ninth score when inconsistency is confirmed.

[0219] In the aforementioned embodiments, multiple candidate questions with different complexity levels were generated, each with its own corresponding complexity level. A mapping relationship between each complexity level and preset weights can be pre-defined. Based on the specific complexity level of the candidate question and the mapping relationship, the dimension weights corresponding to each dimension to be used are determined. For example, the complexity levels of a question may include three levels: simple, medium, and complex. The dimension weight corresponding to the simple level is the first weight, the dimension weight corresponding to the medium level is the second weight, and the dimension weight corresponding to the complex level is the third weight. Each dimension weight includes execution weight, syntax weight, and matching weight, corresponding one-to-one with the aforementioned multiple dimensions.

[0220] The execution score, syntax score, and matching score are weighted using dimensional weights to obtain the first quality score. Optionally, the weighted result can be added to the consistency score to obtain the first quality score. Optionally, additional bonus points can be determined based on the range of the consistency score, and the weighted result can be added to the bonus points to obtain the first quality score.

[0221] It should be noted that since a candidate question corresponds to multiple specified SQL statements, the best-performing SQL statement can be selected from these multiple specified SQL statements while calculating the first quality score. Optionally, when determining the best SQL statement, the specified SQL statements can be sorted in descending order of matching score to obtain a first sequence; the specified SQL statements can be sorted in ascending order of execution time to obtain a second sequence; the score of each specified SQL statement can be calculated based on its ranking in the first and second sequences, and the SQL statement with the highest score can be selected as the best SQL statement. In this way, the best SQL statement has a lower matching degree and shorter execution time.

[0222] S140. For each candidate question-answer pair, calculate the confidence level of the candidate SQL statement at multiple levels to obtain a second quality score.

[0223] For each candidate question pair, in addition to calculating the first quality score mentioned above, a second quality score can be calculated. Both the first and second quality scores are scores obtained after evaluating the quality of the question-answer pair, but they differ in the evaluation perspective.

[0224] The second quality score primarily evaluates candidate SQL statements hierarchically from a confidence level, which can include the text unit level, clause level, and the entire SQL statement level. As one implementation method, calculating the second quality score involves: for each candidate SQL statement, obtaining the clauses, text units, and their corresponding unit types; determining an adjustment factor based on the inclusion relationship between the text units and the business rules; calculating the confidence of each text unit based on its unit type and the adjustment factor; summing the confidence of all text units within each clause for each sentence to obtain the clause confidence; and performing a geometric mean on the clause confidence of all clauses within the candidate SQL statement to obtain the second quality score.

[0225] For each candidate SQL statement, we can break it down into its clauses and each text unit within a clause. For example, if the candidate SQL statement is "SELECT customer_name, SUM(order_amount) FROM customers c JOIN orders o ON c.customer_id = o.customer_id WHERE order_date > '2023-01-01' GROUP BY customer_name", then the clauses can be broken down into multiple clauses based on SELECT, FROM, etc. For each clause, we can further break it down into each text unit, which can also be referred to as a token.

[0226] For each text unit, multiple unit types can be pre-defined, and the definition of each unit type is given. The specific settings can be configured according to actual needs. In this embodiment of the invention, types can include keyword types, pattern types, operator types, and constant types. Keyword types refer to reserved words in SQL statements, such as SELECT, FROM, WHERE, JOIN, GROUP BY, etc.; pattern types refer to identifiers related to database structure information, such as table names, column names, aliases, etc.; operator types refer to comparison and operation logical operators, such as =, >, <, AND, OR, BETWEEN, etc.; constant types refer to literal constants, including strings, numbers, and dates.

[0227] For each text unit, an adjustment factor can be determined based on the inclusion relationship between the text unit and the business rules. Specifically, this can involve obtaining business rules from structured text data, extracting keywords from the business rules, and detecting whether the text involves keywords from the business rules. If keywords are involved, the adjustment factor is set to a first specified value; if no keywords are involved, the adjustment factor is set to a second specified value. The first specified value must be greater than the second specified value, and the specific values ​​can be set according to actual needs. In this embodiment, the first specified value can be set to 0.1, and the second specified value can be set to 0.

[0228] For each text unit, the confidence level of each text unit can be calculated using its unit type and adjustment factor. As one implementation method, when calculating the confidence level of a text unit, the following steps can be taken: First, obtain the type weight and influence coefficient of the text unit based on its corresponding unit type; second, obtain the unit probability of the text unit in the large language model; third, calculate the basic weight of the text unit using the unit probability, the influence coefficient, and the type region; fourth, calculate the average of the basic weights of all text units in a specified clause, where the specified clause is the clause to which the text unit belongs; and finally, calculate the confidence level of the text unit using the average weight and the adjustment factor.

[0229] For each unit type, a type weight and influence coefficient can be pre-set, which can be configured according to actual needs. In this embodiment of the invention, Table 1 shows the type weight and influence coefficient corresponding to each unit type.

[0230] Table 1

[0231]

[0232] For each text unit, the unit probability corresponding to the text unit in the large language model can be obtained. Specifically, the large language model can be used to predict each text unit, and the probability of the large language model predicting the text unit can be obtained as the unit probability.

[0233] Multiply the unit probability by the influence coefficient to obtain the influence weight. Then, add the influence weight to the type weight of the text unit to obtain the basic weight of the text unit. Using the clause to which the text unit belongs as the specified clause, calculate the basic weight of each other text unit in the specified clause. Then, calculate the average of the basic weights of all text units in the clause to obtain the average weight. Add an adjustment factor to the average weight to obtain the candidate confidence level. Compare the candidate confidence level with the specified confidence level, and take the smaller value as the confidence level of the text unit. For example, if the candidate confidence level is 0.85 and the specified confidence level is 1, 0.85 is less than 1, so 0.85 is the confidence level of the text unit.

[0234] For each clause, the confidence scores of all text units within the clause are aggregated to obtain the clause confidence score. Optionally, the confidence scores of all text units can be aggregated by calculating the arithmetic mean of the confidence scores of all text units, which is then used as the confidence score of the entire clause.

[0235] A candidate SQL statement may include multiple clauses. To calculate the confidence score of a candidate SQL statement, the confidence scores of all clauses can be aggregated. Alternatively, a geometric mean can be applied to the confidence scores of all clauses to calculate a second quality score for the candidate SQL statement.

[0236] S150. Combine the first quality score and the second quality score to determine the target SQL dataset from the candidate question-answer pairs.

[0237] For each candidate question-answer pair, a first quality score and a second quality score are calculated for the candidate SQL statements. The target quality score for each candidate question-answer pair is obtained by fusing the first and second quality scores. Optionally, there are several ways to fuse the first and second quality scores; for example, the target quality score can be obtained by directly summing the first and second quality scores.

[0238] Target SQL question-answer pairs can be selected from candidate question-answer pairs using a target quality score. For example, a quality threshold can be set, and candidate question-answer pairs with a target quality score greater than the threshold can be identified as target SQL question-answer pairs. Alternatively, all candidate question-answer pairs can be sorted in descending order of target quality score to obtain a question-answer pair sequence. The top-ranked candidate question-answer pairs in this sequence, representing a specified number of pairs, can be selected as target SQL question-answer pairs. This specified number is an integer and can be set according to actual needs. All identified target SQL question-answer pairs constitute the target SQL dataset.

[0239] The SQL dataset generation scheme provided in this invention can be applied in various scenarios. For example, in the smart manufacturing scenario, the SQL generation algorithm can be optimized by constructing the SQL dataset so that SQL statements can be accurately generated in the intelligent query analysis of manufacturing data, providing a solid foundation for intelligent query analysis in the smart manufacturing scenario.

[0240] The method provided by this invention can extract structured text data from text data and automatically generate candidate question-answer pairs using the structured text data. Then, the candidate question-answer pairs can be evaluated from the execution dimension, matching dimension, and syntax dimension to obtain a first quality score. The confidence of the candidate question-answer pairs at multiple levels is calculated to obtain a second quality score. Finally, the first quality score and the second quality score are combined to filter out the target SQL dataset. In the absence of standard SQL statements, the candidate question-answer pairs are comprehensively and reasonably evaluated from multiple perspectives, realizing a fully automated process of generation and intelligent evaluation, and can quickly generate high-quality target SQL datasets.

[0241] To better implement the above methods, this embodiment of the invention also provides an SQL dataset generation device, which can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster composed of multiple servers.

[0242] For example, in this embodiment, the method of the present invention will be described in detail by taking the SQL dataset generation device specifically integrated into the server as an example.

[0243] For example, such as Figure 4 As shown, the SQL dataset generation device 200 may include a structuring module 210, a candidate module 220, a first calculation module 230, a second calculation module 240, and a generation module 250.

[0244] The structuring module 210 is used to extract information from text data to obtain structured text data, which includes complexity distribution, business rules, and query patterns.

[0245] The candidate module 220 is used to construct multiple candidate question-answer pairs based on the complexity distribution, business rules and query patterns. The candidate question-answer pairs include candidate questions and corresponding candidate SQL statements.

[0246] The first calculation module 230 is used to calculate a first quality score for each candidate question-answer pair based on the dimension scores of the candidate SQL statement on multiple dimensions and the complexity level of the candidate question. The multiple dimensions include execution dimension, matching dimension and syntax dimension.

[0247] The second calculation module 240 is used to calculate the confidence of the candidate SQL statement at multiple levels for each candidate question-answer pair, and obtain a second quality score. The multiple levels include text units and clauses.

[0248] The generation module 250 is used to fuse the first quality score and the second quality score to determine the target SQL dataset from the candidate question-answer pairs.

[0249] In some embodiments, the structuring module 210 is specifically used for:

[0250] Obtain the text data in the business scenario, and the data type corresponding to each piece of text data;

[0251] The text data is divided into text blocks, and the text blocks are vectorized and stored to obtain a text knowledge base, which includes text block vectors and database structure information.

[0252] The specified text block and specified structure information are determined from the text block of the specified data type and the database structure information according to the sampling strategy;

[0253] Analyze the specified structural information and the specified text block to extract the specified information to obtain structured text data.

[0254] In some embodiments, the structuring module 210 is specifically used for:

[0255] Randomly sample a first proportion of text blocks from text blocks of a specified data type;

[0256] For each first text block, retrieve the first structural information corresponding to the first text block and the number of times the first structural information appears in the database structural information;

[0257] Based on the frequency of occurrence of the first structural information, determine the specified structural information and the second structural information from the database structural information;

[0258] From the text block corresponding to the second structural information, collect a second proportion of the second text block;

[0259] The specified text block is determined based on the first text block and the second text block.

[0260] In some embodiments, the structuring module 210 is specifically used for:

[0261] The specified data type, specified structural information, and specified text block are combined with the extraction template to obtain extraction prompt words. The extraction prompt words include data cleaning rules, data extraction rules, and output format. The data extraction rules include query pattern mining rules, complexity analysis rules, and business extraction rules.

[0262] Using the extracted prompt words, the large language model is guided to clean the specified text block according to the data cleaning rules to obtain the cleaned text block;

[0263] Using the extracted prompt words, the large language model is guided to extract specified information from the cleaned text block according to the data extraction rules;

[0264] The specified information is organized according to the output format to obtain structured text data.

[0265] In some embodiments, candidate module 220 is specifically used for:

[0266] The structured text data is combined with the first template to obtain the first prompt word, which includes the question generation requirements related to the complexity distribution, business rules, and query mode.

[0267] Using the first prompt word, the large language model is guided to process the structured text data according to the question generation requirements, generating multiple candidate questions of different complexity levels;

[0268] For each candidate question, multiple candidate SQL statements are generated using various different SQL generation strategies.

[0269] The candidate questions are combined with the candidate SQL statements to obtain multiple candidate question-answer pairs.

[0270] In some embodiments, the first computing module 230 is specifically used for:

[0271] Based on the execution results of the candidate SQL statements, determine the execution score of the candidate question-answer pair;

[0272] The candidate question-answer pairs whose execution scores are greater than a specified value are designated question-answer pairs, and the designated question-answer pairs include candidate questions and specified SQL statements;

[0273] Parse the syntax structure of the specified SQL statement and calculate a syntax score based on the syntax structure;

[0274] Using a large language model, the matching degree between the candidate question and the specified SQL statement is analyzed to obtain a matching score;

[0275] For each specified SQL statement, a first quality score is calculated based on the complexity level of the candidate question, the execution score, the syntax score, and the matching score corresponding to the specified SQL statement.

[0276] In some embodiments, the first computing module 230 is specifically used for:

[0277] For each candidate question, obtain multiple specified SQL statements corresponding to the candidate question;

[0278] For each specified SQL statement, retrieve the return content corresponding to the specified SQL statement;

[0279] A consistency score is obtained by comparing the returned content with the consistency of multiple specified SQL statements.

[0280] The dimensional weights for each dimension are determined based on the complexity level of the candidate problem.

[0281] Based on the aforementioned dimensional weights, the consistency score, execution score, syntax score, and matching score of the specified SQL statement are combined to obtain the first quality score.

[0282] In some embodiments, the second computing module 240 is specifically used for:

[0283] For each candidate SQL statement, obtain the clauses, text units, and corresponding unit types in the candidate SQL statement;

[0284] Based on the text unit and the business rules, determine the adjustment factor of the text unit;

[0285] Calculate the confidence level of each text unit based on the unit type corresponding to each text unit and the adjustment factor;

[0286] For each sentence, the confidence scores of all text units within the clause are aggregated to obtain the clause confidence score;

[0287] For the candidate SQL statement, the clause confidence of all clauses is processed by geometric mean to obtain the second quality score.

[0288] In some embodiments, the second computing module 240 is specifically used for:

[0289] Based on the unit type corresponding to the text unit, obtain the type weight and influence coefficient of the text unit;

[0290] Obtain the unit probability corresponding to the text unit in the large language model;

[0291] The basic weight of the text unit is calculated using the unit probability, the influence coefficient, and the type weight;

[0292] Calculate the average of the base weights of all text units in the specified clause to obtain the average weight, where the specified clause is the clause to which the text unit belongs;

[0293] The confidence level of the text unit is calculated using the average weight and the adjustment factor.

[0294] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0295] As shown above, the SQL dataset generation device in this embodiment can extract structured text data from text data and automatically construct multiple candidate question-answer pairs using the complexity distribution, business rules, and query patterns in the structured text data. For each candidate question-answer pair, its dimensional scores on multiple dimensions are calculated to obtain a first quality score; its confidence scores on multiple levels are calculated to obtain a second quality score; and finally, the SQL dataset is selected from the candidate question-answer pairs based on the first and second quality scores. Automatically extracting information from text data and constructing question-answer pairs, with the calculation of the first and second quality scores including multiple dimensions and levels, ensures multi-faceted evaluation of candidate question-answer pairs, thereby ensuring the rapid generation of a high-quality target SQL dataset.

[0296] This invention also provides an electronic device, which can be a terminal, a server, or other similar devices. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0297] In some embodiments, the SQL dataset generation device may also be integrated into multiple electronic devices, such as multiple servers, with multiple servers implementing the SQL dataset generation method of the present invention.

[0298] In this embodiment, a server will be used as an example for detailed description. For example, ... Figure 5 As shown, it illustrates a structural schematic diagram of the electronic device involved in an embodiment of the present invention, specifically:

[0299] The electronic device may include components such as a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0300] The processor 310 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes various functions and processes data by running or executing software programs and / or modules stored in the memory 320, and by calling data stored in the memory 320. In some embodiments, the processor 310 may include one or more processing cores; in some embodiments, the processor 310 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 310.

[0301] The memory 320 can be used to store software programs and modules. The processor 310 executes various functional applications and data processing by running the software programs and modules stored in the memory 320. The memory 320 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 320 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 320 may also include a memory controller to provide the processor 310 with access to the memory 320.

[0302] The electronic device also includes a power supply 330 that supplies power to the various components. In some embodiments, the power supply 330 can be logically connected to the processor 310 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 330 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0303] The electronic device may also include an input module 340, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0304] The electronic device may also include a communication module 350. In some embodiments, the communication module 350 may include a wireless module, through which the electronic device can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 350 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0305] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 310 in the electronic device loads the executable files corresponding to the processes of one or more application programs into the memory 320 according to instructions, and the processor 310 runs the application programs stored in the memory 320, thereby implementing the steps in the methods of the various embodiments of the present invention. The specific implementation of each of the above steps can be found in the previous embodiments, and will not be described in detail here.

[0306] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0307] To this end, embodiments of the present invention provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps in any of the SQL dataset generation methods provided in the embodiments of the present invention. The storage medium may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0308] According to one aspect of the present invention, a computer program product or computer program is provided, comprising a computer program / instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instructions from the computer-readable storage medium and executes the computer program / instructions, causing the electronic device to perform the methods provided in various alternative implementations of the SQL dataset generation aspect of the above embodiments.

[0309] Since the instructions stored in the storage medium can execute the steps in any of the SQL dataset generation methods provided in the embodiments of the present invention, the beneficial effects that any of the SQL dataset generation methods provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0310] The foregoing has provided a detailed description of an SQL dataset generation method and apparatus provided by embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method of generating a SQL dataset, characterized by, The method comprises: information extraction processing is performed on the text data to obtain structured text data, the structured text data comprising complexity distribution, business rules and query patterns; based on the complexity distribution, business rules and query patterns, a plurality of candidate question and answer pairs are constructed, the candidate question and answer pairs comprising candidate questions and corresponding candidate SQL statements; for each candidate question and answer pair, a first quality score is calculated based on the dimension scores of the candidate SQL statements in a plurality of dimensions and the complexity level of the candidate question, the plurality of dimensions comprising execution dimension, matching dimension and syntax dimension; for each candidate question and answer pair, a second quality score is calculated by calculating the confidence of the candidate SQL statements in a plurality of levels, the plurality of levels comprising text units and clauses; the first quality score and the second quality score are fused to determine a target SQL dataset from the candidate question and answer pairs.

2. The method of claim 1, wherein, The information extraction processing on the text data to obtain structured text data comprises: obtaining text data in a business scenario and data types corresponding to each text data; dividing the text data into text blocks and performing vector storage processing on the text blocks to obtain a text knowledge base, the text knowledge base comprising text block vectors and database structure information; determining specified text blocks and specified structure information from text blocks of a specified data type and database structure information according to a sampling strategy; analyzing the specified structure information and the specified text blocks to extract specified information to obtain structured text data.

3. The method of claim 2, wherein, The determination of specified text blocks and specified structure information from text blocks of a specified data type and database structure information according to a sampling strategy comprises: randomly sampling a first proportion of first text blocks from text blocks of a specified data type; for each first text block, retrieving first structure information corresponding to the first text block and the number of occurrences of the first structure information in the database structure information; determining specified structure information and second structure information from the database structure information according to the number of occurrences of the first structure information; collecting a second proportion of second text blocks from text blocks corresponding to the second structure information; determining specified text blocks according to the first text blocks and the second text blocks.

4. The method of claim 2, wherein, The analysis of the specified structure information and the specified text blocks to extract specified information to obtain structured text data comprises: combining the specified data type, specified structure information and specified text blocks with an extraction template to obtain extraction cues, the extraction cues comprising data cleaning rules, data extraction rules and output formats, the data extraction rules comprising query pattern mining rules, complexity analysis rules and business extraction rules; using the extraction cues to guide a large language model to clean the specified text blocks according to the data cleaning rules to obtain cleaned text blocks; using the extraction cues to guide the large language model to extract specified information from the cleaned text blocks according to the data extraction rules; organizing the specified information according to the output format to obtain structured text data.

5. The method of claim 1, wherein, The method comprises the following steps: combining the structured text data with a first template to obtain a first prompt word, wherein the first prompt word comprises problem generation requirements related to the complexity distribution, business rules and query patterns; using the first prompt word to guide a large language model to process the structured text data according to the problem generation requirements to generate a plurality of candidate questions with different complexity levels; for each candidate question, a plurality of candidate SQL statements corresponding to the candidate question are generated by using a plurality of different SQL generation strategies; combining the candidate question with the candidate SQL statements to obtain a plurality of candidate question and answer pairs.

6. The method of claim 1, wherein, The method comprises the following steps: determining an execution score of the candidate question and answer pair according to an execution result of the candidate SQL statement; combining the candidate question and answer pair with the execution score greater than a specified value as a specified question and answer pair, wherein the specified question and answer pair comprises a candidate question and a specified SQL statement; parsing the syntax structure of the specified SQL statement and calculating a syntax score based on the syntax structure; using a large language model to analyze the matching degree of the candidate question and the specified SQL statement to obtain a matching score; for each specified SQL statement, calculating a first quality score based on the complexity level of the candidate question, the execution score corresponding to the specified SQL statement, the syntax score and the matching score.

7. The method of claim 6, wherein, The method comprises the following steps: for each candidate question, obtaining a plurality of specified SQL statements corresponding to the candidate question; for each specified SQL statement, obtaining the return content corresponding to the specified SQL statement; determining a consistency score according to the consistency of the return content of the plurality of specified SQL statements; determining a dimension weight corresponding to each dimension based on the complexity level of the candidate question; based on the dimension weight, fusing the consistency score, the execution score, the syntax score and the matching score of the specified SQL statement to obtain a first quality score.

8. The method of claim 1, wherein, The method comprises the following steps: for each candidate SQL statement, obtaining clauses, text units and unit types corresponding to the text units in the candidate SQL statement; determining an adjustment factor of the text unit according to the text unit and the business rules; calculating the confidence of each text unit according to the unit type corresponding to each text unit and the adjustment factor; for each sentence, aggregating the confidence of all text units in the clause to obtain a clause confidence; for the candidate SQL statement, performing geometric mean processing on the clause confidence of all clauses in the candidate SQL statement to obtain a second quality score.

9. The method of claim 8, wherein, The calculating the confidence of each text unit according to the unit type corresponding to each text unit and the adjustment factor comprises: obtaining a type weight and an influence coefficient of the text unit according to the unit type corresponding to the text unit; obtaining a unit probability corresponding to the text unit in the large language model; calculating a basic weight of the text unit by using the unit probability, the influence coefficient and the type weight; calculating an average value of the basic weights of all text units in a specified clause to obtain an average weight, the specified clause being a clause to which the text unit belongs; calculating the confidence of the text unit by using the average weight and the adjustment factor.

10. A SQL dataset generation apparatus for implementing the method of any one of claims 1 to 9, characterized in that, The device comprises: a structured module configured to perform information extraction processing on text data to obtain structured text data, the structured text data comprising complexity distribution, business rules and query patterns; a candidate module configured to construct a plurality of candidate question and answer pairs based on the complexity distribution, the business rules and the query patterns, the candidate question and answer pairs comprising a candidate question and a corresponding candidate SQL statement; a first calculation module configured to calculate, for each candidate question and answer pair, a first quality score based on a dimension score of the candidate SQL statement in a plurality of dimensions and a complexity level of the candidate question, the plurality of dimensions comprising an execution dimension, a matching dimension and a syntax dimension; a second calculation module configured to calculate, for each candidate question and answer pair, a confidence of the candidate SQL statement in a plurality of levels to obtain a second quality score, the plurality of levels comprising a text unit and a clause; a generation module configured to fuse the first quality score and the second quality score to determine a target SQL dataset from the candidate question and answer pairs.

Images