SQL data set generation method and device

Abstract

The invention discloses an SQL data set generation method and device, and relates to the technical field of artificial intelligence. The method comprises the steps of performing information extraction processing on text data to obtain structured text data including complexity distribution, business rules and query modes; constructing a plurality of candidate question and answer pairs based on the structured text data; for each candidate question and answer pair, calculating a first quality score based on the dimension scores of the candidate SQL statements in the execution dimension, the matching dimension and the grammar dimension and the complexity level of the candidate questions; calculating confidence coefficients of the candidate SQL statements on text units and clause levels to obtain a second quality score; and fusing the first quality score and the second quality score, and determining a target SQL data set. According to the method and the device, the high-quality target SQL data set can be quickly generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a SQL data set generation method and device. BACKGROUND

[0002] With the rapid development of artificial intelligence and large language model technology, Text2sql technology has attracted widespread attention and application in the field of industrial-level vertical enterprise-level data analysis. The landing of Text2sql technology in the industrial vertical field requires the accumulation of a large amount of high-quality data to build and optimize algorithm strategies, among which question and answer data composed of questions and corresponding SQL are particularly critical.

[0003] Traditional question and answer data acquisition methods are highly dependent on manual annotation, which not only has high cost and low efficiency, but also is difficult to meet the needs of large-scale applications. Although automatic methods based on large language models can produce question and answer data in batches, due to the inherent uncertainty of large language models, there may be a large number of low-quality question and answer pairs in the batch-generated data. These low-quality data introduced into the business process will also seriously affect the performance of the model and the final application effect. Therefore, there is an urgent need for a way to quickly generate a large amount of question and answer data and to systematically evaluate it to achieve the automated production of high-quality SQL question and answer data. SUMMARY

[0004] In view of the above problems, the purpose of the present application is to provide a SQL data set generation method and device, which can automatically generate SQL question and answer data and automatically evaluate and screen the SQL question and answer data from multiple angles, thereby generating a high-quality target SQL data set.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] In one aspect, the present application provides a SQL data set generation method, which comprises:

[0007] performing information extraction processing on the text data to obtain structured text data, the structured text data comprising complexity distribution, business rules and query patterns;

[0008] based on the complexity distribution, business rules and query patterns, constructing a plurality of candidate question and answer pairs, the candidate question and answer pairs comprising candidate questions and corresponding candidate SQL statements;

[0009] for each candidate question and answer pair, calculating a first quality score based on the dimension scores of the candidate SQL statements in multiple dimensions and the complexity level of the candidate question, the multiple dimensions comprising execution dimension, matching dimension and syntax dimension;

[0010] For each candidate question and answer pair, a confidence of the candidate SQL statement on multiple levels is calculated, to obtain a second quality score, the multiple levels including a text unit and a clause;

[0011] The first quality score and the second quality score are fused to determine a target SQL dataset from the candidate question and answer pairs.

[0012] In another aspect, an embodiment of the present application further provides an SQL dataset generation apparatus, which comprises:

[0013] A structured module is configured to perform information extraction processing on text data, to obtain structured text data, the structured text data comprising a complexity distribution, a business rule and a query pattern;

[0014] A candidate module is configured to construct multiple candidate question and answer pairs based on the complexity distribution, the business rule and the query pattern, the candidate question and answer pairs comprising a candidate question and a corresponding candidate SQL statement;

[0015] A first calculation module is configured to calculate, for each candidate question and answer pair, a first quality score based on a dimension score of the candidate SQL statement on multiple dimensions and a complexity level of the candidate question, the multiple dimensions comprising an execution dimension, a matching dimension and a syntax dimension;

[0016] A second calculation module is configured to calculate, for each candidate question and answer pair, a confidence of the candidate SQL statement on multiple levels, to obtain a second quality score, the multiple levels comprising a text unit and a clause;

[0017] A generation module is configured to fuse the first quality score and the second quality score, to determine a target SQL dataset from the candidate question and answer pairs.

[0018] In another aspect, an embodiment of the present application further provides an electronic device comprising a processor and a memory, the memory storing a plurality of instructions; the processor loads the instructions from the memory to perform steps in any of the SQL dataset generation methods provided by embodiments of the present application.

[0019] In another aspect, an embodiment of the present application further provides a computer readable storage medium storing a plurality of instructions, the instructions being adapted to be loaded by a processor to perform steps in any of the SQL dataset generation methods provided by embodiments of the present application.

[0020] In another aspect, an embodiment of the present application further provides a computer program product comprising computer programs / instructions, the computer programs / instructions being executed by a processor to implement steps in any of the SQL dataset generation methods provided by embodiments of the present application.

[0021] The technical scheme provided by the application has at least the following beneficial effects:

[0022] In the embodiment of the application, structured text data is extracted from text data, and a plurality of candidate question-answer pairs are automatically constructed by using complexity distribution, business rules and query patterns in the structured text data; for each candidate question-answer pair, a dimension score in a plurality of dimensions is calculated to obtain a first quality score; a confidence in a plurality of levels is calculated to obtain a second quality score, and finally, a SQL data set is screened from the candidate question-answer pairs based on the first quality score and the second quality score. The application automatically extracts information from text data and constructs question-answer pairs, and the calculation of the first quality score and the second quality score includes a plurality of dimensions and a plurality of levels, which can ensure that the candidate question-answer pairs are evaluated from multiple angles, and thus a high-quality target SQL data set can be quickly generated. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0024] Figure 1 is a schematic diagram of an application scenario of the SQL data set generation method provided by the embodiment of the application;

[0025] Figure 2 is a flowchart of the SQL data set generation method provided by the embodiment of the application;

[0026] Figure 3 is a schematic diagram of determining a specified text block provided by the embodiment of the application;

[0027] Figure 4 is a structural schematic diagram of the SQL data set generation device provided by the embodiment of the application;

[0028] Figure 5 is a structural schematic diagram of the electronic device provided by the embodiment of the application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative effort fall within the scope of the application.

[0030] It can be understood that in the specific embodiments of the present application, the data related to user information and the like need to obtain user permission or consent, and the collection, use and processing of the related data need to comply with the relevant laws, regulations and standards of the country and region.

[0031] The embodiment of the present application provides a SQL data set generation method, which can be referred to Figure 1 , which shows an application scenario diagram of a SQL data set generation method. Among them, the application scenario can include a terminal 101 and a server 102, and the terminal 101 and the server 102 can exchange data through a network, and the terminal 101 can install an application related to question and answer. Among them, the terminal 101 can be a mobile phone, a tablet computer, a smart Bluetooth device, a computer, a large screen device, a robot, etc.; the server 102 can be a single server, or a server cluster composed of multiple servers.

[0032] The user can send text data to the server 102 through the terminal 101, so that the server 102 performs information extraction processing on the text data to obtain structured text data, and the structured text data includes complexity distribution, business rules and query mode; based on the complexity distribution, the business rules and the query mode, a plurality of candidate question and answer pairs are constructed, the candidate question and answer pair includes a candidate question and a corresponding candidate SQL statement; for each candidate question and answer pair, based on the dimension score of the candidate SQL in multiple dimensions and the complexity level of the candidate question, a first quality score is calculated, the multiple dimensions include execution dimension, matching dimension and syntax dimension; for each candidate question and answer pair, the confidence of the candidate SQL statement in multiple levels is calculated to obtain a second quality score, and the multiple levels include text unit and clause; the first quality score and the second quality score are fused to determine the target SQL data set from the candidate question and answer pair.

[0033] After the server 102 generates the target SQL data set, the server 102 can store the target SQL data set to a fixed position, and return the target SQL data set creation success and the address information of the fixed position to the terminal 101, so that the user can view and use the target SQL data set.

[0034] Then the server 102 sends the reply text to the terminal 101, so as to show the reply text to the user.

[0035] In this embodiment, a SQL data set generation method is provided, as shown in Figure 2 , the specific process of the SQL data set generation method can be as follows:

[0036] S110, information extraction processing is performed on the text data to obtain structured text data.

[0037] The text data is text information collected from a business scenario, and the structured text data is data obtained by extracting and arranging specific data from the text data, and is the basis for generating a SQL dataset subsequently.

[0038] Optionally, when the information extraction processing is performed on the text data to obtain the structured text data, text data in a business scenario and a data type corresponding to each text data can be acquired; the text data is divided into text blocks, and vector storage processing is performed on the text blocks to obtain a text knowledge base, the text knowledge base including text block vectors and database structure information; specified text blocks and specified structure information are determined from the text blocks of a specified data type and the database structure information according to a sampling strategy; the specified structure information and the specified text blocks are analyzed to extract specified information to obtain the structured text data.

[0039] Various text information is collected from an actual business scenario. The format of the text information can be a format that can be parsed into text, for example, PDF, Text, word, markdown, etc. In order to facilitate subsequent processing, a plurality of data types corresponding to the text data can also be constructed according to the actual business scenario, for example, in the embodiment of the application, the data types can include business requirements, technical specifications, user manuals, historical query logs, data dictionaries, business rules, etc. For each text data, the text data can be divided into text blocks, and the text blocks are then vectorized to generate corresponding text vectors.

[0040] For each text block, the content of the text block, the text vector, the metadata, and the data type are stored in a document vector library together, and the document vector library can support efficient semantic retrieval and similarity matching. Then a structure meta-database is constructed, which stores table structures, relationships, and statistical information. The table structures can include table names, columns, etc.; the relationships can include foreign keys, association relationships, etc.; and the statistical information can include data sample numbers, null value rates, etc. In addition, a knowledge index can also be established, for example, an association mapping between text data and structure information, a mapping of business terms to database elements, etc. The document vector library and the structure meta-database can jointly constitute a text knowledge base, the document vector library can contain text block vectors corresponding to each text block, the structure meta-database can include all data structure information, and the knowledge index can be selectively put into the text knowledge base according to actual needs.

[0041] The text data extracted from the business scenario is stored in the text knowledge base. Through the text knowledge base, the data type corresponding to each text block can be obtained, and specified text blocks and specified structure information can be determined from the text blocks of a specified data type and the database structure information according to a sampling strategy, for subsequent use when extracting structured data.

[0042] The specified data types are any one or more of the data types, which can be set according to actual needs. When extracting structured data, a large language model can be used for processing. Since the input length of the large language model is limited, part of the data can be extracted from the text blocks of the specified data types and the database structure information for analysis.

[0043] Optionally, part of the data can be randomly sampled from the text blocks and the database structure information to obtain specified text blocks and specified structure information. By analyzing the specified text blocks and the specified structure information, structured text data can be extracted.

[0044] Optionally, to ensure the accuracy of the extracted structured text data, when determining the specified text blocks and the specified structure information from the text blocks of the specified data types and the database structure information according to a sampling strategy, a first proportion of first text blocks can be randomly sampled from the text blocks of the specified data types; for each first text block, first structure information corresponding to the first text block and the number of occurrences of the first structure information are searched in the database structure information; the specified structure information and second structure information are determined from the database structure information according to the number of occurrences of the first structure information; second text blocks of a second proportion are collected from text blocks corresponding to the second structure information; and the first text blocks and the second text blocks are used as the specified text blocks.

[0045] Text blocks of the specified data types are obtained, and a first proportion of the text blocks are randomly sampled as first text blocks, where the first proportion can be set according to actual needs. In an embodiment of the present application, the first proportion can be 10%. For the first text blocks obtained by sampling, a first text block vector corresponding to the first text blocks is obtained, and the first text block vector is used to search in the database structure information to recall a preset number of first structure information. Since each first text block can recall a preset number of first structure information, different first text blocks can recall the same first structure information. Therefore, for each first structure information, the number of occurrences of each first structure information corresponding to the first structure information can be counted.

[0046] The number of occurrences of each first structure information corresponding to the first structure information can be used to determine the specified structure information and the second structure information from the database structure information. The specified structure information is structure information related to the text blocks of the specified data types. As an embodiment, all the first structure information can be directly used as the specified structure information. As another embodiment, the first structure information with the highest number of occurrences can be used as the specified structure information according to the number of occurrences of the first structure information. The preset number is an integer, which can be set according to actual needs.

[0047] The second structural information refers to structural information not involved or less involved in the first text block. Optionally, the first structural information with an occurrence number less than a specified number of occurrence numbers can be taken as the second structural information according to the occurrence numbers of the first structural information. Of course, all structural information other than the second structural information can also be taken as the second structural information.

[0048] To ensure that the sampled text block can cover most of the database structural information, sampling can be continued based on the second structural information. Optionally, a mapping relationship between the structural information and the text block can be obtained, the text block corresponding to the second structural information is obtained based on the mapping relationship, and the second proportion of the second text block is continued to be randomly sampled therefrom. The second proportion is a dynamic proportion, and a preset proportion range can be pre-set. The second proportion is in the preset proportion range, for example, the preset proportion range can be 5%-20%.

[0049] After the second text block is collected from the text block corresponding to the second structural information according to the second proportion, the first text block and the second text block can be taken together as the specified text block.

[0050] Optionally, reference can be made to Figure 3 , a schematic diagram for determining the specified text block is shown. To ensure that the collected text block covers all the database structural information as much as possible, after the second text block is determined, the first text block and the second text block can be taken together as a new first text block. The new first text block is returned to execute the retrieval of the new first structural information corresponding to the first text block in the database structural information, and the coverage rate of the first structural information in the database structural information is calculated. When the coverage rate does not reach the specified coverage rate, the second text block is continued to be sampled according to the original mode based on the occurrence number of the new first structural information, until the coverage rate of the first structural information of the first text block reaches the specified coverage rate, and the first text block is taken as the specified text block.

[0051] Alternatively, the number of times of sampling the second text block can be recorded. If the number of times of sampling the second text block reaches a preset number of times, even if the coverage rate does not reach the specified coverage rate, the new first text block obtained is taken as the specified text block and the process is ended. The occurrence number of the first structural information can facilitate the adjustment of the result information that needs to be supplemented each time, so as to ensure that the specified text block obtained by the final sampling can cover most of the structural information.

[0052] The determined specified text block and the specified structural information are analyzed, and the specified information is extracted to obtain the structured text data. The specified information can be pre-set content that needs to be extracted, for example, business entities, data relationships, query patterns, business rules, explanations of professional terms, query complexity, etc. The extracted content is combined according to a fixed format, and the structured text data is obtained.

[0053] As an implementation form, when the specified structure information and the specified text block are analyzed and the specified information is extracted to obtain the structured text data, the specified data type, the specified structure information and the specified text block can be combined with an extraction template to obtain an extraction prompt word, the extraction prompt word including a data cleaning rule, a data extraction rule and an output format, the data extraction rule including a query pattern mining rule, a complexity analysis rule and a business extraction rule; the extraction prompt word is used to guide a large language model to clean the specified text block according to the data cleaning rule to obtain a cleaned text block; the extraction prompt word is used to guide the large language model to extract the specified information from the cleaned text block according to the data extraction rule; and the specified information is organized according to the output format to obtain the structured text data.

[0054] The extraction template is a prompt word template for extracting the structured text data, and the prompt word template can include a plurality of to-be-filled slots, such as a type slot, a content slot and a structure information slot. The specified data type, the specified structure information and the specified text block are filled into the corresponding to-be-filled slots to obtain the extraction prompt word.

[0055] The extraction prompt word can include requirements related to data processing, such as a data cleaning rule, a data extraction rule and an output format. The data cleaning rule refers to a basic rule for cleaning the input specified text block. The data extraction rule can include extraction rules of each specified information, such as a query pattern, a mining rule, a complexity analysis rule and a business extraction rule. The output format is a final organization mode of the extracted specified information, and the specified information is organized to become the structured text data.

[0056] The extraction template can be set according to actual needs. In the embodiment of the present application, the extraction template can be:

[0057] “Task: Apply the following production materials to the processing task, and extract structured knowledge.

[0058] Input type: [document_type]

[0059] Input content: [document_content]

[0060] Related database Schema (TopK):

[0061] [topk_schema_info]

[0062] Please complete the following processing task:

[0063] 1. Data cleaning and standardization:

[0064] Analyze input content, perform the following cleaning operations:

[0065] - Identify and remove content irrelevant to database queries (e.g., headers and footers, copyright information, redundant punctuation, markdown internal formatting codes, etc.)

[0066] - Standardize professional terms: unify synonyms (e.g., "customer" / "user" / "Customer" to a standard term)

[0067] - Correct obvious spelling errors and formatting issues

[0068] - Extract core business logic descriptions, remove redundant expressions

[0069] Output cleaned core content (200 words or less)

[0070] 2. Information extraction and structuring (based on business document and TopK Schema deep analysis):

[0071] 2.1 Business entity identification:

[0072] - Identify all business entities (e.g., customers, orders, products, etc.) from the document

[0073] - Traverse each entity and try to map it to a specific table in the TopK Schema

[0074] 2.2 Data relationship extraction:

[0075] - Analyze the business processes and data flow described in the document (if any, otherwise skip the relationship extraction process)

[0076] - Identify potential JOIN relationships and association conditions between tables in the Schema

[0077] - Extract primary-foreign key relationships, one-to-many / many-to-many relationships

[0078] 2.3 Query pattern mining:

[0079] - Infer possible query requirements based on business descriptions (e.g., statistical analysis, data filtering, report generation)

[0080] - Identify the complexity level of the query (simple / medium / complex)

[0081] - Generate typical query scenario examples

[0082] 2.4 Constraint rule extraction:

[0083] - Identify business constraints (e.g. VIP discount rules, order amount limits)

[0084] - Extract data validation rules (e.g. field non-empty, value range)

[0085] - Label special business logic (e.g. tiered pricing, time window restrictions)

[0086] 2.5 Professional Term Explanation:

[0087] - Extract domain-specific terms and provide definitions

[0088] - Establish mapping of terms to Schema elements (table names, column names)

[0089] - Explain usage of terms in different scenarios

[0090] 2.6 Complexity Assessment:

[0091] - Assess query complexity based on Schema structure

[0092] - Calculate table association density, possibility of nested queries

[0093] Output Format (strictly match json format):

[0094] {

[0095] "cleaned_content": "cleaned core content",

[0096] "entities": [

[0097] {"name": "entity name", "type": "entity type", "table_mapping": "corresponding table name", "description": "description"}

[0098] ],

[0099] "relationships": [

[0100] {"source": "source table", "target": "target table", "type": "relationship type", "join_condition": "join condition", "description": "relationship description"}

[0101] ],

[0102] "query_patterns": [

[0103] {"pattern": "query pattern", "frequency": "frequency", "complexity": "complexity", "involved_tables": ["involved tables"], "example": "example"}

[0104] ],

[0105] "business_rules": [

[0106] {"rule": "rule description", "tables_involved": ["tables involved"], "columns_involved": ["columns involved"], "validation": "validation logic"}

[0107] ],

[0108] "domain_terms": [

[0109] {"term": "term", "definition": "definition", "table_column_mapping": "table.column mapping", "context": "context of use"}

[0110] ],

[0111] "complexity_metrics": {

[0112] "table_count": "table count",

[0113] "avg_columns_per_table": "average columns per table",

[0114] "relationship_density": "relationship density",

[0115] "query_complexity_distribution": {"simple": 0.3, "medium": 0.5, "complex": 0.2}

[0116] }

[0117] }”

[0118] Wherein, [document_type] can be filled with a specified data type; [document_content] can be filled with a specified text block; and [topk_schema_info] can be filled with specified structure information, and the filling is completed to obtain an extracted prompt word.

[0119] Based on the prompt template, the extracted prompt word obtained after filling the corresponding information can include specific data cleaning rules and specific extraction rules of the specified information.

[0120] The structured text data in the embodiment of the application can include complexity distribution, business rules and query patterns, and the extracted specified information can further include business entities, data relationships, explanations of professional terms, etc. The extracted prompt word is input into a large language model to guide the large language model to perform data processing according to the data cleaning rules and extraction rules therein, and the extracted specified information is assembled according to an output format to obtain structured text data.

[0121] In the process of business entity recognition, the large language model can recognize all business entities such as customers, orders and products from the specified text block, and then traverse each recognized entity and compare it with specific tables in the specified structure information. In the process of data relationship extraction, the business process and data flow described in the specified text block can be analyzed to identify the relationship between potential database tables and the association conditions, and the primary-foreign key relationship can be extracted. In the process of query pattern mining, possible query requirements such as statistical analysis, data filtering and report generation can be inferred from the business description in the specified text block, the complexity of these query requirements is identified to determine the corresponding complexity level, and typical query examples are generated. In the process of business rule extraction, the constraint conditions of the business can be identified from the specified text block, such as discount rules and order amount limits, and special business logic such as ladder prices and time window limits can also be annotated. In the process of professional term explanation, domain-specific nouns can be extracted from the specified text block and corresponding definitions are given, the mapping between nouns and table names and column names is established, and the usage of nouns in different scenarios is explained. In the process of complexity distribution extraction, the query complexity can be evaluated based on the specified structure information, the association density between tables can be calculated, and the possibility of nested queries can be obtained to obtain the complexity distribution.

[0122] The extracted specified information is organized according to the output format to form structured text data. Through the information extraction process, the useless information in the text data can be removed, and the structured text data contains content related to the business scenario, such as professional term explanation and business rules, which can deepen the model's understanding of the business in subsequent processing and ensure the generation of high-quality SQL data sets.

[0123] S120, constructing a plurality of candidate question and answer pairs based on the complexity distribution, business rules and query patterns.

[0124] After the structured text data is extracted, a plurality of candidate question and answer pairs can be constructed based on the complexity distribution, business rules and query patterns therein. Each candidate question and answer pair contains a candidate question and a corresponding candidate SQL statement.

[0125] Optionally, when constructing a plurality of candidate question and answer pairs, the structured text data can be combined with a first template to obtain a first prompt word, the first prompt word including problem generation requirements related to the complexity distribution, business rules and query patterns; using the first prompt word, a large language model is guided to process the structured text data according to the problem generation requirements to generate a plurality of candidate questions of different complexity levels; for each candidate question, a plurality of candidate SQL statements corresponding to the candidate question are generated using a plurality of different SQL generation strategies; the candidate question and the candidate SQL statement are combined to obtain a plurality of candidate question and answer pairs.

[0126] The first template is a prompt word template for generating a candidate question, which can be set in advance according to actual needs. The first template can include a corresponding structured filling slot, and the structured text data extracted as described above is filled into the structured filling slot to obtain the first prompt word. The first prompt word can be used to guide the large language model to generate a candidate question, and the first prompt word can include generation requirements of the candidate question, which are related to the complexity distribution, business rules and query patterns in the structured text data.

[0127] The obtained first prompt word can be input into the large language model to guide the large language model to process the structured text data according to the problem generation requirements to generate a plurality of candidate questions of different complexity levels. The problem generation requirements can include generating candidate questions of different complexity levels according to the complexity distribution, generating candidate questions that conform to business logic using business rules, and generating questions in common scenarios based on query patterns. As for the database table information involved in the candidate question, it can be obtained from the mapped table name in the entity and the extracted data relationship in the structured text data.

[0128] It should be noted that the first prompt word can also include information such as the specific number of candidate questions to be generated. In the embodiment of the present application, the first template can be:

[0129] “Task: Based on the given database structure and business scenario, generate diversified natural language query questions.

[0130] Input structured knowledge:

[0131] - Entity information: [StructuredKnowledge.entities]

[0132] - Relationship information: [StructuredKnowledge.relationships]

[0133] - Business rules: [StructuredKnowledge.business_rules]

[0134] - Query pattern: [StructuredKnowledge.query_patterns]

[0135] - Complexity distribution: [StructuredKnowledge.complexity_metrics]

[0136] Generation requirements:

[0137] 1. Generate problems of different complexities based on the distribution in complexity_metrics.

[0138] 2. Make full use of business_rules to generate queries that conform to business logic.

[0139] 3. Generate common query scenarios based on query_patterns

[0140] 4. Obtain the relevant table information from the entities' table_mapping.

[0141] 5. The questions are expressed naturally and in a variety of ways.

[0142] Please generate 10 query questions, in the following format:

[0143] {

[0144] "question": "Query question text",

[0145] "complexity": "simple / medium / complex",

[0146] "involved_tables": [...], # Infer from entities.table_mapping and relationships

[0147] "query_type": "selection / aggregation / join / subquery",

[0148] "business_rules_applied": ["business rules involved"]

[0149] }”

[0150] Among them, the input structured knowledge, that is, the corresponding structured text data can be filled in to generate the first prompt word. The first prompt word is input into the large language model, so as to guide the large language model to generate a plurality of candidate questions of different complexity levels according to the problem generation requirement.

[0151] Based on the guidance of the first prompt word, each candidate question generated by the large language model can include the question itself, complexity, involved data table, question type, and dependent business rules, etc. Exemplarily, the candidate question can be:

[0152] {

[0153] "question": "Query the total order amount of all VIP customers and apply a 9% discount.",

[0154] "complexity": "medium",

[0155] "involved_tables": ["customers", "orders", "order_items"],

[0156] "query_type": "aggregation",

[0157] "business_rules_applied": ["VIP customer 9% discount"]

[0158] }

[0159] For each candidate question, a plurality of different SQL statement generation strategies can be used to generate a plurality of candidate SQL statements corresponding to the candidate question. The candidate question and each candidate SQL statement are combined to form a plurality of candidate question and answer pairs.

[0160] Optionally, when generating the candidate SQL statement corresponding to the candidate question, a plurality of existing different SQL generation algorithms can be used to process the candidate question to obtain a plurality of candidate SQL statements. For example, a plurality of specially trained SQL statement generation models are used to generate a plurality of candidate SQL statements corresponding to the candidate question, wherein the model architecture and training samples used by different SQL statement generation models are different.

[0161] Optionally, a prompt word can be constructed based on the candidate question, and the prompt word is input into the large language model to generate the candidate SQL statement. The prompt word can include various context information such as database structure information, reference examples, and related complete documents. By setting different prompt words or introducing different contexts in the prompt word, different ways of generating candidate SQL statements can also be considered.

[0162] Each candidate question is combined with the generated multiple candidate SQL statements to obtain multiple candidate question and answer pairs.

[0163] In S130, for each candidate question and answer pair, a first quality score is calculated based on the dimension scores of the candidate SQL statement in multiple dimensions and the complexity level of the candidate question.

[0164] For each candidate question and answer pair, a first quality score of each candidate question and answer pair can be calculated based on the dimension scores of the candidate SQL in multiple dimensions and the complexity level of the candidate question. The multiple dimensions can include execution dimension, matching dimension, and syntax dimension. The dimension scores corresponding to the candidate SQL statement are calculated in the multiple dimensions, and then the first quality score is calculated using the dimension scores and the complexity level of the candidate question. The first quality score can be used to measure the quality of the candidate question and answer pair.

[0165] As an implementation, when calculating the first quality score, the execution score of the candidate question and answer pair can be determined according to the execution result of the candidate SQL statement. The candidate question and answer pair with an execution score greater than a specified value is regarded as a specified question and answer pair, which includes a candidate question and a specified SQL statement. The syntax structure of the specified SQL statement is parsed, and a syntax score is calculated based on the syntax structure. The matching degree between the candidate question and the specified SQL statement is analyzed using a large language model to obtain a matching score. For each specified SQL statement, the complexity level of the candidate question, the specified score corresponding to the specified SQL statement, the syntax score, and the matching score are used to calculate the first quality score.

[0166] For each candidate SQL statement in the candidate question and answer pair, the candidate SQL statement can be executed to obtain the execution result corresponding to the candidate SQL statement. The execution result can include execution status, execution time, returned result, and the like. The execution score can be calculated using the execution result.

[0167] Optionally, a calculation rule of the execution score can be preset, and the execution score is calculated according to the calculation rule. For example, in the embodiment of the present application, the execution state can include execution success or execution failure; the return result can include null or non-null; if the execution state is execution success and the return result is non-null, the execution score can be set as a first score; if the execution state is execution success but the return result is null, the execution score can be set as a second score; if the execution state is execution failure, the execution score can be set as a third score. The first score is greater than the second score, and the second score is greater than the third score.

[0168] After the execution score is calculated, candidate question-answer pairs with the execution score greater than a specified value can be screened out and used as specified question-answer pairs in subsequent processes. The specified value can be set according to actual needs, and in the embodiment of the present application, the numerical value of the specified value can be the same as the third score. That is, in the embodiment of the present application, the difference between the specified question-answer pairs and the candidate question-answer pairs is that there are candidate SQL statements with execution failure in the candidate question-answer pairs, while the specified SQL statements in the specified question-answer pairs can all be successfully executed.

[0169] For the specified SQL statements in the specified question-answer pairs, the syntax structure of the specified SQL statements can be continuously parsed, and a syntax score can be calculated based on the parsed syntax structure. First, the specified SQL statements can be parsed to obtain a parsing result, wherein the parsing result can include a parsing state and a syntax tree, and the syntax score of the specified SQL statements can be calculated according to the parsing state and the syntax tree.

[0170] Optionally, a calculation rule of the syntax score can be preset, and the syntax score is calculated according to the calculation rule. For example, in the embodiment of the present application, the parsing state can include parsing success or parsing failure, and the parsed syntax tree can include select and group by; a parsing score can be calculated according to the parsing state; a syntax consistency score can be calculated according to select and group by in the syntax tree; and the syntax score can be calculated by multiplying the parsing score and the syntax consistency score.

[0171] Specifically, if the parsing state is parsing success, the parsing score can be set as a fourth score; if the parsing state is parsing failure, the parsing score can be set as a fifth score; it is determined whether non-aggregated columns in select all exist in group by; if yes, the syntax consistency score can be set as a sixth score; if no, the syntax consistency score can be set as a seventh score; finally, the product of the parsing score and the syntax consistency score is calculated to obtain the syntax score.

[0172] The execution score and the syntax score calculated in the foregoing are evaluated from the dimension of the SQL statement, and to ensure the comprehensiveness of the evaluation, the matching degree between the candidate question and the specified SQL statement in the specified question and answer pair can be further analyzed.

[0173] Optionally, when analyzing the matching degree between the candidate question and the specified SQL statement, the analysis can be performed from four aspects of query target matching degree, filter condition matching degree, aggregation operation matching degree, and business logic matching degree.

[0174] The query target matching degree can be to judge whether the SELECT clause in the specified SQL statement correctly returns the information required by the candidate question, whether the required fields are included, and the like, and accordingly give a score of 0-1 as the query target matching degree. The filter condition matching degree can be to judge whether the WHERE clause in the specified SQL statement correctly implements the filter condition in the candidate question, whether the condition logic is consistent with the description in the candidate question, and the like, and accordingly give a score of 0-1 as the filter condition matching degree. The aggregation operation matching degree can be to judge whether the aggregation function such as SUM, COUNT, AVG, and the like in the specified SQL statement is consistent with the candidate question, whether the grouping in the group by is correct, and the like, and accordingly give a score between 0-1 as the aggregation operation matching degree. The business logic matching degree can be to judge whether the specified SQL statement implements the business rules, whether the JOIN relationship is reasonable, and the like, and accordingly give a score between 0-1 as the business logic matching degree. It should be noted that the greater the values of the query target matching degree, the filter condition matching degree, the aggregation operation matching degree, and the business logic matching degree, the more the specified SQL statement meets the corresponding requirements.

[0175] The matching score can be calculated by performing a weighted average calculation on the query target matching degree, the filter condition matching degree, the aggregation operation matching degree, and the business logic matching degree.

[0176] As an implementation manner, the matching degree of each aspect can be evaluated by calling a large language model and giving a score, and finally the scores of each aspect are weighted and averaged to obtain the matching score. As another implementation manner, a large language model can be called, and a corresponding prompt word can be designed, which can include the calculation logic of the matching degree of each aspect and the calculation logic of the matching score, so that the matching score can be directly calculated.

[0177] For example, the prompt word can be:

[0178] Task: Evaluate the semantic matching degree between natural language questions and SQL queries

[0179] Question: [question]

[0180] SQL: [sql]

[0181] Related Schema: [schema_info]

[0182] Please analyze and evaluate:

[0183] 1. Query Target Matching (0-1 points):

[0184] - Does the SELECT clause of the SQL correctly return the information required by the question?

[0185] - Does it include all the necessary fields?

[0186] - Are there any extra or missing fields?

[0187] 2. Filter Condition Matching (0-1 points):

[0188] - Does the WHERE clause correctly implement the filtering conditions in the question?

[0189] - Is the condition logic consistent with the question description?

[0190] 3. Aggregation Operation Matching (0-1 points):

[0191] - Are the aggregation functions (SUM / COUNT / AVG, etc.) consistent with the question requirements?

[0192] - Is the GROUP BY correctly grouped?

[0193] 4. Business Logic Matching (0-1 points):

[0194] - Does the SQL correctly implement the business rules?

[0195] - Are the JOIN relationships reasonable?

[0196] Output format:

[0197] {

[0198] "target_match": 0.95,

[0199] "filter_match": 1.0,

[0200] "aggregation_match": 0.9,

[0201] "business_logic_match": 1.0,

[0202] "overall_semantic_match": 0.96, / / Weighted average

[0203] "explanation": "Briefly explain the score reason"

[0204] }

[0205] Example output:

[0206] {

[0207] "target_match": 1.0,

[0208] "filter_match": 1.0,

[0209] "aggregation_match": 1.0,

[0210] "business_logic_match": 0.95,

[0211] "overall_semantic_match": 0.99,

[0212] "explanation": "SQL correctly implements the requirement of querying the total amount of VIP customer orders, including correct aggregation and filter conditions"

[0213] }

[0214] For each specified question and answer pair, fill in the [question] with the candidate question, fill in the [sql] with the specified SQL statement, and fill in the [schema] with the database structure information retrieved based on the candidate question or the specified SQL statement, and pass it to the large language model. In this way, the large language model can output the matching degree of each aspect and calculate the final matching score.

[0215] At this time, each specified question and answer pair calculates the execution score, syntax score, and matching score. Using these three scores and the complexity level of the candidate question, the first quality score can be calculated.

[0216] As an implementation, when calculating the first quality score, for each candidate question, the corresponding multiple specified SQL statements can be obtained; for each specified SQL statement, the corresponding return content can be obtained; the consistency of the multiple specified SQL statements can be compared according to the return content to obtain a consistency score; the complexity level of the candidate question can be used to determine the dimension weight corresponding to each dimension; and based on the dimension weight, the consistency score, execution score, syntax score, and matching score of the specified SQL statement can be fused to obtain the first quality score.

[0217] The foregoing screening of the specified question and answer pair according to the execution score includes a candidate question and a corresponding specified SQL statement. For each candidate question, a plurality of specified SQL statements corresponding to the candidate question can be obtained. For the plurality of specified SQL statements corresponding to the candidate question, each specified SQL statement can be obtained, and an execution result obtained by executing the specified SQL statement can be obtained. The foregoing execution result includes execution time, returned content, and the like.

[0218] For a candidate question, consistency analysis can be performed by comparing the execution results of the specified SQL statements corresponding to the candidate question. For example, it can be determined whether the returned content of all specified SQL statements corresponding to the candidate question is semantically equivalent, or whether the returned content is the same. If the returned content is semantically equivalent or the same, it can be considered consistent. The consistency score can be set to an eighth score when consistency is confirmed, and set to a ninth score when inconsistency is confirmed.

[0219] In the foregoing embodiment, a plurality of candidate questions of different complexity levels are generated, and each candidate question has a corresponding complexity level. A mapping relationship between each complexity level and a preset weight can be set in advance. Based on the specific complexity level of the candidate question and the mapping relationship, a dimension weight corresponding to each dimension that needs to be used can be determined. For example, the complexity level of the question can include three levels: simple, medium, and complex. The dimension weight corresponding to the simple level is a first weight, the dimension weight corresponding to the medium level is a second weight, and the dimension weight corresponding to the complex level is a third weight. Each dimension weight includes an execution weight, a syntax weight, and a matching weight, and each dimension weight corresponds to one of the foregoing plurality of dimensions.

[0220] The execution score, the syntax score, and the matching score are weighted and calculated using the dimension weight to obtain a first quality score. Alternatively, the result obtained after the weighted calculation can be added to the consistency score to obtain the first quality score. Alternatively, an additional score value can be determined based on the range in which the consistency score is located, and the result obtained after the weighted calculation can be added to the additional score value to obtain the first quality score.

[0221] It should be noted that since one candidate question corresponds to multiple specified SQL statements, while calculating the first quality score, the best SQL statement can be selected from the multiple specified SQL statements corresponding to the candidate question. Alternatively, when determining the best SQL statement, the specified SQL statements can be sorted in descending order of matching scores to obtain a first sequence; the specified SQL statements can be sorted in ascending order of execution time to obtain a second sequence; the score of each specified SQL statement is calculated based on the ranking of the specified SQL statement in the first sequence and the second sequence, and the one with the largest score is selected as the best SQL statement. In this way, the matching degree and the execution time of the best SQL statement determined are both small.

[0222] In S140, for each candidate question and answer pair, the confidence of the candidate SQL statement at multiple levels is calculated to obtain a second quality score.

[0223] For each candidate question and answer pair, in addition to calculating the first quality score described above, a second quality score can be calculated. The first quality score and the second quality score are both scores obtained after evaluating the quality of the question and answer pair, and the difference lies in the different angles of evaluation.

[0224] The second quality score mainly evaluates the candidate SQL statement at different levels from the perspective of confidence, and the levels can include text unit level, clause level, and entire SQL statement level. As an implementation, when calculating the second quality score, for each candidate SQL statement, the clauses, text units, and unit types corresponding to the text units in the candidate SQL statement can be obtained; an adjustment factor can be determined according to the inclusion relationship between the text units and the business rules; the confidence of each text unit can be calculated according to the unit type corresponding to each text unit and the adjustment factor; for each sentence, the confidence of all text units in the clause is summed to obtain the clause confidence; for the candidate SQL statement, the clause confidences of all clauses are geometrically averaged to obtain the second quality score.

[0225] For each candidate SQL statement, clauses therein can be split, and each text unit in the clauses can be split. For example, the candidate SQL statement is "SELECT customer_name, SUM(order_amount) FROM customers c JOIN orders o ON c.customer_id = o.customer_id WHERE order_date > '2023-01-01' GROUP BY customer_name", and the clauses can be split into multiple clauses according to SELECT, FROM, etc. For each clause, each text unit therein can be split, and the text unit can also be referred to as a token.

[0226] For each text unit, multiple unit types can be pre-set, and the definition of each unit type is given, which can be set according to actual needs. In the embodiment of the present application, the unit types can include keyword type, pattern type, operator type, and constant type, etc. The keyword type refers to reserved words in the SQL statement, such as SELECT, FROM, WHERE, JOIN, GROUP BY, etc. The pattern type refers to identifiers related to database structure information, such as table name, column name, alias, etc. The operator type refers to comparison and operation logic symbols, such as =, >, <, AND, OR, BETWEEN, etc. The constant type refers to literal constant, including string, number, date.

[0227] For each text unit, an adjustment factor can be determined according to the inclusion relationship between the text unit and the business rule. Specifically, the business rule in the structured text data can be obtained, the keywords in the business rule can be extracted, and it is detected whether the text involves the keywords in the business rule. If the keywords are involved, the adjustment factor is set to a first specified value. If the keywords are not involved, the adjustment factor is set to a second specified value. The first specified value is greater than the second specified value, and the specific values can be set according to actual needs. In the embodiment of the present application, the first specified value can be set to 0.1, and the second specified value can be set to 0.

[0228] For each text unit, the confidence of each text unit can be calculated using the unit type of the text unit and the adjustment factor. As an implementation, in calculating the confidence of the text unit, the type weight and the influence coefficient of the text unit can be obtained according to the unit type corresponding to the text unit; the unit probability of the text unit in the large language model is obtained; the base weight of the text unit is calculated using the unit probability, the influence coefficient and the type area; the average weight is obtained by calculating the average value of the base weights of all text units in the specified clause, and the average weight is obtained by calculating the average weight of the text unit; and the confidence of the text unit is calculated using the average weight and the adjustment factor.

[0229] For each unit type, the type weight and the influence coefficient corresponding to the unit type can be set in advance, which can be set according to actual needs. In the embodiment of the application, refer to Table 1, which shows the type weight and the influence coefficient corresponding to each unit type.

[0230] Table 1

[0231]

[0232] For each text unit, the unit probability corresponding to the text unit in the large language model can be obtained. Specifically, the large language model can be used to predict each text unit, and the probability of the text unit predicted by the large language model is obtained as the unit probability.

[0233] The unit probability is multiplied by the influence coefficient to obtain the influence weight, and the influence weight is added to the type weight of the text unit to obtain the base weight of the text unit. The clause to which the text unit belongs is taken as the specified clause, and the base weight of each text unit in the specified clause is calculated; and the average weight of the base weights of all text units in the clause is calculated to obtain the average weight. The average weight is added to the adjustment factor as the candidate confidence, and the candidate confidence is compared with the specified confidence, and the smaller value between the two is taken as the confidence of the text unit. For example, the candidate confidence is 0.85, the specified confidence is 1, and 0.85 is smaller than 1, so 0.85 is the confidence of the text unit.

[0234] For each clause, the confidence of all text units in the clause is aggregated to obtain the clause confidence corresponding to the clause. Alternatively, the confidence of all text units can be aggregated by calculating the arithmetic mean of the confidence of all text units as the confidence of the entire clause.

[0235] For the candidate SQL sentence, which can include multiple clauses, in order to calculate the confidence of the candidate SQL sentence, the confidence of all the clauses can be aggregated. Optionally, the confidence of all the clauses can be geometrically averaged to calculate the second quality score of the candidate SQL sentence.

[0236] S150, fusing the first quality score and the second quality score, determining the target SQL data set from the candidate question and answer pair.

[0237] For each candidate question and answer pair, the first quality score and the second quality score of the candidate SQL sentence in it are calculated. The first quality score and the second quality score are fused to obtain the target quality score of each candidate question and answer pair. Optionally, there are multiple ways to fuse the first quality score and the second quality score, for example, the first quality score and the second quality score can be directly summed to obtain the target quality score.

[0238] The target quality score can be used to screen the target SQL question and answer pair from the candidate question and answer pair, for example, a quality threshold can be set, and the candidate question and answer pair with a target quality score greater than the quality threshold is determined as the target SQL question and answer pair. For another example, all candidate question and answer pairs can be sorted in descending order of the target quality score to obtain a question and answer pair sequence, and the top specified number of candidate question and answer pairs in the question and answer pair sequence are determined as the target SQL question and answer pair, wherein the specified number is an integer and can be set according to actual needs. All the determined target SQL question and answer pairs are determined as the target SQL data set.

[0239] The SQL data set generation scheme provided by the embodiments of the present application can be used in various scenarios, for example, in the intelligent manufacturing scenario, the SQL data set constructed can be used to optimize the SQL generation algorithm, so that the SQL statement can be accurately generated in the intelligent query analysis of manufacturing data, and a solid foundation is provided for the intelligent query analysis in the intelligent manufacturing scenario.

[0240] The method provided by the embodiments of the present application can extract structured text data from text data, and automatically generate candidate question and answer pairs using the structured text data. Then, the candidate question and answer pairs can be evaluated from the execution dimension, the matching dimension and the syntax dimension to obtain the first quality score, the confidence of the candidate question and answer pairs in multiple levels is calculated to obtain the second quality score, and finally the first quality score and the second quality score are fused to screen the target SQL data set. In the absence of a standard SQL statement, the candidate question and answer pairs are comprehensively and reasonably evaluated from multiple angles, a full-automatic process of generation and intelligent evaluation is realized, and a high-quality target SQL data set can be quickly generated.

[0241] To better implement the above methods, this embodiment of the invention also provides an SQL dataset generation device, which can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster composed of multiple servers.

[0242] For example, in this embodiment, the method of the present invention will be described in detail by taking the SQL dataset generation device specifically integrated into the server as an example.

[0243] For example, such as Figure 4 As shown, the SQL dataset generation device 200 may include a structuring module 210, a candidate module 220, a first calculation module 230, a second calculation module 240, and a generation module 250.

[0244] The structuring module 210 is used to extract information from text data to obtain structured text data, which includes complexity distribution, business rules, and query patterns.

[0245] The candidate module 220 is used to construct multiple candidate question-answer pairs based on the complexity distribution, business rules and query patterns. The candidate question-answer pairs include candidate questions and corresponding candidate SQL statements.

[0246] The first calculation module 230 is used to calculate a first quality score for each candidate question-answer pair based on the dimension scores of the candidate SQL statement on multiple dimensions and the complexity level of the candidate question. The multiple dimensions include execution dimension, matching dimension and syntax dimension.

[0247] The second calculation module 240 is used to calculate the confidence of the candidate SQL statement at multiple levels for each candidate question-answer pair, and obtain a second quality score. The multiple levels include text units and clauses.

[0248] The generation module 250 is used to fuse the first quality score and the second quality score to determine the target SQL dataset from the candidate question-answer pairs.

[0249] In some embodiments, the structuring module 210 is specifically used for:

[0250] Obtain the text data in the business scenario, and the data type corresponding to each piece of text data;

[0251] The text data is divided into text blocks, and the text blocks are vectorized and stored to obtain a text knowledge base, which includes text block vectors and database structure information.

[0252] determining, according to a sampling strategy, specified text blocks and specified structure information from the text blocks of the specified data type and the database structure information;

[0253] analyzing the specified structure information and the specified text blocks, and extracting specified information to obtain structured text data.

[0254] In some embodiments, the structured module 210 is specifically configured to:

[0255] randomly sampling a first proportion of first text blocks from the text blocks of the specified data type;

[0256] For each first text block, searching for first structure information corresponding to the first text block and the number of occurrences of the first structure information in the database structure information;

[0257] determining, according to the number of occurrences of the first structure information, specified structure information and second structure information from the database structure information;

[0258] collecting a second proportion of second text blocks from the text blocks corresponding to the second structure information;

[0259] determining, according to the first text blocks and the second text blocks, specified text blocks.

[0260] In some embodiments, the structured module 210 is specifically configured to:

[0261] combining the specified data type, the specified structure information, and the specified text blocks with an extraction template to obtain an extraction prompt word, the extraction prompt word including data cleaning rules, data extraction rules, and an output format, the data extraction rules including query pattern mining rules, complexity analysis rules, and business extraction rules;

[0262] using the extraction prompt word to guide a large language model to clean the specified text blocks according to the data cleaning rules to obtain cleaned text blocks;

[0263] using the extraction prompt word to guide a large language model to extract specified information from the cleaned text blocks according to the data extraction rules;

[0264] organizing the specified information according to the output format to obtain structured text data.

[0265] In some embodiments, the candidate module 220 is specifically configured to:

[0266] combining the structured text data with a first template to obtain a first prompt word, the first prompt word including problem generation requirements related to the complexity distribution, business rules, and query patterns;

[0267] The first prompt word is used to guide the large language model to process the structured text data according to the requirements of the question to generate a plurality of candidate questions with different complexity levels;

[0268] For each candidate question, a plurality of candidate SQL statements corresponding to the candidate question are generated by using a plurality of different SQL generation strategies;

[0269] The candidate question and the candidate SQL statement are combined to obtain a plurality of candidate question and answer pairs.

[0270] In some embodiments, the first computing module 230 is specifically configured to:

[0271] According to the execution result of the candidate SQL statement, an execution score of the candidate question and answer pair is determined;

[0272] The candidate question and answer pair with an execution score greater than a specified value is taken as a specified question and answer pair, and the specified question and answer pair includes a candidate question and a specified SQL statement;

[0273] The syntax structure of the specified SQL statement is parsed, and a syntax score is calculated based on the syntax structure;

[0274] Using a large language model, the matching degree of the candidate question and the specified SQL statement is analyzed to obtain a matching score;

[0275] For each specified SQL statement, a first quality score is calculated based on the complexity level of the candidate question, the execution score corresponding to the specified SQL statement, the syntax score, and the matching score.

[0276] In some embodiments, the first computing module 230 is specifically configured to:

[0277] For each candidate question, a plurality of specified SQL statements corresponding to the candidate question are obtained;

[0278] For each specified SQL statement, the return content corresponding to the specified SQL statement is obtained;

[0279] According to the consistency of the return content of a plurality of specified SQL statements, a consistency score is obtained;

[0280] Based on the complexity level of the candidate question, a dimension weight corresponding to each dimension is determined;

[0281] Based on the dimension weight, the consistency score, the execution score, the syntax score, and the matching score of the specified SQL statement are fused to obtain a first quality score.

[0282] In some embodiments, the second computing module 240 is specifically configured to:

[0283] For each candidate SQL sentence, obtain clauses, text units and unit types corresponding to the text units in the candidate SQL sentence;

[0284] According to the text unit and the business rule, determine an adjustment factor of the text unit;

[0285] According to the unit type corresponding to each text unit and the adjustment factor, calculate a confidence degree of each text unit;

[0286] For each sentence, aggregate the confidence degrees of all text units in the clause to obtain a clause confidence degree;

[0287] For the candidate SQL sentence, perform geometric mean processing on the clause confidence degrees of all clauses in the candidate SQL sentence to obtain a second quality score.

[0288] In some embodiments, the second calculation module 240 is specifically configured to:

[0289] According to the unit type corresponding to the text unit, obtain a type weight and an influence coefficient of the text unit;

[0290] Obtain a unit probability corresponding to the text unit in the large language model;

[0291] Using the unit probability, the influence coefficient and the type weight, calculate a basic weight of the text unit;

[0292] Calculate an average value of the basic weights of all text units in a specified clause to obtain an average weight, the specified clause being a clause to which the text unit belongs;

[0293] Using the average weight and the adjustment factor, calculate a confidence degree of the text unit.

[0294] In implementation, each of the above modules can be implemented as an independent entity, or can be combined as the same or several entities, and the specific implementation of each of the above modules can refer to the method embodiments described above, which will not be described here.

[0295] As shown above, the SQL dataset generation device in this embodiment can extract structured text data from text data and automatically construct multiple candidate question-answer pairs using the complexity distribution, business rules, and query patterns in the structured text data. For each candidate question-answer pair, its dimensional scores on multiple dimensions are calculated to obtain a first quality score; its confidence scores on multiple levels are calculated to obtain a second quality score; and finally, the SQL dataset is selected from the candidate question-answer pairs based on the first and second quality scores. Automatically extracting information from text data and constructing question-answer pairs, with the calculation of the first and second quality scores including multiple dimensions and levels, ensures multi-faceted evaluation of candidate question-answer pairs, thereby ensuring the rapid generation of a high-quality target SQL dataset.

[0296] This invention also provides an electronic device, which can be a terminal, a server, or other similar devices. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0297] In some embodiments, the SQL dataset generation device can also be integrated into multiple electronic devices, such as multiple servers, with multiple servers implementing the SQL dataset generation method of the present invention.

[0298] In this embodiment, a server will be used as an example for detailed description. For example, ... Figure 5 As shown, it illustrates a structural schematic diagram of the electronic device involved in an embodiment of the present invention, specifically:

[0299] The electronic device may include components such as a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0300] The processor 310 is the control center of the electronic device, and connects various parts of the entire electronic device through various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 320 and calling data stored in the memory 320. In some embodiments, the processor 310 can include one or more processing cores; in some embodiments, the processor 310 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 310.

[0301] The memory 320 can be used to store software programs and modules, and the processor 310 executes various functions and data processing by running the software programs and modules stored in the memory 320. The memory 320 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 320 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 320 can also include a memory controller to provide access for the processor 310 to the memory 320.

[0302] The electronic device also includes a power supply 330 for powering various components, and in some embodiments, the power supply 330 can be logically connected to the processor 310 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 330 can also include one or more direct current or alternating current power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power state indicator, and any other components.

[0303] The electronic device can also include an input module 340, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0304] The electronic device can also include a communication module 350, which in some embodiments can include a wireless module, and the electronic device can perform short-range wireless transmission through the wireless module of the communication module 350, thereby providing the user with wireless broadband Internet access. For example, the communication module 350 can be used to help the user send and receive emails, browse web pages, and access streaming media, etc.

[0305] Although not shown, the electronic device can further include a display unit and the like, which will not be described herein. Specifically, in the present embodiment, the processor 310 in the electronic device will load the executable file corresponding to the process of one or more application programs into the memory 320 according to the instructions, and run the application program stored in the memory 320 by the processor 310, so as to implement the steps in the method of the embodiments of the present application. The specific implementation of each step can refer to the previous embodiments, which will not be described herein.

[0306] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by related hardware controlled by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0307] To this end, the embodiments of the present application provide a computer readable storage medium, which stores a plurality of instructions capable of being loaded by a processor to execute the steps in any of the SQL dataset generation methods provided by the embodiments of the present application. The storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0308] According to an aspect of the present application, a computer program product or computer program is provided, which includes computer programs / instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer programs / instructions from the computer readable storage medium, and the processor executes the computer programs / instructions, so that the electronic device executes the method provided in any of the various optional implementations of the SQL dataset generation aspect provided in the above embodiments.

[0309] Since the instructions stored in the storage medium can execute the steps in any of the SQL dataset generation methods provided by the embodiments of the present application, the beneficial effects that can be achieved by any of the SQL dataset generation methods provided by the embodiments of the present application can be achieved, which will be described in detail in the previous embodiments, and will not be described herein.

[0310] The above describes in detail the SQL dataset generation method and device provided by the embodiments of the present application, and the principles and implementation manners of the present application are described by applying specific examples; the above embodiment descriptions are only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description should not be understood as limiting the present application.

Claims

1. A method of generating a SQL dataset, characterized by, The method comprises: information extraction processing is performed on the text data to obtain structured text data, the structured text data comprising complexity distribution, business rules and query patterns; based on the complexity distribution, business rules and query patterns, a plurality of candidate question and answer pairs are constructed, the candidate question and answer pairs comprising candidate questions and corresponding candidate SQL statements; for each candidate question and answer pair, a first quality score is calculated based on the dimension scores of the candidate SQL statements in a plurality of dimensions and the complexity level of the candidate question, the plurality of dimensions comprising execution dimension, matching dimension and syntax dimension; for each candidate question and answer pair, a second quality score is calculated by calculating the confidence of the candidate SQL statements in a plurality of levels, the plurality of levels comprising text units and clauses; the first quality score and the second quality score are fused to determine a target SQL dataset from the candidate question and answer pairs.

2. The method of claim 1, wherein, The information extraction processing on the text data to obtain structured text data comprises: obtaining text data in a business scenario and data types corresponding to each text data; dividing the text data into text blocks and performing vector storage processing on the text blocks to obtain a text knowledge base, the text knowledge base comprising text block vectors and database structure information; determining specified text blocks and specified structure information from text blocks of a specified data type and database structure information according to a sampling strategy; analyzing the specified structure information and the specified text blocks to extract specified information to obtain structured text data.

3. The method of claim 2, wherein, The determination of specified text blocks and specified structure information from text blocks of a specified data type and database structure information according to a sampling strategy comprises: randomly sampling a first proportion of first text blocks from text blocks of a specified data type; for each first text block, retrieving first structure information corresponding to the first text block and the number of occurrences of the first structure information in the database structure information; determining specified structure information and second structure information from the database structure information according to the number of occurrences of the first structure information; collecting a second proportion of second text blocks from text blocks corresponding to the second structure information; determining specified text blocks according to the first text blocks and the second text blocks.

4. The method of claim 2, wherein, The analysis of the specified structure information and the specified text blocks to extract specified information to obtain structured text data comprises: combining the specified data type, specified structure information and specified text blocks with an extraction template to obtain extraction cues, the extraction cues comprising data cleaning rules, data extraction rules and output formats, the data extraction rules comprising query pattern mining rules, complexity analysis rules and business extraction rules; using the extraction cues to guide a large language model to clean the specified text blocks according to the data cleaning rules to obtain cleaned text blocks; using the extraction cues to guide the large language model to extract specified information from the cleaned text blocks according to the data extraction rules; organizing the specified information according to the output format to obtain structured text data.

5. The method of claim 1, wherein, The method comprises the following steps: combining the structured text data with a first template to obtain a first prompt word, wherein the first prompt word comprises problem generation requirements related to the complexity distribution, business rules and query patterns; using the first prompt word to guide a large language model to process the structured text data according to the problem generation requirements to generate a plurality of candidate questions with different complexity levels; for each candidate question, a plurality of candidate SQL statements corresponding to the candidate question are generated by using a plurality of different SQL generation strategies; combining the candidate question with the candidate SQL statements to obtain a plurality of candidate question and answer pairs.

6. The method of claim 1, wherein, The method comprises the following steps: determining an execution score of the candidate question and answer pair according to an execution result of the candidate SQL statement; combining the candidate question and answer pair with the execution score greater than a specified value as a specified question and answer pair, wherein the specified question and answer pair comprises a candidate question and a specified SQL statement; parsing the syntax structure of the specified SQL statement and calculating a syntax score based on the syntax structure; using a large language model to analyze the matching degree of the candidate question and the specified SQL statement to obtain a matching score; for each specified SQL statement, calculating a first quality score based on the complexity level of the candidate question, the execution score corresponding to the specified SQL statement, the syntax score and the matching score.

7. The method of claim 6, wherein, The method comprises the following steps: for each candidate question, obtaining a plurality of specified SQL statements corresponding to the candidate question; for each specified SQL statement, obtaining the return content corresponding to the specified SQL statement; determining a consistency score according to the consistency of the return content of the plurality of specified SQL statements; determining a dimension weight corresponding to each dimension based on the complexity level of the candidate question; based on the dimension weight, fusing the consistency score, the execution score, the syntax score and the matching score of the specified SQL statement to obtain a first quality score.

8. The method of claim 1, wherein, The method comprises the following steps: for each candidate SQL statement, obtaining clauses, text units and unit types corresponding to the text units in the candidate SQL statement; determining an adjustment factor of the text unit according to the text unit and the business rules; calculating the confidence of each text unit according to the unit type corresponding to each text unit and the adjustment factor; for each sentence, aggregating the confidence of all text units in the clause to obtain a clause confidence; for the candidate SQL statement, performing geometric mean processing on the clause confidence of all clauses in the candidate SQL statement to obtain a second quality score.

9. The method of claim 8, wherein, The calculating the confidence of each text unit according to the unit type corresponding to each text unit and the adjustment factor comprises: obtaining a type weight and an influence coefficient of the text unit according to the unit type corresponding to the text unit; obtaining a unit probability corresponding to the text unit in the large language model; calculating a basic weight of the text unit by using the unit probability, the influence coefficient and the type weight; calculating an average value of the basic weights of all text units in a specified clause to obtain an average weight, the specified clause being a clause to which the text unit belongs; calculating the confidence of the text unit by using the average weight and the adjustment factor.

10. A SQL dataset generation apparatus for implementing the method of any one of claims 1 to 9, characterized in that, The device comprises: a structured module configured to perform information extraction processing on text data to obtain structured text data, the structured text data comprising complexity distribution, business rules and query patterns; a candidate module configured to construct a plurality of candidate question and answer pairs based on the complexity distribution, the business rules and the query patterns, the candidate question and answer pairs comprising a candidate question and a corresponding candidate SQL statement; a first calculation module configured to calculate, for each candidate question and answer pair, a first quality score based on a dimension score of the candidate SQL statement in a plurality of dimensions and a complexity level of the candidate question, the plurality of dimensions comprising an execution dimension, a matching dimension and a syntax dimension; a second calculation module configured to calculate, for each candidate question and answer pair, a confidence of the candidate SQL statement in a plurality of levels to obtain a second quality score, the plurality of levels comprising a text unit and a clause; a generation module configured to fuse the first quality score and the second quality score to determine a target SQL dataset from the candidate question and answer pairs.

Images