Data set construction method and device, equipment and medium

Abstract

The invention belongs to the technical field of natural language processing, and particularly relates to a data set construction method and device, equipment and a medium. Firstly, a graph structure of a database to be processed is generated, data tables in the database serve as nodes, the relation between the data tables serves as edges, and a structure graph of the database is constructed. And then, according to the graph structure, generating at least one target natural language question for operating the database. Then, inputting the generated natural language question sentences into the large language model to obtain structured query sentences corresponding to the question sentences; according to the method, the situation that information of a database structure needs to be recoded every time natural language query is processed is effectively avoided, so that query processing efficiency is improved, and consumption of computing resources is reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of natural language processing, and particularly relates to a data set construction method and device, equipment and a medium. BACKGROUND

[0002] A relational database is difficult for non-professionals to access to obtain required information through autonomous operation due to its complex table structure and rigorous query language.

[0003] The existing solution is to realize database access through natural language to structured query language, specifically, first constructing a data set, then using the data set to train a related model, and then integrating the trained model into a query system to enable the system to convert natural language into structured query statements, so that non-professionals can directly access the relational database through natural language.

[0004] However, in the existing data set construction process, the method of first generating a structured query statement and then generating a natural language question from the structured query statement is prone to generate mechanical or out-of-context questions, which is difficult to meet the use requirements of non-professionals. SUMMARY

[0005] The application provides a data set construction method, device, equipment and medium, which is used to solve the problem that the existing construction method is prone to generate mechanical or out-of-context questions, which is difficult to meet the use requirements of non-professionals.

[0006] In a first aspect, the application provides a data set construction method, comprising:

[0007] generating a graph structure of a to-be-processed database, wherein nodes in the graph structure are data tables in the to-be-processed database, and edges between the nodes are relationships of the data tables in the to-be-processed database;

[0008] generating at least one target natural language question according to the graph structure, wherein the target natural language question is used to operate the to-be-processed database;

[0009] inputting the at least one target natural language question into a large language model to obtain a structured query statement corresponding to each of the at least one target natural language question.

[0010] Optionally, the generating of the graph structure of the to-be-processed database comprises:

[0011] determining a data table in the to-be-processed database, and determining the data table as a node of the graph structure;

[0012] Determine the foreign key between the data tables in the to-be-processed database, and determine the foreign key as the relationship of the data tables in the to-be-processed database.

[0013] Optionally, the generating at least one target natural language question sentence according to the graph structure comprises:

[0014] For any one node, the node and a preset number of nodes connected to the node are determined as a target subgraph corresponding to the node.

[0015] According to the target subgraph corresponding to each node, the at least one target natural language question sentence is generated.

[0016] Optionally, the generating at least one target natural language question sentence according to the target subgraph corresponding to each node comprises:

[0017] According to the target subgraph corresponding to each node, at least one candidate natural language question sentence corresponding to each node is generated respectively.

[0018] According to the at least one candidate natural language question sentence corresponding to each node, the at least one target natural language question sentence is determined.

[0019] Optionally, the generating at least one target natural language question sentence according to the target subgraph corresponding to each node comprises:

[0020] The target subgraph is input into the large language model to obtain an initial keyword set of the target subgraph.

[0021] The initial keyword set is filtered to obtain a target keyword set of the target subgraph.

[0022] According to the target subgraph and the target keyword set of the target subgraph, the at least one candidate natural language question sentence is determined.

[0023] Optionally, the determining the at least one candidate natural language question sentence according to the target subgraph and the target keyword set of the target subgraph comprises:

[0024] A preset structure structured query language sentence type, the target subgraph and the target keyword set of the target subgraph are input into the large language model to obtain the at least one candidate natural language question sentence.

[0025] Optionally, the determining the at least one target natural language question sentence according to the at least one candidate natural language question sentence corresponding to each node comprises:

[0026] The target keyword set is subjected to text embedding processing to obtain a class center vector.

[0027] vectorize the candidate natural language question sentence, to obtain a problem vector corresponding to the candidate natural language question sentence;

[0028] determine a similarity between the class center vector and the problem vector;

[0029] determine the candidate natural language question sentence with the similarity greater than the preset similarity as the target natural language question sentence.

[0030] Optionally, the inputting the at least one target natural language question sentence into the large language model to obtain the structured query statement corresponding to each of the at least one target natural language question sentence comprises:

[0031] for any one target natural language question sentence, inputting the target natural language question sentence into the large language model, obtaining a structure of the target natural language question sentence through the large language model, and obtaining the structured query statement according to the structure.

[0032] In a second aspect, the present application provides a data set construction method and device, comprising:

[0033] a generation module configured to generate a graph structure of a to-be-processed database, wherein nodes in the graph structure are data tables in the to-be-processed database, and edges between the nodes are relationships of the data tables in the to-be-processed database;

[0034] the generation module is further configured to generate at least one target natural language question sentence according to the graph structure, wherein the target natural language question sentence is used to operate the to-be-processed database;

[0035] an input module configured to input the at least one target natural language question sentence into a large language model to obtain a structured query statement corresponding to each of the at least one target natural language question sentence.

[0036] Optionally, the device further comprises a determination module.

[0037] the determination module is configured to determine data tables in the to-be-processed database, and determine the data tables as nodes of the graph structure;

[0038] the determination module is further configured to determine foreign keys associated between the data tables in the to-be-processed database, and determine the foreign keys as relationships of the data tables in the to-be-processed database.

[0039] Optionally, the determination module is further configured to, for any one node, determine the node and a preset number of nodes connected to the node as a target subgraph corresponding to the node.

[0040] The generating module is specifically configured to generate the at least one target natural language question sentence according to the target subgraph corresponding to each node.

[0041] Optionally, the generating module is further configured to generate at least one candidate natural language question sentence corresponding to each node according to the target subgraph corresponding to each node.

[0042] The determining module is specifically configured to determine the at least one target natural language question sentence according to the at least one candidate natural language question sentence corresponding to each node.

[0043] Optionally, the input module is further configured to input the target subgraph into the large language model to obtain an initial keyword set of the target subgraph.

[0044] The device further includes a processing module.

[0045] The processing module is configured to perform filtering processing on the initial keyword set to obtain a target keyword set of the target subgraph.

[0046] The determining module is specifically configured to determine the at least one candidate natural language question sentence according to the target subgraph and the target keyword set of the target subgraph.

[0047] Optionally, the input module is further configured to input a preset structure structured query sentence type, the target subgraph, and the target keyword set of the target subgraph into the large language model to obtain the at least one candidate natural language question sentence.

[0048] Optionally, the processing module is further configured to perform text embedding processing on the target keyword set to obtain a class center vector.

[0049] The processing module is further configured to, for any one candidate natural language question sentence, perform vectorization processing on the candidate natural language question sentence to obtain a problem vector corresponding to the candidate natural language question sentence.

[0050] The determining module is further configured to determine a similarity between the class center vector and the problem vector.

[0051] The determining module is specifically configured to determine the candidate natural language question sentence with a similarity greater than a preset similarity as the target natural language question sentence.

[0052] Optionally, the input module is specifically configured to, for any one target natural language question sentence, input the target natural language question sentence into the large language model to obtain a structure of the target natural language question sentence through the large language model, and obtain the structured query sentence according to the structure.

[0053] In a third aspect, the present application provides a data set construction method and device, comprising:

[0054] a memory;

[0055] a processor;

[0056] The memory stores computer execution instructions.

[0057] The processor executes the computer execution instructions stored in the memory to implement the data set construction method as described in the first aspect and various possible implementation manners of the first aspect.

[0058] In a fourth aspect, the present application provides a computer storage medium, which stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the data set construction method as described in the first aspect and various possible implementation manners of the first aspect.

[0059] In a fifth aspect, the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the data set construction method as described above.

[0060] The data set construction method provided by the present application first generates a graph structure of a to-be-processed database, wherein the nodes of the graph structure are data tables in the to-be-processed database, and the edges between the nodes represent the relationships of the data tables; then at least one target natural language question for operating the to-be-processed database is generated according to the graph structure; finally, the target natural language questions are input into a large language model to obtain respective corresponding structured query statements. The method generates natural language questions from the actual structure of the database, solves the problem of mechanical and business context-independent questions generated by the existing construction method, and makes the generated natural language questions and structured query statements more suitable for real business scenarios and query intentions. BRIEF DESCRIPTION OF DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0062] Figure 1 The flow of the data set construction method provided by the present application Figure 1 ;

[0063] Figure 2 The flow of the data set construction method provided by the present application Figure 2 ;

[0064] Figure 3 The flow of the data set construction method provided by the present application Figure 3 ;

[0065] Figure 4 is a structural schematic diagram of the data set construction method device provided in the application;

[0066] Figure 5 is a structural schematic diagram of the data set construction method device provided in the application.

[0067] Through the above-mentioned drawings, the specific embodiments of the present application have been shown, and will be described in more detail hereinafter. These drawings and the written description are not intended to limit the scope of the present application concept in any way, but to illustrate the present application concept to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0068] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative work belong to the scope of protection of the present application.

[0069] The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0070] In the embodiments of the present application, the words "exemplary" or "for example" are used to mean example, illustration, or description. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design solutions. Rather, the use of "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0071] A relational database is difficult for non-professionals to access the database to obtain the required information through autonomous operation due to its complex table structure and rigorous query language.

[0072] The existing solution is to realize database access through natural language to structured query language, specifically, first constructing a data set containing natural language question to structured query language, then using the data set to train the relevant model, and then integrating the trained model into the query system, so that the system has the ability to convert natural language into structured query language, so that non-professionals can directly access the relational database through natural language.

[0073] However, in the existing data set construction process, the way of generating a structured query statement first and then generating a natural language question from the structured query statement is easy to generate a question that is mechanical or deviates from the actual business context, and it is difficult to meet the use needs of non-professionals.

[0074] To solve the above problems, the present application provides a data set construction method. First, a graph structure of a to-be-processed database is generated, taking the data tables in the database as nodes and the relationships between the data tables as edges; then, a target natural language question is generated according to the graph structure; finally, the target natural language question is input into a large language model to generate a corresponding structured query statement. This method generates a natural language question that fits the business context by starting from the database structure, and matches the structured query statement, so that the problem of mechanicalness or deviation from the actual business context caused by the existing construction method can be solved, and the demand of non-professionals for accessing a relational database through natural language can be better met.

[0075] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0076] Figure 1 The flow of the data set construction method provided in this embodiment Figure 1 As shown in Figure 1 , the data set construction method provided in this embodiment includes:

[0077] S101: Generate a graph structure of a to-be-processed database, the nodes in the graph structure are data tables in the to-be-processed database, and the edges between the nodes are the relationships of the data tables in the to-be-processed database.

[0078] The to-be-processed database may be, for example, a gas emergency management database, a student management database, or a medical case database. The present application does not make special limitations on this.

[0079] Different types of to-be-processed databases correspond to different data tables. For example, when the to-be-processed database is a fire emergency management database, the data tables may include, for example, emergency handling unit data tables, accident record data tables, geographic location information data tables, gas accident data tables, gas pipeline information tables, and accident type tables; and when the to-be-processed database is a student management database, the data tables may include, for example, class tables, age tables, teacher tables, and student information tables.

[0080] The purpose of this step is to convert the scattered data tables and the relationships between the data tables in the to-be-processed database into a structured graph structure.

[0081] It can be understood that in the original to-be-processed database, the number of data tables can be large, and the relationship between the data tables is complex and not easy to observe directly. Therefore, by generating a graph structure taking the data table as a node and the inter-table relationship as an edge, the overall architecture of the to-be-processed database can be presented in the form of a graph structure.

[0082] S102: generating at least one target natural language question according to the graph structure, the target natural language question being used to operate the to-be-processed database.

[0083] The purpose of this step is to generate a natural language question that can operate the to-be-processed database according to the graph structure.

[0084] It can be understood that the original to-be-processed database operation depends on a structured query statement, which is difficult for non-professionals to operate. Therefore, by referring to the graph structure, at least one target natural language question can be generated, which can directly operate the to-be-processed database through a specific instruction in the natural language, so that non-professionals can easily operate the database without mastering complex database query languages.

[0085] S103: inputting the at least one target natural language question into a large language model to obtain at least one structured query statement corresponding to each of the at least one target natural language question.

[0086] The purpose of this step is to convert the target natural language question into a structured query statement that can operate the database.

[0087] It can be understood that the natural language question is a common expression method for non-professionals to operate the database, but the database executes operations depending on a structured query statement. The large language model can accurately convert the natural language question into a structured query statement that meets the requirements of the database.

[0088] Therefore, by inputting the target natural language question into the large language model, the target natural language question can be converted into a structured query statement that can operate the database, so that non-professionals can interact with the database in the natural language, thereby enabling the database to respond to the operation intent.

[0089] Optionally, the present application provides a method for optimizing the large language model after determining the structured query statement corresponding to each of the at least one target natural language question, specifically comprising: fine-tuning the large language model according to the structured query statement corresponding to each of the at least one target natural language question.

[0090] The purpose of this step is to fine-tune the large language model into a model that is highly adapted to the to-be-processed database.

[0091] It can be understood that by fine-tuning the large language model, the large language model can better support the operation of the data processing platform in actual use. When non-professionals operate on the data processing platform corresponding to the large language model, they only need to input a natural language question, and the large language model built in the platform which has been fine-tuned will perform semantic analysis on the natural language question, generate a structured query statement matching the database to be processed, and then the platform executes the structured query statement and outputs the final operation result on the interface.

[0092] The data set construction method provided by the embodiment first generates a graph structure taking the data tables of the database to be processed as nodes and the relationships between the tables as edges, then generates at least one target natural language question for operating the database based on the graph structure, and finally inputs the target natural language question into the large language model to obtain the corresponding structured query statement. This method solves the problem of mechanical and business context-independent questions generated by the existing construction method, and can generate more business-related and accurate query questions, thereby better meeting actual needs.

[0093] Figure 2 Flow of the data set construction method provided by the embodiment Figure 2 As shown in Figure 1 , the embodiment is based on the Figure 3 embodiment and details the data set construction method. The data set construction method provided by the embodiment includes:

[0094] S201: Determine the data tables in the database to be processed, and determine the data tables as the nodes of the graph structure.

[0095] It can be understood that the database to be processed contains multiple data tables, and each data table stores data of a specific type. Therefore, by determining the data tables as the nodes of the graph structure, each data table is treated as a node in the graph, and the data tables are presented in the form of nodes.

[0096] S202: Determine the foreign keys associated between the data tables in the database to be processed, and determine the foreign keys as the relationships between the data tables in the database to be processed.

[0097] The purpose of this step is to determine the edges between the nodes in the graph through the foreign keys associated between the data tables.

[0098] It can be understood that in the original database to be processed, the relationships between the data tables are represented by determining the foreign keys. In the graph structure, the main role of the edge is to reflect the mutual relationship between the nodes.

[0099] Therefore, converting the foreign keys associated between the data tables in the to-be-processed database into edges of the corresponding nodes can make the edges in the graph structure have clear and definite association basis, while reflecting the actual connection relationship between the data tables, and effectively avoiding the case that the definition of the edges is disconnected from the actual structure of the database.

[0100] For example, assuming that the to-be-processed database is a gas emergency management database, the database includes data tables of a gas accident table, geographic location information, gas pipeline information, emergency treatment unit information, sensor data, accident records, and accident types. Among them, the gas accident table is connected with the geographic location information, the accident records, the gas pipeline information, and the accident types through foreign keys respectively; the emergency treatment unit is connected with the geographic location information and the accident records through foreign keys respectively; and the gas pipeline information is connected with the sensor data, the geographic location information, and the gas accident table through foreign keys respectively.

[0101] Then, based on the above information, the gas accident table, the geographic location information, the gas pipeline information, the emergency treatment unit information, the sensor data, the accident records, and the accident types can be taken as nodes of the graph structure respectively, and the connections between the above data tables through the foreign keys can be taken as edges connecting the nodes in the graph structure.

[0102] S203: For any one node, the node and a preset number of nodes connected to the node are determined as a target subgraph corresponding to the node.

[0103] The preset number can be 3, 4, or 6, for example. The present application does not make special limitations on this.

[0104] The target subgraph refers to a local substructure extracted from the graph structure, which takes a single node as the center and includes the node and a preset number of associated nodes directly connected to the node through edges.

[0105] The purpose of this step is to disassemble the complex graph structure into several local substructures.

[0106] It can be understood that the original graph structure includes a large number of nodes and complex node associations. If the natural language question sentence is directly generated based on the whole graph structure, it is easy to cause the natural language question sentence to be logically chaotic due to information overload, or to deviate from the actual multi-table query demand due to insufficient focus.

[0107] Therefore, by determining, for each node, the node itself and a preset number of associated nodes directly connected to the node, the nodes and the association relationship between the nodes can be integrated into the target subgraph corresponding to each node.

[0108] For example, assuming that the preset number is 3, in the graph structure corresponding to the gas emergency management database, the target subgraph formed with the gas accident table as the core node can be determined as the gas accident table, the geographic location information, and the accident type.

[0109] S204: According to the target subgraph corresponding to each node, at least one target natural language question is generated, and the target natural language question is used to operate the to-be-processed database.

[0110] The purpose of this step is to generate a natural language question that can operate the to-be-processed database according to the target subgraph corresponding to each node.

[0111] For example, assuming that the target subgraph is the "gas accident table" as the core node, and the nodes associated with the core node are "geographic location information table" and "accident type table". Then based on the above information, it can be determined that the generated target natural language question includes: querying the accident type corresponding to a specific gas accident, and querying the specific geographic location where the accident occurred.

[0112] S205: For any one target natural language question, input the target natural language question into a large language model, obtain the structure of the target natural language question through the large language model, and obtain a structured query statement according to the structure.

[0113] The structure refers to the framework structure of the structured query statement. The structure includes but is not limited to: query object, constraint condition, associated subject, and operation type.

[0114] The purpose of this step is to convert the target natural language question used by non-professionals into a structured query statement that can be directly executed by the database.

[0115] It can be understood that non-professionals usually use natural language to make database operation requests, but the database cannot directly understand natural language and can only recognize and execute operations through structured query statements, making it difficult for non-professionals to directly operate the database due to lack of understanding of structured query statements.

[0116] Therefore, inputting the target natural language question into the large language model can analyze the natural language question through the semantic parsing ability of the large language model, disassemble the core elements such as the query object, the constraint condition, the associated subject, and the operation type, and form the framework structure supporting the generation of the structured query statement; then based on the framework structure, combined with the data table structure of the to-be-processed database, a structured query statement conforming to the syntax specification is generated, finally allowing non-professionals to interact with the database without mastering professional technical language.

[0117] For example, assume the target natural language question is: List all the types of gas incidents, the time of occurrence, and the detailed address. Then based on the above information, the structure can be determined first as:

[0118] SELECT columns

[0119] FROM gas_incidents

[0120] JOIN incident_types ON gas_incidents.type_id = incident_types.id

[0121] JOIN locations ON gas_incidents.location_id = locations.id

[0122] Subsequently, based on the structure, the structured query statement is determined.

[0123] SELECT

[0124] incident_types.name AS incident_type,

[0125] gas_incidents.reported_time,

[0126] locations.address

[0127] FROM gas_incidents

[0128] JOIN incident_types ON gas_incidents.type_id = incident_types.id

[0129] JOIN locations ON gas_incidents.location_id = locations.id.

[0130] The data set construction method provided by the embodiment first determines a data table in a to-be-processed database as a graph structure node, and takes a foreign key between data tables as a table relationship; then, for each node, a target subgraph composed of the node and a preset number of connected nodes is selected, and at least one target natural language question sentence for operating the database is generated according to each target subgraph; finally, the question sentence is input into a large language model, the structure of the question sentence is parsed, and a corresponding structured query statement is generated. The method generates a subgraph by combining the actual structure of the database and generates a natural language question sentence based on the subgraph, avoiding the problem that the existing method generates a question sentence mechanically and is separated from the business context, so that the generated question sentence and the structured query statement are more suitable for the actual business scenario.

[0131] Figure 3 The flow of the data set construction method provided by the embodiment Figure 4 As shown in Figure 4 , the embodiment is based on the above-mentioned embodiment, and the specific implementation process of generating at least one target natural language question sentence according to the target subgraph corresponding to each node is described in detail. The method specifically includes:

[0132] S301: At least one candidate natural language question sentence corresponding to each node is generated according to the target subgraph corresponding to each node.

[0133] Among them, the candidate natural language question sentence refers to a preliminary natural language question sentence for operating the to-be-processed database. The candidate natural language question sentence needs to be screened in the subsequent processing process.

[0134] The target natural language question sentence is a question sentence that meets the actual demand and is screened from the candidate natural language question sentence.

[0135] The purpose of this step is to generate a preliminary natural language question sentence that can operate the to-be-processed database.

[0136] It can be understood that by generating the candidate natural language question sentence corresponding to each node, it means that various possible query expression methods when querying the to-be-processed database can be prepared in advance, so that when facing the target natural language question sentence actually proposed, the matching and response can be faster and more accurate, thereby better meeting the query needs of non-professionals.

[0137] Optionally, the application provides a possible implementation manner for generating at least one candidate natural language question sentence corresponding to each node according to the target subgraph corresponding to each node, which includes:

[0138] Firstly, the target subgraph is input into a large language model to obtain an initial keyword set of the target subgraph.

[0139] The purpose of this step is to extract information from the target subgraph and obtain a plurality of representative keywords in the form of an initial keyword set.

[0140] It can be understood that the target subgraph is a local substructure determined from the graph structure, which contains specific nodes and the association relationship between the nodes and other information. Therefore, by inputting the target subgraph into the large language model, the nodes in the target subgraph and the relationship between the nodes and other information can be screened by the analysis capability of the large language model, so as to obtain keywords capable of representing the content of the target subgraph.

[0141] For example, assuming that the target subgraph is a gas accident table, geographic location information, and accident type, based on the above information, it can be determined that the initial keyword set includes: gas accident, occurrence location, accident type, gas accident type, gas accident occurrence location, accident location, and table association.

[0142] Secondly, the initial keyword set is filtered to obtain a target keyword set of the target subgraph.

[0143] The purpose of this step is to remove irrelevant or redundant keywords from the initial keyword set, thereby obtaining a more core target keyword set.

[0144] It can be understood that although the initial keyword set covers part of the important information of the target subgraph, it may contain some keywords that are weakly associated with the core query scene, repeated expressions, or not clear in semantics. Therefore, by filtering the initial keyword set, irrelevant or redundant keywords can be removed, and finally the target keyword set that can reflect the key features of the target subgraph is retained.

[0145] For example, assuming that the initial keyword set includes: gas accident, occurrence location, accident type, gas accident type, gas accident occurrence location, accident location, and table association, based on the above information, it can be determined that the target keyword set includes: gas accident type and gas accident occurrence location.

[0146] Thirdly, at least one candidate natural language question is determined according to the target subgraph and the target keyword set of the target subgraph.

[0147] It can be understood that the target subgraph contains specific nodes and the association relationship between the nodes and other information, which constitutes the basic framework of the subgraph. The target keyword set is obtained by screening the initial keyword set, removing irrelevant or redundant keywords, and reflecting the key features of the target subgraph.

[0148] Therefore, by integrating the target subgraph and the target keyword set of the target subgraph, a preliminary natural language question sentence for operating the to-be-processed database can be generated.

[0149] For example, assuming that the target subgraph is a gas accident table, geographic location information, and accident type, and the target keyword set of the target subgraph includes: gas accident type, and gas accident occurrence location. Then, based on the above information, the candidate natural language question sentences that can be determined include: (1) query the location where the gas accident occurs; (2) query the type of the gas accident; (3) query the accident type corresponding to a specific gas accident and the specific geographic location where the accident occurs.

[0150] Optionally, the present application provides a possible implementation manner for determining at least one candidate natural language question sentence according to the target subgraph and the target keyword set of the target subgraph, which includes: inputting a preset structured query statement type, the target subgraph, and the target keyword set of the target subgraph into a large language model to obtain at least one candidate natural language question sentence.

[0151] The preset structured query statement type refers to a structured query statement category that is defined in advance according to common operation scenarios and business requirements of the database and has a fixed syntax logic and a function pointing.

[0152] It can be understood that by introducing the preset structured query statement type, the candidate natural language question sentence generated by the large language model can not only match the structure of the target subgraph and the semantics of the target keyword, but also accurately operate the to-be-processed database.

[0153] S302: determining at least one target natural language question sentence according to the at least one candidate natural language question sentence corresponding to each node.

[0154] The purpose of this step is to filter out the target natural language question sentence that best meets the actual demand and most accurately expresses the query intention from the candidate natural language question sentences corresponding to each node.

[0155] Optionally, the present application provides a possible implementation manner for determining at least one target natural language question sentence according to the at least one candidate natural language question sentence corresponding to each node, which includes:

[0156] Firstly, text embedding processing is performed on the target keyword set to obtain a class center vector.

[0157] The class center vector refers to that, for the target keyword set, each keyword is first converted into a vector with the same dimension through text embedding technology, and then specific mathematical operations are performed on all converted keyword embedding vectors to finally obtain a vector. The class center vector can represent the target semantics contained in the target keyword set.

[0158] The purpose of this step is to determine a vector that can represent the target semantics of the target keyword set.

[0159] It can be understood that the keywords in the target keyword set have diversity, and different keywords may express the same. It is difficult to accurately capture the essential semantics by directly processing the keyword set. Therefore, by text embedding processing, all keywords in the target keyword set can be mapped to a continuous numerical space, so that keywords with similar semantics are closer in the vector space, thereby obtaining a vector that can represent the target semantics of the target keyword set.

[0160] Secondly, for any one candidate natural language question, the candidate natural language question is vectorized to obtain a problem vector corresponding to the candidate natural language question.

[0161] The purpose of this step is to convert the candidate natural language question in natural language form into a vector form that can be compared in features.

[0162] It can be understood that the natural language question cannot be directly used for feature comparison at the semantic level. Therefore, by vectorizing the candidate natural language question, the semantic information contained in the candidate natural language question can be mapped to a numerical space and converted into a vector form that can be compared in features, thereby facilitating feature comparison with the class center vector of the target keyword set.

[0163] Thirdly, the similarity between the class center vector and the problem vector is determined.

[0164] The similarity is used to represent the semantic association degree between the candidate natural language question and the target keyword set. The higher the similarity between the class center vector and the problem vector, the closer the semantic association between the candidate natural language question and the target keyword set; on the contrary, the lower the similarity between the class center vector and the problem vector, the weaker the semantic association between the candidate natural language question and the target keyword set. The similarity can be expressed in percentage, such as 50%, or in decimal form, such as 0.9. The present application does not make special limitations.

[0165] The purpose of this step is to determine the semantic association degree between the candidate natural language question and the target keyword set.

[0166] It can be understood that the class center vector can represent the target semantics implied by the target keyword set, and the problem vector represents the semantic features of the candidate natural language question.

[0167] Therefore, by calculating the similarity between the class center vector and the problem vector, the association degree between the candidate natural language question and the target keyword set in the semantic can be reflected in numerical form.

[0168] In the fourth step, the candidate natural language question with the similarity greater than the preset similarity is determined as the target natural language question.

[0169] The preset similarity can be used as a criterion for measuring the closeness of the semantic association between the candidate natural language question and the target keyword set. For example, the preset similarity can be expressed in percentage, such as 90%, or in decimal form, such as 0.9. The present application does not make special limitations.

[0170] It can be understood that the preset similarity is a criterion for measuring the closeness of the semantic association between the candidate natural language question and the target keyword set. Therefore, by comparing the similarity of the candidate natural language question and the class center vector of the target keyword set with the preset similarity, and selecting the candidate natural language question with the similarity greater than the preset similarity, and determining it as the target natural language question, it can be ensured that the natural language question used for database operation or further interaction with the large model has high accuracy.

[0171] Optionally, the present application provides a way to determine the target natural language question based on the candidate natural language question with the similarity greater than the preset similarity, and provides another specific way to determine the target natural language question according to the similarity, which includes:

[0172] In the first step, the similarity is normalized to obtain a processed similarity.

[0173] It can be understood that since the candidate natural language question itself has diversity, the calculated similarity values may be in different value ranges, for example, some similarity calculation results may be distributed in the interval [-1, 1], and some may be distributed in the interval [0, 10]. Therefore, by normalizing the similarity, the similarity can be unified to the same interval range.

[0174] In the second step, one minus the processed similarity is obtained to obtain the semantic distance of the candidate natural language question.

[0175] It can be understood that the similarity mainly reflects the degree of association between the candidate natural language question and the target keyword set in the semantic level. The semantic distance can measure the distance between the problem vector and the class center vector. The smaller the semantic distance, the closer the distance between the candidate natural language question and the target keyword set; on the contrary, the larger the semantic distance, the farther the distance between the candidate natural language question and the target keyword set.

[0176] Therefore, by calculating the semantic distance, the degree of deviation of the candidate natural language question from the target semantic can be determined, thereby providing a basis for subsequent screening of the candidate natural language question that meets the target semantic.

[0177] In the third step, a preset weighted matching function is used to calculate and process the sentence distance of the candidate natural language question, so as to obtain the semantic weight of the candidate natural language question.

[0178] The preset weighted matching function is as follows:

[0179]

[0180] The semantic weight of the candidate natural language question is represented by w q. The semantic weight of the candidate natural language question is represented by w q. The similarity is used to control the influence. The greater the similarity difference is, the more significant the weight change is, ∈[2,10]; The sentence distance of the candidate natural language question is represented by d q. The length penalty intensity is represented by μ, and μ∈[0.6,0.95]. The length scale is represented by l, which can be the average length of all candidate natural language questions; and |q| represents the length of the candidate natural language question.

[0181] In the fourth step, the candidate natural language question with a semantic weight greater than a preset semantic weight is determined as the target natural language question.

[0182] The preset semantic weight can be 0.9, for example.

[0183] The purpose of this step is to screen out the natural language question that is most semantically related to the target keyword set.

[0184] It can be understood that by determining the semantic weight of the candidate natural language question and comparing the semantic weight with the preset semantic weight, the candidate natural language question can be sorted according to the size of the weight, and then it is ensured that the candidate natural language question with a greater weight is given priority, so that the natural language question that is most semantically related to the target keyword set is screened out.

[0185] The data set construction method provided in this embodiment first generates at least one candidate natural language question for each node according to the target subgraph corresponding to each node; and then determines at least one target natural language question based on the candidate natural language questions corresponding to each node. This method can generate target natural language questions that meet the needs of the nodes in a targeted manner by first generating candidate natural language questions according to the target subgraph and then determining the target natural language questions from the candidate natural language questions, thereby improving the reliability of natural language question generation.

[0186] Figure 5 The structure diagram of the data set construction method and device provided in this application is shown in FIG. 1. Figure 5As shown, the present application provides a data set construction method and device, the data set construction method and device 400 include:

[0187] The generation module 401 is configured to generate a graph structure of the to-be-processed database, wherein nodes in the graph structure are data tables in the to-be-processed database, and edges between the nodes are relationships of the data tables in the to-be-processed database.

[0188] The generation module 401 is further configured to generate at least one target natural language question according to the graph structure, and the target natural language question is used to operate the to-be-processed database.

[0189] The input module 402 is configured to input the at least one target natural language question into a large language model to obtain a structured query statement corresponding to each of the at least one target natural language question.

[0190] Optionally, the device further includes a determination module 403.

[0191] The determination module 403 is configured to determine data tables in the to-be-processed database, and determine the data tables as nodes of the graph structure.

[0192] The determination module 403 is further configured to determine foreign keys associated between the data tables in the to-be-processed database, and determine the foreign keys as relationships of the data tables in the to-be-processed database.

[0193] Optionally, the determination module 403 is further configured to, for any one node, determine the node and a preset number of nodes connected to the node as a target subgraph corresponding to the node.

[0194] The generation module 401 is specifically configured to generate the at least one target natural language question according to the target subgraph corresponding to each node.

[0195] Optionally, the generation module 401 is further configured to generate at least one candidate natural language question corresponding to each node according to the target subgraph corresponding to each node.

[0196] The determination module 403 is specifically configured to determine the at least one target natural language question according to the at least one candidate natural language question corresponding to each node.

[0197] Optionally, the input module 402 is further configured to input the target subgraph into the large language model to obtain an initial keyword set of the target subgraph.

[0198] The device further includes a processing module 404.

[0199] The processing module 404 is configured to perform filtering processing on the initial keyword set to obtain a target keyword set of the target subgraph.

[0200] The determining module 403 is specifically configured to determine at least one candidate natural language question sentence according to the target subgraph and the target keyword set of the target subgraph.

[0201] Optionally, the input module 402 is further configured to input the preset structured query sentence type, the target subgraph and the target keyword set of the target subgraph into the large language model to obtain the at least one candidate natural language question sentence.

[0202] Optionally, the processing module 404 is further configured to perform text embedding processing on the target keyword set to obtain a class center vector.

[0203] The processing module 404 is further configured to perform vectorization processing on the candidate natural language question sentence to obtain a problem vector corresponding to the candidate natural language question sentence.

[0204] The determining module 403 is further configured to determine the similarity between the class center vector and the problem vector.

[0205] The determining module 403 is specifically configured to determine the candidate natural language question sentence with a similarity greater than a preset similarity as the target natural language question sentence.

[0206] Optionally, the input module 402 is specifically configured to input the target natural language question sentence into the large language model to obtain a structure of the target natural language question sentence through the large language model, and obtain a structured query sentence according to the structure.

[0207] ​ The structure of the data set construction method device provided by the present application is shown in the figure. As shown in the figure, the present application provides a data set construction method device, which comprises a receiver 501, a transmitter 502, a processor 503 and a memory 504. ​

[0208] The receiver 501 is used to receive instructions and data.

[0209] The transmitter 502 is used to transmit instructions and data.

[0210] The memory 504 is used to store computer execution instructions.

[0211] The processor 503 is used to execute the computer execution instructions stored in the memory 504 to realize each step of the data set construction method performed in the above-mentioned embodiments. For details, please refer to the related description in the foregoing data set construction method embodiments.

[0212] Optionally, the above-mentioned memory 504 can be independent or integrated with the processor 503. ​

[0213] When the memory 504 is independently provided, the electronic device further includes a bus for connecting the memory 504 and the processor 503.

[0214] The application also provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. When the processor executes the computer execution instructions, the data set construction method performed by the data set construction method device is implemented.

[0215] So far, the technical solution of the application has been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the application is obviously not limited to these specific embodiments, and the above embodiments are only used to illustrate the technical solution of the application, but not to limit it; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solution recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the application.

Claims

1. A method of constructing a data set, characterized by, The method comprises: generating a graph structure of a to-be-processed database, nodes in the graph structure being data tables in the to-be-processed database, and edges between the nodes being relationships of the data tables in the to-be-processed database; generating at least one target natural language question according to the graph structure, the target natural language question being used for operating the to-be-processed database; inputting the at least one target natural language question into a large language model to obtain a structured query statement corresponding to each of the at least one target natural language question.

2. The method of claim 1, wherein, The method comprises: determining data tables in the to-be-processed database, and determining the data tables as nodes of the graph structure; determining foreign keys associated between the data tables in the to-be-processed database, and determining the foreign keys as relationships of the data tables in the to-be-processed database.

3. The method of claim 1, wherein, The method comprises: for any one node, determining the node and a preset number of nodes connected to the node as a target subgraph corresponding to the node; generating the at least one target natural language question according to the target subgraph corresponding to each node.

4. The method of claim 3, wherein, The method comprises: generating at least one candidate natural language question corresponding to each node according to the target subgraph corresponding to the node; determining the at least one target natural language question according to the at least one candidate natural language question corresponding to each node.

5. The method of claim 4, wherein, The method comprises: inputting the target subgraph into the large language model to obtain an initial keyword set of the target subgraph; performing filtering processing on the initial keyword set to obtain a target keyword set of the target subgraph; determining the at least one candidate natural language question according to the target subgraph and the target keyword set of the target subgraph.

6. The method of claim 5, wherein, The method comprises: inputting a preset structured query statement type, the target subgraph and the target keyword set of the target subgraph into the large language model to obtain the at least one candidate natural language question.

7. The method of claim 5, wherein, The method comprises: performing text embedding processing on the target keyword set to obtain a class center vector; for any one candidate natural language question, performing vectorization processing on the candidate natural language question to obtain a problem vector corresponding to the candidate natural language question; determining a similarity between the class center vector and the problem vector; determining the candidate natural language question with the similarity greater than a preset similarity as the target natural language question.

8. The method of claim 1, wherein, The method comprises: For any one target natural language question, the target natural language question is input into the large language model, the structure of the target natural language question is obtained through the large language model, and the structured query statement is obtained according to the structure.

9. A method of constructing a data set, characterized by, Comprise: The generation module is used for generating a graph structure of a to-be-processed database, wherein nodes in the graph structure are data tables in the to-be-processed database, and edges between the nodes are relationships of the data tables in the to-be-processed database; The generation module is also used for generating at least one target natural language question according to the graph structure, wherein the target natural language question is used for operating the to-be-processed database; The input module is used for inputting the at least one target natural language question into a large language model to obtain a structured query statement corresponding to each of the at least one target natural language question.

10. A method of constructing a data set, comprising: Comprise: A memory; A processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement the data set construction method in any one of claims 1-8.

Images