Query method based on knowledge graph and Text2SQL model

Abstract

The invention relates to the technical field of database query optimization, in particular to a query method based on a knowledge graph and a Text2SQL (Structured Query Language) model, which mainly comprises the following steps of: according to the context of user query data and intention recognition, analyzing a relationship among keywords by utilizing the knowledge graph, and judging whether the keywords can meet SQL generation conditions or not; if the SQL generation condition is not met, interacting with the user through multiple rounds of questions and answers, and prompting the user to supplement the deficient condition or delete redundant information until the SQL generation condition is met; the context of the query data, the analysis result of the knowledge graph, metadata information obtained through multi-round question and answer supplementation and the incidence relation are integrated into a prompt, and the Text2SQL model generates an SQL statement based on the adjusted prompt. According to the method and the device, the accuracy of the SQL statement generated when the Text2SQL large model processes the multi-table associated query can be remarkably improved.

Description

Technical Field

[0001] The present application relates to the technical field of database query optimization, and in particular to a query method based on a knowledge graph and a Text2SQL model. Background Art

[0002] With the rapid development of natural language processing technology, Text2SQL system has become an important tool for converting natural language queries into SQL statements. However, when dealing with complex scenarios involving multi-table association queries, traditional large models often lack structural knowledge of specific databases (such as private databases of universities) and find it difficult to accurately understand the relationships between tables, the meaning of fields, and the dependencies between them. This results in low accuracy of the generated SQL statements and low query efficiency. Knowledge graph is a semantic network that reveals the relationships between entities. It stores knowledge in the form of a graph. The nodes in the graph represent entities, and the edges represent the relationships between entities. The architecture of the knowledge graph includes a schema layer and a data layer, which can fully describe the structure and metadata of the database. Knowledge graph technology is usually used in combination with graph retrieval enhancement generation (Graph RAG) technology. Graph RAG uses the structured characteristics of the knowledge graph to enhance the depth and contextual relevance of the retrieved information, becoming a powerful supplement to the traditional vector search method.

[0003] However, when processing multi-table queries, the existing Rag model mainly relies on language vectorization similarity for intent recognition and table extraction, ignoring the complex relationships between tables and fields, such as inclusion relationships, field annotations, primary key and foreign key relationships, etc. These relationships are crucial for accurate SQL generation, but traditional methods often fail to effectively utilize this information.

[0004] Therefore, how to generate SQL statements more accurately in scenarios where knowledge graphs and Text2SQL large models are used for multi-table queries is an urgent problem that needs to be solved. Summary of the invention

[0005] This application provides a query method based on knowledge graph and Text2SQL model, which can generate SQL statements more accurately in the scenario of multi-table query using knowledge graph and Text2SQL large model.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] In a first aspect, the present invention provides a query method based on a knowledge graph and a Text2SQL model, the method comprising:

[0008] Collect metadata to be queried, extract structural information of the metadata, and construct a knowledge graph based on the structural information;

[0009] Creating a vector index based on the metadata, scoring the metadata index based on a weight, and storing the processed metadata in a vector database;

[0010] Collecting the user's query data, using the Text2SQL model to perform intent recognition on the metadata in the vector database that matches the context of the query data, extracting keywords and analyzing the user's query intent;

[0011] According to the context of the user query data and the intent recognition, the relationship between the keywords is analyzed using the knowledge graph to determine whether the keywords can meet the conditions for SQL generation; if it is determined that they do not meet the conditions, the user is prompted to supplement the missing conditions or delete redundant information through multiple rounds of question and answer interaction until the SQL generation conditions are met;

[0012] The context of the query data, the parsing result of the knowledge graph, and the metadata information and association relationships obtained through multiple rounds of question and answer supplementation are integrated into the prompt, and the Text2SQL model generates an SQL statement based on the adjusted prompt;

[0013] A query operation is performed on the vector database based on the SQL statement to obtain a query result.

[0014] In a preferred example of the present application, it can be further configured that the collecting of metadata to be queried includes:

[0015] Collect the basic business metadata to be queried, view and storage information, ETL cleaning SQL information, indicator caliber algorithm information and interface link tracking SQL to obtain metadata.

[0016] In a preferred example of the present application, it can be further configured that the building of the knowledge graph based on the structural information includes:

[0017] Different business domains are divided according to business scenarios, and knowledge graphs are built based on these business domains.

[0018] In a preferred example of the present application, it can be further configured to include:

[0019] When the user enters query data, the user's current input question is matched with the metadata, and an associated prompt is given to the user's input question based on the matching result.

[0020] In a preferred example of the present application, it can be further configured to include:

[0021] When the metadata index is scored based on the weight, the greater the weight, the higher the score of the table and field, and the tables and fields that have not been scored are all given a default basic score;

[0022] When matching is performed in the vector database, the minimum cosine distance between all vectors in the vector database and the query vector closest to the semantics is calculated, and the vectors are sorted according to the scores.

[0023] In a preferred example of the present application, it can be further configured to include:

[0024] When using the knowledge graph to parse the relationship between keywords and determine whether the keywords can satisfy the conditions for SQL generation, determine whether the keywords can satisfy SQL generation based on the relationships between tables, tables and fields, fields and fields, field annotations, or the relationships between fields and primary keys and foreign keys in the knowledge graph. If it is determined to be satisfied, the table and field related information and link conditions in the keywords are added to the prompt.

[0025] In a second aspect, the present application provides a query device based on a knowledge graph and a Text2SQL model, the device comprising:

[0026] A data collection module, used to collect metadata to be queried, extract structural information of the metadata, and construct a knowledge graph based on the structural information;

[0027] A vector database module, used to create a vector index based on the metadata, score the index of the metadata based on the weight, and store the processed metadata into a vector database;

[0028] A context analysis module, for collecting query data of the user, using a Text2SQL model to perform intent recognition on the metadata in the vector database that matches the context of the query data, extracting keywords and analyzing the query intent of the user;

[0029] A judgment module is used to analyze the relationship between keywords based on the context of the user query data and the intent recognition, and to judge whether the keywords can meet the conditions for SQL generation; if it is judged that they do not meet the conditions, the judgment module interacts with the user through multiple rounds of questions and answers, prompting the user to supplement the missing conditions or delete redundant information until the SQL generation conditions are met;

[0030] The query module is used to integrate the context of the query data, the parsing results of the knowledge graph, and the metadata information and association relationships obtained through multiple rounds of question and answer into the prompt. The Text2SQL model generates an SQL statement based on the adjusted prompt; based on the SQL statement, a query operation is performed on the vector database to obtain a query result.

[0031] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the query method based on the knowledge graph and the Text2SQL model as described in any one of the above items are implemented.

[0032] In a fourth aspect, the present application provides a computer-readable storage medium having a program stored thereon, wherein when the program is executed by a processor, the query method based on the knowledge graph and the Text2SQL model as described in any one of the above items is implemented.

[0033] In a fifth aspect, the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the query method based on the knowledge graph and the Text2SQL model as described in any one of the above items.

[0034] In summary, compared with the prior art, the technical solution provided in the embodiment of the present application has at least the following beneficial effects:

[0035] 1. This application constructs a knowledge graph containing multi-dimensional metadata information and integrates it into a system based on the Text2SQL large model, and uses vector database technology to efficiently retrieve metadata tags to provide a basis for intent recognition. At the same time, based on the user query context and the parsed content of intent recognition, keywords are extracted, and the relationship between keywords is analyzed and parsed in advance to determine whether the keywords can meet the SQL generation conditions. If the judgment is satisfied, the table, field related information and link conditions in the keyword are added to the prompt prompt; if the judgment is not satisfied, the user intention is supplemented through multiple rounds of questions and answers. Finally, SQL generation based on the fine-tuned prompt can significantly improve the accuracy of SQL statements generated by the Text2SQL large model when processing multi-table association queries. Experimental data show that compared with the traditional RAG model, the Graph RAG model using the method of the present invention has an accuracy improvement of 65% (from 20% to 85%), and the rate of meeting the demand is also greatly improved (from 1 minute 35 seconds to 17 seconds). This not only optimizes query efficiency, but also improves user experience, and provides new ideas and methods for the development of natural language processing and database query optimization technology.

[0036] 2. This application adds the metadata information and associations supplemented by the query context and the knowledge graph parsing context to the prompt, and provides it to the Text2SQL large model, so that the Text2SQL large model does not need to or reduces the use of associations, and directly outputs results based on known conditions, thereby improving the accuracy of the generated SQL statements. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart of a query method based on a knowledge graph and a Text2SQL model is provided for one embodiment of the present application.

[0038] Figure 2 A structural diagram of a query device based on a knowledge graph and a Text2SQL model provided for one embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] In one embodiment of the present application, a query method based on a knowledge graph and a Text2SQL model is provided. Figure 1 As shown, the method includes:

[0041] S100: Collect metadata to be queried, extract structural information of the metadata, and construct a knowledge graph based on the structural information.

[0042] Specifically, automated scripts or tools are designed to regularly extract metadata from the data warehouse generated by information standards through multi-dimensional metadata collection methods.

[0043] Furthermore, the collecting of metadata to be queried includes:

[0044] Collect the basic business metadata, view and storage information, ETL cleaning SQL information, indicator caliber algorithm information and interface link tracking SQL to obtain metadata;

[0045] S200: Creating a vector index based on the metadata, scoring the metadata index based on a weight, and storing the processed metadata in a vector database;

[0046] In specific implementation, after the metadata is collected, a vector index is created, a series of tags are appended to the main metadata index, and the tags are scored according to the weights and then stored in the vector database.

[0047] S300: collecting query data of the user, using the Text2SQL model to perform intent recognition on the metadata in the vector database that matches the context of the query data, extracting keywords and analyzing the query intent of the user;

[0048] In specific implementation, the Text2SQL large model performs intent recognition, extracts keywords, and analyzes user query intent based on the query context and matching metadata information in the vector database.

[0049] S400: according to the context of the user query data and the intent recognition, the relationship between the keywords is analyzed by using the knowledge graph to determine whether the keywords can meet the conditions for SQL generation; if it is determined that they do not meet the conditions, the user is prompted to supplement the missing conditions or delete the redundant information through multiple rounds of question and answer interaction until the SQL generation conditions are met;

[0050] In specific implementation, keywords are extracted based on the user query context and the parsed content of intent recognition, and the relationship between keywords is analyzed and parsed in advance. And based on the known table-to-table relationship, table-to-field relationship, field-to-field relationship, field annotation, field-to-primary key and foreign key relationship, etc., it is determined whether the keywords can meet the SQL generation. If it is determined to be satisfied, the table and field related information and link conditions in the keyword are added to the prompt word. If it is determined not to be satisfied, the user intent is supplemented through multiple rounds of questions and answers.

[0051] S500: Integrate the context of the query data, the parsing result of the knowledge graph, and the metadata information and association relationships obtained through multiple rounds of question and answer supplementation into the prompt, and the Text2SQL model generates an SQL statement based on the adjusted prompt;

[0052] In the specific implementation, the metadata information and association relationships supplemented by the query context and the knowledge graph parsing context are added to the prompt and provided to the Text2SQL model, so that the Text2SQL model does not need to or reduces the use of associations and directly outputs results based on known conditions.

[0053] Furthermore, it also includes:

[0054] When using the knowledge graph to parse the relationship between keywords and determine whether the keywords can satisfy the conditions for SQL generation, determine whether the keywords can satisfy SQL generation based on the relationships between tables, tables and fields, fields and fields, field annotations, or the relationships between fields and primary keys and foreign keys in the knowledge graph. If it is determined to be satisfied, the table and field related information and link conditions in the keywords are added to the prompt.

[0055] S600: Execute a query operation on the vector database based on the SQL statement to obtain a query result.

[0056] In this embodiment, the beneficial effects include at least:

[0057] 1. This application constructs a knowledge graph containing multi-dimensional metadata information and integrates it into a system based on the Text2SQL large model, and uses vector database technology to efficiently retrieve metadata tags to provide a basis for intent recognition. At the same time, based on the user query context and the parsed content of intent recognition, keywords are extracted, and the relationship between keywords is analyzed and parsed in advance to determine whether the keywords can meet the SQL generation conditions. If the judgment is satisfied, the table, field related information and link conditions in the keyword are added to the prompt prompt; if the judgment is not satisfied, the user intention is supplemented through multiple rounds of questions and answers. Finally, SQL generation based on the fine-tuned prompt can significantly improve the accuracy of SQL statements generated by the Text2SQL large model when processing multi-table association queries. Experimental data show that compared with the traditional RAG model, the Graph RAG model using the method of the present invention has an accuracy improvement of 65% (from 20% to 85%), and the rate of meeting the demand is also greatly improved (from 1 minute 35 seconds to 17 seconds). This not only optimizes query efficiency, but also improves user experience, and provides new ideas and methods for the development of natural language processing and database query optimization technology.

[0058] 2. This application adds the metadata information and associations supplemented by the query context and the knowledge graph parsing context to the prompt, and provides it to the Text2SQL large model, so that the Text2SQL large model does not need to or reduces the use of associations, and directly outputs results based on known conditions, thereby improving the accuracy of the generated SQL statements.

[0059] In some embodiments, constructing a knowledge graph based on the structural information includes:

[0060] Different business domains are divided according to business scenarios, and knowledge graphs are built based on these business domains.

[0061] During the specific implementation, fine-tuning is performed based on metadata, different business domains such as student management domain and human resources domain are divided according to business scenarios, and a knowledge graph is constructed based on the above content.

[0062] In this embodiment, the accuracy of knowledge graph construction is improved, thereby improving the accuracy of subsequent keyword extraction, and at the same time improving the efficiency of Text2SQL large model in processing data.

[0063] In some embodiments, it also includes:

[0064] When the user enters query data, the user's current input question is matched with the metadata, and an associated prompt is given to the user's input question based on the matching result.

[0065] In this embodiment, not only the user's query experience is improved, but also the accuracy of the query statement input by the user is improved, thereby improving the accuracy of the query result.

[0066] In some embodiments, it also includes:

[0067] When the metadata index is scored based on the weight, the greater the weight, the higher the score of the table and field, and the tables and fields that have not been scored are all given a default basic score;

[0068] When matching is performed in the vector database, the minimum cosine distance between all vectors in the vector database and the query vector closest to the semantics is calculated, and the vectors are sorted according to the scores.

[0069] In this embodiment, the accuracy of the query result and the query efficiency are improved by improving the index of the vector database.

[0070] This application also provides a query device based on knowledge graph and Text2SQL model, please refer to Figure 2 As shown, the device comprises:

[0071] The data collection module 100 is used to collect metadata to be queried, extract structural information of the metadata, and construct a knowledge graph based on the structural information;

[0072] The vector database module 200 is used to create a vector index for the metadata, score the index of the metadata based on the weight, and store the processed metadata into a vector database;

[0073] A context analysis module 300 is used to collect user query data, use a Text2SQL model to perform intent recognition on the metadata in the vector database that matches the context of the query data, extract keywords and analyze the user's query intent;

[0074] The judgment module 400 is used to analyze the relationship between keywords based on the context of the user query data and the intent recognition, and to judge whether the keywords can meet the conditions for SQL generation; if it is judged that they do not meet the conditions, the judgment module 400 interacts with the user through multiple rounds of questions and answers, prompting the user to supplement the missing conditions or delete the redundant information until the SQL generation conditions are met;

[0075] The query module 500 is used to integrate the context of the query data, the parsing results of the knowledge graph, and the metadata information and association relationships obtained through multiple rounds of question and answer into the prompt. The Text2SQL model generates an SQL statement based on the adjusted prompt; based on the SQL statement, a query operation is performed on the vector database to obtain a query result.

[0076] The functional implementation of each module in the above-mentioned query device based on knowledge graph and Text2SQL model corresponds to the steps in the above-mentioned query method embodiment based on knowledge graph and Text2SQL model, and its functions and implementation process will not be repeated here one by one.

[0077] The present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the query method based on the knowledge graph and the Text2SQL model as described in any of the above embodiments are implemented.

[0078] The present application also provides a computer-readable storage medium, on which a program is stored, wherein the computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive, and / or a memory stick, etc., and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The working process, working details, and technical effects of the computer-readable storage medium provided in this embodiment can be found in the above embodiment of a query method based on a knowledge graph and a Text2SQL model, which will not be repeated here.

[0079] The application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the query method based on the knowledge graph and the Text2SQL model as described in any of the above embodiments.

[0080] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0081] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above-mentioned embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several deformations and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the patent of this application shall be based on the attached claims.

Claims

1. A query method based on knowledge graph and Text2SQL model, characterized in that: include: Collect metadata to be queried, extract structural information of the metadata, and construct a knowledge graph based on the structural information; Creating a vector index based on the metadata, scoring the metadata index based on a weight, and storing the processed metadata in a vector database; Collecting the user's query data, using the Text2SQL model to perform intent recognition on the metadata in the vector database that matches the context of the query data, extracting keywords and analyzing the user's query intent; According to the context of the user query data and the intent recognition, the relationship between the keywords is analyzed using the knowledge graph to determine whether the keywords can meet the conditions for SQL generation; If it is judged that the conditions are not met, the system will interact with the user through multiple rounds of questions and answers, prompting the user to supplement the missing conditions or delete redundant information until the SQL generation conditions are met; The context of the query data, the parsing result of the knowledge graph, and the metadata information and association relationships obtained through multiple rounds of question and answer supplementation are integrated into the prompt, and the Text2SQL model generates an SQL statement based on the adjusted prompt; A query operation is performed on the vector database based on the SQL statement to obtain a query result.

2. The query method based on knowledge graph and Text2SQL model according to claim 1, characterized in that: The collecting of metadata to be queried includes: Collect the basic business metadata to be queried, view and storage information, ETL cleaning SQL information, indicator caliber algorithm information and interface link tracking SQL to obtain metadata.

3. The query method based on knowledge graph and Text2SQL model according to claim 2 is characterized in that: The constructing of a knowledge graph based on the structural information includes: Different business domains are divided according to business scenarios, and knowledge graphs are built based on these business domains.

4. The query method based on knowledge graph and Text2SQL model according to claim 3 is characterized in that: Also includes: When the user inputs query data, the user's current input question is matched with the metadata, and an associated prompt is given to the user's input question based on the matching result.

5. The query method based on knowledge graph and Text2SQL model according to claim 3 is characterized in that: Also includes: When the metadata index is scored based on the weight, the greater the weight, the higher the score of the table and field, and the tables and fields that have not been scored are all given a default basic score; When matching is performed in the vector database, the minimum cosine distance between all vectors in the vector database and the query vector closest to the semantics is calculated, and the vectors are sorted according to the scores.

6. The query method based on knowledge graph and Text2SQL model according to claim 5 is characterized in that: Also includes: When using the knowledge graph to parse the relationship between keywords and determine whether the keywords can satisfy the conditions for SQL generation, determine whether the keywords can satisfy SQL generation based on the relationships between tables, tables and fields, fields and fields, field annotations, or the relationships between fields and primary keys and foreign keys in the knowledge graph. If it is determined to be satisfied, the table and field related information and link conditions in the keywords are added to the prompt.

7. A query device based on knowledge graph and Text2SQL model, characterized in that: include: A data collection module, used to collect metadata to be queried, extract structural information of the metadata, and construct a knowledge graph based on the structural information; A vector database module, used to create a vector index based on the metadata, score the index of the metadata based on the weight, and store the processed metadata into a vector database; A context analysis module, for collecting query data of the user, using a Text2SQL model to perform intent recognition on the metadata in the vector database that matches the context of the query data, extracting keywords and analyzing the query intent of the user; A judgment module is used to analyze the relationship between keywords based on the context of the user query data and the intent recognition, and to judge whether the keywords can meet the conditions for SQL generation; If it is judged that the conditions are not met, the system will interact with the user through multiple rounds of questions and answers, prompting the user to supplement the missing conditions or delete redundant information until the SQL generation conditions are met; The query module is used to integrate the context of the query data, the parsing results of the knowledge graph, and the metadata information and association relationships obtained through multiple rounds of question and answer into the prompt. The Text2SQL model generates an SQL statement based on the adjusted prompt; based on the SQL statement, a query operation is performed on the vector database to obtain a query result.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the query method based on the knowledge graph and the Text2SQL model as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, wherein when the program is executed by the processor, the query method based on the knowledge graph and the Text2SQL model as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising computer instructions, characterized in that When executed by a processor, the computer instructions implement the steps of the query method based on the knowledge graph and the Text2SQL model described in claims 1 to 6.

Images