A large model-based multi-order retrieval data enhancement method and device

Abstract

The application discloses a kind of based on big model and multi-order search data enhancement method and device, belong to big data technical field, method includes the following steps: index library construction, preliminary search, result reordering, generate SQL draft, multi-stage search optimization, execute query, return result to user and result feedback and iterative optimization, multi-stage search optimization includes first index optimization, secondary index optimization and error identification adjustment optimization again.The method and device of the application improve the relevance of context and user problems through intelligent multi-stage search technology, making it easier for big models to understand business data structures through example embedding, greatly improving the accuracy of generated query statements. By using examples instead of structures, the context length is shortened by avoiding embedding data table structures in the context, solving the performance degradation problem of traditional methods when processing large-scale data tables.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of thread winding, and particularly relates to a large model and multi-stage retrieval data enhancement method and device. BACKGROUND

[0002] Text2Sql is a natural language processing (NLP) technology that converts natural language queries into structured SQL queries. This technology enables non-technical users to interact with databases through natural language without needing to understand the complexity of SQL language. However, the accuracy of Text2Sql technology has always been a problem. With the popularity of large model technology, Text2Sql technology has made great progress relying on the powerful understanding ability of large models. However, the current Text2Sql technology based on large models still has the following problems and defects:

[0003] 1. Insufficient accuracy in parsing complex natural language queries. When the natural language query is complex or the target SQL statement is complex, it is often impossible to generate a normally executable query statement or get the correct query result.

[0004] 2. Many existing technologies rely on providing data table structures as context to large models. When the number of business data tables is very large, the context length may exceed the upper limit of the context length of the large model, causing the model to fail to run. Even if the context length does not exceed the upper limit of the context length of the large model, it will greatly affect the reasoning speed of the large model.

[0005] It should be noted that the above content belongs to the technical cognition of the inventor and does not necessarily constitute prior art. SUMMARY

[0006] To solve the above problems, the purpose of the present application is to provide a large model and multi-stage retrieval data enhancement method and device. Through intelligent multi-stage retrieval technology, the relevance of the context and the user's problem is improved. Through the example embedding method, the large model can better understand the business data structure, greatly improving the accuracy of the generated query statement. Through the example instead of structure method, the data table structure is avoided in the context embedding, the context length is shortened, and the performance decline problem of traditional methods in processing large-scale data tables is solved.

[0007] To achieve the above purpose, the present application provides a large model and multi-stage retrieval data enhancement Text2Sql method, which is applied to the above-mentioned device. The specific implementation process of the method includes the following steps:

[0008] S1: Index library construction;

[0009] Build a basic example index library of business, examples refer to natural language questions and corresponding SQL query question and answer pairs, the question and answer pairs added to the index library need to include the following information: natural language question, vector embedding of natural language question, SQL query, desensitized form of SQL query, and vector embedding of desensitized form of SQL query.

[0010] S2: preliminary retrieval;

[0011] After the user submits a query question, the user question is first converted into a corresponding vector embedding, and then the vector embedding and the original question are submitted to the vector database for retrieval.

[0012] For the query question text itself, BM25 and ranking function are used for similarity ranking calculation of the text itself.

[0013] Finally, the two scores of BM25 and ranking function are combined in a certain proportion for boosting and ranking, and the top N elements are returned.

[0014] S3: result reordering;

[0015] In step S2, N example results similar to the query question are obtained, and the similarity method used is a combination of vector cosine similarity and text BM25+. In this step, the approximate set is reordered to obtain the top K elements.

[0016] S4: generating a SQL draft;

[0017] The similar ranking results obtained from step S3 are combined with the pre-tuned prompt words as the context of the large model input, and then the query question is combined as the input of the large model, and then the output result of the large model is obtained as the SQL draft.

[0018] S5: multi-stage retrieval optimization;

[0019] The SQL draft obtained in S4 is optimized in multiple stages, and more accurate results are obtained after each iteration. The specific stages include first index optimization, second retrieval optimization, and third adjustment optimization.

[0020] S6: executing the query;

[0021] In step S5, the final executable SQL statement is obtained. In this step, the SQL statement is first syntax analyzed to ensure that the statement only involves query and returned data, and the data volume is within the allowed range. After the syntax detection passes, the statement is submitted to the database for query and the returned result is obtained.

[0022] S7: returning the result to the user;

[0023] Return the result from S6 according to the user's requirements in a reasonable form to the user.

[0024] S8: Result feedback and iterative optimization;

[0025] The system continuously enriches the example library and optimizes the model and retrieval algorithm based on user feedback and query results.

[0026] Further, the S5 is expanded as follows:

[0027] S51: First index optimization;

[0028] From the SQL draft obtained in step S4, first, the parameter desensitization is performed on the SQL draft, and then it is converted into an embedding vector. The first retrieval optimization adds the embedding vector similarity of the parameter desensitized SQL draft as a retrieval ranking parameter on the basis of the original first retrieval. At this time, the ranking score is a combination of the query problem BM25+ score, the cosine similarity of the query problem vector, and the cosine similarity of the parameter desensitized SQL draft vector. In steps S2 and S3, the first-stage coarse division is performed. Then, according to the coarse division result, the rerank score of the query problem vector and the rerank score of the parameter desensitized SQL draft vector are combined for fine-grained ranking to obtain a similar example set with higher precision than S3. Then, step S4 is synchronized to obtain the first index optimization draft SQL.

[0029] S52: Second index optimization;

[0030] The steps are the same as S51, and the second index optimization is used in the case of a large number of index examples.

[0031] S53: Error recognition adjustment optimization;

[0032] The optimized SQL draft and example set are integrated into the pre-adjusted error recognition prompt words and are input into the large model together, and then the large model outputs the adjusted SQL statement.

[0033] Further, the difference between the SQL generated in S53 and step S4 is that in S4, the large model is required to write the corresponding SQL statement according to the query question, and in step S53, the large model is required to identify the problems in the SQL statement according to the query question and the SQL statement and correct them. Therefore, an error recognizer is added to change the focus of the large model from the overall statement structure to fine-grained error recognition.

[0034] Further, the vector embedding similarity in S2 is calculated using the following cosine similarity formula:

[0035]

[0036] Further, the similarity ranking calculation formula in S2 is as follows:

[0037]

[0038] Score(D,Q) is the score of the document D relative to the query Q; n is the number of words in the query Q; IDF(q i ) is the inverse document frequency of the i-th word in the query; f(q i ,D) is the frequency of word i in the document D; k1 and b are adjustable parameters; |D| is the length of the document D; Avgdl is the average length of the multiple documents; and δ is a positive parameter.

[0039] Further, the display form in S7 is a table, a chart or a natural language reply.

[0040] A Text2Sql device based on a large model and multi-stage retrieval data enhancement, the application environment of the device includes a server, a large model, a vector database and a business database.

[0041] The server specifically refers to a service receiving a user request, the large model refers to a large language generative model, the vector database refers to a vector database used for saving and retrieving vectors of examples, and the business database refers to a database storing system business data.

[0042] Further, the server and the large model, the server and the vector database and the server and the business database are connected through a network.

[0043] The device is applied to the server.

[0044] The large model and multi-stage retrieval data enhancement method and device based on the large model can bring the following beneficial effects:

[0045] 1. The method and device improve the relevance of the context and the user question through intelligent multi-stage retrieval technology, make the large model more easily understand the business data structure through the example embedding mode, and greatly improve the accuracy of the generated query statement.

[0046] 2. The method and device avoid embedding the data table structure in the context through the example instead of structure mode, shorten the context length, and solve the performance decline problem of the traditional method in processing large-scale data tables. BRIEF DESCRIPTION OF DRAWINGS

[0047] The accompanying drawings, which are included to provide a further understanding of the application and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0048] Figure 1 The application environment architecture diagram for the application environment of the present application.

[0049] Figure 2 The implementation flowchart of the large model and multi-order retrieval data enhancement Text2Sql method of the present application.

[0050] Figure 3 The specific flowchart of the multi-stage retrieval optimization of the present application. DETAILED DESCRIPTION

[0051] In order to more clearly explain the overall concept of the present application, the following will be described in detail in an exemplary manner with reference to the accompanying drawings.

[0052] The embodiment of the present application proposes a large model and multi-order retrieval data enhancement Text2Sql device, which can be applied in an application environment such as Figure 1 The application environment includes a server, a large model, a vector database, and a business database. The server specifically refers to a service that receives user requests, the large model refers to a large language generative model, the vector database refers to a vector database used to save and retrieve vectors of examples, and the business database refers to a database that stores system business data.

[0053] The server and the large model, the server and the vector database, and the server and the business database are all connected through a network, which can be a wired network or a wireless network.

[0054] The large model and multi-order retrieval data enhancement Text2Sql device provided by the embodiment of the present application is applied to the server.

[0055] In an embodiment, a large model and multi-order retrieval data enhancement Text2Sql method is provided, as shown in Figure 2 The method is implemented by the above device, and the specific implementation process of the method includes the following steps:

[0056] S1: Index library construction;

[0057] The basic example index library of the business is constructed, and the example refers to a question and answer pair of a natural language question and a corresponding SQL query. The question and answer pair added to the index library needs to include the following information: a natural language question, a vector embedding of the natural language question, a SQL query, a desensitized form of the SQL query (here, the desensitized form refers to excluding specific scalars in the query statement, and an example is as follows: assuming that the SQL query statement is "select name from user where id = 1", the desensitized form is "select name from user where id =?"), and a vector embedding of the desensitized form of the SQL query. The desensitization of the query statement scalar can reduce the influence of the scalar on retrieval and improve the recall rate of retrieval.

[0058] S2: preliminary retrieval;

[0059] After the user submits a query question, the user question is first converted into a corresponding vector embedding, and then the vector embedding and the original question text are submitted to the vector database for retrieval. The vector embedding similarity is calculated by using the following cosine similarity formula:

[0060]

[0061] The BM25 and ranking function are used for similarity ranking calculation of the text itself of the query question, and the formula is as follows:

[0062]

[0063] wherein Score(D, Q) is the score of the document D relative to the query Q; n is the number of words in the query Q; IDF(qi) is the inverse document frequency of the i th word in the query; f(qi, D) is the frequency of the word i in the document D; k1 and b are adjustable parameters; |D| is the length of the document D; Avgdl is the average length of the multiple documents; and δ is a positive parameter for avoiding excessive punishment of long documents in scoring. i i

[0064] Finally, the two scores of the BM25 and the ranking function are combined in a certain proportion for boosting and ranking, and the top N elements are returned. It should be noted that N is not equal to the total number of examples that need to be embedded into the context. Assuming that K examples need to be embedded into the context, then N = 5K or N = 10K. Example: if 10 examples need to be embedded into the context, then 50 to 100 top-ranked results need to be returned in this step.

[0065] S3: result reordering;

[0066] ​​In step S2, N example results similar to the query question are obtained, and the similarity method used is a combination of vector cosine similarity and text BM25+. This method only roughly obtains an approximate set. After rearranging the approximate set in this step, a more accurate similar ranking result is obtained (the meaning of rearrangement is that we will retrieve some results using similarity, but these results are large and unordered. We can use rearrangement techniques to accurately sort these results and select the results closer to the question. The final result of S3 is a small amount and in order), and then the top K elements (K is the number of embedded examples) are obtained. It should be noted that: the reason why the accurate sorting is not directly performed in step S2 is that the index data volume may be large, and the accurate similarity score needs to be input to the neural network in pairs for operation. The operation amount is large, and if all the examples in the library are calculated, the time consumption is huge. The method relies on vector database for vector operation design and optimization, and the cosine similarity of the query question to the entire index example can be obtained in a very short time, so as to serve as the result of rough division, greatly reducing the calculation amount of fine division. Thus, the balance between efficiency and accuracy is achieved.

[0067] S4: Generate SQL draft;

[0068] The similar ranking result obtained from step S3 is combined with the pre-optimized prompt word as the context of the large model input, and then the query question is combined to serve as the input of the large model. Then the output result of the large model is obtained as the SQL draft.

[0069] S5: Multi-stage retrieval optimization;

[0070] The SQL draft obtained in S4 is optimized in multiple stages, and the result is more accurate than the previous one after each iteration. The specific stages include first index optimization, second retrieval optimization, and third adjustment optimization.

[0071] S6: Execute query;

[0072] In step S5, the final executable SQL statement is obtained. In this step, the SQL statement is first subjected to syntax analysis to ensure that the statement only involves query and returned data, and the data volume is within the allowed range. After syntax detection, the statement is submitted to the database for query and the returned result is obtained.

[0073] S7: Return result to user;

[0074] The returned result obtained from step S6 is displayed to the user in a reasonable form according to the user's requirements, such as table, chart or natural language reply.

[0075] S8: Result feedback and iterative optimization;

[0076] The system continuously enriches the example library and optimizes the model and retrieval algorithm according to user feedback and query effect, to improve the efficiency and accuracy of future queries.

[0077] As Figure 3 shown below for step S5:

[0078] S51: First indexing optimization;

[0079] From the SQL draft obtained in step S4, first perform parameter desensitization on the SQL draft, and then convert it into an embedding vector. The first retrieval optimization adds the embedding vector similarity of the parameter desensitized SQL draft as a retrieval ranking parameter on the basis of the original first retrieval. At this time, the ranking score is a combination of the query question BM25+ score, the cosine similarity of the query question vector, and the cosine similarity of the parameter desensitized SQL draft vector. Synchronously with steps S2 and S3, this step performs the first-stage rough division, and then according to the rough division result, combines the rerank score of the query question vector and the rerank score of the parameter desensitized SQL draft vector to perform fine-grained ranking, to obtain a similar example set with higher precision than S3. Then, synchronously with step S4, the first indexing optimization draft SQL is obtained.

[0080] S52: Second indexing optimization;

[0081] The step is the same as S51. The second indexing optimization is used in the case where the index examples are extremely numerous. The set precision will be improved to a certain extent, but if the index examples are less, step S52 can be skipped as appropriate.

[0082] S53: Error recognition adjustment optimization;

[0083] The results of step S52 (if S52 is skipped, take the results of S51). Integrate the optimized SQL draft and the example set into the pre-adjusted error recognition prompt words, and input them into the large model together, and then the large model outputs the adjusted SQL statement. It should be noted that the difference between this step and step S4 is that in step S4, the large model is required to write the corresponding SQL statement according to the query question. However, in step S53, the large model is required to identify the problems in the SQL statement according to the query question and the SQL statement and correct them. The reason for adding this step is that in practice, after step S52, there are still incorrect SQL statements. Generally, the overall structure is correct, and the error points are usually in the fine-grained places. Therefore, an error recognizer is added to change the focus of the large model from the overall statement structure to fine-grained error recognition, which can effectively improve the accuracy of the SQL statement.

[0084] In an embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method and the device for large model based and multi-order retrieval data enhanced Text2Sql in the above embodiment when executing the computer program, for example Figure 2 The steps S1 to S8 shown.

[0085] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0086] The above only describes the embodiments of the present application and is not used to limit the present application. The present application can have various changes and modifications for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A large model and multi-order retrieval data enhancement Text2Sql method, characterized in that, The specific implementation process of the method comprises the following steps: S1: index library construction; S2: preliminary retrieval; After the user submits a query question, the user question is first converted into a corresponding vector embedding, and then the vector embedding and the original text of the question are submitted to the vector database for retrieval; For the query question text itself, BM25 and ranking function are used for similarity ranking calculation of the text itself; Finally, the two scores of BM25 and ranking function are combined in a certain proportion to enhance the ranking and return the top N elements; S3: result reordering; In step S2, N example results similar to the query question are obtained, and the similarity method used is a combination of vector cosine similarity and text BM25+. In this step, the similar set is reordered to obtain the top K elements; S4: generating a SQL draft; The similar ranking results obtained from step S3 are combined with the pre-tuned prompt words as the context of the large model input, and then the query question is combined as the input of the large model. Then the output result of the large model is obtained as the SQL draft; S5: multi-stage retrieval optimization; The SQL draft obtained in S4 is optimized in multiple stages, and more accurate results are obtained after each iteration. The specific stages include first index optimization, second retrieval optimization, and third adjustment optimization; Specifically, it includes: S51: first index optimization; The SQL draft obtained in step S4 is first subjected to parameter desensitization, and then converted into an embedding vector. The first retrieval optimization adds the embedding vector similarity of the parameter desensitized SQL draft as a retrieval ranking parameter based on the original first retrieval. At this time, the ranking score is a combination of the query question BM25+ score, the cosine similarity of the query question vector, and the cosine similarity of the parameter desensitized SQL draft vector. Steps S2 and S3 are synchronized. This step performs the first stage of rough division. Then, according to the rough division result, the query question vector reordering score and the parameter desensitized SQL draft vector reordering score are combined for fine ranking to obtain a similar example set with higher precision than S3. Then, step S4 is synchronized to obtain the first index optimization draft SQL; S52: second index optimization; The step is the same as S51, and the second index optimization is used in the case of multiple index examples; S53: error recognition adjustment optimization; The optimized SQL draft and the example set are integrated into the pre-adjusted error recognition prompt words, which are input into the large model together. Then the large model outputs the adjusted SQL statement; S6: execute the query.

2. The large model and multi-stage retrieval data enhancement Text2Sql method according to claim 1, characterized in that, The method further comprises: S7: return the result to the user; The returned result from step S6 is displayed to the user in a reasonable form according to the user's requirements; S8: result feedback and iterative optimization; The system continuously enriches the example library, optimizes the model, and improves the retrieval algorithm based on user feedback and query results.

3. The large model and multi-stage retrieval data enhancement Text2Sql method according to claim 2, characterized in that, In the S4 step, the large model is required to write the corresponding SQL statement according to the query question. In the S53 step, the large model is required to identify and correct the problems in the SQL statement according to the query question and the SQL statement. In the large model, an error identifier is added to convert the focus of the large model from the overall statement structure to the fine-grained error identification.

4. The large model and multi-stage retrieval data enhancement Text2Sql method according to claim 3, characterized in that, The S1 includes: A basic example index library of the business is constructed. The example refers to a question and answer pair of a natural language question and a corresponding SQL query. The question and answer pair added to the index library needs to include the following information: a natural language question, a vector embedding of the natural language question, an SQL query, a desensitized form of the SQL query, and a vector embedding of the desensitized form of the SQL query.

5. The large model and multi-stage retrieval data enhancement Text2Sql method according to claim 4, characterized in that, The similarity ranking calculation formula in the S2 is as follows: Score(D, Q) is the score of document D with respect to query Q; n is the number of words in query Q; IDF(q i ) is the inverse document frequency of the i-th word in the query; f(q i , D) is the frequency of word i in document D; k1 and b are adjustable parameters; |D| is the length of document D; Avgdl is the average length of the documents present; δ is a positive parameter.

6. The large model and multi-stage retrieval data enhancement Text2Sql method according to claim 5, characterized in that, The display form in the S7 is a table, a chart, or a natural language reply.

7. The large model and multi-stage retrieval data enhancement Text2Sql method according to claim 6, characterized in that, The S6 specifically includes: In the S5 step, the final executable SQL statement is obtained. In this step, the SQL statement is first subjected to syntax analysis to ensure that the statement only involves queried and returned data, and the data volume is within the allowed range. After the syntax detection passes, the statement is submitted to the database for querying and obtaining the returned result.

8. A large model and multi-order retrieval data enhancement Text2Sql device for implementing the method of any one of claims 1-7. The application environment of the device includes a server, a large model, a vector database, and a business database. The server specifically refers to a service that receives user requests. The large model refers to a large language generative model. The vector database refers to a vector database used to save and retrieve vectors of examples. The business database refers to a database that stores system business data.

9. The large model and multi-stage retrieval data enhancement Text2Sql apparatus of claim 8, wherein, The server and the large model, the server and the vector database, and the server and the business database are connected through a network link. The device is applied to the server.

Images