Table data retrieval method and device, retrieval enhancement generation method and device, electronic equipment and storage medium

Abstract

The invention discloses a table data retrieval method and device, a retrieval enhancement generation method and device, electronic equipment and a storage medium, and the method comprises the steps that in response to a query request of table data, retrieval is conducted through a pre-stored knowledge base in a retrieval enhancement generation RAG scene, and a retrieval result is obtained, the pre-stored knowledge base at least comprises original data of the table data; and filling the retrieval result into a preset cue word template to obtain a cue word prompt, and inputting the cue word prompt into a large language model LLM to obtain a final query result. By means of the table data retrieval method in the retrieval enhancement generation RAG scene, it can be ensured that retrieved table content is accurate and complete.

Description

Technical Field

[0001] This application relates to the technical fields of tabular data processing and large language models, and particularly relates to a tabular data retrieval method, a retrieval-augmented generation method for tabular data, a device, an electronic device, and a storage medium. Background Art

[0002] RAG (Retrieval-Augmented Generation) is an AI architecture that combines information retrieval and large model generation, mainly used to improve the knowledge accuracy and context understanding ability of large models. Since large language models (LLMs) face significant limitations, especially in specific domains or knowledge-intensive tasks, especially when dealing with queries that go beyond the scope of training data or require current information, they often produce "hallucinations", that is, generate inaccurate or false information. In RAG, by introducing external knowledge obtained through semantic retrieval, the hallucination problem is effectively alleviated.

[0003] In the related art, tabular data still cannot be processed using the general RAG process. Therefore, in the RAG scenario, a retrieval strategy for tabular data is required. Summary of the Invention

[0004] Embodiments of this application provide a tabular data retrieval method, a retrieval-augmented generation method, a device, an electronic device, and a storage medium to reduce complex table structures and table scales, and at the same time efficiently understand table semantics.

[0005] Embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a tabular data retrieval method, which is applied to a query server. The retrieval method includes:

[0007] In response to a query request for tabular data, a retrieval result is obtained by retrieving through a pre-stored knowledge base in the Retrieval-Augmented Generation (RAG) scenario, where the pre-stored knowledge base at least includes the original data of the tabular data;

[0008] The retrieval result is filled into a pre-set prompt template to obtain a prompt, which is input into a large language model (LLM) to obtain a final query result.

[0009] In some embodiments, the step of obtaining a retrieval result by retrieving through a pre-stored knowledge base in the Retrieval-Augmented Generation (RAG) scenario in response to a query request for tabular data includes:

[0010] According to the query request for the tabular data, a query instruction is parsed;

[0011] According to the query instruction, in the Retrieval-Augmented Generation (RAG) scenario, retrieve results are obtained through the first storage relationship, the second storage relationship, and the third storage relationship in the pre-stored knowledge base.

[0012] Among them, the first storage relationship is used as structured table data, the second storage relationship is used as table description data, and the third storage relationship is used as table structure description data.

[0013] In some embodiments, the method further includes:

[0014] Adopting the method of globally retrieving data to obtain a first retrieval result in the pre-stored knowledge base;

[0015] Adopting the method of range retrieving data to obtain a second retrieval result in the pre-stored knowledge base;

[0016] Adopting the method of converting text to structured query language to obtain a third retrieval result in the pre-stored knowledge base;

[0017] Among them, the range of the second retrieval result is smaller than that of the first retrieval result, and the second retrieval result is a target table determined first based on the semantic matching result between the table description data and the query request, and the retrieval result obtained from the target table;

[0018] The third retrieval result is to first determine the target table through the semantic matching between the query request and the table description, and then obtain the table structure description of the table.

[0019] In some embodiments, the method further includes:

[0020] According to the first retrieval result, the second retrieval result, and the third retrieval result, at least two of the retrieval results or all of the retrieval results are concatenated and filled into the prompt template to obtain at least one first prompt Prompt1;

[0021] Alternatively, according to the first retrieval result, the second retrieval result, and the third retrieval result, at least one of the retrieval results is filled into the prompt template alone to obtain at least one second prompt Prompt2;

[0022] Input the first prompt Prompt1 or the second prompt Prompt2 into the large language model (LLM) respectively to obtain the retrieval result.

[0023] In some embodiments,

[0024] The response to the query request for table data further includes:

[0025] In response to a query request for tabular data in a target scenario, the first retrieval result and the second retrieval result are obtained from the pre-stored knowledge base in a global data retrieval manner and a range data retrieval manner as reference information for the large language model (LLM).

[0026] Alternatively, in response to a query request for tabular data in the target scenario, the method of using text-to-structured query language is added.

[0027] In a second aspect, an embodiment of the present application further provides a retrieval-enhanced generation method for tabular data, which is applied to a client, and the generation method includes:

[0028] Determine the user's question and the reference information;

[0029] Fill the user's question and the reference information into a pre-set prompt template to obtain a prompt;

[0030] In the retrieval-enhanced generation (RAG) scenario, generate an answer to the user's question through the large language model (LLM) according to the reference information.

[0031] In some embodiments, the determining the user's question and the reference information includes:

[0032] Determine the user's question through the interaction scenario; and

[0033] Determine the reference information through the storage and processing result of the original data of the tabular data.

[0034] In a third aspect, an embodiment of the present application further provides a tabular data retrieval device, where the retrieval device includes:

[0035] A query module, configured to retrieve a retrieval result from a pre-stored knowledge base in a retrieval-enhanced generation (RAG) scenario in response to a query request for tabular data, where the pre-stored knowledge base at least includes the original data of the tabular data;

[0036] An input module, configured to fill the retrieval result into a pre-set prompt template to obtain a prompt, so as to input it into the large language model (LLM) to obtain a final query result.

[0037] In a fourth aspect, an embodiment of the present application further provides an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, and the executable instructions, when executed, cause the processor to execute the above method.

[0038] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, cause the electronic device to execute the above method.

[0039] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: In response to a query request for tabular data, a retrieval result is obtained by retrieving through a pre-stored knowledge base in the Retrieval-Augmented Generation (RAG) scenario, and the pre-stored knowledge base includes at least the original data of the tabular data. Thus, corresponding retrieval results can be obtained based on different reference information in the knowledge base, and the retrieval results are filled into a pre-set prompt template to obtain a prompt, which is input into a large language model (LLM) to obtain the final query result, thereby ensuring the accuracy and completeness of the retrieved tabular content. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0041] Figure 1 is a schematic flowchart of a method for retrieving tabular data in an embodiment of the present application;

[0042] Figure 2 is a schematic flowchart of a retrieval-augmented generation method for tabular data in an embodiment of the present application;

[0043] Figure 3 is a schematic diagram of using global retrieval data in a method for retrieving tabular data in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of using range retrieval data in a method for retrieving tabular data in an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of using the operation result of text-to-SQL in a method for retrieving tabular data in an embodiment of the present application;

[0046] Figure 6 is a schematic structural diagram of a device for retrieving tabular data in an embodiment of the present application;

[0047] Figure 7 is a schematic structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments of this application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts fall within the scope of protection of this application.

[0049] In the RAG retrieval-enhanced generation in the table scenario, the general process of RAG cannot be used to process table data. The main reasons at least include:

[0050] (1) Complex table structure: Considering the complexity of the table structure, the relationships of the table data input, including but not limited to cells, rows, and columns, are difficult for the large language model (LLM) that usually takes text as input to understand, and it needs to be converted into markdown or structured data format before it can be used.

[0051] (2) Table semantics: The table content cannot be represented as text with complete semantics, especially when the table header is information in a professional field or the meaning of the value is unclear. For example, (SaaS: Yes) indicates that the customer has purchased the SaaS service, and professional field information is required as reference information.

[0052] (3) Table scale: The amount of data in the table is usually large, especially when the table has a lot of rows and columns. A single query may need to process a large amount of data. As a result, it may be limited by the input length limit of the LLM. And if the text length is too long, it will also affect the generation effect of the LLM.

[0053] The RAG retrieval-enhanced generation in the table scenario of the related technology mainly adopts the following means:

[0054] (1) Whole table recall, which will result in too long input, affect the LLM generation effect, and the table structure is difficult to understand.

[0055] (2) Table structure-based methods, such as using Text2SQL, generate SQL instructions through understanding the table structure (table schema) to obtain data, but the process is relatively complex and there are also problems with incorrect generation.

[0056] (3) Row and / or column recall method: By encoding / recalling key rows and / or columns, the context limitation problem is solved; however, it is affected by the data encoding effect, especially when the data is too long and the encoding effect is poor, which will result in a still poor retrieval (recall) effect.

[0057] In addition, most of the methods in the related technology only target a single table and do not propose solutions to deal with multiple tables.

[0058] In an embodiment of the present application, a method for retrieving tabular data is provided. First, by representing all data in a structured data form and providing a table description, it is convenient for the LLM to understand. Second, all data is finally recalled by keyword matching or executing SQL, without semantic matching with the Query request. Then, the recalled data is row and / or column data or the entire table data that meets the length limit, thereby reducing the length of the input text. Finally, by combining multiple retrieval contents, the accuracy and completeness of the retrieved table content are ensured.

[0059] The following will, with reference to the accompanying drawings, detail the technical solutions provided by the embodiments of the present application.

[0060] An embodiment of the present application provides a method for retrieving tabular data, as Figure 1 shown, a schematic flowchart of the method for retrieving tabular data in the embodiment of the present application is provided, and the method at least includes the following steps S110 to step S120:

[0061] Step S110, in response to a query request for tabular data, retrieve a retrieval result through a pre-stored knowledge base in the Retrieval-Augmented Generation (RAG) scenario, where the pre-stored knowledge base at least includes the original data of the tabular data.

[0062] According to the query request for tabular data, a retrieval result can be retrieved through the knowledge base in the Retrieval-Augmented Generation (RAG) scenario. For the knowledge base that at least includes the original data of the tabular data, after the tabular data is uploaded and parsed, various storage forms of the tabular data are obtained and saved to the corresponding database. It can be understood that the original data of the tabular data can be stored in the database according to structured tabular data, table description, and table structure description respectively. The database includes but is not limited to relational databases, vector databases, and ElasticSearch databases.

[0063] In the pre-stored knowledge base, the processing of tabular data includes but is not limited to processing all original tabular data (complete Excel files, multiple worksheets) into row and / or column / whole table data; storing the structured data in Elasticsearch in plain text form, storing the original table in a relational database (such as MySQL), storing the table structure description in a text search engine database, and storing the table description in a vector database.

[0064] For example, the row and / or column data to be processed is shown in Table 1,

[0065] Table 1

[0066]

[0067]

[0068] After processing the above Table 1, the following results are obtained:

[0069] 1. {"Name": "Li Si", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}

[0070] 2. {"Name": "Li Siji", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room301, 402, 503"}

[0071] If the entire table content, after being converted into structured data, is less than the set maximum length, the whole table data can be processed as such.

[0072] For example, the above example can be processed as follows:

[0073] [{"Name": "Li Si", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}, {"Name": "Li Siji", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}]

[0074] Alternatively, when the number of rows and / or columns in the table is very small, it can be processed as whole table data or as row and / or column data, depending on the user configuration.

[0075] Specifically, for table descriptions, manual writing and splicing are adopted, including but not limited to:

[0076] (1) Overall table description: The semantic description of the whole table.

[0077] (2) Field description: The description of each field (headers) in the table, such as type, sample value, etc., to facilitate the understanding of the fields by the LLM.

[0078] (3) Sample questions: To improve the matching effect, users can provide sample questions as part of the table description.

[0079] (4) Data example: A piece of structured data can be directly provided as an example.

[0080] Specifically, the processing of the table structure includes but not limited to: for the Text2SQL structured query language.

[0081] When obtaining the "Text to SQL operation result", first determine the target table through semantic matching of the query with the table description, and then obtain the table structure description of that table.

[0082] The table structure description includes, but is not limited to, table name, fields (field name, field type, sample value).

[0083] Step S120: Fill the retrieved result into a pre-set prompt template to obtain a prompt, and input it into a large language model (LLM) to obtain a final query result.

[0084] After obtaining a prompt by filling the retrieved result into a pre-set prompt template, input it into a large language model (LLM) to obtain a retrieved result.

[0085] Through the above method, it can be applied to the retrieval enhancement generation of tabular data or the tabular data retrieval method in the retrieval enhancement generation (RAG) scenario.

[0086] Through the above method, the problem of complex table structures can be solved. Represent all data in a structured data form and provide a table description to facilitate the understanding of the large language model (LLM).

[0087] Through the above method, all data is finally recalled by keyword matching or executing SQL, without semantic matching with the query request for Query tabular data, which can solve the table semantics problem.

[0088] Through the above method, the recalled data is row and / or column data or the entire table data that meets the length limit, reducing the input length and solving the table scale problem.

[0089] Different from the related technology, the problem that retrieving the entire table will lead to too long input and affect the generation effect of the large language model (LLM). The above method represents all data in a structured data form and provides a table description, thus facilitating the understanding of the large language model (LLM).

[0090] Different from the related technology, the complex problem of generating SQL instructions through understanding the table structure (Table Schema) and then obtaining data. All data in the above method is finally recalled by keyword matching or executing SQL, without semantic matching with the Query request.

[0091] Different from the related technology, affected by the data encoding effect, especially when the data is too long and the encoding effect is poor, which will lead to a still poor retrieval effect. The above method retrieves row and / or column data or the entire table data that meets the length limit, thereby reducing the input text length.

[0092] In an embodiment of the present application, for the query request in response to tabular data, the retrieval result obtained by retrieving through a pre-stored knowledge base in the Retrieval-Augmented Generation (RAG) scenario includes: obtaining a query instruction by parsing according to the query request of the tabular data; obtaining a retrieval result by using the first storage relationship, the second storage relationship, and the third storage relationship in the pre-stored knowledge base in the Retrieval-Augmented Generation (RAG) scenario according to the query instruction, where the first storage relationship is used as structured tabular data, the second storage relationship is used as tabular description data, and the third storage relationship is used as tabular structure description data.

[0093] In an embodiment of the present application, the method further includes: obtaining a first retrieval result in the pre-stored knowledge base by using the method of global retrieval of data; obtaining a second retrieval result in the pre-stored knowledge base by using the method of range retrieval of data; obtaining a third retrieval result in the pre-stored knowledge base by using the method of text-to-structured query language; where the range of the second retrieval result is smaller than that of the first retrieval result, and the second retrieval result is the retrieval result obtained on the target table determined based on the semantic matching result between the tabular description data and the query request; the third retrieval result is to first determine the target table through the semantic matching between the query request and the tabular description, and then obtain the tabular structure description of the table.

[0094] It can be understood that the "tabular description" and "tabular structure description" in tabular data are stored in a vector database, such as Qdrant, Milvus, etc. The tabular description and tabular structure description of the same table are stored in the same piece of data. Specifically, a piece of data may include: the tabular description vector obtained by the tabular description of the table through a text embedding model, and the original tabular description text and the original tabular structure description text stored in the metadata. During retrieval, the user's query statement is semantically vector-matched with the tabular description to obtain the original tabular description text or the original tabular structure description text in the metadata.

[0095] For global retrieval of data: Retrieve data through keywords globally, that is, including all table files, Worksheet worksheets, rows, and columns. Global retrieval of data has the characteristic of high efficiency, and it is not necessary to consider the semantic matching between the tabular description and the Query.

[0096] For range retrieval of data: First, consider the semantic matching between the tabular description and the Query, then determine the target table, and then retrieve within the target table. Range retrieval of data is more accurate, reduces the retrieved noise, and provides the tabular description, facilitating the understanding of data by the Large Language Model (LLM).

[0097] For the operation result of text-to-SQL: First, consider the semantic matching between the table description and the Query, and then determine the target table; directly convert the Query into an executable SQL according to the target table structure to obtain the result. The method of text-to-SQL makes the retrieval result of table data more accurate and occupies less input context length. Specifically, first perform semantic matching with the table description, and after determining the target table, obtain the table structure description data corresponding to the target table; then fill the table description data and the query request into the prompt template of text-to-SQL to obtain the text-to-SQL prompt, and then input it into the large model to obtain the text-to-SQL statement and execute the text-to-SQL statement to obtain the retrieval result.

[0098] The above three parts of retrieval content, namely the method of global retrieval of data, the method of range retrieval of data, and the method of text-to-SQL, can ensure the accuracy and completeness of the retrieved table content.

[0099] As Figure 3 shown, the table data includes but is not limited to multiple table files, multiple workbooks (Work Sheet), and multiple table structures. An index is constructed and stored in the ES database. By performing information retrieval in the ES database, relevant data is obtained from the specified table in the structured table data.

[0100] As Figure 3 shown, global retrieval of data: Retrieve within the scope of all table data, similar to the effect of the retrieval part using general RAG. It has the advantages of simplicity and efficiency. When the Query is incomplete, if the matching of the table description in the range retrieval fails and relevant data cannot be retrieved, global retrieval of data can provide a supplement in this case.

[0101] Exemplarily, global retrieval of data.

[0102] (1) The user inputs a Query:

[0103] Which servers have Li Si as the administrator?

[0104] Reference information:

[0105] {"Name": "Li Si", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}

[0106] {"Name": "Li Siji", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}

[0107] …

[0108] (2) Obtain the prompt:

[0109] Answer the question based on the structured data and data field descriptions given in the reference information. Your answer should be concise and accurate.

[0110] Reference information:

[0111] {"Name": "Li Si", "Position Level": "Intermediate Server Administrator", "Managed Machine Rooms": "Room 301, 402, 503"}

[0112] {"Name": "Li Siji", "Position Level": "Intermediate Server Administrator", "Managed Machine Rooms": "Room 301, 402, 503"}

[0113] …

[0114] Question: Which servers are managed by Li Si?

[0115] (3) The large language model LLM outputs the retrieval result: Sorry, the reference information does not contain information about the servers managed by "Li Si".

[0116] When globally retrieving data, the retrieval effect may be poor due to the large amount of data, and there is too much noisy data. Structured tabular data is not text with complete semantics. Especially when the field names are professional terms or written in a certain way, if no description is provided, the LLM may not be able to understand, affecting the answering effect. For example, a "key-value pair" of "SaaS:Yes" cannot be understood, and the actual meaning is "Whether the customer has purchased the SaaS service: Yes".

[0117] Exemplarily, retrieve data within a range.

[0118] (1) User inputs the Query:

[0119] Which servers are managed by Li Si?

[0120] Reference information:

[0121]

[0122] Server Info: Basic information of the server, including fields such as server ID, server name, IP address, administrator name, etc.

[0123]

[0124] {"Server ID": "S001", "Server Name": "Server_A", "IP Address": "xxx.xxx.xx.xx", "Administrator": "Li Si"}

[0125] {"Server ID": "S005", "Server Name": "Server_E", "IP Address": "xxx.xxx.xx.xx", "Administrator": "Li Si"}

[0126] …

[0127]

[0128] Server Admin List: List of server administrators, including administrator name, rank, and managed computer rooms, etc.

[0129]

[0130] {"Name": "Li Si", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}

[0131] {"Name": "Li Siji", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}

[0132] …

[0133] (2) Obtain the prompt:

[0134] Answer the question based on the structured data and data field descriptions given in the reference information. Your answer should be concise and accurate.

[0135] Reference Information:

[0136]

[0137] Server Info: Basic information of the server, including fields such as server ID, server name, IP address, administrator name, etc.

[0138]

[0139] {"Server ID": "S001", "Server Name": "Server_A", "IP Address": "xxx.xxx.xx.xx", "Administrator": "Li Si"}

[0140] {"Server ID": "S005", "Server Name": "Server_E", "IP Address": "xxx.xxx.xx.xx", "Administrator": "Li Si"}

[0141]

[0142] Server Admin List: List of server administrators, including administrator name, rank, and managed computer rooms, etc.

[0143]

[0144] {"Name": "Li Si", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}

[0145] {"Name": "Li Siji", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}

[0146] …

[0147] Question: Which servers are administered by Li Si?

[0148] (3) The large language model LLM outputs retrieval results: According to the reference data, there are 5 servers administered by "Li Si" as follows:

[0149] Server_A

[0150] Server E

[0151] …

[0152] The range retrieval data first narrows the range and then performs an exact search. It has the advantages of more accurate retrieval and returns the matching table description to help the model understand the data semantics. However, it is also limited by the input context length of the large language model LLM and can only splice the top-K data of each target table, resulting in incomplete data.

[0153] For example, in the example, the matching servers may be more than top-5. The top-5 is only for illustration and does not limit the protection scope in the embodiments of this application. It is not applicable to some Queries, such as "How many servers are administered by Li Si?", "Which servers were repaired before February 28th?", "Which servers are in the shutdown state" (for example, the table content is "status": "0", and it cannot be matched by keywords), and these must use SQL queries.

[0154] As Figure 4 shown, the table data includes but is not limited to multiple table files, multiple workbooks (Work Sheet), and multiple table structures. An index is constructed and stored in the ES database. By performing information retrieval in the ES database, relevant data is obtained from the specified table in the structured table data. An index is constructed and stored in the vector database, and the target data range is narrowed and the table description is obtained through semantic retrieval in the table description.

[0155] Exemplarily, the text-to-SQL running result.

[0156] (1) User inputs Query:

[0157] How many servers are administered by Li Si?

[0158] Reference information:

[0159] ```SQLSELECT COUNT(*)

[0160] FROM`Server Info`

[0161] WHERE administrator = 'Li Si';``` executed successfully

[0162] Execution result: 13

[0163] (2) Get the prompt:

[0164] Answer the question based on the structured data and data field descriptions given in the reference information. Your answer should be concise and accurate.

[0165] Reference information:

[0166] ```SQLSELECT COUNT(*)

[0167] FROM`Server Info`

[0168] WHERE administrator = 'Li Si';``` executed successfully

[0169] Execution result: 13

[0170] Question: How many servers are administered by Li Si?

[0171] (3) The large language model LLM outputs the retrieval result: According to the reference data, Li Si manages 13 servers.

[0172] Using text-to-SQL to run the result has the advantage of being able to provide the most accurate and shortest retrieval result, but there is a deficiency that text-to-SQL may produce errors, resulting in execution failure.

[0173] As Figure 5 shown, the Query inputs the large language model LLM, generates the SQL retrieval result and executes the SQL execution result. The Query matches the corresponding semantic information in the table representation and table structure through semantic retrieval and inputs it into the large language model LLM.

[0174] In one embodiment of the present application, the method further includes: according to the first retrieval result, the second retrieval result, and the third retrieval result, splicing at least two of the retrieval results or all of the retrieval results and filling them into a prompt template to obtain at least one first prompt Prompt1; or, according to the first retrieval result, the second retrieval result, and the third retrieval result, filling at least one of the retrieval results into the prompt template alone to obtain at least one second prompt Prompt2; inputting the first prompt Prompt1 or the second prompt Prompt2 into the large language model LLM respectively to obtain retrieval results.

[0175] The first prompt can be obtained by splicing, or the second prompt can be generated by directly putting it into the prompt template without splicing.

[0176] Exemplarily, global retrieval data + range retrieval data + text-to-SQL operation result

[0177] User input Query:

[0178] How many servers have Li Si as the administrator?

[0179] Reference information:

[0180]

Global retrieval data

[0181] {"Name": "Li Si", "Position level": "Intermediate server administrator", "Managed computer rooms": "Room 301, 402, 503"}

[0182] {"Name": "Li Siji", "Position level": "Intermediate server administrator", "Managed computer rooms": "Room 301, 402, 503"}

[0183] …

[0184]

Range retrieval data

[0185]

[0186] Server Info: Basic information of the server, including fields such as server ID, server name, IP address, and administrator name.

[0187]

[0188] {"Server ID": "S001", "Server name": "Server_A", "IP address": "xxx.xxx.xx.xx", "Administrator": "Li Si"}

[0189] {"Server ID": "S005", "Server Name": "Server_E", "IP Address": "xxx.xxx.xx.xx", "Administrator": "Li Si"}

[0190]

[0191] Server Admin List: List of server administrators, including administrator name, rank, and managed computer rooms, etc.

[0192]

[0193] {"Name": "Li Si", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}

[0194] {"Name": "Li Siji", "Rank": "Intermediate Server Administrator", "Managed Computer Rooms": "Room 301, 402, 503"}

[0195] …

[0196]

Text to Structured Query Language Execution Result

[0197] ```SQLSELECT COUNT(*)

[0198] FROM `Server Info`

[0199] WHERE Administrator = 'Li Si';``` executed successfully

[0200] Execution result: 13

[0201] Question: How many servers have Li Si as the administrator?

[0202] (3) Large Language Model LLM Output Retrieval Result: According to the reference information, there are 13 servers with Li Si, an intermediate server administrator, as the administrator. For example

[0203] Server_A, IP Address: xxx.xxx.xx.xx

[0204] Server_E, IP Address: xxx.xxx.xx.xx

[0205] …

[0206] Using global retrieval data, data can be retrieved globally by keywords; using range retrieval data: first consider the semantic matching between the table description and the query request to determine the target table, and then retrieve within the target table; using the result of text-to-structured query language execution: first consider the semantic matching between the table description and the query request to determine the target table; directly convert the query request into an executable structured query language according to the target table structure to obtain the result.

[0207] The above method combines the three parts of retrieval content and takes into account the mutual conversion or complementarity among (range retrieval data, global retrieval data, and the result of text-to-structured query language execution). When the query request is too short in range retrieval data and fails to match the table description, global retrieval data will be used; when the retrieval in global retrieval data is incomplete and leads to retrieval failure, the result of text-to-structured query language execution will be used; similarly, when the structured query language execution fails in the result of text-to-structured query language execution, global retrieval data will be used, and if the scope needs to be narrowed and noise reduced in global retrieval data, range retrieval data will be used), ensuring the accuracy and completeness of the retrieved table content.

[0208] In addition, the above-mentioned ways of mutual conversion or complementarity among range retrieval data, global retrieval data, and the result of text-to-structured query language execution are only examples and are not used to limit the protection scope in the embodiments of the present application.

[0209] In an embodiment of the present application, the response to the query request for table data further includes: in a target scenario, in response to the query request for table data, using the global retrieval data method and the range retrieval data method to obtain a first retrieval result and a second retrieval result in the pre-stored knowledge base as reference information for the large language model LLM; or, in the target scenario, in response to the query request for table data, adding the method of using text-to-structured query language.

[0210] The target scenario can be a query scenario that requires quick response, and in this case, global retrieval data can be used as reference information.

[0211] The target scenario can also be a query scenario that does not consider the server computing cost and focuses on high hit rate, and in this case, the result of text-to-structured query language execution can be used as reference information.

[0212] The target scenario can also be a regular query scenario, and in this case, a combination of global retrieval data and range retrieval data can be used.

[0213] Meanwhile, when the query request in the range retrieval data is too short and the matching table description fails, the global retrieval data can be used; when the retrieval in the global retrieval data is incomplete and leads to retrieval failure, the result of running the text-to-structured query language can be used; similarly, when the structured query language execution fails in the result of running the text-to-structured query language, the global retrieval data can be used. If you want to narrow the scope and reduce noise in the global retrieval data, the range retrieval data is used.

[0214] For the complementary relationship of the three parts of data, in most cases, the first retrieval result and the second result are obtained from the pre-stored knowledge base; the first retrieval result and the second result are spliced into the prompt template to obtain the target prompt. Using the range retrieval data and the global retrieval data can achieve a relatively accurate effect.

[0215] As Figure 2 shown, in the embodiment of the present application, a retrieval enhancement generation method for tabular data is further provided, which is applied to the client. The generation method includes:

[0216] Step S210, determining the user's question and reference information.

[0217] As Figure 2 shown, the user's question is obtained through interaction on the client, and the reference information includes but is not limited to the table structure description, table description, and structured table data. The knowledge base storing the reference information can be constructed through the table structure description, table description, and structured table data.

[0218] Step S220, filling the user's question and the reference information into a pre-set prompt template to obtain the prompt prompt.

[0219] Filling the user's question and reference information into the prompt template to obtain, for example, a prompt prompt that answers the question according to the structured data and data field description given in the reference information. Your answer should be concise and accurate.

[0220] Step S230, in the retrieval enhancement generation (RAG) scenario, generating an answer to the user's question according to the reference information through a large language model (LLM).

[0221] As Figure 2 shown, in the retrieval enhancement generation (RAG) scenario, generating an answer to the user's question according to the reference information through a large language model (LLM). Since the selection of the reference information is different, the answer to the user's question may also be different.

[0222] In one embodiment of the present application, determining the user's question and the reference information includes: determining the user's question through an interaction scenario; and determining the reference information through the storage processing result of the original data of the tabular data.

[0223] The client determines the user's question through the interaction scenario by storing tabular data in vector databases such as Qdrant and Milvus respectively. A text search engine database such as an ElasticSearch database. A relational database such as an SQL database.

[0224] It can be understood that the "table description" and "table structure description" in the tabular data are stored in vector databases such as Qdrant and Milvus. The table description and table structure description of the same table are stored in the same piece of data. Specifically, a piece of data may include: the table description vector obtained by the table description of the table through a text embedding model, and the original text of the table description and the original text of the table structure description stored in the metadata. During retrieval, the user's query statement is semantically vector-matched with the table description to obtain the original text of the table description or the original text of the table structure description in the metadata.

[0225] The embodiment of the present application also provides a tabular data retrieval device 600, as Figure 6 shown, which provides a schematic structural diagram of the tabular data retrieval device in the embodiment of the present application. The tabular data retrieval device 600 at least includes: a query module 610 and an input module 620, where:

[0226] In one embodiment of the present application, the query module 610 is specifically configured to: in response to a query request for tabular data, retrieve a retrieval result through a pre-stored knowledge base in the Retrieval-Augmented Generation (RAG) scenario, where the pre-stored knowledge base at least includes the original data of the tabular data.

[0227] According to the query request for tabular data, a retrieval result can be retrieved through the knowledge base in the Retrieval-Augmented Generation (RAG) scenario. The knowledge base at least includes the original data of the tabular data. After the tabular data is uploaded and parsed, various storage forms of the tabular data are obtained and saved to the corresponding databases. It can be understood that the original data of the tabular data can be stored in the database according to structured tabular data, table description, and table structure description respectively. The databases include but are not limited to relational databases, vector databases, and ElasticSearch databases.

[0228] It should be noted that the text search engine database specifically refers to the database used to store and manage text data crawled from the Internet. For example, Elasticsearch is a highly scalable, full-text retrieval and analysis engine that can store, search, and analyze massive amounts of data in real time. In the Elasticsearch database, data is stored in indexes and queried through the HTTP RESTful API, supporting multi-condition queries and aggregation queries.

[0229] In the pre-stored knowledge base, the processing of tabular data includes, but is not limited to, processing all original tabular data (Excel files, multiple worksheets) into row and / or column / whole table data; storing structured data in Elasticsearch (ES database) in plain text format, storing the original table in a relational database (such as MySQL), storing the table structure description in the text search engine database, and storing the table description in the vector database.

[0230] In an embodiment of the present application, the input module 620 is specifically configured to: fill the retrieval result into a pre-set prompt template to obtain a prompt word prompt, and input it into the large language model LLM to obtain the final query result.

[0231] After obtaining the prompt word by filling the retrieval result into the pre-set prompt template, input it into the large language model LLM to obtain the retrieval result.

[0232] In an embodiment of the present application, the query module 610 is further configured to:

[0233] Parse the query request of the tabular data to obtain a query instruction;

[0234] According to the query instruction, in the Retrieval-Augmented Generation (RAG) scenario, obtain the retrieval result through the first storage relationship, the second storage relationship, and the third storage relationship in the pre-stored knowledge base,

[0235] wherein, the first storage relationship is used as structured tabular data, the second storage relationship is used as table description data, and the third storage relationship is used as table structure description data.

[0236] In an embodiment of the present application, the query module 610 is further configured to:

[0237] Obtain the first retrieval result in the pre-stored knowledge base by using the global retrieval data method;

[0238] Obtain the second retrieval result in the pre-stored knowledge base by using the range retrieval data method;

[0239] Obtain a third retrieval result in the pre-stored knowledge base by means of text-to-structured query language;

[0240] Among them, the range of the second retrieval result is smaller than that of the first retrieval result, and the second retrieval result is a target table determined based on the semantic matching result between the tabular description data and the query request first, and the retrieval result obtained from the target table;

[0241] The third retrieval result first determines the target table through the semantic matching of the query request and the table description, and then obtains the table structure description of the table.

[0242] In an embodiment of the present application, the query module 610 is further configured to:

[0243] According to the first retrieval result, the second retrieval result, and the third retrieval result, splice at least two of the retrieval results or all of the retrieval results and fill them into the prompt template to obtain at least one first prompt Prompt1;

[0244] Alternatively, according to the first retrieval result, the second retrieval result, and the third retrieval result, fill at least one of the retrieval results into the prompt template alone to obtain at least one second prompt Prompt2;

[0245] Input the first prompt Prompt1 or the second prompt Prompt2 into the large language model LLM respectively to obtain the retrieval result.

[0246] In an embodiment of the present application, the query module 610 is further configured to:

[0247] In response to a query request for tabular data in a target scenario, obtain a first retrieval result and a second retrieval result in the pre-stored knowledge base by means of global retrieval data and range retrieval data as reference information for the large language model LLM;

[0248] Alternatively, in response to a query request for tabular data in the target scenario, add the method of text-to-structured query language.

[0249] It can be understood that the above tabular data retrieval device can implement each step of the tabular data retrieval method provided in the foregoing embodiments. The relevant explanations of the tabular data retrieval method are applicable to the tabular data retrieval device and will not be elaborated here.

[0250] Figure 7 It is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 7, at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include internal memory, such as high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0251] The processor, network interface, and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 only a two-way arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0252] The memory is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory can include internal memory and non-volatile memory, and provide instructions and data to the processor.

[0253] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a table data retrieval device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0254] In response to a query request for table data, retrieve a retrieval result through a pre-stored knowledge base in the retrieval-augmented generation (RAG) scenario, where the pre-stored knowledge base includes at least the original data of the table data;

[0255] Fill the retrieval result into a pre-set prompt template to obtain a prompt, and input it into a large language model (LLM) to obtain the final query result.

[0256] The above is as described in this application Figure 1The method executed by the table data retrieval device disclosed in the illustrated embodiment can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The above processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0257] The electronic device can also execute Figure 1 the method executed by the table data retrieval device in Figure 1 the illustrated embodiment and implement the functions of the table data retrieval device in

[0258] Embodiments of the present application also propose a computer-readable storage medium that stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including multiple application programs, can enable the electronic device to execute Figure 1 the method executed by the table data retrieval device in the illustrated embodiment, and specifically used to execute:

[0259] In response to a query request for table data, retrieve a retrieval result through a pre-stored knowledge base in the Retrieval-Augmented Generation (RAG) scenario, where the pre-stored knowledge base includes at least the original data of the table data;

[0260] Fill the retrieved results into a pre - set prompt template to obtain a prompt, and input it into a large - language model (LLM) to obtain the final query result.

[0261] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer - usable storage media (including but not limited to disk storage, CD - ROM, optical storage, etc.) containing computer - usable program code.

[0262] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general - purpose computer, a special - purpose computer, an embedded processor, or other programmable data - processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data - processing devices generate means for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0263] These computer program instructions can also be stored in a computer - readable memory that can direct a computer or other programmable data - processing device to work in a specific manner, so that the instructions stored in the computer - readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0264] These computer program instructions can also be loaded onto a computer or other programmable data - processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer - implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0265] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0266] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0267] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0268] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0269] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0270] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A table data retrieval method, wherein: Applied to a query server, the retrieval method includes: In response to a query request for the table data, obtaining a search result by searching a pre-stored knowledge base in a retrieval enhancement generation RAG scenario, wherein the pre-stored knowledge base at least includes original data of the table data; The search result is filled into a preset prompt word template to obtain a prompt word prompt, which is then input into the large language model LLM to obtain the final query result.

2. The method of claim 1, wherein: In response to the query request of the table data, obtaining the search result by searching the pre-stored knowledge base in the search enhancement generation RAG scenario includes: According to the query request of the table data, a query instruction is obtained by parsing; According to the query instruction, in the retrieval enhancement generation RAG scenario, the retrieval result is obtained by using the first storage relationship, the second storage relationship and the third storage relationship in the pre-stored knowledge base, The first storage relation is used as structured table data, the second storage relation is used as table description data, and the third storage relation is used as table structure description data.

3. The method according to claim 2, further comprising: Using a global search data method, a first search result is obtained in the pre-stored knowledge base; Using a range search method to obtain a second search result in the pre-stored knowledge base; Obtaining a third search result in the pre-stored knowledge base by converting the text into a structured query language; The scope of the second search result is smaller than that of the first search result, and the second search result is a target table determined based on the semantic matching result between the table description data and the query request, and a search result obtained in the target table; The third search result is to first determine the target table through semantic matching between the query request and the table description, and then obtain the table structure description of the table.

4. The method according to claim 2 or 3, further comprising: According to the first search result, the second search result and the third search result, at least two of the search results or all of the search results are spliced ​​and filled into a prompt word template to obtain at least one first prompt word Prompt1; Alternatively, according to the first search result, the second search result and the third search result, at least one of the search results is separately filled into a prompt word template to obtain at least one second prompt word Prompt2; The first prompt word Prompt1 or the second prompt word Prompt2 is respectively input into the large language model LLM to obtain the search result.

5. The method of claim 3, wherein: The response to the query request of the table data also includes: In response to a query request for table data in a target scenario, a global data search method and a range data search method are used to obtain a first search result and a second search result in the pre-stored knowledge base as reference information of a large language model LLM; Alternatively, in response to a query request for table data in the target scenario, the method of converting text to structured query language is added.

6. A retrieval enhancement generation method for tabular data, wherein: Applied to the client, the generation method includes: Identify user issues and reference information; Filling the user question and the reference information into a preset prompt word template to obtain a prompt word prompt; In the retrieval enhancement generation RAG scenario, the answer to the user's question is generated through the large language model LLM according to the reference information.

7. The method of claim 6, wherein: The determining of the user question and the reference information includes: Determine the user problem through the interaction scenario; and The reference information is determined by the stored processing result of the raw data of the table data.

8. A table data retrieval device, wherein: Applied to a query server, the retrieval device comprises: A query module, for responding to a query request of the table data, and obtaining a search result by searching a pre-stored knowledge base in a retrieval enhancement generation RAG scenario, wherein the pre-stored knowledge base at least includes original data of the table data; The input module is used to fill the search result into a preset prompt word template to obtain a prompt word prompt, so as to input it into the large language model LLM to obtain the final query result.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 5, and / or any one of claims 6 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute any of the methods of claims 1 to 5 and / or any of the methods of claims 6 to 7.

Images