A table content retrieval method, program product, device and medium

Abstract

The present application discloses a table content retrieval method, program product, device and medium, which relates to the technical field of data retrieval, including: obtaining a data table, constructing a question and answer database and a table block database based on the data table; receiving question information raised by a user, querying the question and answer database according to the question information to obtain a first query result, querying the table block database according to the question information to obtain a second query result; merging the question information, the first query result and the second query result to form input content, and inputting the input content into a preset language model to output a retrieval result. The present application clarifies a method for converting a data table into a question and answer database and a table block database for storage and indexing, which solves the problem of effectively reducing the storage space of long tables, and at the same time can solve the problem that the long table content cannot be input all at once due to the limitation of the large preset language model context window, thereby improving the retrieval accuracy of the data table.

Description

Technical Field

[0001] The present application relates to the technical field of data retrieval, and in particular to a table content retrieval method, program product, device and medium. Background Art

[0002] Retrieval-Augmented Generation (RAG) is a preset language model architecture that combines retrieval and generation. It is particularly suitable for answering open-domain questions and answers, generating relevant text, and tasks that require rich background information. It combines the characteristics of retrieval systems and generation preset language models. It generates more accurate and context-related answers or content by retrieving relevant information from a large database and processing it using a generation preset language model.

[0003] Among some related technologies, the current RAG is mainly based on embedded vector technology. After dividing the text into blocks, it retrieves relevant information by analyzing the semantic content. This method is only applicable to the expression logic that conforms to natural language, and has poor performance when processing documents containing data tables. Since the data in the table is organized according to specific results and levels, and there is no fixed order or hierarchical organization of the content, the accuracy of the retrieval output results is not high; at the same time, for longer data tables, when processing data, it is impossible to process the entire data table, and the need for segmented retrieval may cause the loss of content in the table. Summary of the invention

[0004] In order to solve at least one of the problems mentioned in the above background technology, the present application provides a table content retrieval method, program product, device and medium, which clearly converts data tables into question and answer databases and table block databases for storage and indexing, effectively reducing the storage space of long tables, and at the same time solving the problem that the long table content cannot be entered all at once due to the limitation of the large preset language model context window, thereby improving the retrieval accuracy of data tables.

[0005] The specific technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, a table content retrieval method is provided, the method comprising:

[0007] Obtain data tables, and build a question-and-answer database and a table block database based on the data tables;

[0008] Receive question information raised by the user, query the question-answer database according to the question information to obtain a first query result, and query the table block database according to the question information to obtain a second query result;

[0009] The question information, the first query result, and the second query result are combined to form input content, and the input content is input into a preset language model to output a search result.

[0010] In a specific embodiment, a question-and-answer database is constructed based on a data table, specifically including: using a preset language model to generate question information based on the data table, generating answer information by combining the question information and the data table content associated with the question information; constructing an association relationship between the question information and the answer information, wherein the question information and the answer information correspond one-to-one; and combining all the association relationships between the question information and the answer information to form a question-and-answer database.

[0011] In a specific embodiment, a preset context window size of a preset language model is obtained, and a data table size is obtained; if the table data size exceeds the preset context window size of the preset language model, the data table is split into several sub-tables according to the row data of the data table, and the sub-tables are set to have column information.

[0012] In a specific embodiment, it is confirmed that the size of the sub-tables meets the preset context window size, and the sub-tables are arranged in sequence according to the row numbers in the data table to form a table queue; in response to obtaining the answer information, the sub-tables in the table queue are input into the preset language model in sequence.

[0013] In a specific embodiment, a table block database is constructed based on a data table, and the table block database includes pattern data and cell data, specifically including: extracting the column name, data type and data value range in the data table; setting the column name, data type and the corresponding association with the data value range to form the pattern data; extracting the cell information and cell value in the data table, and setting the association between the cell information and the cell data to form the cell data.

[0014] In a specific embodiment, all data in a data table are acquired and arranged according to the data size to form a data queue; a minimum value in the data queue and a maximum value in the data queue are set to form a numerical range of the data.

[0015] In a specific embodiment, all cell data of a data table are obtained, and it is determined whether the number of cells in the data table exceeds a preset threshold; if the number of cells exceeds the preset threshold, the repetition frequency of the cell values ​​is obtained; the cell values ​​whose repetition frequency exceeds the preset frequency are stored in a table block database, and other cell values ​​are deleted.

[0016] In a specific embodiment, obtaining a first query result by querying a question and answer database based on question information specifically includes: obtaining feature information of the question information, querying a question and answer database in combination with the feature information to obtain question information matching the question information; querying answer information associated with the question information, and configuring the answer information associated with the question information as the first query result.

[0017] In a specific embodiment, keywords and semantic features of question information are obtained, keyword similarity is calculated by combining the keywords with question information in a question-and-answer database, and semantic similarity is calculated by combining the semantic features with the question information; keyword similarity and semantic similarity are weighted to obtain a comprehensive similarity; and question information associated with the comprehensive similarity that meets preset rules is obtained as question information that matches the question information.

[0018] In a specific embodiment, a table block database is queried according to the question information to obtain a second query result, which specifically includes: identifying the query intent of the question information, setting the query intent that meets the preset conditions to query through pattern data to obtain matching pattern data; setting the query intent that does not meet the preset conditions to query through cell data to obtain matching cell data, and combining the matching pattern data and the matching cell data to form the second query result.

[0019] In a specific embodiment, the number of columns of the matching pattern data and / or the matching cell data is obtained; when the number of columns is set to be greater than or equal to two columns, the data in the matching pattern data and / or the matching cell data are arranged in sequence according to the column numbers in the column information.

[0020] In a specific embodiment, the data table background information is obtained, in response to receiving the question information raised by the user, and confirming that the question information satisfies the background information; if the question information does not satisfy the background information, querying and confirming that the question information corresponds to the data table.

[0021] In a second aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0022] Step A: Obtain a data table, and build a question-answer database and a table block database based on the data table;

[0023] Step B: receiving question information raised by the user, querying the question-answer database according to the question information to obtain a first query result, and querying the table block database according to the question information to obtain a second query result;

[0024] Step C: Combine the question information, the first query result, and the second query result to form input content, and input the input content into a preset language model to output a search result.

[0025] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0026] Step A: Obtain a data table, and build a question-answer database and a table block database based on the data table;

[0027] Step B: receiving question information raised by the user, querying the question-answer database according to the question information to obtain a first query result, and querying the table block database according to the question information to obtain a second query result;

[0028] Step C: Combine the question information, the first query result, and the second query result to form input content, and input the input content into a preset language model to output a search result.

[0029] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0030] Step A: Obtain a data table, and build a question-answer database and a table block database based on the data table;

[0031] Step B: receiving question information raised by the user, querying the question-answer database according to the question information to obtain a first query result, and querying the table block database according to the question information to obtain a second query result;

[0032] Step C: Combine the question information, the first query result, and the second query result to form input content, and input the input content into a preset language model to output a search result.

[0033] The embodiments of the present application have the following beneficial effects:

[0034] 1. The solution provided in the embodiment of the present application, before asking questions to the user, builds a question and answer database and a table block database based on the initial data table, respectively, so as to query the above databases according to the user's question information, and inputs the query results on both sides into the preset language model to obtain the retrieval results. The embodiment of the present application clarifies the method of converting the data table into a question and answer database and a table block database for storage and indexing, which effectively reduces the storage space of long tables, and at the same time can solve the problem that the long table content cannot be input all at once due to the limitation of the large preset language model context window, thereby improving the retrieval accuracy of the data table. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 A schematic diagram of the application environment according to the present application is shown;

[0037] Figure 2 A schematic diagram of a method for retrieving content according to the application form is shown;

[0038] Figure 3 A schematic diagram showing a question-answer database formed according to the present application;

[0039] Figure 4 A schematic diagram of forming a table block database according to the present application is shown;

[0040] Figure 5 A schematic diagram showing obtaining a second query result according to the present application is shown;

[0041] Figure 6 A schematic diagram of a computer device according to the present application is shown. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] The table content retrieval method provided in this application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. Among them, the terminal 102 stores a data table to realize the construction of a question and answer database and a table block database based on the data table; receiving the question information raised by the user, querying the question and answer database according to the question information to obtain a first query result, and querying the table block database according to the question information to obtain a second query result; merging the question information, the first query result, and the second query result to form input content, and inputting the input content into the preset language model to output the retrieval result. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0044] In one embodiment, Figure 2 As shown, a table content retrieval method is provided, which is applied to Figure 1 The terminal in the example is used for explanation, wherein the terminal stores a data table and a preset language model. In this embodiment, the LLM model (Large Language Model, LLM model for short) is selected as the preset language model for explanation, wherein the LLM model is configured with a RAG question-answering function, such as Figure 2 As shown, the following steps are included:

[0045] Step 101: Obtain a data table, and build a question-answer database based on the data table.

[0046] The data table includes but is not limited to a table in an Excel or PDF document, and a question-answering database is constructed based on the data table, specifically including: using a preset language model to generate question information based on the data table, and generating answer information by combining the question information and the data table content associated with the question information; constructing an association relationship between the question information and the answer information, wherein the question information and the answer information correspond one to one; and combining all the association relationships between the question information and the answer information to form a question-answering database.

[0047] For example, Figure 3 As shown, the LLM model in this embodiment uses the moonshot-v1-128k API, and LLM can be replaced with any other deep learning model. Various question information is generated by using the LLM model based on the content in the data table, and answer information is generated based on the question information and the data table content, that is, the answer information is equivalent to the answer to the question, and the question and the answer are in a one-to-one correspondence, and a question and answer database is generated by combining all question and answer information.

[0048] Before using the LLM model to process the data table, in order to ensure that the word language model can process the data table, it is also necessary to judge the size of the data table, including: obtaining the preset context window size of the preset language model, and obtaining the data table size; if the table data size exceeds the preset context window size of the preset language model, the data table is split into several sub-tables according to the row data of the data table, and the sub-tables are set to have column information.

[0049] In this embodiment, if it is determined that the size of the data table exceeds the context window of the LLM model, the data table is split. Specifically, the table is split into multiple sub-tables according to the number of rows in the data table, and each sub-table is set to retain the column name to achieve accurate labeling of the data in each sub-table.

[0050] Furthermore, in order to ensure that the split tables meet the corresponding size requirements, it is set to confirm that the size of the sub-tables meets the preset context window size, and the sub-tables are arranged in sequence according to the row numbers in the data table to form a table queue; in response to obtaining the answer information, the sub-tables in the table queue are input into the preset language model in sequence.

[0051] Through the above settings, the split tables are arranged in sequence according to their initial row numbers in the data table during the splitting process to avoid omissions when the table data is input into the LLM model.

[0052] Step 102: construct a table block database based on the data table.

[0053] like Figure 4 As shown, the table block database in this embodiment includes pattern data and cell data. Specifically, the column name, data type and data value range in the data table are extracted; the column name, data type and the corresponding association with the data value range are set to form pattern data; the cell information and cell value in the data table are extracted, and the cell information is associated with the cell data to form cell data.

[0054] Among them, the data range in the data table is obtained by obtaining the data in the data table, specifically obtaining all the data in the data table and arranging them according to the data size to form a data queue; setting the minimum value in the data queue and the maximum value in the data queue to form the numerical range of the data.

[0055] For example, when extracting the original data table, extract the column name, data type and corresponding value range of the table, such as "mathematics, integer, minimum 58, maximum 138". Obtain the cell data in the data table, such as "mathematics, 76".

[0056] Furthermore, due to the duplication of cell data, when the amount of data is large, a score of 76 in mathematics may involve many students. At this time, in order to facilitate data search and statistics, it is set to obtain all cell data of the data table, and determine whether the number of cells in the data table exceeds the preset threshold; if the number of cells exceeds the preset threshold, the repetition frequency of the cell value is obtained; the cell values ​​with a repetition frequency exceeding the preset frequency are stored in the table block database, and other cell values ​​are deleted.

[0057] For example, when the preset frequency is set to 10 times, when the number of people with a score of 76 in math is 20, the cell data of 76 in math is retained and the cell data is directly stored in the table block data; when the number of people with a score of 76 in math is 4, the cell data of 76 in math is not retained and the cell data of other scores of 76 in math is deleted.

[0058] For example, the data table is a student score table as shown in Table 1. Based on the scheme in this embodiment, the data in Table 1 are converted into a question and answer database and a table block database respectively, where the question and answer database is the question and answer pairs generated by LLM based on the data table, and the table block database is the pattern data and cell data formed based on the data table.

[0059] Table 1 Student performance table (partial)

[0060]

[0061] Exemplarily, the constructed question-and-answer database content includes, "{query: What is the math score of student A?, answer: 77}", "{query: How many students are there in the senior year (ten) class? answer: 41}". The constructed table block database content, in which the pattern data includes "{column name: class, type: string, value example: senior year (one) class, senior year (ten) class}", "{column name: gender, type: BOOL, value example: male, female}", and the cell data includes "{class, senior year (one) class}", "{name, student A}".

[0062] Step 103: Receive question information raised by the user, and query the question-answer database according to the question information to obtain a first query result.

[0063] Among them, querying the question and answer database according to the question information to obtain the first query result specifically includes: obtaining feature information of the question information, querying the question and answer database in combination with the feature information to obtain question information matching the question information; querying answer information associated with the question information, and configuring the answer information associated with the question information as the first query result.

[0064] Among them, in order to ensure that the first query result obtained by the query is highly correlated with the question information, the corresponding database is queried through the following scheme to obtain the corresponding matching results, specifically including: setting keywords and semantic features for obtaining question information, combining keywords with question information in the question and answer database to calculate keyword similarity, combining semantic features with question information to calculate semantic similarity; performing weighted calculation on keyword similarity and semantic similarity to obtain comprehensive similarity; obtaining question information associated with the comprehensive similarity that meets preset rules as question information matching the question information.

[0065] For example, when searching for related questions in the question-answer database, a semantic search method is used to calculate the comprehensive similarity between the user question and the question in the question-answer database, and the top K values ​​with the highest comprehensive similarity are obtained as the query result. For example, K in this embodiment is 5, and the comprehensive similarity is calculated using formula (1):

[0066] (1)

[0067] in, Indicates comprehensive similarity; represents the weight, In this embodiment Take 0.2; Indicates keyword similarity; Indicates semantic similarity.

[0068] Furthermore, the keyword similarity adopts the BM25 algorithm, where the calculation formula (2) of the keyword similarity is:

[0069] (2)

[0070] in, and They respectively represent the user question information and the keyword representation of the question information in the question and answer database; in this embodiment, k1 is 1.2 and b is 0.8.

[0071] Furthermore, the semantic similarity adopts the KNN algorithm, and its calculation formula (3) is:

[0072] (3)

[0073] in, and They respectively represent the semantic vector representation of user question information and question information in the question and answer database; in this embodiment, the bge-large-zh-v1.5 model is adopted, and n is 1024.

[0074] Step 104: query the table block database according to the problem information to obtain a second query result.

[0075] like Figure 5 As shown, it specifically includes: identifying the query intent of the question information, setting the query intent that meets the preset conditions to query through the pattern data to obtain matching pattern data; setting the query intent that does not meet the preset conditions to query through the cell data to obtain matching cell data, and combining the matching pattern data and the matching cell data to form a second query result.

[0076] Exemplarily, the recognition model used is the LLM model, which uses the moonshot-v1-8k API. For example, the user's question information is "What is the average score of boys in the third grade (tenth) class in mathematics?" The pattern data obtained by the recognition model include: "class", "gender", "mathematics", and the recognized cell information and cell data are: "Class: Third Grade (Tenth) Class", "Gender: Male", "Mathematics, 77", "Mathematics, 78", "Mathematics, 140", etc.

[0077] In a specific embodiment, in order to prevent data reading omissions or confusion when the amount of data in the query results is large, the number of columns for obtaining matching pattern data and / or matching cell data is set; when the number of columns is set to be greater than or equal to two columns, the data in the matching pattern data and / or matching cell data are arranged in sequence according to the column numbers in the column information.

[0078] Step 105: merge the question information, the first query result, and the second query result to form input content, and input the input content into a preset language model to output a search result.

[0079] In order to ensure that the final search results match the information in the data table, it is set to obtain the background information of the data table before obtaining the first query result and the second query result, respond to the question information raised by the user, and confirm that the question information satisfies the background information; if the question information does not satisfy the background information, query and confirm that the question information corresponds to the data table.

[0080] Specifically, in this solution, the instruction template and input content are obtained, and the input content is used as the input of the LLM model to obtain the LLM generation result and return it to the user.

[0081] The instruction template is used to indicate answers based on user questions and retrieved data. The instruction template is implemented by calling a code execution tool, and a check module is configured in the instruction template. The check module is used to configure the logic of the search results before outputting the search results so that the expression in the search results conforms to the commonly used logical order.

[0082] In a specific embodiment, before outputting the search results, the number of answers in the search results is obtained. If the number of answers is greater than or equal to two, when setting the output search results, the corresponding numbers are set in sequence according to the output time sequence of the answers in the search results to improve the convenience of reading the search results.

[0083] In this embodiment, by explicitly converting the data table into a question and answer database and a table block database for storage and indexing, the space for storing long tables is effectively reduced. At the same time, the problem that the long table content cannot be input all at once due to the limitation of the large preset language model context window is solved, thereby improving the retrieval accuracy of the data table.

[0084] Embodiment 2

[0085] Corresponding to the above embodiment, the present application provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0086] Step 201: Obtain a data table, and construct a question-answer database and a table block database based on the data table;

[0087] Step 202: receiving question information raised by the user, querying the question-answer database according to the question information to obtain a first query result, and querying the table block database according to the question information to obtain a second query result;

[0088] Step 203: merge the question information, the first query result, and the second query result to form input content, and input the input content into a preset language model to output a search result.

[0089] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0090] A preset language model is used to generate question information based on a data table, and answer information is generated by combining the question information and the data table content associated with the question information; an association relationship between the question information and the answer information is constructed, wherein the question information and the answer information correspond one to one; and the association relationships between all question information and answer information are combined to form a question and answer database.

[0091] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0092] Obtain a preset context window size of a preset language model and obtain a data table size; if the table data size exceeds the preset context window size of the preset language model, split the data table into several sub-tables according to the row data of the data table, and set the sub-tables to have column information.

[0093] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0094] Confirm that the size of the sub-tables meets the preset context window size, and arrange the sub-tables in sequence according to the row numbers in the data table to form a table queue; in response to obtaining the answer information, input the sub-tables in the table queue into the preset language model in sequence.

[0095] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0096] Extract the column name, data type and data value range in the data table; set the column name, data type and corresponding association with the data value range to form pattern data; extract the cell information and cell value in the data table, set the association between the cell information and the cell data to form the cell data.

[0097] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0098] Get all the data in the data table and arrange them according to the data size to form a data queue; set the minimum value and the maximum value in the data queue to form the data value range.

[0099] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0100] Obtain all cell data of the data table and determine whether the number of cells in the data table exceeds a preset threshold; if the number of cells exceeds the preset threshold, obtain the repetition frequency of the cell values; store the cell values ​​whose repetition frequency exceeds the preset frequency in the table block database, and delete other cell values.

[0101] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0102] Acquire feature information of the question information, query a question-answer database in combination with the feature information to obtain question information matching the question information; query answer information associated with the question information, and configure the answer information associated with the question information as the first query result.

[0103] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0104] Obtain keywords and semantic features of question information, calculate keyword similarity by combining keywords with question information in a question-and-answer database, and calculate semantic similarity by combining semantic features with question information; perform weighted calculation on keyword similarity and semantic similarity to obtain comprehensive similarity; obtain question information associated with comprehensive similarity that meets preset rules as question information that matches the question information.

[0105] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0106] Identify the query intent of the question information, set the query intent that meets the preset conditions to query through pattern data to obtain matching pattern data; set the query intent that does not meet the preset conditions to query through cell data to obtain matching cell data, and combine the matching pattern data and the matching cell data to form a second query result.

[0107] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0108] Get the number of columns of the matched pattern data and / or matched cell data; when the number of columns is set to be greater than or equal to two columns, arrange the data in the matched pattern data and / or matched cell data in sequence according to the column numbers in the column information.

[0109] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0110] Acquire data table background information, respond to receiving question information raised by the user, and confirm that the question information satisfies the background information; if the question information does not satisfy the background information, query and confirm that the question information corresponds to the data table.

[0111] Embodiment 3

[0112] A computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0113] Step 301: Obtain a data table, and construct a question-answer database and a table block database based on the data table;

[0114] Step 302: Receive question information raised by the user, query the question-answer database according to the question information to obtain a first query result, and query the table block database according to the question information to obtain a second query result;

[0115] Step 303: merge the question information, the first query result, and the second query result to form input content, and input the input content into a preset language model to output a search result.

[0116] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0117] A preset language model is used to generate question information based on a data table, and answer information is generated by combining the question information and the data table content associated with the question information; an association relationship between the question information and the answer information is constructed, wherein the question information and the answer information correspond one to one; and the association relationships between all question information and answer information are combined to form a question and answer database.

[0118] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0119] Obtain a preset context window size of a preset language model and obtain a data table size; if the table data size exceeds the preset context window size of the preset language model, split the data table into several sub-tables according to the row data of the data table, and set the sub-tables to have column information.

[0120] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0121] Confirm that the size of the sub-tables meets the preset context window size, and arrange the sub-tables in sequence according to the row numbers in the data table to form a table queue; in response to obtaining the answer information, input the sub-tables in the table queue into the preset language model in sequence.

[0122] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0123] Extract the column name, data type and data value range in the data table; set the column name, data type and corresponding association with the data value range to form pattern data; extract the cell information and cell value in the data table, set the association between the cell information and the cell data to form the cell data.

[0124] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0125] Get all the data in the data table and arrange them according to the data size to form a data queue; set the minimum value and the maximum value in the data queue to form the data value range.

[0126] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0127] Obtain all cell data of the data table and determine whether the number of cells in the data table exceeds a preset threshold; if the number of cells exceeds the preset threshold, obtain the repetition frequency of the cell values; store the cell values ​​whose repetition frequency exceeds the preset frequency in the table block database, and delete other cell values.

[0128] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0129] Acquire feature information of the question information, query a question-answer database in combination with the feature information to obtain question information matching the question information; query answer information associated with the question information, and configure the answer information associated with the question information as the first query result.

[0130] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0131] Obtain keywords and semantic features of question information, calculate keyword similarity by combining keywords with question information in a question-and-answer database, and calculate semantic similarity by combining semantic features with question information; perform weighted calculation on keyword similarity and semantic similarity to obtain comprehensive similarity; obtain question information associated with comprehensive similarity that meets preset rules as question information that matches the question information.

[0132] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0133] Identify the query intent of the question information, set the query intent that meets the preset conditions to query through pattern data to obtain matching pattern data; set the query intent that does not meet the preset conditions to query through cell data to obtain matching cell data, and combine the matching pattern data and the matching cell data to form a second query result.

[0134] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0135] Get the number of columns of the matched pattern data and / or matched cell data; when the number of columns is set to be greater than or equal to two columns, arrange the data in the matched pattern data and / or matched cell data in sequence according to the column numbers in the column information.

[0136] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0137] Acquire data table background information, respond to receiving question information raised by the user, and confirm that the question information satisfies the background information; if the question information does not satisfy the background information, query and confirm that the question information corresponds to the data table.

[0138] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a question and answer database and a table block database. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a table content retrieval method is implemented.

[0139] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] Embodiment 4

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0142] Step 401: Obtain a data table, and construct a question-answer database and a table block database based on the data table;

[0143] Step 402: receiving question information raised by the user, querying the question-answer database according to the question information to obtain a first query result, and querying the table block database according to the question information to obtain a second query result;

[0144] Step 403: merge the question information, the first query result, and the second query result to form input content, and input the input content into a preset language model to output a search result.

[0145] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0146] A preset language model is used to generate question information based on a data table, and answer information is generated by combining the question information and the data table content associated with the question information; an association relationship between the question information and the answer information is constructed, wherein the question information and the answer information correspond one to one; and the association relationships between all question information and answer information are combined to form a question and answer database.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0148] Obtain a preset context window size of a preset language model and obtain a data table size; if the table data size exceeds the preset context window size of the preset language model, split the data table into several sub-tables according to the row data of the data table, and set the sub-tables to have column information.

[0149] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0150] Confirm that the size of the sub-tables meets the preset context window size, and arrange the sub-tables in sequence according to the row numbers in the data table to form a table queue; in response to obtaining the answer information, input the sub-tables in the table queue into the preset language model in sequence.

[0151] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0152] Extract the column name, data type and data value range in the data table; set the column name, data type and corresponding association with the data value range to form pattern data; extract the cell information and cell value in the data table, set the association between the cell information and the cell data to form the cell data.

[0153] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0154] Get all the data in the data table and arrange them according to the data size to form a data queue; set the minimum value and the maximum value in the data queue to form the data value range.

[0155] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0156] Obtain all cell data of the data table and determine whether the number of cells in the data table exceeds a preset threshold; if the number of cells exceeds the preset threshold, obtain the repetition frequency of the cell values; store the cell values ​​whose repetition frequency exceeds the preset frequency in the table block database, and delete other cell values.

[0157] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0158] Acquire feature information of the question information, query a question-answer database in combination with the feature information to obtain question information matching the question information; query answer information associated with the question information, and configure the answer information associated with the question information as the first query result.

[0159] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0160] Obtain keywords and semantic features of question information, calculate keyword similarity by combining keywords with question information in a question-and-answer database, and calculate semantic similarity by combining semantic features with question information; perform weighted calculation on keyword similarity and semantic similarity to obtain comprehensive similarity; obtain question information associated with comprehensive similarity that meets preset rules as question information that matches the question information.

[0161] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0162] Identify the query intent of the question information, set the query intent that meets the preset conditions to query through pattern data to obtain matching pattern data; set the query intent that does not meet the preset conditions to query through cell data to obtain matching cell data, and combine the matching pattern data and the matching cell data to form a second query result.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0164] Get the number of columns of the matched pattern data and / or matched cell data; when the number of columns is set to be greater than or equal to two columns, arrange the data in the matched pattern data and / or matched cell data in sequence according to the column numbers in the column information.

[0165] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0166] Acquire data table background information, respond to receiving question information raised by the user, and confirm that the question information satisfies the background information; if the question information does not satisfy the background information, query and confirm that the question information corresponds to the data table.

[0167] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0168] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0169] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A table content retrieval method, characterized in that: The method comprises: Obtain a data table, and construct a question-and-answer database and a table block database based on the data table; The question-answer database is a question-answer pair generated by the LLM model based on a data table. The table block database includes pattern data and cell data. The column name, data type and data value range in the data table are extracted; the column name, the data type and the data value range are set to correspond and associate to form the pattern data; the cell information and cell value in the data table are extracted, and the cell information is set to associate with the cell data to form the cell data; Receive question information raised by a user, and query the question-answer database according to the question information to obtain a first query result, specifically including: calculating the comprehensive similarity between the user question and the questions in the question-answer database by a semantic search method, and obtaining the top K values ​​with the highest comprehensive similarity as the query result; Querying the table block database according to the question information to obtain a second query result specifically includes: identifying the query intent of the question information, setting the query intent that meets the preset conditions to query through the pattern data to obtain matching pattern data; setting the query intent that does not meet the preset conditions to query through the cell data to obtain matching cell data, and combining the matching pattern data and the matching cell data to form the second query result; The question information, the first query result, and the second query result are combined to form input content, and the input content is input into a preset language model to output a search result.

2. The table content retrieval method according to claim 1, characterized in that: Constructing a question-answer database based on the data table specifically includes: Generate question information based on the data table using the preset language model, and generate answer information by combining the question information and the data table content associated with the question information; Establishing an association relationship between the question information and the answer information, wherein the question information corresponds to the answer information one by one; The association relationship between all the question information and the answer information is combined to form the question and answer database.

3. The table content retrieval method according to claim 2, characterized in that: The method further comprises: Obtaining a preset context window size of the preset language model, and obtaining a size of the data table; If the table data size exceeds the preset context window size of the preset language model, the data table is split into a plurality of sub-tables according to the row data of the data table, and the sub-tables are set to have column information.

4. The table content retrieval method according to claim 3, characterized in that: The method further comprises: Confirming that the size of the sub-tables meets the preset context window size, and arranging the sub-tables in sequence according to the row numbers in the data table to form a table queue; In response to obtaining the answer information, the sub-tables in the table queue are sequentially input into the preset language model.

5. The table content retrieval method according to claim 1, characterized in that: The method further comprises: Acquire all the data in the data table, and arrange them according to the data size to form a data queue; The minimum value in the data queue and the maximum value in the data queue are set to form the numerical range of the data.

6. The table content retrieval method according to claim 1, characterized in that: The method further comprises: Acquire all cell data of the data table, and determine whether the number of cells in the data table exceeds a preset threshold; If the number of cells exceeds a preset threshold, the repetition frequency of the cell value is obtained; The cell values ​​whose repetition frequency exceeds the preset frequency are stored in the table block database, and the other cell values ​​are deleted.

7. The table content retrieval method according to claim 2, characterized in that: Querying the question-answer database according to the question information to obtain a first query result specifically includes: Acquire characteristic information of the question information, and query the question-answer database in combination with the characteristic information to obtain the question information matching the question information; The answer information associated with the question information is queried, and the answer information associated with the question information is configured as the first query result.

8. The table content retrieval method according to claim 7, characterized in that: The method further comprises: Acquire keywords and semantic features of the question information, calculate keyword similarity by combining the keywords with the question information in the question-answer database, and calculate semantic similarity by combining the semantic features with the question information; Performing weighted calculation on the keyword similarity and the semantic similarity to obtain a comprehensive similarity; The question information associated with the comprehensive similarity that meets a preset rule is obtained as question information matched with the question information.

9. The table content retrieval method according to claim 1, characterized in that: The method further comprises: Obtaining the number of columns of the matched pattern data and / or the matched cell data; When the number of columns is set to be greater than or equal to two columns, the matched pattern data and / or the data in the matched cell data are arranged in sequence according to the column numbers in the column information.

10. The table content retrieval method according to claim 1 or 2, characterized in that: The method further comprises: Acquire the data table background information, in response to receiving question information raised by a user, and confirm that the question information satisfies the background information; If the question information does not satisfy the background information, query and confirm that the question information corresponds to the data table.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 10 are implemented.

12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images