Data table information acquisition method and device and computer equipment

Abstract

The invention provides a data table information acquisition method and device, computer equipment, a storage medium and a computer program product, and the method comprises the steps: receiving a to-be-questioned and answered data table and question input by a user, and inputting the data table and question into a pre-trained question and answer model, the question and answer model is obtained through training of a large language model, according to the method, row data information, column data information, corresponding header information, header information and table data content information of a data table can be obtained, knowledge enhancement is carried out on the table data content information based on the header information and the header information, and overall information representing table data content logic relevance is obtained. Target table information matched with the question is recognized from the data table based on the row data information, the column data information and the overall information, the target table information output by the question answering model is obtained, the information is displayed as question answering information, and the question answering accuracy for the table information can be improved by adopting the method.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to a method, device, computer device, storage medium, and computer program product for obtaining information of a data table. Background Art

[0002] In the logistics field, 40% of the data is a large amount of tabular data such as excel and csv. The title and header in the tabular data are the keys to understanding its content. Since the title and header may be in different rows, columns, or even cells, the structure is very diverse. At present, a large amount of manual processing and analysis is required for complex tabular data.

[0003] For complex structured data, there are various formats, and a large amount of manpower is required for statistical analysis, resulting in problems such as low efficiency, high error rate, and non-uniformity. For simple structures, important information such as headers and titles cannot be intelligently identified and extracted, and human participation is required. Therefore, the current technology cannot achieve question and answer for tabular information, and the accuracy of the question and answer is not high. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for obtaining information of a data table.

[0005] In a first aspect, this application provides a method for obtaining information of a data table, including:

[0006] Receiving a data table and a question to be answered input by a user;

[0007] Inputting the data table and the question into a pre-trained question and answer model; the pre-trained question and answer model is trained based on a large language model and is used to obtain row data information and column data information of the input data table, as well as header information, title information, and table data content information corresponding to the data table, and enhance the table data content information based on the header information and title information to obtain overall information representing the logical relevance between the table data contents, and then based on the row data information, column data information, and overall information, identifying target table information adapted to the question from the data table;

[0008] Obtaining the target table information output by the question and answer model, and presenting the target table information as a reply message to the question.

[0009] In one of the embodiments, after inputting the data table and the question into the pre-trained question and answer model, the pre-trained question and answer model is further used to: convert the data table into corresponding row text data information and column text data information, and remove abnormal text data information therein according to rows and columns to obtain row data information and column data information of the data table;

[0010] Cluster the row data information and column data information of the data table, and obtain the corresponding header information, title information, and table data content information based on the clustering results.

[0011] In one embodiment, clustering the row data information and column data information of the data table, and obtaining the corresponding header information, title information, and table data content information based on the clustering results, including:

[0012] Convert the row data information and column data information of the data table into corresponding row vectors and column vectors respectively;

[0013] Cluster based on the row vectors and column vectors, and obtain the corresponding header information, title information, and table data content information of the data table based on the clustering results.

[0014] In an exemplary embodiment, clustering based on the row vectors and column vectors, and obtaining the corresponding header information, title information, and table data content information of the data table, including:

[0015] Use the k-means algorithm to cluster the row vectors and column vectors, and obtain the corresponding header information, title information, and table data content information of the data table according to the clustering results.

[0016] In one embodiment, the question-and-answer model is trained through the following steps:

[0017] Obtain the sample data table and sample questions input by the user;

[0018] Convert the sample data table into corresponding row text data information and column text data information, and remove the abnormal text data information therein row by row and column by column to obtain the row data information and column data information of the sample data table;

[0019] Classify based on the row data information and column data information of the sample data table through a clustering model to obtain the corresponding header information, title information, and table data content information of the sample data table;

[0020] Input the corresponding header information, title information, and table data content information of the sample data table into the machine learning model to be trained for knowledge enhancement to obtain the overall information representing the logical relevance between the table data contents, and then based on the row data information, column data information, and overall information corresponding to the sample data table, identify the target sample table information adapted to the sample questions from the sample data table;

[0021] Based on the target sample table information and the reference answer information preset for the sample questions, the clustering model, machine learning model, and large language model are correspondingly adjusted, and a question-answering model is obtained based on the clustering model, machine learning model, and large language model at the end of training.

[0022] In an exemplary embodiment, the question-answering model is trained through the following steps:

[0023] Receive the sample data table and sample questions input by the user as question training data;

[0024] Obtain the reference answer information set by the user for the sample questions as answer training data;

[0025] Input the question training data into the large language model to be trained, and predict the answer data information adapted to the sample questions through the large language model;

[0026] According to the predicted answer data information and answer training data, correspondingly adjust the large language model, and obtain a question-answering model based on the large language model at the end of training.

[0027] In a second aspect, the present application also provides an information acquisition device for a data table, including:

[0028] A receiving module for receiving the data table and questions to be answered input by the user;

[0029] An input module for inputting the data table and questions into a pre-trained question-answering model; the pre-trained question-answering model is trained based on a large language model and is used to obtain the row data information and column data information of the input data table, as well as the header information, title information, and table data content information corresponding to the data table, and perform knowledge enhancement on the table data content information based on the header information and title information to obtain the overall information representing the logical relevance between the table data contents, and then based on the row data information, column data information, and overall information, identify the target table information adapted to the questions from the data table;

[0030] A display module for obtaining the target table information output by the question-answering model and displaying the target table information as the reply information to the questions.

[0031] In a third aspect, the present application also provides a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] Receive the data table and questions to be answered input by the user;

[0033] Input a data table and a question into a pre-trained question-answering model; the pre-trained question-answering model is trained based on a large language model and is used to obtain the row data information and column data information of the input data table, as well as the header information, title information, and table data content information corresponding to the data table, and perform knowledge enhancement on the table data content information based on the header information and title information to obtain the overall information representing the logical relevance between the table data contents, and then based on the row data information, column data information, and overall information, identify the target table information suitable for the question from the data table;

[0034] Obtain the target table information output by the question-answering model and display the target table information as the answer information to the question.

[0035] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0036] Receive the data table and question to be answered input by the user;

[0037] Input the data table and the question into a pre-trained question-answering model; the pre-trained question-answering model is trained based on a large language model and is used to obtain the row data information and column data information of the input data table, as well as the header information, title information, and table data content information corresponding to the data table, and perform knowledge enhancement on the table data content information based on the header information and title information to obtain the overall information representing the logical relevance between the table data contents, and then based on the row data information, column data information, and overall information, identify the target table information suitable for the question from the data table;

[0038] Obtain the target table information output by the question-answering model and display the target table information as the answer information to the question.

[0039] In a fifth aspect, the present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0040] Receive the data table and question to be answered input by the user;

[0041] Input the data table and the question into a pre-trained question-answering model; the pre-trained question-answering model is trained based on a large language model and is used to obtain the row data information and column data information of the input data table, as well as the header information, title information, and table data content information corresponding to the data table, and perform knowledge enhancement on the table data content information based on the header information and title information to obtain the overall information representing the logical relevance between the table data contents, and then based on the row data information, column data information, and overall information, identify the target table information suitable for the question from the data table;

[0042] Obtain the target table information output by the question-and-answer model, and display the target table information as the reply information to the question.

[0043] The above-mentioned method, device, computer device, storage medium and computer program product for obtaining information of a data table receive a data table and a question to be answered input by a user, and input the data table and the question into a pre-trained question-and-answer model. This question-and-answer model is trained by a large language model, and it can obtain the row data information and column data information of the data table, as well as the corresponding header information, title information and table data content information. And based on the header information, title and information, knowledge enhancement is performed on the table data content information to obtain the overall information representing the logical relevance between the table data contents. Furthermore, based on the row data information, column data information and overall information, the target table information suitable for the question is identified from the data table. Based on this, further, the target table information output by the question-and-answer model can be obtained, and this information is displayed as the reply information to the question. Through a series of processing of the data table to be answered by the pre-trained question-and-answer model, it helps the server better understand the table content of the data table, and based on this, output the reply information corresponding to the question, improving the accuracy and efficiency of the question and answer. Description of the Drawings

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0045] Figure 1 It is an application environment diagram of the method for obtaining information of a data table in an embodiment;

[0046] Figure 2 It is a schematic flowchart of the method for obtaining information of a data table in an embodiment;

[0047] Figure 3 It is a schematic flowchart of the training steps of a question-and-answer model in an embodiment;

[0048] Figure 4 It is a schematic flowchart of the training steps of a question-and-answer model in another embodiment;

[0049] Figure 5 It is a schematic flowchart of the method for obtaining information of a data table in another embodiment;

[0050] Figure 6 It is a structural block diagram of the device for obtaining information of a data table in an embodiment;

[0051] Figure 7 It is the internal structure diagram of a computer device in an embodiment. Specific implementation manners

[0052] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] The method for obtaining information of a data table provided in an embodiment of the present application can be applied to, for example Figure 1 the application environment shown in the figure. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers. The server 104 receives the data table and question to be answered input by the user through the terminal 102, and then inputs the data table and question into a pre-trained question and answer model. Through this question and answer model, the row data information and column data information of the data table, as well as the corresponding header information, title information and table data content information are obtained. Then, based on the header information and title information, knowledge enhancement is performed on the table data content to obtain the overall information representing the logical relevance between the table data contents. Further, according to the row data information, column data information and overall information, the target table information suitable for the question is identified from the data table. The server 104 obtains the target table information output by the question and answer model, and displays this target table information to the terminal 102 as the answer information for the question and answer. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0054] In an exemplary embodiment, as Figure 2 shown in the figure, a method for obtaining information of a data table is provided. Taking the method applied to Figure 1 the server 104 in the figure as an example, it includes the following steps S210 to step S220. Among them:

[0055] Step S210, receiving the data table and question to be answered input by the user.

[0056] Among them, the data table to be questioned and answered can be understood as a table from which users need to find relevant information. It can be known that the table is in the form of excel and csv, and there is a large amount of data content in it. Manual search is time-consuming and laborious, and the accuracy of the searched content is not high.

[0057] Optionally, the user inputs the data table and related questions that they want to question and answer through the terminal. The server 104 then obtains this information input by the user through the communication network and continues with the next step of processing. The server specifically obtains the data table that needs to be questioned and answered instead of receiving all data tables. For example, for a table in the logistics field, it only receives data table information related to the logistics field and uses this as an answer library for question and answer. Specifically receiving information is beneficial to improving the accuracy of question and answer and reducing the cost of resource acquisition.

[0058] Step S220: Input the data table and the question into a pre-trained question and answer model; the pre-trained question and answer model is trained based on a large language model and is used to obtain the row data information and column data information of the input data table, as well as the header information, title information, and table data content information corresponding to the data table. And based on the header information and title information, knowledge enhancement is performed on the table data content information to obtain the overall information representing the logical relevance between the table data contents. Furthermore, based on the row data information, column data information, and overall information, the target table information adapted to the question is identified from the data table.

[0059] Among them, the pre-trained question and answer model can be understood as a large language model whose weights have been adaptively adjusted. The large language model can be used to screen out the target data information required and adapted to the question from a large amount of data information, which is equivalent to an answer library that can automatically match answers. As for the header information, title information, and data table content information corresponding to the data table, the specific form can be as shown in Table 1 below:

[0060] Table 1: Seafood Product Transportation Table

[0061]

[0062] It can be known that the header information in this table is seafood products, the title information includes squid, lobster, oyster, and scallop, purchase price, transportation price, etc., and the table data content information includes A1...G4, etc.

[0063] Knowledge enhancement can be understood as, due to problems with structured data, the need to utilize common sense or domain knowledge, which often do not appear in tabular data. Additionally, the data descriptions in the table are often missing. To solve this problem, it is necessary to perform knowledge enhancement on the table, such as identifying and establishing the entity types of the columns in the table (such as people, locations, units, metrics, etc.), the relationships between column entities, and their semantic roles in language, and enabling the algorithm to use this knowledge to understand the user's question, eliminate semantic ambiguity, and improve the understanding accuracy.

[0064] Exemplarily, after inputting the data table and the question into a pre-trained question answering model, the question answering model first obtains the row data information and column data information of the data table, as well as the header, title information, and table data content information corresponding to the data table. At this time, the logical relevance between the header, title information and the table data content information is relatively poor. Therefore, further knowledge enhancement is performed on the table data content information based on the header and title information, and the overall information representing the logical relevance between the table data contents is obtained. Then, based on the row data information, column data information, and this overall information, the target table information matching the question is identified from the data table.

[0065] By preprocessing the content information contained in the data table through the above means, the logical relevance between the header, title and the table data content is enhanced, and important information such as the header and title is intelligently identified and extracted to help the server better understand the content of the data table, which is beneficial to improving the accuracy of question answering and reducing the cost of manual operations at the same time.

[0066] Step S230, obtain the target table information output by the question answering model, and display the target table information as the reply information to the question.

[0067] As an implementation manner, the server can provide the target table information to the terminal, and the relevant information is displayed as the reply information to the user's input question through the terminal.

[0068] In the method for obtaining information from the above data table, the data table to be questioned and the question input by the user are received, and the data table and the question are input into a pre-trained question-and-answer model. This question-and-answer model is trained by a large language model and can obtain the row data information and column data information of the data table, as well as the corresponding header information, title information, and table data content information. Based on the header information, title, and information, knowledge enhancement is performed on the table data content information to obtain the overall information representing the logical relevance between the table data contents. Furthermore, based on the row data information, column data information, and overall information, the target table information suitable for the question is identified from the data table. Based on this, further, the target table information output by the question-and-answer model can be obtained and presented as the answer information to the question. By performing a series of processes on the data table to be questioned through the pre-trained question-and-answer model, it helps the server better understand the table content of the data table, and based on this, the answer information corresponding to the question is output, improving the accuracy and efficiency of the question and answer.

[0069] In one embodiment, after inputting the data table and the question into the pre-trained question-and-answer model, the pre-trained question-and-answer model is further configured to:

[0070] Convert the data table into corresponding row text data information and column text data information, and remove the abnormal text data information therein according to rows and columns to obtain the row data information and column data information of the data table; perform clustering on the row data information and column data information of the data table, and obtain the corresponding header information, title information, and table data content information based on the clustering result.

[0071] Among them, the text data information can be understood as removing the table in the original table data and only retaining the content information inside the table. The abnormal text data information can be understood as spaces or some content information that does not conform to the title, as well as some duplicate text data with different row titles but exactly the same row data, etc. For example, for two completely different row titles of height and age, the corresponding data content is exactly the same. At this time, the text data is abnormal and the incorrect data needs to be deleted.

[0072] Optionally, first convert the data table into row text data information and column text data information that are easy to process content information, and then use rows and columns as inspection units to check one by one whether there are abnormal row or column data in the row and column text data information. Here, the abnormality may be that the data value is abnormal or the title and the corresponding data information are abnormal, that is, the entire row or column shows an unreasonable similarity. Once such a situation is found, the row or column needs to be cleared immediately. Perform clustering on the row data information and column data information after clearing, and extract the header information, title information, and table data content information corresponding to the table from the information.

[0073] By converting data tables into easy-to-process text data and removing duplicate and abnormal data according to rows and columns, the accuracy of question and answer is further improved. After that, the processed data information is clustered to obtain table headers, titles, and table data content information, laying a data foundation for the subsequent use of table headers and title information to enhance the knowledge of table data content information.

[0074] In one embodiment, clustering is performed on row data information and column data information of a data table, and corresponding header information, title information, and table data content information are obtained based on the clustering result, including:

[0075] Convert the row data information and column data information of the data table into corresponding row vectors and column vectors respectively;

[0076] Clustering is performed based on row vectors and column vectors, and header information, title information, and table data content information corresponding to the data table are obtained based on the clustering results.

[0077] Among them, the row vector can be understood as data with a horizontal direction, and the column vector can be understood as data with a vertical direction. Data with a direction is not easily confused when clustering.

[0078] For example, the row data information is converted into a data vector with a horizontal direction, and the column data information is converted into a data vector with a vertical direction. These row vectors and column vectors are clustered, and the header information, title information and table data content information corresponding to the data table can be obtained based on the clustering results. Because rows and columns may have titles, and sometimes there are multi-level titles, these titles may have similar descriptions, so converting them into vectors and then clustering them can help the server better understand the title content, thereby improving the accuracy of questions and answers.

[0079] In one embodiment, clustering is performed based on row vectors and column vectors, and header information, title information, and table data content information corresponding to the data table are obtained based on the clustering results, including: clustering row vectors and column vectors using a k-means algorithm, and obtaining header information, title information, and table data content information corresponding to the data table based on the clustering results.

[0080] The main function of the k-means algorithm is to cluster data and divide the data points in the data set into k different categories. It iteratively assigns data points to k categories so that the distance between each data point and the center point of its category (i.e., the cluster center) is minimized.

[0081] Optionally, it is first divided into k categories. By iterating multiple row vectors and column vectors through the k-means algorithm, they are assigned to the corresponding categories. It can be understood that the k categories mentioned here can refer to table headers and title information. Generally, the table header information is less than the title information. The object of classification is mainly the table data content. Finally, the table header information, title information, and table data content information corresponding to the data table can be obtained.

[0082] By using the k-means algorithm to classify and group the data, it helps the server better understand the internal results in the data and is beneficial to improving the accuracy of question answering.

[0083] In an exemplary embodiment, as Figure 3 shown, the question answering model is trained through the following steps, including step S310 to step S350, where:

[0084] Step S310, obtain the sample data table and sample question input by the user.

[0085] Among them, the sample data table can be understood as similar data tables in the same field, and the sample question can be understood as a question that already has reference answer information.

[0086] Step S320, convert the sample data table into corresponding row text data information and column text data information, and remove the abnormal text data information therein according to rows and columns to obtain the row data information and column data information of the sample data table.

[0087] Step S330, based on the row data information and column data information of the sample data table, classify through a clustering model to obtain the table header information, title information, and table data content information corresponding to the sample data table.

[0088] Among them, the clustering model can be understood as a machine learning model used to divide the data points in a data set into several different groups or categories, so that the data points within the same group have a relatively high similarity, while the data points between different groups have a relatively low similarity. The goal of this model is to automatically group the data points according to their similarity without knowing the categories to which the data points belong. In the clustering model, some algorithms (such as k-means, hierarchical clustering, DBSCAN, etc.) are usually used to cluster the data points. These algorithms determine the grouping method of the data points according to the distance or similarity between the data points, and appropriate algorithms and parameters can be selected according to different clustering requirements. This application discloses but is not limited to using the k-means algorithm.

[0089] Exemplarily, according to the row data information and column data information of the sample data table, it is input into a clustering model for classification, and the header information, title information, and table data content information corresponding to the sample data table are output. In this way, the data information in the sample data table can be processed, improving the speed of data processing.

[0090] Step S340: Input the header information, title information, and table data content information corresponding to the sample data table into a machine learning model to be trained for knowledge enhancement, obtain the overall information used to characterize the logical relevance between the table data contents, and then identify the target sample table information adapted to the sample problem from the sample data table based on the row data information, column data information, and overall information corresponding to the sample data table.

[0091] Among them, the machine learning model is, but not limited to, bert (Bidirectional Encoder Representations from Transformers) in NLP (Natural Language Processing) disclosed in this application.

[0092] Step S350: Based on the target sample table information and the reference answer information preset for the sample problem, correspondingly adjust the clustering model, machine learning model, and large language model, and obtain a question-answering model based on the clustering model, machine learning model, and large language model at the end of training.

[0093] Optionally, according to the differences between the target sample table information and the reference answer information preset for the sample, correspondingly adjust the clustering model, machine learning model, and large language model, and obtain a question-answering model based on the clustering model, machine learning model, and large language model after training. This method of comprehensively using different models can improve the robustness and performance of the system, and further improve the question-answering accuracy for table information.

[0094] In one embodiment, as Figure 4 shown, the question-answering model is trained through the following steps, including steps S410 to S440, where:

[0095] Step S410: Receive the sample data table and sample problem input by the user as question training data.

[0096] Step S420: Obtain the reference answer information set by the user for the sample problem as answer training data.

[0097] Step S430: Input the question training data into the large language model to be trained, and predict the answer data information adapted to the sample problem through the large language model.

[0098] Step S440: Adjust the large language model according to the predicted answer data information and the answer training data, and obtain a question-answering model based on the large language model at the end of training.

[0099] In this embodiment, the received sample data table and sample questions input by the user are used as answer training data, and the reference answer information set by the user for the sample questions is obtained as answer training data. The question training data is input into the large language model to be trained, and the large language model predicts the answer data information suitable for the sample questions. According to the predicted answer data information and the answer training data, the large language model is adjusted correspondingly, and the final question-answering model can be obtained. This method directly adjusts the large language model itself to obtain the question-answering model, reducing the processing steps and improving the efficiency of model construction.

[0100] In an exemplary embodiment, as Figure 5 shown, a method for obtaining information of a data table is implemented. The detailed steps include step S510 to step S570, where:

[0101] Step S510: The server receives the data table and questions to be answered input by the user.

[0102] Step S520: Convert the data table into corresponding row text data information and column text data information, and remove the abnormal text data information therein according to rows and columns to obtain the row data information and column data information of the data table.

[0103] Step S530: Convert the row data information and column data information of the data table into corresponding row vectors and column vectors.

[0104] Step S540: Use the k-means algorithm to cluster the row vectors and column vectors, and obtain the header information, title information, and table data content information corresponding to the data table according to the clustering results.

[0105] Step S550: Perform knowledge enhancement on the table data content information based on the header information and title information to obtain the overall information representing the logical relevance between the table data contents.

[0106] Step S560: Identify the target table information suitable for the question from the data table according to the row data information and column data information obtained in step S520 and the overall information obtained in step S550.

[0107] Step S570: Obtain the target table information output by the question-answering model, and display this target table information as the reply information to the question.

[0108] Compared with the existing technical solutions, the present application has the following advantages:

[0109] 1) Pre-process the content of the data table for Q&A in advance, which improves the quality and effectiveness of the data table content and enhances the accuracy of the answers to the questions input by users.

[0110] 2) The Q&A model constructed jointly by the clustering model, machine learning model and large language model avoids manually statistical analysis of the data table, reduces the error rate, improves the efficiency, ensures the unity of the data, and is conducive to the server's better understanding of the logical relationship between data contents, thereby improving the accuracy of Q&A.

[0111] 3) The Q&A model directly trained by the large language model shortens the process of data table processing, improves the reply efficiency of Q&A, and also improves the accuracy of Q&A for table information.

[0112] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0113] Based on the same inventive concept, the embodiments of the present application also provide an information acquisition device for a data table for implementing the information acquisition method of the data table involved above. The implementation solutions provided by this device to solve problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more embodiments of the information acquisition device for the data table provided below can refer to the limitations on the information acquisition method of the data table in the above text, and will not be repeated here.

[0114] In an exemplary embodiment, as Figure 6 shown, an information acquisition device for a data table is provided, including: a receiving module 610, an input module 620, and a display module 630, where:

[0115] The receiving module 610 is configured to receive the data table and questions to be answered input by the user.

[0116] An input module 620 for inputting a data table and questions into a pre-trained question-answering model; the pre-trained question-answering model is trained based on a large language model and is used to obtain row data information and column data information of the input data table, as well as header information, title information, and table data content information corresponding to the data table, and perform knowledge enhancement on the table data content information based on the header information and title information to obtain overall information representing the logical relevance between the table data contents, and then identify target table information adapted to the questions from the data table based on the row data information, column data information, and overall information.

[0117] A display module 630 for obtaining the target table information output by the question-answering model and displaying the target table information as a reply message to the question.

[0118] In one embodiment, the device further includes a data processing module and a data clustering module, where:

[0119] The data processing module is used to convert the data table into corresponding row text data information and column text data information, and remove abnormal text data information therein according to rows and columns to obtain the row data information and column data information of the data table.

[0120] The data clustering module is used to perform clustering according to the row vectors and the column vectors, and obtain the header information, title information, and table data content information corresponding to the data table based on the clustering result.

[0121] In an exemplary embodiment, the data clustering module further includes a vector conversion module and a vector clustering module, where:

[0122] The vector conversion module is used to convert the row data information and column data information of the data table into corresponding row vectors and column vectors respectively.

[0123] The vector clustering module is used to perform clustering based on the row vectors and column vectors, and obtain the header information, title information, and table data content information corresponding to the data table based on the clustering result.

[0124] In one embodiment, the vector clustering module is further specifically used to perform clustering according to the row vectors and column vectors, and obtain the header information, title information, and table data content information corresponding to the data table based on the clustering result.

[0125] In one embodiment, the question-and-answer model in the device is obtained through the pre-training unit 1. The pre-training unit 1 is used to obtain the sample data table and the sample question input by the user; convert the sample data table into the corresponding row text data information and column text data information, and remove the abnormal text data information therein according to rows and columns to obtain the row data information and column data information of the sample data table; classify through a clustering model based on the row data information and column data information of the sample data table to obtain the header information, title information, and table data content information corresponding to the sample data table; input the header information, title information, and table data content information corresponding to the sample data table into the machine learning model to be trained for knowledge enhancement to obtain the overall information characterizing the logical relevance between the table data contents, and then based on the row data information, column data information, and overall information corresponding to the sample data table, identify the target sample table information adapted to the sample question from the sample data table; based on the target sample table information and the reference answer information preset for the sample question, correspondingly adjust the clustering model, machine learning model, and large language model, and obtain the question-and-answer model based on the clustering model, machine learning model, and large language model at the end of training.

[0126] In one embodiment, the question-and-answer model in the device is obtained through the pre-training unit 2. The pre-training unit 2 is used to receive the sample data table and the sample question input by the user, obtain the reference answer information set by the user for the sample question as the answer training data, and use it as the question training data; input the question training data into the large language model to be trained, and predict the answer data information adapted to the sample question through the large language model; correspondingly adjust the large language model according to the predicted answer data information and the answer training data, and obtain the question-and-answer model based on the large language model at the end of training.

[0127] Each module in the above information acquisition device of the data table can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or be independent of it, or be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0128] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 7As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store tabular data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a method for obtaining information of a data table.

[0129] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0130] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements the method for obtaining information of the data table in the above embodiment.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, it implements the method for obtaining information of the data table in the above embodiment.

[0132] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, it implements the method for obtaining information of the data table in the above embodiment.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0134] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0135] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0136] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for obtaining information of a data table, characterized in that The method includes: Receiving a data table and a question to be answered input by a user; Inputting the data table and the question into a pre-trained question-answering model; the pre-trained question-answering model is trained based on a large language model and is used to obtain the row data information and column data information of the input data table, as well as the header information, title information, and table data content information corresponding to the data table, and perform knowledge enhancement on the table data content information based on the header information and title information to obtain the overall information representing the logical relevance between the table data contents, and then based on the row data information, column data information, and overall information, identifying target table information adapted to the question from the data table; Obtaining the target table information output by the question-answering model and presenting the target table information as the answer information to the question.

2. The method according to claim 1, characterized in that, After inputting the data table and the question into the pre-trained question-answering model, the pre-trained question-answering model is further used to: convert the data table into corresponding row text data information and column text data information, and remove abnormal text data information therein row by row and column by column to obtain the row data information and column data information of the data table; Clustering the row data information and column data information of the data table and obtaining corresponding header information, title information, and table data content information based on the clustering result.

3. The method according to claim 2, wherein The clustering of the row data information and column data information of the data table and obtaining corresponding header information, title information, and table data content information based on the clustering result includes: Converting the row data information and column data information of the data table into corresponding row vectors and column vectors respectively; Performing clustering based on the row vectors and the column vectors and obtaining the header information, title information, and table data content information corresponding to the data table based on the clustering result.

4. The method according to claim 3, wherein The performing clustering based on the row vectors and the column vectors and obtaining the header information, title information, and table data content information corresponding to the data table based on the clustering result includes: Using the k-means algorithm to cluster the row vectors and the column vectors and obtaining the header information, title information, and table data content information corresponding to the data table according to the clustering result.

5. The method according to claim 1, characterized in that, The question-answering model is trained through the following steps: Obtaining a sample data table and a sample question input by a user; Converting the sample data table into corresponding row text data information and column text data information, and removing abnormal text data information therein row by row and column by column to obtain the row data information and column data information of the sample data table; Classifying based on the row data information and column data information of the sample data table through a clustering model and obtaining the header information, title information, and table data content information corresponding to the sample data table; Input the header information, title information, and table data content information corresponding to the sample data table into the machine learning model to be trained for knowledge enhancement, obtain the overall information characterizing the logical relevance between the table data contents, and then identify the target sample table information adapted to the sample question from the sample data table based on the row data information, column data information, and overall information corresponding to the sample data table; Based on the target sample table information and the reference answer information preset for the sample question, correspondingly adjust the clustering model, the machine learning model, and the large language model, and obtain the question and answer model based on the clustering model, machine learning model, and large language model at the end of training.

6. The method according to claim 1, characterized in that The question and answer model is trained through the following steps: Receive the sample data table and sample question input by the user as question training data; Obtain the reference answer information set by the user for the sample question as answer training data; Input the question training data into the large language model to be trained, and predict the answer data information adapted to the sample question through the large language model; Correspondingly adjust the large language model according to the predicted answer data information and the answer training data, and obtain the question and answer model based on the large language model at the end of training.

7. An information acquisition device for a data table, characterized in that, The device includes: A receiving module, configured to receive the data table and question to be answered input by the user; An input module, configured to input the data table and question into a pre-trained question and answer model; the pre-trained question and answer model is trained based on a large language model, and is configured to obtain the row data information and column data information of the input data table, as well as the header information, title information, and table data content information corresponding to the data table, and perform knowledge enhancement on the table data content information based on the header information and title information to obtain the overall information characterizing the logical relevance between the table data contents, and then identify the target table information adapted to the question from the data table based on the row data information, column data information, and overall information; A display module, configured to obtain the target table information output by the question and answer model and display the target table information as the reply information to the question.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

Images