Training method, device, medium and equipment for large language model of tabular data processing

Abstract

The present invention relates to a training method, device, equipment and medium for a large language model for processing tabular data, which is used to solve the problem of low accuracy of large language models in the prior art for processing tabular data. The method comprises: obtaining tabular question and answer data, which comprises a table, a question and an answer; inputting the table and the question into an intelligent question and answer model to generate an extended answer to the answer, and forming extended training data based on the table, the question and the extended answer; generating a new question based on the table, inputting the table and the new question into the intelligent question and answer model, and generating a first answer and a second answer based on different methods; comparing the first answer and the second answer for cross-validation, and if the cross-validation is passed, generating training data based on the table, the new question and the first answer and / or the second answer; and training a pre-trained model based on a mixed training set formed by the generated training data and the extended training data to obtain a large language model for processing tabular data.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a training method, device, medium and equipment for a large language model for processing tabular data. Background Art

[0002] With the advent of the digital age, a large amount of data is structured in tabular form and is widely used in various industries and fields. From transaction monitoring in the banking industry to human resource management, to census in government agencies, tabular data plays an irreplaceable role in modern society. Although tabular data is ubiquitous, specific tasks related to tables are usually time-consuming and labor-intensive and require professional skills. Automating these tabular tasks is of great significance to both academia and industry, and has attracted the attention of many researchers.

[0003] Traditional methods for processing tabular data mainly focus on adjusting the language model architecture, combining elements such as position embedding and attention mechanisms to encode the inherent structural properties of tabular data. However, these methods usually require a lot of manpower and time investment and are prone to errors. With the rise of large language models such as GPT-3.5 and GPT-4, researchers have begun to explore more efficient and intelligent methods for processing tabular data. The latest research emphasizes making precise prompts, integrating key information of tabular data, and using external programming languages ​​for reasoning. However, existing research often fails to reflect the actual needs in real office scenarios, resulting in limitations in existing methods when processing different types of tabular data.

[0004] According to the survey, office workers have clear preferences for tasks such as table Q&A, table revision, chart creation, and table matching. Excel / CSV and Word / PDF formats and long tables are the first choice of participants. The survey results show that there are two major challenges in processing table data in actual offices: (1) Diverse operational requirements, including query, update, merge, and drawing, go beyond traditional table Q&A tasks; (2) Different formats of table data require different processing methods, such as Word / PDF documents often contain mixed queries of text and table data, while Excel / CSV spreadsheets often contain complex structures such as long tables.

[0005] Based on this, the present invention proposes a large language model training method for tabular data processing to improve the accuracy of the tabular data processing language model. Summary of the invention

[0006] In view of the above analysis, the embodiments of the present invention aim to provide a method, apparatus, medium and device for training a large language model for processing tabular data, so as to solve the problem of low accuracy of large language models in the prior art for processing tabular data.

[0007] The present invention provides a method for training a large language model for table data processing, comprising the following steps:

[0008] Acquire form question and answer data, wherein the form question and answer data includes a form, questions and answers;

[0009] Inputting the table and question into the intelligent question-answering model to generate an extended answer to the answer, and forming extended training data based on the table, question and extended answer;

[0010] Generate a new question based on the table, input the table and the new question into the intelligent question-answering model, and generate a first answer and a second answer based on different methods;

[0011] Compare the first answer and the second answer for cross-validation, and if the cross-validation passes, generate training data based on the table, the new question, and the first answer and / or the second answer;

[0012] The pre-training model is trained based on a mixed training set formed by generating training data and expanding training data to obtain the large language model for processing tabular data, and the large language model for processing tabular data is used to generate answers based on tables and questions.

[0013] In some embodiments, the tabular question and answer data is derived from one or more of the WikiTQ, FeTaQA, or TAT-QA datasets;

[0014] The tabular question and answer data also comes from WikiSQL and / or Spider datasets.

[0015] In some embodiments, the intelligent question answering model includes one or more of GPT3.5, GPT4, and deepseek-coder-33B.

[0016] In some embodiments, the table and the question are input into an intelligent question-answering model to generate an expanded answer to the answer, including:

[0017] Determine the type of the table;

[0018] If the type is a document embedded table, a process expansion answer having a text query and a description of the reasoning process is generated by the intelligent question-answering model;

[0019] If the type is a spreadsheet, a code expansion answer is generated by the intelligent question and answer model, and the code expansion answer includes a table operation code that can generate the answer.

[0020] In some embodiments, the table and the new question are input into the intelligent question-answering model, and the first answer and the second answer are generated based on different methods, including:

[0021] Determine the type of the table;

[0022] If the type is a document embedded table, based on the same new question, the intelligent question-answering model generates a first answer with a text query and a description of the reasoning process and a description of the result through different prompt words, and generates a second answer including a table operation code;

[0023] If the type is a spreadsheet, the intelligent question-answering model generates a first answer including a first table operation code and a second answer including a second table operation code based on the new question.

[0024] In some embodiments, the first answer and the second answer are compared for cross-validation, and if the cross-validation is passed, training data is generated based on the table, the new question, and the first answer and / or the second answer, including:

[0025] Determine the type of the table;

[0026] If the type is a document embedded table, and if the result description of the first answer is consistent with the running result of the table operation code of the second answer, then through cross-validation, first generated training data is formed based on the table, the new question and the first answer, and second generated training data is formed based on the table, the new question and the second answer. If they are inconsistent, cross-validation is not passed and the table and the new question are discarded;

[0027] If the type is a spreadsheet, and if the running results of the spreadsheet operation codes of the first answer and the second answer are consistent, the generated training data is formed based on the spreadsheet, the new question and one of the first answer or the second answer through cross-validation.

[0028] In some embodiments, when the pre-training model is trained based on the generated training data and the expanded training data to form a mixed training set to obtain the large language model for processing the tabular data, the loss function of the training is expressed as:

[0029] Where x represents input and y represents output. TableLLM(y i |x,y j＜i ) represents the probability of generating the i-th token given the j-th token preceding the i-th token in x and y.

[0030] The present invention also provides a training device for a large language model for processing tabular data, comprising:

[0031] An acquisition module is used to acquire form question and answer data, wherein the form question and answer data includes a form, questions and answers;

[0032] A data expansion module, inputting the table and question into the intelligent question-answering model to generate an expanded answer to the answer, and forming expanded training data based on the table, question and expanded answer;

[0033] A data generation module generates a new question based on the table, inputs the table and the new question into the intelligent question-answering model, and generates a first answer and a second answer based on different methods;

[0034] A cross-validation module, comparing the first answer and the second answer for cross-validation, and generating training data based on the table, the new question, and the first answer and / or the second answer if the cross-validation passes;

[0035] The training module trains the pre-trained model according to the mixed training set formed by the generated training data and the expanded training data to obtain the large language model for processing the tabular data, wherein the large language model for processing the tabular data is used to generate answers according to the tables and questions.

[0036] The invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method for training a large language model for processing tabular data as described in any of the above embodiments is implemented.

[0037] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for training a large language model for processing tabular data as described in any of the above embodiments is implemented.

[0038] The above embodiments of the present invention have at least the following beneficial effects:

[0039] The embodiment of the present invention uses data reasoning process expansion strategy and data cross-validation strategy to improve the efficiency of model understanding reasoning mode and ensure data quality. For tables in files of different formats, a large model that can skillfully process table data in embedded documents and electronic tables is trained through internal parameter-driven and code-driven methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0041] Figure 1 A flow chart of a method for training a large language model using tabular data provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the training device architecture for processing a large language model using tabular data provided by the present invention;

[0043] Figure 3 This is a schematic diagram of the electronic device architecture provided by the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. It should be noted that, in the absence of conflict, the embodiments and features in the embodiments disclosed in this disclosure can be combined, separated, interchanged and / or rearranged with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0045] The terms used here are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used here, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used here, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0046] The present disclosure is described below through several specific embodiments. In order to keep the following description of the embodiments of the present invention clear and concise, the present invention omits the detailed description of known functions and known components. Figure 1 The embodiment of the present invention provides a method for training a large language model for table data processing, comprising the following steps:

[0047] Step 1: Obtain form question and answer data, wherein the form question and answer data includes a form, questions and answers.

[0048] In some embodiments, the tabular question and answer data is derived from one or more of the WikiTQ, FeTaQA, or TAT-QA datasets;

[0049] The tabular question and answer data also comes from WikiSQL and / or Spider datasets.

[0050] In the specific implementation, the initial table question answering data comes from the existing table question answering task benchmark data set. Specifically, for the query of document embedded table data, the training data is mainly collected from the widely used table question answering benchmarks, including WikiTQ, FeTaQA and TAT-QA. For the spreadsheet-like embedded table data operations, the training data is obtained from the two Text2SQL benchmarks WikiSQL and Spider.

[0051] Step 2: Input the table and question into the intelligent question-answering model to generate an extended answer to the answer, and form extended training data based on the table, question and extended answer.

[0052] In some embodiments, the intelligent question answering model includes one or more of GPT3.5, GPT4, and deepseek-coder-33B.

[0053] In some embodiments, the table and the question are input into an intelligent question-answering model to generate an expanded answer to the answer, including:

[0054] Determine the type of the table;

[0055] If the type is a document embedded table, a process expansion answer having a text query and a description of the reasoning process is generated by the intelligent question-answering model;

[0056] If the type is a spreadsheet, a code expansion answer is generated by the intelligent question and answer model, and the code expansion answer includes a table operation code that can generate the answer.

[0057] It should be understood that although the existing benchmark provides sufficient training data for table question answering tasks, the short answers provided by a single instance are not enough to solve complex table data operation tasks, which usually require complex reasoning processes to effectively obtain answers. Therefore, this embodiment proposes a data reasoning process expansion strategy to enhance training data by expanding the reasoning process of the existing benchmark to promote the training of large language models.

[0058] In specific implementation, question-answer pairs are provided to GPT-3.5, and it is instructed to enhance the reasoning process to expand the provided short answers. This reasoning process expansion is presented in text form to be consistent with queries with mixed text and table data input. The present invention believes that expanding the reasoning process of short answers to training data can enhance the reasoning ability of large language models. Since WikiTQ and FeTaQA only provide raw table data and do not involve text information, in order to simulate the office scene of mixed text and tables in the real world, the present invention also uses GPT-3.5 to supplement the table description information.

[0059] In some embodiments, the document embedding table is less than 500 tokens in length.

[0060] Since the answers in the training data are presented in text form, the solution of question-answer pairs is achieved through the model's internal parameter-driven approach. In connection with the form tasks in office scenarios, a constraint of less than 500 tokens is imposed on the form input in this way to ensure that the model can more accurately build the connection between questions and answers.

[0061] In addition, in order to ensure the quality of the text-based reasoning process using GPT-3.5, a large language model specifically used for scoring, CritiqueLLM, is used to evaluate the consistency between the reasoning process and the answer provided in the benchmark. In other words, this embodiment uses CritiqueLLM to evaluate the consistency between the result of the expanded answer and the original answer.

[0062] Since data operations on tables embedded in spreadsheets mainly involve simple table input and complex table operations, such tasks are more suitable for solving in a code-driven way. In combination with the characteristics of WikiSQL and Spider training sets corresponding to SQL queries, this example selects training data from them, and optionally expands the reasoning process of question-answer pairs into Pandas code form, for example, through the Deepseek large language model to ensure flexibility in data operations. By comparing the quality of the generated code, it is ensured that the execution results after the generated Pandas code are consistent with the original answers provided in the benchmark. Among them, for the Spider dataset, in order to better keep consistent with the needs of single-table operations in office scenarios, data with empty results after multi-table queries and SQL queries are excluded to better reflect actual applications.

[0063] Step three: Generate a new question based on the table, input the table and the new question into the intelligent question-answering model, and generate a first answer and a second answer based on different methods.

[0064] Step 4: Compare the first answer and the second answer for cross-validation. If the cross-validation passes, generate training data based on the table, the new question, and the first answer and / or the second answer.

[0065] In some embodiments, the table and the new question are input into the intelligent question-answering model, and the first answer and the second answer are generated based on different methods, including:

[0066] Determine the type of the table;

[0067] If the type is a document embedded table, based on the same new question, the intelligent question-answering model generates a first answer with a text query and a description of the reasoning process and a description of the result through different prompt words, and generates a second answer including a table operation code;

[0068] If the type is a spreadsheet, the intelligent question-answering model generates a first answer including a first table operation code and a second answer including a second table operation code based on the new question.

[0069] In some embodiments, the table and the new question are input into the intelligent question-answering model, and the first answer and the second answer are generated based on different methods, including:

[0070] Determine the type of the table;

[0071] If the type is a document embedded table, based on the same new question, the intelligent question-answering model generates a first answer with a text query and a description of the reasoning process and a description of the result through different prompt words, and generates a second answer including a table operation code;

[0072] If the type is a spreadsheet, the intelligent question-answering model generates a first answer including a first table operation code and a second answer including a second table operation code based on the new question.

[0073] In some embodiments, the first answer and the second answer are compared for cross-validation, and if the cross-validation is passed, training data is generated based on the table, the new question, and the first answer and / or the second answer, including:

[0074] Determine the type of the table;

[0075] If the type is a document embedded table, and if the result description of the first answer is consistent with the running result of the table operation code of the second answer, then through cross-validation, first generated training data is formed based on the table, the new question and the first answer, and second generated training data is formed based on the table, the new question and the second answer. If they are inconsistent, cross-validation is not passed and the table and the new question are discarded;

[0076] If the type is a spreadsheet, and if the running results of the spreadsheet operation codes of the first answer and the second answer are consistent, the generated training data is formed based on the spreadsheet, the new question and one of the first answer or the second answer through cross-validation.

[0077] It should be understood that although the existing benchmark training data is of high quality, the types of question-answer pairs, especially the types of operations such as table update, merge, and drawing, are relatively limited. In order to enrich the types of table operations to better meet the needs of real office scenarios, the embodiment of the present invention introduces a data cross-validation strategy, which uses the table information provided in the benchmark to automatically or manually generate new question-answer pairs. The specific process is as follows:

[0078] First, generate questions: In the specific implementation, a total of 15,000 tables with less than 500 tokens are selected from the WikiTQ, TAT-QA, and FeTaQA data sets to simulate document-embedded table data. For each table, GPT-3.5 is used to generate questions involving single or multiple table query operations. For WikiTQ and FeTaQA tables, GPT-3.5 also creates a contextual description of the table content, and retains the original context information for TAT-QA tables. In addition, 1,300 long tables were selected from GitTables. For each table, 20 questions involving various table operations were generated. Existing benchmarks usually focus on table query operations, so when generating questions, this embodiment also adds problems or operation requirements such as table update, merge and drawing operations. For query, update and drawing operations, GPT-3.5 can be used to generate questions through prompts; for merge operations, merge questions can be generated by directly constructing prompts.

[0079] Next is answer generation and cross-validation: For problems based on document-embedded tabular data, this embodiment uses prompt words for GPT-3.5 to generate solutions through internal parameters (i.e., generated directly by the model) and code-driven methods (i.e., generating operation codes that can get results through the model). The generated code is executed to produce an answer, and the answer is used as a reference, and then CritiqueLLM is used to evaluate the consistency between the internal parameter-driven answer and this reference, thereby improving the quality of the answer. This internal parameter-driven and code-driven technology provides a variety of solutions and constitutes a form of cross-validation. For problems with tabular data embedded in spreadsheets, GPT-3.5 is first used to generate a pandas code solution, and then GPT-4 is used again to generate an alternative code solution. The accuracy of the execution results from the first code is verified by comparing with the results of the second code. Considering the potential diversity of the two encoding solutions, which lead to the same answer, this dual encoding strategy can be regarded as originating from different distributions. Therefore, it also serves as a cross-validation method to ensure the reliability of the solution.

[0080] Step 5: Train the pre-trained model based on a mixed training set formed by the generated training data and the expanded training data to obtain the large language model for processing tabular data, wherein the large language model for processing tabular data is used to generate answers based on tables and questions.

[0081] In some embodiments, when the pre-training model is trained based on the generated training data and the expanded training data to form a mixed training set to obtain the large language model for processing the tabular data, the loss function of the training is expressed as:

[0082] Where x represents input and y represents output. TableLLM(y i |x,y j＜i ) represents the probability of generating the i-th token given the j-th token preceding the i-th token in x and y.

[0083] This loss function is similar to the standard language model loss and specifically considers the loss calculated on y. This embodiment mixes the document table embedding training data and the spreadsheet training data in a 1:1 ratio, shuffles them thoroughly, and then divides them into training batches.

[0084] Specifically, in the context of document-embedded tabular data, the input to the LLM includes both the text and the entire table content. However, in the case of spreadsheet-embedded tabular data, since tables usually have a wide range of lengths, only the header and a portion of the rows are provided as input to the LLM. The trained single model covers both types of data sources. Given that code models tend to outperform text models on reasoning-intensive tasks, combining these two data sources can enhance text-level reasoning with code-level reasoning.

[0085] In some embodiments, in order to verify the effectiveness of generating and expanding training data in the present invention, the impact of different training data sets is analyzed by comparing five different training configurations:

[0086] 1. CodeLlama (13B): The original version without any training is evaluated on the corresponding test set.

[0087] 2. Original training data of existing benchmarks. We use 2000 training instances from TAT-QA and WikiTQ to train CodeLlama, and then evaluate it on the corresponding test sets.

[0088] 3. Extended training data for existing benchmarks: Train on 2,000 training instances from WikiTQ / TAT-QA, and supplement the extended reasoning process of each instance to form the extended training data with process extended answers, and then evaluate the corresponding test set. Train on 2,000 training instances from Spider, and supplement the extended pandas code to form extended training data with code extended answers, and evaluate Spider and the created test set.

[0089] 4. Use the generated training data: Train 2000 generated training data based on the tabular data of WikiTQ / TAT-QA, and then test on the corresponding test set. Train 2000 generated training data with code output based on the tabular data of GitLab, and evaluate the Spider and the created test set.

[0090] 5. Use mixed data: Train with 2000 extended instances and 2000 generated instances from TAT-QA / WikiTQ, and then evaluate on the corresponding test set. Train with a mix of 2000 extended Spider training data and 2000 generated training data with code output instances, and evaluate on both Spider and the created test set.

[0091] The experimental results are shown in Table 1 below:

[0092] Table 1. Impact of different training data sources

[0093]

[0094] The results presented in the table above demonstrate the effectiveness of incorporating the extended reasoning process, improving performance by 3.8% and 9.2% on WikiTQ and TAT-QA, respectively, compared to using the original training data (i.e., question and answer pairs) alone. This improvement is mainly attributed to the inclusion of detailed textual explanations of the results, which helps large language models recognize reasoning patterns. In addition, the addition of generated data improves performance by 1.6% and 6.4% compared to the original training data on WikiTQ and TATQA, further highlighting the value of including the reasoning process in the answers. Notably, hybrid training with extended and generated training data significantly improves performance by 4.8% and 10.1% compared to using only original data, highlighting the advantages of integrating different data sources. The results observed on Spider and the test set we created further confirm the benefits of extended and generated training data.

[0095] The data cross-validation strategy in this invention uses the consistency between the direct answer generation driven by internal parameters and the code generation solution. To further verify the effectiveness of this method, we compared it with two other verification methods: same-path verification, which generates two direct answers through the same internal parameter technology of GPT-3.5, and selects the correct samples as training data after comparing their results; self-checking verification, which enables GPT-3.5 to generate a text solution and check whether its answer is correct by itself. The experimental results are shown in Table 2 below:

[0096] Table 2. Experimental results of cross-validation strategy

[0097] Verification strategy WikiTQ TAT Self-checking and verification strategy 49.4％ 55.8％ Same-path verification strategy 49.6％ 58.2％ Cross-validation strategy (present invention) 51.5％ 59.8％

[0098] According to the results in Table 2, the data cross-validation strategy of the present invention is superior to other methods. This superior performance is attributed to the fact that it uses two different responses, thereby obtaining more reliable data results. This proves the accuracy of the cross-validation strategy of the present invention.

[0099] The present invention also collected test sets for document-embedded table query tasks from established benchmarks, including WikiTQ, FeTAQA, and TAT-QA. In addition, the present invention also integrated the OTT-QA dataset, whose tables and questions are significantly different from the training data of the present invention, to evaluate the generalization ability of the model of the present invention. For table-embedded table tasks, the present invention utilized test sets from WikiSQL and Spider, which meet the query operation requirements of the present invention. Since there are no benchmarks for table updates, merges, and chart operations, the present invention created test sets through manual annotation. The present invention selected 50 long tables from InfiAgent-DABench to ensure that they are completely different from the training data of the present invention. In terms of evaluation, the present invention adopted a classification evaluation method to evaluate the performance of the model on different operations. Query operation: For the answers obtained by code execution, the present invention performs an exact match comparison to determine the correctness of the model output with the standard answer. However, for the answers directly inferred by internal parameters, since the answers generated by the model are long and cannot be accurately matched, the present invention relies on CritiqueLLM to assign a score of 1 to 10 by comparing the model output with the standard answer. According to the manual verification judgment of the present invention, the evaluation results with scores of ≥7 can be considered correct. In terms of chart operation, it is difficult to evaluate the correctness of the drawing operation by drawing comparison. Therefore, the present invention compares the code output of the model with the corresponding code of the standard answer. The present invention also uses CritiqueLLM to compare the output code of the model with the standard answer code, and the scoring threshold during evaluation is set to 5.

[0100] According to the above standards, the present invention selects a variety of methods such as pre-training and fine-tuning large models, general large models, code customized large models and prompt word class large models for comparison. The results are shown in Table 3 below:

[0101] Table 3. Overall evaluation of various methods in document-embedded table and spreadsheet scenarios

[0102]

[0103] A “–” in the table means that the method does not support the data set or the test accuracy is too low.

[0104] From the results in the above table, it can be found that the large table model trained by the present invention is generally superior to other existing methods in the spreadsheet embedding scenario, and is comparable to GPT-3.5 in the document embedding table scenario, second only to GPT-4, which has the highest commercial level in the industry, proving the high efficiency of the present invention.

[0105] The present invention also provides a training device for a large language model for processing tabular data, such as Figure 2 As shown, including:

[0106] An acquisition module is used to acquire form question and answer data, wherein the form question and answer data includes a form, questions and answers;

[0107] A data expansion module, inputting the table and question into the intelligent question-answering model to generate an expanded answer to the answer, and forming expanded training data based on the table, question and expanded answer;

[0108] A data generation module generates a new question based on the table, inputs the table and the new question into the intelligent question-answering model, and generates a first answer and a second answer based on different methods;

[0109] A cross-validation module, comparing the first answer and the second answer for cross-validation, and generating training data based on the table, the new question, and the first answer and / or the second answer if the cross-validation passes;

[0110] The training module trains the pre-trained model according to the mixed training set formed by the generated training data and the expanded training data to obtain the large language model for processing the tabular data, wherein the large language model for processing the tabular data is used to generate answers according to the tables and questions.

[0111] The invention also provides an electronic device, such as Figure 3 As shown, it includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, it implements the training method of a large language model for processing tabular data as described in any of the above embodiments.

[0112] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for training a large language model for processing tabular data as described in any of the above embodiments is implemented.

[0113] Computer readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0114] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0115] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0116] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for training a large language model for processing tabular data, characterized in that: The following steps are involved: Acquire form question and answer data, wherein the form question and answer data includes a form, questions and answers; Inputting the table and question into the intelligent question-answering model to generate an extended answer to the answer, and forming extended training data based on the table, question and extended answer; Generate a new question based on the table, input the table and the new question into the intelligent question-answering model, and generate a first answer and a second answer based on different methods; Compare the first answer and the second answer for cross-validation, and if the cross-validation passes, generate training data based on the table, the new question, and the first answer and / or the second answer; The pre-training model is trained according to the mixed training set formed by the generated training data and the expanded training data to obtain the large language model for processing tabular data, wherein the large language model for processing tabular data is used to generate answers according to the table and the question; The step of inputting the table and the question into the intelligent question-answering model to generate an extended answer to the answer includes: Determine the type of the table; If the type is a document embedded table, a process expansion answer having a text query and a description of the reasoning process is generated by the intelligent question-answering model; If the type is a spreadsheet, a code expansion answer is generated by the intelligent question-answering model, and the code expansion answer includes a table operation code capable of generating the answer; The inputting the table and the new question into the intelligent question-answering model, and generating the first answer and the second answer based on different methods, includes: Determine the type of the table; If the type is a document embedded table, based on the same new question, the intelligent question-answering model generates a first answer with a text query and a description of the reasoning process and a description of the result through different prompt words, and generates a second answer including a table operation code; If the type is a spreadsheet, the intelligent question-answering model generates a first answer including a first table operation code and a second answer including a second table operation code based on the new question.

2. The method for training a large language model for table data processing according to claim 1, characterized in that: The tabular question-answering data is derived from one or more of the WikiTQ, FeTaQA or TAT-QA datasets; The tabular question and answer data also comes from WikiSQL and / or Spider datasets.

3. The method for training a large language model for table data processing according to claim 1, characterized in that: The intelligent question answering model includes one or more of GPT3.5, GPT4, and deepseek-coder-33B.

4. The method for training a large language model for table data processing according to claim 1, characterized in that: Comparing the first answer and the second answer for cross-validation, and if the cross-validation is passed, generating training data based on the table, the new question, and the first answer and / or the second answer, including: Determine the type of the table; If the type is a document embedded table, and if the result description of the first answer is consistent with the running result of the table operation code of the second answer, then through cross-validation, first generated training data is formed based on the table, the new question and the first answer, and second generated training data is formed based on the table, the new question and the second answer. If they are inconsistent, cross-validation is not passed and the table and the new question are discarded; If the type is a spreadsheet, and if the running results of the spreadsheet operation codes of the first answer and the second answer are consistent, the generated training data is formed based on the spreadsheet, the new question and one of the first answer or the second answer through cross-validation.

5. The method for training a large language model for table data processing according to claim 1, characterized in that: The pre-training model is trained by forming a mixed training set based on the generated training data and the expanded training data to obtain the large language model for processing the tabular data. The loss function of the training is expressed as: in Indicates input, Represents the output, TableLLM(y i |x,y j<i ) indicates that in a given and The probability of generating the i-th token given the j-th token before the i-th token.

6. A training device for a large language model for processing tabular data, characterized in that: include: An acquisition module is used to acquire form question and answer data, wherein the form question and answer data includes a form, questions and answers; A data expansion module, inputting the table and question into the intelligent question-answering model to generate an expanded answer to the answer, and forming expanded training data based on the table, question and expanded answer; A data generation module generates a new question based on the table, inputs the table and the new question into the intelligent question-answering model, and generates a first answer and a second answer based on different methods; A cross-validation module, comparing the first answer and the second answer for cross-validation, and generating training data based on the table, the new question, and the first answer and / or the second answer if the cross-validation passes; A training module, which trains the pre-trained model according to a mixed training set formed by generating training data and expanding training data to obtain the large language model for processing table data, wherein the large language model for processing table data is used to generate answers according to tables and questions; The step of inputting the table and the question into the intelligent question-answering model to generate an extended answer to the answer includes: Determine the type of the table; If the type is a document embedded table, a process expansion answer having a text query and a description of the reasoning process is generated by the intelligent question-answering model; If the type is a spreadsheet, a code expansion answer is generated by the intelligent question-answering model, and the code expansion answer includes a table operation code capable of generating the answer; The inputting the table and the new question into the intelligent question-answering model, and generating the first answer and the second answer based on different methods, includes: Determine the type of the table; If the type is a document embedded table, based on the same new question, the intelligent question-answering model generates a first answer with a text query and a description of the reasoning process and a description of the result through different prompt words, and generates a second answer including a table operation code; If the type is a spreadsheet, the intelligent question-answering model generates a first answer including a first table operation code and a second answer including a second table operation code based on the new question.

7. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the training method for a large language model for processing tabular data as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for training a large language model for processing tabular data as described in any one of claims 1 to 5 is implemented.

Images