Table analysis method and table understanding model training method

Abstract

The invention discloses a table analysis method and a table understanding model training method. The table analysis method comprises the following steps: receiving table data uploaded by a target object; according to a query request of a target object, a table understanding model is adopted to process table data to obtain a table analysis result, the table understanding model is used for analyzing a table structure and internal logic of the table data and is obtained through training of historical table data after multi-dimensional semantic enhancement and expression enhancement, and the table analysis result is obtained; semantic enhancement is used for enriching content diversity of the historical table data, and expression enhancement is used for enriching description diversity of the historical table data. According to the method and the device, the technical problems of insufficient understanding ability, relatively weak generalization ability and over-high dependence on a specific format when a large language model is used for processing structured table data in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the field of table comprehension, and more specifically, to a table analysis method and a method for training a table comprehension model. Background Art

[0002] With the rapid development of big data and artificial intelligence technologies, table understanding has become a core technology in key business processes in many industries, especially in the fields of finance, scientific research, and healthcare. The ability to automatically extract important information from structured table data and perform analysis has become an urgent need.

[0003] However, the tabular training data construction methods in related technologies usually have the following problems. First, most methods focus on converting tabular data into two-dimensional formats such as Markdown, but ignore the advantages of large language models in processing one-dimensional natural language text, resulting in pre-training models being difficult to efficiently adapt and understand the deep semantics of tabular data. Second, related technical solutions mainly focus on the recognition of table structure and the extraction of local information, ignoring the in-depth understanding and analysis of the overall content of the table, especially the lack of training in comprehensive analysis capabilities such as time series and cross-column comparisons. Third, the expression of training data is often single, and the model is prone to overfitting specific expression patterns, which reduces the generalization ability to diverse scenarios, and lacks effective quality control measures, making it impossible to ensure that the content learned by the model is both valuable and diverse.

[0004] These problems limit the effectiveness of large models in table understanding tasks, especially when processing complex table analysis and generating natural language descriptions. The model's performance is affected by template dependence and insufficient generalization capabilities, making it difficult to meet the modern enterprise's demand for immediate and accurate data insights.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0006] The embodiments of the present application provide a table analysis method and a table understanding model training method to at least solve the technical problems that large language models in related technologies have insufficient understanding ability, weak generalization ability, and excessive dependence on specific formats when processing structured table data.

[0007] According to one aspect of an embodiment of the present application, a table analysis method is provided, including: receiving table data uploaded by a target object; processing the table data using a table understanding model based on a query request of the target object to obtain a table analysis result, wherein the table understanding model is used to analyze the table structure and internal logic of the table data, and is obtained by training historical table data after multi-dimensional semantic enhancement and expression enhancement, the semantic enhancement is used to enrich the content diversity of the historical table data, and the expression enhancement is used to enrich the description diversity of the historical table data.

[0008] Optionally, the table understanding model is trained in the following manner: obtaining historical table data and performing key-value processing on the historical table data, wherein the key-value processing is used to convert the historical table data into a one-dimensional key-value pair format; performing multi-dimensional semantic enhancement processing on the key-value processed historical table data to obtain a first data set; performing multi-dimensional expression enhancement processing on the first data set to obtain a second data set; using a high-quality data set in the second data set to train the initial model to obtain a table understanding model, wherein the high-quality data set is a table data set whose quality assessment result obtained after quality assessment of the second data set is greater than a preset threshold.

[0009] Optionally, multi-dimensional semantic enhancement processing is performed on the key-value processed historical table data, including multi-dimensional semantic analysis of the historical table data: determining the indicator name and number of indicators in the historical table data to obtain indicator statistical results; determining the existence of preset indicators in the historical table data to obtain indicator judgment results; and determining the consistency between the preset data statement and the historical table data to obtain data verification results.

[0010] Optionally, multi-dimensional semantic enhancement processing is performed on the key-valued historical table data, which also includes multi-dimensional summary analysis of the historical table data: determining data question and answer results based on the row data and column data in the historical table data, wherein the question and answer results are used to represent the query and response relationship between indicators and numerical values in the historical table data; summarizing the unit data in the historical table data to obtain data summary results; performing qualitative analysis on the historical table data to obtain data analysis results; and generating a logical reasoning path corresponding to the historical table data based on the qualitative analysis data and the data analysis results.

[0011] Optionally, the first data set is subjected to multi-dimensional expression enhancement processing, including: processing the first data set using different data processing models based on preset prompt words to obtain a first expression result; adjusting the preset prompt words, processing the first data set using the same data processing model or a different data processing model to obtain a second expression result; adjusting model parameters, and processing the first data set using the adjusted data processing model to obtain a third expression result; and processing the first data set using a text retelling model to obtain a fourth expression result, wherein the text retelling model is used to convert the first data set into tabular data of different structures.

[0012] Optionally, the initial model is trained using a high-quality data set in the second data set, including: performing quality evaluation on the second data set according to a first evaluation rule and a second evaluation rule to obtain a quality evaluation result, wherein the first evaluation rule is used to express the data accuracy requirements for the historical tabular data, and the second evaluation rule is used to express the expression logic and fluency requirements for the historical tabular data; determining the tabular data that exceeds a preset threshold in the quality evaluation result as a high-quality data set in the second data set; and determining the tabular data that does not exceed the preset threshold in the quality evaluation result as a low-quality data set in the second data set.

[0013] Optionally, the method also includes: using high-quality data as positive samples for training the initial model, and using low-quality data as negative samples for training the initial model; training the initial model based on the positive samples, negative samples and contrast loss function to obtain a table understanding model, wherein the contrast loss function is used to improve the data expression ability of the initial model by comparing the similarity between positive samples and negative samples.

[0014] According to another aspect of an embodiment of the present application, a method for training a table understanding model is also provided, including: obtaining historical table data and performing key-value processing on the historical table data, wherein the key-value processing is used to convert the historical table data into a one-dimensional key-value pair format; performing multi-dimensional semantic enhancement processing on the key-value processed historical table data to obtain a first data set, wherein the semantic enhancement is used to enrich the content diversity of the historical table data; performing multi-dimensional expression enhancement processing on the first data set to obtain a second data set, wherein the expression enhancement is used to enrich the description diversity of the historical table data; using a high-quality data set in the second data set to train an initial model to obtain a table understanding model, wherein the high-quality data set is a set of table data whose quality assessment result obtained after quality assessment of the second data set is greater than a preset threshold.

[0015] According to another aspect of the embodiment of the present application, a table analysis device is also provided, including: a receiving module for receiving table data uploaded by a target object; an analysis module for processing the table data using a table understanding model based on a query request of the target object to obtain a table analysis result, wherein the table understanding model is used to analyze the table structure and internal logic of the table data, and is obtained by training historical table data after multi-dimensional semantic enhancement and expression enhancement, the semantic enhancement is used to enrich the content diversity of the historical table data, and the expression enhancement is used to enrich the description diversity of the historical table data.

[0016] According to another aspect of the embodiments of the present application, a training device for a table understanding model is also provided, including: an acquisition module for acquiring historical table data and performing key-value processing on the historical table data, wherein the key-value processing is used to convert the historical table data into a one-dimensional key-value pair format; a first enhancement module for performing multi-dimensional semantic enhancement processing on the historical table data after key-value processing to obtain a first data set, wherein the semantic enhancement is used to enrich the content diversity of the historical table data; a second enhancement module for performing multi-dimensional expression enhancement processing on the first data set to obtain a second data set, wherein the expression enhancement is used to enrich the description diversity of the historical table data; a training module for using a high-quality data set in the second data set to train the initial model to obtain a table understanding model, wherein the high-quality data set is a table data set whose quality assessment result obtained after quality assessment of the second data set is greater than a preset threshold.

[0017] According to another aspect of an embodiment of the present application, an electronic device is also provided, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the training method for implementing the above-mentioned table analysis method or table understanding model.

[0018] According to another aspect of an embodiment of the present application, a non-volatile storage medium is further provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned table analysis method or table understanding model training method by running the computer program.

[0019] According to another aspect of the embodiments of the present application, a computer program product is also provided, including computer instructions, which, when executed by a processor, implement the above-mentioned table analysis method or table understanding model training method.

[0020] In an embodiment of the present application, by receiving table data uploaded by a target object; and processing the table data using a table understanding model based on a query request of the target object, a table analysis result is obtained, wherein the table understanding model is used to analyze the table structure and internal logic of the table data, and is obtained through training of historical table data after multi-dimensional semantic enhancement and expression enhancement, semantic enhancement is used to enrich the content diversity of historical table data, and expression enhancement is used to enrich the description diversity of historical table data, thereby achieving the purpose of deeply mining the intrinsic semantics and logic of the table data, thereby realizing the technical effect of enhancing the ability of the large language model to understand the table data, and further solving the technical problems of the large language model in the related technology having insufficient understanding ability, weak generalization ability and excessive dependence on specific formats when processing structured table data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 is a hardware structure diagram of a computer terminal for implementing a table analysis method or a table comprehension model training method according to an embodiment of the present application;

[0023] Figure 2 is a flow chart of a table analysis method according to an embodiment of the present application;

[0024] Figure 3 is a flowchart of a method for training a table comprehension model according to an embodiment of the present application;

[0025] Figure 4 is a flowchart of another method for training a table comprehension model according to an embodiment of the present application;

[0026] Figure 5 is a structural diagram of a table analysis device according to an embodiment of the present application;

[0027] Figure 6 It is a structural diagram of a training device for a table comprehension model according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] First, some nouns or terms that appear in the process of explaining the embodiments of this application are subject to the following explanations:

[0031] Large Language Model (LLM): A deep learning model built through self-supervised or semi-supervised learning on massive amounts of text data. These models typically have millions to billions of parameters and are capable of processing and generating complex natural language text, performing tasks such as text generation, translation, question answering, and summarization.

[0032] Table understanding refers to the ability of a computer program or AI system to parse and analyze table content, including but not limited to identifying table structure, extracting key information, and understanding and expressing relationships between data. This process converts table data into a machine-readable format for further use in scenarios such as data analysis, report generation, and decision support.

[0033] Key-value pair: A format commonly used in data storage and transmission, where the "key" serves as an identifier to uniquely identify data, and the "value" is the data content corresponding to the key.

[0034] In order to solve the problem of poor table comprehension ability in the related art, the embodiment of the present application provides a table analysis method or a table comprehension model training method, which can be run on Figure 1 Among the computer terminals shown, the computer terminal will be described below.

[0035] The table analysis method or table understanding model training method provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computer terminal for implementing a table analysis method or a table understanding model training method. Figure 1As shown, the computer terminal 10 may include one or more (illustrated by 102a, 102b, ..., 102n in the figure) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0036] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10. As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the table analysis method or the table understanding model training method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned table analysis method or table understanding model training method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0038] The transmission module 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission module 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission module 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0039] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0040] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0041] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a table analysis method and a table understanding model training method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] Figure 2 is a flow chart of a table analysis method according to an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps:

[0043] Step S202: receiving the table data uploaded by the target object.

[0044] In step S202, a table data upload request is first received from a user or system. The target object is an end user or application that requires table understanding services. The table data can be in Excel, CSV, or other formats, and contains a series of structured data units.

[0045] Step S204: Based on the query request of the target object, a table understanding model is used to process the table data to obtain a table analysis result, wherein the table understanding model is used to analyze the table structure and internal logic of the table data, and is obtained through training of historical table data after multi-dimensional semantic enhancement and expression enhancement. Semantic enhancement is used to enrich the content diversity of historical table data, and expression enhancement is used to enrich the description diversity of historical table data.

[0046] In step S204, the uploaded table data can be parsed and analyzed using a pre-trained table understanding model based on the query request raised by the target object to generate detailed table analysis results. The table understanding model is trained based on historical table data through multi-dimensional semantic enhancement and representation enhancement. Its core purpose is to enhance the diversity of data content through semantic enhancement, enabling the model to more comprehensively understand the table structure and the inherent logic between the data. Representation enhancement enriches data descriptions, allowing the model to generate more diverse representations, reduce reliance on specific templates, and improve the quality and flexibility of the output text.

[0047] Through the above steps S202 to S204, the purpose of deeply mining the intrinsic semantics and logic of tabular data is achieved, thereby realizing the technical effect of enhancing the ability of the large language model to understand tabular data, thereby solving the technical problems of insufficient understanding ability, weak generalization ability and excessive dependence on specific formats of the large language model in related technologies when processing structured tabular data.

[0048] Optionally, the table understanding model is trained in the following manner: obtaining historical table data and performing key-value processing on the historical table data, wherein the key-value processing is used to convert the historical table data into a one-dimensional key-value pair format; performing multi-dimensional semantic enhancement processing on the key-value processed historical table data to obtain a first data set; performing multi-dimensional expression enhancement processing on the first data set to obtain a second data set; using a high-quality data set in the second data set to train the initial model to obtain a table understanding model, wherein the high-quality data set is a table data set whose quality assessment result is greater than a preset threshold after quality assessment of the second data set. The following is combined with Figure 3 Provide detailed explanation.

[0049] Figure 3 is a flowchart of a method for training a table comprehension model according to an embodiment of the present application, such as Figure 3 As shown, the method includes the following steps:

[0050] Step S302 : acquiring historical table data and performing key-value processing on the historical table data, wherein the key-value processing is used to convert the historical table data into a one-dimensional key-value pair format.

[0051] In the above step S302 , the historical table data may come from various application scenarios, such as financial statements, scientific research data, medical records, etc.

[0052] In real-world applications, tabular data is typically stored in formats like Excel or CSV, essentially a two-dimensional dataset organized in rows and columns. However, large language models (LLMs) were originally trained on large scales on plain text (one-dimensional data such as novels, news, and code). They are more adept at processing linearly arranged textual information rather than directly parsing complex tabular structures. When directly feeding tabular data into an LLM, it can encounter difficulties in interpretation and comprehension, reducing the model's effectiveness in table comprehension tasks.

[0053] Therefore, these diverse and complex historical tabular data can be key-value processed, that is, the two-dimensional tabular data can be converted into a one-dimensional key-value pair format so that large language models can better understand and learn the structured information therein.

[0054] Specifically, the content (value) of each cell is combined with the header (key) of its row and column to form a series of key-value pairs. These key-value pairs, when concatenated, form a one-dimensional representation of the original table data. For example, from a table containing information such as item type, time, location, and attributes, we can extract key-value pairs such as "item type: item X," "time: T1," and "location: S1." These key-value pairs not only preserve the integrity of the original data but also enhance the model's understanding of the semantics it carries.

[0055] In the embodiment of the present application, the specific key-value conversion examples are as follows:

[0056] Table 1 Original historical table data

[0057]

[0058] The historical table data in Table 1 above is key-value processed to obtain the following format data:

[0059] Item type: Item X, Time: T1, Location: S1, Attribute A: X11, Company A's price during the same period last year: X12, Company A's selling price: X13, Company B's price during the same period last year: X14, Company B's selling price: X15, ...

[0060] Item type: Item Y, Time: T1, Location: S2, Attribute A: Y21, Company A's price during the same period last year: Y22, Company A's selling price: Y23, Company B's price during the same period last year: Y24, Company B's selling price: Y25, ...

[0061] By converting tabular data into a one-dimensional key-value pair format, we not only address format compatibility issues faced by large language models when processing tabular data, but also facilitate the model's deep understanding and flexible application of tabular content. This preprocessing step lays a solid foundation for subsequent semantic enhancement and richer representations, enabling the model to more efficiently extract key information from tabular data, identify relationships between data, and generate high-quality natural language descriptions or analysis reports, significantly improving the performance and practicality of large models for table understanding and data analysis tasks.

[0062] Step S304 : performing multi-dimensional semantic enhancement processing on the key-valued historical table data to obtain a first data set, wherein the semantic enhancement is used to enrich the content diversity of the historical table data.

[0063] In step S304, multi-dimensional semantic enhancement is performed on the keyed data to enrich its content diversity, enabling the model to access and understand a wider range of information types and relationships during training. Specifically, semantic enhancement can be used to construct different types of task data, such as extracting, analyzing, comparing, and reasoning about tabular information. This ensures that the model goes beyond superficial table structure recognition and instead deeply understands the data's inherent logic and its value to business scenarios.

[0064] Step S306 : performing multi-dimensional expression enhancement processing on the first data set to obtain a second data set, wherein the expression enhancement is used to enrich the description diversity of the historical table data.

[0065] In step S306, the first dataset generated in step S304 undergoes multi-dimensional representation enhancement, aiming to enrich the different ways of describing the same data, thereby enhancing the model's representational diversity and flexibility. Specifically, representation enhancement can generate multiple versions of representations by adjusting prompt words, parameter settings, or introducing a paraphrase model. This prevents the model from overly relying on specific templates or representational conventions when generating output, thereby improving the model's ability to respond to the diverse needs of actual users.

[0066] Step S308: Use a high-quality dataset in the second dataset to train the initial model to obtain a table understanding model, wherein the high-quality dataset is a table data set whose quality assessment result obtained after quality assessment of the second dataset is greater than a preset threshold.

[0067] In the above step S308, it is first necessary to perform a quality assessment on the second data set to screen out a high-quality data set, and then use the high-quality data set to train the initial model to obtain the final table understanding model.

[0068] Quality assessment is a critical step in ensuring the validity and accuracy of training data. By setting a preset threshold for quality assessment results to distinguish high-quality from low-quality data, we ensure that the model focuses on learning valuable expressions and information rather than noise or meaningless data. This allows the trained model to generate more accurate, high-quality table comprehension output, reducing the possibility of errors and misunderstandings.

[0069] Through steps S302 to S308 above, the goal of establishing an intelligent model capable of efficiently understanding, analyzing, and representing tabular data is achieved, thereby achieving the technical effect of enhancing the large language model's ability to understand tabular data. This further addresses the technical issues of large language models in related technologies that suffer from insufficient understanding, weak generalization, and excessive reliance on specific formats when processing structured tabular data. This is explained in detail below.

[0070] In the above step S304, multi-dimensional semantic enhancement processing is performed on the key-valued historical table data, including multi-dimensional semantic analysis of the historical table data: determining the indicator name and the number of indicators in the historical table data to obtain the indicator statistical results; determining the existence of preset indicators in the historical table data to obtain the indicator judgment results; and determining the consistency between the preset data statement and the historical table data to obtain the data verification results.

[0071] Optionally, multi-dimensional semantic enhancement processing is performed on the key-valued historical table data, which also includes multi-dimensional summary analysis of the historical table data: determining data question and answer results based on the row data and column data in the historical table data, wherein the question and answer results are used to represent the query and response relationship between indicators and numerical values in the historical table data; summarizing the unit data in the historical table data to obtain data summary results; performing qualitative analysis on the historical table data to obtain data analysis results; and generating a logical reasoning path corresponding to the historical table data based on the qualitative analysis data and the data analysis results.

[0072] In the embodiment of the present application, step S304 mainly focuses on the diversified semantic enhancement strategy of tabular data, and its core goal is to enhance the understanding and reasoning ability of large language models on tabular data. Given that LLMs are usually pre-trained on large-scale text data, they perform well in processing one-dimensional text sequences, but have limited direct understanding and generation capabilities for two-dimensional tabular data. Therefore, this step is committed to building diversified training data so that the model can deeply analyze the semantic relationships in the table, go beyond simple structure recognition, and then demonstrate excellent table processing skills in a wider range of application scenarios. This diversified semantic enhancement strategy mainly includes the following two aspects:

[0073] 1. Semantic analysis of table key-value pair data.

[0074] The main purpose is to conduct a more detailed semantic analysis of the key-value historical table data, so that the model can deeply understand the structure and content of the data. The specific analysis is as follows:

[0075] (1) Indicator identification and feature extraction: By identifying the key (i.e., indicator) and extracting its features, the model can identify the names and counts of all indicators, establish a basic understanding of the table structure framework, and understand the attribute information in the dataset. The data example is as follows:

[0076] {"input":"Given the following data segment, output the number of indicators contained in the data segment and the name of the indicator. The data segment is as follows:\nItem type: Item X, Time: T1, Location: S1, Attribute A: X11, Company A's price during the same period last year: X12, Company A's price: X13, Company B's price during the same period last year: X14, Company B's price: X15, ......",

[0077] "output": "The given data segment contains n indicators, the indicator names are: item type, time, location, attribute A, company A's price in the same period last year, company A's price, company B's price in the same period last year, company B's price"}

[0078] (2) Key existence determination: The training model determines whether a specific indicator exists in a given key-value pair data to enhance the model's recognition ability and accuracy and reduce uncertainty when processing data. The data example is as follows:

[0079] {"input":"Based on the data snippet below, determine whether it contains the indicators: attribute A and company C's selling price.\nThe data snippet is as follows:\nItem type: item X, time: T1, location: S1, attribute A: X11, company A's selling price during the same period last year: X12, company A's selling price: X13, company B's selling price during the same period last year: X14, company B's selling price: X15, ......",

[0080] "output": "The given data segment contains the indicator: A attribute. The given data segment does not contain the indicator: Company C's selling price"}

[0081] (3) Data statement consistency verification: Design tasks to allow the model to check whether the statements based on key-value pairs are consistent with the original data, in order to enhance the model's critical thinking and error correction capabilities. The system will provide a statement based on the data and require the model to judge its authenticity. For false statements, further errors need to be corrected and correct statements generated.

[0082] The data examples are as follows:

[0083] {"input": I will give you a data source and a statement. Based on the data in the data source, please answer whether the statement is correct and provide reasons. \nData source: \nItem type: Item X, Time: T1, Location: S1, Attribute A: X11, Company A's price during the same period last year: X12, Company A's selling price: X13, Company B's price during the same period last year: X14, Company B's selling price: X15\nStatement: Company A's price for item X is X14.",

[0084] "output": "Error. According to the data source, the price of item X at Company A is X13."}

[0085] 2. Summary and analysis of table key-value pair data.

[0086] In addition to deepening understanding at the semantic level, historical table data can also be summarized and analyzed in multiple dimensions, allowing the model to flexibly apply data rather than just memorize it. The specific analysis is as follows:

[0087] (1) Table question-answer pair generation: By converting row and column data in historical table data into question and answer pairs, the training model learns to accurately extract indicators and corresponding numerical information from key-value pairs, understand the relationship between various fields, and express them in the form of natural language, thereby enhancing the model's retrieval ability and cultivating its ability to convert structured data into human-readable text, providing users with intuitive and fast data query services. The data example is as follows:

[0088] {“input”: “What is the value of attribute A of item X at time T1 and location S1?”, “output”: “X11”}

[0089] {“input”: “What was the selling price of item X at Company A at time T1 and location S1 during the same period last year?”, “output”: “X12”}

[0090] {“input”: “What is the selling price of item X at Company A at time T1 and location S1?”, “output”: “X13”}

[0091] {“input”: “What was the selling price of item X at Company B at time T1 and location S1 during the same period last year?”, “output”: “X14”}

[0092] {“input”: “What is the selling price of item X at Company B at time T1 and location S1?”, “output”: “X15”}

[0093] (2) Tabular Data Summarization: The model is required to generate concise summaries based on key-value pairs, helping it learn how to extract key information while maintaining the integrity and relevance of the data. By understanding the meaning of table cell data and combining row and column information, the model is trained to learn to summarize the main trends, key features, or comprehensive information of the data in natural language. The data examples are as follows:

[0094] {“input”: Please understand the indicator name and corresponding number in the given data source, and output the relevant data summary:\nThe data is as follows:\nItem type: Item X, time: T1, location: S1, attribute A: X11, company A’s price in the same period last year: 8, company A’s price: 12, company B’s price in the same period last year: 15, company B’s price: 20",

[0095] "output": "The A attribute of item X at time T1 and location S1 is X11. Company A's selling price was 8 last year, and the current selling price is 12. Company B's selling price was 15 last year, and the current selling price is 20."

[0096] (3) Tabular data analysis: train the model to perform qualitative analysis, thereby helping the model understand the meaning and value behind the data. For example: 1) Analyze the meaning of the cell data corresponding to the indicator, such as a negative growth rate indicates a decline in the indicator, and a leading ranking indicates excellent performance of the indicator; 2) Compare and analyze the cell data corresponding to different indicators; 3) Analyze the correlation between different indicators, such as the relationship between the value of attribute A and the sales price of Company A in the same period last year. The data examples are as follows:

[0097] {"input": "I will give you a set of data. Please use it to output a brief analysis.\nData:\nItem type: Item X, Time: T1, Location: S1, Attribute A: X11, Company A's price during the same period last year: 8, Company A's price: 12, Company B's price during the same period last year: 15, Company B's price: 20",

[0098] "output": "Indicates that the price of item X at time T1 at location S1 generally increased. Company A's price was higher than the same period last year. Company B's price also increased, and Company B's price remained higher than Company A's."}

[0099] (4) Logical reasoning and analysis based on thought chains: Based on the above analysis, the training model constructs a logical reasoning path (i.e., thought chain) to transform the qualitative analysis results into the key steps of a coherent reasoning process. By guiding the model to start from the data and gradually build a chain from data to conclusion, the model can learn the thought process of analyzing data and avoid jumping directly to conclusions while ignoring the logic and rationality of the analysis process. The data example is as follows:

[0100] {“input”: “Based on the given data, please gradually output the key data to be analyzed, the results of the comparison and analysis, and then output the final brief analysis results.\nData source is as follows: Item type: Item X, Time: T1, Location: S1, Attribute A: X11, Company A’s selling price during the same period last year: 8, Company A’s selling price: 12, Company B’s selling price during the same period last year: 15, Company B’s selling price: 20”

[0101] "output": "Data to be analyzed and analysis process: 1. The price of item X at Company A at location S1 at time T1 was 8 last year, and the current price is 12, indicating that Company A's price has increased. 2. The price of item X at Company B at location S1 at time T1 was 15 last year, and the current price is 20, indicating that Company B's price has also increased. 3. Comparing the current prices of Company A and Company B, Company B's price is consistently higher than Company A's. Final analysis results: The price of item X at location S1 at time T1 generally increased. Both Company A and Company B saw price increases, and Company B's price remained consistently higher than Company A's."}

[0102] This data example shows that the final conclusion of the qualitative analysis was obtained by comparing "Company A's selling price in the same period last year" with "Company A's selling price", "Company B's selling price in the same period last year" with "Company B's selling price", and "Company A's selling price" with "Company B's selling price", demonstrating the thinking chain process.

[0103] In summary, the introduction of multi-dimensional semantic enhancement and summary analysis significantly enhances the model's ability to understand and apply tabular data. From understanding indicators to verifying data statements, to generating questions and answers, summarizing, and analyzing, each step gradually deepens the model's understanding of the data's connotations and trains its analytical skills. In particular, the introduction of logical reasoning training based on thought chains enables the model to systematically engage in logical thinking when processing complex tabular analysis and generate coherent analytical results. This not only improves the model's practicality and generalization capabilities, but also achieves significant breakthroughs in enhancing the model's analytical depth and accuracy, providing users with a one-stop table understanding solution and greatly improving the efficiency and quality of data analysis.

[0104] In the above-mentioned step S306, the first data set is subjected to multi-dimensional expression enhancement processing, including: according to the preset prompt words, using different data processing models to process the first data set to obtain a first expression result; adjusting the preset prompt words, using the same data processing model or a different data processing model to process the first data set to obtain a second expression result; adjusting the model parameters, and processing the first data set through the adjusted data processing model to obtain a third expression result; and using a text retelling model to process the first data set to obtain a fourth expression result, wherein the text retelling model is used to convert the first data set into tabular data of different structures.

[0105] In the embodiments of this application, the expression enhancement process aims to break the model's reliance on a single expression method and enhance its expression diversity and flexibility. The specific methods are as follows:

[0106] F1: For the same prompt word, different large model outputs are used.

[0107] Unlike previous single-model training methods, multiple different data processing models, such as Telechat, Qwen, and GPT, are used here to process the same task. This allows the model to be exposed to different understanding and generation styles, learn more diverse expression methods, and avoid over-reliance on a specific model or style, thereby enhancing the model's generalization ability and creativity, enabling it to provide high-quality output when faced with various inputs.

[0108] F2: Adjust the prompt word and use the same or different large model output.

[0109] By changing the prompt word, even using the same model, different representations can be obtained. For example, replacing "Please describe the main trends in the following data" with "Please summarize the key changes in the following data" can guide the model to generate richer and more varied text representations from different perspectives. This helps the model understand the subtle semantic differences between different representations, enhances its understanding ability and output diversity, and reduces its reliance on a single representation.

[0110] F3: Adjust the temperature parameters of the large model and then output.

[0111] By fine-tuning parameters like temperature, the style and richness of the model's output can be significantly altered. For example, setting a higher temperature parameter might make the model's output more creative, while a lower one might make it more conservative and accurate. This strategy aims to allow the model to explore a wider range of representations within controlled limits, thereby increasing representational diversity while maintaining output quality.

[0112] F4: Introduce text retelling model for output.

[0113] A text paraphrase model is introduced to convert the first dataset into tabular data with different structures and presentation formats. Unlike direct processing of raw data, the paraphrase model allows the model to understand and present data from a new perspective, such as converting data into narrative or graphical descriptions. This not only enriches the model's presentation capabilities but also enables it to learn paraphrase techniques, enhancing the readability and appeal of the output text. This cross-structure paraphrase training enables the model to process and generate data in a variety of formats, providing users with more diverse ways to understand and present data.

[0114] The data example is as follows:

[0115] Statement 1: The selling price of Company A's item X at location S1 at time T1 increased to 12 yuan compared with 8 yuan in the same period last year, while the selling price of Company B's item X at location S1 at time T1 increased to 20 yuan compared with 15 yuan in the same period last year.

[0116] Statement 2: Item X is at time T1 and location S1. Attribute A has a value of X11. Company A's selling price was 8 last year, and it's currently 12. Company B's selling price was 15 last year, and it's currently 20.

[0117] In the above step S308, the high-quality data set in the second data set is used to train the initial model, including: performing quality evaluation on the second data set according to the first evaluation rule and the second evaluation rule to obtain a quality evaluation result, wherein the first evaluation rule is used to express the data accuracy requirements for the historical table data, and the second evaluation rule is used to express the expression logic and fluency requirements for the historical table data; the table data that exceeds the preset threshold in the quality evaluation result is determined as the high-quality data set in the second data set; and the table data that does not exceed the preset threshold in the quality evaluation result is determined as the low-quality data set in the second data set.

[0118] In the embodiments of the present application, data quality control and screening are indispensable links in the training process of large language models, especially when constructing a diverse training data set. Although the strategy of enhancing the diversity of expressions can increase the learning breadth of the model, the introduction of too many expressions without screening, especially those with unclear semantics, confusing logic or errors, may have a negative impact on the training of the model, causing the model to learn invalid or even wrong information. Therefore, effectively screening and controlling data quality is the key to ensuring the effectiveness of model training.

[0119] Assume that the first data set obtained in the above step S306 is D1, and the second data set obtained in the above step S308 is D2, then D2 = {F1(x), F2(x), F3(x), F4(x)|x∈D1}, which represents the diversified representations obtained by transforming the seed data x through different representation enhancement methods.

[0120] To ensure that the model learns from high-quality and valuable data, this application introduces a reference prompt word p and an evaluation mechanism for evaluating the quality of the seed data x and the diversified expression data D2. The quality evaluation includes:

[0121] (1) Accuracy evaluation: This is the first evaluation rule mentioned above. Based on the prompt word p and specific data, verify whether the data representation is accurate and the logical reasoning is correct, ensuring that the model learns factual information and avoiding misleading training.

[0122] (2) Evaluation of logical expression and fluency: This is the second evaluation rule mentioned above. Even if the data is accurate, the logical expression and fluency of the expression are crucial, which is related to the readability of the model output and the difficulty of user understanding. By evaluating whether the data expression logic is clear and the language is fluent, the quality of the training data can be further improved, making the text generated by the model more natural and professional.

[0123] Furthermore, the quality screening strategy includes the following two methods:

[0124] (1) Scoring mechanism: Directly score each piece of data x and its diverse representation D2. Comprehensive evaluation is performed using large language models such as Telechat, Qwen, and GPT, or manual scoring. By calculating the average score s and setting a threshold θ, data above θ is considered high-quality data and moved to the high-quality dataset; data below θ is considered low-quality data and moved to the low-quality dataset to prevent these data from interfering with model training.

[0125] (2) Sorting and screening: Instead of using a scoring method, the data is sorted directly based on its overall quality. The data ranked before the kth place after sorting are considered high-quality data and moved into the high-quality data set; the data ranked after the kth place after sorting are considered low-quality data and also moved into the low-quality data set.

[0126] Optionally, the above method also includes: using high-quality data as positive samples for training the initial model, and using low-quality data as negative samples for training the initial model; training the initial model based on the positive samples, negative samples and contrast loss function to obtain a table understanding model, wherein the contrast loss function is used to improve the data expression ability of the initial model by comparing the similarity between positive samples and negative samples.

[0127] The specific expression can be as follows:

[0128]

[0129] Where L represents the contrast loss function, N represents the number of samples; x i Represents the i-th instance of the positive sample; Represents the corresponding positive sample; Represents the corresponding negative sample; t represents the temperature parameter of the model.

[0130] In the examples of this application, the high-quality dataset consists of rigorously screened data, each of which meets high standards for grammar, logic, and information accuracy. Using this data as positive examples for model training provides a clear learning objective for the model: the model should learn how to accurately extract information from tabular data and express it in clear, fluent natural language.

[0131] Low-quality datasets, on the other hand, contain data with flawed or inaccurate representations, such as grammatical errors, logical contradictions, and missing information. These data serve as negative examples in model training, acting as "negative teaching materials." By learning from low-quality data, the model can identify and avoid repeating similar errors when generating output, enhancing its discernment and enabling it to generate output more cautiously and accurately when faced with complex or ambiguous inputs.

[0132] The contrastive loss function is a special loss function used in model training. Its design aims to enable the model to more accurately distinguish between high-quality and low-quality data by quantifying the difference in similarity between positive and negative samples. During training, the model attempts to minimize the difference between positive samples and the expected output, while maximizing the difference between negative samples and the expected output. This "pulling positive samples closer and pushing negative samples further away" training mechanism helps the model form clearer decision boundaries during learning and improves its data representation capabilities. By continuously optimizing the contrastive loss, the model learns to correctly locate data in the similarity space, ensuring that the output representation is both accurate and coherent, consistent with the characteristics of high-quality data.

[0133] Figure 4 This is a flowchart of another method for training a table understanding model according to an embodiment of the present application. Figure 4 As shown in Figure 2, the whole process from raw tabular data to building high-quality and diverse training data is described in more detail. Specifically:

[0134] First, (historical) tabular data is acquired. Subsequently, through data preprocessing, the two-dimensional tabular data is converted into a one-dimensional key-value pair format to facilitate efficient understanding and learning by the large language model. Next, a variety of semantic enhancement strategies are employed. On the one hand, semantic parsing of the tabular key-value pair data is performed, including identification and feature extraction of key elements, determination of key existence, and verification of the consistency of the key-value pair data statements. On the other hand, summary and analysis of the tabular key-value pair data is performed, including generation of tabular question-answer pairs, tabular data summarization, tabular data analysis, and tabular logical reasoning and analysis based on thought chains. Furthermore, to enhance the diversity of data representation, representation enhancement methods such as different model outputs, adjustment of prompt words or model parameters, and generation of multi-dimensional representations using a paraphrase model can be used to diversify the representation of the same seed data, thereby improving the model's generalization ability. Finally, data quality control and screening are performed, with a comprehensive evaluation of data accuracy and representation quality. For example, screening based on a scoring parameter θ or a quality ranking k ensures that the model focuses on learning valuable, diverse, high-quality data. Low-quality data is also moved to a dedicated dataset to serve as negative samples for comparative learning, further optimizing the model's output quality.

[0135] In the embodiment of the present application, a one-dimensional key-value preprocessing strategy for tabular data is proposed, which effectively simplifies the data structure and enables the model to absorb and parse tabular information more efficiently. At the same time, through diversified semantic enhancement, including in-depth analysis of key-value pairs, summary analysis of data, and logical reasoning training based on thought chains, not only the model's understanding of the semantics of the table is strengthened, but also its analysis and reasoning capabilities are significantly improved, making the model output more accurate and logical. In addition, by enriching the diversity of data representation and implementing strict quality control, combined with the optimization training of the contrast loss function, the model's ability to learn high-quality and varied expressions is effectively promoted, avoiding excessive reliance on a single template, significantly enhancing the model's generalization and output quality, and being able to process tabular data more stably and professionally in practical applications to meet the needs of a wide range of fields.

[0136] According to an embodiment of the present application, a table analysis device is provided. It should be noted that the table analysis device of the embodiment of the present application can be used to execute the table analysis method provided in the embodiment of the present application. The table analysis device provided in the embodiment of the present application is introduced below.

[0137] Figure 5 This is a structural diagram of a table analysis device provided according to an embodiment of the present application. Figure 5 As shown, the device includes:

[0138] Receiving module 50, for receiving table data uploaded by the target object;

[0139] The analysis module 52 is used to process the table data using a table understanding model based on the query request of the target object to obtain the table analysis results, wherein the table understanding model is used to analyze the table structure and internal logic of the table data, and is obtained by training historical table data after multi-dimensional semantic enhancement and expression enhancement. The semantic enhancement is used to enrich the content diversity of the historical table data, and the expression enhancement is used to enrich the description diversity of the historical table data.

[0140] Through the receiving module and analysis module in the above-mentioned table analysis device, the purpose of deeply mining the intrinsic semantics and logic of the table data is achieved, thereby realizing the technical effect of enhancing the ability of the large language model to understand the table data, and thus solving the technical problems that the large language model in the related technology has insufficient understanding ability, weak generalization ability and excessive dependence on specific formats when processing structured table data.

[0141] According to an embodiment of the present application, a table comprehension model training device is also provided. It should be noted that the table comprehension model training device of the embodiment of the present application can be used to execute the table comprehension model training method provided in the embodiment of the present application. The table comprehension model training device provided in the embodiment of the present application is described below.

[0142] Figure 6 This is a structural diagram of a training device for a table comprehension model provided in accordance with an embodiment of the present application. Figure 6 As shown, the device includes:

[0143] An acquisition module 60 is used to acquire historical table data and perform key-value processing on the historical table data, wherein the key-value processing is used to convert the historical table data into a one-dimensional key-value pair format;

[0144] A first enhancement module 62 is configured to perform multi-dimensional semantic enhancement on the key-valued historical table data to obtain a first data set, wherein the semantic enhancement is used to enrich the content diversity of the historical table data;

[0145] A second enhancement module 64 is configured to perform multi-dimensional expression enhancement processing on the first data set to obtain a second data set, wherein the expression enhancement is used to enrich the description diversity of the historical table data;

[0146] The training module 66 is used to train the initial model using a high-quality dataset in the second dataset to obtain a table understanding model, wherein the high-quality dataset is a table data set whose quality assessment result obtained after quality assessment of the second dataset is greater than a preset threshold.

[0147] Through the acquisition module, first enhancement module, second enhancement module, and training module in the aforementioned table understanding model training device, the goal of establishing an intelligent model capable of efficiently understanding, analyzing, and representing tabular data is achieved, thereby achieving the technical effect of enhancing the ability of large language models to understand tabular data. This, in turn, addresses the technical issues of large language models in related technologies that suffer from insufficient understanding, weak generalization, and excessive reliance on specific formats when processing structured tabular data. This is described in detail below.

[0148] In the training device of the table comprehension model provided in the embodiment of the present application, the first enhancement module is also used to determine the indicator name and the number of indicators in the historical table data to obtain the indicator statistical results; determine the existence of preset indicators in the historical table data to obtain the indicator judgment results; and determine the consistency between the preset data statement and the historical table data to obtain the data verification results.

[0149] In the training device of the table comprehension model provided in an embodiment of the present application, the first enhancement module is also used to determine the data question and answer results based on the row data and column data in the historical table data, wherein the question and answer results are used to represent the query and response relationship between indicators and numerical values in the historical table data; summarize the unit data in the historical table data to obtain data summary results; perform qualitative analysis on the historical table data to obtain data analysis results; and generate a logical reasoning path corresponding to the historical table data based on the qualitative analysis data and the data analysis results.

[0150] In the training device of the table comprehension model provided in the embodiment of the present application, the second enhancement module is also used to process the first data set using different data processing models based on preset prompt words to obtain a first expression result; adjust the preset prompt words, use the same data processing model or a different data processing model to process the first data set to obtain a second expression result; adjust the model parameters, and process the first data set through the adjusted data processing model to obtain a third expression result; and use a text retelling model to process the first data set to obtain a fourth expression result, wherein the text retelling model is used to convert the first data set into tabular data of different structures.

[0151] In the training device of the table comprehension model provided in the embodiment of the present application, the training module is also used to perform quality evaluation on the second data set based on the first evaluation rule and the second evaluation rule to obtain a quality evaluation result, wherein the first evaluation rule is used to express the data accuracy requirements for the historical table data, and the second evaluation rule is used to express the expression logic and fluency requirements for the historical table data; the table data that exceeds the preset threshold in the quality evaluation result is determined as a high-quality data set in the second data set; and the table data that does not exceed the preset threshold in the quality evaluation result is determined as a low-quality data set in the second data set.

[0152] In the training device of the table understanding model provided in an embodiment of the present application, the training module is also used to use high-quality data as positive samples for training the initial model, and to use low-quality data as negative samples for training the initial model; the initial model is trained based on the positive samples, negative samples and the contrast loss function to obtain a table understanding model, wherein the contrast loss function is used to improve the data expression ability of the initial model by comparing the similarity between the positive samples and the negative samples.

[0153] An embodiment of the present application also provides an electronic device, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the training method for implementing the above-mentioned table analysis method or table understanding model.

[0154] It should be noted that the above electronic equipment is used to perform Figure 2 The table analysis method shown or Figure 3 The training method of the table understanding model shown, therefore the relevant explanations in the above table analysis method or the training method of the table understanding model are also applicable to the electronic device and will not be repeated here.

[0155] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned table analysis method or table understanding model training method by running the computer program.

[0156] It should be noted that the above non-volatile storage medium is used to execute Figure 2 The table analysis method shown or Figure 3 The training method of the table understanding model shown, therefore the relevant explanations in the above table analysis method or the training method of the table understanding model are also applicable to the non-volatile storage medium and will not be repeated here.

[0157] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-mentioned table analysis method or table understanding model training method.

[0158] It should be noted that the above-mentioned computer program product is used to execute Figure 2 The table analysis method shown or Figure 3 The training method of the table understanding model shown, therefore the relevant explanations in the above-mentioned table analysis method or the training method of the table understanding model are also applicable to this computer program product and will not be repeated here.

[0159] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0160] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0162] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0163] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0164] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0165] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A table analysis method, characterized in that: include: Receive the table data uploaded by the target object; Based on the query request of the target object, the table data is processed using a table understanding model to obtain a table analysis result, wherein the table understanding model is used to analyze the table structure and internal logic of the table data, and is obtained by training historical table data after multi-dimensional semantic enhancement and expression enhancement. The semantic enhancement is used to enrich the content diversity of the historical table data, and the expression enhancement is used to enrich the description diversity of the historical table data.

2. The method according to claim 1, characterized in that The table comprehension model is trained in the following way: Acquire historical table data, and perform key-value processing on the historical table data, wherein the key-value processing is used to convert the historical table data into a one-dimensional key-value pair format; Performing multi-dimensional semantic enhancement processing on the key-valued historical table data to obtain a first data set; Performing multi-dimensional representation enhancement processing on the first data set to obtain a second data set; The initial model is trained using a high-quality dataset in the second dataset to obtain the table understanding model, wherein the high-quality dataset is a table data set whose quality assessment result obtained after quality assessment of the second dataset is greater than a preset threshold.

3. The method according to claim 2, characterized in that Performing multi-dimensional semantic enhancement processing on the key-valued historical table data, including performing multi-dimensional semantic analysis on the historical table data: Determine the indicator name and the number of indicators in the historical table data to obtain indicator statistical results; Determine whether a preset indicator exists in the historical table data and obtain an indicator determination result; And determine the consistency between the preset data statement and the historical table data to obtain a data verification result.

4. The method according to claim 2, characterized in that Performing multi-dimensional semantic enhancement processing on the key-valued historical table data also includes performing multi-dimensional summary analysis on the historical table data: Determining a data question-and-answer result based on the row data and column data in the historical table data, wherein the question-and-answer result is used to represent a query-and-answer relationship between an indicator and a value in the historical table data; Summarizing the unit data in the historical table data to obtain a data summary result; Performing qualitative analysis on the historical table data to obtain data analysis results; And generating a logical reasoning path corresponding to the historical table data based on the qualitative analysis data and the data analysis results.

5. The method according to claim 2, characterized in that Performing multi-dimensional representation enhancement processing on the first data set includes: According to the preset prompt words, different data processing models are used to process the first data set to obtain a first expression result; Adjusting the preset prompt words, and processing the first data set using the same data processing model or a different data processing model to obtain a second expression result; Adjusting model parameters, and processing the first data set using the adjusted data processing model to obtain a third representation result; And using a text paraphrase model to process the first data set to obtain a fourth expression result, wherein the text paraphrase model is used to convert the first data set into tabular data with different structures.

6. The method according to claim 2, characterized in that The initial model is trained using the high-quality dataset in the second dataset, including: Performing a quality assessment on the second data set according to a first assessment rule and a second assessment rule to obtain a quality assessment result, wherein the first assessment rule is used to express a requirement for data accuracy of the historical table data, and the second assessment rule is used to express a requirement for logical expression and fluency of the historical table data; Determining the tabular data exceeding the preset threshold in the quality assessment result as a high-quality data set in the second data set; The tabular data that does not exceed the preset threshold in the quality evaluation result is determined as a low-quality data set in the second data set.

7. The method according to claim 6, characterized in that The method further comprises: Using the high-quality data as positive samples for training the initial model, and using the low-quality data as negative samples for training the initial model; The initial model is trained based on the positive samples, the negative samples, and a contrastive loss function to obtain the table comprehension model, wherein the contrastive loss function is used to improve the data expression ability of the initial model by comparing the similarity between the positive samples and the negative samples.

8. A method for training a table comprehension model, characterized in that: include: Acquire historical table data, and perform key-value processing on the historical table data, wherein the key-value processing is used to convert the historical table data into a one-dimensional key-value pair format; Performing multi-dimensional semantic enhancement processing on the key-valued historical table data to obtain a first data set, wherein the semantic enhancement is used to enrich the content diversity of the historical table data; Performing multi-dimensional expression enhancement processing on the first data set to obtain a second data set, wherein the expression enhancement is used to enrich the description diversity of the historical table data; The initial model is trained using a high-quality dataset in the second dataset to obtain a table understanding model, wherein the high-quality dataset is a set of table data whose quality assessment result obtained after quality assessment of the second dataset is greater than a preset threshold.

9. A table analysis device, characterized in that: include: Receiving module, used to receive the table data uploaded by the target object; An analysis module is used to process the table data using a table understanding model based on the query request of the target object to obtain a table analysis result, wherein the table understanding model is used to analyze the table structure and internal logic of the table data, and is obtained by training historical table data after multi-dimensional semantic enhancement and expression enhancement, the semantic enhancement is used to enrich the content diversity of the historical table data, and the expression enhancement is used to enrich the description diversity of the historical table data.

10. A training device for a table comprehension model, characterized in that: include: an acquisition module, configured to acquire historical table data and perform key-value processing on the historical table data, wherein the key-value processing is configured to convert the historical table data into a one-dimensional key-value pair format; a first enhancement module, configured to perform multi-dimensional semantic enhancement processing on the key-valued historical table data to obtain a first data set, wherein the semantic enhancement is used to enrich the content diversity of the historical table data; a second enhancement module, configured to perform multi-dimensional expression enhancement processing on the first data set to obtain a second data set, wherein the expression enhancement is used to enrich the description diversity of the historical table data; A training module is used to train the initial model using a high-quality data set in the second data set to obtain a table understanding model, wherein the high-quality data set is a table data set whose quality assessment result obtained after quality assessment of the second data set is greater than a preset threshold.

11. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the table analysis method described in any one of claims 1 to 7 or the table understanding model training method described in claim 8.

12. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the table analysis method described in any one of claims 1 to 7 or the table understanding model training method described in claim 8 by running the computer program.

13. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the table analysis method described in any one of claims 1 to 7 or the table understanding model training method described in claim 8 is implemented.

Images