Method, system, storage medium and electronic device for processing tabular data

Abstract

The present application discloses a method, system, storage medium and electronic device for processing table data, which obtains cell vector information and text vector information in the table to be processed, obtains data features of each row and each column, fuses the data features of each row and each column to obtain a table feature representation of the table to be processed, matches the table feature representation and the text feature representation, obtains a matching result of the table feature representation and the text feature representation, optimizes a preset training model through the matching result, and obtains structured table information in the table to be processed based on the optimized preset training model. Through the above, the semantic representation of free text and structured table is jointly learned, and different information source data, such as cell vector information, text vector information, semantic representation of structured table, etc. of the table are fused from different spaces and levels to obtain multi-granularity information, and the structured information of the table is effectively obtained through the multi-granularity information.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a method, system, storage medium and electronic device for processing table data. Background Art

[0002] With the rapid development of pre-trained language models (LMs) for natural language (NL) understanding tasks of text, pre-trained language representation models (Bidirectional Encoder Representations from Transformer, BERT) and other pre-trained models use a large amount of long text natural language as a prediction, train the model by setting self-supervised pre-training tasks, and use the trained model to perform text classification, entity recognition, text matching and other operations.

[0003] Although pre-trained models such as BERT, TaBERT, and Generative Pre-Training (GPT) have significant effects in downstream tasks such as text classification, text matching, and text generation, these models are usually trained mainly on long natural language text data, and the pre-training tasks they set are only for natural text, making it impossible to effectively obtain structured table information in tables.

[0004] Therefore, how to effectively obtain structured table information in a table has become an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present application discloses a method, system, storage medium and electronic device for processing table data, aiming to effectively obtain structured information of the table through the multi-granularity information.

[0006] In order to achieve the above purpose, the disclosed technical solutions are as follows:

[0007] In a first aspect, the present application discloses a method for processing tabular data, the method comprising:

[0008] Obtaining cell vector information in a table to be processed; the cell vector information includes cell row vector information and cell column vector information;

[0009] Obtaining data features for each row and data features for each column; the data features for each row are obtained by encoding the cell row vector information; the data features for each column are obtained by encoding the cell column vector information;

[0010] By using a preset fusion method, the data features of each row and the data features of each column are fused to obtain a table feature representation of the table to be processed; the table feature representation is used to characterize the feature representation of the cell vector information affected by the row context and the feature representation affected by the column context;

[0011] Matching the table feature representation with the pre-acquired text feature representation using a preset matching function to obtain a matching result;

[0012] If the matching result is a matching result in which the table feature representation matches the text feature representation, the preset training model is optimized through the matching result, and the structured table information in the table to be processed is obtained based on the optimized preset training model.

[0013] Preferably, the obtaining of cell vector information in the table to be processed includes:

[0014] Obtain cell vector information in the table to be processed through the preset training model.

[0015] Preferably, the features of each row of data are obtained, including:

[0016] Encoding the cell row vector information according to a preset row order to obtain a row feature representation; the row feature representation is a feature representation of the cell vector information affected by the row context;

[0017] A first preset character is added to the row feature representation to obtain the data feature of each row; the first preset character is used to represent the character of each row of table features.

[0018] Preferably, obtaining the characteristics of each column of data includes:

[0019] Encoding the cell column vector information according to a preset column order to obtain a column feature representation; the column feature representation is a feature representation of the cell vector information affected by the column context;

[0020] A second preset character is added to the column feature representation to obtain the data feature of each column; the second preset character is used to represent the character of each column table feature.

[0021] Preferably, the matching of the table feature representation and the pre-acquired text feature representation by a preset matching function to obtain a matching result includes:

[0022] Obtaining text vector information in the table to be processed, and determining text feature representation based on the text vector information;

[0023] When matching the table feature representation with the text feature representation using a preset matching function, obtaining a matching similarity score between the table corresponding to the table feature representation and the text corresponding to the text feature representation;

[0024] When the matching similarity score is greater than a preset similarity score, determining that the table feature representation and the text feature representation match, and generating a matching result indicating that the table feature representation and the text feature representation match;

[0025] When the matching similarity score is less than a preset similarity score, it is determined that the table feature representation and the text feature representation do not match, and a result indicating that the table feature representation and the text feature representation do not match is generated.

[0026] A second aspect of the present application discloses a system for processing tabular data, the system comprising:

[0027] A first acquiring unit is configured to acquire cell vector information in a table to be processed; the cell vector information includes cell row vector information and cell column vector information;

[0028] A second acquisition unit is configured to acquire data features of each row and data features of each column; the data features of each row are obtained by encoding the cell row vector information; the data features of each column are obtained by encoding the cell column vector information;

[0029] a fusion unit, configured to fuse the data features of each row and the data features of each column in a preset fusion manner to obtain a table feature representation of the table to be processed; the table feature representation is used to characterize the feature representation of the cell vector information affected by the row context and the feature representation of the cell vector information affected by the column context;

[0030] A matching unit, configured to match the table feature representation with the pre-acquired text feature representation using a preset matching function to obtain a matching result;

[0031] The third acquisition unit is used to optimize the preset training model through the matching result, and obtain the structured table information in the table to be processed based on the optimized preset training model.

[0032] Preferably, the first acquiring unit is specifically configured to:

[0033] Obtain cell vector information in the table to be processed through the preset training model.

[0034] Preferably, the second acquisition unit for acquiring the feature of each row of data includes:

[0035] A first encoding module is configured to encode the cell row vector information according to a preset row order to obtain a row feature representation; the row feature representation is a feature representation of the cell vector information affected by a row context;

[0036] The first adding module is used to add a first preset character to the row feature representation to obtain the data feature of each row; the first preset character is used to represent the character of each row of table features.

[0037] A third aspect of the present application discloses a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the table data processing method as described in any one of the first aspects.

[0038] The fourth aspect of the present application discloses an electronic device comprising a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to perform the method for processing tabular data as described in any one of the first aspects.

[0039] It can be seen from the above technical solution that the present application discloses a method, system, storage medium and electronic device for processing tabular data, obtaining cell vector information in the table to be processed, the cell vector information including cell row vector information and cell column vector information, obtaining the data features of each row and the data features of each column, the data features of each row are obtained by encoding the cell row vector information, and the data features of each column are obtained by encoding the cell column vector information, and the data features of each row and the data features of each column are fused by a preset fusion method to obtain a table feature representation of the table to be processed; the table feature representation is used to characterize the feature representation of the cell vector information affected by the row context and the feature representation affected by the column context, and the table feature representation and the pre-acquired text feature representation are matched by a preset matching function to obtain a matching result, if the matching result is a matching result that matches the table feature representation and the text feature representation, the preset training model is optimized by the matching result, and the structured table information in the table to be processed is obtained based on the optimized preset training model. Through the above scheme, the semantic representations of free text and structured tables are jointly learned, and different information source data, such as table cell vector information, text vector information, and semantic representation of structured tables, are fused from different spaces and levels to obtain multi-granularity information. The structured information of the table can be effectively obtained through this multi-granularity information. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0041] Figure 1 A flowchart of a method for processing tabular data disclosed in an embodiment of the present application;

[0042] Figure 2 This is an example diagram of the pre-trained BERT model disclosed in the embodiments of this application;

[0043] Figure 3 This is an example diagram of the pre-trained table and text data disclosed in the embodiments of this application;

[0044] Figure 4 This is an example diagram of column name prediction disclosed in the embodiments of this application;

[0045] Figure 5 This is an example diagram of column value prediction disclosed in the embodiments of this application;

[0046] Figure 6 This is an example diagram of text and table relevance prediction disclosed in the embodiments of this application;

[0047] Figure 7 A schematic diagram of the structure of a table data processing system disclosed in an embodiment of the present application;

[0048] Figure 8 This is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0050] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0051] As we know from the background, although pre-trained models such as BERT, TaBERT, and GPT have shown significant results in downstream tasks such as text classification, text matching, and text generation, these models are typically trained primarily on long natural language text data, and their pre-training tasks are designed to target only natural text, making it difficult to effectively capture structured table information within tables. Therefore, how to effectively capture structured table information within tables has become a pressing issue for those skilled in the art.

[0052] To address the above issues, the present application discloses a method, system, storage medium, and electronic device for processing tabular data. These methods jointly learn the semantic representations of free text and structured tables, fuse different information source data from different spaces and levels, such as table cell vector information, text vector information, and semantic representations of structured tables, to obtain multi-granular information. This multi-granular information effectively captures the structured information of the table. The specific implementation is described in detail in the following embodiments.

[0053] refer to Figure 1 FIG. 1 is a flow chart of a method for processing tabular data disclosed in an embodiment of the present application. The method for processing tabular data mainly includes the following steps:

[0054] S101: Obtain cell vector information in a table to be processed; the cell vector information includes cell row vector information and cell column vector information.

[0055] In S101, cell vector information in the table to be processed is obtained through a preset training model.

[0056] Among them, the pending forms are forms that have not been processed by the preset training model.

[0057] The preset training model can be a pre-trained (Bidirectional Encoder Representation from Transformers, BERT) model or other types of models. The specific preset training model is determined by the technician according to the actual situation and is not specifically limited in this application. The preset training model of this application is preferably a pre-trained BERT model.

[0058] The pre-trained BERT model includes the RowTransformer model, the ColumnTransformer model, and the TextTransformer model. To facilitate the understanding of the pre-trained BERT model, Figure 2 To explain, Figure 2 An example diagram showing a pre-trained BERT model.

[0059] According to the traditional BERT pre-training method, the column and row information in the table is combined into a linear long text sequence for training (for example, the content of the above table is linearized into "Package Name Changshou Dawang Card 4G [SEP] Price 39 [SEP] Partner JD [SEP] Package Name Taobao Dawang Card..."). However, this data input method makes the cell values ​​between different columns and rows independent during model training, and it is impossible to obtain the contextual dependencies between them (for example, Figure 2 The “Changshou Card 4G Package” and the “Taobao King Card 5G Package” belong to the same column and have a certain degree of similarity).

[0060] Therefore, it is necessary to encode the cell row vector information through the RowTransformer model to obtain the data features of each row, encode the cell column vector information through the ColumnTransformer model to obtain the data features of each column, encode the text vector information through the TextTransformer model to obtain the text vector information, and obtain the context dependency through the data features of each row, the data features of each column and the text vector information.

[0061] The text vector information is obtained through the pre-trained BERT model.

[0062] The text vector information is encoded using the TextTransformer model to obtain a text feature representation. This text feature representation is then matched with the table feature representation obtained below to obtain the structured table information in the table to be processed.

[0063] The specific pre-training process of the BERT model includes pre-training of the table and text data of the BERT model, modeling of the Masked Language Model (MLM), Masked Entity Prediction, Masked Entity Prediction, Column Name Prediction, Column Value Prediction, and text table relevance prediction NL&TableMatch.

[0064] Pre-training BERT model on tabular & textual data:

[0065] In the pre-training task of table and text data, a large amount of table and table-related text data is collected from e-commerce and public websites to form Table&NL (natural language) prediction pairs for pre-training. Figure 3 shown. Figure 3 Example graphs showing pre-trained tabular and text data.

[0066] Figure 3 The table includes holidays, dates, and holiday duration (excluding adjusted holidays). Holidays include New Year's Day, Spring Festival, Tomb-Sweeping Day, Labor Day, Dragon Boat Festival, Mid-Autumn Festival, and National Day. Dates include January 1, the first, second, and third day of the first lunar month, Tomb-Sweeping Day, May 1, the Dragon Boat Festival, Mid-Autumn Festival, and October 1, 2, and 3. Holiday duration (excluding adjusted holidays) includes 1 day, 3 days, etc.

[0067] The content of the text data is "Major traditional festivals in China include the Spring Festival, Lantern Festival, Tomb-Sweeping Day, etc. In addition, ethnic minorities also retain their own traditional festivals, such as the Water-Splashing Festival of the Dai people, the Nadam Festival of the Mongolian people..."

[0068] Masked Language Modeling (MLM) modeling:

[0069] This task refers to the pre-training task in the BERT model. For all the natural language (NL) of the text, a word is randomly masked using the [Masked] vocabulary, and the masked word is predicted through the context NL. For example, for Figure 3The sentence "In addition, ethnic minorities also keep their own traditional festivals, such as the Water-Splashing Festival of the Dai people, the Nadam Festival of the Mongolian people..." is randomly replaced with "In addition, ethnic minorities also keep their own traditional festivals, such as the Water-Splashing Festival of the Dai people, the Nadam Festival of the Mongolian people..." and the masked words are predicted by the language model.

[0070] Masked Entity Prediction:

[0071] Traditional MLM only predicts a masked word based on the contextual features of the entire natural language and is unable to capture table information features. To enhance the information interaction between the table pre-training task NL and Table and capture the knowledge sharing involved, a new pre-training task is proposed. This task first uses n-gram character matching to identify the entity word (either column name or column value) of a cell in the Table mentioned in the NL. This entity word is then masked according to a certain ratio. The masked entity word is then predicted using the overall contextual features of the NL and Table. For example, in the NL "Major traditional festivals in China include the Spring Festival, Lantern Festival, and Qingming Festival," where "Spring Festival, Lantern Festival, and Qingming Festival" are all entity words mentioned in the table column values, the words are randomly masked and predicted using the NL and Table features: "Major traditional festivals in China include [MASK][MASK], Lantern Festival, and Qingming Festival."

[0072] Column Name Prediction:

[0073] In order to strengthen the learning of structured information within the table, two pre-training tasks for table information learning are proposed. Column Name Prediction mainly masks the column names in the table randomly, and then predicts the column based on the spatial information and hierarchical information related to the column. Figure 4 Provide explanation.

[0074] Figure 4 In the example, "partner" is masked to obtain spatial information (information obtained horizontally based on column names) "Package Name [SEP] Price [SEP] Partner" -> "Package Name [SEP] Price [SEP] [MASK] [MASK]" and hierarchical information "[COL] | Partner | JD | Alibaba | ... " -> "[COL] | [MASK] [MASK] | JD | Alibaba | ... ", and the column name will be predicted using horizontal and vertical information.

[0075] RowValue Prediction:

[0076] The specific process of column value prediction is combined with Figure 5 Provide explanation. Figure 5 In the column value prediction, a column value in the table is randomly masked, and then the spatial information of the column and the hierarchical information of the row are used as context to predict the masked column value.

[0077] Text table relevance prediction NL&TableMatch:

[0078] In order to further enhance the interactive knowledge fusion of the model's NL and Table information representation, an additional task of text table relevance prediction is added. During the data collection process, the collected NL text descriptions related to the Table are used as positive samples, and NL is combined with other random Tables to construct negative samples. Figure 6 To explain, Figure 6 An example graph showing text and table relevance predictions.

[0079] Figure 6 In , simScore is the similarity score when matching text and a column / row. A higher similarity score indicates a match between the two, while a lower score indicates a mismatch between the two.

[0080] During the training process, we use the self-attention learning to obtain the table representation of each row data feature [ROW] obtained by Row transformer and each column data feature [COL] obtained by Column transformer ( Figure 6 [ROW] and [COL] in the text). The [CLS] representation is then obtained through the BERT transformer as the text feature representation; finally, a classification function is used to determine whether the two representations match.

[0081] S102: Obtaining data features of each row and each column; the data features of each row are obtained by encoding the cell row vector information; the data features of each column are obtained by encoding the cell column vector information.

[0082] In S102, the cell row vector information is encoded through the RowTransformer model to obtain the data features of each row; the cell column vector information is encoded through the ColumnTransformer model to obtain the data features of each column.

[0083] The process of obtaining the characteristics of each row of data is shown in A1-A2, and combined with Figure 2 Provide explanation.

[0084] A1: Encode the cell row vector information according to the preset row order to obtain the row feature representation; the row feature representation is the feature representation of the cell vector information affected by the row context.

[0085] Among them, according to Figure 2 Each row in the sequence encodes the cell information from left to right. Assume that Row_i contains the vector information of cells such as c_1, c_2, c_n. This sequence is integrated into the RowTransformer model. Then, the same self-attention mechanism as BERT is used to obtain the row feature representations r_1, r_2, r_3, etc. of each cell information affected by the row context.

[0086] A2: Add a first preset character to the row feature representation to obtain the data feature of each row; the first preset character is used to represent the feature of each row of the table.

[0087] Among them, add the following at the beginning of each line of data: Figure 2 The special character [ROW] in , i.e. the first preset character, gets the table features of each row.

[0088] The specific process of obtaining the characteristics of each column of data is shown in B1-B2, and combined with Figure 2 Provide explanation.

[0089] B1: Encode the cell column vector information according to the preset column order to obtain the column feature representation; the column feature representation is the feature representation of the cell vector information affected by the column context.

[0090] Among them, according to Figure 2 Each column encodes the cell information from top to bottom. Assume that Column_i contains the vector information of cells such as c_1, c_2, c_n, and the sequence is fused into the ColumnTransformer unit. Then, the same self-attention mechanism as BERT is used to obtain the column feature representations c_1, c_2, c_3, etc., in which each cell information is affected by the column context.

[0091] B2: Add a second preset character to the column feature representation to obtain the data feature of each column; the second preset character is used to represent the character of each column table feature.

[0092] At the beginning of each column of data, add Figure 2 The special character [COL] in , i.e. the second preset character, gets the table features of each column.

[0093] S103: By using a preset fusion method, the data features of each row and the data features of each column are fused to obtain a table feature representation of the table to be processed; the table feature representation is used to characterize the feature representation of cell vector information affected by the row context and the feature representation affected by the column context.

[0094] Among them, the preset fusion method can be a mean pooling (Mean_pooling) fusion method, or other types of fusion methods. The specific preset fusion method is determined by technical personnel according to actual conditions and is not specifically limited in this application. The preset fusion method of this application is preferably a mean pooling fusion method.

[0095] After being encoded by the RowTransformer model and the ColumnTransformer model, each cell has two different feature representations (in the RowTransformer model, Figure 2 The "JD.com" in the second row represents a table value vector feature. In the ColumnTransformer model, Figure 2 The "JD.com" in the figure represents the vector feature of a table value in the third column. The two granularities of information (the two different feature representations mentioned above) are fused through mean pooling to obtain the final feature representation of the cell.

[0096] In order to obtain the overall table features, the data features of each column [COL], the data features of each row [ROW] and the final feature representation of the cell are fused into a single vector after self-attention learning as the overall feature representation, that is, the table feature representation of the table to be processed.

[0097] S104: Matching the table feature representation and the pre-acquired text feature representation using a preset matching function to obtain a matching result.

[0098] In S104, the preset matching function may be an n-gram character matching function or other matching functions. The specific preset matching function is determined by technicians according to actual conditions and is not limited in this application. The preset matching function of this application is preferably an n-gram character matching function.

[0099] The text vector information is obtained through the pre-trained BERT model, and the text vector information is encoded through the TextTransformer model to obtain the text feature representation.

[0100] Assume that the text contains multiple characters w_i, w_2, w_3…w_n (where n is an integer greater than or equal to 1). After encoding, text feature representations such as h_1, h_2, h_3…h_n are obtained.

[0101] Specifically, the table feature representation and the text feature representation are matched by a preset matching function, and the process of obtaining the matching result is shown in C1-C4.

[0102] C1: Obtain text vector information in the table to be processed and determine text feature representation based on the text vector information.

[0103] For example, reference Figure 2 As shown in the figure, “This Changshou Card 4G package is launched in cooperation with JD.com” is text vector information.

[0104] C2: When matching the table feature representation and the text feature representation using a preset matching function, obtaining a matching similarity score between the table corresponding to the table feature representation and the text corresponding to the text feature representation.

[0105] C3: When the matching similarity score is greater than the preset similarity score, it is determined that the table feature representation and the text feature representation match, and a matching result indicating that the table feature representation and the text feature representation match is generated.

[0106] C4: When the matching similarity score is less than the preset similarity score, it is determined that the table feature representation and the text feature representation do not match, and a result representing the mismatch between the table feature representation and the text feature representation is generated.

[0107] S105: If the matching result is a matching result in which the table feature representation and the text feature representation match, the preset training model is optimized through the matching result, and structured table information in the table to be processed is obtained based on the optimized preset training model.

[0108] In S105, if the table relevance is consistent with the text relevance, a matching result is obtained that represents the matching between the table feature representation and the text feature representation, and the preset training model is optimized through the matching result, and the structured table information in the table to be processed is obtained based on the optimized preset training model.

[0109] During the training process of the preset training model, the model's learning of structured information in text information and tables is optimized by continuously optimizing the matching results.

[0110] The structured information in the table to be processed means that the information in the table to be processed can be decomposed into multiple interrelated components after analysis, and there is a clear hierarchical structure between the components.

[0111] In an embodiment of the present application, the semantic representations of free text and structured tables are jointly learned, and different information source data, such as cell vector information of the table, text vector information, semantic representation of the structured table, etc., are fused from different spaces and levels to obtain multi-granularity information, and the structured information of the table is effectively obtained through this multi-granularity information.

[0112] Based on the above embodiment Figure 1 A method for processing table data is disclosed. The present application also discloses a table data processing system. Figure 7 As shown, the table data processing system includes a first acquisition unit 701 , a second acquisition unit 702 , a fusion unit 703 , a matching unit 704 and a third acquisition unit 705 .

[0113] The first acquiring unit 701 is used to acquire cell vector information in the table to be processed; the cell vector information includes cell row vector information and cell column vector information.

[0114] The second acquisition unit 702 is used to acquire the data features of each row and each column; the data features of each row are obtained by encoding the cell row vector information; the data features of each column are obtained by encoding the cell column vector information.

[0115] The fusion unit 703 is used to fuse the data features of each row and the data features of each column through a preset fusion method to obtain a table feature representation of the table to be processed; the table feature representation is used to characterize the feature representation of cell vector information affected by the row context and the feature representation affected by the column context.

[0116] The matching unit 704 is configured to match the table feature representation with the pre-acquired text feature representation using a preset matching function to obtain a matching result.

[0117] The third acquisition unit 705 is used to optimize the preset training model through the matching result if the matching result is a matching result in which the table feature representation and the text feature representation match, and obtain structured table information in the table to be processed based on the optimized preset training model.

[0118] Furthermore, the first acquiring unit 701 is specifically configured to acquire cell vector information in the table to be processed through a preset training model.

[0119] Furthermore, the second acquiring unit 702 for acquiring the features of each row of data includes a first encoding module and a first adding module.

[0120] The first encoding module is used to encode the cell row vector information according to a preset row order to obtain a row feature representation; the row feature representation is a feature representation of the cell vector information affected by the row context.

[0121] The first adding module is used to add a first preset character to the row feature representation to obtain the data feature of each row; the first preset character is used to represent the character of each row of the table feature.

[0122] Furthermore, the second acquiring unit 702 for acquiring the feature of each column of data includes a second encoding module and a second adding module.

[0123] The second encoding module is used to encode the cell column vector information according to a preset column order to obtain a column feature representation; the column feature representation is a feature representation of the cell vector information affected by the column context.

[0124] The second adding module is used to add a second preset character to the column feature representation to obtain the data feature of each column; the second preset character is used to represent the character of each column table feature.

[0125] Furthermore, the matching unit 704 includes a first acquisition module, a second acquisition module, a first determination module, and a second determination module.

[0126] The first acquisition module is used to obtain text vector information in the table to be processed and determine text feature representation through the text vector information.

[0127] The second acquisition module is configured to acquire a matching similarity score between a table corresponding to the table feature representation and a text corresponding to the text feature representation when matching the table feature representation and the text feature representation using a preset matching function.

[0128] The first determining module is configured to determine that the table feature representation and the text feature representation match when the matching similarity score is greater than a preset similarity score, and generate a matching result indicating that the table feature representation and the text feature representation match.

[0129] The second determining module is configured to determine that the table feature representation and the text feature representation do not match when the matching similarity score is less than a preset similarity score, and generate a result indicating that the table feature representation and the text feature representation do not match.

[0130] In an embodiment of the present application, the semantic representations of free text and structured tables are jointly learned, and different information source data, such as cell vector information of the table, text vector information, semantic representation of the structured table, etc., are fused from different spaces and levels to obtain multi-granularity information, and the structured information of the table is effectively obtained through this multi-granularity information.

[0131] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned table data processing method.

[0132] The embodiment of the present invention further provides an electronic device, the structural diagram of which is shown in FIG. Figure 8 As shown, it specifically includes a memory 801 and one or more instructions 802, wherein the one or more instructions 802 are stored in the memory 801 and are configured to be executed by one or more processors 803 to execute the one or more instructions 802 to perform the above-mentioned table data processing method.

[0133] The specific implementation processes and derivative methods of the above embodiments are all within the protection scope of the present invention.

[0134] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0135] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0136] The above description of the disclosed embodiments will enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein, but is to be construed in the widest manner consistent with the principles and novel features disclosed herein.

[0137] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for processing tabular data, characterized in that: The method comprises: Obtaining cell vector information in a table to be processed; the cell vector information includes cell row vector information and cell column vector information; Obtaining data features for each row and data features for each column; the data features for each row are obtained by encoding the cell row vector information; the data features for each column are obtained by encoding the cell column vector information; By using a preset fusion method, the data features of each row and the data features of each column are fused to obtain a table feature representation of the table to be processed; the table feature representation is used to characterize the feature representation of the cell vector information affected by the row context and the feature representation affected by the column context; Matching the table feature representation with the pre-acquired text feature representation using a preset matching function to obtain a matching result; If the matching result is a matching result in which the table feature representation matches the text feature representation, the preset training model is optimized through the matching result, and the structured table information in the table to be processed is obtained based on the optimized preset training model.

2. The method according to claim 1, characterized in that The step of obtaining cell vector information in a table to be processed includes: Obtain cell vector information in the table to be processed through the preset training model.

3. The method according to claim 1, characterized in that Get the characteristics of each row of data, including: Encoding the cell row vector information according to a preset row order to obtain a row feature representation; the row feature representation is a feature representation of the cell vector information affected by the row context; A first preset character is added to the row feature representation to obtain the data feature of each row; the first preset character is used to represent the character of each row of table features.

4. The method according to claim 1, wherein Get the data characteristics of each column, including: Encoding the cell column vector information according to a preset column order to obtain a column feature representation; the column feature representation is a feature representation of the cell vector information affected by the column context; A second preset character is added to the column feature representation to obtain the data feature of each column; the second preset character is used to represent the character of each column table feature.

5. The method according to claim 1, wherein The matching result is obtained by matching the table feature representation with the pre-acquired text feature representation using a preset matching function, including: Obtaining text vector information in the table to be processed, and determining text feature representation based on the text vector information; When matching the table feature representation with the text feature representation using a preset matching function, obtaining a matching similarity score between the table corresponding to the table feature representation and the text corresponding to the text feature representation; When the matching similarity score is greater than a preset similarity score, determining that the table feature representation and the text feature representation match, and generating a matching result indicating that the table feature representation and the text feature representation match; When the matching similarity score is less than a preset similarity score, it is determined that the table feature representation and the text feature representation do not match, and a result indicating that the table feature representation and the text feature representation do not match is generated.

6. A table data processing system, characterized in that: The system comprises: A first acquiring unit is configured to acquire cell vector information in a table to be processed; the cell vector information includes cell row vector information and cell column vector information; A second acquisition unit is configured to acquire data features of each row and data features of each column; the data features of each row are obtained by encoding the cell row vector information; the data features of each column are obtained by encoding the cell column vector information; a fusion unit, configured to fuse the data features of each row and the data features of each column in a preset fusion manner to obtain a table feature representation of the table to be processed; the table feature representation is used to characterize the feature representation of the cell vector information affected by the row context and the feature representation of the cell vector information affected by the column context; A matching unit, configured to match the table feature representation with the pre-acquired text feature representation using a preset matching function to obtain a matching result; The third acquisition unit is used to optimize the preset training model through the matching result if the matching result is a matching result that matches the table feature representation and the text feature representation, and obtain the structured table information in the table to be processed based on the optimized preset training model.

7. The system according to claim 6, characterized in that The first acquiring unit is specifically configured to: Obtain cell vector information in the table to be processed through the preset training model.

8. The system according to claim 6, wherein: The second acquisition unit for acquiring the characteristics of each row of data includes: A first encoding module is configured to encode the cell row vector information according to a preset row order to obtain a row feature representation; the row feature representation is a feature representation of the cell vector information affected by a row context; The first adding module is used to add a first preset character to the row feature representation to obtain the data feature of each row; the first preset character is used to represent the character of each row of table features.

9. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the table data processing method according to any one of claims 1 to 5.

10. An electronic device, characterized in that: The system comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to cause one or more processors to execute the table data processing method according to any one of claims 1 to 5.

Images