Table information extraction method and system for target attribute details table and medium
By employing a model-plus-path query approach, utilizing the BERT model, convolutional neural networks, and global pointer networks to extract table information, the accuracy and versatility issues of auction table information extraction are resolved, achieving efficient and accurate data acquisition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HEFEI DAZHIHUI CAIHUI DATA TECH CO LTD
- Filing Date
- 2024-09-13
- Publication Date
- 2026-07-24
AI Technical Summary
Existing technologies suffer from insufficient accuracy and versatility in extracting information from auction tables, especially due to the difficulty in adapting information extraction to all situations caused by differences in the format and content of auction activities.
An information extraction method combining model and path query is adopted. The BERT model is used for text encoding and feature extraction, and convolutional neural network and global pointer network are combined for cell classification. Table data is obtained through path query, feature matrix is constructed and path query is performed to obtain complete data sets.
It improves the accuracy and completeness of information extraction, can handle complex tabular data, is particularly suitable for the auction field, and enhances the model's generalization ability and robustness.
Smart Images

Figure CN119271815B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of natural language processing and information extraction technology, specifically a method, system, and medium for extracting tabular information from a table of object attribute details. Background Technology
[0002] With the rapid development of e-commerce and internet technology, auctions have become increasingly popular as an important method of commodity transaction, resulting in a massive surge of transaction data on online platforms. In the auction field, the auction item attribute details table typically contains a large amount of key information such as product details, price data, and bidder information. Traditional manual extraction methods are not only time-consuming and labor-intensive, but also struggle to guarantee the accuracy and completeness of the data. Therefore, developing an efficient and accurate auction table information extraction technology is of great significance for improving the efficiency and accuracy of auction activities.
[0003] Despite progress in information extraction technology, several problems and challenges remain in extracting information from auction tables. For example, the format and content of tables often vary depending on the auction activity, making it difficult for general information extraction algorithms to adapt to all situations. Furthermore, the naming of proper nouns differs across platforms, making it difficult to distinguish them using simple, unified rules, thus limiting the accuracy and versatility of table information extraction. Therefore, these issues urgently need to be addressed. Summary of the Invention
[0004] To address the technical problem that the accuracy and versatility of table information extraction in existing technologies need to be improved, this invention provides a method, system, and medium for extracting table information from a subject's attribute details table.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] This invention discloses a method for extracting tabular information from a table of object attribute details, comprising:
[0007] Obtain the HTML document of the target table, and preprocess the HTML document to obtain the text string of the target table;
[0008] Multiple features are extracted from the text string to construct a feature matrix; the multiple features include text features, positional features, and tag features.
[0009] The feature matrix is input into the trained cell classification model, and the output is the classification results of all cells in the target table.
[0010] Based on the cell category determination results, a path query is performed in all cells of the target table to obtain all data information in the target table that can form groups.
[0011] As a further improvement to the above scheme, the table information extraction method further includes the following steps before constructing the feature matrix:
[0012] The text string is split into several tokens to form a token sequence;
[0013] Using a predefined vocabulary, each token in the token sequence is mapped to a unique identifier.
[0014] Padded token sequences to the maximum fixed length;
[0015] Create an attention mask for the padded token sequence to distinguish between the actual token and the padded token.
[0016] As a further improvement to the above solution, the extraction of multiple features from the text string includes:
[0017] The text string is encoded using the BERT model, and a first text feature, a second text feature, and a third text feature are extracted. These text features are used as inputs to the convolutional neural network and the global pointer network. The first text feature is used to represent the numerical sequence of the text content; the second text feature is used to indicate the actual token in the token sequence; and the third text feature is used to indicate the token in the token sequence that corresponds to the first cell of the row and the first cell of the column.
[0018] Extract the index position of the token corresponding to the non-filled cell in the target table in the token sequence to form a position feature;
[0019] Extract one or more tags to each cell to form tag features; wherein, the tags are divided into indicator name tags or value tags, and the indicator name tags have at least one level.
[0020] As a further improvement to the above scheme, the training method for the cell classification model includes:
[0021] Initialize the pre-trained BERT model;
[0022] Cell category discrimination is performed based on convolutional neural networks and global pointer networks; convolutional neural networks are used to capture local features and patterns, while global pointer networks are used to determine the global relationships between different elements in the table.
[0023] Select the Adam optimizer to update the parameters of the cell classification model;
[0024] The cell classification model is trained using the training dataset, and its performance is periodically evaluated on the validation dataset to monitor for overfitting.
[0025] The hyperparameters are adjusted based on the performance of the cell classification model on the validation set; the hyperparameters include the learning rate, batch size, and number of layers.
[0026] Iterative perturbation using PGD is employed to add adversarial training;
[0027] The final performance of the cell classification model is evaluated on an independent test dataset using preset metrics. Once the evaluation is passed, the trained cell classification model is obtained. The preset metrics include accuracy, recall, and F1 score.
[0028] As a further improvement to the above scheme, the category determination result of each cell includes four parameters, in order: label category, cell row, cell column, and label confidence.
[0029] As a further improvement to the above scheme, the path query is performed in all cells of the target table based on the cell category discrimination result to obtain all data information in the target table that can form groups, including:
[0030] S41. Locate any initial cell with a first-level indicator name label, and search for the next cell to the right of the initial cell;
[0031] S42. If the next cell is a cell with a value label, then the initial cell and the cell with the value label are grouped together, and the data information of this group is obtained horizontally;
[0032] S43. If the next cell is a cell with a secondary indicator name label, continue searching to the right, and satisfy the condition that the indicator name label level decreases between cells, until a cell with a value label appears. Form a group with cells whose indicator name label levels decrease and cells with value labels, and obtain the data information of this group horizontally.
[0033] S44. If the next cell is still a cell with a first-level indicator name label, it is determined that the current path query to the right of the initial cell has failed, and then step S45 is executed;
[0034] S45. Search for the next cell below the initial cell, and refer to steps S42 to S43 to determine whether a group can be formed in the cells along the vertical direction. If so, obtain the data information of that group in the vertical direction.
[0035] S46. Following the steps S41 to S45, traverse all cells in the target table that have labels with first-level indicator names to obtain all data information in the target table that can form groups.
[0036] As a further improvement to the above solution, the preprocessing of the HTML document to obtain the text string of the target table includes:
[0037] Remove garbled characters from HTML documents and perform uniform conversion of punctuation marks;
[0038] Use regular expressions to obtain the complete text string of the target table;
[0039] Remove unnecessary tags from the HTML document;
[0040] Fill blank cells in the target table to create filled cells and unfilled cells.
[0041] The present invention also discloses a tabular information extraction system for a subject property attribute details table, comprising: a data acquisition module and a data processing module.
[0042] The data acquisition module is used to obtain the HTML document of the target table and preprocess the HTML document to obtain the text string of the target table.
[0043] The data processing module is used to extract multiple features from the text string to construct a feature matrix; the multiple features include text features, position features, and label features; the data processing module is also used to input the feature matrix into a trained cell classification model and output the category discrimination results of all cells in the target table; the data processing module is also used to perform path queries in all cells of the target table based on the cell category discrimination results, thereby obtaining all data information in the target table that can form groups.
[0044] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, it implements the steps of the method for extracting tabular information from a table of object attribute details as described above.
[0045] Compared with the prior art, the beneficial effects of the present invention are:
[0046] 1. This invention employs a model-plus-path-query information extraction method, achieving higher accuracy than pure model extraction. Path-query allows for understanding the logical structure of tables; for example, when an indicator name has multiple sub-indicators, path-query can extract the complete data set in hierarchical order. This method improves the accuracy and completeness of information extraction, enabling the handling of tables with complex structures, and is particularly suitable for table data extraction tasks in the auction field, thereby enhancing the versatility of table information extraction.
[0047] 2. This invention utilizes the BERT model to automatically encode and extract features from the text strings of tables, reducing the workload of manual feature engineering, and enhancing the model's generalization ability and robustness through adversarial training. Attached Figure Description
[0048] Figure 1 This is a flowchart of the method for extracting tabular information from the target object attribute details table in Embodiment 1 of the present invention.
[0049] Figure 2 This is a structural diagram of the table information extraction system for the target object attribute details table in Embodiment 2 of the present invention. Detailed Implementation
[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0051] Example 1
[0052] Please see Figure 1 This embodiment provides a method for extracting tabular information from a table of object attribute details, including steps S1 to S4.
[0053] S1. Obtain the HTML document of the target table, and preprocess the HTML document to obtain the text string of the target table. The preprocessing process includes steps S11 to S14.
[0054] S11. Remove garbled characters from HTML documents and perform uniform conversion of punctuation marks.
[0055] S12. Use regular expressions to obtain the complete text string of the target table.
[0056] S13. Remove unnecessary tags from the HTML document, keeping only special tags such as colspan and rowspan.
[0057] S14. Fill the blank cells in the target table with the content "PAD", thus creating filled cells and unfilled cells.
[0058] After preprocessing, the table information extraction method may also include the following data loading and preliminary processing steps, namely S15 to S18.
[0059] S15. Using a tokenizer to divide a text string into several tokens, forming a token sequence, is fundamental to natural language processing. Mathematically, this can be viewed as a mapping process, mapping the input text T to the token sequence t1, t2, ..., t3. n Above, n is the maximum fixed length.
[0060] S16. Using a predefined vocabulary V, map each token in the token sequence to a unique identifier. This is a mapping from the vocabulary to a set of integers, i.e., V: t → id.
[0061] S17. Sequence padding: To ensure the uniformity of the input sequence, the token sequence is padded to a maximum fixed length l; mathematically, this can be expressed as P(t) = t◇[0] l-|t| Where ◇ represents sequence concatenation; P(t) represents the processing performed on each token, converting it into a numerical form that the BERT model can understand; [0] represents padding terms used to achieve the fixed sequence length required by the model.
[0062] S18. Create attention masks (i.e., attention_masks) for the padded token sequence to distinguish between the actual tokens and the padded tokens. This helps the model ignore the padded parts and focus on the effective information.
[0063] S2. After data preprocessing, the tabular text data is converted into a numerical feature representation that the model can process, and feature extraction is performed, that is, extracting multiple features from the text string to construct a feature matrix; the multiple features include text features, positional features, and label features. Step S2 may include S21 to S23.
[0064] S21. The text string (i.e., the text content in the table, such as the auction item name, certificate information, etc.) is encoded using the BERT model, and the first text feature, the second text feature, and the third text feature are extracted. The text features are used as input to the convolutional neural network and the global pointer network. The first text feature is used to represent the numerical sequence of the text content; the second text feature is used to indicate the actual token in the token sequence, usually 1 represents the actual token and 0 represents the padding token; the third text feature is used to indicate the token in the token sequence that corresponds to the starting cell of the row and the starting cell of the column, so that the start of different rows or columns can be distinguished for the table data.
[0065] S22. Extract the index position of the token corresponding to the non-filled cell in the target table in the token sequence to form positional features, which helps the model understand the relative position of the cell in the table.
[0066] S23. Extract one or more labels belonging to each cell to form label features; wherein, the labels are divided into indicator name labels or value labels. In this embodiment, the indicator name labels have at least one level and at most two levels, and in other embodiments, they can have more than one level.
[0067] Each cell can have multiple labels because there are different types of indicator values, such as rights restriction status - indicator value, auction item name - indicator value, other indicator values, etc. A cell may belong to both special indicator values and ordinary indicator values. When a table contains multiple auction items, which is equivalent to a sub-table, the cells in the first column of the sub-table will be considered to have a "serial identifier" label in addition to the original labels, which serves as an identifier for finding the sub-table.
[0068] As shown in the original text of Table 1, in this table, "Item Name", "Item Owner", and "Rights Restriction Status" are all first-level indicator name labels, "Mortgage Status" is a second-level indicator name label, and "Electric Vehicle", "Zhang San", and "Mortgaged" are all value labels.
[0069] Table 1: Example of the original table text
[0070]
[0071] (1) Use BertTokenizer to encode the text content in Table 1.
[0072] The maximum fixed length of the sequence is l = 32. If the number of tokens in a sequence is less than 32, a [PAD] marker needs to be added to the end of the sequence. The identity code of [PAD] is 0.
[0073] The following is an example of the padded input_ids (identity identifier code corresponding to the token sequence) and attention_mask (attention mask corresponding to the token sequence).
[0074] Regarding the text "electric vehicle":
[0075] input_ids:[101,2687,102,0,0,...,0];
[0076] attention_mask:[1,1,1,0,0,...,0].
[0077] Regarding the text "Zhang San":
[0078] input_ids:[101,3214,102,0,0,...,0];
[0079] attention_mask:[1,1,1,0,0,...,0].
[0080] Regarding the text "No mortgage":
[0081] input_ids:[101,1503,102,1704,102,0,...,0];
[0082] attention_mask:[1,1,1,1,1,0,...,0].
[0083] In the above examples, all are filled with 32 tokens; 101 is the [CLS] token of the BERT model, used for the beginning of the sequence; 102 is the [SEP] token, used for the end of the sequence; 2687, 3214, 1503, and 1704 are the identification codes for the four words "electric vehicle", "Zhang San", "mortgage status", and "no mortgage", respectively.
[0084] (2) Constructing the tag matrix
[0085] The label matrix will be used to record the position of each label in the table. Since the table is 3 rows and 3 columns, the label matrix can be viewed as a two-dimensional matrix with dimensions (number of rows, number of columns). In this example, the label matrix has dimensions (3,3), where the cell corresponding to indicator name 1 is labeled 0, the cell corresponding to indicator name 2 is labeled 1, and the cell corresponding to indicator value is labeled 2. An example of a label matrix is shown below:
[0086] [0,2,2], which represents the first row: ["Indicator Name 1","Indicator Value","Indicator Value"]
[0087] [0,2,2], which is the second line: ["Indicator Name 1","Indicator Value","Indicator Value"]
[0088] [0,1,2], which is the 3rd row: ["Indicator Name 1","Indicator Name 2","Indicator Value"]
[0089] (3) Constructing the location list
[0090] The position list records the index position of each label in the label matrix. In this example, each label corresponds to one position because each label appears only once. An example of the position list is shown below:
[0091] [1, 1], meaning "Indicator Name 1" is in the 1st row and 1st column.
[0092] [1, 2], meaning the "indicator value" is in the 1st row and 2nd column.
[0093] [1, 3], meaning the "indicator value" is in the 1st row and 3rd column.
[0094] [2, 1], meaning "Indicator Name 1" is in the 2nd row and 1st column.
[0095] [2, 2], meaning the "indicator value" is in the 2nd row and 2nd column.
[0096] [2, 3], meaning the "indicator value" is in the 2nd row and 3rd column.
[0097] [3, 1], meaning "Indicator Name 1" is in the 3rd row and 1st column.
[0098] [3, 2], meaning "Indicator Name 2" is in the 3rd row and 2nd column.
[0099] [3, 3], meaning the "indicator value" is in the 3rd row and 3rd column.
[0100] S3. Input the feature matrix into the trained cell classification model and output the category classification results of all cells in the target table. The training method of the cell classification model includes S31 to S37.
[0101] S31. Model initialization: Load the parameters θ of the pre-trained BERT model. The BERT model is able to capture the complex relationships between words.
[0102] S32. Cell category determination is performed based on two classification algorithms: convolutional neural network and global pointer network. Convolutional neural network is used to capture local features and patterns, while global pointer network is used to determine the global relationship between different elements in the table.
[0103] Classification methods based on Convolutional Neural Networks (CNNs) capture local features and patterns, using CNNs to process tabular data. CNNs are particularly well-suited for processing data with a grid-like topology, such as images and tables. The core idea behind this approach is to use convolutional layers to extract local features from the input data. Specifically, a two-dimensional convolution can be represented as:
[0104] (f*g)(i,j)=∑ m ∑ n f(m, n)·g(im, jn)
[0105] In the formula, f is the input data (e.g., a local region of an image or table), g is the convolution kernel, * denotes the convolution operation, and i and are the indices of the output feature map. m represents the stride of the convolution kernel as it slides across the input data; n represents the dimension of the convolution kernel matrix.
[0106] The Global Pointer network is used to determine the relationships between different elements in a table. This is an attention-based model. The Global Pointer network captures global dependencies by calculating the attention score of each element in the input sequence to other elements.
[0107] S33. Using the Adam optimizer, update the parameters of the cell classification model based on the gradient of the loss function. The update rule is as follows:
[0108]
[0109] In the formula, v is the momentum term; v t This represents the momentum estimate at step t; v t+1 η represents the momentum estimate at step t+1, used to consider the exponentially weighted average of past gradients; t This is the adjusted learning rate, and β1 is a hyperparameter in the Adam optimizer used to calculate the first moment estimate; This indicates that in the parameter θ t The gradient of the loss function J at point θ; J represents the loss function; θ t This represents the parameter update at step t; θ t+1 β1 represents the parameter update at step t+1; β2 represents the hyperparameters in the Adam optimizer, used to calculate the second-order moment estimate.
[0110] S34. Train the cell classification model using the training dataset and periodically evaluate the model performance on the validation dataset to monitor for overfitting.
[0111] S35. Adjust the hyperparameters based on the performance of the cell classification model on the validation set; the hyperparameters include the learning rate, batch size, and number of layers.
[0112] S36. Adversarial Training: Adversarial training techniques such as PGD are used to enhance model robustness. The iterative perturbation addition in PGD can be represented as:
[0113]
[0114] In the formula, Clip ∈ (·) indicates a clipping operation, ensuring that the updated parameter θ t+1 With the original parameter θ t The difference is kept within the ∈ sphere, which ensures that the magnitude of the perturbation will not exceed a certain limit ∈; α represents the perturbation step size (or learning rate), which controls the magnitude of the perturbation in each iteration; sign represents the sign of the gradient, which is used to determine the direction of the perturbation.
[0115] For example, inputting the feature matrix formed in Table 1 into the cell classification model yields the following results:
[0116] {["Primary Indicator Name Label",0,0,0.9999999999021745],["Value Label",0,1,0.9999999943139526],["Value Label",0,2,0.9999999943129543],["Primary Indicator Name Label",1,0,0.999999999997289],["Value Label",1,1,0.9999999986302166],["Value Label",1,1,0.9999999986302253],["Primary Indicator Name Label",2,0,1.0],["Secondary Indicator Name Label",2,1,0.9999999997916862],["Value Label",2,2,1.0]}.
[0117] Each cell returns four parameters as the category classification result for that cell. The first parameter represents the label category, the second parameter represents the row number of the cell, the third parameter represents the column number of the cell, and the fourth parameter represents the confidence score of the label.
[0118] By introducing adversarial perturbations, models are trained to recognize and adapt to subtle changes in input data, which helps them remain stable when faced with noise and outliers in real-world applications. In the context of tabular information extraction, adversarial training can help models better handle tabular data of different formats and styles, improving the accuracy and reliability of extraction results. This is a method to improve the model's generalization ability and robustness.
[0119] S37. Evaluate the final performance of the cell classification model on an independent test dataset using preset metrics. Once the evaluation is successful, the trained cell classification model is obtained. The preset metrics include accuracy, recall, and F1 score.
[0120] S4. Based on the cell category determination results, perform path queries in all cells of the target table to obtain all data information in the target table that can form groups.
[0121] Due to inherent thinking habits, tables can only be built to the right or downwards. In addition, since the specific types of tables have been found when building the corpus training model, step S4 in this embodiment may include S41 to S46.
[0122] S41. Locate any initial cell with a first-level indicator name label, and search for the next cell to the right of the initial cell;
[0123] S42. If the next cell is a cell with a value label, then the initial cell and the cell with the value label are grouped together, and the data information of this group is obtained horizontally;
[0124] S43. If the next cell is a cell with a secondary indicator name label, continue searching to the right, and satisfy the condition that the indicator name label level decreases between cells, until a cell with a value label appears. Form a group with cells whose indicator name label levels decrease and cells with value labels, and obtain the data information of this group horizontally.
[0125] S44. If the next cell is still a cell with a first-level indicator name label, it is determined that the current path query to the right of the initial cell has failed, and then step S45 is executed;
[0126] S45. Search for the next cell below the initial cell, and refer to steps S42 to S43 to determine whether a group can be formed in the cells along the vertical direction. If so, obtain the data information of that group in the vertical direction.
[0127] S46. Following the steps S41 to S45, traverse all cells in the target table that have labels with first-level indicator names to obtain all data information in the target table that can form groups.
[0128] Example 2
[0129] This embodiment provides a tabular information extraction system for a target object attribute details table, including: a data acquisition module 101 and a data processing module 102.
[0130] The data acquisition module 101 is used to acquire the HTML document of the target table and preprocess the HTML document to obtain the text string of the target table.
[0131] The data processing module 102 is used to extract multiple features from the text string to construct a feature matrix; the multiple features include text features, position features, and label features; the data processing module 102 is also used to input the feature matrix into a trained cell classification model and output the category discrimination results of all cells in the target table; the data processing module 102 is also used to perform path queries in all cells of the target table based on the cell category discrimination results, thereby obtaining all data information in the target table that can form groups.
[0132] In this embodiment, the table information extraction system can implement the steps of the table information extraction method described in Embodiment 1.
[0133] Example 3
[0134] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the steps of the table information extraction method for the target attribute details table in Embodiment 1.
[0135] The computer-readable storage medium may include flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Of course, the storage medium may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.
[0136] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for extracting tabular information from a table of object attribute details, characterized in that, include: Obtain the HTML document of the target table, and preprocess the HTML document to obtain the text string of the target table; Extract various features from the text string to construct a feature matrix; The various features include text features, location features, and tag features; The feature matrix is input into the trained cell classification model, which outputs the classification results of all cells in the target table. The classification result of each cell includes four parameters, in order: label category, row of cell, column of cell, and label confidence. Based on the cell category identification results, a path query is performed in all cells of the target table to obtain all data information in the target table that can form groups; The extraction of multiple features from the text string includes: The text string is encoded using the BERT model, and a first text feature, a second text feature, and a third text feature are extracted. These text features are used as inputs to the convolutional neural network and the global pointer network. The first text feature is used to represent the numerical sequence of the text content; the second text feature is used to indicate the actual token in the token sequence; and the third text feature is used to indicate the token in the token sequence that corresponds to the first cell of the row and the first cell of the column. Extract the index position of the token corresponding to the non-filled cell in the target table in the token sequence to form a position feature; Extract one or more tags to each cell to form tag features; wherein, the tags are divided into indicator name tags or value tags, and the indicator name tags have at least one level; The training method for the cell classification model includes: Initialize the pre-trained BERT model; Cell category discrimination is performed based on convolutional neural networks and global pointer networks; convolutional neural networks are used to capture local features and patterns, while global pointer networks are used to determine the global relationships between different elements in the table. Select the Adam optimizer to update the parameters of the cell classification model; The cell classification model is trained using the training dataset, and its performance is periodically evaluated on the validation dataset to monitor for overfitting. The hyperparameters are adjusted based on the performance of the cell classification model on the validation set; the hyperparameters include the learning rate, batch size, and number of layers. Iterative perturbation using PGD is employed to add adversarial training; The final performance of the cell classification model is evaluated on an independent test dataset using preset metrics. Once the evaluation is passed, the trained cell classification model is obtained. The preset metrics include accuracy, recall, and F1 score.
2. The method for extracting tabular information from a subject's attribute details table according to claim 1, characterized in that, Before constructing the feature matrix, the table information extraction method further includes: The text string is split into several tokens to form a token sequence; Using a predefined vocabulary, each token in the token sequence is mapped to a unique identifier. Padded token sequences to the maximum fixed length; Create an attention mask for the padded token sequence to distinguish between the actual token and the padded token.
3. The method for extracting tabular information from a subject's attribute details table according to claim 1, characterized in that, The method of performing path queries in all cells of the target table based on cell category discrimination results to obtain all data information in the target table that can form groups includes: S41. Locate any initial cell with a first-level indicator name label, and search for the next cell to the right of the initial cell; S42. If the next cell is a cell with a value label, then the initial cell and the cell with the value label are grouped together, and the data information of this group is obtained horizontally; S43. If the next cell is a cell with a secondary indicator name label, continue searching to the right, and satisfy the condition that the indicator name label level decreases between cells, until a cell with a value label appears. Form a group with cells whose indicator name label levels decrease and cells with value labels, and obtain the data information of this group horizontally. S44. If the next cell is still a cell with a first-level indicator name label, it is determined that the current path query to the right of the initial cell has failed, and then step S45 is executed; S45. Search for the next cell below the initial cell, and refer to steps S42 to S43 to determine whether a group can be formed in the cells along the vertical direction. If so, obtain the data information of that group in the vertical direction. S46. Following the steps S41 to S45, traverse all cells in the target table that have labels with first-level indicator names to obtain all data information in the target table that can form groups.
4. The method for extracting tabular information from a subject's attribute details table according to claim 1, characterized in that, The preprocessing of the HTML document to obtain the text string of the target table includes: Remove garbled characters from HTML documents and perform uniform conversion of punctuation marks; Use regular expressions to obtain the complete text string of the target table; Remove unnecessary tags from the HTML document; Fill blank cells in the target table to create filled cells and unfilled cells.
5. A system for extracting tabular information from a table of object attribute details, characterized in that, The table information extraction system implements the table information extraction method as described in any one of claims 1 to 4, and the table information extraction system includes: The data acquisition module is used to acquire the HTML document of the target table and preprocess the HTML document to obtain the text string of the target table. The data processing module is used to extract multiple features from the text string to construct a feature matrix; the multiple features include text features, position features, and label features; the data processing module is also used to input the feature matrix into a trained cell classification model and output the category discrimination results of all cells in the target table; the data processing module is also used to perform path queries in all cells of the target table based on the cell category discrimination results, thereby obtaining all data information in the target table that can form groups.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the table information extraction method for the target attribute details table as described in any one of claims 1 to 4.
Citation Information
Patent Citations
Table index information extraction method based on graph attention network
CN116795833A
Method and device for restoring wireless table in PDF file and storage medium
CN117710997A