Table structuring method, device and system and storage medium

Abstract

The invention discloses a table structuring method, device and system and a storage medium. The table structuring method comprises the following steps: S1, preprocessing table data; step S2, adopting a Transform architecture and introducing a sparse attention mechanism to construct a table structured model; s3, training the table structured model according to the preprocessed table data; and S4, inputting the pre-processed new table data into the trained table structured model, and outputting a table structure prediction result. By adopting the technical scheme of the invention, the problems of low accuracy, high computing resource consumption, low processing efficiency and the like when the existing table structuring technology is used for processing a complex table are solved.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing technology, and in particular relates to a table structuring method and device, system, and storage medium. Background Art

[0002] In today's digital age, tables, as an important form of data organization and presentation, are widely present in various documents. In many fields such as document understanding, data analysis, and information retrieval, it is crucial to accurately structure tables. However, existing table structuring technologies have many shortcomings. When dealing with complex tables, traditional methods have difficulty effectively capturing long-distance dependencies in tables, resulting in inaccurate recognition of table structures. For example, for complex tables containing a large number of rows and columns, nested structures, and merged cells, the recognition accuracy of existing technologies is low. At the same time, when processing large-scale data, computing resources are consumed too much, and processing efficiency is low, which cannot meet the real-time and high efficiency requirements in practical applications. With the continuous growth of data volume and the increase in the complexity of table structure, these problems have become increasingly prominent, and there is an urgent need for a more advanced table structuring technology to solve them. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a table structuring method, device, system, and storage medium to address the problems of low accuracy, high computational resource consumption, and low processing efficiency in existing table structuring technologies when processing complex tables. By introducing a sparse attention mechanism and an optimized Transformer architecture, the accuracy of table structure recognition is improved, computational complexity is reduced, processing efficiency is increased, and the model's generalization ability is enhanced, making it adaptable to the needs of table structuring in different fields and formats.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A table structuring method, comprising:

[0006] Step S1, preprocessing the table data;

[0007] Step S2: Use the Transformer architecture and introduce the sparse attention mechanism to build a table structured model;

[0008] Step S3: training the table structured model based on the preprocessed table data;

[0009] Step S4: input the pre-processed new table data into the trained table structured model and output the table structure prediction result.

[0010] Preferably, in step S1, if the table data is an image table, OCR technology is used to extract text and infer the table bounding box positioning information based on the text block coordinates; if the table data is a plain text table, key information such as row and column separators are identified through text parsing rules, and the extracted data is normalized.

[0011] Preferably, in step S2, a sparse attention module is embedded in the encoder and decoder of the Transformer; wherein, the encoder extracts tabular data features in combination with the sparse attention module, and the decoder performs grammatical constraint decoding in combination with the encoder output and prior information of the table structure to generate a structured representation; an adaptive sliding window sparse attention is used to constrain the attention range, allowing the current token to only pay attention to the row separator features within the range of its adjacent k rows or the column separator features within the range of its adjacent m columns, and the local and global structural features are fused through a double-layer spatial semantic sparse attention gating.

[0012] As a preference, the loss function of the trained tabular structured model is for:

[0013]

[0014] Among them, cross-unit merging loss for:

[0015]

[0016] Syntactic Constraint Loss for:

[0017]

[0018] The present invention also provides a table structuring device, comprising:

[0019] A first processing module, configured to pre-process the table data;

[0020] The second processing module is used to build a table structured model using the Transformer architecture and introducing a sparse attention mechanism;

[0021] A third processing module is used to train the table structured model based on the preprocessed table data;

[0022] The fourth processing module is used to input the pre-processed new table data into the trained table structured model and output the table structure prediction result.

[0023] Preferably, the first processing module includes:

[0024] A first processing unit is configured to extract text using an OCR technique and infer table bounding box positioning information based on text block coordinates if the table data is an image table;

[0025] The second processing unit is used to identify key information such as row and column separators through text parsing rules if the table data is a plain text table, and to normalize the extracted data.

[0026] Preferably, a sparse attention module is embedded in the encoder and decoder of the Transformer; the encoder combines the sparse attention module to extract tabular data features, and the decoder combines the encoder output and the table structure prior information to perform grammatical constraint decoding to generate a structured representation; an adaptive sliding window sparse attention is used to constrain the attention range, allowing the current token to only pay attention to the row separator features within the adjacent k rows or the column separator features within the adjacent m columns, and the local and global structural features are fused through a double-layer spatial semantic sparse attention gating.

[0027] As a preference, the loss function of the trained tabular structured model is for:

[0028]

[0029] Among them, cross-unit merging loss for:

[0030]

[0031] Syntactic Constraint Loss for:

[0032]

[0033] The present invention also provides a table structuring system, comprising: a memory and a processor, wherein the memory stores a computer program to be executed by the processor, and the computer program executes the table structuring method when executed by the processor.

[0034] The present invention also provides a storage medium, wherein a computer program is stored on the storage medium, and the computer program executes the table structuring method when running.

[0035] The beneficial effects of the present invention are as follows:

[0036] (1) Improved Accuracy: By introducing a sparse attention mechanism and an optimized Transformer architecture, we can more accurately capture long-range dependencies in tables, effectively improving the accuracy of recognizing complex table structures. Experiments show that compared to traditional methods, our technology improves the accuracy of complex table structuring tasks by 5%-18%.

[0037] (2) Reduced computational resource consumption: The sparse attention processing significantly reduces computational effort, significantly reducing memory usage and significantly improving computational efficiency when processing large-scale tabular data. Under the same hardware conditions, the processing speed is increased by approximately 27%, which can better meet application scenarios with high real-time requirements.

[0038] (3) Enhanced generalization capability: Models trained with a large amount of diverse data have strong generalization capabilities and can adapt to table structures in different fields and formats. There is no need for specialized customized development for specific types of tables, which improves the versatility and applicability of the technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0040] Figure 1 This is a flow chart of a table structuring method according to an embodiment of the present invention;

[0041] Figure 2 This is a flow chart of another table structuring method according to an embodiment of the present invention;

[0042] Figure 3 This is a bank funds flow diagram according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] Example 1:

[0046] like Figure 1 As shown, an embodiment of the present invention provides a table structuring method, comprising the following steps:

[0047] Step S1, Data Preprocessing: Documents containing tables are processed and converted into a format suitable for model input. For image tables, OCR technology is used to extract the text and infer the table's bounding box positioning information based on the text block coordinates. For plain text tables, key information such as row and column separators is identified using text parsing rules. The extracted data is then normalized to unify the data format and remove noise and outliers. For image tables, an improved OCR coordinate correction algorithm is used, and a table grid positioning matrix is ​​generated through bilinear interpolation:

[0048]

[0049] Among them, (x, y) is the original coordinate, (x', y') is the corrected coordinate, a ij ,b ij are learnable parameters.

[0050] Step S2, model construction: Use the Transformer architecture and introduce the sparse attention mechanism to build a table structured model. Embed the sparse attention module in the Transformer encoder and decoder. By sparsifying the attention matrix, the computational complexity is reduced and the model's processing efficiency for tabular data is improved. The encoder and decoder structures are optimized. The encoder combined with the sparse attention module can effectively extract tabular data features. The decoder combines the encoder output and the table structure prior information to perform grammatical constraint decoding to generate a structured representation. That is, the dynamic window sparse attention mechanism is implemented in the Transformer encoder, and the attention score is defined as:

[0051]

[0052] Among them, Attention(Q,K,V) is the result of sparse attention output, Q, K, V are the query, key and value vectors of the input sequence, d k is the dimension of the key vector, M sparse is a doubly constrained sparse mask matrix.

[0053] Local window mask:

[0054]

[0055] Among them, M local(i,j) is the value of position (i, j) in the local window mask matrix, row(i) and row(j) represent the row numbers corresponding to positions i and j respectively, and k is the window radius, which is used to control the context range.

[0056] Semantic Similarity Mask:

[0057]

[0058] Among them, M global(i,j) is the value of position (i, j) in the semantic similarity mask matrix, h i , h j is the semantic vector representation of the content at position i, j in the input, sim(h i ,h j ) represents h i Yu h j The semantic similarity between Indicates that h i The set of top k vectors with the highest similarity.

[0059] Step S3, Model Training: Collect a large amount of tabular data from different fields and structural complexity as a training set and accurately annotate it, marking structural information such as the number of rows and columns, table header information, and cell merging status. Use the annotated data to train the model, using a hierarchical loss function L to measure the difference between the predicted results and the actual structure. Use the backpropagation algorithm and optimizer to adjust the model parameters to improve model accuracy and generalization ability.

[0060]

[0061] in, is the cell-level loss, is the row and column positioning loss, is the grammatical structure loss; λ1, λ2, λ3 are the weight coefficients of the corresponding losses.

[0062] Pooling losses across units Using improved Dice Loss:

[0063]

[0064] Among them, p i is the i-th predicted value, g i is the corresponding true label, and ∈ is a small constant used to prevent division by zero.

[0065] Syntactic Constraint Loss Verify by finite state automaton:

[0066]

[0067] Among them, FSM represents finite state machine, which is used to verify the legality of the structure; 1:t Represents the historical state sequence of the model to the tth moment; y t is the true label at time t.

[0068] Step S4, structured prediction: Improve the coordinate overlap resolution through confidence weighted post-processing, and input the pre-processed new table data into the trained model. The model outputs the table structure prediction result through the collaborative work of the encoder and decoder. In order to improve the bounding box positioning accuracy, a weighted coordinate fusion mechanism is introduced. Specifically, each candidate box bbox i is assigned weight w i , the weight is determined by the OCR confidence αc ocr and semantic similarity s sem The weight calculation uses a linear weighting method, and the fusion factor α controls the importance of the two. The final prediction box is obtained by taking a weighted average of all candidate boxes. The prediction results are post-processed and converted into a convenient format such as HTML or JSON.

[0069]

[0070] Among them, bbox i Indicates the position of the i-th candidate box, α=0.8, c ocr is the OCR confidence, s sem is the semantic similarity.

[0071] As an implementation of an embodiment of the present invention, in step S1, data preprocessing includes:

[0072] S11. When extracting text from an image table using OCR, correct the tilted or distorted table bounding box coordinates using a bilinear interpolation algorithm, and calculate the pixel positions of the center point and four corner coordinates of the text block;

[0073] S12. When parsing plain text tables, match row separators (such as `---` or `===`) and column separators (such as vertical bars `|` or tabs `\t`) according to predefined regular expressions, and identify nested table structures in combination with indentation levels;

[0074] S13. During the normalization process, missing values ​​are filled with the placeholder "[NULL]", and texts in different languages ​​are uniformly converted into Unicode encoding. At the same time, a multi-dimensional tensor input is constructed using the text block coordinates and text length features.

[0075] As an implementation method of an embodiment of the present invention, in step S2, the sparse attention module is implemented as follows:

[0076] (1) During the multi-head attention calculation process, a sparse mask matrix with directional bias is dynamically generated for each attention head. The attention range is constrained by the sliding window mechanism, allowing the current token to only focus on the row separator features within the adjacent k rows or the column separator features within the adjacent m columns.

[0077] (2) A two-layer sparse attention selection mechanism is established. The first layer adopts local window attention based on table structure coordinates, focusing on the local area centered on the current text block and calculating the attention of adjacent cells with a cell spacing < δ; the second layer adopts global sparse attention and dynamically selects the top-k global connections to be retained based on the semantic similarity of row and column headers;

[0078] (3) Adaptively fuse the multi-head sparse attention outputs through a learnable gating network, impose sparse constraints on the attention heads related to the table border, and suppress the attention weights of non-structure related features. The multi-head sparse attention adaptive fusion gate mechanism is defined as;

[0079]

[0080] Among them, g h ∈[0,1] is the gating weight, H is the total number of attention heads, and || represents vector concatenation.

[0081] (4) Tabular grammatical rule priors are introduced in the decoder stage, and the generation logic of the output sequence is constrained by hard-coding the mask matrix to ensure that the hierarchically nested tag structure conforms to grammatical specifications such as HTML or JSON.

[0082] As an implementation of an embodiment of the present invention, in step S3, the optimization strategy for model training includes:

[0083] (1) Using a hierarchical loss function, the loss weight for cell merging errors is increased by 3 times, and Focal Loss is designed for the row and column number prediction task to alleviate class imbalance;

[0084] (2) Adopt a dynamic course learning strategy, start training with simple table structure samples, and gradually increase the proportion of difficult samples such as cross-page tables and nested tables;

[0085] (3) Load the pre-trained LayoutLMv3 model weights as encoder initialization parameters and freeze the first three layer parameters to improve training stability;

[0086] (4) When using the AdamW optimizer, the learning rate of the sparse attention head parameters is set lower than that of the fully connected layer (ratio 1:5) to prevent overfitting.

[0087] As an implementation of an embodiment of the present invention, in step S4, the structured prediction post-processing includes:

[0088] (1) Check the HTML tag sequence generated by the decoder through a finite state machine (FSM) and automatically repair missing closing tags or hierarchical misaligned tags;

[0089] (2) When outputting in JSON format, the row and column indexes are calculated based on the cell coordinates. If multiple rows / columns are merged, the `rowspan` or `colspan` attribute is added;

[0090] (3) Start the conflict detection module for cells whose bounding box coordinates overlap more than the threshold τ, and perform voting fusion with OCR confidence and semantic similarity as weights;

[0091] (4) When returning structured results, the location confidence score of each cell is attached so that the downstream system can filter low-quality prediction areas according to the threshold.

[0092] The embodiment of the present invention constructs a Transformer model that integrates a sparse attention mechanism to perform efficient feature extraction and structural analysis on tabular data. The innovative introduction of a sparse attention weight dynamic allocation mechanism and a table structure syntax constraint decoding method significantly improves the processing efficiency and accuracy of complex tables. The adaptive sliding window sparse attention is used to reduce the computational complexity to O(nlogn), and the local and global structural features are fused through double-layer spatial semantic attention gating. While optimizing the Transformer architecture to enhance the recognition ability of complex table structures, combined with table data preprocessing, model training optimization and structured prediction output processes, accurate and efficient table structured processing is achieved, which can be widely used in document understanding, corporate financial report analysis and other fields, and solves the problems of low accuracy and high computing resource consumption in the existing technology when processing complex tables.

[0093] Example 2:

[0094] for Figure 3 Take the bank funds flow table as an example, Figure 2 As shown, an embodiment of the present invention further provides a table structuring method, including:

[0095] Step 1: Data preprocessing

[0096] (1) Image table processing

[0097] (1.1) OCR recognition

[0098] by Figure 3 Take the bank funds flow table as an example. The table exists in the form of an image and contains multiple merged cells and multi-level headers. First, using advanced optical character recognition (OCR) technology, the present invention uses PaddleOCR to recognize text blocks in the image and obtain text block information and coordinates. For example, a text block "operating income" is identified, and its coordinates are the upper left corner (x1, y1), the upper right corner (x2, y1), the lower left corner (x1, y2), the lower right corner (x2, y2), and the center coordinates are

[0099] (1.2) Coordinate correction and text block center coordinate calculation

[0100] The table bounding box coordinates are corrected using a bilinear interpolation algorithm to effectively handle tilted or distorted tables and ensure that the pixel positions of the center point and the four corners of the text block are accurate. The coordinates of the upper left corner, upper right corner, lower left corner, lower right corner, and center point of the text block are used as position embedding vectors, which together with the text information vector represent the feature vector that constitutes the text block. The coordinate correction calculation is performed using formula (1) to ensure that the pixel position of each coordinate is accurate. The formula is:

[0101] x corr =x orig ×θ x ,y corr =y orig ×θ y

[0102] Among them, the original coordinates are (x orig ,y orig ), the corrected coordinates are (x corr ,y corr ).

[0103] (1.3) Spatial embedding generation

[0104] Convert the row number (row_num) and column number (col_num) of the text block into a learnable embedding vector, and generate a spatial encoding vector (space_merge) for the merged cells. For example, for a cell that spans two rows, generate a corresponding spatial encoding vector to indicate the range and position of its merging.

[0105] (1.4) Feature vector fusion

[0106] The position coordinate vector, text information vector, and spatial embedding vector are combined to form the final feature vector. For example, the vector representation of the text "operating income", the position coordinate vector, and the spatial embedding vector are combined to form a comprehensive feature vector V_input.

[0107] (2) Plain text table processing

[0108] (2.1) Text parsing and normalization

[0109] For plain text tables in the financial sector, such as stock quotes, carefully designed text parsing rules accurately identify key information such as row and column separators. During the normalization phase, the extracted table data is uniformly formatted, for example, by standardizing numbers and dates in different formats to remove noise and outliers.

[0110] (2.2) Data enhancement and feature construction

[0111] Specifically, missing values ​​are filled with the placeholder "[NULL]," and text in different languages ​​is uniformly converted to Unicode. Furthermore, multidimensional tensor input is constructed using text block coordinates and text length features to ensure data standardization and integrity. For example, the stock code "600000" has coordinates (row 3, column 1) in the table, and the text length is 6. These features, along with the text information of the stock code, will constitute the input data.

[0112] Step 2: Model construction

[0113] (1) Transformer architecture and sparse attention mechanism

[0114] (1.1) Model architecture initialization

[0115] We use the Transformer architecture and introduce a sparse attention mechanism to build a tabular structured model. By embedding the sparse attention module in the Transformer encoder and decoder, we reduce computational complexity and improve the model's efficiency in processing tabular data by sparsifying the attention matrix.

[0116] (1.2) Local attention window calculation

[0117] The fused feature vector is input into the local attention window, which contains two sparse mask matrices, corresponding to the sparse masks in the row direction and column direction respectively. Through these two sparse mask matrices, attention calculation is performed only on the adjacent n rows and m columns in the row and column directions. For example, in the row direction, the current token only pays attention to the tokens in the n rows above and below it; in the column direction, the current token only pays attention to the tokens in the m columns to the left and right of it. The local window mask matrix M is defined using formula (2) local(i,j) , where the value of position (i, j) is determined by the difference between the row and column numbers and the window radius r:

[0118]

[0119] (1.3) Multi-head attention calculation and feature splicing

[0120] In the local attention window, multi-head attention calculation is performed to obtain attention features in the row and column directions. These attention features are spliced ​​to form a 2D-dimensional feature vector. The spliced ​​2D-dimensional feature vector is linearly projected and reduced back to d-dimension to achieve alignment of the feature space. In the invention, a fully connected layer is used to map the 2D-dimensional feature vector back to d-dimension. Formula (3) is used for multi-head attention calculation:

[0121] MultiHead(Q,K,V)=Concat(head1,…,head h )W o

[0122] in,

[0123] (1.4) Global semantic screening

[0124] The reduced feature vector is input into the global semantic screening module, and the semantic similarity with the true value is calculated by cosine similarity, and the top-K results are screened. Specifically, the semantic feature vector of the true value is V_true. Formula (4) is used to calculate the cosine similarity between the reduced input feature vector V_input_cur and V_true, and the K feature vectors with the highest similarity are selected.

[0125]

[0126] (1.5) Cross-attention calculation

[0127] The cross-attention calculation is performed on the filtered TOP-K feature vectors to obtain the attention weights of the row and column names of specific rows and columns. Specifically, the correlation between the row and column names is calculated through the cross-attention mechanism using formula (5) to obtain an attention weight matrix.

[0128]

[0129] (1.6) Dynamic Gating Network Fusion

[0130] The feature vector obtained by the cross-attention calculation is input into the dynamic gating network. The weight is calculated based on the coordinate vector and spatial embedding, and the weighted calculation is performed to adjust the importance ratio of visual and text features. The dynamic gating network learns a weight vector V_weight based on the input feature vector to adjust the weights of visual features and text features, and finally outputs the fused feature vector. The weight calculation of the dynamic gating network is defined using formula (9):

[0131] g=σ(W g [h vis ;h text ]+b g )

[0132] Among them, h vis and h text represent visual features and text features respectively, and σ is the activation function.

[0133] (2) Syntax Constraint Decoding and Post-Processing

[0134] (2.1) Syntax Constraint Decoding

[0135] The feature vectors output by the dynamic gating network are input into the decoder for grammar-constrained decoding. Only the results that meet the grammar constraints are output to the post-processing layer. That is, when generating a structured table structure such as HTML or JSON, the grammar constraint loss is calculated using formula (7) to ensure that the hierarchical structure of the table header is correct and the cell merging complies with the specifications.

[0136]

[0137] where y t represents the label predicted at the t-th moment, y<t represents the previously predicted label sequence, x represents the input feature, and FSM represents the constraint rules of the finite state machine. This formula calculates the log-likelihood loss of the model predicting the correct label sequence considering the FSM constraints, thus prompting the model to learn a decoding path that conforms to the grammar specifications.

[0138] (2.2) Post-processing optimization

[0139] Calculate the position confidence score using formula (8) and post-process the results output by the decoder, including automatic repair of the HTML tag sequence, calculation and attribute addition of row and column indexes in JSON format, conflict detection and fusion of bounding box coordinate overlaps, and addition of the position confidence score. In the present invention, a method of using a finite state machine (FSM) to check and repair the HTML tag sequence to ensure that the generated HTML code is correct is adopted.

[0140] confidence = α·OCR confidence + β·semantic_similarity

[0141] where α and β are weight coefficients, representing the importance of the OCR confidence and semantic similarity respectively.

[0142] Step three, model training

[0143] (1) Data collection and annotation

[0144] (1.l) Data collection

[0145] Collect a large amount of table data from different financial fields with different structural complexities to construct a training dataset. These data include bank statements, corporate financial statements, stock market data tables, etc.

[0146] (1.2) Data annotation

[0147] Detailed annotations are made to these tabular data, accurately marking each table's structural information, including the number of rows and columns, header information, cell merging status, etc. For example, for a company's financial statements, information such as the header hierarchy, the merge range of each cell, and the data type are marked.

[0148] (2) Hierarchical Loss Function and Optimizer

[0149] (2.1) Loss function and optimizer selection

[0150] During the training process, a hierarchical loss function is used to measure the difference between the model prediction results and the actual table structure. For example, the loss weight for cell merging errors is increased by 3 times, and Focal Loss is designed for the row and column number prediction task to alleviate class imbalance. The hierarchical loss function is defined using formula (5):

[0151]

[0152] in, is the cell-level loss, is the row and column positioning loss, is the grammatical structure loss; λ1, λ2, λ3 are the corresponding loss weight coefficients.

[0153] At the same time, the AdamW optimizer is used in the present invention to optimize the model training process and accelerate the model convergence. Experiments show that this effectively prevents the occurrence of overfitting.

[0154] (2.2) Dynamic course learning strategies

[0155] It should be noted that during training, the present invention adopts a dynamic course learning strategy, starting with simple table structure samples and gradually increasing the proportion of high-difficulty samples such as cross-page tables and nested tables to improve the generalization ability of the model.

[0156] (3) Pre-training model and learning rate setting

[0157] (3.1) Loading pre-trained model

[0158] Load the pre-trained LayoutLMv3 model weights as the encoder initialization parameters and freeze the first three layer parameters to improve training stability.

[0159] (3.2) Learning rate setting

[0160] When using the AdamW optimizer, the learning rate of the sparse attention head parameters is set lower than that of the fully connected layer (ratio 1:5) to prevent overfitting.

[0161] Step 4: Structured Prediction

[0162] (1) Preprocessing and model input

[0163] When a fully trained model receives new financial table data, it first performs the preprocessing described in step 1 to convert it into a format acceptable to the model. Specifically, for a new image of a bank's funds flow chart, it first performs OCR recognition and coordinate extraction to construct a feature vector. For plain text tables, it performs text parsing and normalization to construct a multi-dimensional tensor input, and then feeds the preprocessed data into the model.

[0164] (2) Model calculation and prediction result output

[0165] (2.1) Model calculation

[0166] The model outputs the prediction result (result_head) of the table structure through the collaborative operation of the encoder and decoder. During the decoding process, the model generates a structured representation (result_word) that conforms to the specification based on the learned table grammar rules. Finally, the final structured prediction result is calculated according to formula (10):

[0167] result final =Decoder(result head ,result word )

[0168] (2.2) Prediction result output

[0169] For the bank funds flow table, the result_word generated by the model can accurately predict the hierarchical structure of the table header, the merging of each cell, and the row and column attribution of the data.

[0170] Step 5: Post-processing

[0171] When post-processing the prediction results, the structured representation of the model output is converted into a format that is easy to understand and use, generating a table structure description in HTML or JSON format to facilitate subsequent integration with other systems and further processing. The specific steps are as follows:

[0172] (1) Automatic repair of HTML tag sequence

[0173] When structuring financial tables, generating accurate HTML tag sequences is crucial for subsequent data display and analysis. A finite state machine (FSM) checks the HTML tag sequences generated by the decoder, automatically repairing missing closing tags or misplaced tags. For example, if the generated HTML code contains unclosed tags, the FSM can automatically detect and add the corresponding tags to ensure the correctness of the HTML code.

[0174] (2) JSON format optimization

[0175] For table-structured results output in JSON format, row and column indices are calculated based on cell coordinates. If multiple rows or columns are merged, the rowspan or colspan attributes are added. Specifically, in the JSON structure of a corporate financial report, if a cell spans two rows, the rowspan=2 attribute is added to it to ensure that the merged cell can be correctly identified and handled in subsequent data processing.

[0176] (3) Bounding box coordinate overlap conflict detection and fusion

[0177] When processing complex financial tables, bounding box coordinates may overlap, for example, when the coordinate ranges of different cells intersect. For cells whose bounding box coordinates overlap by more than a threshold τ, the conflict detection module is activated, and a voting fusion is performed using OCR confidence and semantic similarity as weights. For example, if the coordinates of two cells overlap and the OCR confidence is 0.9 and 0.8, respectively, and the semantic similarity is 0.85 and 0.75, respectively, the fused cell information is calculated based on the weights to ensure the accuracy and reliability of the final result.

[0178] (4) Position confidence score addition

[0179] When returning structured results, a location confidence score is appended to each cell, allowing downstream systems to filter out low-quality predictions based on thresholds. In the generated HTML or JSON structure, a confidence attribute is added to each cell, indicating the confidence level of the cell's location prediction. Downstream systems can filter out high-quality predictions based on the set confidence threshold, improving the overall credibility and usability of the data.

[0180] Example 3:

[0181] An embodiment of the present invention further provides a table structuring device, comprising:

[0182] A first processing module, configured to pre-process the table data;

[0183] The second processing module is used to build a table structured model using the Transformer architecture and introducing a sparse attention mechanism;

[0184] A third processing module is used to train the table structured model based on the preprocessed table data;

[0185] The fourth processing module is used to input the pre-processed new table data into the trained table structured model and output the table structure prediction result.

[0186] As an implementation of an embodiment of the present invention, the first processing module includes:

[0187] A first processing unit is configured to extract text using an OCR technique and infer table bounding box positioning information based on text block coordinates if the table data is an image table;

[0188] The second processing unit is used to identify key information such as row and column separators through text parsing rules if the table data is a plain text table, and to normalize the extracted data.

[0189] As an implementation method of an embodiment of the present invention, a sparse attention module is embedded in the encoder and decoder of the Transformer; wherein, the encoder combines the sparse attention module to extract tabular data features, and the decoder combines the encoder output and the table structure prior information to perform grammatical constraint decoding to generate a structured representation; adaptive sliding window sparse attention is used to constrain the attention range, only allowing the current token to pay attention to the row separator features within the adjacent k rows or the column separator features within the adjacent m columns, and the local and global structural features are fused through double-layer spatial semantic sparse attention gating.

[0190] As an implementation method of an embodiment of the present invention, the loss function of the trained table structured model is for:

[0191]

[0192] Among them, cross-unit merging loss for:

[0193]

[0194] Syntactic Constraint Loss for:

[0195]

[0196] Example 4:

[0197] An embodiment of the present invention further provides a table structuring system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a table structuring method when executed by the processor.

[0198] Example 5:

[0199] An embodiment of the present invention further provides a storage medium having a computer program stored thereon, wherein the computer program executes the table structuring method when running.

[0200] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A table structuring method, characterized in that: include: Step S1, preprocessing the table data; Step S2: Use the Transformer architecture and introduce the sparse attention mechanism to build a table structured model; Step S3: training the table structured model based on the preprocessed table data; Step S4: input the pre-processed new table data into the trained table structured model and output the table structure prediction result.

2. The table structuring method according to claim 1, wherein: In step S1, if the table data is an image table, OCR technology is used to extract text and infer the table bounding box positioning information based on the text block coordinates; if the table data is a plain text table, key information such as row and column separators are identified through text parsing rules, and the extracted data is normalized.

3. The table structuring method according to claim 2, wherein: In step S2, a sparse attention module is embedded in the encoder and decoder of the Transformer; the encoder combines the sparse attention module to extract tabular data features, and the decoder combines the encoder output and the table structure prior information to perform grammatical constraint decoding to generate a structured representation; an adaptive sliding window sparse attention is used to constrain the attention range, allowing the current token to only pay attention to the row separator features within the adjacent k rows or the column separator features within the adjacent m columns, and the local and global structural features are fused through a double-layer spatial semantic sparse attention gating.

4. The table structuring method according to claim 3, wherein: The loss function of the trained tabular structured model for: Among them, cross-unit merging loss for: Syntactic Constraint Loss for:

5. A table structuring device, characterized in that: include: A first processing module, configured to pre-process the table data; The second processing module is used to build a table structured model using the Transformer architecture and introducing a sparse attention mechanism; A third processing module is used to train the table structured model based on the preprocessed table data; The fourth processing module is used to input the pre-processed new table data into the trained table structured model and output the table structure prediction result.

6. The table structuring device according to claim 5, wherein: The first processing module includes: A first processing unit is configured to extract text using an OCR technique and infer table bounding box positioning information based on text block coordinates if the table data is an image table; The second processing unit is used to identify key information such as row and column separators through text parsing rules if the table data is a plain text table, and to normalize the extracted data.

7. The table structuring device according to claim 6, wherein: A sparse attention module is embedded in the encoder and decoder of the Transformer. The encoder combines the sparse attention module to extract tabular data features, and the decoder combines the encoder output and prior information of the table structure to perform grammatical constraint decoding to generate a structured representation. Adaptive sliding window sparse attention is used to constrain the attention range, allowing the current token to only pay attention to the row separator features within the adjacent k rows or the column separator features within the adjacent m columns, and a double-layer spatial semantic sparse attention gating is used to fuse local and global structural features.

8. The table structuring device according to claim 7, wherein: The loss function of the trained tabular structured model for: Among them, cross-unit merging loss for: Syntactic Constraint Loss for:

9. A table structuring system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the table structuring method according to any one of claims 1 to 4 is executed.

10. A storage medium, characterized in that: The storage medium stores a computer program, which executes the table structuring method according to any one of claims 1 to 4 when running.

Images