Table structure identification method, model training method, system and equipment

Abstract

The invention relates to the technical field of document image analysis, in particular to a table structure recognition method, a model training method, a system and equipment, and the method comprises the steps: receiving a document image, and extracting multi-dimensional features; constructing a node initial vector fusing the position information; global feature aggregation is carried out by introducing a two-dimensional position offset self-attention mechanism; predicting a structural relationship probability based on the updated node characterization; and reconstructing and outputting the spreadsheet file. According to the method, the self-attention mechanism is fused with the original two-dimensional relative position offset, so that the model can simultaneously capture the long-distance semantic dependency and spatial geometry relationship between the text blocks, and the accuracy and robustness of structure recognition of a frameless and complex-layout table are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of document image analysis technology, and more specifically, to a table structure recognition method, model training method, system, and device. Background Technology

[0002] In the wave of global digital transformation, massive amounts of unstructured document images, such as scanned reports, financial statements, academic papers, and invoices, are being transformed into machine-readable electronic data on a large scale. In this process, tables, as an efficient and well-organized form of data organization, are key carriers of core information within documents. Therefore, how to accurately and automatically parse the complete structure of tables from document images—that is, to restore pixel information into electronic tables containing logical information such as cells, row-column relationships, and cross-row / column relationships—has become a core task and research hotspot in the field of document image analysis. Achieving high-precision table structure recognition is not only the foundation for automated data extraction and information entry but also a key bottleneck enabling advanced applications such as downstream big data analysis, knowledge graph construction, and business intelligence decision-making. In recent years, with the development of deep learning technology, methods such as graph neural networks have been used to model text blocks as graph nodes and learn the relationships between nodes through the model, providing a promising technical path to solve this complex problem. This has significant research and application value for improving the efficiency and intelligence level of information processing. However, currently, existing table structure recognition technologies still have significant bottlenecks when handling complex scenarios. On the one hand, rule-based or traditional computer vision methods heavily rely on the explicit borders, lines, and other visual features of tables. When faced with tables without borders, lines, or irregular layouts, their recognition accuracy drops sharply, exhibiting poor robustness. On the other hand, early deep learning methods, such as models based on convolutional neural networks (CNNs) or recurrent neural networks (RNNs), are limited by their network structures. CNNs, due to their inherent local receptive fields, struggle to capture long-distance dependencies between widely spaced cells in a table; while RNNs tend to process text sequentially, which doesn't match the inherent two-dimensional spatial properties of tables. Moreover, even subsequent models based on graph attention mechanisms, while capable of modeling global relationships, typically "flatten" the two-dimensional text block sequence into a one-dimensional sequence before computation. This results in the significant loss of valuable spatial geometric information between text blocks (such as their relative positions up, down, left, and right), making it difficult for the model to distinguish between semantically similar but spatially distinct complex structures, thus limiting its performance ceiling for complex table recognition tasks. Summary of the Invention

[0003] The purpose of this invention is to provide a table structure recognition method, model training method, system and device. By integrating the self-attention mechanism with the original two-dimensional relative position bias, the model can simultaneously capture the long-distance semantic dependencies and spatial geometric relationships between text blocks, which significantly improves the accuracy and robustness of structure recognition for borderless and complex layout tables.

[0004] This invention is achieved through the following technical solution:

[0005] A table structure recognition method, the steps of which include: Receive the document image to be identified, and extract the visual features, semantic features and geometric position features of each text block in the document image through a text detection and recognition model; Based on the aforementioned visual features, text semantic features, and geometric position features, the position information is encoded through a geometric position embedding function and multiple features are concatenated to construct the initial vector of the text block's nodes. Global feature aggregation is performed on the node initial vector. The global feature aggregation is achieved by introducing a self-attention mechanism that includes learnable two-dimensional relative position bias. The scaling dot product attention weights between nodes are calculated and the spatial position bias term is superimposed to obtain an updated node representation that integrates spatial geometric relationships and semantic information. Based on the updated node representation, the probability of structural relationship between any two text blocks is predicted by a relationship discriminant function. The structural relationship includes at least the relationship between the same cell, the same row, and the same column. Based on the structural relationship probabilities, text blocks are combined into cells, rows, and columns through connected component analysis and topology correction, and a spreadsheet file containing the complete structure is reconstructed and output.

[0006] Optionally, the initial vector of the nodes for constructing the text block is given by the following formula:

[0007] in, Let represent the node initialization vector of the i-th text block. Represents visual feature vectors. This represents the recognition of text vectors. This indicates geometric information including center point coordinates, width, height, and relative proportions. It is a learnable geometric position embedding function used to encode the two-dimensional position and size information of nodes.

[0008] Optionally, the introduction includes a self-attention mechanism that can learn two-dimensional relative position biases, specifically employing scaled dot product attention, calculated as follows:

[0009] in, , , H is the node feature matrix. , , The projection matrix is ​​learnable. Let Q be the query vector matrix, representing the current node's attention needs towards other nodes. K is the key vector matrix, representing the feature identifiers of each node, used for matching calculations with the query vector. V is the value vector matrix, representing the actual feature information of the nodes. The aggregated output is then processed after attention weighting. To represent the transpose of the key vector matrix K, it is used to perform a dot product operation with the query vector Q to calculate the relevance score between nodes.

[0010] Optionally, the learnable two-dimensional relative position offset is specifically as follows:

[0011] in, This represents the attention score between node i and node j. Let be the query vector corresponding to node i, obtained by linear transformation of the node features. Let be the key vector corresponding to node j, which is also obtained from the node features through a linear mapping. A learnable mapping function used to measure the difference in lateral distance between nodes. Longitudinal distance difference Width difference and height difference It is transformed into an additivity bias term to achieve the fusion of spatial and semantic information.

[0012] Optionally, the specific calculation formula for predicting the structural relationship probability between any two text blocks using the relation discriminant function is as follows:

[0013] in, This represents the predicted probability that the relationship between node i and node j belongs to the cell, row, or col structure. This is a relation discrimination function used to map the features of node pairs and output a score for the corresponding relation. These correspond to relationships within the same cell, row, and column, respectively. and This represents the updated node representation after L-level aggregation. The function represents the mapping function of node geometric relationships. This is the geometric position information vector corresponding to node j, including spatial features such as the bounding box coordinates, width, and height of the text block.

[0014] Optionally, the reconstructing and outputting of a spreadsheet file containing the complete structure specifically includes: Based on the predicted adjacency matrices of the same cell, row, and column, the connected component analysis algorithm is used to identify independent cell regions. Correct the topological relationships of the identified cell range across rows and columns, and handle merged cells and broken borders; The corrected structure is mapped to HTML code or an Excel document for output.

[0015] A training method for a table structure recognition model, wherein the table structure recognition model is used to perform a table structure recognition method, and the training steps include: Obtain training samples containing text blocks and their true structural relationships; The training samples are input into a predefined table structure recognition model architecture to obtain the predicted structural relationship probabilities; Based on the predicted structural relationship probability and the actual structural relationship, the loss value is calculated using a preset loss function; Based on the loss value, the parameters of the table structure recognition model are updated using the backpropagation algorithm to complete the training of the table structure recognition model.

[0016] Optionally, the loss value is calculated using a preset loss function, and the specific calculation formula is as follows:

[0017] in, , , Here, CE represents the cross-entropy loss, and the weighting coefficients are used. This is a label indicating whether node i and node j belong to the same cell in the actual annotation. The value is 1 when node i and node j belong to the same cell, and 0 otherwise. Let be the probability value predicted by the model that node i and node j belong to the same cell, and its value ranges from [0,1]. Labels are the actual adjacency matrix labels. These are the model's predicted values. , , are the true adjacency matrix labels of nodes i and j in the same row relationship, and , representing the probability values ​​of nodes i and j predicted by the model in the same row relationship.

[0018] A table structure recognition system, including: The text feature construction module is used to receive document images, extract multidimensional features of text blocks, and construct node initial vectors; The structural feature aggregation module is used to model global dependencies and update features of node initial vectors by introducing a self-attention mechanism with two-dimensional relative position bias. The relationship determination module is used to predict the structural relationships between text blocks in the same cell, same row, and same column based on the updated node representation. The structure reconstruction module is used to reconstruct the table topology based on the predicted structural relationships and output an editable electronic document.

[0019] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a table structure recognition method or a training method for a table structure recognition model.

[0020] The technical solution of the present invention has at least the following advantages and beneficial effects: This invention, on the one hand, introduces a self-attention mechanism, enabling parallel computation of dependencies between any two text blocks in an image. This completely eliminates the limitations of local receptive fields or sequential processing, effectively capturing the global contextual information of the entire table. It has a natural advantage in understanding long-distance structural relationships such as those spanning rows and columns. On the other hand, this invention innovatively designs and introduces a learnable two-dimensional relative position bias term, directly integrating it into the calculation of attention scores. This design allows the model to simultaneously and explicitly consider the relative spatial geometric relationships between text blocks on the horizontal and vertical axes while evaluating their semantic connections. Therefore, by deeply coupling semantic information with two-dimensional spatial information, this invention enables the model to accurately perceive the true physical layout of the table, thereby greatly improving its ability to discriminate key structural relationships such as those within the same cell, row, and column. It exhibits excellent recognition performance and generalization ability even in borderless tables with complex layouts. Attached Figure Description

[0021] Figure 1 A flowchart illustrating the table structure recognition method provided by this invention; Figure 2 This is a schematic diagram illustrating the principle of the table structure recognition system provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] The core inventive concept of this invention lies in proposing a novel deep learning-based table structure recognition technology system, aiming to solve the bottleneck problem of recognition accuracy caused by the loss of spatial geometric information when processing complex and irregular tables in existing technologies. This technical solution first treats each text block in the document image as a node in graph theory and extracts multi-dimensional features for each node, including visual, semantic, and geometric positional features. Its key innovation lies in the original introduction of a learnable "two-dimensional relative position bias" term when using a self-attention mechanism to model the global relationship between all nodes. This bias term is directly and additively integrated into the calculation of the attention score, enabling the model to simultaneously and explicitly perceive the relative spatial layout of any two text blocks in a two-dimensional plane when judging the content correlation between them. Through this deep coupling of semantic and spatial information, the model can accurately understand the true physical structure of the table, effectively overcoming the information loss caused by traditional methods that "flatten" the two-dimensional structure into a one-dimensional sequence. Ultimately, based on this node representation that integrates global context and precise spatial awareness, high-precision and robust structured reconstruction of tables with complex cross-row, cross-column, and borderless conditions is achieved through relationship discrimination and topology reconstruction.

[0024] Example 1 Reference Figure 1 , Figure 1 This is a flowchart illustrating a table structure recognition method provided in an exemplary embodiment of the present invention. The method can be applied to computing devices such as servers, personal computers, or embedded devices to automatically convert document images containing tables into machine-readable structured electronic documents. Specifically, the method may include the following steps: Step S100: Receive the document image and extract multidimensional features.

[0025] This step is the data input and preprocessing stage of the entire table structure recognition process. The computing device first receives an image of the document to be recognized, which can be a scanned paper document, a digital photograph, or a page from a PDF file.

[0026] Upon receiving the image, the system invokes a pre-trained text detection and recognition model, often referred to as an Optical Character Recognition (OCR) engine, to analyze the image. The OCR engine's task is to locate all discrete text blocks in the image and identify the text content within each block. A text block can be a word, a number, a phrase, or, in some cases, an entire paragraph of text within a cell.

[0027] For each detected text block, this step requires extracting three types of key features: The first category is visual features. These features are extracted from the image regions corresponding to each text block by the backbone network of deep convolutional neural networks (CNNs), such as ResNet and VGG. Visual features mainly capture image-level information such as font, weight, color, and background texture of the text. This information is very important for distinguishing text with different functions such as titles, body text, and annotations.

[0028] The second category is text semantic features. These features convert the text strings recognized by the OCR engine into high-dimensional semantic vectors using a pre-trained language model, such as BERT or RoBERTa. Semantic features encode the inherent meaning of text blocks, enabling the model to understand the different functions of words like "total" and "unit price" in a table.

[0029] The third category is geometric position features. These features describe the physical location and size of each text block on the entire page. Specifically, it includes at least the bounding box information for each text block, such as the (x, y) coordinates of the top-left corner, the width w, and the height h of the bounding box. To help the model better understand relative positions, it can further calculate and include information such as the normalized center point coordinates, aspect ratio, and the proportion of the text block area to the total page area.

[0030] After this step, the original document image is transformed into a set of structured data, where each data unit corresponds to a text block and includes its complete visual, semantic, and geometric location features. This lays a solid data foundation for subsequent graph model construction and structural relationship analysis.

[0031] Step S200: Construct the initial vector of nodes with fused location information.

[0032] After extracting the multidimensional features of each text block in step S100, the goal of this step is to integrate these discrete features into a unified vector that represents the initial state of each text block, i.e., the node initial vector. From the perspective of graph theory, each text block is regarded as a graph node, and this initial vector is the initial representation of the node before the graph neural network computation begins.

[0033] Specifically, for the i-th text block, its node initialization vector It is constructed through a multi-feature concatenation operation. This construction process can be precisely described by the following formula:

[0034] in, This represents the initial vector of the node for the i-th text block. The superscript 0 indicates that this is the 0th layer, i.e., the feature representation of the input layer. Represents visual feature vectors. This represents the recognition of text vectors. It represents geometric information including the coordinates of the center point, width, height, and relative proportions. It is a crucial geometric location embedding function. Due to the geometric location features These are the original coordinates and dimensions; directly concatenating them with other high-dimensional features may lead to inconsistencies in numerical scale and difficulties in model learning. Therefore, an embedding function is needed. This is mapped to a high-dimensional space compatible with visual and semantic feature dimensions. This function typically consists of one or more fully connected layers, and its parameters are learnable. Through continuous optimization during model training, this function learns how to most effectively encode positional information, enabling it to be understood and utilized by subsequent self-attention modules.

[0035] By performing this step, the system generates a high-dimensional initial vector for each text block in the image. This vector not only contains the content and appearance information of the text block itself, but also encodes its spatial location information through a learnable embedding function, providing the necessary prerequisite for the subsequent model to accurately perceive and infer the two-dimensional spatial structure of the table.

[0036] Step S300: Global feature aggregation is performed by introducing a self-attention mechanism with two-dimensional positional bias.

[0037] This step is the core of the technical solution of this invention. Its goal is to establish global dependencies among all text block nodes and update the feature representation of each node so that it can perceive its own context within the entire table structure. This process is achieved through a specially modified self-attention mechanism.

[0038] Traditional self-attention mechanisms, such as those used in Transformer models, can calculate the correlation between any two elements in a sequence, thereby capturing long-distance dependencies. This is very helpful for understanding relationships across rows and columns in a table. Its core operation is scaled dot product attention, and its standard form is:

[0039] Among them, the Q (query), K (key), and V (value) matrices are all derived from the feature vector matrix H of all nodes (derived from all features in step S200). Stacked together) and then subjected to different linear projections (multiplied by a learnable weight matrix). , , This formula calculates the attention weights of each node to all other nodes, and uses these weights to weighted aggregate the information of all nodes, thus obtaining the updated node representation.

[0040] However, standard self-attention mechanisms are position-independent during computation, failing to distinguish between two identical text blocks located at different positions on the page. To address this issue, this invention innovatively introduces a learnable two-dimensional relative position bias term when calculating the attention score.

[0041] Specifically, in calculating the attention score of node i to node j... At that time, query vectors are no longer used alone. and key vector Instead of the dot product, the following improved formula is used:

[0042] in, This represents the attention score between node i and node j. Let be the query vector corresponding to node i, obtained by linear transformation of the node features. Let be the key vector corresponding to node j, which is also obtained from the node features through a linear mapping. A learnable mapping function used to measure the difference in lateral distance between nodes. Longitudinal distance difference Width difference and height difference These are transformed into additivity bias terms to achieve the fusion of spatial and semantic information. These relative geometric differences are input into a learnable mapping function. In this context, the mapping function, typically implemented using a multilayer perceptron (MLP), transforms the raw geometric difference values ​​into a scalar bias term. This bias term is then directly added to the content-based attention score.

[0043] In this way, when determining how much attention node i should pay to node j, the model considers not only the relevance of their content but also their spatial arrangement explicitly and quantitatively. For example, if two nodes are aligned vertically ( Even if the distance is very small, the model can learn to give them a larger positional bias, thereby increasing the likelihood that they will be identified as co-column relationships.

[0044] This feature aggregation step can be stacked in multiple layers, with the output of each layer serving as the input to the next. After L layers of such aggregation, each node's final representation... It deeply integrates the global semantic context information of the entire table with precise two-dimensional spatial geometric information.

[0045] Step S400: Predict the probability of structural relationships based on the updated node representations.

[0046] After the multi-layer global feature aggregation in step S300, each text block node obtains its final, context-rich updated node representation. The task of this step is to use these refined node representations to determine the possible structural relationships between any two text blocks.

[0047] In one specific embodiment of the present invention, the structural relationships that need to be determined include at least three basic types: 1. Cell Relationship: Determines whether two text blocks belong to the same table cell. This typically occurs when text within a cell wraps to a new line.

[0048] 2. Row relation: Determines whether two text blocks are located in the same row of a table.

[0049] 3. Col relation (col): Determines whether two text blocks are located in the same column of a table.

[0050] To predict these relationships, the system designs a dedicated relationship discrimination function for each relationship type. This function takes the updated representations of a pair of nodes (e.g., node i and node j) as input and outputs a probability value representing the likelihood that the two nodes have that relationship.

[0051] Specifically, for any relation type (using the symbol...) The representation (which can be cell, row, or col) is the adjacency matrix value that predicts the relationship between node i and node j. The calculation process is as follows:

[0052] in, These correspond to relationships within the same cell, row, and column, respectively. and This represents the updated node representation after L-level aggregation. It is a node geometric relationship mapping function, similar to the position bias function in step S300, but applied during the prediction stage. It directly maps the original geometric features of the two nodes. and (or their relative differences) are mapped into a feature vector as additional geometric cue information. It is a discriminative network corresponding to a specific relationship type, usually a small multilayer perceptron (MLP). For the three relationships of same cell, same row, and same column, there will be three independent discriminative networks respectively. . The function converts the output of the discriminant network into a probability distribution, thereby obtaining the probability that there is a relationship between node i and node j.

[0053] By analyzing all possible node pairs in the image After performing the above calculations, the system will eventually generate three adjacency matrices: The element values ​​of these matrices represent the probability that there is a corresponding structural relationship between any two text blocks, and they are the direct basis for subsequent table structure reconstruction.

[0054] Step S500: Rebuild and output the spreadsheet file.

[0055] This step is the final step in the table structure recognition process. Its goal is to use the three relational probability adjacency matrices (same cell, same row, same column) generated in step S400 to reorganize discrete text blocks into a spreadsheet with a complete logical structure.

[0056] The reconstruction process mainly includes the following sub-steps: First, cell construction. The system primarily relies on an adjacency matrix representing relationships between cells. By setting a probability threshold (e.g., 0.5), all node pairs with probabilities higher than this threshold are considered connected. Then, a connectivity analysis algorithm (e.g., depth-first search or breadth-first search) is applied to find all connected components in the graph formed by these connections. Each connected component represents an independent table cell containing one or more text blocks belonging to that cell.

[0057] Secondly, topological structure correction. After constructing the initial cells, it is necessary to utilize the row and column relationship matrices. and The system determines the row and column layout of these cells and handles complex cross-row and cross-column cells (also known as merged cells). For example, if all text blocks in cell A have a high probability of being in the same row as all text blocks in cell B, then cells A and B are determined to be in the same row. For a cell C that spans two columns, it will simultaneously exhibit a row relationship with cells in the two columns to its left and right. By analyzing these complex row and column relationship patterns, the system can accurately identify which cells are standard cells, which are cross-row or cross-column merged cells, and determine the extent to which they span. This process can also correct initial identification errors caused by broken or missing table borders.

[0058] Finally, a spreadsheet file is generated and output. Once all cells, their row and column positions, and spans are determined, a complete logical structure of the table is reconstructed. The system can then render and output this logical structure as a user-friendly, editable electronic file format. Common output formats include HTML (Hypertext Markup Language) files, which can perfectly display the table structure in a web browser; or the more commonly used Excel format document (such as a .xlsx file), which allows users to directly edit, calculate, and further analyze within the spreadsheet software.

[0059] By performing this step, the system successfully transforms a static, unstructured document image into a dynamic, structured, and interactive spreadsheet, thus completing the entire recognition task.

[0060] Example 2 Based on Embodiment 1 above, the present invention also provides a training method for training the table structure recognition model described in Embodiment 1. An untrained model cannot perform accurate recognition tasks; therefore, this training method is a necessary prerequisite for achieving the technical effects of the present invention. The training method specifically includes the following steps: Step S610: Obtain training samples.

[0061] The first step in training is preparing a large amount of training data. Each training sample consists of two parts: one part is a document image, similar to the image to be identified in Example 1; the other part is the corresponding ground truth label. This ground truth label is an accurate description of the table structure in the image, which needs to be pre-annotated manually or generated through other reliable methods. Specifically, the ground truth label needs to clearly indicate which cell, row, and column each text block in the image belongs to. Based on this information, three ground truth adjacency matrices can be generated: In these matrices, if node i and node j have a real relationship, then the corresponding element... The value is 1 if it is 1, otherwise it is 0.

[0062] Step S620: Perform forward propagation to obtain the predicted probability.

[0063] The training sample images obtained in step S610 are input into the constructed model architecture. This model architecture is the complete process described in Example 1, from feature extraction, node vector construction, to self-attention aggregation with two-dimensional positional bias, and then to the relation discriminant function. After receiving the image, the model performs a complete forward propagation calculation and finally outputs its prediction results of the structural relationships between various text blocks in the image, namely, three predicted adjacency matrices: The elements in these matrices are probability values ​​between 0 and 1.

[0064] Step S630: Calculate the loss value using a preset loss function.

[0065] The purpose of this step is to quantify the gap between the model's predictions and the true labels. This gap is called the loss. The larger the loss value, the less accurate the model's predictions are. In a preferred embodiment of the invention, a three-way weighted cross-entropy loss function is used to jointly optimize the prediction of cell, row, and column relationships. The specific formula for this loss function is as follows:

[0066] in, , , The weights are introduced because the number of samples for different relationships in a table may be very unbalanced (e.g., the number of relationships within the same row is much greater than the number of relationships within the same cell), or some relationships may be more difficult to learn. By adjusting these weights, the contributions of the three tasks to the total loss can be balanced, guiding the model to learn more stably and efficiently. CE represents the cross-entropy loss. This is a label indicating whether node i and node j belong to the same cell in the actual annotation. The value is 1 when node i and node j belong to the same cell, and 0 otherwise. Let be the probability value predicted by the model that node i and node j belong to the same cell, and its value ranges from [0,1]. Labels are the actual adjacency matrix labels. The value is the model prediction, and L represents the final calculated total loss value. This represents the standard cross-entropy loss function, used to measure the difference between two probability distributions (in this case, the 0 / 1 distribution of the true labels and the probability distribution predicted by the model). It is the label value (0 or 1) in the actual adjacency matrix. , , are the true adjacency matrix labels of nodes i and j in the same row relationship, and , representing the probability values ​​of nodes i and j predicted by the model in the same row relationship.

[0067] Step S640: Update the model parameters using the backpropagation algorithm.

[0068] After calculating the total loss value L, the training process enters its final step: parameter update. The system employs the backpropagation algorithm, starting from the loss function and proceeding in the reverse direction of the model's computation path, to calculate the gradient of the loss value L with respect to each learnable parameter in the model (e.g., the weights of each fully connected layer, the projection matrix of the self-attention module, the parameters of the position bias function, etc.). The gradient indicates the direction in which the parameters should be adjusted to reduce the loss value.

[0069] Then, using an optimizer algorithm, such as stochastic gradient descent (SGD), Adam, or AdamW, a small update is made to all the parameters of the model based on the calculated gradient.

[0070] Repeat steps S620 to S640, continuously feeding batches of training samples into the model, calculating the loss, and updating the parameters. After thousands of iterations, the model's parameters will gradually converge to an optimal or near-optimal state. At this point, the total loss calculated by the model will be very small, meaning it has learned how to accurately predict the structural relationships of tables from document images. Thus, a well-trained model ready for practical recognition tasks is born.

[0071] Example 3 This embodiment provides a specific implementation of the table structure recognition method as described in Embodiment 1. (Refer to...) Figure 2 , Figure 2 This is a schematic diagram of the functional modules of the table structure recognition system according to an embodiment of the present invention. The system can be deployed on a cloud server or a local computer, and includes: The text feature construction module is configured to execute steps S100 and S200 described in Example 1. Specifically, it is responsible for receiving the document image to be recognized and internally integrating or calling a high-efficiency OCR engine to perform text detection and recognition. Subsequently, it extracts visual features, semantic features, and geometric position features for each detected text block through a parallel feature extraction pipeline. Finally, this module fuses these three types of features into a unified node initial vector through a concatenation unit containing a learnable embedding layer and outputs it to the next module.

[0072] The structural feature aggregation module, the core computational unit of the system, is configured to execute step S300 as described in Example 1. Internally, it implements one or more stacked self-attention layers that introduce two-dimensional relative positional biases. This module receives initial vectors from all nodes in the text feature construction module and, through multi-layer global feature aggregation, computes an updated node representation for each node that integrates global context and precise spatial location information. The design of this module ensures that the model can effectively handle long-range dependencies and complex spatial layouts.

[0073] The relationship discrimination module is configured to perform step S400 as described in Example 1. It receives updated node representations from the structural feature aggregation module. Internally, this module contains three parallel sub-networks specifically designed to discriminate relationships within the same cell, row, and column. For any input pair of node representations, the module calculates the probabilities of these three structural relationships existing between them through these three sub-networks. Finally, the module outputs three complete relationship probability adjacency matrices.

[0074] The structure reconstruction module, which is the system's output unit, is configured to execute step S500 as described in Example 1. It receives three relation probability adjacency matrices from the relation discrimination module. Internally, this module implements a graph theory-based post-processing algorithm. First, it constructs initial cells through connected component analysis, then performs topological correction based on row and column relationships to accurately identify and reconstruct merged cells spanning multiple rows and columns. Finally, based on the reconstructed complete table logical structure, the module calls the corresponding renderer to generate and save it in a user-specified electronic file format, such as an HTML or Excel document.

[0075] Example 4 The present invention also provides an electronic device. This electronic device may be a smartphone, tablet computer, laptop computer, desktop computer, server, etc.

[0076] The electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor.

[0077] The processor is the control center of a device and can use various computing architectures. A processor can include a single processing unit or multiple processing units.

[0078] The memory can be volatile or non-volatile, or a combination of both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), electrically erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as external cache memory.

[0079] When the computer program stored in the memory is executed by the processor, it can implement the table structure recognition method as described in Embodiment 1, or implement the training method of the table structure recognition model as described in Embodiment 2.

[0080] Example 5 The present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program can implement the table structure recognition method as described in Embodiment 1, or implement the training method for the table structure recognition model as described in Embodiment 2.

[0081] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0082] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0083] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations. These advantages, benefits, and effects should not be considered as essential features of each embodiment of the present invention.

[0084] It should also be noted that in the apparatus, device, and method of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered equivalents of the present invention. The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0085] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for recognizing table structures, characterized in that, The steps of this method include: Receive the document image to be identified, and extract the visual features, semantic features and geometric position features of each text block in the document image through a text detection and recognition model; Based on the aforementioned visual features, text semantic features, and geometric position features, the position information is encoded through a geometric position embedding function and multiple features are concatenated to construct the initial vector of the text block's nodes. Global feature aggregation is performed on the node initial vector. The global feature aggregation is achieved by introducing a self-attention mechanism that includes learnable two-dimensional relative position bias. The scaling dot product attention weights between nodes are calculated and the spatial position bias term is superimposed to obtain an updated node representation that integrates spatial geometric relationships and semantic information. Based on the updated node representation, the probability of structural relationship between any two text blocks is predicted by a relationship discriminant function. The structural relationship includes at least the relationship between the same cell, the same row, and the same column. Based on the structural relationship probabilities, text blocks are combined into cells, rows, and columns through connected component analysis and topology correction, and a spreadsheet file containing the complete structure is reconstructed and output.

2. The table structure recognition method according to claim 1, characterized in that, The initial vector of the nodes used to construct the text block is given by the following formula: in, Let represent the node initialization vector of the i-th text block. Represents visual feature vectors. This represents the recognition of text vectors. This indicates geometric information including center point coordinates, width, height, and relative proportions. It is a learnable geometric position embedding function used to encode the two-dimensional position and size information of nodes.

3. The table structure recognition method according to claim 2, characterized in that, The introduction of a self-attention mechanism that includes learnable two-dimensional relative position bias specifically employs scaled dot product attention, calculated as follows: in, , , H is the node feature matrix. , , The projection matrix is ​​learnable. Let Q be the query vector matrix, representing the current node's attention needs towards other nodes. K is the key vector matrix, representing the feature identifiers of each node, used for matching calculations with the query vector. V is the value vector matrix, representing the actual feature information of the nodes. The aggregated output is then processed after attention weighting. To represent the transpose of the key vector matrix K, it is used to perform a dot product operation with the query vector Q to calculate the relevance score between nodes.

4. The table structure recognition method according to claim 3, characterized in that, The learnable two-dimensional relative position offset is specifically as follows: in, This represents the attention score between node i and node j. Let be the query vector corresponding to node i, obtained by linear transformation of the node features. Let be the key vector corresponding to node j, which is also obtained from the node features through a linear mapping. A learnable mapping function used to measure the difference in lateral distance between nodes. Longitudinal distance difference Width difference and height difference It is transformed into an additivity bias term to achieve the fusion of spatial and semantic information.

5. The table structure recognition method according to claim 4, characterized in that, The specific calculation formula for predicting the structural relationship probability between any two text blocks using the relation discriminant function is as follows: in, This represents the predicted probability that the relationship between node i and node j belongs to the cell, row, or col structure. This is a relation discrimination function used to map the features of node pairs and output a score for the corresponding relation. These correspond to relationships within the same cell, row, and column, respectively. and This represents the updated node representation after L-level aggregation. The function represents the mapping function of node geometric relationships. This is the geometric position information vector corresponding to node j, including spatial features such as the bounding box coordinates, width, and height of the text block.

6. The table structure recognition method according to claim 1, characterized in that, The process of reconstructing and outputting a spreadsheet file containing the complete structure specifically includes: Based on the predicted adjacency matrices of the same cell, row, and column, the connected component analysis algorithm is used to identify independent cell regions. Correct the topological relationships of the identified cell range across rows and columns, and handle merged cells and broken borders; The corrected structure is mapped to HTML code or an Excel document for output.

7. A training method for a table structure recognition model, characterized in that, The table structure recognition model is used to execute the table structure recognition method as described in claim 1, and its training steps include: Obtain training samples containing text blocks and their true structural relationships; The training samples are input into a predefined table structure recognition model architecture to obtain the predicted structural relationship probabilities; Based on the predicted structural relationship probability and the actual structural relationship, the loss value is calculated using a preset loss function; Based on the loss value, the parameters of the table structure recognition model are updated using the backpropagation algorithm to complete the training of the table structure recognition model.

8. The training method for the table structure recognition model according to claim 7, characterized in that, The loss value is calculated using a preset loss function, and the specific calculation formula is as follows: in, , , Here, CE represents the cross-entropy loss, and the weighting coefficients are used. This is a label indicating whether node i and node j belong to the same cell in the actual annotation. The value is 1 when node i and node j belong to the same cell, and 0 otherwise. Let be the probability value predicted by the model that node i and node j belong to the same cell, and its value ranges from [0,1]. Labels are the actual adjacency matrix labels. These are the model's predicted values. , , are the true adjacency matrix labels of nodes i and j in the same row relationship, and , representing the probability values ​​of nodes i and j predicted by the model in the same row relationship.

9. A table structure recognition system, characterized in that, include: The text feature construction module is used to receive document images, extract multidimensional features of text blocks, and construct node initial vectors; The structural feature aggregation module is used to model global dependencies and update features of node initial vectors by introducing a self-attention mechanism with two-dimensional relative position bias. The relationship determination module is used to predict the structural relationships between text blocks in the same cell, same row, and same column based on the updated node representation. The structure reconstruction module is used to reconstruct the table topology based on the predicted structural relationships and output an editable electronic document.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the table structure recognition method as described in any one of claims 1 to 6 or the training method for the table structure recognition model as described in any one of claims 7 to 8.

Images