Table structure recognition method and system based on adaptive anchor box, terminal and medium

Abstract

The present application relates to the technical field of computer vision and document analysis, and discloses a table structure recognition method and system based on adaptive anchor boxes, a terminal and a medium. The method obtains multiple typical table image samples and labels the structured data and merging types of table cells, and constructs a row channel dataset and a column channel dataset through multi-modal processing. A table structure recognition model is constructed, which includes a backbone feature extraction network, a double-branch detection head and a merged cell classification module. The table structure recognition model is trained based on the row channel dataset and the column channel dataset. The trained model is used to perform a structure recognition task on an input table image, and structured data containing cell physical coordinates and logical position relationships is generated according to the recognition result. The present application can effectively solve the cumulative error problem caused by the asymmetric row and column detection logic in traditional methods, and avoid the inconsistency problem of row width and column height caused by complete dynamic adjustment.

Description

Technical Field

[0001] This invention relates to the field of computer vision and document analysis technology, specifically to a table structure recognition method, system, terminal, and medium based on adaptive anchor boxes. Background Technology

[0002] In the field of general object detection, dynamic anchor box adjustment mechanisms, represented by Mask R-CNN (He et al., 2017), can flexibly adapt to changes in the shape and position of different objects by simultaneously optimizing the coordinates and size of the bounding box. Such methods perform well in object detection in natural scenes because they can flexibly adapt to changes in the scale and position of different objects. However, tables, as highly structured objects, have strict geometric alignment and size consistency in their rows and columns. Existing technologies have the following problems and drawbacks in table recognition scenarios:

[0003] 1. Lack of geometric constraints: The rows and columns of the table have strict size consistency and alignment, but the free adjustment of the dynamic anchor frame will cause prediction distortion. The predicted values ​​of the table row height / column width are easy to deviate from the actual physical values, causing row width compression or column height stretching.

[0004] 2. Error accumulation problem: When the dynamically adjusted anchor frame coordinates are divided into consecutive rows and columns, the error is passed on layer by layer, eventually causing the boundary to tilt or cross. Summary of the Invention

[0005] To address the technical problems existing in the prior art, this invention provides a table structure recognition method, system, terminal, and medium based on adaptive anchor frames. It can effectively solve the problem of accumulated errors caused by the asymmetry of row and column detection logic in traditional methods, while avoiding the problem of inconsistent row width and column height caused by fully dynamic adjustment.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] This invention discloses a table structure recognition method based on adaptive anchor boxes, which includes the following steps, namely S1 to S4.

[0008] S1. Obtain typical table image samples of various types and label the structured data and merging types of the table cells. Construct travel channel datasets and column channel datasets through multimodal processing; wherein, the column channel datasets are obtained by geometric transformation of the original table images and their labels;

[0009] S2. Construct a table structure recognition model, which includes a backbone feature extraction network, a dual-branch detection head, and a merged cell classification module. The backbone feature extraction network is used to extract row features and column features from the input row channel image and column channel image, respectively. The dual-branch detection head is used to perform target detection based on the row features and column features, respectively detecting blank areas between rows and blank areas between columns. The detection results are used to reconstruct the grid structure of the table. The merged cell classification module is used to identify and merge cell regions that span multiple rows or columns in the grid structure. During the detection process of the dual-branch detection head, the horizontal geometric parameters of the anchor frame, i.e., the x-coordinate and width, are fixed, while the vertical geometric parameters of the anchor frame, i.e., the y-coordinate and height, are dynamically adjusted.

[0010] S3. Train the table structure recognition model based on the row channel dataset and the column channel dataset;

[0011] S4. Use the trained table structure recognition model to perform a structure recognition task on the input table image, and generate structured data containing the physical coordinates and logical positional relationships of the cells based on the recognition results.

[0012] As a further improvement to the above scheme, step S1 includes the following specific steps:

[0013] S11. Collect and label several image samples covering multiple typical tables to obtain the first table structure recognition dataset; among them, the structured data includes the physical coordinates and logical positional relationships of the cells; the merging type includes labels for four merging states of the cells, namely: no merging, merging to the left, merging upwards, and bidirectional merging;

[0014] S12. By constructing detection labels for inter-row blank areas and inter-column blank areas respectively from the structured data in the first table structure recognition dataset, without considering merging relationships, a second table structure recognition dataset is obtained.

[0015] S13. Perform data augmentation on the second table structure recognition dataset, including horizontal flipping and normalization with set probabilities, to obtain the third table structure recognition dataset;

[0016] S14. Divide the third table structure recognition dataset into a training set and a validation set according to a set ratio;

[0017] S15. Preserve the original image samples and spatial relationships in the training and validation sets to construct the row channel dataset; similarly, rotate the original image samples in the training and validation sets 90° clockwise and transform the labels synchronously to construct the column channel dataset.

[0018] As a further improvement to the above scheme, in step S2, the HRNet32-FPN architecture is used as the backbone feature extraction network, and weights are shared between the two input channels. When the input is a table image sample in the row channel dataset, the saliency hierarchical features of the row channel are output, and when the input is a table image sample in the column channel dataset, the saliency hierarchical features of the column channel are output.

[0019] As a further improvement to the above scheme, in step S2, a row detection head branch and a column detection head branch with the same structure but independent parameters are constructed based on Mask R-CNN, namely the dual-branch detection head; the input of the row detection head branch is the saliency hierarchical features of the row channels, and the output is the row detection result; the input of the column detection head branch is the saliency hierarchical features of the column channels, and the output is the post-processed column detection result; wherein, both the row detection result and the column detection result include the corresponding detection box, confidence score and segmentation mask; the post-processing is to restore the original column detection result by rotating it counterclockwise by 90°; the specific settings of the anchor box in the dual-branch detection head are as follows:

[0020] In the row detection head branch, the horizontal coordinate of the anchor box is locked to the left boundary position of the original image sample, and the width is locked to the width of the table image sample; in the column detection head branch, the original image sample is processed according to the same anchor box strategy as the row detection after being rotated 90° clockwise.

[0021] As a further improvement to the above solution, the reconstructed grid structure of the table includes the following specific steps:

[0022] For the row detection results and the column detection results, respectively set corresponding thresholds to filter out results with low confidence;

[0023] Based on the filtered row detection results, the vertical midpoint of each row detection box is calculated as the row center line; based on the filtered column detection results, the horizontal midpoint of each column detection box is calculated as the column center line.

[0024] The center lines of rows and columns are sorted to ensure the correct topological order during mesh generation;

[0025] The four corner points are determined by defining the center lines of two adjacent rows and two adjacent columns, and the grid structure of the table is determined based on the corner point positions of all cells.

[0026] As a further improvement to the above scheme, step S3 adopts phased training, as follows:

[0027] Phase 1: Training the backbone feature extraction network and the dual-branch detection head; where the multi-task loss is obtained by weighted fusion of row detection loss and column detection loss; the row detection loss is obtained by adding row classification loss, row detection box regression loss and row mask loss, and the column detection loss is obtained by adding column classification loss, column detection box regression loss and column mask loss;

[0028] The second stage involves freezing the parameters of the backbone feature extraction network and the dual-branch detection head, and fine-tuning the merged cell classification module. The merged cell classification loss uses a dual-focus loss function to jointly optimize the merge type prediction and the starting grid. The main focus loss function is calculated for the four merge states, while the secondary focus loss function simplifies the problem into a binary classification task and calculates whether it is the starting grid separately.

[0029] As a further improvement to the above scheme, when calculating the row detection loss, the row classification loss adopts cross-entropy loss, the row detection box regression loss only calculates the smooth L1 loss of the detection box's ordinate and height, and the detection box's abscissa and width do not participate in gradient update, the row mask loss adopts pixel-wise binary cross-entropy loss, and the column detection loss is calculated in the same way.

[0030] The present invention also discloses a table structure recognition system based on adaptive anchor boxes, which applies the table structure recognition method based on adaptive anchor boxes as described above; the system includes: a data acquisition module, a model building module, a training module and a task execution module.

[0031] The data acquisition module is used to acquire various typical table image samples and label the structured data and merging types of the table cells. After multimodal processing, travel channel datasets and column channel datasets are constructed.

[0032] The model building module is used to construct a table structure recognition model, which includes a backbone feature extraction network, a dual-branch detection head, and a merged cell classification module. The backbone feature extraction network is used to extract row features and column features from the input row channel image and column channel image, respectively. The dual-branch detection head is used to perform target detection based on row features and column features, respectively detecting blank areas between rows and blank areas between columns. The detection results are used to reconstruct the grid structure of the table. The merged cell classification module is used to identify merged cell regions that span multiple rows or columns in the grid structure. During the detection process of the dual-branch detection head, the horizontal geometric parameters of the anchor frame, namely the x-coordinate and width, are fixed, while the vertical geometric parameters of the anchor frame, namely the y-coordinate and height, are dynamically adjusted.

[0033] The training module is used to train the table structure recognition model in stages based on the row channel dataset and the column channel dataset;

[0034] The task execution module is used to perform a structure recognition task on the input table image using the trained table structure recognition model, and generate structured data containing the physical coordinates and logical positional relationships of the cells based on the recognition results.

[0035] The present invention also discloses a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the table structure recognition method based on adaptive anchor boxes as described above.

[0036] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, characterized in that, when the program is executed by a processor, it implements the steps of the table structure recognition method based on adaptive anchor boxes as described above.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] 1. The table structure recognition method based on adaptive anchor boxes disclosed in this invention transforms the column detection task into an equivalent row detection task by rotating the original image by 90°, so that the row and column segmentation adopt the same fixed anchor box processing logic. On the one hand, it eliminates the systematic error caused by inconsistent detection standards: after unifying the row and column detection logic, it avoids the cumulative error caused by the asymmetry of row and column processing in traditional methods. On the other hand, it simplifies the model learning objective: reduces the dimension of dynamic parameters (only predicts y and h, without calculating x and w), and improves the training convergence speed.

[0039] 2. This invention proposes a dual-branch independent detection architecture, which unifies column detection tasks into row detection modes through geometric transformation, resolving the task conflict problem of the single-branch model. Verification shows that the AP50 for row and column detection is improved to 0.857 (rows) and 0.901 (columns), respectively, representing improvements of 17.2% and 18.2% compared to the single-branch model. Furthermore, it improves computational efficiency, reducing the number of model parameters by nearly 30M compared to two independent model schemes, resulting in a 46% speedup.

[0040] 3. This invention eliminates dimensional distortion by fixing key geometric parameters during row / column detection and dynamically adjusting only necessary parameters. Verification has shown that the prediction errors for row width and column height have been reduced from over 10% to less than 5%. Furthermore, it suppresses misalignment accumulation: by fixing the dimensions, it avoids the misalignment accumulation caused by dynamically adjusting parameters in traditional methods. Attached Figure Description

[0041] Figure 1 This is a flowchart of the table structure recognition method based on adaptive anchor boxes in Embodiment 1 of the present invention.

[0042] Figure 2 This is a logic block diagram of the table structure recognition model in Embodiment 1 of the present invention.

[0043] Figure 3 This is an anchor block diagram of the region proposal network in Embodiment 1 of the present invention.

[0044] Figure 4 This is a structural diagram of the table structure recognition system based on adaptive anchor frames in Embodiment 2 of the present invention.

[0045] Figure 5 This is a structural diagram of the computer terminal in Embodiment 3 of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Example 1

[0048] Please see Figure 1 This embodiment provides a table structure recognition method based on adaptive anchor boxes, including the following steps, namely S1 to S4.

[0049] S1. Obtain typical table image samples of various types and label the structured data and merging types of the table cells. Construct a travel channel dataset and a column channel dataset through multimodal processing; wherein, the column channel dataset is obtained by geometric transformation of the original table images and their labels. Step S1 includes specific steps S11 to S15.

[0050] S11. Collect and label several image samples covering multiple typical tables to obtain the first table structure recognition dataset T1; wherein, the structured data includes the physical coordinates and logical positional relationships of the cells; the merging type includes labels for four merging states of the cells, namely: no merging, merging to the left, merging upward, and bidirectional merging.

[0051] In this embodiment, the sample set contains 3,000 labeled table images, covering six typical tables such as financial statements and performance reports. Each sample provides 1) the physical coordinates and logical positional relationship of the cells; 2) four-category labels for merged cells (0: no merging, 1: merge to the left, 2: merge to the top, 3: merge in both directions).

[0052] S12. By constructing detection labels for inter-row blank areas and inter-column blank areas respectively from the structured data in the first table structure recognition dataset, without considering merging relationships, a second table structure recognition dataset T2 is obtained.

[0053] S13. Perform data augmentation on the second table structure recognition dataset T2, including horizontal flipping and normalization with a 50% probability, to obtain the third table structure recognition dataset T3.

[0054] S14. Divide the third table structure recognition dataset T3 into training set T in a 9:1 ratio. train (2,700 images) and validation set T val (300 images).

[0055] S15. Transfer the training set T train and validation set T val Preserving the original image samples and spatial relationships, construct the row channel dataset T. row (Including row channel training set T) row_train And the verification set T row_val ), used for line detection tasks; similarly, the training set T train and validation set T val The original image samples are rotated 90° clockwise, and the labels are transformed synchronously to construct a column channel dataset T. col (Including column channel training set T) col_train Column channel validation set T col_val ), used for column detection tasks.

[0056] At this point, the column detection task is transformed into an equivalent row detection task, unifying the logic of row and column detection and improving the consistency of the algorithm.

[0057] S2. Construct a table structure recognition model, which includes a backbone feature extraction network, a dual-branch detection head, and a merged cell classification module. The backbone feature extraction network is used to extract row features and column features from the input row channel image and column channel image, respectively. The dual-branch detection head is used to perform target detection based on the row features and column features, respectively detecting blank areas between rows and blank areas between columns. The detection results are used to reconstruct the grid structure of the table. The merged cell classification module is used to identify and merge cell regions that span multiple rows or columns in the grid structure. During the detection process of the dual-branch detection head, the horizontal geometric parameters of the anchor frame, namely the x-coordinate and width, are fixed, while the vertical geometric parameters of the anchor frame, namely the y-coordinate and height, are dynamically adjusted.

[0058] Please see Figure 2 In step S2, the HRNet32-FPN architecture is used as the backbone feature extraction network, and weights are shared between the two input channels to reduce the complexity of the model.

[0059] When the input to the backbone feature extraction network is the row channel dataset T row When taking a table image sample, output the saliency hierarchical features F of the row channels.row .

[0060] When the input to the backbone feature extraction network is a column channel dataset T col When using table image samples, the saliency hierarchical features F of the output column channels are... col .

[0061] Based on Mask R-CNN, a dual-branch detection head is constructed, consisting of row and column detection head branches with identical structures but independent parameters. Each branch only needs to detect a single category of target (inter-row or inter-column blank areas), simplifying the classification task.

[0062] The input to the row detection head branch is the saliency hierarchical feature F of the row channel. row The output is the row detection result.

[0063] The row detection results include row detection boxes. Confidence score and row splitting mask , , n This represents the number of rows detected.

[0064] The input to the column detection head branch is the saliency hierarchical feature F of the column channel. col The output is the post-processed column detection result.

[0065] The column detection results include column detection boxes. , list confidence scores Column splitting mask , , m This represents the number of columns detected.

[0066] The post-processing involves restoring the original column detection results by rotating them counterclockwise by 90°; the specific settings for the anchor frame in the dual-branch detection head are as follows:

[0067] In the row detection head branch, the horizontal parameter x and width w of the anchor box are fixed; where x is always locked to the left boundary position of the table, and the width w is always locked to the width of the current table sample; under this constraint, the model only needs to dynamically predict the vertical y coordinate and height h; in the column detection head branch, since the original image is rotated by 90°, column detection is converted into equivalent row detection, and the rotated image is processed according to the same anchor box strategy as row detection: the horizontal parameters x (left boundary after rotation, corresponding to the top boundary of the original table) and w (row width after rotation, corresponding to the original column height) are fixed, and only the vertical parameters y and h (corresponding to the left boundary position of the original column and the column spacing) are dynamically adjusted.

[0068] In this embodiment, combined with Figure 3The region proposal network uses anchor boxes. The detection head consists of a 3×3 convolution (256 output channels) and two parallel 1×1 convolutions, which output the probability that each anchor box contains an object (number of channels k) and the vertical y-bias and h-bias of each anchor box (number of channels 2k), respectively, where k is the number of anchor boxes corresponding to each feature map pixel. By fixing x and w and predicting only y and h, the detection degrees of freedom can be effectively reduced, and the accuracy of row and column boundary prediction can be improved.

[0069] The reconstruction of the grid structure of the table includes the following specific steps:

[0070] For the row detection results and the column detection results, corresponding thresholds are set to filter out results with low confidence.

[0071] First, threshold filtering is applied to the input row and column segmentation results; for row detection results, a row threshold is set. Filtering results with low confidence; the filtering process The expression is:

[0072] ;

[0073] Filtering results The expression is:

[0074] ;

[0075] In the formula, , , These represent the row detection bounding box, row segmentation mask, and row confidence score in the filtering results, respectively. n’ This represents the number of rows in the filtered results.

[0076] Similarly, for column detection results, set a column threshold. Filtering low-confidence results, the filtering process The expression is:

[0077] ;

[0078] Filtering results The expression is:

[0079] ;

[0080] In the formula, , , These are the column detection boxes, column split masks, and column confidence scores in the filtering results, respectively. m’ This represents the number of columns in the filtered results.

[0081] Based on the filtered row detection results, the vertical midpoint of each row detection box is calculated as the row center line; based on the filtered column detection results, the horizontal midpoint of each column detection box is calculated as the column center line.

[0082] For the filtered row detection results Calculate the detection box for each row. The vertical midpoint is used as the row centerline The calculation formula is:

[0083] ;

[0084] In the formula, and For the first The top and bottom borders of the row detection box coordinate.

[0085] Similarly, for the filtered column detection results Calculate the detection box for each column. The horizontal midpoint is used as the column centerline The calculation formula is:

[0086] ;

[0087] In the formula, and For the first The left and right lines of the column detection box coordinate.

[0088] Then, the row and column centerlines are sorted to ensure the correct topological order during grid generation.

[0089] Finally, the positions of the four corner points (arranged in the order of top left, top right, bottom right, and bottom left) are determined by the cells defined by the center lines of two adjacent rows and two adjacent columns, thus determining the grid structure of the table based on the corner points of all cells.

[0090] In this embodiment, the merged cell classification module can use a four-category (no merging, upward merging, leftward merging, bidirectional merging) merging algorithm to identify and merge cell regions that span multiple rows or columns.

[0091] S3. Train the table structure recognition model based on the row channel dataset and the column channel dataset. Step S3 employs a phased training method, as follows:

[0092] Phase 1: Training the backbone feature extraction network and the dual-branch detection head.

[0093] Using training set T train Optimize multi-tasking loss. Loss due to line detection And column detection loss The weighted fusion yields the following expression:

[0094] ;

[0095] In the formula, and are weighting coefficients, all of which are hyperparameters.

[0096] Among them, the row and column detection loss inherits the multi-task loss of Mask R-CNN, but is adjusted for the fixed anchor box strategy. Taking the row detection loss as an example:

[0097] ;

[0098] In the formula, For row classification loss, cross-entropy loss is used; For bounding box regression loss, only the smooth L1 loss of the dynamic parameters (y,h) is calculated, while the fixed parameters (x,w) do not participate in gradient update; For mask loss, pixel-wise binary cross-entropy loss is used.

[0099] The calculation of column detection loss is the same as that of row detection loss.

[0100] Phase 2: Freeze the parameters of the main feature extraction network and the dual-branch detection head, and fine-tune the merged cell classification module.

[0101] Among them, the training set T is used train Optimize the category loss for merged cells. A dual-focus loss function is used to jointly optimize the prediction of merge type and the initial grid (the grid that is not merged). The primary focus loss function is calculated for the four types of merged states; the secondary focus loss function simplifies the problem into a binary classification task and calculates whether it is the initial grid separately.

[0102] In this embodiment, model training may include the following steps, namely S31 to S34.

[0103] S31. Set the first stage training parameters: number of epochs = 100, batch size = 4, and learning rate lr = 2 × 10⁻⁶. -2 Set the learning rate and decay step size set steps={32, 64, 80}, set the number of detection classes (excluding background) num_class=4; set the hyperparameters λ1=λ2=1;

[0104] S32. Set the training parameters for the second stage: number of epochs = 100, batch size = 16, and learning rate lr = 2 × 10⁻⁶. -4 ;

[0105] S33. In both the first and second stages, 2700 pairs of image samples and their corresponding label data are input into the model for training;

[0106] S33. After each round of training in both phases, use 300 pairs of validation sets T. val The samples were used for validation to obtain the optimal model parameters.

[0107] By integrating the model parameters from the two stages in step S3, the parameter file of the entire model is obtained, and the model training is completed.

[0108] S4. Use the trained table structure recognition model to perform a structure recognition task on the input table image, and generate structured data containing the physical coordinates and logical positional relationships of the cells based on the recognition results.

[0109] To verify the effectiveness of the method of the present invention, the following experiments are also provided in this embodiment:

[0110] This embodiment compares with a model built on Mask R-CNN, including:

[0111] 1. The single-branch model (Baseline) adopts the standard Mask R-CNN architecture: it processes row and column segmentation tasks simultaneously through a single detector head and dynamically adjusts all anchor box parameters (x, y, w, h).

[0112] 2. Dual-branch independent model: Two completely independent Mask R-CNN models are used to handle row detection and column detection respectively, with no parameter sharing.

[0113] The experiments used an NVIDIA L20 professional-grade GPU (48GB VRAM) as the computing platform and were implemented based on the PyTorch 2.4.0 deep learning framework. The hardware environment included CUDA 12.2 and the cuDNN 8.6.0 acceleration library. At the software implementation level, stochastic gradient descent (SGD) was used as the optimizer, with an initial learning rate of 0.02. A multi-step decay strategy was employed for learning rate scheduling, adjusting the learning rate by a factor of 0.1 at epochs 32, 64, and 80. All experiments were repeated 5 times and the average value was taken to eliminate the influence of randomness.

[0114] Table 1 presents the quantitative comparison results, with bold and underlined indicating the best and second-best results, respectively. Evaluation indicators include:

[0115] AP50: Average accuracy when the IoU threshold is 0.5, a core metric for measuring the accuracy of model detection;

[0116] Geometric dimension prediction error: including the relative error rate of row width and the relative error rate of column height.

[0117] Line width error rate = |Predicted line width - Actual line width| / Image width

[0118] Column height error rate = |Predicted column height - Actual column height| / Image height

[0119] Convergence rate: The number of training epochs required to achieve 90% of the model's final AP50;

[0120] Model parameter count: Reflects model complexity, expressed in millions (M);

[0121] Inference speed: Average processing time per image (short side scaled to 800 pixels, long side not exceeding 1333 pixels, aspect ratio unchanged), in milliseconds (ms).

[0122] Table 1: Quantitative Comparison of Different Methods

[0123] ;

[0124] As shown in Table 1, compared with traditional single-branch and dual-independent models, this method achieves significant improvements in several key metrics: row / column detection AP50 reaches 0.85 and 0.90 respectively (an improvement of 17.2% / 18.1%); the geometric size error rate is reduced to 4.3% / 3.8%, a reduction of over 60% compared to the single-branch model (which suffers from a large number of missed row and column detections, as a single detection head cannot simultaneously handle targets with significant morphological differences) and over 45% compared to the dual-branch independent model (which loses the beneficial feature sharing between row and column tasks); simultaneously, the number of model parameters is reduced by 30% compared to the dual-independent model, while maintaining a practical inference speed of 65ms. These improvements validate the effectiveness of the fixed-dynamic hybrid parameter strategy.

[0125] Example 2

[0126] This embodiment provides a table structure recognition system based on adaptive anchor boxes, which can be applied to the table structure recognition method based on adaptive anchor boxes as described in Embodiment 1.

[0127] Please see Figure 4 The system includes: a data acquisition module 11, a model building module 12, a training module 13, and a task execution module 14.

[0128] Data acquisition module 11 is used to acquire multiple typical table image samples and label the structured data and merging types of table cells. After multimodal processing, travel channel dataset and column channel dataset are constructed.

[0129] The model building module 12 is used to build a table structure recognition model, which includes a backbone feature extraction network, a dual-branch detection head, and a merged cell classification module. The backbone feature extraction network is used to extract row features and column features from the input row channel image and column channel image, respectively. The dual-branch detection head is used to perform target detection based on the row features and column features, respectively detecting blank areas between rows and blank areas between columns. The detection results are used to reconstruct the grid structure of the table. The merged cell classification module is used to identify and merge cell regions that span multiple rows or columns in the grid structure. During the detection process of the dual-branch detection head, the horizontal geometric parameters of the anchor frame, namely the x-coordinate and width, are fixed, while the vertical geometric parameters of the anchor frame, namely the y-coordinate and height, are dynamically adjusted.

[0130] Training module 13 is used to perform phased training on the table structure recognition model based on the row channel dataset and the column channel dataset;

[0131] The task execution module 14 is used to perform a structure recognition task on the input table image using the trained table structure recognition model, and generate structured data containing the physical coordinates and logical positional relationships of the cells based on the recognition results.

[0132] Example 3

[0133] This embodiment provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the table structure recognition method based on adaptive anchor boxes as described in Embodiment 1.

[0134] like Figure 5 As shown, the computer terminal provided in this embodiment includes: at least one processor 101, and a memory 102 connected to at least one processor 101. This embodiment does not limit the specific connection medium between the processor 101 and the memory 102. Figure 5 The example shown is the connection between processor 101 and memory 102 via bus 100. Bus 100 is... Figure 5 The connections between other components are shown in bold lines and are for illustrative purposes only, not as limiting information. Bus 100 can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 5 The bus is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. Alternatively, the processor 101 may also be called a controller; there is no restriction on the name.

[0135] In this embodiment, the memory 102 stores instructions that can be executed by at least one processor 101. The at least one processor 101 can execute the aforementioned method by executing the instructions stored in the memory 102.

[0136] The processor 101 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 102 and calling data stored in memory 102, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0137] In one possible design, processor 101 may include one or more processing units. Processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 101. In some embodiments, processor 101 and memory 102 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0138] Processor 101 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the table structure recognition method based on adaptive anchor boxes disclosed in Embodiment 1 can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules in processor 101.

[0139] Memory 102, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 102 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 102 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In this embodiment, memory 102 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0140] By designing and programming the processor 101, the code corresponding to the security verification method described in the foregoing embodiments can be embedded into the chip, thereby enabling the chip to execute the code during operation. Figure 1 The steps of the table structure recognition method based on adaptive anchor boxes are shown. How to design and program the processor 101 is a technique well-known to those skilled in the art and will not be described further here.

[0141] Example 4

[0142] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the steps of the table structure recognition method based on adaptive anchor boxes as described in Embodiment 1.

[0143] The computer-readable storage medium may include flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Of course, the storage medium may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0144] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A table structure recognition method based on adaptive anchor box, characterized in that, include: S1. Obtain typical table image samples of various types and label the structured data and merging type of the table cells. Construct travel channel dataset and column channel dataset through multimodal processing. The column channel dataset is obtained by geometric transformation of the original table image and its labels. The geometric transformation refers to rotating the original image sample 90° clockwise. S2. Construct a table structure recognition model, which includes a backbone feature extraction network, a dual-branch detection head, and a merged cell classification module. The backbone feature extraction network is used to extract row features and column features from the input row channel image and column channel image, respectively. The dual-branch detection head is used to perform target detection based on the row features and column features, respectively detecting blank areas between rows and blank areas between columns. The detection results are used to reconstruct the grid structure of the table. The merged cell classification module is used to identify and merge cell regions that span multiple rows or columns in the grid structure. During the detection process of the dual-branch detection head, the horizontal geometric parameters of the anchor frame, i.e., the x-coordinate and width, are fixed, while the vertical geometric parameters of the anchor frame, i.e., the y-coordinate and height, are dynamically adjusted. S3. Train the table structure recognition model based on the row channel dataset and the column channel dataset; S4. Use the trained table structure recognition model to perform a structure recognition task on the input table image, and generate structured data containing the physical coordinates and logical positional relationships of the cells based on the recognition results.

2. The adaptive anchor box based table structure recognition method according to claim 1, wherein, Step S1 includes the following specific steps: S11. Collect and label several image samples covering multiple typical tables to obtain the first table structure recognition dataset; among them, the structured data includes the physical coordinates and logical positional relationships of the cells; the merging type includes labels for four merging states of the cells, namely: no merging, merging to the left, merging upwards, and bidirectional merging; S12. By constructing detection labels for inter-row blank areas and inter-column blank areas respectively from the structured data in the first table structure recognition dataset, without considering merging relationships, a second table structure recognition dataset is obtained. S13. Perform data augmentation on the second table structure recognition dataset, including horizontal flipping and normalization with set probabilities, to obtain the third table structure recognition dataset; S14. Divide the third table structure recognition dataset into a training set and a validation set according to a set ratio; S15. Preserve the original image samples and spatial relationships in the training and validation sets to construct the row channel dataset; similarly, rotate the original image samples in the training and validation sets 90° clockwise and transform the labels synchronously to construct the column channel dataset.

3. The adaptive anchor box based table structure recognition method according to claim 2, characterized in that, In step S2, the HRNet32-FPN architecture is used as the backbone feature extraction network, and weights are shared between the two input channels. When the input is a table image sample in the row channel dataset, the saliency hierarchical features of the row channel are output, and when the input is a table image sample in the column channel dataset, the saliency hierarchical features of the column channel are output.

4. The table structure recognition method based on adaptive anchor boxes according to claim 3, characterized in that, In step S2, a row detection head branch and a column detection head branch with the same structure but independent parameters are constructed based on Mask R-CNN, namely the dual-branch detection head. The input of the row detection head branch is the saliency hierarchical features of the row channels, and the output is the row detection result. The input of the column detection head branch is the saliency hierarchical features of the column channels, and the output is the post-processed column detection result. Both the row detection result and the column detection result include the corresponding detection box, confidence score, and segmentation mask. The post-processing involves rotating the original column detection result counterclockwise by 90° to restore it. The specific settings for the anchor boxes in the dual-branch detection head are as follows: In the row detection head branch, the horizontal coordinate of the anchor box is locked to the left boundary position of the original image sample, and the width is locked to the width of the table image sample; in the column detection head branch, the original image sample is processed according to the same anchor box strategy as the row detection after being rotated 90° clockwise.

5. The table structure recognition method based on adaptive anchor boxes according to claim 4, characterized in that, The reconstruction of the grid structure of the table includes the following specific steps: For the row detection results and the column detection results, respectively set corresponding thresholds to filter out results with low confidence; Based on the filtered row detection results, the vertical midpoint of each row detection box is calculated as the row center line; based on the filtered column detection results, the horizontal midpoint of each column detection box is calculated as the column center line. The center lines of rows and columns are sorted to ensure the correct topological order during mesh generation; The four corner points are determined by defining the center lines of two adjacent rows and two adjacent columns, and the grid structure of the table is determined based on the corner point positions of all cells.

6. The table structure recognition method based on adaptive anchor boxes according to claim 2, characterized in that, Step S3 employs a phased training approach, as detailed below: Phase 1: Training the backbone feature extraction network and the dual-branch detection head; where the multi-task loss is obtained by weighted fusion of row detection loss and column detection loss; the row detection loss is obtained by adding row classification loss, row detection box regression loss and row mask loss, and the column detection loss is obtained by adding column classification loss, column detection box regression loss and column mask loss; The second stage involves freezing the parameters of the backbone feature extraction network and the dual-branch detection head, and fine-tuning the merged cell classification module. The merged cell classification loss uses a dual-focus loss function to jointly optimize the merge type prediction and the starting grid. The main focus loss function is calculated for the four merge states, while the secondary focus loss function simplifies the problem into a binary classification task and calculates whether it is the starting grid separately.

7. The table structure recognition method based on adaptive anchor boxes according to claim 6, characterized in that, When calculating the row detection loss, the row classification loss uses cross-entropy loss, the row detection box regression loss only calculates the smooth L1 loss of the detection box's ordinate and height, and the detection box's x-coordinate and width do not participate in gradient updates, the row mask loss uses pixel-wise binary cross-entropy loss, and the column detection loss is calculated in the same way.

8. A table structure recognition system based on adaptive anchor boxes, characterized in that, The system employs the table structure recognition method based on adaptive anchor boxes as described in any one of claims 1 to 7; the system comprises: The data acquisition module is used to acquire various typical table image samples and label the structured data and merging types of the table cells. After multimodal processing, travel channel datasets and column channel datasets are constructed. The model building module is used to construct a table structure recognition model, which includes a backbone feature extraction network, a dual-branch detection head, and a merged cell classification module. The backbone feature extraction network is used to extract row features and column features from the input row channel image and column channel image, respectively. The dual-branch detection head is used to perform target detection based on the row features and column features, respectively detecting blank areas between rows and blank areas between columns. The detection results are used to reconstruct the grid structure of the table. The merged cell classification module is used to identify and merge cell regions that span multiple rows or columns in the grid structure. During the detection process of the dual-branch detection head, the horizontal geometric parameters of the anchor frame, namely the x-coordinate and width, are fixed, while the vertical geometric parameters of the anchor frame, namely the y-coordinate and height, are dynamically adjusted. The training module is used to train the table structure recognition model in stages based on the row channel dataset and the column channel dataset; The task execution module is used to perform a structure recognition task on the input table image using the trained table structure recognition model, and generate structured data containing the physical coordinates and logical positional relationships of the cells based on the recognition results.

9. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the table structure recognition method based on adaptive anchor boxes as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the table structure recognition method based on adaptive anchor boxes as described in any one of claims 1 to 7.

Images