Table structure recognition method, device, equipment and storage medium

Abstract

The present application discloses a table structure recognition method, device, equipment and storage medium, which can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and Internet of Vehicles. The method includes: dividing a table image into at least two text boxes; extracting features from each text box to obtain appearance features, position features, and content features of each text box; determining an appearance cumulative context feature set, a position cumulative context feature set, and a content cumulative context feature set; fusing the appearance cumulative context feature set and the position cumulative context feature set to obtain a first fused feature; fusing the position cumulative context feature set and the content cumulative context feature set to obtain a second fused feature; and obtaining table structure information of the table image based on the first fused feature and the second fused feature. The present application improves the recognition accuracy of table structure information.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a table structure recognition method, device, equipment and storage medium. Background Art

[0002] Table structure recognition technology is to recognize the structural information of a table in a table image into a machine-readable format, which contains the span dependency information of cells, rows, and columns. Table structure recognition technology has a wide range of applications, such as automatic question answering and dialogue systems.

[0003] In the related art, the method based on boundary extraction has a good recognition effect for relatively regular table scenes, but it cannot accurately recognize the table structure for complex table scenes that contain cells spanning multiple table elements. In addition, the cell boundaries of some complex tables are fuzzy, making it difficult to recognize the table structure.

[0004] Therefore, it is necessary to provide a table structure recognition method, device, equipment and storage medium, which can accurately recognize the table structure in an image. Summary of the invention

[0005] The present application provides a table structure recognition method, apparatus, device and storage medium, which can accurately recognize the table structure in an image.

[0006] In one aspect, the present application provides a table structure recognition method, the method comprising:

[0007] Divide the table image into at least two text boxes;

[0008] Extracting features from each text box to obtain appearance features, position features, and content features of each text box; the appearance features of the text box represent image features of the text box;

[0009] Determine an appearance cumulative context feature set, a position cumulative context feature set, and a content cumulative context feature set based on the appearance features, position features, and content features of each text box;

[0010] Fusing the appearance accumulated context feature set and the position accumulated context feature set to obtain a first fused feature;

[0011] Fusing the position accumulated context feature set and the content accumulated context feature set to obtain a second fused feature;

[0012] Based on the first fusion feature and the second fusion feature, table structure information of the table image is obtained; the table structure information includes row information, column information and cell information of the text in each text box in the table.

[0013] Another aspect provides a table structure recognition device, the device comprising:

[0014] A text box determination module, used for dividing the table image into at least two text boxes;

[0015] A feature extraction module is used to extract features from each text box to obtain appearance features, position features and content features of each text box; the appearance features of the text box represent the image features of the text box;

[0016] A context feature set determination module, used to determine an appearance cumulative context feature set, a position cumulative context feature set, and a content cumulative context feature set based on the appearance features, position features, and content features of each text box;

[0017] A first fusion module, used for fusing the appearance accumulated context feature set and the position accumulated context feature set to obtain a first fused feature;

[0018] A second fusion module, configured to fuse the position accumulated context feature set and the content accumulated context feature set to obtain a second fusion feature;

[0019] The table structure information determination module is used to obtain the table structure information of the table image based on the first fusion feature and the second fusion feature; the table structure information includes the row information, column information and cell information of the text in each text box in the table.

[0020] On the other hand, a table structure recognition device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the table structure recognition method as described above.

[0021] On the other hand, a computer storage medium is provided, wherein the computer storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the table structure recognition method as described above.

[0022] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the table structure recognition method as described above.

[0023] The table structure recognition method, device, equipment and storage medium provided by this application have the following technical effects:

[0024] The present application divides a table image into at least two text boxes; performs feature extraction on each text box to obtain appearance features, position features and content features of each text box; the appearance features of the text box represent the image features of the text box; based on the appearance features, position features and content features of each text box, determines the appearance cumulative context feature set, the position cumulative context feature set and the content cumulative context feature set; fuses the appearance cumulative context feature set and the position cumulative context feature set to obtain a first fused feature; fuses the position cumulative context feature set and the content cumulative context feature set to obtain a second fused feature; based on the first fused feature and the second fused feature, obtains the table structure information of the table image. The present application extracts multimodal features such as appearance features, position features and content features from the table image, and constructs the cumulative context feature set of each feature in a decoupled manner, thereby improving the accuracy of the fused features corresponding to the two modal features, thereby improving the recognition accuracy of the table structure information. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a schematic diagram of a table structure recognition system provided in an embodiment of the present application;

[0027] Figure 2 It is a flowchart of a table structure recognition method provided in an embodiment of the present application;

[0028] Figure 3 It is a flowchart of a method for constructing a cumulative context feature set corresponding to a multimodal feature provided in an embodiment of the present application;

[0029] Figure 4 is a flowchart of a method for determining a first fusion feature provided in an embodiment of the present application;

[0030] Figure 5 It is a flowchart of a method for determining table structure information of a table image provided by an embodiment of the present application;

[0031] Figure 6is a flowchart of a method for determining table structure information of a table image based on the table feature set provided in an embodiment of the present application;

[0032] Figure 7 It is a schematic diagram of the structural framework of the multimodal table structure recognition method provided in an embodiment of the present application;

[0033] Figure 8 is a schematic diagram of the structure of a neural collaboration module provided in an embodiment of the present application;

[0034] Fig. 9 It is a structural schematic diagram of a table structure recognition device provided in an embodiment of the present application;

[0035] Fig.10 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is an interdisciplinary subject that covers a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics.

[0037] Specifically, the solution provided in the embodiments of the present application relates to the field of machine learning in artificial intelligence. Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all fields of artificial intelligence. The table structure recognition method of the present application can be applied to systems such as intelligent dialogue question and answer.

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] See also Figure 1 , Figure 1 is a schematic diagram of a table structure recognition system provided in an embodiment of the present application, such as Figure 1 As shown, the table structure recognition system may include at least a server 01 and a client 02 .

[0041] Specifically, in the embodiment of the present application, the server 01 may include an independently running server, or a distributed server, or a server cluster composed of multiple servers, and may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 01 may include a network communication unit, a processor, and a memory, etc. Specifically, the server 01 can be used to divide the table image into at least two text boxes; perform feature extraction on each text box to obtain the appearance features, position features, and content features of each text box; determine the appearance cumulative context feature set, the position cumulative context feature set, and the content cumulative context feature set based on the appearance features, position features, and content features of each text box; fuse the appearance cumulative context feature set and the position cumulative context feature set to obtain a first fused feature; fuse the position cumulative context feature set and the content cumulative context feature set to obtain a second fused feature; based on the first fused feature and the second fused feature, obtain the table structure information of the table image.

[0042] Specifically, in the embodiment of the present application, the client 02 may include physical devices such as smart phones, desktop computers, tablet computers, laptop computers, digital assistants, smart wearable devices, smart speakers, vehicle terminals, smart TVs, etc., and may also include software running in physical devices, such as web pages provided by some service providers to users, or applications provided by these service providers to users. Specifically, the client 02 may be used to display the table structure information of the table image.

[0043] The following introduces a table structure recognition method of this application. Figure 2 It is a flowchart of a table structure recognition method provided in an embodiment of the present application. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 2 As shown, the method may include:

[0044] S201: Divide the table image into at least two text boxes.

[0045] In the embodiment of the present application, the table image may be an image corresponding to a table, and the table image may include one or more tables. The text box may be divided according to words or phrases, for example, a word or a phrase may correspond to a text box, and the text box may be a word enclosing box, and a word enclosing box may include a phrase, a word, or a number.

[0046] S203: Extract features from each text box to obtain appearance features, position features, and content features of each text box; the appearance features of the text box represent image features of the text box.

[0047] In the embodiment of the present application, the position feature of the text box can be determined according to the geometric coordinates (x, y, w, h, θ) of the text box. The position feature can be F Geom ={g1,g2,...,gN}∈RN×d, where g=M(x / W,y / H,w / W,h / H)T, where (x,y) represents the center point of the text box, w and h represent the width and height of the text box respectively, W and H represent the width and height of the table image respectively, and M∈R4×d is the parameter matrix.

[0048] In the embodiment of the present application, a standard 50-layer residual network (Res50) with a feature pyramid network (FPN) can be used as a feature extraction network. The RoI Align method is used to extract appearance features on the pyramid feature map upsampled to a size of 1 / 4 of the original image size. App ={f1,f2,...,fN}∈RN×d. The appearance feature characterizes the image feature of the text box. The image feature of the text box may include but is not limited to the background color, shadow and font features of the text box. The font features may include font type, font size, font color, font format, character shading and other features. Among them, ROI Align is a regional feature aggregation method. The idea of ​​ROI Align is to cancel the quantization operation and use the bilinear interpolation method to obtain the image value of the pixel point with floating point coordinates, thereby converting the entire feature aggregation process into a continuous operation.

[0049] In some embodiments, a word vector model (word2vec) can be used to map the text content corresponding to each word bounding box into a semantic space. Then, a convolutional layer with a kernel size of 7×1×d and a stride of 1 is used to model the text sequence as the content feature F Cont ={t1,t2,...,tN}∈RN×d. The word2vec model can be used to map each word to a vector, which can be used to represent the relationship between words. The vector is the hidden layer of the neural network.

[0050] S205: Determine an appearance cumulative context feature set, a position cumulative context feature set, and a content cumulative context feature set based on the appearance features, position features, and content features of each text box.

[0051] In the embodiments of the present application, Figure 3 As shown, the method of determining the appearance cumulative context feature set, the position cumulative context feature set, and the content cumulative context feature set based on the appearance features, position features, and content features of each text box includes:

[0052] S2051: constructing a first graph adjacency matrix based on the appearance features of each text box;

[0053] In some embodiments, a first graph adjacency matrix may be constructed based on the appearance features of multiple file boxes; each appearance feature is first converted into a feature vector, and then the feature vector is used as a node in the graph. The context corresponding to each node is dynamically selected by a KNN algorithm, that is, each point is only connected to its five neighboring points, and the length of the connected side is determined according to the inner product function between the two points, and the calculation formula is as follows:

[0054] Gi,j App =Prod(fi,fj); where i,j∈{1,2,...,N};

[0055] KNN (K-Nearest Neighbor) algorithm is one of the most basic and simple algorithms in machine learning. It can be used for both classification and regression. KNN performs classification by measuring the distance between different feature values. The idea of ​​KNN algorithm is very simple: for any n-dimensional input vector, each corresponds to a point in the feature space, and the output is the category label or predicted value corresponding to the feature vector.

[0056] The KNN algorithm is a very special machine learning algorithm because it does not have a learning process in the general sense. Its working principle is to use training data to divide the feature vector space and use the division result as the final algorithm model. There is a sample data set, also called a training sample set, and each data in the sample set has a label, that is, we know the corresponding relationship between each data in the sample set and the classification to which it belongs. After inputting the unlabeled data, each feature of this unlabeled data is compared with the feature corresponding to the data in the sample set, and then the classification label of the data (nearest neighbor) with the closest feature in the sample is extracted. Usually the first K most similar data in the sample data set are selected, which is the origin of K in the KNN algorithm. Usually K is an integer not greater than 20. Finally, the category with the most occurrences in the K most similar data is selected as the classification of the new data. In this embodiment, K can take the value of 5 to construct the first graph adjacency matrix corresponding to the appearance feature.

[0057] S2053: constructing a second graph adjacency matrix based on the position features of each text box;

[0058] In some embodiments, a second graph adjacency matrix may be constructed based on the position features of multiple file frames; each position feature is first converted into a feature vector, and then the feature vector is used as a node in the graph. The context corresponding to each node is dynamically selected by a KNN (K=5) algorithm, that is, each point is only connected to its five neighboring points, and the length of the connected side is determined according to the inner product function between the two points, and the calculation formula is as follows:

[0059] Gi,j Geom =Prod(gi,gj);

[0060] Among them, i, j∈{1,2,...,N}.

[0061] S2055: constructing a third graph adjacency matrix based on the content features of each text box;

[0062] In some embodiments, a second graph adjacency matrix may be constructed based on content features of multiple file frames; each content feature is first converted into a feature vector, and then the feature vector is used as a node in the graph. The context corresponding to each node is dynamically selected by a KNN (K=5) algorithm, that is, each point is only connected to its five neighboring points, and the length of the connected side is determined according to the inner product function between the two points, and the calculation formula is as follows:

[0063] Gi,j Cont =Prod(gi,gj);

[0064] Where, i, j∈{1,2,...,N};

[0065] In an embodiment of the present application, a graph adjacency matrix corresponding to the appearance features, position features, and content features of the text box may be constructed respectively; and then an accumulated context feature set (Aggregated Context) corresponding to each feature may be determined.

[0066] In an embodiment of the present application, the K values ​​corresponding to the first graph adjacency matrix, the second graph adjacency matrix, and the third graph adjacency matrix are all the same. For example, the K values ​​may all be 5.

[0067] S2057: performing convolution processing on the first graph adjacency matrix according to the first graph convolution network to obtain the appearance cumulative context feature set;

[0068] In some embodiments, the first graph adjacency matrix can be input into a first graph convolutional network, which may include a three-layer stacked convolutional network (Graph Convolutional Network, GCN); the GCN includes a normalization layer and a non-linear layer; the first graph adjacency matrix is ​​first normalized and then non-linearly processed to obtain an appearance cumulative context feature set.

[0069] Among them, a layer of GCN is defined as:

[0070] Z=SELU(InstanceNorm(GXW))+X, where G is the adjacency matrix mentioned above, X is the corresponding set of modal eigenvectors, W is the weight matrix, SELU is the nonlinear layer, and InstanceNorm represents the normalization layer.

[0071] In some embodiments, the first graph convolutional network, the second graph convolutional network, and the third graph convolutional network may all be GCNs.

[0072] S2059: performing convolution processing on the second graph adjacency matrix according to the second graph convolution network to obtain the position accumulated context feature set;

[0073] In some embodiments, the second graph adjacency matrix can be input into a second graph convolutional network, which may include a three-layer stacked convolutional network (Graph Convolutional Network, GCN); the GCN includes a normalization layer and a non-linear layer; the second graph adjacency matrix is ​​first normalized and then non-linearly processed to obtain a position-accumulated context feature set.

[0074] S20511: Perform convolution processing on the third graph adjacency matrix according to the third graph convolutional network to obtain the content accumulated context feature set.

[0075] In some embodiments, the third graph adjacency matrix can be input into a third graph convolutional network, which may include a three-layer stacked convolutional network (Graph Convolutional Network, GCN); the GCN includes a normalization layer and a non-linear layer; the third graph adjacency matrix is ​​first normalized and then non-linearly processed to obtain a content accumulation context feature set.

[0076] In an embodiment of the present application, corresponding graph adjacency matrices can be constructed for the appearance features, position features, and content features of the text box, respectively, and then the cumulative context feature set of various features can be obtained through a graph convolutional network; thereby, the cumulative context feature set of multimodal features is obtained, which can improve the accuracy of the determined table structure information.

[0077] S207: Fusing the appearance accumulated context feature set and the position accumulated context feature set to obtain a first fused feature.

[0078] In an embodiment of the present application, the method further includes:

[0079] A neural synergy module is constructed based on a multi-head cross-attention network, and the neural synergy module includes two multi-head cross-attention networks.

[0080] In an embodiment of the present application, a neural collaborative module (NCM) can be constructed based on two multi-head cross-attention networks; the accumulated context feature sets of the three modalities are first sent to the NCM in pairs to obtain CA-G and CC-G, and then one NCM is used for the final fusion to obtain the collaborative graph embedding (CGE).

[0081] In some embodiments, the fusing the appearance accumulated context feature set and the position accumulated context feature set to obtain a first fused feature includes:

[0082] Based on the neural synergy module, the appearance cumulative context feature set and the position cumulative context feature set are fused to obtain a first fused feature.

[0083] In an embodiment of the present application, the neural collaboration module includes a first multi-head cross-attention network and a second multi-head cross-attention network, such as Figure 4 As shown, based on the neural collaborative module, the appearance cumulative context feature set and the position cumulative context feature set are fused to obtain a first fused feature, including:

[0084] S2071: Inputting the appearance accumulated context feature set into the first multi-head cross attention network to obtain a first result;

[0085] S2073: Inputting the position accumulated context feature set into the second multi-head cross attention network to obtain a second result;

[0086] In an embodiment of the present application, a multi-head cross attention model (MCA) may include a first multi-head cross attention network and a second multi-head cross attention network; the first multi-head cross attention network and the second multi-head cross attention network have a symmetrical structure, thereby eliminating the problem caused by the different order of sending modal features into the model.

[0087] S2075: Fusing the first result and the second result to obtain the first fusion feature.

[0088] In an embodiment of the present application, two symmetrically structured multi-head cross-attention networks can be used to process the appearance cumulative context feature set and the position cumulative context feature set respectively, and then the processing results of the two can be fused, so that the appearance features can be accurately matched with the corresponding position features, thereby improving the accuracy of the determined table structure.

[0089] In an embodiment of the present application, the neural synergy module includes a first normalization layer, a second normalization layer, a third normalization layer, a first feedforward layer, and a second feedforward layer, and the first result and the second result are fused to obtain the first fusion feature, including:

[0090] Inputting the appearance accumulated context feature set into the first multi-head cross attention network to obtain a first result;

[0091] Inputting the position accumulated context feature set into the second multi-head cross attention network to obtain a second result;

[0092] The first result and the second result are normalized based on the first normalization layer to obtain a first normalized result.

[0093] In an embodiment of the present application, the first normalization layer may be an Add&Norm layer, where Add represents a residual connection (Residual Connection) used to prevent network degradation, and Norm represents Layer Normalization, which is used to normalize the activation values ​​of each layer.

[0094] The first normalization result is processed based on the first feed-forward layer to obtain a first processing result.

[0095] In an embodiment of the present application, the first feed-forward layer may be a FFN (position-wise feed-forward networks).

[0096] Normalizing the first result and the second result based on the second normalization layer to obtain a second normalized result;

[0097] In an embodiment of the present application, the second normalization layer may be an Add&Norm layer, where Add represents a residual connection (Residual Connection) used to prevent network degradation, and Norm represents Layer Normalization, which is used to normalize the activation values ​​of each layer.

[0098] The second normalization result is processed based on the second feed-forward layer to obtain a second processing result.

[0099] In an embodiment of the present application, the second feed-forward layer may be FFN (position-wise feed-forward networks).

[0100] In an embodiment of the present application, after obtaining the first processing result and the second processing result, the first processing result can be re-used as the first result, and the second processing result can be re-used as the second result; and the above processing steps for the first result and the second result are repeated to obtain the first updated processing result and the second updated processing result.

[0101] The first processing result and the second processing result are fused based on the third normalization layer to obtain the first fused feature.

[0102] In the embodiment of the present application, the features can be normalized through multiple normalization layers, and the normalized features can be processed through a feed-forward layer to obtain a first fusion feature with a higher fusion accuracy.

[0103] In the embodiment of the present application, the fusing the first processing result and the second processing result based on the third normalization layer to obtain the first fusion feature may include:

[0104] The first update processing result and the second update processing result are fused based on the third normalization layer to obtain the first fused feature.

[0105] In an embodiment of the present application, the third normalization layer can be an Add&Norm layer, where Add represents a residual connection (Residual Connection) used to prevent network degradation, and Norm represents Layer Normalization, which is used to normalize the activation values ​​of each layer.

[0106] S209: Fusing the location accumulated context feature set and the content accumulated context feature set to obtain a second fused feature.

[0107] In the embodiment of the present application, the fusion processing of the position accumulated context feature set and the content accumulated context feature set to obtain the second fusion feature includes:

[0108] Based on the neural synergy module, the position accumulated context feature set and the content accumulated context feature set are fused to obtain a second fused feature.

[0109] In the embodiment of the present application, the features can be normalized through multiple normalization layers, and the normalized features can be processed through a feedforward layer to obtain a second fusion feature with a higher fusion accuracy.

[0110] In the embodiment of the present application, the fusion processing of the position accumulated context feature set and the content accumulated context feature set to obtain the second fusion feature includes:

[0111] Inputting the position accumulated context feature set into the first multi-head cross attention network to obtain a third result;

[0112] The content accumulated context feature set is input into the second multi-head cross attention network to obtain a fourth result.

[0113] In an embodiment of the present application, the first multi-head cross-attention network and the second multi-head cross-attention network can be the same network, and the input features of the two networks can be swapped.

[0114] The third result and the fourth result are fused to obtain the second fusion feature.

[0115] In an embodiment of the present application, two symmetrically structured multi-head cross-attention networks can be used to process the position-accumulated context feature set and the content-accumulated context feature set respectively, and then the processing results of the two can be fused, so that the position features can be accurately matched with the corresponding content features, thereby improving the accuracy of the determined table structure.

[0116] In the embodiment of the present application, the fusing process of the third result and the fourth result to obtain the second fusion feature includes:

[0117] Normalizing the third result and the fourth result based on the first normalization layer to obtain a third normalized result;

[0118] Processing the third normalized result based on the first feed-forward layer to obtain a third processed result;

[0119] Normalizing the third result and the fourth result based on the second normalization layer to obtain a fourth normalized result;

[0120] Processing the fourth normalization result based on the second feed-forward layer to obtain a fourth processing result;

[0121] In an embodiment of the present application, after obtaining the third processing result and the fourth processing result, the third processing result can be reused as the third result, and the fourth processing result can be reused as the fourth result; and the above processing steps for the third result and the fourth result are repeated to obtain the third updated processing result and the fourth updated processing result.

[0122] The third processing result and the fourth processing result are fused based on the third normalization layer to obtain the second fused feature.

[0123] In the embodiments of the present application, Figure 8 As shown in the figure, the outputs M1 and M2 of the two MCAs pass through the "interlocking gate" structure to control the information flow of the two modes to obtain P1 and P2. Among them, ⊙ represents the Hadamard inner product, and σ represents the sigmoid function. After that, the final output is obtained by passing through several "Add&Norm" and "FFN" layers. Add represents the residual connection (Residual Connection) used to prevent network degradation, and Norm represents LayerNormalization, which is used to normalize the activation values ​​of each layer.

[0124] Where "Add&Norm" represents the addition operation and layer normalization operation, and "FFN" represents the feed-forward layer. The whole process is defined as follows:

[0125] M1=MCA1→2(Q1, K2, V2), M2=MCA2→1(Q2, K1, V1);

[0126] P1=σ(M2)⊙M1+M1, P2=σ(M1)⊙M2+M2;

[0127] A1=Add&Norm(Q1,P1), A2=Add&Norm(Q2,P2);

[0128] Y1=Add&Norm(FFN(A1),A1), Y2=Add&Norm(FFN(A2),A2);

[0129] Where MCA1→2 represents the first MCA, and Q=X1, K=X2, V=X2; MCA2→1 represents another MCA, where Q=X2, K=X1, V=X1. The NCM module can accumulate N layers, and this method sets N to 3. Finally, the output accumulated after N layers is fused through the following operation Y=Add&Norm(Y1(N), Y2(N)). In this way, the model can capture the two-way interaction of the context features of the two modalities, which will play a positive role in predicting the table structure. At this point, the collaborative graph embedding E={e1, e2,..., eN}∈RN×d with inter-modal contextual interaction information is obtained.

[0130] S2011: Based on the first fusion feature and the second fusion feature, obtain table structure information of the table image; the table structure information includes row information, column information and cell information of the text in each text box in the table.

[0131] In the embodiments of the present application, Figure 5 As shown, the table structure information of the table image is obtained based on the first fusion feature and the second fusion feature, including:

[0132] S20111: Based on a neural collaboration module, the first fusion feature and the second fusion feature are fused to obtain a collaboration graph embedding result;

[0133] S20113: Parse the collaborative graph embedding result to obtain table structure information of the table image.

[0134] In some embodiments, Figure 6 As shown, the collaborative graph embedding result is parsed to obtain the table structure information of the table image, including:

[0135] S201131: Based on the collaborative graph embedding result, determine at least two table feature pairs, each set of table feature pairs includes a preset number of table features, and the table features include content features and appearance features and position features corresponding to the content features.

[0136] In an embodiment of the present application, at least two table feature pairs can be determined based on the collaborative graph embedding result, and each group of table feature pairs includes a preset number of table features; the preset number can be the number of features in the graph adjacency matrix. For example, the K value in the graph adjacency matrix is ​​5, that is, a feature and its 5 adjacent features constitute the graph adjacency matrix. The features corresponding to a graph adjacency matrix can be used as a table feature pair; a table feature is constructed based on the graph adjacency matrix corresponding to the three modal features; each content feature can be matched and fused with its corresponding appearance feature and position feature according to the three graph adjacency matrices to obtain the fused table feature.

[0137] S201133: performing a series operation on the at least two table feature pairs according to the dimension information to obtain a table feature set; the table features in the table feature set are located in the same dimension;

[0138] In an embodiment of the present application, table feature pairs may be serially connected along corresponding dimensional axes (channel axes) to obtain table feature sets; the obtained table feature sets may be multiple, and the table features in each table feature set are located in the same dimension.

[0139] S201135: Based on the table feature set, determine the table structure information of the table image.

[0140] In some embodiments, the neural synergy module includes a fully connected layer, and determining the table structure information of the table image based on the table feature set includes:

[0141] Classify the table features in the table feature set based on the fully connected layer to determine a first table feature set located in the same row, a second table feature set located in the same column, and a third table feature set located in the same cell;

[0142] In the embodiment of the present application, three groups of fully connected layers (FC layers) can be used to predict the binary classification results of the features in the table feature set, that is, it can be predicted whether any two feature vectors in the table feature set, such as the i-th and j-th feature vectors, belong to the same row, the same column, or the same cell relationship. Thus, the first table feature set located in the same row, the second table feature set located in the same column, and the third table feature set located in the same cell are obtained.

[0143] Based on the first table feature set, the second table feature set, and the third table feature set, table structure information of the table image is determined.

[0144] In an embodiment of the present application, the table structure information of the table image can be determined based on the first table feature set located in the same row, the second table feature set located in the same column, and the third table feature set located in the same cell, that is, the number of rows and columns of the table and the text information in each cell are determined, so that the table corresponding to the table image can be constructed.

[0145] In an embodiment of the present application, a collaborative graph with inter-modal contextual interaction information is embedded E={e1,e2,...,eN}∈RN×d. Based on this embedding, it is constructed into sample pairs and connected in series along the channel axis to obtain sample pair features U={u1,1,u1,2,...,ui,j,...,uN,N}∈RN2×2de, and then three groups of FC layers are used to predict the binary classification relationship of U, that is, whether the i-th and j-th CGE vectors belong to the same row, column or cell relationship. The feature vectors of the same row, column or cell can be set with the same identification information, and the identification information can be a color identification, font size identification, font type identification, font format identification, etc.; for example, text boxes with the same color indicate that they belong to the same structural relationship. Each FC group consists of 3 256-dimensional FC layers and a two-dimensional FC layer containing a normalization (softmax) layer.

[0146] In some embodiments, Figure 7 As shown, Figure 7 The schematic diagram of the structure framework of the multimodal table structure recognition method is shown in Figure 1. First, the multimodal features are extracted to obtain the appearance features F of each text box in the table image. App , Geometric features (position features) F Geom and content feature F Cont ; Then get the graph adjacency matrix of each feature and get the appearance graph G App , geometric figure G Geom and content graph G Cont ; Then input each image into the GCN network to obtain the cumulative context feature set corresponding to each feature; then the appearance cumulative context feature set C App And the position accumulated context feature set C Geom Input an NCM for fusion, get the first fusion result, and accumulate the context feature set C Geom and the content accumulated context feature set C Cont Input another NCM for fusion to obtain a second fusion result. Input the first fusion result and the second fusion result into an NCM for fusion to obtain a collaborative graph embedding, thereby constructing a sample pair and obtaining a prediction result of the table structure information, including row relationships, column relationships, and cell relationships. In practical applications, each network can be trained in advance, and feature processing can be performed after the training is completed.

[0147] This application extracts multimodal features of table elements from the appearance dimension, geometric dimension and content dimension respectively, and constructs their contextual relationships as a graph in a decoupled manner. In order to make full use of their contextual interactions within the modality, GCN is used for context accumulation. It is then sent to the proposed attention-based neural collaboration module (NCM) to generate a collaborative graph embedding (CGE) with inter-modal interactions. Based on CGE, the relationship between table elements is predicted through the structure prediction module to obtain the final table structure. CGE can better extract and model the relationships within and between different modalities of the table, thereby realizing fast and accurate recognition of table structures in table images.

[0148] In one embodiment, the structure recognition of table images is performed using the related technologies Graph TSR (Table Structure Recognition), image analysis convolutional neural networks (Dynamic Graph Convolutional Neural Networks, DGCNN) and the method of this embodiment, and the obtained test results are shown in Table 1 below, where F1-score is a comprehensive evaluation indicator of recall and precision.

[0149] Table 1

[0150] method Accuracy recall F1-score GraphTSR 95.9 94.8 95.3 DGCNN 97.0 98.1 97.6 This embodiment 99.4 99.2 99.3

[0151] It can be seen from the technical solution provided by the above embodiment of the present application that the embodiment of the present application divides the table image into at least two text boxes; performs feature extraction on each text box to obtain the appearance features, position features and content features of each text box; the appearance features of the text box represent the image features of the text box; based on the appearance features, position features and content features of each text box, the appearance cumulative context feature set, the position cumulative context feature set and the content cumulative context feature set are determined; the appearance cumulative context feature set and the position cumulative context feature set are fused to obtain a first fused feature; the position cumulative context feature set and the content cumulative context feature set are fused to obtain a second fused feature; based on the first fused feature and the second fused feature, the table structure information of the table image is obtained. The present application extracts multimodal features such as appearance features, position features and content features in the table image, and constructs the cumulative context feature set of each feature in a decoupled manner, thereby improving the accuracy of the fused features corresponding to the two modal features, thereby improving the recognition accuracy of the table structure information.

[0152] The present application also provides a table structure recognition device, such as Fig. 9 As shown, the device comprises:

[0153] A text box determination module 910, used to divide the table image into at least two text boxes;

[0154] The feature extraction module 920 is used to extract features from each text box to obtain appearance features, position features and content features of each text box; the appearance features of the text box represent the image features of the text box;

[0155] A context feature set determination module 930, configured to determine an appearance cumulative context feature set, a position cumulative context feature set, and a content cumulative context feature set based on the appearance features, position features, and content features of each text box;

[0156] A first fusion module 940 is used to fuse the appearance accumulated context feature set and the position accumulated context feature set to obtain a first fused feature;

[0157] A second fusion module 950 is used to fuse the position accumulated context feature set and the content accumulated context feature set to obtain a second fused feature;

[0158] The table structure information determination module 960 is used to obtain the table structure information of the table image based on the first fusion feature and the second fusion feature; the table structure information includes the row information, column information and cell information of the text in each text box in the table.

[0159] In some embodiments, the context feature set determination module may include:

[0160] A first graph adjacency matrix construction unit, configured to construct a first graph adjacency matrix based on the appearance features of each text box;

[0161] A second graph adjacency matrix construction unit, used to construct a second graph adjacency matrix based on the position features of each text box;

[0162] A third graph adjacency matrix construction unit, used to construct a third graph adjacency matrix based on the content features of each text box;

[0163] an appearance cumulative context feature set determining unit, configured to perform convolution processing on the first graph adjacency matrix according to a first graph convolutional network to obtain the appearance cumulative context feature set;

[0164] A position-accumulated context feature set determining unit, configured to perform convolution processing on the second graph adjacency matrix according to a second graph convolutional network to obtain the position-accumulated context feature set;

[0165] The content accumulation context feature set determination unit is used to perform convolution processing on the third graph adjacency matrix according to the third graph convolution network to obtain the content accumulation context feature set.

[0166] In some embodiments, the apparatus may further include:

[0167] A neural collaborative module building module is used to build a neural collaborative module based on a multi-head cross-attention network, and the neural collaborative module includes two multi-head cross-attention networks.

[0168] The first fusion module includes:

[0169] The first fusion feature determination unit is used to fuse the appearance cumulative context feature set and the position cumulative context feature set based on the neural synergy module to obtain a first fusion feature.

[0170] The second fusion module includes:

[0171] The second fusion feature determination unit is used to fuse the position accumulated context feature set and the content accumulated context feature set based on the neural synergy module to obtain a second fusion feature.

[0172] In some embodiments, the neural synergy module includes a first multi-head cross-attention network and a second multi-head cross-attention network, and the first fusion feature determination unit includes:

[0173] A first result determination subunit, configured to input the appearance accumulated context feature set into the first multi-head cross attention network to obtain a first result;

[0174] A second result determination subunit is used to input the position accumulated context feature set into the second multi-head cross attention network to obtain a second result;

[0175] The first fusion feature determination subunit is used to fuse the first result and the second result to obtain the first fusion feature.

[0176] In some embodiments, the neural synergy module includes a first normalization layer, a second normalization layer, a third normalization layer, a first feedforward layer, and a second feedforward layer, and the first fusion feature determination subunit includes:

[0177] a first normalization result determining subunit, configured to perform normalization processing on the first result and the second result based on the first normalization layer to obtain a first normalization result;

[0178] a first processing result determining subunit, configured to process the first normalization result based on the first feed-forward layer to obtain a first processing result;

[0179] a second normalization result determination subunit, configured to perform normalization processing on the first result and the second result based on the second normalization layer to obtain a second normalization result;

[0180] a second processing result determining subunit, configured to process the second normalization result based on the second feed-forward layer to obtain a second processing result;

[0181] The first fusion feature subunit is used to fuse the first processing result and the second processing result based on the third normalization layer to obtain the first fusion feature.

[0182] In some embodiments, the table structure information determination module includes:

[0183] A collaborative graph embedding result determining unit, configured to fuse the first fusion feature and the second fusion feature based on a neural collaborative module to obtain a collaborative graph embedding result;

[0184] The table structure information determining unit is used to parse the collaborative graph embedding result to obtain the table structure information of the table image.

[0185] In some embodiments, the table structure information determining unit includes:

[0186] A table feature pair determination subunit, configured to determine at least two table feature pairs based on the collaborative graph embedding result, each set of table feature pairs including a preset number of table features, the table features including content features and appearance features and position features corresponding to the content features;

[0187] a table feature set determination subunit, configured to perform a series operation on the at least two table feature pairs according to the dimension information to obtain a table feature set; wherein the table features in the table feature set are located in the same dimension;

[0188] The table structure information determination subunit is used to determine the table structure information of the table image based on the table feature set.

[0189] In some embodiments, the neural synergy module includes a fully connected layer, and the table structure information determination subunit includes:

[0190] A classification subunit, configured to classify the table features in the table feature set based on the fully connected layer, and determine a first table feature set located in the same row, a second table feature set located in the same column, and a third table feature set located in the same cell;

[0191] An information determination subunit is used to determine the table structure information of the table image based on the first table feature set, the second table feature set and the third table feature set.

[0192] The device and method embodiments in the described device embodiments are based on the same inventive concept.

[0193] An embodiment of the present application provides a table structure recognition device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the table structure recognition method provided in the above method embodiment.

[0194] An embodiment of the present application also provides a computer storage medium, which can be set in a terminal to store at least one instruction or at least one program related to a table structure recognition method in a method embodiment, and the at least one instruction or at least one program is loaded and executed by the processor to implement the table structure recognition method provided by the above method embodiment.

[0195] The embodiment of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the table structure recognition method provided by the above method embodiment.

[0196] Optionally, in the embodiment of the present application, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in the present embodiment, the storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0197] The memory described in the embodiment of the present application can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs required for functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0198] The table structure recognition method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Fig.10 1 is a hardware structure block diagram of a server of a table structure recognition method provided in an embodiment of the present application. Fig.10 As shown, the server 1000 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1010 (the central processing unit 1010 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1030 for storing data, and one or more storage media 1020 (such as one or more mass storage devices) for storing application programs 1023 or data 1022. Among them, the memory 1030 and the storage medium 1020 can be short-term storage or permanent storage. The program stored in the storage medium 1020 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the central processing unit 1010 can be configured to communicate with the storage medium 1020 and execute a series of instruction operations in the storage medium 1020 on the server 1000. The server 1000 may also include one or more power supplies 1060, one or more wired or wireless network interfaces 1050, one or more input and output interfaces 1040, and / or one or more operating systems 1021, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0199] The input / output interface 1040 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the server 1000. In one example, the input / output interface 1040 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices through a base station so as to communicate with the Internet. In one example, the input / output interface 1040 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0200] It can be understood by those skilled in the art that Fig.10 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.10 More or fewer components as shown, or with Fig.10 Different configurations shown.

[0201] It can be seen from the embodiments of the table structure recognition method, device, equipment or storage medium provided by the present application that the present application divides the table image into at least two text boxes; performs feature extraction on each text box to obtain the appearance features, position features and content features of each text box; the appearance features of the text box represent the image features of the text box; based on the appearance features, position features and content features of each text box, the appearance cumulative context feature set, the position cumulative context feature set and the content cumulative context feature set are determined; the appearance cumulative context feature set and the position cumulative context feature set are fused to obtain a first fused feature; the position cumulative context feature set and the content cumulative context feature set are fused to obtain a second fused feature; based on the first fused feature and the second fused feature, the table structure information of the table image is obtained. The present application extracts multimodal features such as appearance features, position features and content features in the table image, and constructs the cumulative context feature set of each feature in a decoupled manner, thereby improving the accuracy of the fused features corresponding to the two modal features, thereby improving the recognition accuracy of the table structure information.

[0202] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of this specification are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0203] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0204] A person skilled in the art will appreciate that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0205] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A table structure recognition method, characterized in that: The method comprises: Divide the table image into at least two text boxes; Performing feature extraction on each text box to obtain appearance features, position features, and content features of each text box; the appearance features of the text box represent image features of the text box; Determine an appearance cumulative context feature set, a position cumulative context feature set, and a content cumulative context feature set based on the appearance features, position features, and content features of each text box; Fusing the appearance accumulated context feature set and the position accumulated context feature set to obtain a first fused feature; Fusing the position accumulated context feature set and the content accumulated context feature set to obtain a second fused feature; Based on the first fusion feature and the second fusion feature, table structure information of the table image is obtained; the table structure information includes row information, column information and cell information of the text in each text box in the table.

2. The method according to claim 1, characterized in that The determining of the appearance cumulative context feature set, the position cumulative context feature set, and the content cumulative context feature set based on the appearance features, the position features, and the content features of each text box includes: Based on the appearance features of each text box, construct a first graph adjacency matrix; Based on the position features of each text box, construct a second graph adjacency matrix; Based on the content features of each text box, construct a third graph adjacency matrix; Performing convolution processing on the first graph adjacency matrix according to the first graph convolutional network to obtain the appearance accumulated context feature set; Performing convolution processing on the second graph adjacency matrix according to the second graph convolution network to obtain the position accumulated context feature set; The third graph adjacency matrix is ​​convolved according to the third graph convolutional network to obtain the content accumulation context feature set.

3. The method according to claim 1, characterized in that The method further comprises: Building a neural synergy module based on a multi-head cross-attention network, wherein the neural synergy module includes two multi-head cross-attention networks; The fusing the appearance accumulated context feature set and the position accumulated context feature set to obtain a first fused feature includes: Based on the neural synergy module, the appearance cumulative context feature set and the position cumulative context feature set are fused to obtain a first fused feature; The fusing the position accumulated context feature set and the content accumulated context feature set to obtain a second fused feature includes: Based on the neural synergy module, the position accumulated context feature set and the content accumulated context feature set are fused to obtain a second fused feature.

4. The method according to claim 3, characterized in that: The neural synergy module includes a first multi-head cross attention network and a second multi-head cross attention network. Based on the neural synergy module, the appearance cumulative context feature set and the position cumulative context feature set are fused to obtain a first fused feature, including: Inputting the appearance accumulated context feature set into the first multi-head cross attention network to obtain a first result; Inputting the position accumulated context feature set into the second multi-head cross attention network to obtain a second result; The first result and the second result are fused to obtain the first fusion feature.

5. The method according to claim 4, characterized in that The neural synergy module includes a first normalization layer, a second normalization layer, a third normalization layer, a first feedforward layer, and a second feedforward layer. The first result and the second result are fused to obtain the first fusion feature, including: Normalizing the first result and the second result based on the first normalization layer to obtain a first normalized result; Processing the first normalization result based on the first feed-forward layer to obtain a first processing result; Normalizing the first result and the second result based on the second normalization layer to obtain a second normalized result; Processing the second normalized result based on the second feed-forward layer to obtain a second processing result; The first processing result and the second processing result are fused based on the third normalization layer to obtain the first fused feature.

6. The method according to claim 3, characterized in that: The obtaining the table structure information of the table image based on the first fusion feature and the second fusion feature includes: Based on the neural collaboration module, the first fusion feature and the second fusion feature are fused to obtain a collaboration graph embedding result; The collaborative graph embedding result is parsed to obtain table structure information of the table image.

7. The method according to claim 6, characterized in that The step of parsing the collaborative graph embedding result to obtain the table structure information of the table image includes: Based on the collaborative graph embedding result, determining at least two table feature pairs, each set of table feature pairs includes a preset number of table features, and the table features include content features and appearance features and position features corresponding to the content features; The at least two table feature pairs are connected in series according to the dimension information to obtain a table feature set; the table features in the table feature set are located in the same dimension; Based on the table feature set, table structure information of the table image is determined.

8. The method according to claim 7, characterized in that The neural synergy module includes a fully connected layer, and the determining of the table structure information of the table image based on the table feature set includes: Classify the table features in the table feature set based on the fully connected layer to determine a first table feature set located in the same row, a second table feature set located in the same column, and a third table feature set located in the same cell; Based on the first table feature set, the second table feature set, and the third table feature set, table structure information of the table image is determined.

9. A table structure recognition device, characterized in that: The device comprises: A text box determination module, used for dividing the table image into at least two text boxes; A feature extraction module is used to extract features from each text box to obtain appearance features, position features and content features of each text box; the appearance features of the text box represent the image features of the text box; A context feature set determination module, used to determine an appearance cumulative context feature set, a position cumulative context feature set, and a content cumulative context feature set based on the appearance features, position features, and content features of each text box; A first fusion module, configured to fuse the appearance accumulated context feature set and the position accumulated context feature set to obtain a first fused feature; A second fusion module, configured to fuse the position accumulated context feature set and the content accumulated context feature set to obtain a second fusion feature; The table structure information determination module is used to obtain the table structure information of the table image based on the first fusion feature and the second fusion feature; the table structure information includes the row information, column information and cell information of the text in each text box in the table.

10. A table structure recognition device, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the table structure recognition method as described in any one of claims 1-8.

11. A computer storage medium, characterized in that: The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the table structure recognition method according to any one of claims 1 to 8.

12. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the table structure recognition method as described in any one of claims 1-8 is implemented.

Images