Table positioning methods, apparatus, devices and computer-readable storage media

Abstract

This application provides a table location method, apparatus, device, and computer-readable storage medium. The method includes: determining the line segment positions of line segment objects and the text positions and text content of text objects in a file to be processed, including text objects and line segment objects; generating text vectors based on the text positions and text content; generating line segment vectors based on the text positions and line segment positions; inputting the text vectors and line segment vectors into a multi-channel composite model to obtain prediction results; determining the target text object located in the table based on the prediction results; and determining the table location result based on the target text object. Thus, by combining the features of the line segment positions, text positions, and text content information in the file to be processed using a multi-channel composite model, the table in the file to be processed can be located.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and to, but is not limited to, a table positioning method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Portable Document Format (PDF) documents boast excellent cross-platform compatibility, and PDF has become one of the most widely used electronic document formats today. However, due to inherent formatting limitations, table data in PDF documents lacks specific definitions; it's simply a combination of lines and text positions, making it often difficult to position tables within a PDF document. Summary of the Invention

[0003] In view of this, this application provides a table positioning method, apparatus, device, and computer-readable storage medium, which at least solves the problem of difficulty in locating tables in a document.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] At least one embodiment of this application provides a table positioning method, wherein the file to be processed includes text objects and line segment objects, the method comprising:

[0006] Determine the position of line segments in line segment objects and the position and content of text objects in the file to be processed;

[0007] Generate a text vector based on the text position and the text content;

[0008] Generate a line segment vector based on the text position and the line segment position;

[0009] The text vector and the line segment vector are input into a multi-channel composite model to obtain the prediction result;

[0010] The target text object located in the table is determined based on the prediction result, and the table positioning result is determined based on the target text object.

[0011] At least one embodiment of this application provides a table positioning device, wherein the file to be processed includes text objects and line segment objects, the device comprising:

[0012] The first determining module is used to determine the line segment position of the line segment object in the file to be processed, as well as the text position and text content of the text object.

[0013] The first generation module is used to generate a text vector based on the text position and the text content;

[0014] The second generation module is used to generate a line segment vector based on the text position and the line segment position;

[0015] The input module is used to input the text vector and the line segment vector into the multi-channel composite model to obtain the prediction result;

[0016] The second determining module is used to determine the target text object located in the table based on the prediction result;

[0017] The third determining module is used to determine the table positioning result based on the target text object.

[0018] At least one embodiment of this application provides a table positioning device, comprising:

[0019] Processor; and

[0020] Memory for storing computer programs that can run on the processor;

[0021] The computer program, when executed by the processor, implements the steps of the table positioning method described above.

[0022] At least one embodiment of this application provides a computer-readable storage medium storing computer-executable instructions configured to perform the steps of the table positioning method described above.

[0023] The table localization method provided in this application first determines the line segment positions of the line segment objects and the text positions and text content of the text objects from the file to be processed, which includes text objects and line segment objects. Then, it generates text vectors based on the text positions and text content, and generates line segment vectors based on the text positions and line segment positions. Next, it inputs the text vectors and line segment vectors into a pre-trained multi-channel composite model to obtain prediction results. Finally, it determines the target text object located in the table based on the prediction results, and then determines the table localization result based on the target text object. In this way, by combining the features between line segment positions, text positions, and text content information through a multi-channel composite model, it achieves table localization in the file to be processed, effectively solving the problem of locating tables in a document. Attached Figure Description

[0024] In the accompanying drawings (which are not necessarily drawn to scale), similar reference numerals may describe similar parts in different views. The drawings illustrate, by way of example and not limitation, the various embodiments discussed herein.

[0025] Figure 1 A schematic diagram illustrating an implementation flow of the table positioning method provided in this application embodiment;

[0026] Figure 2A schematic diagram illustrating another implementation flow of the table positioning method provided in the embodiments of this application;

[0027] Figure 3 This is a flowchart illustrating a PDF table location method that combines text semantics and location information, as proposed in an embodiment of this application.

[0028] Figure 4 A schematic diagram illustrating the representation of text vectors provided in an embodiment of this application;

[0029] Figure 5 A schematic diagram illustrating the representation of the feature matrix provided in an embodiment of this application;

[0030] Figure 6 The network architecture diagram of the multi-channel TextCNN composite model provided in the embodiments of this application;

[0031] Figure 7 A schematic diagram illustrating the training process of the multi-channel TextCNN composite model provided in this application embodiment;

[0032] Figure 8 A schematic diagram of the table positioning result output by the table positioning method provided in the embodiments of this application;

[0033] Figure 9 This is a schematic diagram of the composition of the table positioning device provided in the embodiments of this application;

[0034] Figure 10 This is a schematic diagram of the composition structure of the table positioning device provided in the embodiments of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0037] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0039] To address the problems existing in related technologies, this application provides a table positioning method, which is applied to a table positioning device. The method provided in this application can be implemented by a computer program, which, when executed, completes each step of the table positioning method provided in this application. In some embodiments, the computer program can be executed by a processor in the table positioning device. Figure 1 This is a schematic diagram illustrating an implementation flow of the table positioning method provided in an embodiment of this application, such as... Figure 1 As shown, the table positioning method includes the following steps:

[0040] Step S101: Determine the line segment position of the line segment object and the text position and text content of the text object in the file to be processed.

[0041] The method provided in this application embodiment can be executed by a table positioning device. This table positioning device can be a user equipment (UE), mobile device, terminal, laptop, tablet, desktop computer, or other device capable of performing table positioning.

[0042] The table positioning method provided in this application can be applied to locating tables in documents such as PDFs where tables are not defined and are merely combinations of text and lines. It can also be applied to locating tables in documents with specific definitions or tags for table data, such as DOC (Document), RTF (Rich Text Format), and HTML (Hypertext Markup Language) formats. The following explanation uses locating tables in PDF documents as an example.

[0043] First, obtain the file to be processed. The file to be processed can be a PDF file. The file to be processed includes text objects and line objects. The text objects can include text objects in the table as well as text objects outside the table. The text objects can be text (including Chinese, English, numbers and other types of text). The line objects can be line segments in the table.

[0044] After the table positioning device obtains the original file to be used for table positioning, if the original file does not include scanned pages (i.e., the original file only contains tables composed of text and lines, and does not contain images or other forms of tables), then the original file can be determined as the file to be processed. If the original file includes scanned pages, then the Optical Character Recognition (OCR) method can be used to locate the tables in the scanned pages. The method of this application embodiment is used to locate the tables in the non-scanned pages. One implementation of obtaining the file to be processed includes: obtaining the original file to be used for table positioning, where the original file contains at least one page; selecting a target page from the at least one page, the target page including text objects and line segment objects; and generating the file to be processed based on the target page. After obtaining the original file, the table positioning device deletes the scanned pages in the original file, and the remaining non-scanned pages are the target pages. The non-scanned pages are used to assemble the file to be processed. If the original file only contains scanned pages, then the file to be processed is empty.

[0045] After obtaining the file to be processed, it is parsed to extract text objects and line segment objects, while also obtaining the text information of the text objects and the line segment information of the line segment objects. In one implementation, the open-source Java library PDFBox can be used to extract the text objects and line segment objects, as well as the text information of the text objects and the line segment information of the line segment objects.

[0046] The text information includes the text position and text content. The extracted text information can be represented as: [x, y, char, width, height]. Here, x represents the horizontal coordinate of the text object, y represents the vertical coordinate of the text object, char represents the text content of the text object, width represents the text width of the text object, and height represents the text height of the text object. When the (x, y) coordinate point is the position of the bottom left pixel of the text object, the coordinate point (x + width, y) can be the position of the bottom right pixel of the text object; the coordinate point (x, y + height) can be the position of the top left pixel of the text object; and the coordinate point (x + width, y + height) can be the position of the top right pixel of the text object. This is how the text position of the text object is obtained.

[0047] The line segment information includes the line segment position, and the extracted line segment information can be represented as: [x0, y0, x1, y1]. Here, coordinate point (x0, y0) is the starting point of the line segment object, and coordinate point (x1, y1) is the ending point of the line segment object. x0 represents the x-coordinate of the starting point, y0 represents the y-coordinate of the starting point, x1 represents the x-coordinate of the ending point, and y1 represents the y-coordinate of the ending point.

[0048] Step S102: Generate a text vector for the text object based on the text position and text content.

[0049] Text representation is a fundamental task in natural language processing (NLP), and the quality of text representation directly impacts the performance of the entire NLP system. Text vectorization, which represents text as a series of vectors that express its semantics, is an important method of text representation. Each text object in the file to be recognized is vectorized to determine whether each text object is present in the table.

[0050] Most current research on text vectorization focuses on word vectorization, while some researchers use sentences as the basic unit of text processing, leading to the development of word2vec and str2vec technologies. Since the text in tables typically consists of attributes, features, or other words, this embodiment uses individual characters as the basic unit for vectorization, employing word2vec technology to achieve text content vectorization.

[0051] In the vectorization process of text objects, this application considers not only the text position but also the text content. By vectorizing the text object based on the text position and text content, the resulting text vector can not only determine which "character" the text object is, but also the position of the character, so that each text object corresponds to a unique text vector.

[0052] Step S103: Generate a line segment vector for the line segment object based on the text position and the line segment position.

[0053] Generally, text closer to a line segment is more likely to be in a table than text farther away. However, related technologies do not consider the spatial relationship between text and line segments when locating tables; they only rely on the spatial position of the text. This leads to misjudgments in handling poorly defined tables or situations where table line segments are close to text, resulting in low accuracy in table location.

[0054] In this embodiment of the application, when performing text vectorization, not only the text position and text content are considered, but also the line segment objects are vectorized according to the text position and the line segment position. The spatial positional relationship between the text and the line segment is considered. By combining the semantic relationship between the text and the spatial positional relationship between the text and the line segment, it is possible to determine whether each text object is in the table, which can improve the positioning accuracy.

[0055] In this embodiment of the application, the execution order of steps S102 and S103 is not limited. Step S102 can be executed first, or step S103 can be executed first, or steps S102 and S103 can be executed in parallel.

[0056] Step S104: Input the text vector and line segment vector into the multi-channel composite model to obtain the prediction result.

[0057] The table positioning device can pre-initialize a multi-channel composite model, which has multiple input channels and can be used to determine whether a text object is text within a table. In practical applications, this multi-channel composite model can be pre-trained using training samples from a training set.

[0058] During table localization, the text vectors obtained in step S102 and the line segment vectors obtained in step S103 are input into the multi-channel composite model to obtain the prediction results for each text object in the file to be processed. These prediction results reflect whether each text object is a text object in the table, i.e., whether it is a target text object. For example, for a certain text object, if the prediction result is "yes", it indicates that the text object is a text object in the table, i.e., the text object is a target text object; if the prediction result is "no", it indicates that the text object is not a text object in the table, i.e., the text object is outside the table, and the text object is not a target text object.

[0059] In this embodiment, the multi-channel composite model can be a multi-channel TextConvolutional Neural Network (TextCNN) composite model. The most significant difference between this multi-channel TextCNN composite model and the traditional TextCNN model is that the input has multiple channels, at least two channels: one channel's input vector is updated, while the other channel's input vector is not updated.

[0060] This application embodiment combines the features of line segment positions, text positions, and text content information in the file to be processed based on a multi-channel composite model to locate tables in the file to be processed. It can effectively solve problems such as misidentification of non-table text lines as table data, incomplete table data recognition, and omission of tables when recognizing tables in the file to be processed, thereby improving the accuracy of table recognition.

[0061] Step S105: Determine the target text object located in the table based on the prediction result, and determine the table positioning result based on the target text object.

[0062] Based on the prediction results obtained in step S104, it is possible to determine which text objects are located within the table, and these text objects are then identified as target text objects. These target text objects are used to determine the table borders, thus achieving table positioning.

[0063] In this embodiment, one way to determine the table border is to fit the table boundary based on the position of the target text object. After fitting, all target text objects are within the fitted table boundary, and non-target text objects are outside the fitted table boundary. This fitted table boundary is the table border. When fitting the boundary, the fitting method can be to calculate the minimum bounding rectangle of the text positions of all target text objects within the table.

[0064] In this embodiment, another way to determine the table border is as follows: Determine the boundary target text objects based on the text positions of each target text object within the table; determine the table boundary information based on the text positions of the boundary target text objects; determine the table position based on the table boundary information, and use the table boundary and table position as the table positioning result. The text position includes the x-coordinate and y-coordinate of the text object. From all target text objects, determine the target text object with the smallest x-coordinate, the target text object with the largest x-coordinate, the target text object with the smallest y-coordinate, and the target text object with the largest y-coordinate as the boundary target text objects. Determine the left border line of the table based on the smallest x-coordinate, the right border line based on the largest x-coordinate, the top border line based on the smallest y-coordinate, and the bottom border line based on the largest y-coordinate, thus obtaining the table border.

[0065] In this embodiment, a multi-channel composite model is used to combine features of line segment position, text position, and text content information, which improves the accuracy of recognizing text objects located in tables within PDF documents. This allows for table location based on the text objects within the table, thus enhancing the accuracy of table location. Furthermore, the method provided in this embodiment is based on line segment position recognition. The table shape can be rectangular or any shape that line segments can form; therefore, this method is not limited by table format and can be applied to tables with diverse location formats, offering not only accurate location but also wide applicability.

[0066] The table localization method provided in this application includes: acquiring a file to be processed and parsing the file to obtain text information of text objects and line segment information of line segment objects in the file to be processed. The text information includes text position and text content, and the line segment information includes line segment position. The text objects are vectorized according to their text position and text content to obtain text vectors. The line segment objects are vectorized according to their text position and line segment position to obtain line segment vectors. The text vectors and line segment vectors are input into a pre-trained multi-channel composite model to obtain prediction results. The target text object located in the table is determined based on the prediction results, and the table localization result is determined and output based on the target text object. Based on the multi-channel composite model, the line segment position, text position, and text content information are combined to locate tables in a document. This effectively solves problems such as misidentification of non-table text as table text, incomplete table text recognition, and missed table recognition when identifying tables in a document. It improves the accuracy of recognizing text objects located in tables in a document, thereby improving the accuracy of table recognition and making table localization more accurate.

[0067] In some embodiments, the above Figure 1 In the illustrated embodiment, step S102, "generating a text vector for the text object based on the text position and text content," can be achieved through steps S1021 to S1024:

[0068] Step S1021: Vectorize the text content to obtain character vectors.

[0069] In this embodiment, a single character can be used as the basic unit for vectorization. Text content can be vectorized using word2vec technology to obtain character vectors. The dimension K of this character vector can be determined by a first preset value. To ensure that each character vector is unique, this first preset value can be set to any value greater than the number of characters on each target page. Generally, PDF documents have less than 1000 characters per page; therefore, the first preset value can be set to any value greater than 1000, for example, 1024.

[0070] Step S1022: Vectorize the text position to obtain the initial position vector.

[0071] The text information includes text position and text content. The extracted text information can be represented as: [x, y, char, width, height], where (x, y, width, height) represents the position information of the text object. PDF document page sizes are generally not very large, with less than 1000 pixels per page. Therefore, the values ​​of x, y, width, and height are set to any value between 0 and 999.9. Representing each number 999.9 in binary requires at least 14 binary digits. In this embodiment, the dimension of the initial position vector is determined by a second preset value, for example, 60. 60 bits are used to vectorize the text position, and each number in the text position is represented using 15 bits.

[0072] Step S1023: Based on the dimension of the word vector, perform vector alignment on the initial position vector to obtain the target position vector.

[0073] Here, the dimension of the target position vector is equal to the dimension of the word vector. Within the word vector range, the initial position vector is aligned bit-by-bit with the word vector, with each number occupying K / 4 bits. For example, if the word vector dimension K is 1024, each number, originally represented by 15 bits, is adjusted to use 1024 / 4 = 256 bits, so that the dimension of the aligned target position vector is equal to the dimension of the word vector.

[0074] Step S1024: The character vector and the target position vector are fused to obtain the text vector.

[0075] After all text content and text position are vectorized, the character vectors and target position vectors are fused to obtain the text vector of the text object. In this embodiment, the fusion of character vectors and target position vectors can be performed using vector addition to obtain the text vector.

[0076] In this embodiment of the application, when vectorizing text objects, not only the text position but also the text content is considered, so that the text vector comprehensively considers the semantic and positional relationships between text objects, thereby improving the accuracy when determining whether a text object is in a table based on the text vector.

[0077] In some embodiments, the above Figure 1 In the illustrated embodiment, step S103, "generating a line segment vector for the line segment object based on the text position and the line segment position," can be achieved through the following steps:

[0078] Step S1031: Construct an adjacency matrix of the graph based on the first preset value and the text position of each text object in the file to be processed.

[0079] The vertices of the graph include the text objects, and the edges of the graph include the distances between the text objects.

[0080] In some embodiments, constructing the adjacency matrix of the graph can be achieved by: determining the distance between each text object and other text objects based on the text position of each text object in the file to be processed; determining the dimension of the adjacency matrix based on a first preset value; and constructing the adjacency matrix of the graph based on the dimension of the adjacency matrix and the distance between each text object and other text objects.

[0081] Each page of the PDF document is represented as a vertices, and the distances between text objects are represented as edges. All text objects on each page are represented as an undirected graph. The number of vertices in this undirected graph is equal to a first preset value K, where the number of empty vertices is equal to the first preset value minus the number of text objects on the current page. Based on the first preset value K, each text object, and the distances between text objects, the adjacency matrix of the constructed graph has a dimension of (K, K).

[0082] The data in the adjacency matrix represents the distance between text objects. A larger value indicates a greater distance between the two text objects, and a lower probability that they coexist in the same table. Therefore, the distance between text objects can be used to assist in table positioning. In this embodiment, the distance between two text objects can be Euclidean distance, calculated based on the x-coordinate and y-coordinate of the two text objects.

[0083] Step S1032: Construct a feature matrix based on the first preset value and the distance features of each text object.

[0084] Here, the distance feature of a text object is characterized as the distance from the text object to each line segment object in the file to be processed.

[0085] In step S1031, the data in the adjacency matrix represents the distance between text objects. In step S1032, the data in the feature matrix represents the distance between text objects and line segment objects. The columns of the feature matrix represent the distances from the same text object to each line segment object. Let N be the number of line segment objects, and the dimension of the constructed feature matrix is ​​(K, N).

[0086] The larger the data value in the feature matrix, the farther the text object is from the line segment object, and the lower the probability that the text object exists in the table containing the line segment object. Therefore, the distance between the text object and the line segment object can be used to assist in table positioning. In this embodiment, the distance between the text object and the line segment object can be Euclidean distance, which can be calculated based on the x-coordinate and y-coordinate of the text object and the midpoint coordinate of the line segment object. In one implementation, "obtaining the distance feature of a text object" can be implemented as follows: determine the midpoint coordinates of each line segment object based on the line segment position of each line segment object; calculate the distance from a text object to each line segment object based on the text position of a text object and the midpoint coordinates of each line segment object, obtaining N values; these N values ​​are the distances from the text object to each line segment object, which is a column of data in the feature matrix.

[0087] Based on practical experience, the shorter the distance between a text object and a line segment object, the more likely it is to exist in a table. Therefore, when the number of line segments N is large, it indicates the presence of multiple tables on the page. In this case, when obtaining the distance features of a text object, after obtaining the distances from the text object to each line segment object, the feature matrix can be reduced in dimensionality to decrease computation. Dimensionality reduction of the feature matrix can be achieved by selecting the smallest M values ​​from the N distance values ​​as the distance features of a single text object, where M is a preset value. For example, if M is 10, the distances between the text object and the 10 nearest line segment objects can be used as the distance features of that text object, forming a column of the feature matrix. This reduces the dimensionality of the feature matrix without affecting the accuracy of table location, thereby reducing computation, accelerating table location, and improving efficiency.

[0088] Step S1033: Determine the line segment vector based on the adjacency matrix and the characteristic matrix.

[0089] In this embodiment of the application, the adjacency matrix representing the distance between text objects and the feature matrix representing the distance between text objects and line segment objects can be directly determined as line segment vectors.

[0090] In some embodiments, to further improve computational speed, the adjacency matrix and feature matrix can be normalized. Determining the line segment vector can then be achieved as follows: normalize the adjacency matrix and feature matrix respectively to obtain the processed adjacency matrix and processed feature matrix; determine the columns corresponding to the text objects in the processed adjacency matrix and the columns corresponding to the text objects in the processed adjacency matrix and the processed feature matrix respectively; and determine the corresponding columns of the text objects in the processed adjacency matrix and the columns corresponding to the text objects in the processed feature matrix as line segment vectors. Here, when constructing the adjacency matrix and feature matrix, the text objects can be sorted first, and the adjacency matrix and feature matrix can be constructed based on the sequence of the same text object. This ensures that the i-th column in the adjacency matrix and the i-th column in the feature matrix contain data from the same text object, and the data in the corresponding columns of the adjacency matrix and feature matrix are determined as line segment vectors.

[0091] In this embodiment of the application, when vectorizing line segment objects, not only the spatial position between text objects is considered, but also the spatial relationship between text objects and line segment objects. This makes the line segment vector integrate the positional relationships between text objects and between text objects and line segment objects, thereby improving the accuracy when determining whether a text object is in a table based on the line segment vector.

[0092] In some embodiments, the above Figure 1 In the illustrated embodiment, step S105, "determine the table location result based on the target text object," can be achieved through the following steps S1051 to S1053:

[0093] Step S1051: Determine the boundary target text objects based on the text positions of each target text object in the table.

[0094] The text position includes the x-coordinate and y-coordinate of the text object. From all target text objects, the target text object with the smallest x-coordinate, the target text object with the largest x-coordinate, the target text object with the smallest y-coordinate, and the target text object with the largest y-coordinate are determined as the boundary target text objects.

[0095] Step S1052: Determine the table boundary information based on the text position of the boundary target text object.

[0096] The left border of the table is determined by the minimum x-coordinate, the right border by the maximum x-coordinate, the top border by the minimum y-coordinate, and the bottom border by the maximum y-coordinate, thus obtaining the table borders. The table boundary information can be the equations of the left, right, top, and bottom border lines; the intersection of these four lines are the four vertices of the table.

[0097] In other embodiments, table boundary information can be obtained through fitting. In this case, steps S1051 and S1052 can be replaced by: fitting the table boundary based on the position of the target text object to obtain the table boundary information. After fitting, all target text objects are within the fitted table boundary, and non-target text objects are outside the fitted table boundary. This fitted table boundary is the table border. When performing boundary fitting, the fitting method can be to calculate the minimum bounding rectangle of the text positions of all target text objects within the table.

[0098] Step S1053: Determine the table position based on the table boundary information, and use the table boundary and table position as the table positioning result.

[0099] The table position includes the table's horizontal and vertical coordinates, width, and height, which can be represented as (xb, yb, widthb, heightb). Here, (xb, yb) are the reference coordinates, which can be the coordinates of the bottom left corner of the table or the coordinates of the center point of the table. widthb is the width of the table, and heightb is the height of the table. Based on the reference coordinates, width, and height, the position of the table can be determined, and the table positioning result can be obtained.

[0100] In some embodiments, after determining the table location result, the table location result can be further output. This can be output as a prompt message, or the table and its text can be rendered on the interface simultaneously for user viewing.

[0101] The method provided in this application, after obtaining the prediction results of each text object, determines the table boundary information based on the target text object located within the table determined by the prediction results, and further determines the table positioning result, thereby achieving table positioning.

[0102] In some embodiments, the training process of a multi-channel composite model can be achieved through the following steps:

[0103] Step S11: Obtain the training dataset.

[0104] Here, the training dataset includes multiple training text objects, multiple training line segment objects, and annotation information for each training text object. The annotation information is used to indicate whether the training text object exists in the table.

[0105] The training dataset can be pre-labeled data for the user.

[0106] Step S12: Vectorize the training text object according to its text position and text content to obtain the training text vector.

[0107] The step of vectorizing the training text object here is implemented in the same way as the step of vectorizing the text object in the file to be processed, as described in step S102 above.

[0108] Step S13: Vectorize the training line segment object according to the text position of the training text object and the line segment position of the training line segment object to obtain the training line segment vector.

[0109] The steps for vectorizing the training line segment objects are implemented in the same way as those for vectorizing the line segment objects in the file to be processed, as described in step S103 above.

[0110] Step S14: Construct a multi-channel composite model.

[0111] The multi-channel composite model includes a text feature extraction sub-model, a line segment feature extraction sub-model, and a labeling sub-model. The multi-channel composite model is a shallow model.

[0112] The table localization device pre-initializes a multi-channel composite model with multiple input channels. This model is used to determine whether a text object is text within a table. The trained multi-channel composite model is obtained by training the model using training data. This multi-channel composite model can be a TextCNN composite model. The most significant difference between this multi-channel TextCNN composite model and the traditional TextCNN model is that it uses multiple input channels, at least two channels. The vector of one input channel is updated, while the vector of the other input channel is not updated.

[0113] Initializing the multi-channel TextCNN composite model is divided into defining the feature extraction network and defining the sequence labeling model. The feature extraction network definition part includes defining the text feature extraction network and the line segment feature extraction network.

[0114] The text feature extraction network can be a dual-channel TextCNN text feature extraction network. The definition process can be implemented as follows: initialize two different embedding channels: one is a randomly initialized embedding defined as a dynamic channel, whose value will change as the network is trained; the other is a pre-trained embedding defined as a static channel, whose value will not be updated as the network is trained.

[0115] The line segment feature extraction network can be a type of line segment feature extraction network based on Graph Convolutional Network (GCN). The definition process can be implemented as follows: initialize a feature matrix, an adjacency matrix, and a GCN network respectively. When performing feature extraction, input the feature matrix and the adjacency matrix into the GCN network to extract line segment features.

[0116] Separate feature extraction is performed for text and line segments. Then, the text extraction network and the line segment extraction network are concatenated into vectors to fuse the extracted features from the two parts.

[0117] The sequence labeling model can be defined as follows: a fully connected layer is defined to fuse text features and line segment features, and the output vector has a first preset dimension (K); the sequence is labeled using a Conditional Random Field (CRF) model, two types of labels are defined, N represents non-table text and Y represents table text, and the vector dimension is set to M based on the consistency between the final labeling result and the input text sequence.

[0118] Step S15: Train the text feature extraction sub-model, the line segment feature extraction model, and the annotation sub-model based on the training text vector, the training line segment vector, and the annotation information to obtain the trained multi-channel composite model.

[0119] The method provided in this application embodiment abstracts the task of table localization into a sequence labeling model during the model modeling stage. Text located inside the table is labeled as Y (yes), and text located outside the table is labeled as N (no). A multi-channel composite model is trained to determine whether a text object is text in the table.

[0120] Based on the above embodiments, this application further provides a table positioning method. Figure 2 This is a schematic diagram illustrating another implementation flow of the table positioning method provided in the embodiments of this application, such as... Figure 2 As shown, this method can be implemented through the following steps:

[0121] Step S201: Obtain the original file for table positioning.

[0122] The original file contains at least one page and can be a file stored in PDF format.

[0123] Step S202: Select the target page from at least one page.

[0124] The target page includes text objects and line segment objects. After the table positioning device obtains the original file, it deletes the scanned pages from the original file, and the remaining non-scanned pages are the target pages. The scanned pages here are pages generated from scanned images. For the scanned pages, methods such as OCR can be used for table positioning.

[0125] Step S203: Generate a file to be processed based on the target page.

[0126] The unscanned pages are grouped into a file to be processed, which includes text objects and line segment objects.

[0127] Step S204: Determine the line segment position of the line segment object and the text position and text content of the text object in the file to be processed.

[0128] After obtaining the file to be processed, it is parsed to extract text objects and line segment objects. Simultaneously, the text information of the text objects and the line segment information of the line segment objects are obtained. The text information includes the text position and text content, while the line segment information includes the line segment position.

[0129] In one implementation, the open-source Java library PDFBox can be used to extract text objects and line segment objects, as well as the text information of the text objects and the line segment information of the line segment objects.

[0130] Step S205: Vectorize the text content to obtain character vectors.

[0131] In this embodiment, a single character can be used as the basic unit for vectorization. Text content is vectorized using word2vec technology to obtain character vectors. The dimension K of this character vector can be determined by a first preset value. To ensure that each character vector is unique, this first preset value can be set to any value greater than the number of characters on each target page. Generally, PDF documents have less than 1000 characters per page; therefore, the first preset value can be set to any value greater than 1000, for example, 1024.

[0132] Step S206: Vectorize the text position to obtain the initial position vector.

[0133] The dimension of the initial position vector is determined by a second preset value. In this embodiment, the extracted text information can be represented as: [x, y, char, width, height], where (x, y, width, height) represents the position information of the text object. The page size of a PDF document is generally not very large, and the number of pixels per page is generally less than 1000. Therefore, the size range of each number x, y, width, and height is set to any value between 0 and 999.9. Representing each number 999.9 in binary requires at least 14 binary digits. This second preset value can be set to 60, using 60 bits to vectorize the text position, with each number in the text position represented using 15 bits.

[0134] Step S207: Based on the dimension of the word vector, perform vector alignment on the initial position vector to obtain the target position vector.

[0135] The dimension of the target position vector is equal to the dimension of the word vector. Within the word vector range, the initial position vector is aligned bit-by-bit with the word vector, with each number occupying K / 4 bits. For example, if the word vector dimension K is 1024, each number originally represented by 15 bits is adjusted to use 1024 / 4 = 256 bits, so that the dimension of the aligned target position vector is equal to the dimension of the word vector.

[0136] Step S208: The character vector and the target position vector are fused to obtain the text vector.

[0137] After all text content and text position are vectorized, the character vectors and target position vectors are fused to obtain the text vector of the text object. In this embodiment, the fusion of character vectors and target position vectors can be performed using vector addition to obtain the text vector.

[0138] Step S209: Construct an adjacency matrix of the graph based on the first preset value and the text position of each text object in the file to be processed.

[0139] The vertices of the graph consist of individual text objects, and the edges consist of the distances between each text object. Constructing the adjacency matrix of the graph can be achieved as follows: Based on the text positions of each text object in the file to be processed, determine the distance between each text object and all other text objects; determine the dimension of the adjacency matrix based on a first preset value; construct the adjacency matrix of the graph based on the dimension of the adjacency matrix and the distances between each text object and all other text objects. The data in the adjacency matrix represents the distance between text objects. Larger data values ​​indicate greater distance between two text objects, and a lower probability that they coexist in the same table. Therefore, the distance between text objects can be used to assist in table positioning. Here, the distance between two text objects can be Euclidean distance, calculated based on the x-coordinate and y-coordinate of the two text objects.

[0140] Step S210: Obtain the distance features of each text object.

[0141] The distance feature of a text object refers to the distance from the text object to each line segment object. Obtaining the distance feature of a text object can be achieved as follows: Based on the position of each line segment object, determine the midpoint coordinates of each line segment object; based on the text position of the text object and the midpoint coordinates of each line segment object, calculate the distance from the text object to each line segment object, obtaining N values, where N is the number of line segment objects; select the M smallest values ​​from the N values ​​as the distance feature of a single text object, where M is a preset value.

[0142] Based on practical experience, the shorter the distance between a text object and a line segment object, the more likely it is to exist within a table. Therefore, when the number of line segments N is large, it indicates the presence of multiple tables on the page. In this case, when obtaining the distance features of a text object, after obtaining the distances from the text object to each line segment object, the feature matrix can be reduced in dimensionality to decrease computational load. Thus, without affecting the accuracy of table location, reducing the dimensionality of the feature matrix and decreasing computational load can accelerate table location and improve efficiency.

[0143] Step S211: Construct a feature matrix based on the first preset value and the distance features of each text object.

[0144] The distance feature of a text object is represented by the distance from the text object to each line segment object in the file to be processed. The data in the feature matrix represents the distance between the text object and the line segment objects, and the columns of the feature matrix represent the distances from the same text object to each line segment object. Let N be the number of line segment objects, and the dimension of the constructed feature matrix is ​​(K, N).

[0145] Step S212: Determine the line segment vector based on the adjacency matrix and the characteristic matrix.

[0146] In some embodiments, determining the line segment vector can be achieved by: normalizing the adjacency matrix and the feature matrix respectively to obtain the processed adjacency matrix and the processed feature matrix; determining the column corresponding to the text object in the processed adjacency matrix and the column corresponding to the text object in the processed feature matrix respectively; and determining the column corresponding to the text object in the processed adjacency matrix and the column corresponding to the processed feature matrix as the line segment vector. This can further improve the calculation speed, accelerate table positioning, and improve efficiency.

[0147] Step S213: Input the text vector and line segment vector into the pre-trained multi-channel composite model to obtain the prediction result.

[0148] In some embodiments, training a multi-channel composite model can be implemented as follows: Obtaining a training dataset, which includes multiple training text objects, multiple training line segment objects, and annotation information for each training text object, wherein the annotation information is used to indicate whether the training text object exists in a table; vectorizing the training text objects according to their text positions and text content to obtain training text vectors; vectorizing the training line segment objects according to their text positions and line segment positions to obtain training line segment vectors; constructing a multi-channel composite model, which includes a text feature extraction sub-model, a line segment feature extraction sub-model, and an annotation sub-model, wherein the multi-channel composite model is a shallow model; training the text feature extraction sub-model, the line segment feature extraction sub-model, and the annotation sub-model based on the training text vectors, training line segment vectors, and annotation information to obtain a trained multi-channel composite model.

[0149] In this embodiment, based on a multi-channel composite model, the features of line segment position, text position and text content information are combined to realize the function of recognizing tables in PDF documents. This can improve the recognition accuracy of text objects located in tables in PDF documents and effectively solve the problems of misidentifying non-table text lines as table data, incomplete table data recognition, and missed table recognition when recognizing tables in documents, thereby improving the accuracy of table recognition.

[0150] Step S214: Determine the target text object located in the table based on the prediction results.

[0151] By predicting each text object, we can determine which text objects are text in the table and identify those text objects as target text objects.

[0152] Step S215: Determine the boundary target text objects based on the text positions of each target text object in the table.

[0153] The text position includes the x-coordinate and y-coordinate of the text object. From all target text objects, the target text object with the smallest x-coordinate, the target text object with the largest x-coordinate, the target text object with the smallest y-coordinate, and the target text object with the largest y-coordinate are determined as the boundary target text objects.

[0154] Step S216: Determine the table boundary information based on the text position of the boundary target text object.

[0155] The left border of the table is determined by the minimum x-coordinate, the right border by the maximum x-coordinate, the top border by the minimum y-coordinate, and the bottom border by the maximum y-coordinate, thus obtaining the table borders. The table boundary information can be the equations of the left, right, top, and bottom border lines; the intersection of these four lines are the four vertices of the table.

[0156] Step S217: Determine the table position based on the table boundary information.

[0157] The table position includes the table's horizontal and vertical coordinates, width, and height, which can be represented as (xb, yb, widthb, heightb). Here, (xb, yb) are the reference coordinates, which can be the coordinates of the bottom left corner of the table or the coordinates of the center point of the table. widthb is the width of the table, and heightb is the height of the table. The position of the table can be determined based on the reference coordinates, width, and height.

[0158] Step S218: Determine the table boundaries and table position as the table positioning result.

[0159] Step S219: Output the table location results.

[0160] The table location information can be output as a prompt message, and the table and its text can be rendered on the interface simultaneously for easy viewing by the user. In this embodiment, the table format is not limited and can be applied to tables with various positioning formats, ensuring accurate positioning and wide applicability.

[0161] The table localization method provided in this application uses a multi-channel composite model to combine features of line segment position, text position and text content information, which can improve the accuracy of identifying text objects located in tables in PDF documents. Therefore, table localization can be performed based on text objects located in the table, thereby improving the accuracy of table localization.

[0162] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0163] Contracts, as a common carrier of business information, contain various key information and are usually stored in PDF document format. Conducting risk audits on contracts is one of the effective ways for enterprises to avoid various risks in advance. A large number of audit targets are found within tables, such as the contract subject matter (product name, unit price, tax rate, amount). Furthermore, table data in PDF documents lacks specific definitions; it's simply a combination of lines and text positions. This makes extracting table data from PDF documents very difficult. Common problems include: misidentifying non-table text lines as table data, incomplete table data identification, and missed table identification. These problems are all related to inaccurate table location. Therefore, accurately locating tables in PDF documents is a key technical challenge that needs to be overcome.

[0164] For locating tables in PDF documents, there are three relevant technical solutions:

[0165] The first approach involves converting the PDF document pages into images and then using image-based template similarity to determine the location. First, a template for a reference location is predefined. Then, grayscale template matching is performed on the collected table image at preset intervals from top to bottom and from left to right. The match with the highest similarity to the reference location identifier is used as the target location identifier, and the location of the target location identifier is obtained.

[0166] The second approach involves first converting the PDF document pages into images, then using an image regression model to locate the table. This approach is a general image object detection method that treats all elements in the image as pixels and the table as a target similar to cat or dog recognition. It uses deep learning for object detection (Regions with CNN features, RCNN) or the YOLO (You Only Look Once) object detection algorithm to locate the table and outputs a bounding box (x, y, w, h).

[0167] The third approach: a PDF table extraction method based on positional relationships. This method extracts text characters and their positions from PDF documents, uses the text's position coordinates for layout, and utilizes spatial information to draw the table positions.

[0168] Existing technologies have the following drawbacks: 1) Template matching is a pattern recognition method with inherent limitations. It can only perform parallel movements; if the target in the original image rotates or changes size, the algorithm becomes ineffective. 2) Image-based object detection models, such as R-CNN or YOLO, are deep convolutional models with 16 or more layers and numerous parameters. These large networks require a Graphics Processing Unit (GPU) to run, resulting in relatively slow recognition speeds. Furthermore, this approach only considers pixel-level relationships and cannot utilize the semantic information of the text itself. 3) Table drawing based on text spatial location information primarily relies on character layout features to locate tables. This approach has issues with irregularly defined tables and situations where table rows are close to non-table rows. Additionally, this approach lacks the semantic information of the text itself and does not utilize line segment spatial information, which is valuable for table localization.

[0169] This application proposes a method and system for locating PDF tables by combining text semantics and location information, aiming to solve the above-mentioned problems. Specifically, this application proposes a multi-channel TextCNN composite model, which effectively integrates line segments, text position, text content spatial information, and semantic information to achieve table location functionality based on PDF documents. This effectively solves problems such as misidentification of non-table text lines as table data, incomplete table data recognition, and missed table recognition. Furthermore, multi-channel TextCNN is a shallow semantic model, and its training and prediction have advantages over deep image models such as YOLO and RCNN, including faster speed, lower hardware consumption, and no reliance on GPUs.

[0170] This application primarily utilizes text semantics, text position, and line segment position for table localization. The relative positional relationship between line segments and text reflects the table's location. The relative positions of line segments and text within the table are vectorized, and the geometric characteristics of these line segments and text are used to assist in table localization. Table text positions exhibit certain layout characteristics; text within the table is often a key-value model, differing from text outside the table. This positional difference is used to assist in table localization. Table text and text outside the table have semantic differences; text within the table is often nouns such as concepts and items, along with corresponding value types, exhibiting aggregation characteristics. This semantic difference is used to assist in table localization. In the modeling stage, the table localization task is abstracted into a sequence labeling model, where all text within the table is labeled Y, and all text outside the table is labeled N. After the sequence labeling task, information on whether the target text is within the table is obtained. Post-processing is used for table border fitting, which can obtain multiple table regions in the PDF document, achieving the final goal.

[0171] This application proposes a PDF table localization system that combines text semantics and location information. The system includes two processes: training and prediction. Training primarily involves offline model training, and includes three modules: a PDF line segment text parsing module, a text and line segment vectorization module, and a multi-channel TextCNN composite model module. The PDF line segment text parsing module parses the PDF document into text content and line segment content. The text and line segment vectorization module represents the text content and line segment content as word vector structures, facilitating subsequent model processing. The multi-channel TextCNN composite model module uses the model to determine whether the text is table information. Prediction primarily involves table localization service, and includes four modules: a PDF line segment text parsing module, a text and line segment vectorization module, a TensorFlow-based prediction module, and a post-processing and output module. The TensorFlow Serving module uses the model file to predict whether the text in the PDF document is table information. The post-processing and output module uses the text information to construct table borders.

[0172] Figure 3 This is a flowchart illustrating a PDF table location method combining text semantics and location information proposed in an embodiment of this application. Figure 3 As shown, the table positioning method includes the following steps:

[0173] Step S301: Obtain the PDF document.

[0174] Step S302: Determine whether the PDF document is a scanned copy. If the PDF document is a scanned copy, proceed to step S303; if the PDF document is not a scanned copy, proceed to step S308.

[0175] Step S303: Parse the non-scanned PDF document to obtain the parsing result.

[0176] The parsing results include text information and line segment information from PDF documents. The PDF line segment text parsing module receives various contract documents stored in PDF format, excluding PDF scans. It uses pdfbox for code customization to extract text and line segment information. The text information is collected in the format [x, y, char, width, height], where x represents the horizontal coordinate, y represents the vertical coordinate, char represents the text content, width represents the text width, and height represents the text height. The line segment information is collected in the format [x0, y0, x1, y1], where x0 represents the starting point's horizontal coordinate, y0 represents the starting point's vertical coordinate, x1 represents the ending point's horizontal coordinate, and y1 represents the ending point's vertical coordinate.

[0177] Step S304: Vectorize the parsing results to obtain text vectors and line segment vectors.

[0178] Here, you can use the text and line vectorization module to perform vectorization operations, including text vectorization and line vectorization.

[0179] Among them, text vectorization is to receive text extracted in the format of [x, y, char, width, height], specifically including: (1) First, the text content is vectorized by word2vec, assuming the vectorization dimension is K; the number of characters in each page of PDF document is generally less than 1000, so K = 1024 is defined; (2) Second, (x, y, width, height) represents the position information of the text, and its size range is 0 to 999.9. It is vectorized by 15 bits, and 60 bits are set to represent all position information vectorization. Within the range of text word vectors, it is aligned with the word vector bit by bit. The alignment method is that each coordinate occupies K / 4 bits; (3) After all the text and position are vectorized, the text vector is obtained by fusion through vector addition operation. The text vector is represented as follows. Figure 4 As shown.

[0180] The line segment vectorization process acquires the line segment acquisition format as [x0, y0, x1, y1]; its specific implementation steps include: (1) First, determine the adjacency matrix A, define the number of characters in the space as the dimension of A, and define the edge value as the distance between two vertices; the number of characters is the same as that of text vectorization, taking a fixed value K; the dimension of A is (K, K); (2) Second, determine the feature matrix X, the representation of the feature matrix is ​​as follows Figure 5 As shown. The text is defined as a node, and the feature of each node is defined as the distance from the node to all line segments in the space; the 10 with the smallest absolute distance value are uniformly selected as features, where the dimension of X is (K, 10); (3) Finally, the values ​​in the two-dimensional matrix A and X are normalized, and the distance values ​​are normalized by z-value.

[0181] Step S305: Obtain the trained multi-channel TextCNN composite model.

[0182] The multi-channel TextCNN composite model module includes a feature extraction network and a sequence labeling model. Figure 6 The network architecture diagram of the multi-channel TextCNN composite model provided in the embodiments of this application is as follows: Figure 6 As shown, Figure 6As shown, the model definition process includes: (1) defining a dual-channel TextCNN text feature extraction network, initializing two different embedding channels, one of which is a randomly initialized embedding defined as a dynamic channel, whose value will change with network training; the other is a pre-trained embedding defined as a static channel, whose value will not be updated with network training; (2) defining a line segment feature extraction network based on GCN graph convolution, inputting the feature matrix X and the adjacency matrix A into the GCN network for feature extraction; (3) performing separate feature extraction for text and line segments, and then concatenating the text extraction network and the line segment extraction network into vectors to fuse the extracted features of the two parts; (4) defining a fully connected layer FC to fuse text features and line segment features, and finally outputting a vector with the number of text sequences (K) in dimension; (5) using the CRF model for sequence labeling, defining two types of labels, "N" for non-table text and "Y" for table text, and setting the vector dimension to K based on the consistency between the final labeling result and the input text sequence.

[0183] Figure 7 This is a schematic diagram illustrating the training process of the multi-channel TextCNN composite model provided in the embodiments of this application, as shown below. Figure 7 As shown, the training process includes the following steps:

[0184] Step S701: Obtain the training PDF document.

[0185] Step S702: Parse the training PDF document to obtain the parsing result.

[0186] Step S703: The parsing results are vectorized to obtain text vectors and line segment vectors.

[0187] Step S704: Initialize the multi-channel TextCNN composite model.

[0188] Step S705: Input the text vector and line segment vector into the initialized multi-channel TextCNN composite model to obtain the trained multi-channel TextCNN composite model.

[0189] Step S306: Input the text vector and line segment vector into the multi-channel TextCNN composite model to obtain the prediction result.

[0190] Here, the Tensorflow Serving module can receive the PDF parsing module as input, use the model file for model prediction, and output the corresponding text of the input. It can also determine whether the output text is table text. This part uses Tensorflow Serving to build a service for model publishing and online service.

[0191] Step S307: Process the prediction results to obtain the table positioning results.

[0192] Here, the prediction results output by the Tensorflow Serving module can be processed by the post-processing module, and table boundaries can be fitted to obtain the table location results. The fitting method can be to calculate the maximum bounding matrix of the text positions within all tables. Based on this method, the prediction effect of multiple tables on a single page can be achieved. After obtaining the table location results, proceed to step S309.

[0193] Step S308: Determine the table positioning result according to the OCR scheme.

[0194] The table location result determined by the OCR scheme can be either the first scheme or the second scheme mentioned above.

[0195] Step S309: Output the table location results.

[0196] Figure 8 This is a schematic diagram of the table positioning result output by the table positioning method provided in the embodiments of this application, as shown below. Figure 8 As shown, the final table rectangle 801 is output through the output module.

[0197] This application proposes a table localization scheme based on sequence labeling. A composite model combining multi-channel TextCNN and GCN is constructed based on text semantics, text position, and line segment position to extract table features. Sequence labeling is performed based on a CRF model. This scheme achieves table localization based on sequence labeling and post-processing, and can support multiple table detections. It utilizes the Euclidean space distance between text and line segments, and extracts Euclidean space characteristics based on a graph convolutional network (GCN) to characterize whether text is within a table.

[0198] This application addresses the problem of table localization in PDF documents by proposing a scheme combining a feature extraction network and a sequence labeling model. This transforms the object detection problem (table localization) into a task combining sequence labeling and post-processing, simplifying the complex problem and achieving multi-table localization with limited computational resources, thus improving recognition accuracy and performance. Furthermore, it proposes a multi-channel TextCNN feature extraction scheme that combines text semantics and text position, effectively utilizing the semantic features and spatial layout features of text within tables. It also proposes a GCN network feature extraction scheme built using the Euclidean distance between text and line segments, effectively using line segments to locate tables. This solves problems such as the absence of table borders, irregular table layout, close distance between tables and adjacent paragraphs, multiple table recognitions, and few table recognitions, ultimately ensuring general table localization and recognition performance and improving recognition accuracy. Moreover, using TextCNN and GCN for feature extraction and CRF for sequence labeling ensures performance while meeting production requirements in terms of shallow neural network processing speed and hardware requirements. Compared to RCNN, YOLO has significant performance and resource consumption advantages.

[0199] Based on the foregoing embodiments, this application provides a table positioning device. The modules and units included in the device can be implemented by a processor in a computer device; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0200] This application embodiment further provides a table positioning device. Figure 9 This is a schematic diagram of the composition structure of the table positioning device provided in the embodiments of this application, as shown below. Figure 9 As shown, the table positioning device 900 includes:

[0201] The first determining module 901 is used to determine the line segment position of the line segment object in the file to be processed, as well as the text position and text content of the text object.

[0202] The first generation module 902 is used to generate a text vector based on the text position and the text content;

[0203] The second generation module 903 is used to generate a line segment vector based on the text position and the line segment position;

[0204] Input module 904 is used to input the text vector and the line segment vector into a multi-channel composite model to obtain prediction results;

[0205] The second determining module 905 is used to determine the target text object located in the table based on the prediction result;

[0206] The third determining module 906 is used to determine the table positioning result based on the target text object.

[0207] In some embodiments, the table positioning device 900 may further include: a first acquisition module, configured to acquire an original file to be positioned, the original file containing at least one page; a filtering module, configured to filter out a target page from the at least one page, the target page including text objects and line segment objects; and a third generation module, configured to generate a file to be processed based on the target page.

[0208] In some embodiments, the first generation module 902 is further configured to: vectorize the text content to obtain a character vector, the dimension of which is determined by a first preset value; vectorize the text position to obtain an initial position vector, the dimension of which is determined by a second preset value; perform vector alignment on the initial position vector based on the dimension of the character vector to obtain a target position vector, the dimension of which is equal to the dimension of the character vector; and fuse the character vector and the target position vector to obtain a text vector.

[0209] In some embodiments, the second generation module 903 is further configured to: construct an adjacency matrix of a graph based on the first preset value and the text position of each text object in the file to be processed, wherein the vertices of the graph include each text object and the edges of the graph include the distances between each text object; construct a feature matrix based on the first preset value and the distance features of each text object, wherein the distance features of the text object represent the distances from the text object to each line segment object in the file to be processed; and determine line segment vectors based on the adjacency matrix and the feature matrix.

[0210] In some embodiments, the second generation module 903 is further configured to: determine the distance between each text object and other text objects based on the text position of each text object in the file to be processed; determine the dimension of the adjacency matrix based on the first preset value; and construct an adjacency matrix of a graph based on the dimension of the adjacency matrix and the distance between each text object and other text objects.

[0211] In some embodiments, the table positioning device 900 may further include: a second acquisition module, configured to acquire distance features of each text object; the second acquisition module is further configured to: determine the midpoint coordinates of each line segment object based on the line segment position of each line segment object; calculate the distance from the text object to each line segment object based on the text position of the text object and the midpoint coordinates of each line segment object, obtaining N values, where N is the number of line segment objects; and select the smallest M values ​​from the N values ​​as the distance features of the text object, where M is a preset value.

[0212] In some embodiments, the second generation module 903 is further configured to: normalize the adjacency matrix and the feature matrix respectively to obtain a processed adjacency matrix and a processed feature matrix; determine the column corresponding to the text object in the processed adjacency matrix and the column corresponding to the text object in the processed feature matrix respectively; and determine the column corresponding to the text object in the processed adjacency matrix and the column corresponding to the text object in the processed feature matrix as a line segment vector.

[0213] In some embodiments, the third determining module 906 is further configured to: determine boundary target text objects based on the text positions of each target text object located in the table; determine table boundary information based on the text positions of the boundary target text objects; determine the table position based on the table boundary information; and determine the table boundary and the table position as the table positioning result.

[0214] In some embodiments, the table positioning device 900 may further include: a training module;

[0215] The training module is used to acquire a training dataset, which includes multiple training text objects, multiple training line segment objects, and annotation information for each training text object. The annotation information is used to indicate whether the training text object exists in a table. The training text objects are vectorized based on their text positions and text content to obtain training text vectors. The training line segment objects are vectorized based on their text positions and line segment positions to obtain training line segment vectors. A multi-channel composite model is constructed, which includes a text feature extraction sub-model, a line segment feature extraction sub-model, and an annotation sub-model. The multi-channel composite model is a shallow model. The text feature extraction sub-model, the line segment feature extraction sub-model, and the annotation sub-model are trained based on the training text vectors, the training line segment vectors, and the annotation information to obtain a trained multi-channel composite model.

[0216] It should be noted that the descriptions of the table positioning device embodiments above are similar to the method descriptions above, and have the same beneficial effects as the method embodiments. For technical details not disclosed in the table positioning device embodiments of this application, those skilled in the art should refer to the descriptions of the method embodiments of this application for understanding.

[0217] It should be noted that, in the embodiments of this application, if the above-described table positioning method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0218] Accordingly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in the table positioning method provided in the above embodiments.

[0219] This application provides a table positioning device. Figure 10 This is a schematic diagram of the composition structure of the table positioning device provided in the embodiments of this application. Figure 10 The exemplary structure of the table positioning device 1000 shown can be used to deduce other exemplary structures of the table positioning device 1000. Therefore, the structure described herein should not be considered a limitation. For example, some components described below may be omitted, or components not described below may be added to suit the specific needs of certain applications.

[0220] Figure 10 The table positioning device 1000 shown includes: a processor 1001, at least one communication bus 1002, a user interface 1003, at least one external communication interface 1004, and a memory 1005. The communication bus 1002 is configured to enable communication between these components. The user interface 1003 may include a display screen 10031, and the external communication interface 1004 may include standard wired and wireless interfaces. The processor 1001 is configured to execute a program of a table positioning method stored in the memory to implement the steps of the table positioning method provided in the above embodiments.

[0221] The descriptions of the table positioning device and storage medium embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the table positioning device and storage medium embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0222] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0223] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0224] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0225] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0226] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0227] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0228] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an AC to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0229] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A table positioning method, wherein the file to be processed includes text objects and line segment objects, characterized in that, The method includes: Determine the position of line segments in line segment objects and the position and content of text objects in the file to be processed; Generate a text vector based on the text position and the text content; Based on a first preset value and the text position of each text object in the file to be processed, an adjacency matrix of a graph is constructed, wherein the vertices of the graph include each text object, and the edges of the graph include the distances between each text object; the first preset value is determined by the number of text objects. Based on the first preset value and the distance features of each text object, a feature matrix is ​​constructed, wherein the distance features of the text object are characterized as the distances from the text object to each line segment object in the file to be processed. The line segment vector is determined based on the adjacency matrix and the feature matrix; The text vector and the line segment vector are input into a multi-channel composite model to obtain the prediction result; The target text object located in the table is determined based on the prediction result, and the table positioning result is determined based on the target text object.

2. The method according to claim 1, characterized in that, The step of generating a text vector based on the text position and the text content includes: The text content is vectorized to obtain character vectors, and the dimension of the character vectors is determined by a first preset value; The text position is vectorized to obtain an initial position vector, the dimension of which is determined by a second preset value; Based on the dimension of the character vector, the initial position vector is vector aligned to obtain the target position vector, wherein the dimension of the target position vector is equal to the dimension of the character vector; The character vector and the target position vector are fused to obtain the text vector.

3. The method according to claim 1, characterized in that, The step of constructing an adjacency matrix of the graph based on the first preset value and the text positions of each text object in the file to be processed includes: Based on the text position of each text object in the file to be processed, determine the distance between each text object and all other text objects; The dimension of the adjacency matrix is ​​determined based on the first preset value; Based on the dimensions of the adjacency matrix and the distance between each text object and all other text objects, construct the adjacency matrix of the graph.

4. The method according to claim 1, characterized in that, The method further includes: obtaining the distance features of each text object; Among them, obtaining the distance features of a text object includes: Based on the position of each line segment object, determine the coordinates of the midpoint of each line segment object; Based on the text position of the text object and the midpoint coordinates of each line segment object, calculate the distance from the text object to each line segment object to obtain N values, where N is the number of line segment objects; The smallest M values ​​are selected from the N values ​​as the distance features of the text object, where M is a preset value.

5. The method according to claim 1, characterized in that, Determining the line segment vector based on the adjacency matrix and the feature matrix includes: The adjacency matrix and the feature matrix are normalized respectively to obtain the processed adjacency matrix and the processed feature matrix. The columns corresponding to the text object in the processed adjacency matrix and the columns corresponding to the processed feature matrix are determined respectively. The columns corresponding to the text object in the processed adjacency matrix and the columns corresponding to the processed feature matrix are determined as line segment vectors.

6. The method according to claim 2, characterized in that, The determination of table positioning results based on the target text object includes: Based on the text position of each target text object located in the table, the boundary target text objects are determined; The table boundary information is determined based on the text position of the boundary target text object; The table position is determined based on the table boundary information, and the table boundary and the table position are determined as the table positioning result.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the original file to be used for table positioning, the original file containing at least one page; Filter out a target page from the at least one page, the target page including text objects and line segment objects; A file to be processed is generated based on the target page.

8. The method according to claim 1, characterized in that, The method further includes: Obtain a training dataset, which includes multiple training text objects, multiple training line segment objects, and annotation information for each training text object. The annotation information is used to indicate whether the training text object exists in the table. The training text object is vectorized based on its text position and text content to obtain a training text vector. The training line segment object is vectorized based on the text position of the training text object and the line segment position of the training line segment object to obtain the training line segment vector. A multi-channel composite model is constructed, which includes a text feature extraction sub-model, a line segment feature extraction sub-model, and a labeling sub-model. The multi-channel composite model is a shallow model. The text feature extraction sub-model, the line segment feature extraction model, and the annotation sub-model are trained based on the training text vector, the training line segment vector, and the annotation information to obtain a trained multi-channel composite model.

9. A table positioning device, wherein the file to be processed includes text objects and line segment objects, characterized in that, The device includes: The first determining module is used to determine the line segment position of the line segment object in the file to be processed, as well as the text position and text content of the text object. The first generation module is used to generate a text vector based on the text position and the text content; The second generation module is used to construct an adjacency matrix of a graph based on a first preset value and the text positions of each text object in the file to be processed. The vertices of the graph include each text object, and the edges of the graph include the distances between each text object. The first preset value is determined by the number of text objects. Based on the first preset value and the distance features of each text object, a feature matrix is ​​constructed, where the distance features of the text objects represent the distances from each text object to each line segment object in the file to be processed. Line segment vectors are determined based on the adjacency matrix and the feature matrix. The input module is used to input the text vector and the line segment vector into the multi-channel composite model to obtain the prediction result; The second determining module is used to determine the target text object located in the table based on the prediction result; The third determining module is used to determine the table positioning result based on the target text object.

10. A table positioning device, characterized in that, The device includes: Processor; and Memory for storing computer programs that can run on the processor; When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The device stores computer-executable instructions configured to perform the steps of the method according to any one of claims 1 to 8.

Images