Table information extraction method, device, equipment and storage medium

Abstract

The present application relates to the field of artificial intelligence technology, and discloses a table information extraction method, device, equipment and storage medium. The method includes: identifying a table to be processed in a picture to be identified, and obtaining multiple text fragments; performing character-level classification processing on the text fragments to obtain multiple character labels; constructing a first adjacency matrix and a label feature matrix based on the character labels; using a pre-trained graph convolutional network to perform position feature extraction processing on the first adjacency matrix and the label feature matrix, and constructing a second adjacency matrix based on the position feature extraction processing results; determining the row position relationship of each text entity in the table to be processed based on the adjacency relationship of each node in the second adjacency matrix, and correcting the position of each text entity in the table to be processed based on the determined row position relationship. The embodiment of the present application can improve the accuracy of information extraction for image table information.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a table information extraction method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of image recognition technology, automated document information extraction is increasingly being used in real-world production and life. When extracting table information from a table in an image, the clarity of the table in the image directly affects the accuracy of the extracted information.

[0003] To improve the accuracy of image information extraction, existing technologies propose an image slope recovery algorithm based on the least squares method to correct images. However, this correction can only be performed when the entire image is rotated. It cannot correct images that are distorted or tilted upward or downward in the row direction of a table. Summary of the Invention

[0004] The purpose of this application is to provide a table information extraction method, device, equipment and storage medium, aiming to improve the accuracy of information extraction from image table information.

[0005] In a first aspect, a table information extraction method is provided, comprising:

[0006] Identify the table to be processed in the image to be identified and obtain multiple text fragments;

[0007] Performing character-level classification processing on the text segment to obtain a plurality of character labels, wherein the character labels include character contents of corresponding text entities in the text segment, and the text entities are character-level information in the text segment;

[0008] Constructing a first adjacency matrix and a label feature matrix based on the character labels, wherein the first adjacency matrix represents the row-column adjacency relationship of each character label, and the label feature matrix represents the feature information of each character label;

[0009] Performing position feature extraction processing on the first adjacency matrix and the label feature matrix using a pre-trained graph convolutional network, and constructing a second adjacency matrix based on the position feature extraction processing results;

[0010] The row position relationship of each text entity in the table to be processed is determined based on the adjacency relationship of each node in the second adjacency matrix, and the position of each text entity in the table to be processed is corrected based on the determined row position relationship.

[0011] In some embodiments, the step of identifying a table to be processed in a picture to be identified and obtaining multiple text segments includes:

[0012] Correcting the orientation of text in the image to be recognized;

[0013] Segmenting the text of the table to be processed to obtain a plurality of initial text blocks;

[0014] The initial text block is fed into a pre-trained OCR recognition model for character recognition to obtain the text segment.

[0015] In some embodiments, the character-level classification processing of the text segment to obtain multiple character labels includes:

[0016] Obtaining a character feature matrix of the text entity in each of the text segments;

[0017] Performing maximum pooling on the character feature matrix to obtain a character feature vector;

[0018] Performing labeling processing on the character feature vector to obtain a character level label;

[0019] The character-level labels are classified using a pre-trained classification model, and the classified character-level labels are converted to obtain character labels.

[0020] In some embodiments, constructing a first adjacency matrix and a label feature matrix based on the character labels includes:

[0021] According to the character categories corresponding to the character labels, a plurality of character labels with the same character category within a nearest neighbor distance are associated to obtain a first adjacency relationship graph;

[0022] Representing the first adjacency relationship graph in a matrix to obtain a first adjacency matrix;

[0023] Obtaining several label feature vectors of the character labels;

[0024] The label feature matrix is ​​constructed using each of the label feature vectors as a matrix element.

[0025] In some embodiments, performing position feature extraction processing on the first adjacency matrix and the label feature matrix using a pre-trained graph convolutional network, and constructing a second adjacency matrix based on the position feature extraction processing results, includes:

[0026] Calculating tag aggregation information, where the tag aggregation information represents a result of position feature aggregation operation of the character tag and neighbor tags of the character tag, where the neighbor tags of the character tag are other character tags adjacent to the current character tag in the first adjacency matrix;

[0027] Using the tag aggregation information to update the character tag to obtain a new character tag;

[0028] According to the feature categories of the new character labels, connecting a plurality of new character labels with the same feature categories within a nearest neighbor distance to obtain a second adjacency relationship graph;

[0029] The second adjacency relationship graph is represented by a matrix to obtain a second adjacency matrix.

[0030] In some embodiments, calculating tag aggregation information includes:

[0031] Adding the first adjacency matrix to the identity matrix to obtain a new first adjacency matrix;

[0032] Performing weighted summation on features of character labels and neighbor labels of the character labels in the new first adjacency matrix to obtain a summation result;

[0033] Normalizing the summation result to obtain the tag aggregation information.

[0034] In some embodiments, determining the row position relationship of each text entity in the table to be processed based on the adjacency relationship of each node in the second adjacency matrix, and correcting the position of each text entity in the table to be processed based on the determined row position relationship, includes:

[0035] According to the adjacency relationship of each node in the second adjacency matrix, each node in the second adjacency matrix is ​​arranged, and the nodes in the second adjacency matrix are converted into the corresponding text entities, so as to construct a recognized table.

[0036] In a second aspect, a table information extraction device is provided, the device comprising:

[0037] A recognition module is used to identify the table to be processed in the image to be recognized and obtain multiple text fragments;

[0038] a classification module, configured to perform character-level classification processing on the text segment to obtain a plurality of character labels, wherein the character labels include character contents of corresponding text entities in the text segment, and the text entities are character-level information in the text segment;

[0039] A feature construction module, configured to construct a first adjacency matrix and a label feature matrix based on the character labels, wherein the first adjacency matrix represents the row-column adjacency relationship of each character label, and the label feature matrix contains position feature information of each character label;

[0040] a feature extraction module, configured to perform position feature extraction processing on the first adjacency matrix and the label feature matrix using a pre-trained graph convolutional network, and construct a second adjacency matrix based on the position feature extraction processing results;

[0041] A determination module is used to determine the row position relationship of each text entity in the table to be processed based on the adjacency relationship of each node in the second adjacency matrix, and correct the position of each text entity in the table to be processed based on the determined row position relationship.

[0042] According to a third aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the table information extraction method according to the first aspect when executing the computer program.

[0043] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the table information extraction method described in the first aspect is implemented.

[0044] The beneficial effects of the present application are as follows: by further character-level recognition and classification of the table text content in the image to be identified, character labels at multiple character levels are obtained, thereby improving the recognition accuracy of the table text content, and using character labels to construct a first adjacency matrix and a label feature matrix, the first adjacency matrix represents the row and column adjacency relationship of each character label, and the label feature matrix represents the feature information of each character label, and then the first adjacency matrix and the label feature matrix are processed by a pre-trained graph convolutional network to perform position feature extraction processing, and the row position relationship between each character label is identified, and this is used as the position feature extraction processing result, and the row position relationship between each character label represented by the position feature extraction processing result is converted into the adjacency relationship of each node in the second adjacency matrix, thereby determining the row position relationship of each text entity in the table to be processed, and obtaining the recognition result of the table text content. When the image is distorted or tilted upward or downward in the row direction of the table, the row position relationship of the table text content in the image can also be identified, thereby improving the accuracy of information extraction from the image table information. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of a table information extraction method provided in an embodiment of the present application;

[0046] Figure 2 yes Figure 1 Flowchart of step S101 in FIG.

[0047] Figure 3 yes Figure 1 Flowchart of step S102 in FIG.

[0048] Figure 4 yes Figure 3 Flowchart of step S103 in FIG.

[0049] Figure 5 yes Figure 3 Flowchart of step S104 in FIG.

[0050] Figure 6 It is a structural diagram of the table information extraction device provided in an embodiment of the present application;

[0051] Figure 7 Schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application;

[0052] Figure 8 Schematic diagram of the recognition result of a text segment provided in an embodiment of the present application;

[0053] Figure 9 1 is a schematic structural diagram of a layout lm model provided in one embodiment of the present application;

[0054] Figure 10 This is a schematic structural diagram of a first adjacency relationship graph provided in one embodiment of the present application;

[0055] Figure 11 is a schematic structural diagram of a first adjacency matrix provided in one embodiment of the present application;

[0056] Figure 12 is a schematic structural diagram of a second adjacency relationship graph provided in one embodiment of the present application;

[0057] Figure 13 Schematic diagram of the structure of the second adjacency matrix provided in one embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0059] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0061] First, let’s analyze some of the terms used in this application:

[0062] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and create new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0063] Natural Language Processing (NLP): NLP uses computers to process, understand, and apply human languages ​​(such as Chinese and English). It is a branch of artificial intelligence and an interdisciplinary field between computer science and linguistics, often referred to as computational linguistics. Natural language processing encompasses grammatical analysis, semantic analysis, and discourse comprehension. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information intent identification, information extraction and filtering, text classification and clustering, public opinion analysis, and opinion mining. It encompasses data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research related to language processing, and linguistics research related to language computing.

[0064] Information Extraction (I nformat i on Extraction) is a text processing technology that extracts specified types of entity, relationship, event and other factual information from natural language text and forms structured data output. Information extraction is a technology that extracts specific information from text data. Text data is composed of some specific units, such as sentences, paragraphs, and chapters. Text information is composed of some small specific units, such as characters, words, phrases, sentences, paragraphs, or a combination of these specific units. Extracting noun phrases, names, place names, etc. from text data is all text information extraction. Of course, the information extracted by text information extraction technology can be of various types.

[0065] In the related technologies, the main application technologies of table information extraction solutions are divided into image-based technology and natural language processing-based technology. In the table information extraction solution based on image technology, the table cells are determined by identifying visible and invisible table lines, and then the relationship between cells is analyzed through the row and column correspondence. However, in some specific areas, the layout of the table is not regular enough, and the cells are closely pasted, resulting in low parsing accuracy. At this time, using NLP technology to parse the table from the perspective of text semantics can achieve better information extraction results. However, when using NLP technology to solve the table information extraction solution, the image needs to be corrected in advance, but when the image is distorted or tilted upward or downward in the direction of the table row, it cannot be corrected, and the accuracy of image information extraction is reduced.

[0066] Based on this, the embodiments of the present application provide a table information extraction method, apparatus, device and storage medium, aiming to improve the accuracy of information extraction from image table information.

[0067] The recommended method and device, electronic device, and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the recommended method in the embodiments of the present application is described.

[0068] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0069] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0070] The table information extraction method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The table information extraction method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the table information extraction method, etc., but is not limited to the above forms.

[0071] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0072] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0073] Figure 1 This is an optional flowchart of the table information extraction method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0074] Step S101, identifying a table to be processed in a picture to be identified, and obtaining a plurality of text segments;

[0075] Step S102: performing character-level classification processing on the text segment to obtain a plurality of character labels, wherein the character labels include character contents of corresponding text entities in the text segment, and the text entities are character-level information in the text segment;

[0076] Step S103: constructing a first adjacency matrix and a label feature matrix based on the character labels, wherein the first adjacency matrix represents the row-column adjacency relationship of each character label, and the label feature matrix represents the feature information of each character label;

[0077] Step S104: using a pre-trained graph convolutional network to perform position feature extraction processing on the first adjacency matrix and the label feature matrix, and constructing a second adjacency matrix based on the position feature extraction processing results;

[0078] Step S105 , based on the adjacency relationship of each node in the second adjacency matrix, determine the row position relationship of each text entity in the table to be processed, and correct the position of each text entity in the table to be processed based on the determined row position relationship.

[0079] In step S101 of some embodiments, identifying the table to be processed in the image to be identified may be performed by using optical character recognition (OCR) to identify the table to be processed in the image.

[0080] A text segment (bounding box) contains at least one text entity. This includes the coordinates of the text entity in the table, as well as information such as the character text. Generally speaking, a text segment only contains one text entity. However, some closely spaced text entities may be covered by the same text segment. For example, text entities of numerical values ​​and text entities of units are difficult to distinguish using text segment-level recognition methods, resulting in both text entities of numerical values ​​and units being covered by the same text segment.

[0081] In step S102 of some embodiments, the text fragment is subjected to character-level classification processing to obtain a plurality of character labels, which may be performed by using a pre-trained classification model. Specifically, each text fragment is input into a pre-trained classification model, the classification model performs a sequence labeling task, fine-tunes the text entities in the labeled text fragment, and performs multi-label classification on the text entities in the text fragment to obtain character labels corresponding to the target text entity. The target text block corresponds to one of the text entities in the text fragment, and the character label represents the category and character content of the text entity.

[0082] It can be understood that by performing character-level classification processing on multiple text entities in a text segment, character labels corresponding to each text entity having content records are obtained.

[0083] In step S103 of some embodiments, a first adjacency matrix is ​​constructed based on character labels. This may be when performing character-level classification processing on a text segment and obtaining character labels, arranging each character label according to the coordinate information of the text entity corresponding to the character label in a table, so that the obtained coordinate information of each character label matches the coordinate information of the text entity of the target text block in the table, presetting the adjacency range of the character label in the row and column directions, using each character label as a node of the matrix, connecting other character labels within the adjacency range of the current character label to form a first adjacency matrix, so that the obtained first adjacency matrix represents the row and column adjacency relationship of each character label.

[0084] In step S103 of some embodiments, a label feature matrix is ​​constructed based on character tags. This may be done by arranging the character tags according to the coordinate information of the corresponding text entities in the table, generating a number of position features according to the coordinate information of each character tag, and generating a text type feature according to the character text of the text entity corresponding to the character tag. The above-mentioned features are combined in the form of a matrix to form a label feature matrix, so that the label feature matrix represents the feature information of each character tag.

[0085] In step S104 of some embodiments, a pre-trained graph convolutional network is used to perform position feature extraction processing on the first adjacency matrix and the label feature matrix, and a second adjacency matrix is ​​constructed based on the position feature extraction processing results, which may be to update the node itself by aggregating the features of the surrounding nodes. Specifically, for each node in the first adjacency matrix, the neighbor nodes of the current node are determined according to the adjacency relationship between each node, the features of the current node and the features of its neighbor nodes are aggregated, and the result of the aggregation operation is used as the feature of the current node, and the feature of the current node is updated. This is repeated and the features of each node are iteratively updated. When the number of iterations reaches a set threshold or the feature extraction meets the preset expectations, the iteration is stopped and the feature extraction results are output. The output feature extraction results contain the latest node adjacency relationships, and then a second adjacency matrix is ​​constructed based on the position feature extraction processing results to redetermine the adjacency relationships between each node. It can be understood that the nodes in the first adjacency matrix and the second adjacency matrix refer to character labels.

[0086] In step S105 of some embodiments, the position feature extraction processing results eliminate redundant node adjacency relationships. The node adjacency relationships of the second adjacency matrix may include the row position relationships and column position relationships of each node, that is, the row position relationships and column position relationships of the text entity in the table to be processed. The node adjacency relationships of the second adjacency matrix may also include only the row position relationships of each node. The column position relationships of each node are determined by the initial column position coordinate information of each node. The obtained row position relationships of the text entity in the table to be processed are not affected by the distortion of the image or the tilting upward or downward in the row direction of the table.

[0087] In steps S101 to S105 shown in the embodiment of the present application, further character-level recognition and classification are performed on the table text content in the image to be identified, thereby obtaining character labels at multiple character levels, improving the recognition accuracy of the table text content, and using the character labels to construct a first adjacency matrix and a label feature matrix. The first adjacency matrix represents the row and column adjacency relationship of each character label, and the label feature matrix represents the feature information of each character label. The first adjacency matrix and the label feature matrix are then subjected to position feature extraction processing by a pre-trained graph convolutional network to identify the row position relationship between each character label, and this is used as the position feature extraction processing result. The row position relationship between each character label represented by the position feature extraction processing result is converted into the adjacency relationship of each node in the second adjacency matrix, thereby determining the row position relationship of each text entity in the table to be processed, and obtaining the recognition result of the table text content. When the image is distorted or tilted upward or downward in the row direction of the table, the row position relationship of the table text content in the image can also be identified, thereby improving the accuracy of information extraction from the image table information.

[0088] See also Figure 2 In some embodiments, step S101 may include but is not limited to steps S201 to S203.

[0089] Step S201, correcting the orientation of the text in the image to be recognized;

[0090] Step S202, segmenting the text of the table to be processed to obtain a plurality of initial text blocks;

[0091] Step S203: sending the initial text block to a pre-trained OCR recognition model for character recognition to obtain the text segment.

[0092] In step S201 of some embodiments, the orientation of the text in the image to be identified is corrected. Specifically, after obtaining the image to be identified, the orientation of the text in the image to be identified is first identified and the text orientation angle is corrected. Since the image to be identified is often not perfect and may have a tilt angle, the image to be identified is corrected. The tilt correction algorithm based on the least squares method is used to perform tilt correction on the image to be identified.

[0093] In some embodiments, a tilt correction algorithm based on the least squares method is used to perform tilt correction on the image to be identified. The image to be identified can be scanned row by row from left to right, and the scanned text is roughly sorted. The tilt correction algorithm based on the least squares method fits the straight lines of each row in the rough sorting result, calculates the slope of each row and the corresponding normal slope, and obtains the row straight line slope and the normal slope. For a single element in each row, a straight line with a slope of the row straight line slope and passing through the current element and the intersection of the straight line and the vertical axis is determined. When the intersection falls within the preset range of the row, it is determined to be an element belonging to the row. For a single element in each column, a straight line with a slope of the normal line slope and passing through the current element and the intersection of the straight line and the horizontal axis is determined. When the intersection falls within the preset range of the column, it is determined to be an element belonging to the column.

[0094] In step S202 of some embodiments, the text of the table to be processed is segmented to obtain multiple initial text blocks. The layout of the table to be processed can be analyzed, each row of the table to be processed is segmented, the text of each row is cut out, and then each row of text is segmented into columns to cut out each character, thereby obtaining multiple initial text blocks.

[0095] In step S203 of some embodiments, it is understood that the text segment includes the recognized text content and the coordinate information of the text content. Figure 8As shown, for each text segment, the first 8 elements are the coordinate information of the initial text block, the 9th element is the recognition confidence of the initial text block, and the 10th element is the recognized text of the initial text block.

[0096] In steps S201 to S203 shown in the embodiment of the present application, the image to be recognized is pre-processed by the OCR recognition model, the text content in the table to be processed is recognized, the text is segmented and integrated, and the coordinate information of each text content is initially determined.

[0097] See also Figure 3 In some embodiments, step S102 may include but is not limited to steps S301 to S304.

[0098] Step S301, obtaining a character feature matrix of the text entity in each of the text segments;

[0099] Step S302, performing maximum pooling on the character feature matrix to obtain a character feature vector;

[0100] Step S303: labeling the character feature vector to obtain a character-level label;

[0101] Step S304: Use a pre-trained classification model to classify the character-level labels, and convert the classified character-level labels to obtain character labels.

[0102] In step S301 of some embodiments, the character feature matrix of the text entity in each text segment is obtained, specifically by Figure 9 The LayoutLm model shown in the figure obtains the character feature matrix of the text entities in each text segment. Using the pre-trained LayoutLm model, the five-layer Embedding input layer and the LayoutLm Embedding layer in the LayoutLm model are used as feature extractors for semantic paragraphs, outputting a 512-dimensional character feature vector. The one-dimensional character feature matrix of the text entities belonging to the same column is extracted. For example, the one-dimensional character feature matrix of two text entities belonging to the same column and adjacent to each other is: and

[0103] like Figure 4 As shown in Figure 1, the LayoutLM model structure adds two new embedding layers: 2-DPos it ion Embedding and Image Embedding to the existing pre-trained model:

[0104] 2-D Position Embedding: Based on the text fragments obtained by OCR technology, we can obtain the specific location of the text in the document. After converting the corresponding coordinates into virtual coordinates, we calculate the corresponding representation of the coordinates in the four embedding sub-layers of x, y, w, and h. The final 2-D Position Embedding is the sum of the embeddings of the four sub-layers.

[0105] Image Embedding,In the Image Embedding section, we treat the corresponding text block of each document as a proposal in Faster R-CNN to extract the corresponding local features.,In particular, since the [CLS] symbol is used to represent the semantics of the entire input text, we also use the entire document image as the Image Embedding at this position to maintain modality alignment.

[0106] In step S302 of some embodiments, the character feature matrix is ​​subjected to maximum pooling to obtain a character feature vector, specifically, the one-dimensional character feature matrices of the two text entities are pooled. and Perform max-pooling to obtain the character feature vectors corresponding to the two text entities: and

[0107] In step S303 of some embodiments, the character feature vectors are labeled to obtain character-level labels, specifically, word-level labels for identifying text entities, and the word-level labels of the text entities are converted into character-level labels using a preset conversion rule. For example, the word-level label of the value "2.27" is identified as "B_val, I_val, I_val, I_val", and converted into a character-level label "5.78" using a preset conversion rule, and the category is "val".

[0108] In step S304 of some embodiments, the classification model is a classification model based on the Xgboost model. The character-level labels are classified using a pre-trained classification model, and the classified character-level labels are converted to obtain character labels. Specifically, the character-level labels are input into the Xgboost model, and a binary classification task is performed based on the hit frequency of the title keywords through the Xgboost model to obtain character labels. Exemplarily, the rule scheme for the binary classification task can set two labels, and the two labels correspond to two regular expressions respectively. According to the keyword hit frequency of the title of the table to be processed, the keywords that hit the current title are assigned the labels corresponding to the current title, and the keywords that do not hit the current title are recorded as other labels.

[0109] It is understandable that other machine learning models can also be selected in the above embodiments, and the embodiments of this application are not limited thereto.

[0110] In steps S301 to S304 shown in the embodiment of the present application, the character feature matrix of the text entity is obtained through the LayoutLm model and converted into a character feature vector, the character feature vector is labeled, and then the character-level labels obtained by the labeling are classified through the Xgboost model to improve the recognition of text entities of different categories in the table to be processed.

[0111] See also Figure 4 In some embodiments, step S103 may include but is not limited to steps S401 to S404.

[0112] Step S401, according to the character categories corresponding to the character tags, a plurality of character tags with the same character category within a nearest neighbor distance are associated to obtain a first adjacency relationship graph;

[0113] Step S402: performing matrix representation on the first adjacency relationship graph to obtain a first adjacency matrix;

[0114] Step S403, obtaining several label feature vectors of the character label;

[0115] Step S404: construct the label feature matrix using each of the label feature vectors as a matrix element.

[0116] In step S401 of some embodiments, according to the character category corresponding to the character label, several character labels with the same character category within the nearest distance are associated, and the nearest distance of the character label and several character labels with the same character category within the nearest distance can be determined based on the k-nearest neighbor (KNN) algorithm. Specifically, the input of the k-nearest neighbor algorithm is the character feature vector of the character label, corresponding to the point in the feature space, and the output is the category of the character label, which can be multiple categories. The k-neighbor algorithm assumes that a training data set is given, in which the category of the character label is pre-set. According to the category of the training instance of its k nearest neighbors, prediction is made by majority voting and other methods to determine the adjacency relationship between each character label, and the character labels with adjacency relationship are connected to obtain a first adjacency relationship graph. For example, during classification, for the current character label to be classified, find the k other character labels closest to the current character label to be classified in the training sample set and save them to the set N. Count the number of each type of character labels in the set N, and connect the current character label to be classified with other character labels of the same category in the set N in pairs, and get the following: Figure 10The first adjacency graph shown.

[0117] In step S402 of some embodiments, the first adjacency relationship graph is represented by a matrix to obtain a first adjacency matrix, specifically, the adjacency relationship between each character label in the first adjacency relationship graph is represented by a matrix. For example, Figure 11 As shown, the first adjacency matrix obtained by matrix representation of the first adjacency relationship graph, "1" indicates that two character labels have an adjacency relationship (adjacent), and "0" indicates that two character labels do not have an adjacency relationship (non-adjacent). For example, character label p1 is connected to character label p1, character label p2, character label p5 and character label p6 respectively, then the matrix elements corresponding to character label p1 in the first row and character label p1, character label p2, character label p5 and character label p6 in the first column are 1, and the matrix elements corresponding to character label p1 in the first row and other character labels in the first column are 0.

[0118] In step S403 of some embodiments, obtaining several label feature vectors of the character label may be to obtain relative position features between the character labels and the text type of the text entity corresponding to the character label. Exemplarily, based on the character label generated by the layout lm model, multiple features such as position embedding, text embedding and label are generated, wherein position embedding represents the position information of the character label, including the coordinate information of the character label located at the upper left corner of the standardized character label and the coordinate information of the character label located at the lower right corner of the standardized character label, text embedding represents the text feature information extracted by the recurrent neural network (RNN), and label is the classification annotation information about the character label output by the layout lm model or the xgboost model.

[0119] In step S404 of some embodiments, the label feature matrix is ​​constructed with each of the label feature vectors as a matrix element, and the relative position features and text type of the text entity obtained are combined in the form of a row matrix to form a label feature matrix, so that the label feature matrix represents the feature information of each character label. For example, based on the character labels generated by the layout lm model, three features, namely, position embedding, text embedding, and label, are generated, and these three features are combined in the form of a row matrix to obtain a label feature matrix.

[0120] h=[pos it i on embedded i ng, text embedded i ng, l abe l].

[0121] Steps S401 to S404 shown in the embodiment of the present application construct a first adjacency matrix based on the category of the character labels and a label feature matrix based on the position features of the character labels, which can preliminarily determine the neighbor relationship and relative position between each character label, providing a basis for the accuracy of the feature extraction step.

[0122] See also Figure 5 In some embodiments, step S104 may include but is not limited to steps S501 to S504.

[0123] Step S501, calculating tag aggregation information, wherein the tag aggregation information represents a position feature aggregation operation result of the character tag and its neighbor tags, where the neighbor tags are other character tags adjacent to the current character tag in the first adjacency matrix;

[0124] Step S502: using the tag aggregation information to update the character tag to obtain a new character tag;

[0125] Step S503, connecting a plurality of new character tags with the same feature category within a nearest neighbor distance based on the feature category of the new character tags to obtain a second adjacency relationship graph;

[0126] Step S504: Perform matrix representation on the second adjacency relationship graph to obtain a second adjacency matrix.

[0127] In step S501 of some embodiments, calculating tag aggregation information includes:

[0128] Adding the first adjacency matrix to the identity matrix to obtain a new first adjacency matrix;

[0129] Performing weighted summation on features of character labels and neighbor labels of the character labels in the new first adjacency matrix to obtain a summation result;

[0130] Normalizing the summation result to obtain the tag aggregation information.

[0131] Specifically, the calculation function of tag aggregation information is:

[0132]

[0133]

[0134] Among them, H (l)Represents the label feature matrix of the lth layer, which can include multiple label feature vectors such as position embedding, text embedding and label, σ represents a nonlinear activation function, such as RELU activation function, A represents the first adjacency matrix, I n represents the identity matrix, represents the new first adjacency matrix, W (l) Represents the trainable parameter matrix of the convolution transformation of the current layer.

[0135] right The solution can be understood as performing a weighted sum operation on the features of the current character label and its neighbor labels. For example, character label X has neighbor labels Y and Z, and the features of X, Y, and Z are X = (0.5, 0.6), Y = (0, 3, 0.7), and Z = (0.6, 0.8), respectively. When aggregating character label X, the values ​​of X = (0.5, 0.6), Y = (0, 3, 0.7), and Z = (0.6, 0.8) are weighted summed to obtain the summation result.

[0136] The calculation function of the above label aggregation information can also take the character label information into account, but this formula still has problems: the new first adjacency matrix No normalization,

[0137] The vectors of all neighboring labels of the character label will be added together, so that the eigenvector values ​​of some character labels will be very large after multiple layers of convolution. It is not normalized, which may cause problems when extracting graph features. For example, character labels with more neighbor labels tend to have larger eigenvalues. It can be the new first adjacency matrix Multiply both sides by It can be understood that in the process of updating the feature values ​​of the character labels, the summation results are normalized to obtain label aggregation information.

[0138] The calculation function of label aggregation information after normalization is:

[0139]

[0140] Among them, H l+1 Indicates tag aggregation information, H (l) Represents the label feature matrix of the lth layer, which can include multiple label feature vectors such as position embedding, text embedding and label, σ represents a nonlinear activation function, such as RELU activation function, represents the new first adjacency matrix, represents the degree matrix of the first adjacency matrix, W (l) Represents the trainable parameter matrix of the convolution transformation of the current layer.

[0141] In step S502 of some embodiments, the label aggregation information is used to update the character label to obtain a new character label, wherein the label aggregation information aggregates the features of the current character label and the features of its neighbor labels, and the label aggregation information replaces the original features of the current character label, thereby updating the features of the current node. This is repeated to iteratively update the features of each character label. When the number of iterations reaches a set threshold or the feature extraction meets the preset expectations, the iteration is stopped and a new character label is output, and the adjacency relationship between each new character label is updated.

[0142] In step S503 of some embodiments, according to the feature category of the new character label, several new character labels with the same feature category within the nearest neighbor distance are connected to obtain a second adjacency relationship graph, which can be based on the k-nearest neighbor (KNN, k-NearestNe i ghbor) algorithm to determine the nearest neighbor distance of the character label and several character labels with the same character category within the nearest neighbor distance. Specifically, the input of the k-nearest neighbor algorithm is the character feature vector of the new character label, corresponding to the point in the feature space, and the output is the category of the new character label. The category of the new character label is the same as the category of the character label in the first adjacency relationship graph. According to the category of the training instance of its k nearest neighbors, prediction is made by majority voting and other methods to determine the adjacency relationship between each new character label, and the new character labels with adjacency relationship are connected to obtain a second adjacency relationship graph. For example, during classification, for the new character label to be classified, find the k other new character labels closest to the new character label to be classified in the training sample set and save them to the set N. Count the number of new character labels of each category in the set N, and connect the new character label to be classified with other new character labels of the same category in the set N in pairs, and get the following: Figure 12 The second adjacency graph is shown.

[0143] In step S504 of some embodiments, the second adjacency relationship graph is represented by a matrix to obtain a second adjacency matrix, specifically, the adjacency relationship between each character label in the second adjacency relationship graph is represented by a matrix. For example, Figure 13As shown, the second adjacency matrix obtained by matrix representation of the second adjacency relationship graph, "1" indicates that two character labels have an adjacency relationship (adjacent), and "0" indicates that two character labels do not have an adjacency relationship (non-adjacent). For example, after the position feature extraction process updates the adjacency relationship between each character label, character label p1 is connected to character label p1 and character label p2 respectively, then the matrix elements corresponding to character label p1 in the first row and character label p1 and character label p2 in the first column are 1, and the matrix elements corresponding to character label p1 in the first row and other character labels in the first column are 0.

[0144] Steps S501 to S504 shown in the embodiment of the present application update the features of the current character label by iteratively aggregating the features of the current character label and its neighbor labels, and update the row position relationship between each character label based on the category information of the character label and the feature information of the character label in the first adjacency matrix. When the image is distorted or tilted upward or downward in the row direction of the table, the row position relationship of the table text content in the image can also be identified.

[0145] In some embodiments, step S105 includes:

[0146] According to the adjacency relationship of each node in the second adjacency matrix, each node in the second adjacency matrix is ​​arranged, and the nodes in the second adjacency matrix are converted into the corresponding text entities, so as to construct a recognized table.

[0147] It can be understood that the nodes in the second adjacency matrix refer to character labels whose adjacency relationships have been updated after position feature extraction. Constructing the recognized table can be to identify the text entities corresponding to the character labels, convert the character labels into corresponding text entities, arrange the text entities according to the adjacency relationships of the corresponding character labels, arrange each text entity according to the row position relationship in the second adjacency relationship graph, form table borders at corresponding positions, and make the arranged text entities each located in an independent table frame, thereby constructing the recognized table.

[0148] See also Figure 6 The present application also provides a table information extraction device that can implement the above table information extraction method. The device includes:

[0149] Recognition module 610, used to identify the table to be processed in the image to be recognized and obtain multiple text segments;

[0150] A classification module 620 is configured to perform character-level classification on the text segment to obtain a plurality of character labels, each of which contains character contents of a corresponding text entity in the text segment, where the text entity is character-level information in the text segment;

[0151] A feature construction module 630 is configured to construct a first adjacency matrix and a label feature matrix based on the character labels, wherein the first adjacency matrix represents the row-column adjacency relationship of each character label, and the label feature matrix contains position feature information of each character label;

[0152] A feature extraction module 640 is configured to perform position feature extraction processing on the first adjacency matrix and the label feature matrix using a pre-trained graph convolutional network, and construct a second adjacency matrix based on the position feature extraction processing results;

[0153] The determination module 650 is used to determine the row position relationship of each text entity in the table to be processed based on the adjacency relationship of each node in the second adjacency matrix, and correct the position of each text entity in the table to be processed based on the determined row position relationship.

[0154] The specific implementation of the table information extraction device is basically the same as the specific embodiment of the table information extraction method described above, and will not be repeated here.

[0155] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned table information extraction method when executing the computer program. The electronic device can be any smart terminal including a tablet computer, an in-vehicle computer, or the like.

[0156] See also Figure 7 , Figure 7 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0157] The processor 701 may be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0158] The memory 702 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 702 and is called by the processor 701 to execute the table information extraction method of the embodiments of this application.

[0159] Input / output interface 703, used to implement information input and output;

[0160] Communication interface 704, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, Wi-Fi, Bluetooth, etc.);

[0161] Bus 705 , which transmits information between various components of the device (e.g., processor 701 , memory 702 , input / output interface 703 , and communication interface 704 );

[0162] The processor 701 , the memory 702 , the input / output interface 703 and the communication interface 704 are connected to each other in communication within the device via a bus 705 .

[0163] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned table information extraction method is implemented.

[0164] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0165] The table information extraction method, device, electronic device and storage medium provided in the embodiments of the present application further perform character-level recognition and classification on the table text content in the image to be recognized, thereby obtaining character labels at multiple character levels, improving the recognition accuracy of the table text content, and using the character labels to construct a first adjacency matrix and a label feature matrix, the first adjacency matrix represents the row and column adjacency relationship of each character label, and the label feature matrix represents the feature information of each character label, and then uses the pre-trained graph convolutional network to perform position feature extraction processing on the first adjacency matrix and the label feature matrix to identify the row position relationship between each character label, and use this as the position feature extraction processing result, and convert the row position relationship between each character label represented by the position feature extraction processing result into the adjacency relationship of each node in the second adjacency matrix, thereby determining the row position relationship of each text entity in the table to be processed, and obtaining the recognition result of the table text content. When the image is distorted or tilted upward or downward in the row direction of the table, the row position relationship of the table text content in the image can also be identified, thereby improving the accuracy of information extraction from the image table information.

[0166] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0167] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0168] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0169] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0170] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0171] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0173] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0175] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store programs.

[0176] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A table information extraction method, characterized in that: include: Identify the table to be processed in the image to be identified and obtain multiple text fragments; Performing character-level classification processing on the text segment to obtain a plurality of character labels, wherein the character labels include categories and character contents of corresponding text entities in the text segment, and the text entities are character-level information in the text segment; Constructing a first adjacency matrix and a label feature matrix based on the character labels, wherein the first adjacency matrix represents the row-column adjacency relationship of each of the character labels, and the label feature matrix contains position feature information of each of the character labels; Using a pre-trained graph convolutional network to calculate the first adjacency matrix and the label feature matrix to obtain label aggregation information, and constructing a second adjacency matrix based on the label aggregation information; the label aggregation information represents the position feature aggregation operation result of the character label and the neighbor labels of the character label, where the neighbor labels of the character label are other character labels adjacent to the current character label in the first adjacency matrix; Determining a row position relationship of each text entity in the table to be processed based on the adjacency relationship of each node in the second adjacency matrix, and determining a position of each text entity in the table to be processed based on the determined row position relationship; The step of constructing a first adjacency matrix and a label feature matrix based on the character labels includes: According to the character categories corresponding to the character labels, a plurality of character labels with the same character category within a nearest neighbor distance are associated to obtain a first adjacency relationship graph; Representing the first adjacency relationship graph in a matrix to obtain a first adjacency matrix; Obtaining several label feature vectors of the character labels; The label feature matrix is ​​constructed using each of the label feature vectors as a matrix element.

2. The table information extraction method according to claim 1, characterized in that: The process of identifying a table to be processed in a picture to be identified and obtaining multiple text segments includes: Correcting the orientation of text in the image to be recognized; Segmenting the text of the table to be processed to obtain a plurality of initial text blocks; The initial text block is fed into a pre-trained OCR recognition model for character recognition to obtain the text segment.

3. The table information extraction method according to claim 1, characterized in that: The character-level classification process is performed on the text segment to obtain a plurality of character labels, including: Obtaining a character feature matrix of the text entity in each of the text segments; Performing maximum pooling on the character feature matrix to obtain a character feature vector; Performing labeling processing on the character feature vector to obtain a character level label; The character-level labels are classified using a pre-trained classification model, and the classified character-level labels are converted to obtain character labels.

4. The table information extraction method according to claim 1, characterized in that: The constructing a second adjacency matrix based on the label aggregation information includes: Using the tag aggregation information to update the character tag to obtain a new character tag; According to the feature categories of the new character labels, connecting a plurality of new character labels with the same feature categories within a nearest neighbor distance to obtain a second adjacency relationship graph; The second adjacency relationship graph is represented by a matrix to obtain a second adjacency matrix.

5. The table information extraction method according to claim 1, characterized in that: The using of the pre-trained graph convolutional network to calculate the first adjacency matrix and the label feature matrix to obtain label aggregation information includes: Adding the first adjacency matrix to the identity matrix to obtain a new first adjacency matrix; Performing weighted summation on features of character labels and neighbor labels of the character labels in the new first adjacency matrix to obtain a summation result; Normalizing the summation result to obtain the tag aggregation information.

6. The table information extraction method according to claim 1, characterized in that: The determining, based on the adjacency relationship of each node in the second adjacency matrix, a row position relationship of each text entity in the table to be processed, and determining the position of each text entity in the table to be processed based on the determined row position relationship, includes: The nodes in the second adjacency matrix are arranged according to the adjacency relationship of the nodes in the second adjacency matrix, and the nodes in the second adjacency matrix are converted into the corresponding text entities, thereby constructing a recognized table.

7. A table information extraction device, characterized in that: The device comprises: A recognition module is used to identify the table to be processed in the image to be recognized and obtain multiple text fragments; a classification module, configured to perform character-level classification processing on the text segment to obtain a plurality of character labels, wherein the character labels include the category and character content of the corresponding text entity in the text segment, and the text entity is the character-level information in the text segment; A feature construction module, configured to construct a first adjacency matrix and a label feature matrix based on the character labels, wherein the first adjacency matrix represents the row-column adjacency relationship of each character label, and the label feature matrix contains position feature information of each character label; A feature extraction module is configured to use a pre-trained graph convolutional network to calculate the first adjacency matrix and the label feature matrix to obtain label aggregation information, and construct a second adjacency matrix based on the label aggregation information; the label aggregation information represents the positional feature aggregation operation results of the character label and the neighbor labels of the character label, where the neighbor labels of the character label are other character labels adjacent to the current character label in the first adjacency matrix; a determination module, configured to determine a row position relationship of each of the text entities in the table to be processed based on the adjacency relationship of each node in the second adjacency matrix, and determine a position of each of the text entities in the table to be processed based on the determined row position relationship; The step of constructing a first adjacency matrix and a label feature matrix based on the character labels includes: According to the character categories corresponding to the character labels, a plurality of character labels with the same character category within a nearest neighbor distance are associated to obtain a first adjacency relationship graph; Representing the first adjacency relationship graph in a matrix to obtain a first adjacency matrix; Obtaining several label feature vectors of the character labels; The label feature matrix is ​​constructed using each of the label feature vectors as a matrix element.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the table information extraction method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the table information extraction method according to any one of claims 1 to 6 is implemented.

Images