Table information recognition model training method and device, equipment and medium

Abstract

The application discloses a table information recognition model training method and device, equipment and medium, relates to the technical fields of artificial intelligence and digital medical treatment, can learn the general structure law in different table image data, and is applied to various downstream tasks. The method comprises the following steps: obtaining positive sample data for representing a table image by identifying the table image; performing cutting processing on the positive sample data, and obtaining negative sample data by splicing processing; pre-training a graph neural network model according to a data sample set constructed from the positive sample data and the negative sample data; performing splicing processing on the pre-trained graph neural network model and a classifier model, and training a table information recognition model. The application is suitable for table information extraction operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the fields of artificial intelligence technology and digital medicine, in particular to a table information recognition model training method and device, equipment and medium. BACKGROUND

[0002] During the generation or storage of a table, the table often exists in the form of an image or a portable document format (PDF) document, and the original structural information that is easy for a computer to understand is lost. If a table is reprocessed and entered by using a manual method, there are problems such as low efficiency, a large amount of data, and a high error rate. Therefore, how to enable a computer to automatically recognize a table and extract information from an image or a PDF document has become an important research problem in the field of document recognition.

[0003] In the big data era, in order to efficiently access data and extract effective information from massive data, important technologies are urgently needed in various industries. Early table recognition is mostly for relatively simple or templated tables. For example, starting from the layout structure of a table, a table line or a text block is extracted, and then a rule method is used for analysis. However, these methods often have poor generalization ability and are difficult to process complex tables. With the development of deep learning, great progress has been made in both machine vision and natural language processing. Various table recognition schemes have been proposed, and researchers have begun to try to process tables in natural scenes. For example, when using deep learning technology to recognize a table, similar to other fields, this data-driven method has good performance and precision, but it needs a large amount of manually annotated training samples. In order to learn a good model, a large amount of labeled data needs to be defined and optimized to learn the goal. It can be seen that in the face of large-scale labeled data, the construction of an application system that supports functions such as disease auxiliary diagnosis, health management, and remote consultation usually has the technical problems that labeled data of table structures are difficult to obtain, and the labeling of different data requires a high level of professional knowledge of the labeling personnel. SUMMARY

[0004] In view of this, the present application provides a table information recognition model training method, device, equipment and medium, and the main purpose is to solve the technical problems that in the face of large-scale labeled data, the construction of an application system that supports functions such as disease auxiliary diagnosis, health management, and remote consultation usually has the technical problems that labeled data of table structures are difficult to obtain, and the labeling of different data requires a high level of professional knowledge of the labeling personnel.

[0005] According to one aspect of the present application, a table information recognition model training method is provided, and the method comprises the following steps:

[0006] The positive sample data for representing the table image is obtained by recognizing the table image;

[0007] The positive sample data is split and processed, and the negative sample data is obtained by splicing processing;

[0008] The graph neural network model is pre-trained according to the data sample set constructed by the positive sample data and the negative sample data;

[0009] The graph neural network model and the classifier model pre-trained are spliced and processed, and the table information recognition model is trained.

[0010] According to another aspect of the present application, a table information recognition model training device is provided, which comprises:

[0011] The recognition module is configured to obtain positive sample data for representing the table image by recognizing the table image;

[0012] The sample processing module is configured to split and process the positive sample data, and obtain negative sample data by splicing processing;

[0013] The first training module is configured to pre-train a graph neural network model according to a data sample set constructed by the positive sample data and the negative sample data;

[0014] The second training module is configured to splice and process the graph neural network model and the classifier model pre-trained, and train a table information recognition model.

[0015] According to another aspect of the present application, a computer storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the table information recognition model training method.

[0016] According to another aspect of the present application, a computer device is provided, which comprises a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, and the processor implements the table information recognition model training method when executing the program.

[0017] By means of the technical solutions, the training method, device, equipment and medium of the table information recognition model are provided, and compared with the prior table recognition technical solutions using deep learning technology and the like, the positive sample data for representing the table image is obtained by recognizing the table image, the negative sample data is obtained by performing segmentation processing on the positive sample data and performing splicing processing, the graph neural network model is pre-trained according to the data sample set constructed based on the positive sample data and the negative sample data, the splicing processing is performed on the pre-trained graph neural network model and the classifier model, and the table information recognition model is trained. As can be seen, the table image related to the application system supporting functions such as disease auxiliary diagnosis, health management and remote consultation can be avoided from being manually marked by constructing the positive and negative sample data, the graph neural network model is pre-trained based on the unlabeled data sample to learn the general features in the table image, and the table information recognition model of the target field such as the medical system is trained by combining the classifier model and fine-tuning the graph neural network model.

[0018] The above description is only a summary of the technical solutions of the present application, in order to enable the technical means of the present application to be more clearly understood, the present application can be implemented according to the content of the description, and in order to enable the above and other purposes, features and advantages of the present application to be more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0019] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The schematic embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A flowchart of a table information recognition model training method provided by an embodiment of the present application is shown;

[0021] Figure 2 A flowchart of another table information recognition model training method provided by an embodiment of the present application is shown;

[0022] Figure 3 A structure diagram of a table information recognition model training device provided by an embodiment of the present application is shown;

[0023] Figure 4 A structure diagram of another table information recognition model training device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In the following, the present application will be described in detail with reference to the drawings and in combination with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0025] Tables are widely used in various documents within application systems supporting functions such as disease-aided diagnosis, health management, and remote consultation. They are used to compactly store and present data, containing a wealth of useful information. Table recognition is the foundation for the reuse of table information and has significant application value, making it one of the research hotspots in the field of pattern recognition. However, with the development of deep learning, due to the wide range of application scenarios, numerous styles, and inconsistent image quality, many problems still need to be solved in the field of table recognition. Similarly, natural language processing and computer vision also face similar challenges. To learn a good model, a large amount of labeled data is needed to define and optimize learning objectives, presenting a technical challenge requiring high levels of domain expertise from the labelers. This embodiment provides a training method for a table information recognition model. Based on unlabeled table images, a graph neural network model can be pre-trained to learn general features in the table images. By combining a classifier model and fine-tuning the graph neural network model, a table information recognition model for the target domain can be trained. Figure 1 As shown, this method is illustrated using an example of its application to computer devices such as servers. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, such as intelligent medical systems and digital medical platforms. The method includes the following steps:

[0026] Step 101: By recognizing the table image, positive sample data is obtained to characterize the table image.

[0027] In this embodiment, a regular document containing a table is used as raw data for processing. This regular document can also be a professional document in the field of digital healthcare. Specifically, an OCR model or system is used to recognize and process the document page (table image) containing the table to obtain a set of text blocks. Each text block contains one or more of its text information, location information, and visual information in the table image, so as to construct a fully connected directed graph to represent the table image based on one or more of the text information, location information, and visual information.

[0028] Step 102: The positive sample data is segmented and then spliced ​​to obtain the negative sample data.

[0029] In the embodiment, since the constructed full connection directed graph is obtained for a real table image, the constructed full connection directed graph is defined as positive sample data, the positive sample data includes an adjacency matrix of the full connection directed graph, a head or a tail of the adjacency matrix of the full connection directed graph is randomly split, the randomly split adjacency matrix is spliced to obtain a false full connection directed graph, and the false full connection directed graph is used as negative sample data after disturbance, so as to realize training of the graph neural network model.

[0030] In step 103, the graph neural network model is pre-trained according to the data sample set constructed by the positive sample data and the negative sample data.

[0031] In the embodiment, for a group of text blocks used for representing a table image, a graph-based method is modeled as a full connection directed graph, each text block is represented as a node, which can be a word or a text line, and an initial representation of each node can combine one or more of text information, position information and visual information of the corresponding text block, so that the graph neural network propagates information between adjacent nodes in the full connection directed graph, learns a richer general feature representation of each node, and pre-trains the graph neural network model.

[0032] In step 104, the graph neural network model and the classifier model are spliced, and a table information recognition model is trained.

[0033] In the embodiment, the general feature representation of each node output by the graph neural network model is input into the classifier model, or the features of each node in the full connection directed graph in the input data of the graph neural network model are jointly input into the classifier model, so as to realize training of the table information recognition model. According to the requirements of the actual application scene, the classifier model can be a sequence labeling model, which is not specifically limited here.

[0034] For the embodiment, the above scheme can be used to obtain positive sample data for representing a table image by recognizing the table image; the positive sample data is split, and negative sample data is obtained by splicing; a graph neural network model is pre-trained according to a data sample set constructed by the positive sample data and the negative sample data; the graph neural network model and the classifier model are spliced, and a table information recognition model is trained. Compared with the prior art, the embodiment can avoid manual labeling of table images by constructing positive and negative sample data, pre-train a graph neural network model based on unlabeled data samples to learn general features in table images, and train a target field table information recognition model by combining a classifier model and fine-tuning the graph neural network model.

[0035] Table images, as an important carrier of data, have the characteristics of information extraction and concentration, and are widely used in various industries. For example, in the field of digital medicine, tables are applied in various diagnostic results, medical receipts, and physician information introductions; in the field of education, tables are applied in various test papers and questions; in the field of finance, tables are used to display and analyze data; in the field of science, tables are used to record various experimental configurations and results; and in real life, tables are often seen on slides and station time cards. Therefore, region detection, structure recognition, and information recognition of table images have broad application prospects. Further, as a refinement and extension of the specific implementation of the above embodiments, in order to fully describe the specific implementation process of the present embodiment, another training method of a table information recognition model is provided, as shown in Figure 2 The method comprises the following steps:

[0036] Step 201, recognizing the table image by using an OCR model to obtain a group of text blocks corresponding to the table image.

[0037] Step 202, defining each text block as a node, constructing a fully connected directed graph corresponding to the group of text blocks, and taking the fully connected directed graph as positive sample data for representing the table image.

[0038] In the implementation, the group of text blocks is defined as a fully connected directed graph, that is, each text block constitutes a node, and each node is connected to at least one other node. The initial representation of each node can combine one or more of the text information, position information, and visual information of the corresponding text block, so as to obtain graph data (positive sample data) conforming to the input data format of the graph neural network, thereby realizing the pre-training of the graph neural network model.

[0039] Step 203, randomly cutting the head or tail of the adjacency matrix of the positive sample data, and splicing the randomly cut adjacency matrix to obtain perturbed negative sample data.

[0040] In the implementation, the positive sample data includes the adjacency matrix of the fully connected directed graph. For each graph data, the head or tail of the adjacency matrix A is randomly cut from the unlabeled graph data sample pool, and the randomly cut adjacency matrix is spliced to obtain perturbed negative sample data. According to the requirements of the actual application scene, other samples can also be randomly used to replace about 30% of the nodes for splicing to construct perturbed negative sample data, that is, false graph data, so as to learn to distinguish between real graph data (positive data samples) and perturbed false graph data (negative data samples) by using a self-supervised learning algorithm, and to learn the compact information representation of each node in the graph data.

[0041] Step 204, according to the data sample set constructed by the positive sample data and the negative sample data, a graph neural network model is pre-trained by using self-supervised learning; wherein, the pre-training target is to learn the full connection directed graph features and the attribute features of each node in the full connection directed graph by distinguishing the positive sample data and the negative sample data.

[0042] In implementation, graph as a data structure, graph neural network model GNN is a branch of deep learning on graph structure, common graph structure contains node and edge, node contains entity information, edge contains relation information between entities. Therefore, in order to utilize graph neural network to propagate information between adjacent nodes in full connection directed graph and learn more rich general feature representation of each node, the initial feature of edge is set as the relative coordinates of current text block (current node) and adjacent text block (adjacent node), and the length and width information of adjacent text block, and the width of current text block is normalized to make it have affine invariance.

[0043] It should be noted that the input data of the graph neural network model is a graph structure with node and edge attributes, including the adjacency matrix A of the graph and the attribute information X of the node, the dependence relationship of the graph is captured by message passing between the nodes of the graph and the result is output, for example, the node classification task outputs the node label, the graph classification task outputs the graph label, and the link prediction task outputs whether there is a link. Taking the graph classification task as an example, the graph neural network model trains the implicit vector representation of each node in the graph according to the graph structure and node attribute, so that the implicit vector representation can contain strong enough expression information, which can help each node to extract information, and finally obtain the information vector representation of the whole graph through average pooling and other methods, that is, taking the cell in the table image as the node, according to the node features and edge information between nodes in the table image, the information vector representation of the whole table image is extracted.

[0044] Formally, a graph G is defined as a triple G = (V, H, E), where, V represents the set of nodes, h i represents the attribute vector of node i, u represents the global attribute, E represents the set of edges, e k represents the attribute vector of edge, r k S is the index of the receiver node, k is the index of the sender node.

[0045] In implementation, take the graph network GN block structure as an example, a graph network GN block contains three update functions (φ) and three aggregation functions (ρ), as follows:

[0046]

[0047]

[0048]

[0049] wherein,

[0050] The aggregation function has invariance to the order of input data and can accept different numbers of model parameters.

[0051] The calculation steps of GN block are as follows:

[0052] (1) using update function φ e Process each edge, the input parameters are The return vector is e ′ k The set of output edges connected to each node i is The set of all output edges is

[0053]

[0054] (2) using aggregation function ρ e→h Process the set E i Get the information collection vector of the output edge connected to node i And update the next node.

[0055] (3) using update function φ h Process each node i, used to calculate and update the attribute vector h of node i i ′ The output set of all nodes is

[0056] (4) using aggregation function ρ e→u Process the set E, get the information collection vector of all edges For global update in the next step.

[0057] (5) using aggregation function ρ h→u Process the set H, get the information collection vector of all nodes For global update in the next step.

[0058] with the update function φ u For each graph, process it for computing the updated global attribute vector to obtain the global attribute u.

[0059] Unlike the prior art, training a graph neural network model requires a large amount of labeled data to define and optimize the learning goal. In the face of large-scale labeled data scenarios, including natural language processing and computer vision, it is often difficult to obtain labeled data based on graph structure table images, and different data labeling requires high professional knowledge. The embodiment designs a self-supervised learning task on table image data, pre-trains a graph neural network model based on unlabeled target domain (for example, digital medical field) table image data, and fine-tunes learning on a small-scale data set in the target domain to obtain the expected recognition model in the target domain.

[0060] The pre-trained graph neural network model has universality and basicness based on the constructed large-scale unlabeled data sample set, and can support an indefinite number of subsequent target tasks through migration. Therefore, the model training for various target tasks such as table image recognition in diagnosis and treatment business no longer needs a large amount of labeled data for training, thereby saving the model training time of the target task and improving the training efficiency.

[0061] In the implementation, based on the pre-training learning of contrastive learning, based on the constructed large-scale unlabeled data sample set, the graph neural network model is guided to distinguish positive and negative data samples to learn the information features in the input data. Specifically, the graph neural network model based on contrastive learning is pre-trained in a self-supervised manner, positive and negative data samples are constructed based on graph data represented by table images, so that the graph neural network model distinguishes and learns positive and negative data samples in the implicit representation space, to construct a self-supervised signal for contrast training, and then based on the given positive sample data and negative sample data, the encoder is distinguished to learn to capture the information features with discriminative characteristics in the data sample.

[0062] In the implementation, when the graph neural network model learns that the negative sample data spliced out is false table image data, the training is completed, at this time the graph neural network model has good table image understanding ability and can better learn the table image. For sample data x, the pre-training learning goal of contrastive learning is:

[0063] score(f(x),f(x + ))>>score(f(x),f(x - )

[0064] Where x + is similar or equal to x, called positive sample data, x -is data that is dissimilar to x, referred to as negative sample data; the score function is a metric function used to evaluate the similarity between two features; x is usually referred to as an "anchor" data point, and a softmax classifier is constructed to correctly classify positive sample data and negative sample data, and the softmax classifier encourages the score function to assign a larger value to positive sample data and a smaller value to negative sample data. According to the requirements of the actual application scene, it can also be an N-way softmax classifier, and a cross-entropy loss function is constructed to represent the ratio of positive sample data to negative sample data, as follows:

[0065]

[0066] In implementation, for self-supervised contrastive learning of table image data, for any two data, the more similar (belonging to the same class), the closer the table image representation will be. Specifically:

[0067] (1) Use the original large amount of unlabeled table image dataset N, for each positive sample data (its label is 1), randomly retain the original or replace and splice with a probability of 0.5, to obtain the perturbed negative sample data corresponding to each positive sample data, whose label is 0, that is, the N table image data sets contain N / 2 positive sample data and negative sample data;

[0068] (2) Randomly extract a certain size batch of processed sample data (Batch), containing positive sample data and negative sample data of various classes, and perform graph neural network model training;

[0069] (3) The training and learning goal of the graph neural network model is to learn to distinguish whether the sample data is the original positive sample data or the replaced false negative sample data, in order to learn to understand and express the table image.

[0070] Step 205, splice the pre-trained graph neural network model and the classifier model, and initialize the model parameters of the output layer in the graph neural network model.

[0071] Step 206, train the table information recognition model by inputting the output result of the graph neural network model into the classifier model.

[0072] To illustrate the specific implementation of step 206, as a preferred embodiment, it specifically comprises: obtaining node features in a full connection directed graph for representing the table image in the input data of the graph neural network model; and performing concatenation processing on the output data of the graph neural network model and the node features in the full connection directed graph to obtain input data of the classifier model; wherein the node features in the full connection directed graph at least include one of text features, position features and visual features for representing each text block in the table image.

[0073] To illustrate the specific implementation of step 206, as a preferred embodiment, the training method of the table information recognition model comprises: determining different classifier models according to different business categories; and performing concatenation processing on the pre-trained graph neural network model and the determined classifier model to obtain the table information recognition model.

[0074] In implementation, the pre-trained graph neural network model can learn global information of the table image, efficiently calculate and learn key or more rich representation information of the table image, thus learn general structural rules in different table image data from large-scale unlabeled data samples, make the graph neural network model have good generalization, and can be applied to various downstream tasks, such as diagnosis and treatment business, etc. Unlike the manual parameter tuning, relying on machine learning engineers and experts in the prior art, the industrial implementation mode applicable to large-scale and replication can be obtained, that is, the fitting ability in the downstream table image mining tasks of unlimited types can be given. Specifically, when a specific downstream task needs to be implemented, the table field classification (i.e., node classification), for example, the table field content is address or amount, the pre-trained graph neural network model is concatenated with the corresponding classifier model to obtain the table information recognition model, and the table information recognition model is fine-tuned to avoid training a brand-new model from scratch for each downstream task.

[0075] By applying the technical solution of the embodiment, the table image is recognized to obtain positive sample data for representing the table image; the positive sample data is segmented and processed, and negative sample data is obtained through splicing processing; a data sample set constructed according to the positive sample data and the negative sample data is used to pre-train a graph neural network model; the pre-trained graph neural network model and a classifier model are spliced and processed, and a table information recognition model is trained. Compared with the prior art of using deep learning technology to recognize tables, the embodiment can avoid manual marking of table images by constructing positive and negative sample data, and pre-train a graph neural network model based on unlabeled data samples to learn general features in table images. By combining a classifier model and fine-tuning the graph neural network model, a table information recognition model for a target field is trained.

[0076] Further, as Figure 1 A specific implementation of the method, the embodiment of the present application provides a table information recognition model training device, as Figure 3 shown, the device comprises an identification module 31, a sample processing module 32, a first training module 33, and a second training module 34.

[0077] The identification module 31 is configured to recognize a table image to obtain positive sample data for representing the table image.

[0078] The sample processing module 32 is configured to segment and process the positive sample data, and obtain negative sample data through splicing processing.

[0079] The first training module 33 is configured to pre-train a graph neural network model according to a data sample set constructed from the positive sample data and the negative sample data.

[0080] The second training module 34 is configured to splice and process the pre-trained graph neural network model and a classifier model, and train a table information recognition model.

[0081] In a specific application scenario, as Figure 4 shown, the identification module 31 comprises an OCR unit 311 and a construction unit 312.

[0082] The OCR unit 311 can be configured to recognize a table image using an OCR model to obtain a group of text blocks corresponding to the table image.

[0083] The construction unit 312 can be configured to define each text block as a node, construct a fully connected directed graph corresponding to the group of text blocks, and use the fully connected directed graph as positive sample data for representing the table image.

[0084] In a specific application scenario, the positive sample data includes an adjacency matrix of a full connection directed graph, and the sample processing module 32 can be configured to perform random cutting processing on the head or tail of the adjacency matrix of the positive sample data, and perform splicing processing on the adjacency matrix after the random cutting processing to obtain negative sample data after perturbation.

[0085] In a specific application scenario, the first training module 33 can be configured to pre-train a graph neural network model by using self-supervised learning according to a data sample set constructed based on the positive sample data and the negative sample data; and the pre-training target is to learn the full connection directed graph features and the attribute features of each node in the full connection directed graph by distinguishing the positive sample data and the negative sample data.

[0086] In a specific application scenario, the second training module 34 includes a splicing unit 341 and a fine-tuning training unit 342.

[0087] The splicing unit 341 can be configured to perform splicing processing on the pre-trained graph neural network model and the classifier model, and initialize the model parameters of the output layer in the graph neural network model.

[0088] The fine-tuning training unit 342 can be configured to train the table information recognition model by inputting the output result of the graph neural network model into the classifier model.

[0089] In a specific application scenario, the fine-tuning training unit 342 can be specifically configured to obtain the node features in the full connection directed graph for representing the table image according to the input data of the graph neural network model, perform splicing processing on the output data of the graph neural network model and the node features in the full connection directed graph to obtain the input data of the classifier model, and the node features in the full connection directed graph at least include one of the text features, the position features and the visual features for representing each text block in the table image.

[0090] In a specific application scenario, the training method of the table information recognition model includes: determining different classifier models according to different business categories; performing splicing processing on the pre-trained graph neural network model and the determined classifier model, and training the table information recognition model.

[0091] It should be noted that other corresponding descriptions of the functions of the various functional units involved in the training device of the table information recognition model provided in the embodiments of the present application can be referred to the corresponding descriptions in Figure 1 and Figure 2 , which will not be repeated here.

[0092] Based on the above as Figure 1 and Figure 2The method shown, accordingly, the embodiment of the application also provides a computer storage medium, which stores a computer program, the program is executed by the processor to realize the above-mentioned as Figure 1 and Figure 2 The training method of the table information recognition model shown.

[0093] Based on such understanding, the technical scheme of the present application can be embodied in the form of a software product, which can be stored in a storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in each embodiment scenario of the present application.

[0094] Based on the above-mentioned method as Figure 1 , Figure 2 The method shown, and Figure 3 The virtual device embodiment shown, in order to realize the above-mentioned purpose, the embodiment of the application also provides a computer device, specifically can be personal computer, server, network device, etc., the entity device includes storage medium and processor; the storage medium is used for storing computer program; the processor is used for executing computer program to realize the above-mentioned as Figure 1 and Figure 2 The training method of the table information recognition model shown.

[0095] Optionally, the computer device can also include a user interface, a network interface, a camera, a radio frequency (Radio Frequency, RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface can include a display screen (Display), an input unit such as a keyboard (Keyboard), etc. The optional user interface can also include a USB interface, a card reader interface, etc. The network interface can optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0096] Those skilled in the art can understand that the structure of the computer device provided by the embodiment does not constitute a limitation on the entity device, which can include more or fewer components, or combine certain components, or different component arrangements.

[0097] The storage medium can also include an operating system, a network communication module. The operating system is a program that manages computer device hardware and software resources, supports information processing programs and the running of other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the entity device.

[0098] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software with a necessary general hardware platform, or by hardware. By applying the technical solutions of the present application, compared with the prior art, the present embodiment can avoid manual marking of table images by constructing positive and negative sample data, and pre-train a graph neural network model based on unlabelled data samples to learn general features in table images, and train a table information recognition model of a target field by combining a classifier model and fine-tuning the graph neural network model.

[0099] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred implementation scenario, and the modules or flows in the drawings are not necessarily required for implementing the present application. Those skilled in the art can understand that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the description of the implementation scenario, or can be changed and located in one or more devices different from the implementation scenario. The modules of the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0100] The above application numbers are only for description, and do not represent the advantages and disadvantages of the implementation scenario. The above disclosure is only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any changes that those skilled in the art can think of should fall within the protection scope of the present application.

Claims

1. A method for training a table information recognition model, comprising: The method comprises the following steps: obtaining positive sample data for representing the table image by recognizing the table image; performing segmentation processing on the positive sample data, and obtaining negative sample data through splicing processing; pre-training a graph neural network model according to a data sample set constructed based on the positive sample data and the negative sample data; performing splicing processing on the pre-trained graph neural network model and a classifier model, and training a table information recognition model; wherein the positive sample data comprises an adjacency matrix of a fully connected directed graph, and the step of performing segmentation processing on the positive sample data and obtaining negative sample data through splicing processing comprises: randomly performing segmentation processing on the head or tail of the adjacency matrix of the positive sample data, and performing splicing processing on the randomly segmented adjacency matrix to obtain perturbed negative sample data.

2. The method of claim 1, wherein, The step of obtaining positive sample data for representing the table image by recognizing the table image comprises: recognizing the table image by using an OCR model to obtain a group of text blocks corresponding to the table image; defining each text block as a node, constructing a fully connected directed graph corresponding to the group of text blocks, and taking the fully connected directed graph as the positive sample data for representing the table image.

3. The method of claim 2, wherein, The step of pre-training a graph neural network model according to a data sample set constructed based on the positive sample data and the negative sample data comprises: pre-training a graph neural network model by using self-supervised learning based on the data sample set constructed based on the positive sample data and the negative sample data; wherein the pre-training target is to learn the features of the fully connected directed graph and the attribute features of each node in the fully connected directed graph by distinguishing the positive sample data from the negative sample data.

4. The method of claim 1, wherein, The step of performing splicing processing on the pre-trained graph neural network model and the classifier model, and training a table information recognition model comprises: performing splicing processing on the pre-trained graph neural network model and the classifier model, and initializing the model parameters of the output layer of the graph neural network model; training a table information recognition model by inputting the output result of the graph neural network model into the classifier model.

5. The method of claim 4, wherein, The step of inputting the output result of the graph neural network model into the classifier model comprises: obtaining the node features in the fully connected directed graph used for representing the table image according to the input data of the graph neural network model; performing splicing processing on the output data of the graph neural network model and the node features in the fully connected directed graph to obtain the input data of the classifier model; wherein the node features in the fully connected directed graph at least comprise one of text features, position features and visual features used for representing each text block in the table image.

6. The method of claim 1, wherein, The training method of the table information recognition model comprises: determining different classifier models according to different business categories; performing splicing processing on the pre-trained graph neural network model and the determined classifier model, and training a table information recognition model. 7.A device for training a table information recognition model, comprising: The method comprises the following steps: a recognition module is configured to obtain positive sample data for representing the table image by recognizing the table image; A sample processing module is configured to perform a splitting processing on the positive sample data, and obtain negative sample data through a splicing processing. A first training module is configured to pre-train a graph neural network model according to a data sample set constructed based on the positive sample data and the negative sample data. A second training module is configured to perform a splicing processing on the pre-trained graph neural network model and a classifier model, and train a table information recognition model. The positive sample data includes an adjacency matrix of a full connection directed graph, and the sample processing module is configured to perform a random splitting processing on a head or a tail of the adjacency matrix of the positive sample data, and perform a splicing processing on the randomly split adjacency matrix to obtain perturbed negative sample data.

8. A computer storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the training method of the table information recognition model in any one of claims 1 to 6.

9. A computer device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, The processor executes the program to implement the training method of the table information recognition model in any one of claims 1 to 6.

Images