Data processing method and related device

Abstract

Disclosed in the embodiments of the present application is a data processing method. In the method, the location of a nested table in a document can be identified by means of a detection module, such that on the basis of features such as the location of the nested table, the content of the nested table in the document is targetedly recognized by means of a table content recognition model, thereby accurately restoring the content of the nested table. It can be seen that in the method, when a document comprises a relatively complex table structure such as a nested table, the content of the nested table can still be accurately recognized by combining a detection model and a table content recognition model, thereby improving the accuracy of table content recognition in documents comprising complex structures such as a nested table.

Description

Data processing method and related device

[0001] The present application claims priority to the Chinese patent application No. 202410450975.8, filed on April 11, 2024, and entitled "A data processing method and related device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of table recognition, in particular to a data processing method and related device. BACKGROUND

[0003] In many application scenarios, information needs to be extracted from various documents for subsequent data processing. For example, in the financial field, information needs to be extracted from documents such as contracts and financial reports for financial analysis. Since tables are simple and clear in form, many documents use tables to carry data.

[0004] Current table recognition methods usually directly recognize the content of the entire table in the document. When the structure of the table is complex, it is often difficult to accurately recognize the content of the table, resulting in inaccurate detection results of the content of the table. SUMMARY

[0005] The embodiments of the present application provide a data processing method, which can accurately recognize the content of a table in a document when the document includes a nested table. The present application also provides corresponding devices, equipment, computer readable storage media and computer program products, etc.

[0006] The first aspect of the present application provides a data processing method. In the method, a to-be-processed document can be acquired. A detection model is used to detect the position of each nested table in at least one nested table included in the to-be-processed document, and the at least one nested table is located in an outermost table included in the to-be-processed document. Then, a table content recognition model is used to recognize the content of each nested table in the at least one nested table.

[0007] In the first aspect, the position of a nested table in a document can be recognized by a detection model, so that the content of the nested table in the document can be recognized by a table content recognition model according to the position and other characteristics of the nested table, thereby accurately restoring the content of the nested table. It can be seen that in the first aspect, when a document includes a nested table and other complex table structures, the content of the nested table can still be accurately recognized by combining the detection model and the table content recognition model, thereby improving the recognition accuracy of the table content in the document including the nested table and other complex structures.

[0008] In a possible implementation manner of the first aspect, the detection model comprises a first detection model and a second detection model; and detecting, by the detection model, the position of each of the at least one layer of nested tables included in the to-be-processed document comprises: detecting, by the first detection model, a table region in the to-be-processed document, the table region comprising an outermost table; and detecting, by the second detection model, the position of each of the at least one layer of nested tables included in the table region.

[0009] In this possible implementation manner, the first detection model can be regarded as a layout analysis model, which can perform layout analysis on the to-be-processed document to detect the table region in the to-be-processed document. Compared with a conventional manner of using a text box to detect and identify a table, and a manner of directly identifying the content of the table in the to-be-processed document, the layout analysis model can adapt to various layout forms of the document, and can accurately identify the table region from the to-be-processed document, thereby providing a better data basis for subsequent identification of the content of the table and the like.

[0010] The second detection model can be regarded as a nested table detection model, which is used to detect the position of the nested table from the table region. The second detection model can be a trained model. Compared with a conventional manner of directly identifying the content of the table in the document, the second detection model can learn the features of the nested table in the pre-training process, thereby accurately identifying the position of the nested table in the table region. In this way, the first detection model and the second detection model are combined to perform multi-level detection, which can greatly improve the detection accuracy of the position of each nested table in the document.

[0011] In a possible implementation manner of the first aspect, after the content of each of the at least one layer of nested tables is identified by the table content identification model, the method further comprises: obtaining the content of the table region according to the content of each of the at least one layer of nested tables.

[0012] In this possible implementation manner, the table region comprises the outermost table of the to-be-processed document, and thus the content of the table region can be regarded as the complete content of the table in the to-be-processed document.

[0013] In a possible implementation manner of the first aspect, the method further comprises: covering a first region in the table region to obtain a covered outermost table, the first region being a region in the table region that comprises the at least one layer of nested tables; identifying, by the table content identification model, the content of the covered outermost table; and obtaining the content of the table region according to the content of each of the at least one layer of nested tables, comprising: obtaining the content of the table region according to the content of the covered outermost table and the content of the at least one layer of nested tables.

[0014] In the possible implementation manner, the internal complex structure formed in the outermost table due to the nested tables can be covered, so that the structure form in the covered outermost table is relatively simple and clear, and the content of the covered outermost table is convenient for the table content recognition model to recognize, so as to obtain the complete and accurate content of the table region according to the content of the covered outermost table and the content of the at least one nested table.

[0015] In a possible implementation manner of the first aspect, the at least one nested table includes a first nested table and a second nested table, and the first nested table is an outer table of the second nested table; the identifying, by the table content recognition model, the content of each nested table in the at least one nested table includes: covering a second region in the first nested table to obtain a covered first nested table, the second region being a region in the first nested table including the second nested table; identifying, by the table content recognition model, the content of the covered first nested table; identifying, by the table content recognition model, the content of the second nested table; and obtaining the content of the first nested table according to the content of the covered first nested table and the content of the second nested table.

[0016] In the possible implementation manner, the complex structure in the first nested table due to the second nested table can be covered, so that the structure form in the covered first nested table is relatively simple and clear, and the content of the covered first nested table is convenient for the table content recognition model to recognize.

[0017] It can be seen that when there are multiple nested tables in the to-be-processed document, the content of the innermost nested table and the content of the covered table corresponding to other tables except the innermost nested table can be obtained by referring to the identification manner of the content of the first nested table and the second nested table, and by the table content recognition model. Then, the content of the innermost nested table and the content of the covered table corresponding to other tables can be combined according to the positions of the nested tables from inside to outside to obtain the complete content of each nested table.

[0018] In a possible implementation manner of the first aspect, before the identifying, by the table content recognition model, the content of each nested table in the at least one nested table, the method further includes: detecting, by a detection model, the to-be-processed document to output first coordinate information of the first nested table and second coordinate information of the second nested table; and determining, according to the first coordinate information and the second coordinate information, that the first nested table is an outer table of the second nested table.

[0019] In the possible implementation manner, the second detection model can output the coordinate information of each nested table in the at least two nested tables, and identify the inner-outer relationship between the nested tables according to the coordinate information.

[0020] In a possible implementation form of the first aspect, the table content recognition model comprises a third detection model and a text detection model; and the table content recognition model is configured to recognize the content of each of the at least one layer of nested tables by: the third detection model, configured to detect structure information of each of the at least one layer of nested tables, the structure information being used to describe at least one sub-region in the nested table; and the text detection model, configured to recognize the content of each of the at least one sub-region.

[0021] In this possible implementation form, the structure information of the nested table can be accurately detected by the third detection model to recognize various structures such as text boxes in the nested table, so as to split the nested table into at least one sub-region with simple structure. Then, the text detection model is used to detect the text in the sub-region with simple structure, so as to avoid the interference of the complex structure such as the border of the nested table on the table content recognition, and to accurately detect the content of the sub-region, thereby accurately obtaining the complete information in the nested table through the content of the sub-regions.

[0022] In a possible implementation form of the first aspect, the detection model is configured to detect the position of each of the at least one layer of nested tables included in the to-be-processed document by: converting the to-be-processed document into data in a target format; and detecting the position of each of the at least one layer of nested tables included in the data in the target format by the detection model.

[0023] In this possible implementation form, the to-be-processed document is converted into a fixed target format before being input into the detection model, so as to ensure the uniformity of the format of the input data of the detection model, avoid the interference of various document types on the detection of the detection model, and help to improve the accuracy of the detection model.

[0024] The second aspect of the present application provides a data processing apparatus, which has the function of implementing the method of the first aspect or any possible implementation form of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software comprises one or more modules corresponding to the above functions, such as an acquisition module and a processing module.

[0025] The third aspect of the present application provides a computing device cluster, which comprises at least one computing device, and the at least one computing device comprises a processor and a memory. The memory of the at least one computing device stores computer-executable instructions that can be executed on the processor. When the computer-executable instructions are executed by the processor, the processor executes the method of the first aspect or any possible implementation form of the first aspect.

[0026] The fourth aspect of the present application provides a computer readable storage medium storing one or more computer-executable instructions that, when executed by a processor, cause the processor to perform the method of the first aspect or any possible implementation of the first aspect.

[0027] The fifth aspect of the present application provides a computer program product storing one or more computer-executable instructions that, when executed by a processor, cause the processor to perform the method of the first aspect or any possible implementation of the first aspect.

[0028] The sixth aspect of the present application provides a chip system including a processor for supporting the processor to implement the functions involved in the first aspect or any possible implementation of the first aspect. In a possible design, the chip system can further include a memory for storing necessary program instructions and data. The chip system can be composed of a chip, or can include the chip and other discrete devices.

[0029] The technical effects brought by the second aspect to the sixth aspect or any possible implementation thereof can be referred to the technical effects brought by the first aspect or any possible implementation of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0030] FIG. 1 is a schematic diagram of an exemplary system framework provided by an embodiment of the present application;

[0031] FIG. 2 is a schematic diagram of an embodiment of a data processing method provided by an embodiment of the present application;

[0032] FIG. 3 is a schematic diagram of an exemplary flow provided by an embodiment of the present application;

[0033] FIG. 4 is a schematic diagram of a multi-layer nested table provided by an embodiment of the present application;

[0034] FIG. 5 is a schematic diagram of obtaining the content of a first nested table provided by an embodiment of the present application;

[0035] FIG. 6 is a schematic diagram of model training provided by an embodiment of the present application;

[0036] FIG. 7 is a schematic diagram of an embodiment of a data processing apparatus provided by an embodiment of the present application;

[0037] FIG. 8 is a structural schematic diagram of a computing device provided by an embodiment of the present application;

[0038] FIG. 9 is a structural schematic diagram of a computing device cluster provided by an embodiment of the present application;

[0039] FIG. 10 is a structural schematic diagram of a computing device cluster provided by an embodiment of the present application; DETAILED DESCRIPTION

[0040] The embodiments of the present application will be described below in conjunction with the accompanying drawings. The terms used in the implementation part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0041] Those of ordinary skill in the art can know that, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0042] In the present application, "at least one" means one or more, and "multiple" means two or more. The "and / or" describes the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a way of distinguishing the objects with the same properties used in the description of the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the process, method, system, product or equipment containing a series of units does not have to be limited to those units, but can include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0043] First, some concepts related to the embodiments of the present application are introduced.

[0044] 1. Document

[0045] The document is a file capable of recording text information.

[0046] The specific type of the document can be various, which can be an existing or subsequent developed file type.

[0047] Exemplarily, the document can include editable documents and non-editable documents.

[0048] The specific type of the editable document can include extensible markup language (XML) documents, word documents, txt format documents, etc. For the editable document, the information in the document can be directly obtained by using a corresponding editing tool with a relatively simple rule.

[0049] The specific type of the non-editable document can include a document image, a portable document format (PDF), and the like. The document image can be obtained by scanning or photographing a paper document, or by format conversion of an editable document.

[0050] 2. Nested table

[0051] The nested table is a table located in a table cell.

[0052] 3. Neural network

[0053] The neural network can be composed of neural units, and the neural unit can refer to an operation unit with x s (input data) and an intercept 1 as inputs, and the output of the operation unit can be:

[0054] wherein s = 1, 2, … n, n is a natural number greater than 1, W s is a weight parameter of x s , b is a bias of the neural unit. f is an activation function of the neural unit, which is used to introduce a nonlinear characteristic into the neural network to convert the input signal in the neural unit into an output signal. The output signal of the activation function can be used as the input of the next layer of the convolution layer, and the activation function can be a sigmoid function. The neural network is a network formed by connecting a plurality of the above single neural units, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be a region composed of a plurality of neural units.

[0055] 4. Loss function

[0056] In the process of training a neural network, because the output of the neural network is expected to be as close as possible to the value that is actually intended to be predicted, the weight vector of each layer of the neural network can be updated according to the difference between the predicted value of the current network and the target value that is actually intended to be predicted (of course, before the first update, there is usually an initialization process, that is, the parameters of each layer in the deep neural network are pre-configured), for example, if the predicted value of the network is too high, the weight vector is adjusted to make it predict lower, and the adjustment is continuously made until the neural network can predict the target value that is actually intended to be predicted or a value very close to the target value that is actually intended to be predicted. Therefore, it is necessary to define "how to compare the difference between the predicted value and the target value", which is the loss function or the objective function, which is an important equation for measuring the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, and then the training of the neural network becomes a process of trying to minimize the loss.

[0057] 5. Back propagation algorithm

[0058] The neural network can use the back propagation (BP) algorithm to correct the size of the parameters in the initial super-resolution model during the training process, so that the reconstruction error loss of the super-resolution model becomes smaller and smaller. Specifically, the forward propagation of the input signal until the output produces an error loss, and the error loss information is updated by back propagation to update the parameters in the initial super-resolution model, so as to make the error loss converge. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the super-resolution model, such as the weight matrix.

[0059] In many application scenarios, information needs to be extracted from various documents for subsequent data processing. For example, in the financial field, information often needs to be extracted from documents such as contracts and financial reports for financial analysis. In scenarios where the document is an uneditable document or an editable document cannot be edited using an editing tool for editable documents, information is usually extracted from the document through a corresponding recognition method.

[0060] In actual application scenarios, information is often recorded in a document through a table, so when extracting information from a document, table recognition is often needed to extract information from a table contained in the document.

[0061] However, the inventors of the present application have found that the current table recognition method does not focus on the recognition scenario of nested tables, and usually treats nested tables as ordinary tables, resulting in poor performance in restoring the content of nested tables.

[0062] Based on this, the data processing method provided in the embodiments of the present application can accurately identify the content of the table in the document when the document includes a nested table.

[0063] The data processing method provided in the embodiments of the present application can be applied to a computing device cluster, which can include one or more computing devices.

[0064] Here, the type of any computing device is not limited. For example, any computing device can be a terminal device, a server, a server cluster, a container, a virtual machine, or the like. When the computing device cluster includes multiple computing devices, the types of different computing devices can be the same or different.

[0065] In the embodiments of the present application, the system architecture in which the computing device cluster is located can have various cases, which are not limited here.

[0066] For example, the computing device cluster can include one or more servers located in the cloud, and after receiving the document of the client device, the data processing method provided in the embodiments of the present application is implemented by calling related microservices. Alternatively, the computing device cluster can also be a client device, and the data processing method provided in the embodiments of the present application is implemented by an installed application program or plug-in. Alternatively, the computing device cluster can include servers located in the cloud and client devices, and the data processing method provided in the embodiments of the present application is implemented through information interaction between the cloud and the client devices.

[0067] For example, as shown in the document analysis scenario in FIG. 1, the computing device cluster can receive a document sent by a user through a client device, and after receiving the document, the document is preprocessed.

[0068] For the text information in the document, text recognition can be performed by optical character recognition (OCR) or the like to extract the text information. For the table in the document, the content of the table can be obtained through table recognition.

[0069] In this way, the computing device cluster can restore the information in the document, so that it can be applied to downstream information extraction tasks, for example, business analysis can be performed by extracting key information (such as information of key fields) in the document.

[0070] The computing device cluster can take the extracted key field information in the document as a processing result, or perform business analysis according to the key field information and take the obtained business analysis result as a processing result.

[0071] In the example shown in FIG. 1, the processing result can be output to the client device. Of course, in other examples, the processing result can also be output to other devices.

[0072] The data processing method of the embodiments of the present application can include the related steps of the table identification stage shown in FIG. 1, thereby identifying the content of the table in the document.

[0073] The specific content of the data processing method of the embodiments of the present application will be introduced below.

[0074] As shown in FIG. 2, based on the above-mentioned computing device cluster, the data processing method includes steps 201-203.

[0075] Step 201, obtaining a to-be-processed document.

[0076] In the embodiments of the present application, the source of the to-be-processed document can be various. For example, the to-be-processed document can be stored locally by the computing device cluster executing the embodiments of the present application, can be transmitted to the computing device cluster from other devices (such as client devices), or can be collected by the computing device cluster through a webpage or other applications. The specific type of the to-be-processed document can also be various, which is not limited here. Illustratively, the to-be-processed document can be a PDF or a document image, etc.

[0077] Step 202, detecting the position of each layer of nested table in at least one layer of nested table included in the to-be-processed document by a detection model.

[0078] The at least one layer of nested table is located in the outermost table included in the to-be-processed document.

[0079] In the embodiments of the present application, the detection model can be a trained neural network.

[0080] The specific type of the detection model can be various, for example, it can be an existing or subsequent developed neural network.

[0081] Illustratively, the detection model can include a yolo model or a dino model, or can include a combination of one or more existing neural networks.

[0082] In some examples, the to-be-detected document can be directly input into the detection model.

[0083] And in other examples, the to-be-detected document can be preprocessed before being input into the detection model.

[0084] Specifically, in some embodiments, the above-mentioned step 202 includes:

[0085] Converting the to-be-processed document into data in a target format;

[0086] The detection of the target format data includes the position of each of the at least one layer of nested tables.

[0087] Exemplarily, in the exemplary detection process shown in FIG. 3, the target format data can be in a picture format. That is, in the example shown in FIG. 3, the document to be processed can be converted into data in a picture format, and then input into the detection model. At this time, the detection model can be an image detection model, so as to accurately detect the data in the picture format.

[0088] In this example, the document to be processed is converted into a fixed target format and then input into the detection model, which can ensure the uniformity of the format of the input data of the detection model, avoid the interference of diversified document types on the detection of the detection model, and help improve the accuracy of the detection model.

[0089] Through the detection model, the position of each of the at least one layer of nested tables included in the document to be processed can be detected.

[0090] The nested table is a table located in a table cell, that is, in this example, the at least one layer of nested tables included in the document to be processed is located in the outermost table of the document to be processed.

[0091] The position of each of the at least one layer of nested tables can be used to describe the area where each of the at least one layer of nested tables is located. The position of each of the at least one layer of nested tables can be described by the coordinates of the identification points of each of the at least one layer of nested tables. In this example, the detection model can output the coordinates of the identification points of each of the at least one layer of nested tables. For example, generally, the nested table is in a rectangular structure, and the position of the nested table can be described by the coordinates of the four vertices of the rectangular structure.

[0092] In some examples, the detection model can be a neural network, and the neural network can be directly used to infer the document to be processed to output the position of each of the at least one layer of nested tables included in the document to be processed.

[0093] In other examples, at least two levels of sub-models can be used to respectively perform layout analysis and nested table detection, so as to accurately identify the position of each of the at least one layer of nested tables in the document to be processed.

[0094] Specifically, in some embodiments, the detection model includes a first detection model and a second detection model.

[0095] Step 202 includes:

[0096] Through the first detection model, the table area in the document to be processed is detected, and the table area includes the outermost table.

[0097] The second detection model is used to detect the position of each nested table in the at least one nested table included in the table region.

[0098] The first detection model can be a layout analysis model, which is used to perform layout analysis on the to-be-processed document to detect the table region in the to-be-processed document. For example, the first detection model can be a yolo model or a dino model.

[0099] The layout analysis model can be a trained model, and can be used to analyze various layouts of the to-be-processed document to accurately identify the table region in various layout cases. For example, the layout analysis model can be used to identify a wireless table or a table region including a nested table and the like.

[0100] The table region can include an area covered by all tables in the to-be-processed document. Specifically, the table region can include an outermost table in the to-be-processed document, and the nested table and the like can be an internal structure of the outermost table.

[0101] It can be seen that, compared with a conventional method of using a text box to detect and identify a table, and a method of directly identifying the content of a table in a to-be-processed document, the layout analysis model can adapt to various layout forms of the document, and can accurately identify the table region from the to-be-processed document to provide a better data basis for subsequent identification of the content of the table and the like.

[0102] The second detection model can be considered as a nested table detection model, which is used to detect the position of the nested table from the table region.

[0103] The second detection model can be a trained model. It can be understood that, compared with a conventional method of directly identifying the content of a table in a document, the second detection model can learn the features of the nested table in a targeted manner in a pre-training process, so as to accurately identify the position of the nested table in the table region. Thus, the first detection model and the second detection model can be used for multi-level detection, which can greatly improve the detection accuracy of the position of each nested table in the document.

[0104] In the multi-level detection by the first detection model and the second detection model, the table region can be cropped from the to-be-processed document after the first detection model detects the table region in the to-be-processed document, and the cropped table region can be input into the second detection model to detect at least one nested table in the table region by the second detection model.

[0105] In the multi-level detection by the first detection model and the second detection model, after the first detection model detects the table region in the to-be-processed document, the table region can be cropped from the to-be-processed document, and the cropped table region can be input into the second detection model to detect at least one nested table in the table region by the second detection model.

[0106] As shown in the example detection flow shown in FIG. 3, an example flow of nested table detection by the first detection model and the second detection model in the detection model is shown.

[0107] In the example shown in FIG. 3, the to-be-processed document can be converted into picture format data and then input into the first detection model to perform layout analysis, so as to detect the table region in the to-be-processed document.

[0108] After the first detection model detects the table region in the to-be-processed document, the table region can be cropped from the to-be-processed document, and the cropped table region can be input into the second detection model to detect at least one nested table in the table region by the second detection model.

[0109] Since the table region includes the outermost table in the to-be-processed document, after the at least one nested table is detected by the second detection model, the positions of the multiple layers of tables in the to-be-processed document can be obtained, including the position of the outermost table and the positions of at least one layer of nested tables in the outermost table.

[0110] In addition, the inner-outer layer relationship between the tables in the to-be-processed document can also be determined.

[0111] In some examples, if the to-be-processed document includes only one layer of nested tables, it can be determined that the layer of nested tables is an inner table of the outermost table, that is, the inner-outer layer relationship between the tables in the table region can be directly known.

[0112] At this time, the content of the layer of nested tables can be directly recognized by the table content recognition model.

[0113] In some examples, if the to-be-processed document includes at least two layers of nested tables, the inner-outer layer relationship between the nested tables in the table region can be determined.

[0114] Hereinafter, the first nested table and the second nested table in the at least two layers of nested tables are taken as examples for introduction.

[0115] For example, the detection model can directly output the inner-outer layer relationship between the nested tables in the at least two layers of nested tables.

[0116] Alternatively, the detection model can output coordinate information of each of the at least two layers of nested tables, and identify the inner-outer relationship between the nested tables according to the coordinate information.

[0117] Specifically, in some embodiments, the method further comprises:

[0118] detecting, by the detection model, the document to be processed to output first coordinate information of a first nested table and second coordinate information of a second nested table;

[0119] determining, according to the first coordinate information and the second coordinate information, that the first nested table is an outer table of the second nested table.

[0120] The first coordinate information of the first nested table can include coordinates of vertex edge identification points of the first nested table to describe edge positions of the first nested table by the first coordinate information. The second coordinate information of the second nested table can include coordinates of vertex edge identification points of the second nested table to describe edge positions of the second nested table by the second coordinate information.

[0121] In this way, the containing relationship between the edge positions of the first nested table and the edge positions of the second nested table can be determined by the first coordinate information and the second coordinate information.

[0122] For example, as shown in the example of FIG. 4, it can be determined that table1 is the outermost table in the document to be processed, and the area where table1 is located can be considered as the table area in the document to be processed. According to the first coordinate information of table2 and the second coordinate information of table3 output by the detection model, it can be determined that table2 and table3 are both nested tables in table1, and table2 is the outer table of table3.

[0123] In this example, it can be determined that the first nested table is the outer table of the second nested table, that is, in this example, the table located in the outer layer of the two layers of nested tables can be referred to as the first nested table, and the table located in the inner layer can be referred to as the second nested table.

[0124] In addition, in some examples, the structure of the table can also be corrected, for example, the area where the corresponding inner nested table is located can be corrected according to the structure of the outer table to obtain an accurate and complete nested table structure.

[0125] For example, the second detection model outputs coordinate information of vertices of the first nested table and the initial second nested table respectively, and determines that the initial second nested table has an edge located outside the first nested table based on the coordinate information, and the distance between the edge and an edge of the first nested table is less than a specified distance threshold, it can be considered that the edge needs to be corrected, so that the edge can be moved to the first nested table. In this way, the corrected initial second nested table can be completely contained in the first nested table, and the content of the corrected initial second nested table is not affected. In this way, the corrected initial second nested table can be used as the second nested table, and subsequent related operations can be performed.

[0126] In step 203, the content of each nested table in the at least one nested table is identified by a table content identification model.

[0127] In the embodiments of the present application, after the at least one nested table in the to-be-processed document is detected by the detection model, the content of each nested table can be accurately identified by the table content identification model, so that the accurate result of the content of each nested table can be obtained from the to-be-processed document containing complex structures such as nested tables.

[0128] The table content identification model can identify the content of each nested table in the region where each nested table is located.

[0129] The specific structure of the table content identification model is not limited herein. For example, the table content identification model can include one or a combination of optical character recognition (OCR) model, convolutional neural network, and transformer model.

[0130] In actual applications, after the region where each nested table is located is determined, the table content identification model can be used to identify the region where each nested table is located in the to-be-processed document, so as to extract the content information of each nested table from the region where each nested table is located.

[0131] For example, in some embodiments, the table content identification model includes a third detection model and a text detection model.

[0132] The above step 203 includes:

[0133] The third detection model is used to detect the structure information of each nested table, and the structure information is used to describe at least one sub-region in the nested table.

[0134] The text detection model is used to identify the content of each sub-region in the at least one sub-region.

[0135] In the embodiments of the present application, the third detection model can be used to identify the structural information of the nested table, for example, to identify the text box in the nested table.

[0136] In this way, at least one sub-region in the nested table can be detected by the third detection model, for example, a sub-region can be a text box in the nested table. For example, the third detection model can be a yolo model or a dino model.

[0137] Then, the content of each sub-region in the nested table can be detected by the text detection model, specifically, the text information in each sub-region can be extracted. In this way, the content of each sub-region in the nested table can be combined to obtain the complete content of the nested table. For example, the text detection model can be an OCR model.

[0138] As can be seen, in the embodiments of the present application, the structural information of the nested table can be accurately detected by the third detection model to identify various text boxes and other structures in the nested table, so as to split the nested table into at least one sub-region with simple structure. Then, the text detection model is used to detect the text of the sub-region with simple structure to avoid the interference of the border and other complex structures of the nested table on the identification of the table content, so as to accurately detect the content of the sub-region, and thus accurately obtain the complete information in the nested table through the content of the multiple sub-regions.

[0139] There are various ways to detect the region where each nested table is located.

[0140] For example, in one example, the entire region where the nested table is located can be detected by the table content identification model.

[0141] For example, for the innermost nested table, the entire region where the nested table is located can be detected by the table content identification model.

[0142] In another example, the region where the nested table is located can be processed and then detected by the table content identification model.

[0143] Specifically, in some embodiments, the at least one layer of nested table includes a first nested table and a second nested table, and the first nested table is an outer table of the second nested table.

[0144] Step 203 includes:

[0145] The second region in the first nested table is covered to obtain a covered first nested table, and the second region is a region in the first nested table including the second nested table.

[0146] The content of the first nested table is identified through the table content identification model.

[0147] The content of the second nested table is identified through the table content identification model.

[0148] The content of the first nested table is obtained according to the content of the first nested table after covering and the content of the second nested table.

[0149] In the embodiments of the present application, taking two layers of adjacent nested tables as an example, the identification manner of the content of the multi-layer nested table is exemplarily introduced.

[0150] The first nested table is an outer table of the second nested table. In this example, for the convenience of description, it can be considered that there is no other layer of nested table between the first nested table and the second nested table. Moreover, the number of the second nested table can be one or more, and when the number of the second nested table is more than one, the regions where the plurality of second nested tables are located do not overlap.

[0151] For the first nested table, the second region where the second nested table is located in the first nested table can be covered, and then the content of the first nested table after covering is identified through the table content identification model.

[0152] The manner of covering the second region in the first nested table is not limited herein. For example, a solid color image consistent with the size of the second region can be overlaid on the second region. For example, as shown in the example of FIG. 5, a gray image is overlaid on the second region to cover the information in the second region.

[0153] In this way, the complex structure in the first nested table due to the second nested table can be covered, so that the structure form in the first nested table after covering is relatively simple and clear, which facilitates the table content identification model to identify the content of the first nested table after covering.

[0154] The manner in which the table content identification model identifies the content of the first nested table after covering can refer to the related introduction of the table content identification model described above. For example, the content of the first nested table after covering can be identified through the third detection model and the text detection model in the table content identification model.

[0155] For the second nested table, as shown in the example of FIG. 5, if the second nested table does not include a nested table, the content of the second nested table can be directly identified through the table content identification model.

[0156] Then, as shown in the example of FIG. 5, the content of the first nested table after covering and the content of the second nested table can be combined to obtain the complete content of the first nested table.

[0157] If the second nested table also includes a nested table (referred to as a third nested table), the area in the second nested table where the third nested table is located can be covered, and the content of the covered second nested table can be identified by the table content identification model. In this way, the content of the innermost nested table of the to-be-processed document can be identified by the table content identification model.

[0158] As can be seen, when there are multiple nested tables in the to-be-processed document, the content of the innermost nested table and the content of the covered table corresponding to the other nested tables except the innermost nested table can be obtained by referring to the manner of identifying the content of the first nested table and the second nested table, and by the table content identification model. Then, the content of the innermost nested table and the content of the covered table corresponding to the other nested tables can be combined according to the positions of the nested tables from inside to outside, so as to obtain the complete content of each nested table.

[0159] Based on any of the above embodiments, the content of each nested table in the at least one nested table can be identified by the table content identification model.

[0160] In addition, in some examples, the content of the table region in the to-be-processed document can also be obtained according to the content of each nested table, the table region including the outermost table of the to-be-processed document.

[0161] For example, in some embodiments, the method further includes:

[0162] covering a first area in the table region to obtain a covered outermost table, the first area being an area in the table region including the at least one nested table;

[0163] identifying the content of the covered outermost table by the table content identification model;

[0164] obtaining the content of the table region according to the content of the covered outermost table and the content of the at least one nested table.

[0165] The inner complex structure in the outermost table due to the nested table can be covered, so that the structure in the covered outermost table is relatively simple and clear, and the table content identification model can identify the content of the covered outermost table, thereby obtaining the complete and accurate content of the table region according to the content of the covered outermost table and the content of the at least one nested table.

[0166] It can be seen that in this example, after the table region in the to-be-processed document is identified by the first detection model, the position of the nested table in the table region can be accurately identified by the second detection model, so that the content of the table region containing the nested table in the document can be identified by the table content identification model according to the position of the nested table, thereby accurately restoring the content of the table region. It can be seen that when the document includes a relatively complex table structure such as a nested table, the content of the table can still be accurately identified by combining the first detection model, the second detection model, and the table content identification model.

[0167] For example, as shown in the example of FIG. 3, after the position of the outermost table and each nested table in the to-be-processed document is determined by the second detection model, the covered outermost table can be obtained, and the content of the covered outermost table can be identified by the table content identification model. In addition, the content of each nested table can also be identified by the table content identification model, so as to obtain the content of the table region by combining the content of the covered outermost table and the content of at least one nested table.

[0168] In the embodiments of the present application, the position of the nested table in the document can be identified by the detection model, so that the content of the nested table in the document can be identified by the table content identification model according to the position of the nested table and other features, thereby accurately restoring the content of the nested table. It can be seen that in the embodiments of the present application, when the document includes a relatively complex table structure such as a nested table, the content of the nested table can still be accurately identified by combining the detection model and the table content identification model, thereby improving the identification accuracy of the table content in the document including a complex structure such as a nested table.

[0169] In any of the above embodiments of the present application, the detection model (for example, including the first detection model and the second detection model) and the table content identification model are involved.

[0170] The detection model and the table content identification model are both trained artificial intelligence models. The detection model and / or the table content identification model can be obtained by training by a computing device cluster executing the embodiments of the present application, or can be obtained by training by other devices and then deployed to the computing device cluster.

[0171] The training process of the first detection model, the second detection model, and the table content identification model will be exemplarily introduced below by taking the detection model including the first detection model and the second detection model as an example.

[0172] In the embodiments of the present application, the first detection model to be trained, the second detection model to be trained, and the table content identification model to be trained can be trained by a data set, respectively.

[0173] In the example shown in FIG. 6, the data set can include, but is not limited to, one or more of the following data:

[0174] 1) Business data

[0175] The business data can include historical data collected from the business scenarios of the user, for example, can be collected from the historical documents of the user.

[0176] The business data can include documents containing nested tables, and can also include documents not containing nested tables. The documents not containing nested tables can be documents not containing tables, or documents containing only one layer of tables.

[0177] 2) Open source data

[0178] The open source data is data obtained from publicly available resources. The open source data can include documents containing nested tables, and can also include documents not containing nested tables.

[0179] 3) Generated data

[0180] The generated data can be data additionally collected according to the analysis results of the bad cases in the training process. For example, the generated data can include one or more of the following: difficult examples, documents containing nested tables in different forms, small sample table data, etc.

[0181] 4) Output data of the trained model

[0182] Since the data processing processes of the detection models (for example, including the first detection model and the second detection model) and the table content recognition model are serial, the output data of the detection model obtained after training can be used as the input of the table content recognition model.

[0183] Based on this, the output data of the trained model can be collected as the training data of the next model of the trained model.

[0184] For example, the first detection model to be trained can be trained first, and the first detection model can be obtained after training.

[0185] The output data of the first detection model obtained after training can be layout analysis result data, which can be used as the training data of the second detection model to be trained.

[0186] Similarly, the second detection model to be trained can also be trained first, and the second detection model can be obtained after training.

[0187] The output data of the second detection model obtained after the training can be nested table detection data, which can be used as training data of the table content recognition model to be trained.

[0188] Any of the training data described above can be label data, that is, any of the training data can correspond to a label.

[0189] In the training of different models, the content of the training data and the labels used can have various cases.

[0190] For example, the training data used to train the first detection model to be trained can be a document or a picture converted from the document, and the label of the training data can describe the table region in the corresponding training data. The training data used to train the second detection model to be trained can be a table region cropped from the training data used to train the first detection model to be trained, and the label of the training data can describe the position of the nested table in the corresponding training data. The training data used to train the table content recognition model to be trained can be a table with the inner nested table covered or the innermost table, and the label of the training data can describe the content of the table that is not covered in the corresponding training data.

[0191] After obtaining the data set, the first detection model to be trained, the second detection model to be trained, and the table content recognition model to be trained can be trained through multiple iterations based on a loss function using backpropagation and the like.

[0192] The first detection model to be trained, the second detection model to be trained, and the table content recognition model to be trained can be jointly trained, and the first detection model, the second detection model, and the table content recognition model can be obtained after the joint training.

[0193] Alternatively, the first detection model to be trained, the second detection model to be trained, and the table content recognition model to be trained can also be trained separately.

[0194] For example, the first detection model to be trained can be trained first, and the first detection model can be obtained after the training. Then, the second detection model to be trained can be trained according to the output data of the first detection model and / or the training data in the data set, and the second detection model can be obtained after the training. Next, the table content recognition model to be trained can be trained according to the output data of the second detection model and / or the training data in the data set, and the table content recognition model can be obtained after the training.

[0195] In this example, the loss functions used in the training processes of different models can be the same, for example, can be a cross-entropy function, or the loss functions used can be different, which is not limited here.

[0196] Based on any of the above examples, the first detection model to be trained, the second detection model to be trained, and the table content recognition model to be trained can be trained, and after training, the first detection model, the second detection model, and the table content recognition model are obtained, and the first detection model, the second detection model, and the table content recognition model can be deployed to the computing device cluster.

[0197] It can be seen that in the training process, the plurality of models respectively learn different detection capabilities and information processing capabilities, which can make the detection capabilities and information processing capabilities learned by each model strong. For example, the first detection model can reduce the detection error of the table region in the document containing a complex table structure such as a nested table, and the second detection model can well adapt to various forms of nested tables by learning, accurately recognize one or more layers of nested tables, provide an accurate data basis for the subsequent table content recognition model, and reduce the recognition difficulty of the table content recognition model, so that the table content recognition model can accurately and efficiently recognize the table content.

[0198] The above describes the data processing method provided by the embodiments of the present application from multiple aspects. Next, the data processing apparatus provided by the embodiments of the present application is described in combination with the drawings.

[0199] As shown in FIG. 7, the data processing apparatus 70 provided by the embodiments of the present application includes:

[0200] The acquisition module 701 is configured to acquire a document to be processed.

[0201] The processing module 702 is configured to:

[0202] detect, by a detection model, a position of each layer of nested tables in the at least one layer of nested tables included in the document to be processed, the at least one layer of nested tables being located in an outermost table included in the document to be processed;

[0203] recognize, by a table content recognition model, content of each layer of nested tables in the at least one layer of nested tables.

[0204] Optionally, the detection model includes a first detection model and a second detection model.

[0205] The processing module 702 is configured to:

[0206] detect, by the first detection model, a table region in the document to be processed, the table region including an outermost table;

[0207] detect, by the second detection model, a position of each layer of nested tables in the at least one layer of nested tables included in the table region.

[0208] Optionally, the processing module 702 is further configured to:

[0209] obtaining the content of the table region according to the content of each layer of nested tables.

[0210] Optionally, the processing module 702 is configured to:

[0211] covering a first region in the table region to obtain a covered outermost table, the first region being a region in the table region including at least one layer of nested tables;

[0212] recognizing the content of the covered outermost table by using the table content recognition model;

[0213] obtaining the content of the table region according to the content of the covered outermost table and the content of the at least one layer of nested tables.

[0214] Optionally, the at least one layer of nested tables includes a first nested table and a second nested table, and the first nested table is an outer table of the second nested table.

[0215] The processing module 702 is configured to:

[0216] covering a second region in the first nested table to obtain a covered first nested table, the second region being a region in the first nested table including the second nested table;

[0217] recognizing the content of the covered first nested table by using the table content recognition model;

[0218] recognizing the content of the second nested table by using the table content recognition model;

[0219] obtaining the content of the first nested table according to the content of the covered first nested table and the content of the second nested table.

[0220] Optionally, the processing module 702 is further configured to:

[0221] detecting the to-be-processed document by using the detection model to output first coordinate information of the first nested table and second coordinate information of the second nested table;

[0222] determining that the first nested table is an outer table of the second nested table according to the first coordinate information and the second coordinate information.

[0223] Optionally, the table content recognition model includes a third detection model and a text detection model.

[0224] The processing module 702 is configured to:

[0225] detecting, by using the third detection model, structure information of each layer of nested tables, the structure information being used to describe at least one sub-region in the nested table;

[0226] The content of each of the at least one sub-region is identified by the text detection model.

[0227] Optionally, the processing module 702 is configured to:

[0228] convert the to-be-processed document into data in the target format;

[0229] detect, by the detection model, a position of each of at least one layer of nested tables included in the data in the target format.

[0230] The processing module and the obtaining module can be implemented by software or by hardware. For example, the implementation of the processing module is described below. Similarly, the implementation of the obtaining module can refer to the implementation of the processing module.

[0231] As an example of a software functional unit, the processing module can include code running on a computing instance. The computing instance can include at least one of a physical host (computing device), a virtual machine, and a container. Further, the computing instance can be one or more. For example, the processing module can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers running the code can be distributed in the same region, or in different regions. Further, the multiple hosts / virtual machines / containers running the code can be distributed in the same availability zone (AZ), or in different AZs. Each AZ includes one data center or multiple data centers in close geographical proximity. Generally, one region includes multiple AZs.

[0232] Similarly, the multiple hosts / virtual machines / containers running the code can be distributed in the same virtual private cloud (VPC), or in multiple VPCs. Generally, one VPC is set in one region, and communication between two VPCs in the same region, or between VPCs in different regions, needs to be set in each VPC to set a communication gateway, and the interconnection between VPCs is realized through the communication gateway.

[0233] As an example of a hardware functional unit, the processing module can include at least one computing device, such as a server or the like. Alternatively, the processing module can also be a device implemented by a central processing unit (CPU), or implemented by an application-specific integrated circuit (ASIC), or implemented by a programmable logic device (PLD), and the like. Among them, the PLD can be implemented by a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a neural network processing unit (NPU), a system on chip (SoC), an offload card, an acceleration card, or any combination thereof.

[0234] The plurality of computing devices included in the processing module can be distributed in the same region, or distributed in different regions. The plurality of computing devices included in the processing module can be distributed in the same AZ, or distributed in different AZs. Similarly, the plurality of computing devices included in the processing module can be distributed in the same VPC, or distributed in multiple VPCs. Among them, the plurality of computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, DPUs, NPUs, SoCs, offload cards, acceleration cards, and the like.

[0235] It should be noted that in other embodiments, the processing module can be used to perform any step of the data processing method, and the acquisition module can be used to perform any step of the data processing method. The steps responsible for the implementation of the processing module and the acquisition module can be specified as needed, and the entire function of the data processing device can be achieved by implementing different steps of the data processing method through the processing module and the acquisition module respectively.

[0236] The present application also provides a chip system, which includes a processor and a power supply circuit, the power supply circuit is used to supply power to the processor, and the processor is used to execute the operation steps corresponding to the data processing method. For brevity, it will not be repeated here. Among them, the processor can be implemented by a GPU, or implemented by a DPU, NPU, XPU, SoC, offload card, acceleration card, and the like.

[0237] The present application also provides a computing device 80. As shown in FIG. 8, the computing device 80 includes a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate with each other through the bus 802. The computing device 80 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 80.

[0238] The bus 802 can be a peripheral component interconnect Express (PCIe) bus or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. Among them, the unified bus is also called a smart bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is used in FIG. 8, but it does not mean that there is only one bus or one type of bus. The bus 804 can include a path for transmitting information between various components (e.g., the memory 806, the processor 804, the communication interface 808) of the computing device 80. Among them, the unified bus can also be called a smart bus.

[0239] The processor 804 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), an ASIC, an FPGA, a CPLD, an NPU, a SoC, an offload card, an acceleration card, etc. computing device.

[0240] The memory 806 can include volatile memory (volatile memory), such as random access memory (RAM). The memory 806 can also include non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, a hard disk drive (HDD) or a solid state drive (SSD). In addition, the memory 806 can also be implemented by storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0241] It is worth noting that the same type of storage medium can be configured in the same computing device to implement the memory 806 function, and two or more types of storage media can also be configured to implement the function of the memory 806, which is not limited in the present application.

[0242] The memory 806 stores executable program code, and the processor 804 executes the executable program code to implement the functions of the aforementioned processing modules and acquisition modules, respectively, so as to implement the data processing method. That is, the memory 806 has instructions for executing the data processing method.

[0243] The communication interface 803 uses a transceiver module such as but not limited to a network interface card and a transceiver to implement communication between the computing device 80 and other devices or communication networks.

[0244] As a possible implementation, the computing device 80 can also include a chip system including a processor and a power supply circuit for supplying power to the processor, and the processor is used to execute the operation steps corresponding to the data processing method. For the sake of brevity, it will not be repeated here. Among them, the processor can be implemented by GPU, and can also be implemented by DPU, NPU, XPU, SoC, offload card, acceleration card and other computing devices or AI chips.

[0245] As a possible implementation, the computing device 80 can include multiple types of processors 804, that is, the computing device 80 is a heterogeneous device, for example, the computing device 80 includes CPU and GPU, and at least one of the processors 804 can execute the operation steps corresponding to the data processing method. For the sake of brevity, it will not be repeated here.

[0246] The embodiments of the present application also provide a computing device cluster. The computing device cluster comprises at least one computing device. The computing device can be a server, for example, a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a notebook computer, or a smart phone.

[0247] As shown in FIG. 9, the computing device cluster comprises at least one computing device 80. The same instructions for performing the data processing method can be stored in the memory 806 of one or more computing devices 80 in the computing device cluster.

[0248] In some possible implementation manners, partial instructions for performing the data processing method can also be respectively stored in the memory 806 of one or more computing devices 80 in the computing device cluster. In other words, the combination of one or more computing devices 80 can collectively execute the instructions for performing the data processing method.

[0249] It should be noted that the memory 806 in different computing devices 80 in the computing device cluster can store different instructions, respectively used for performing partial functions of the data processing apparatus. That is, the instructions stored in the memory 806 in different computing devices 80 can implement the functions of one or more of the processing module and the obtaining module.

[0250] In some possible implementation manners, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network or a local area network, etc. FIG. 10 shows a possible implementation manner. As shown in FIG. 10, two computing devices 80A and 80B are connected through a network. Specifically, the communication interface in each computing device is connected to the network. In this type of possible implementation manner, the memory 806 in the computing device 80A stores instructions for performing the functions of the processing module. Meanwhile, the memory 806 in the computing device 80B stores instructions for performing the functions of the obtaining module.

[0251] It should be understood that the functions of the computing device 80A shown in FIG. 10 can also be completed by multiple computing devices 80. Similarly, the functions of the computing device 80B can also be completed by multiple computing devices 80.

[0252] The embodiments of the present application also provide another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to the connection manners of the computing device clusters in FIG. 9 and FIG. 10. The difference is that the same instructions for performing the data processing method can be stored in the memory 806 of one or more computing devices 80 in the computing device cluster.

[0253] In some possible implementations, partial instructions for performing the data processing method can also be respectively stored in the memory 806 of one or more computing devices 80 in the computing device cluster. In other words, the combination of one or more computing devices 80 can collectively execute the instructions for performing the data processing method.

[0254] The embodiments of the present application also provide a computer program product containing instructions. The computer program product can be a software or program product containing instructions, which can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device is caused to perform the data processing method, or the data processing method.

[0255] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium can be any available medium that the computing device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk), etc. The computer readable storage medium contains instructions, which instruct the computing device to perform the data processing method, or instruct the computing device to perform the data processing method.

[0256] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Get documents to be processed; Detecting, by means of a detection model, a position of each nested table in at least one layer of nested tables included in the document to be processed, wherein the at least one layer of nested tables is located in an outermost layer of tables included in the document to be processed; The content of each nested table in the at least one nested table is identified using a table content identification model.

2. The method according to claim 1, characterized in that The detection model includes a first detection model and a second detection model; The detecting model detects the position of each nested table in at least one nested table included in the to-be-processed document, including: Detecting a table area in the document to be processed using the first detection model, wherein the table area includes the outermost table; The position of each layer of nested tables in the at least one layer of nested tables included in the table area is detected by using the second detection model.

3. The method according to claim 2, characterized in that After identifying the content of each nested table in the at least one nested table using the table content identification model, the method further includes: The content of the table area is obtained according to the content of each nested table layer.

4. The method according to claim 3, characterized in that The method further comprises: Covering a first area in the table area to obtain an outermost table after covering, wherein the first area is an area in the table area including the at least one nested table; Identifying the contents of the masked outermost table using the table content recognition model; The obtaining of the content of the table area according to the content of each nested table layer includes: The content of the table area is obtained according to the content of the outermost table after masking and the content of the at least one nested table.

5. The method according to any one of claims 1 to 4, characterized in that The at least one nested table includes a first nested table and a second nested table, wherein the first nested table is an outer table of the second nested table; The identifying the content of each nested table in the at least one nested table by using a table content identification model includes: Masking the second area in the first nested table to obtain the masked first nested table, where the second area is the area in the first nested table that includes the second nested table; Identifying the content of the masked first nested table using the table content recognition model; Identifying the content of the second nested table using the table content identification model; The content of the first nested table is obtained according to the content of the masked first nested table and the content of the second nested table.

6. The method according to claim 5, characterized in that Before identifying the content of each nested table in the at least one nested table using the table content identification model, the method further includes: Detecting the document to be processed by using the detection model to output first coordinate information of the first nested table and second coordinate information of the second nested table; According to the first coordinate information and the second coordinate information, the first nested table is determined to be an outer table of the second nested table.

7. The method according to any one of claims 1 to 6, characterized in that The table content recognition model includes a third detection model and a text detection model; The identifying the content of each nested table in the at least one nested table by using a table content identification model includes: detecting structural information of each layer of nested tables using the third detection model, where the structural information is used to describe at least one sub-region in the nested tables; The content of each sub-region in the at least one sub-region is identified by using the text detection model.

8. The method according to any one of claims 1 to 7, characterized in that The detecting model detects the position of each nested table in at least one nested table included in the to-be-processed document, including: Converting the document to be processed into data in a target format; The position of each nested table in at least one layer of nested tables included in the data in the target format is detected by using the detection model.

9. A data processing device, characterized in that: include: The acquisition module is used to obtain documents to be processed; Processing module for: Detecting, by means of a detection model, a position of each nested table in at least one layer of nested tables included in the document to be processed, wherein the at least one layer of nested tables is located in an outermost layer of tables included in the document to be processed; The content of each nested table in the at least one nested table is identified using a table content identification model.

10. The device according to claim 9, characterized in that The detection model includes a first detection model and a second detection model; The processing module is used for: Detecting a table area in the document to be processed using the first detection model, wherein the table area includes the outermost table; The position of each layer of nested tables in the at least one layer of nested tables included in the table area is detected by using the second detection model.

11. The device according to claim 10, characterized in that The processing module is further configured to: The content of the table area is obtained according to the content of each nested table layer.

12. The device according to claim 11, characterized in that The processing module is used for: Covering a first area in the table area to obtain an outermost table after covering, wherein the first area is an area in the table area including the at least one nested table; Identifying the contents of the masked outermost table using the table content recognition model; The content of the table area is obtained according to the content of the outermost table after masking and the content of the at least one nested table.

13. The device according to any one of claims 9 to 12, characterized in that The at least one nested table includes a first nested table and a second nested table, wherein the first nested table is an outer table of the second nested table; The processing module is used for: Masking the second area in the first nested table to obtain the masked first nested table, where the second area is the area in the first nested table that includes the second nested table; Identifying the content of the masked first nested table using the table content recognition model; Identifying the content of the second nested table using the table content identification model; The content of the first nested table is obtained according to the content of the masked first nested table and the content of the second nested table.

14. The device according to claim 13, characterized in that The processing module is further configured to: Detecting the document to be processed by using the detection model to output first coordinate information of the first nested table and second coordinate information of the second nested table; According to the first coordinate information and the second coordinate information, the first nested table is determined to be an outer table of the second nested table.

15. The device according to any one of claims 9 to 14, characterized in that The table content recognition model includes a third detection model and a text detection model; The processing module is used for: detecting structural information of each layer of nested tables using the third detection model, where the structural information is used to describe at least one sub-region in the nested tables; The content of each sub-region in the at least one sub-region is identified by using the text detection model.

16. The device according to any one of claims 9 to 15, characterized in that The processing module is used for: Converting the document to be processed into data in a target format; The position of each nested table in at least one layer of nested tables included in the data in the target format is detected by using the detection model.

17. A computing device, characterized in that The computing device includes a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the computing device executes the method according to any one of claims 1 to 8.

18. A computing device cluster, characterized in that: comprising at least one computing device, the at least one computing device comprising a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the computing device cluster executes the method according to any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program runs on a processor, the processor is enabled to perform the method according to any one of claims 1 to 8.

20. A computer program product comprising instructions, characterized in that When the instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images