Method and device for acquiring recognition model, method and device for recognizing table structure and medium

Abstract

The invention discloses a method and device for obtaining a recognition model, a method and device for recognizing a table structure and a medium, and belongs to the technical field of artificial intelligence. In the method for obtaining the recognition model, a first training sample is obtained, the first training sample comprises a first cross-page table, the first cross-page table is obtained by combining at least two first areas, and the at least two first areas are obtained by conducting layout analysis on a first document. And then, according to a recognition model obtained by training the first training sample, the recognition model is used for carrying out structure recognition on the input table to obtain first structure data, and the first structure data is used for representing the structure of the input table. In the application, the first training sample comprises the first cross-page table obtained based on the layout analysis process, and has high learning difficulty, so that the recognition model can learn the characteristics of the cross-page table, thereby ensuring that the recognition model obtained by training has high recognition ability for the cross-page table, and can output accurate first structure data.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for obtaining a recognition model, a method, device, and medium for recognizing a table structure. Background Art

[0002] With the development of artificial intelligence technology, the application of artificial intelligence models is becoming more and more widespread. Recognition models are one of them. Recognition models are used to output structured data based on the input table. Structural data is used to represent the structure of the table.

[0003] In related technologies, single-page tables are used as training samples for recognition models, enabling the trained recognition models to output structural data of single-page tables. However, this type of training sample is relatively limited, affecting the recognition capabilities of the recognition model and resulting in low accuracy of the structural data output by the recognition model. Summary of the Invention

[0004] This application provides a method for obtaining a recognition model, a method for recognizing a table structure, an apparatus, and a medium to improve the problem of low accuracy of structural data output by the recognition model. The technical solution provided by this application includes the following aspects.

[0005] In a first aspect, a method for obtaining a recognition model is provided. In this method, a first training sample is obtained, and a recognition model is trained based on the first training sample. The first training sample includes a first spread-page table, which is obtained by merging at least two first regions, each of which is obtained by performing layout analysis on a first document. The recognition model trained based on the first training sample is used to perform structural recognition on an input table to obtain first structural data, where the first structural data is used to characterize the structure of the input table.

[0006] In the present application, since the first training sample includes at least the first cross-page table, the recognition model can learn the features of the first cross-page table during the process of training the recognition model based on the first training sample. Therefore, the recognition model can better recognize the input table when the input table is a cross-page table, fully utilize the context information between different regions in the input table, so that the output first structure data has a higher accuracy. Moreover, the structure of the input table can be completely restored based on the first structure data without the need for additional merging and other processing, thereby improving recognition efficiency. In addition, since the first cross-page table is obtained by merging at least two first regions, and at least two first regions are obtained through layout analysis and may contain errors, the first cross-page table may also contain errors. The first cross-page table has a higher learning difficulty for the recognition model, which can improve the recognition ability of the recognition model. Even if the input table is a table with errors, the recognition model can still better recognize the input table, so that the output first structure data has a higher accuracy.

[0007] In one possible implementation, a recognition model is obtained by training based on a first training sample, including: obtaining an initial model, inputting the first training sample into the initial model, obtaining second structural data output by the initial model, wherein the second structural data is used to characterize the structure of the first cross-page table; obtaining a second training sample based on the second structural data; and training the initial model based on the second training sample to obtain a recognition model.

[0008] Because the second structure data is output by the initial model, it can reflect the training status of the initial model. Therefore, a second training sample can be obtained accordingly based on the second structure data. This is equivalent to supplementing the first training sample with the second training sample in a targeted manner. This increases the number of training samples, which helps improve the training effect and makes the trained recognition model have stronger recognition capabilities.

[0009] In a possible implementation, obtaining a second training sample according to the second structural data includes: determining abnormal structural data in the second structural data, where the structure represented by the abnormal structural data has an abnormality; and obtaining the second training sample according to the abnormal structural data.

[0010] In the case where the structure represented by the abnormal structure data is different from the true structure, the structure represented by the abnormal structure data is considered to be abnormal. If the initial model outputs abnormal structure data, it indicates that the initial model's recognition is incorrect, and therefore a corresponding second training sample is needed to improve the initial model's recognition ability for this true structure and avoid errors when the trained recognition model subsequently recognizes input tables with this true structure.

[0011] In one possible implementation, the first training sample also includes at least one of a first single-page table, a second cross-page table, or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

[0012] In this implementation, the sample types and quantity included in the first training sample are increased, so that the recognition model can learn the features of more samples, thereby ensuring the recognition ability of the recognition model.

[0013] In a second aspect, a method for identifying a table structure is provided. In this method, a table to be identified is obtained and input into a recognition model, thereby obtaining first structural data output by the recognition model. The first structural data is used to characterize the structure of the table to be identified. The recognition model is trained based on a first training sample, the first training sample including a first spread-page table, the first spread-page table being obtained by merging at least two first regions, each of which being obtained by performing layout analysis on a first document.

[0014] In the present application, the structure of the table to be identified is identified by a recognition model. Since the recognition model is trained based on a first training sample, and since the first training sample includes at least a first cross-page table, the recognition model can learn the characteristics of the first cross-page table. In the case where the table to be identified is a cross-page table, the recognition model can better identify the table to be identified, fully utilizing the contextual information between different regions in the table to be identified, so that the output first structure data has high accuracy, and the structure of the table to be identified can be completely restored based on the first structure data without the need for additional merging or other processing, thereby improving recognition efficiency. In addition, since the first cross-page table is obtained by merging at least two first regions, and the at least two first regions are obtained through layout analysis and may contain errors, the first cross-page table may also contain errors. The first cross-page table has a higher learning difficulty for the recognition model, which can improve the recognition ability of the recognition model. Even if the table to be identified is a table with errors, the recognition model can still better identify the table to be identified, so that the output first structure data has high accuracy.

[0015] In a possible implementation, obtaining the table to be identified includes: obtaining a third document, performing layout analysis on the third document to obtain a third region; and obtaining the table to be identified based on the third region.

[0016] Since the training process of the recognition model involves the layout analysis process, the table to be recognized can also be obtained through the layout analysis process during the application of the recognition model. The recognition model has strong adaptability to such tables to be recognized, so that the process of obtaining the table to be recognized does not need to be limited to manual labeling, thereby improving the recognition efficiency while ensuring recognition accuracy.

[0017] In one possible implementation, obtaining a table to be identified based on the third area includes: when the third area is a cross-page table and the number of third areas is at least two, merging at least two third areas to obtain the table to be identified; or, when the third area is a single-page table, using the third area as the table to be identified.

[0018] Therefore, no matter the third area belongs to a cross-page table or a single-page table, the table to be identified can be obtained based on the third area, and the acquisition method is relatively flexible.

[0019] In a possible implementation, the method further includes: determining characters in a table to be recognized, the table to be recognized being in a non-editable format; and generating a table in an editable format according to the characters and the first structure data.

[0020] In this implementation, a table in a non-editable format is converted into a table in an editable format, so that the table in the editable format can be flexibly applied according to actual needs, and can be widely used in various scenarios to meet user needs.

[0021] In one possible implementation, the first training sample also includes at least one of a first single-page table, a second cross-page table, or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

[0022] Since this implementation method increases the sample types and quantity included in the first training sample, the recognition model can learn the features of more samples, thereby ensuring the recognition ability of the recognition model.

[0023] In a third aspect, a device for obtaining a recognition model is provided, the device comprising:

[0024] An acquisition module is configured to acquire a first training sample, the first training sample comprising a first cross-page table, the first cross-page table being obtained by merging at least two first regions, the at least two first regions being obtained by performing layout analysis on a first document;

[0025] The training module is used to obtain a recognition model based on the first training sample training. The recognition model is used to perform structural recognition on the input table to obtain first structural data. The first structural data is used to represent the structure of the input table.

[0026] In one possible implementation, a training module is used to obtain an initial model, input a first training sample into the initial model, obtain second structural data output by the initial model, and the second structural data is used to characterize the structure of the first cross-page table; obtain a second training sample based on the second structural data; and train the initial model based on the second training sample to obtain a recognition model.

[0027] In a possible implementation, the training module is configured to determine abnormal structural data in the second structural data, where the structure represented by the abnormal structural data is abnormal; and obtain a second training sample based on the abnormal structural data.

[0028] In one possible implementation, the first training sample also includes at least one of a first single-page table, a second cross-page table, or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

[0029] In a fourth aspect, a device for identifying a table structure is provided, the device comprising:

[0030] An acquisition module, used to obtain the table to be identified;

[0031] The recognition module is used to input the table to be recognized into the recognition model to obtain the first structural data output by the recognition model. The first structural data is used to characterize the structure of the table to be recognized. The recognition model is trained based on the first training sample. The first training sample includes a first cross-page table. The first cross-page table is obtained by merging at least two first areas. The at least two first areas are obtained by performing layout analysis on the first document.

[0032] In a possible implementation, the acquisition module is configured to acquire a third document, perform layout analysis on the third document to obtain a third region; and acquire the table to be identified based on the third region.

[0033] In one possible implementation, the acquisition module is used to merge at least two third regions to obtain a table to be identified when the third region belongs to a cross-page table and the number of third regions is at least two; or, when the third region belongs to a single-page table, use the third region as the table to be identified.

[0034] In a possible implementation, the apparatus further includes:

[0035] The generating module is used for determining characters in a table to be recognized, where the table to be recognized is in a non-editable format; and generating a table in an editable format according to the characters and the first structure data.

[0036] In one possible implementation, the first training sample also includes at least one of a first single-page table, a second cross-page table, or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

[0037] In a fifth aspect, a computing device is provided, which includes a processor and a memory, and the processor is used to execute instructions stored in the memory so that the computing device executes the method for obtaining a recognition model provided by the first aspect or any implementation of the first aspect, or executes the method for identifying a table structure provided by the second aspect or any implementation of the second aspect.

[0038] In a sixth aspect, a computing device cluster is provided, the computing device cluster including at least one computing device, each computing device including a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method for obtaining a recognition model provided by the first aspect or any implementation of the first aspect, or executes the method for identifying a table structure provided by the second aspect or any implementation of the second aspect.

[0039] In the seventh aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the method for obtaining a recognition model provided by the first aspect or any implementation of the first aspect, or executes the method for identifying a table structure provided by the second aspect or any implementation of the second aspect.

[0040] In an eighth aspect, a computer-readable storage medium is provided, which includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method for obtaining a recognition model provided by the first aspect or any implementation of the first aspect, or executes the method for identifying a table structure provided by the second aspect or any implementation of the second aspect.

[0041] In the ninth aspect, a chip is provided, comprising a processor for calling and executing instructions stored in a memory from a memory, so that a computer equipped with the chip executes the method for obtaining a recognition model provided by the first aspect or any one of the implementations of the first aspect, or implements the method for recognizing a table structure provided by the second aspect or any one of the implementations of the second aspect.

[0042] In the tenth aspect, another chip is provided, comprising: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected through an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, a computer equipped with the chip executes the method for obtaining a recognition model provided by the first aspect or any implementation of the first aspect, or implements the method for recognizing a table structure provided by the second aspect or any implementation of the second aspect.

[0043] It should be understood that the technical effects achieved by the technical solutions provided in the third to tenth aspects of this application and the corresponding possible implementation methods can be referred to the above description of the technical effects achieved by the technical solutions provided in the first and second aspects and the corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of an implementation environment provided for an embodiment of the present application;

[0045] Figure 2 A schematic diagram of another implementation environment provided for an embodiment of the present application;

[0046] Figure 3 A flowchart of a method for obtaining a recognition model provided in an embodiment of the present application;

[0047] Figure 4 A flowchart of a method for obtaining a recognition model provided in an embodiment of the present application;

[0048] Figure 5 A flowchart of a method for identifying a table structure provided in an embodiment of the present application;

[0049] Figure 6 A flowchart of a method for identifying a table structure provided in an embodiment of the present application;

[0050] Figure 7 A flowchart of another method for identifying a table structure provided in an embodiment of the present application;

[0051] Figure 8 A schematic diagram of merging a third region provided in an embodiment of the present application;

[0052] Figure 9 A schematic diagram of a process for obtaining a recognition model and a recognition table structure provided in an embodiment of the present application;

[0053] Figure 10 A schematic diagram of the structure of a device for obtaining a recognition model provided in an embodiment of the present application;

[0054] Figure 11A schematic diagram of a device for identifying a table structure provided in an embodiment of the present application;

[0055] Figure 12 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0056] Figure 13 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0057] Figure 14 A schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0059] The embodiment of the present application provides an implementation environment, such as Figure 1 As shown, the implementation environment includes at least one computing device 101. Each computing device 101 can be a server or a terminal, or a component included in a server or a terminal, such as a chip.

[0060] In exemplary embodiments, a terminal may also be referred to as user equipment (UE), user device, access terminal, user unit, user station, mobile station, mobile station (MS), remote station, remote terminal, mobile device, user terminal, terminal unit, terminal station, terminal device, user agent, or user device. Exemplarily, a terminal includes but is not limited to: a mobile phone, a personal digital assistant (PDA), a laptop computer, a tablet computer (Pad), a notebook computer, a desktop computer, a smart watch, a virtual reality (VR) terminal, an augmented reality (AR) terminal, and the like.

[0061] In the embodiment of the present application, a method for obtaining a recognition model and a method for recognizing a table structure are provided. These two methods can be applied to the same computing device 101 or to different computing devices 101. For example, see Figure 2At least one computing device 101 includes computing device A and computing device B. Computing device A is configured to execute the method for acquiring a recognition model to obtain a recognition model. Computing device B is configured to execute the method for identifying a table structure. For example, computing device B may be connected to computing device A to acquire the recognition model from computing device A, thereby executing the method for identifying a table structure using the acquired recognition model.

[0062] The present invention provides a method for obtaining a recognition model. Figure 1 or Figure 2 In the computing device shown. Figure 3 As shown, the method for obtaining the recognition model includes the following steps 301 and 302.

[0063] Step 301 : Obtain a first training sample, the first training sample including a first cross-page table, the first cross-page table being obtained by merging at least two first regions, and the at least two first regions being obtained by performing layout analysis on a first document.

[0064] The first training sample is a sample used to train a recognition model. The first training sample includes at least a first cross-page table. The first cross-page table occupies at least two pages in the first document, and each of the occupied pages includes a first region, thereby providing at least two first regions. Therefore, the first document can be subjected to layout analysis to obtain at least two first regions in the first document, and then the at least two first regions can be merged to obtain the first cross-page table.

[0065] Exemplarily, the process of performing layout analysis is based on deep learning, has high accuracy and universality, supports documents of various formats and various forms of tables in documents, so that the embodiment of the present application does not need to limit the format of the first document and the form of the first cross-page table in the first document. For example, the first document can be in an editable format or a non-editable format. Editable formats include but are not limited to formats such as editable portable document format (portable document format, PDF). Non-editable formats include but are not limited to formats such as images, scans, and non-editable PDFs. The first cross-page table may have frame lines (i.e., wired tables) or may not have frame lines (i.e., wireless tables). In the case where the first cross-page table has frame lines, the frame lines may be clear and continuous, or they may be blurred and broken. Of course, the embodiments of the present application do not limit the formats of other documents mentioned later, and the forms of cross-page tables included in other documents, and will not be repeated hereafter.

[0066] In an exemplary embodiment, the process of performing layout analysis on a first document and obtaining at least two first areas in the first document includes detecting and obtaining at least two first areas from the first document through a layout analysis model. The layout analysis model can be a model obtained through deep learning. For example, in a document including a table, the location of the table is marked as a region, a reference training sample is obtained, and a layout analysis model is obtained by training based on the reference training sample. The layout analysis model is used to output region data based on the input document, and the region data is used to indicate the region in the document. The region can be determined from the document based on the region data, and the region is the location of the table in the document. For example, the region is a rectangle, and the region data can be the coordinates of the four vertices of the rectangular region, or the coordinates of the two diagonal points of the rectangular region. The coordinate system where the coordinates are located can be set based on the document. For the case where the region is a shape other than a rectangle, examples are not given here one by one.

[0067] In some implementations, each page in a document is sequentially input into a layout analysis model, resulting in multiple sets of region data outputted by the model. Each set of region data indicates the location of a region within the page. For example, page 1 of a document is input into the layout analysis model, resulting in the first set of coordinates outputted by the model. Region 1 is then determined from page 1 based on the first set of coordinates. Page 2 of the document is input into the layout analysis model, resulting in the second set of coordinates outputted by the model. Region 2 is then determined from page 2 based on the second set of coordinates. Similarly, regions can be detected from a document using the layout analysis model.

[0068] In other embodiments, each page in a document is batch-fed into a layout analysis model, resulting in multiple sets of region data outputted by the layout analysis model. Each set of region data indicates the location of a region in the document. For example, each page in a document is batch-fed into a layout analysis model, resulting in outputs such as page number 1 and the first set of coordinates, page number 2 and the second set of coordinates, etc. Region 1 is determined from page 1 indicated by page number 1 based on page number 1 and the first set of coordinates. Region 2 is determined from page 2 indicated by page number 2 based on page number 2 and the second set of coordinates. Similarly, regions can be detected from a document using the layout analysis model.

[0069] After performing layout analysis on the document to obtain the various areas in the document, it can also be determined whether each area belongs to a cross-page table. If an area belongs to a cross-page table, the area needs to be merged with other areas to obtain a cross-page table. If an area does not belong to a cross-page table, it means that the area belongs to a single-page table. A single-page table refers to a table that only occupies one page in the document. Then, the area does not need to be merged with other areas, and the area itself is a single-page table. Exemplarily, in the process of determining whether each area belongs to a cross-page table, different areas can be merged first, and the merged areas can be input into a two-classification model. If the two-classification model outputs the first result, it means that these different areas belong to a cross-page table. If the two-classification model outputs the second result, it means that these different areas do not belong to a cross-page table, but belong to a single-page table. Alternatively, different areas can also be input into a header recognition model. The header recognition model can determine whether different areas belong to a cross-page table by analyzing the headers in the area.

[0070] Based on the above introduction to the layout analysis model, the following describes a process of detecting and obtaining at least two first regions from a first document using the layout analysis model, and a process of merging the at least two first regions to obtain a first cross-page table.

[0071] In some embodiments, if the format of the acquired first document is consistent with the reference format, the first document in the reference format is directly input into the layout analysis model. In other embodiments, if the format of the acquired first document is inconsistent with the reference format, the format of the first document is first converted into the reference format to obtain the first document in the reference format, and then the first document in the reference format is input into the layout analysis model. The embodiments of the present application do not limit the reference format, and the reference format is, for example, an image format. In the case where the reference format is an image format, the at least two first regions detected from the first document in the image format are also in the image format, and the first cross-page table obtained by subsequently merging the at least two first regions is also in the image format.

[0072] For example, in an embodiment of the present application, each page in the first document may be input into the layout analysis model sequentially, or each page in the first document may be input into the layout analysis model in batches, which is not limited in this embodiment of the present application. Regardless of the input method, the layout analysis model can output region data, which is used to indicate the location of the region in the first document. Then, the region can be determined in the first document based on the region data.

[0073] In some embodiments, the first document obtained only includes a cross-page table, and it is not necessary to determine whether the area in the first document belongs to the cross-page table according to the above description, but it is assumed that the areas in the first document all belong to the cross-page table and are all first areas. In all the first areas, different first areas belonging to the same cross-page table are merged to obtain the first cross-page table. In other embodiments, the first document obtained includes both a cross-page table and a single-page table, and it is necessary to determine whether the area in the first document belongs to the cross-page table according to the above description, and the area belonging to the cross-page table is the first area. In all the first areas, different first areas belonging to the same cross-page table are merged to obtain the first cross-page table. Optionally, the process of merging at least two first areas includes splicing at least two first areas in the order of the pages where the at least two first areas are located, so as to obtain the first cross-page table.

[0074] In an exemplary embodiment, in addition to the first cross-page table described above, the first training sample may also include at least one training sample of a first single-page table, a second cross-page table, or a second single-page table. Methods for obtaining these three training samples are described below.

[0075] A first single-page table includes a second region, where the second region is obtained by performing layout analysis on a second document. The first single-page table occupies only one page in the second document, and the occupied page includes the second region. Therefore, after obtaining the second document, layout analysis can be performed on the second document to obtain the second region, and the second region can be used as the first single-page table. For example, performing layout analysis on the second document to obtain the second region includes detecting the second region from the second document using a layout analysis model.

[0076] For example, if the second document is in a reference format, the second document in the reference format is directly input into the layout analysis model. If the second document is in a format other than the reference format, the second document is converted to the reference format to obtain the second document in the reference format, and the second document in the reference format is input into the layout analysis model. For example, the reference format is an image format, and the second region detected from the second document in the image format is also in image format, and the first single-page table is also in image format.

[0077] In an embodiment of the present application, each page in the second document can be sequentially input or batch-input into a layout analysis model to obtain regional data output by the layout analysis model. The regional data is used to indicate the position of the region in the second document, and the second region is determined in the second document based on the regional data. In some embodiments, the acquired second document only includes a single-page table, then it is assumed that the regions in the second document all belong to the single-page table and are all the second region, and a first single-page table is obtained. In other embodiments, the acquired second document includes both a cross-page table and a single-page table, then it is necessary to determine whether the region in the first document belongs to the cross-page table according to the above description, and the region that does not belong to the cross-page table is the second region, thereby obtaining the first single-page table.

[0078] Exemplarily, the second document and the above-mentioned first document can be different or the same document. For example, when a document only includes a cross-page table, the document can only be used as a first document and cannot be used as a second document, and the second document is different from the first document. For another example, when a document includes both a cross-page table and a single-page table, the document can be used as both a first document and a second document, and the second document is the same as the first document. Optionally, the first document and the second document can be open source documents or non-open source documents, and the non-open source document can be, for example, a business document.

[0079] The second spread table is obtained by annotation, and the second spread table can be in image format. For example, the document including the spread table is manually annotated to obtain the second spread table. For example, each region in the document including the spread table is manually annotated, and regions belonging to the same spread table are merged to obtain the second spread table.

[0080] The second single-page table is obtained by annotation. The second single-page table can be in image format. For example, the document including the single-page table is manually annotated to obtain the second single-page table. For example, each region in the document including the single-page table is manually annotated, and each region is used as a second single-page table.

[0081] Exemplarily, the document including the cross-page table may be different from or the same as the first document mentioned above, and the document including the single-page table may be different from or the same as the second document mentioned above, and the embodiments of the present application do not limit this. Optionally, although the process of performing layout analysis already has a high accuracy, the area obtained by performing layout analysis may still have errors, so that the first cross-page table and the first single-page table also have errors. For example, compared to the second cross-page table and the second single-page table obtained by marking, the accuracy of the first cross-page table and the first single-page table is lower. However, for the training process, the first cross-page table and the first single-page table are training samples with a higher learning difficulty. Training with such training samples can enable the recognition model obtained by subsequent training to have a stronger table recognition ability, and will not affect the accuracy of recognition due to errors in the table.

[0082] Step 302: A recognition model is obtained by training based on the first training sample. The recognition model is used to perform structural recognition on the input table to obtain first structural data. The first structural data is used to represent the structure of the input table.

[0083] Training according to the first training sample can enable the recognition model to learn the characteristics of the first training sample, so that the trained recognition model has the ability to perform structural recognition on the input table and obtain the first structural data. The first structural data is used to characterize the structure of the input table, and the input table structure can be restored according to the first structural data. Exemplarily, the input table includes at least one grid, and the first structural data is used to characterize the positioning information and size of each grid. For any grid, the positioning information is used to indicate the position of the grid in the input table, for example, the grid is the first grid in the upper left corner of the input table. The size is used to indicate the size of the grid. Optionally, the size of a grid is represented by the cell array included in the grid. For example, when the size of a grid is expressed as 2×5, it means that the grid includes 2 rows and 5 columns of cells.

[0084] Since the first training sample includes at least the first cross-page table, the recognition model can learn the characteristics of the first cross-page table. For example, the first cross-page table has a large aspect ratio. The aspect ratio refers to the ratio of the vertical size of the first cross-page table to the horizontal size of the first cross-page table. The aspect ratio is also called the length-to-width ratio. Since the recognition model can learn the characteristics of the first cross-page table, when the input table is a cross-page table, although the cross-page table has a large aspect ratio, the recognition model can still better recognize the input table, so that the output first structural data has a higher accuracy. Since cross-page tables are widely used in multiple fields such as finance, the recognition model provided in the embodiment of the present application can be applied to multiple fields where cross-page tables are widely used, and is more practical.

[0085] Because the first spread-page table is obtained by merging at least two first regions, and the at least two first regions are obtained by performing layout analysis, the at least two first regions may contain errors, which in turn causes the first spread-page table to also contain errors. This creates a high learning difficulty for the recognition model, thereby improving the recognition capability of the recognition model. Thus, even if the input table contains errors, the recognition model can still effectively recognize the input table, resulting in a high degree of accuracy in the output first structure data.

[0086] This shows that the recognition model is highly adaptable to complex situations, such as input tables with large aspect ratios (i.e., tables that span multiple pages) and errors in the input tables. It can accurately identify these complex situations, fully utilizing the contextual information between the different regions included in the first spanning-page table to identify and obtain accurate first structural data. Because the first structural data is used to characterize the structure of the input table, the structure of the input table can be directly and completely restored based on the first structural data, eliminating the need for additional processing steps to restore the structure of the input table. This reduces the difficulty and workload of the work and improves recognition efficiency.

[0087] In the related art, a single-page table is obtained by manual annotation, and the single-page table is used as a training sample to train a recognition model. This recognition model can only recognize accurate structural data when the input table is a single-page table without errors. If the input table has errors, the recognition model will have a poor recognition effect on the table with errors because the training sample with errors is not learned during the training process, and the output structural data is not accurate enough. If the input table is a cross-page table with a large aspect ratio, the recognition model will have a poor recognition effect on the cross-page table because the training sample does not learn for the training sample belonging to the cross-page table, and the output structural data is not accurate enough. Even if the recognition model can recognize the regions in the cross-page table one by one (or recognize the cross-page table page by page), the recognition model can only output the structural data of each region separately, and an additional processing flow needs to be added to merge the structural data of each region, which is difficult and labor-intensive, and has low recognition efficiency. Furthermore, the method of identifying regions one by one in a table spanning multiple pages isolates different regions from each other, making low use of contextual information between different regions, which affects recognition accuracy. It can be seen that the related art has various problems, and combined with the above description, it can be seen that the method for obtaining a recognition model provided in the embodiments of the present application can improve various problems existing in the related art.

[0088] For example, the first training sample may further include at least one of a first single-page table, a second double-page table, or a second single-page table. The more types of first training samples there are, the stronger the recognition capability of the trained recognition model will be.

[0089] For example, when the first training sample also includes a first single-page table, the recognition model can also learn the features of the first single-page table. When the input table is a single-page table, the input table can also be well recognized. The second area included in the first single-page table is obtained by performing layout analysis on the second document and may contain errors. It has a higher learning difficulty for the recognition model, which can further improve the ability of the recognition model. For another example, when the first training sample also includes a second spread-page table, the recognition model can also learn the features of the second spread-page table. The features of the spread-page table learned by the recognition model are richer. When the input table is a spread-page table, the recognition model can better recognize the input table. For another example, when the first training sample also includes a second single-page table, the recognition model can also learn the features of the first single-page table. When the input table is a single-page table, the input table can also be well recognized.

[0090] In an exemplary embodiment, the process of training a recognition model based on the first training sample includes but is not limited to the following steps.

[0091] The first step is to obtain an initial model, input the first training sample into the initial model, and obtain second structural data output by the initial model. The second structural data is used to represent the structure of the first cross-page table.

[0092] The embodiment of the present application does not limit the initial model, and the initial model can be a model obtained through pre-training. After obtaining the first initial model, the first training sample is input into the initial model to obtain the second structural data output by the initial model. Since the first training sample includes at least the first cross-page table, the second structural data is at least used to characterize the structure of the first cross-page table. Exemplarily, when the first training sample also includes the first single-page table, the second cross-page table or at least one training sample in the second single-page table, the second structural data is also used to characterize the structure of the at least one training sample. For example, the second structural data includes multiple groups of structural data, a part of the structural data in the multiple groups of structural data is used to characterize the structure of the first cross-page table, and another part of the structural data in the multiple groups of structural data is used to characterize the structure of the at least one training sample. The way in which the second structural data performs structural characterization can refer to the way in which the first structural data performs structural characterization in the above description, and will not be repeated here.

[0093] The second step is to obtain a second training sample according to the second structure data.

[0094] In the embodiment of the present application, the second structure data output by the initial model is analyzed to determine the training status, so that a second training sample can be obtained accordingly, and training is continued based on the second training sample. The second training sample can include at least one of a single-page table or a cross-page table, which is not limited in the embodiment of the present application. Thus, it is equivalent to supplementing the first training sample with a second training sample, which not only increases the number of training samples, but also increases the targeted training samples, which is conducive to improving the training effect and making the trained recognition model have stronger recognition capabilities.

[0095] Exemplarily, the process of obtaining a second training sample based on the second structural data includes determining abnormal structural data in the second structural data, wherein the structure represented by the abnormal structural data is abnormal, and obtaining a second training sample based on the abnormal structural data. Wherein, when the structure represented by the abnormal structural data is different from the true structure, it is considered that the structure represented by the abnormal structural data is abnormal. The abnormal structural data is output by the initial model, and the structure represented by the abnormal structural data is the structure obtained by the initial model recognition. When the structure represented by the abnormal structural data is different from the true structure, it indicates that there is an error in the recognition of the initial model, resulting in the recognized structure being different from the true structure. Therefore, it is necessary to supplement the second training sample in a targeted manner. For example, the second training sample is made to have the same structural features as the true structure, thereby improving the recognition ability of the initial model for such structural features, and avoiding errors in the recognition model obtained by training when recognizing the input table with such structural features.

[0096] In an exemplary embodiment, the second training sample and the real structure have the same structural features, including: the second training sample includes at least one reference grid, and the size of the reference grid is similar to the size of the real structure. If the size of the real structure is 2×5 (including 2 rows and 5 columns of cells), the size of the reference grid is also 2×5. For another example, when the size of the real structure is 2×5, the size of the reference grid is 4×10, 6×15, etc., that is, the reference grid has the same aspect ratio as the real structure. For another example, since the size of the real structure is 2×5, it can be determined based on this size that the real structure has a large number of columns, so the reference grid can also have a large number of columns. Taking the example that a large number of columns is considered when the number of columns is greater than or equal to 4, the size of the reference grid can be 2×4, 2×5, 2×6, 3×7, etc. For example, the position of the reference grid in the second training sample can be the same as or different from the position of the real structure in the first cross-page table, and this embodiment of the present application is not limited to this. For example, when the true structure is the first cell in the upper left corner of the first spread-page table, the reference cell may be the first cell in the upper left corner of the second training sample, or may be located at another position in the second training sample.

[0097] For example, the embodiment of the present application can also determine the number of second training samples based on the difference between the structure represented by the abnormal structure data and the true structure. For example, the larger the difference, the more structural errors represented by the abnormal structure data and the worse the recognition effect of the initial model, and thus the number of second training samples is increased to ensure the training effect. The smaller the difference, the fewer structural errors represented by the abnormal structure data and the better the recognition effect of the initial model, and thus the number of second training samples is decreased to reduce training costs and improve training efficiency.

[0098] The third step is to train the initial model according to the second training sample to obtain the recognition model.

[0099] After obtaining the second training sample, the second training sample is input into the initial model to continue training the initial model until a recognition model is obtained. Exemplarily, the embodiment of the present application trains the initial model according to the second training sample to obtain a transition model, and then tests the transition model. For example, multiple test samples are obtained, and the multiple test samples are respectively input into the transition model to obtain multiple groups of structural data output by multiple models. For any group of structural data, if the structure represented by the group of structural data is the same as the structure of the corresponding test sample, then the group of structural data is considered to be correct. If the accuracy of the multiple groups of structural data is greater than or equal to the accuracy threshold, it is determined that the transition model has passed the test and the training is completed, and the transition model is the recognition model obtained by training. If the accuracy of the multiple groups of structural data is less than the accuracy threshold, it is determined that the transition model has not passed the test, and the training continues until the trained model can pass the test, and the model that can pass the test is the recognition model obtained by training.

[0100] In some embodiments, the transition model is trained further by referring to the first through third steps described above. For example, after inputting the second training sample into the initial model, third structural data output by the initial model is obtained. The third structural data is used to characterize the structure of the second training sample. A third training sample is obtained based on the third structural data, and the transition model is trained based on the third training sample until a recognition model is obtained. In other embodiments, the transition model is trained further using other methods, and the present embodiment does not limit the training method.

[0101] In the above, the process of obtaining the recognition model (also called the training process) is described in general. Figure 4 An example is given to illustrate the training process.

[0102] like Figure 4As shown, in an embodiment of the present application, a first training sample is obtained, and the first training sample includes a first cross-page table and a first single-page table obtained by layout analysis, and also includes a second cross-page table and a second single-page table obtained by annotating open source documents / business documents. Training is performed based on the tables included in the first training sample to obtain abnormal structure data, and a second training sample is obtained based on the abnormal structure data. Iterative training is performed based on the tables included in the second training sample (for example, at least one of the cross-page table and the single-page table) until a recognition model is obtained.

[0103] In summary, in the embodiment of the present application, a recognition model is obtained by training based on the first training sample. Since the first training sample includes at least the first cross-page table, the recognition model can learn the characteristics of the first cross-page table. When the input table is a cross-page table, the recognition model can better recognize the input table, make full use of the context information between different regions in the input table, so that the output first structure data has a higher accuracy, and can completely restore the structure of the input table according to the first structure data without the need for additional merging and other processing, thereby improving the recognition efficiency. In addition, since the first cross-page table is obtained by merging at least two first regions, and at least two first regions are obtained through layout analysis and may contain errors, the first cross-page table may also contain errors. The first cross-page table has a higher learning difficulty for the recognition model, which can improve the recognition ability of the recognition model. Even if the input table is a table with errors, the recognition model can still better recognize the input table, so that the output first structure data has a higher accuracy.

[0104] In addition, the present invention also provides a method for identifying a table structure, which can be applied to Figure 1 or Figure 2 In the computing device shown. Figure 5 As shown, the method for identifying a table structure includes the following steps 501 and 502.

[0105] Step 501: Obtain a table to be identified.

[0106] Among them, the table to be identified is a table that needs to be identified by a recognition model, and the recognition process is used to obtain first structural data, and the first structural data is used to characterize the structure of the table to be identified. The embodiment of the present application does not limit the method of obtaining the table to be identified. In some embodiments, the table to be identified is obtained by annotation, and the annotation method includes but is not limited to manual annotation. In other embodiments, obtaining the table to be identified includes: obtaining a third document, performing layout analysis on the third document to obtain a third area, and obtaining the table to be identified based on the third area. For example, the third area is detected from the third document by a layout analysis model.

[0107] Exemplarily, the third document can be obtained by uploading by the user. For example, a document upload page is displayed, and the document upload page includes a document upload application programming interface (API). When it is detected that the document upload interface is selected, at least one document is displayed, and the document selected by the user in the at least one document is the third document. Exemplarily, the third document can be in an editable format or in a non-editable format. The table included in the third document may or may not have a frame line. It can be seen that the embodiment of the present application does not need to limit the format of the third document and the form of the table in the third document. Even in complex scenarios such as a third document in a non-editable format and a table without a frame line, the third area can be obtained by layout analysis, so that the table to be identified can be obtained based on the third area, which has strong universality.

[0108] After obtaining the third document, the third region can be detected from the third document using the layout analysis model. For example, each page in the third document is input into the layout analysis model sequentially or in batches to obtain region data output by the layout analysis model, and the third region is determined from the third document based on the region data. Exemplarily, when the third document is in a reference format, the third document in the reference format is directly input into the layout analysis model. When the third document is in a format other than the reference format, the format of the third document is converted into the reference format to obtain the third document in the reference format, and the third document in the reference format is input into the layout analysis model. The reference format is, for example, an image format. Accordingly, the third region output by the layout analysis model and the table to be identified are also in image format.

[0109] In the process of obtaining the table to be identified based on the third area, it can be determined whether the third area belongs to a cross-page table. In some embodiments, obtaining the table to be identified based on the third area includes: when the third area belongs to a cross-page table and the number of third areas is at least two, merging at least two third areas to obtain the table to be identified. For example, when it is determined that the third area belongs to a cross-page table by a binary classification model or a header recognition model, the third areas belonging to the same cross-page table can be merged to obtain the table to be identified, and the table to be identified belongs to a cross-page table. In other embodiments, obtaining the table to be identified based on the third area includes: when the third area belongs to a single-page table, using the third area as the table to be identified. For example, when it is determined that the third area does not belong to a cross-page table but belongs to a single-page table by a binary classification model or a header recognition model, there is no need to merge the third area, and the third area is the table to be identified, and the table to be identified belongs to a single-page table.

[0110] In step 502, the table to be identified is input into the recognition model to obtain first structural data output by the recognition model. The first structural data is used to characterize the structure of the table to be identified. The recognition model is trained based on a first training sample. The first training sample includes a first cross-page table. The first cross-page table is obtained by merging at least two first areas. The at least two first areas are obtained by performing layout analysis on the first document.

[0111] Among them, the recognition model is Figure 3 The recognition model obtained by training the method for obtaining the recognition model shown, the training process can be found in step 302 above, which will not be described in detail here. For example, in addition to the first cross-page table, the first training sample also includes at least one of a first single-page table, a second cross-page table or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation. For the description of the various tables included in the first training sample, please refer to step 301 above, which will not be described in detail here.

[0112] Since the first structural data is used to characterize the structure of the table to be identified, the embodiment of the present application can directly and completely restore the structure of the table to be identified based on the first structural data after obtaining the first structural data. For example, the size of the first grid in the upper left corner of the table to be identified is 2×5 (including 2 rows and 5 columns of cells), etc. Afterwards, the restored structure can also be applied according to actual needs. For example, new information can be filled into the restored structure according to actual needs. For another example, the method provided by the embodiment of the present application also includes: determining the characters in the table to be identified, the table to be identified is in a non-editable format, and generating a table in an editable format based on the characters and the first structural data.

[0113] Among them, the embodiment of the present application can determine the characters of the table to be identified by optical character recognition (OCR) and other methods, which are not limited here. After determining the characters, a table in an editable format can be generated based on the characters and the first structural data. For example, after restoring the structure of the table to be identified according to the first structural data, the characters are filled into each grid of the table to be identified, thereby generating a table in an editable format. Thus, converting the table to be identified in a non-editable format into a table in an editable format is more practical and can be applied to a variety of scenarios.

[0114] See also Figure 6After the user uploads the third document and performs preprocessing (including but not limited to the format conversion in step 501), a table to be recognized is obtained. Characters in the table to be recognized are determined through OCR, and the recognition model is used to determine the first structural data, thereby generating an editable table based on the characters and the first structural data. This editable table can be used for subsequent key field extraction, and the extracted key fields can be further applied, such as performing business analysis based on the key fields, etc., which is not limited in this embodiment of the present application.

[0115] In the above, the process of identifying the table structure (also called the reasoning process) is described in general. Figure 7 An example is given to illustrate the reasoning process.

[0116] After obtaining the third document uploaded by the user, the third document is converted into a reference format, and the third document in the reference format is input into the layout analysis model to obtain the region data output by the layout analysis. The third region is determined from the third document based on the region data. If the third region is a table that spans two pages, different third regions are merged to obtain a table to be identified. The third region and the merged table to be identified can be referred to. Figure 8 Alternatively, if the third area is a single-page table, the table to be identified is directly obtained. Subsequently, the table to be identified is input into the recognition model to obtain first structural data output by the recognition model. The first structural data is used to characterize the structure of the table to be identified, so that the complete table to be identified can be restored based on the first structural data.

[0117] In summary, in the embodiment of the present application, the structure of the table to be identified is identified by a recognition model. Since the recognition model is trained based on the first training sample, and since the first training sample includes at least the first cross-page table, the recognition model can learn the characteristics of the first cross-page table. In the case where the table to be identified is a cross-page table, the recognition model can better identify the table to be identified, fully utilizing the contextual information between different regions in the table to be identified, so that the output first structure data has high accuracy, and the structure of the table to be identified can be completely restored based on the first structure data without the need for additional merging or other processing, thereby improving recognition efficiency. In addition, since the first cross-page table is obtained by merging at least two first regions, and the at least two first regions are obtained through layout analysis and may contain errors, the first cross-page table may also contain errors. The first cross-page table has a higher learning difficulty for the recognition model, which can improve the recognition ability of the recognition model. Even if the table to be identified is a table with errors, the recognition model can still better identify the table to be identified, so that the output first structure data has high accuracy.

[0118] The above describes the method for obtaining a recognition model and the method for identifying a table structure provided by the embodiments of the present application. As mentioned above, these two methods can be applied to the same computing device or to different computing devices. Taking the application of these two methods to different computing devices (such as computing device A and computing device B) as an example, Figure 9 Give an example.

[0119] In step 901 , computing device A obtains a first training sample, where the first training sample includes a first cross-page table. The first cross-page table is obtained by merging at least two first regions, and the at least two first regions are obtained by performing layout analysis on a first document.

[0120] The implementation of step 901 can refer to the description of step 301 above, which will not be repeated here.

[0121] In step 902, computing device A obtains a recognition model through training based on the first training sample.

[0122] The implementation of step 902 can refer to the description in step 302 above and will not be repeated here. For example, after obtaining the recognition model, computing device A can actively send (i.e., push) the recognition model to computing device B. Alternatively, after obtaining the recognition model, computing device A stores the recognition model, and subsequently returns the recognition model to computing device B after receiving the recognition model acquisition request sent by computing device B. Of course, computing device A can also call and use the recognition model according to actual needs, and the implementation of this application is not limited to this.

[0123] Step 903: computing device B obtains the recognition model from computing device A.

[0124] Computing device B can receive the recognition model actively sent by computing device A, or send a recognition model acquisition request to computing device A and receive the recognition model returned by computing device A. In this way, computing device B acquires the recognition model.

[0125] Step 904: computing device B obtains the table to be identified.

[0126] The implementation of step 904 can refer to the description of step 501 above and will not be repeated here.

[0127] In step 905 , computing device B inputs the table to be recognized into the recognition model to obtain first structural data output by the recognition model. The first structural data is used to represent the structure of the table to be recognized.

[0128] The implementation of step 905 can refer to the description of step 502 above, which will not be repeated here.

[0129] The above describes the method for obtaining a recognition model provided by the embodiment of the present application. Corresponding to the above method, the embodiment of the present application also provides a device for obtaining a recognition model. The device is used to obtain a recognition model by Figure 10 The modules shown above perform the Figure 3 The method embodiment shown, or Figure 9 The steps performed by the computing device A in the method embodiment shown are as follows. Figure 10 As shown, the device for obtaining a recognition model provided in an embodiment of the present application includes the following modules.

[0130] An acquisition module 1001 is configured to acquire a first training sample, the first training sample comprising a first cross-page table, the first cross-page table being obtained by merging at least two first regions, the at least two first regions being obtained by performing layout analysis on a first document;

[0131] The training module 1002 is used to obtain a recognition model based on the first training sample. The recognition model is used to perform structural recognition on the input table to obtain first structural data. The first structural data is used to represent the structure of the input table.

[0132] In an exemplary embodiment, the training module 1002 is used to obtain an initial model, input a first training sample into the initial model, obtain second structural data output by the initial model, and the second structural data is used to characterize the structure of the first cross-page table; obtain a second training sample based on the second structural data; and train the initial model based on the second training sample to obtain a recognition model.

[0133] Illustratively, the training module 1002 is configured to determine abnormal structural data in the second structural data, where the structure represented by the abnormal structural data is abnormal; and obtain a second training sample based on the abnormal structural data.

[0134] In an exemplary embodiment, the first training sample also includes at least one of a first single-page table, a second cross-page table, or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

[0135] In addition, with the above Figure 5 Corresponding to the method for identifying table structure shown in FIG, the embodiment of the present application also provides a device for identifying table structure. Figure 11 The modules shown above perform the Figure 5 The method embodiment shown, or Figure 9 The steps performed by the computing device B in the method embodiment shown are as follows. Figure 11 As shown, the device for obtaining a recognition model provided in an embodiment of the present application includes the following modules.

[0136] An acquisition module 1101 is used to acquire a table to be identified;

[0137] Identification module 1102 is used to input the table to be identified into the recognition model to obtain first structural data output by the recognition model. The first structural data is used to characterize the structure of the table to be identified. The recognition model is trained based on a first training sample. The first training sample includes a first cross-page table. The first cross-page table is obtained by merging at least two first areas. The at least two first areas are obtained by performing layout analysis on the first document.

[0138] Illustratively, the acquisition module 1101 is configured to acquire a third document, perform layout analysis on the third document to obtain a third region; and acquire a table to be identified based on the third region.

[0139] Exemplarily, the acquisition module 1101 is used to merge at least two third regions to obtain a table to be identified when the third region belongs to a cross-page table and the number of third regions is at least two; or, when the third region belongs to a single-page table, use the third region as the table to be identified.

[0140] In an exemplary embodiment, the apparatus further includes: a generating module configured to determine characters in a table to be recognized, the table to be recognized being in a non-editable format; and generate a table in an editable format according to the characters and the first structure data.

[0141] Exemplarily, the first training sample also includes at least one of a first single-page table, a second cross-page table, or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

[0142] in, Figure 10 The acquisition module 1001 and the training module 1002 shown, and Figure 11 The acquisition module 1101 and recognition module 1102 shown can be implemented in software or hardware. For example, the implementation of acquisition module 1001 will be described below using acquisition module 1001 as an example. Similarly, the implementation of training module 1002, acquisition module 1101, and recognition module 1102 can refer to the implementation of acquisition module 1001.

[0143] As an example of a software functional unit, the acquisition module 1001 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the acquisition module 1001 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0144] For example, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0145] As an example of a hardware functional unit, the acquisition module 1001 may include at least one computing device, such as a server. Alternatively, the acquisition module 1001 may be implemented using a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a neural network processing unit (NPU), a system on chip (SoC), an offload card, an accelerator card, or any combination thereof.

[0146] The multiple computing devices included in the acquisition module 1001 can be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module 1001 can be distributed in the same AZ or in different AZs. For example, the multiple computing devices included in the acquisition module 1001 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, GALs, DPUs, NPUs, SoCs, offload cards, and accelerator cards.

[0147] In an exemplary embodiment, the acquisition module 1001 can be used to perform any step in the method for acquiring a recognition model, and the training module 1002 can be used to perform any step in the method for acquiring a recognition model. Figure 3 The method shown can also be Figure 9 The steps implemented by the computing device A in the method shown in the figure can be specified as needed, and the steps implemented by the acquisition module 1001 and the training module 1002 can be specified as needed, and the acquisition module 1001 and the training module 1002 can respectively implement different steps in the method of obtaining the recognition model to achieve Figure 10 The entire functionality of the device for obtaining the recognition model is shown.

[0148] For example, the acquisition module 1101 can be used to execute any step in the method for identifying a table structure, and the identification module 1102 can be used to execute any step in the method for identifying a table structure. Figure 5 The method shown can also be Figure 9 The steps implemented by the computing device B in the method shown in the figure can be specified as needed, and the steps implemented by the acquisition module 1101 and the identification module 1102 can be specified as needed, and the acquisition module 1101 and the identification module 1102 can respectively implement different steps in the method of identifying the table structure to achieve Figure 11 The overall functionality of the device for identifying table structures is shown.

[0149] It should be understood that the above Figure 10 The device shown in the figure has the following beneficial effects when it realizes its function: Figure 3 The method for obtaining the recognition model shown above has the same beneficial effects. Figure 11 The device shown in the figure has the following beneficial effects when it realizes its function: Figure 5 In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments, which will not be repeated here.

[0150] The present application also provides a computing device 1200. Figure 12 As shown, computing device 1200 includes a bus 1202, a processor 1204, a memory 1206, and a communication interface 1208. Processor 1204, memory 1206, and communication interface 1208 communicate with each other via bus 1202. Computing device 1200 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 1200.

[0151] The bus 1202 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 12 The bus 1204 may include a path for transmitting information between various components of the computing device 1200 (eg, the memory 1206, the processor 1204, and the communication interface 1208).

[0152] The processor 1204 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0153] The memory 1206 may include volatile memory, such as random access memory (RAM). The processor 1204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0154] Memory 1206 stores executable program code, and processor 1204 executes the executable program code to implement the functions of the aforementioned acquisition module 1001 and training module 1002 (for implementing the method for acquiring a recognition model), and / or the functions of the aforementioned acquisition module 1101 and recognition module 1102 (for implementing the method for recognizing a table structure). In other words, memory 1206 stores at least one of instructions for executing the method for acquiring a recognition model and instructions for executing the method for recognizing a table structure.

[0155] The communication interface 1208 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1200 and other devices or a communication network.

[0156] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0157] like Figure 13 As shown, the computing device cluster includes at least one computing device 1300, and each computing device 1300 includes a bus 1302, a processor 1304, a memory 1306, and a communication interface 1308. The memory 1306 in one or more computing devices 1300 in the computing device cluster may store the same instructions, which include at least one of instructions for executing the method for obtaining a recognition model and instructions for executing the method for recognizing a table structure.

[0158] In some possible implementations, the memory 1306 of one or more computing devices 1300 in the computing device cluster may also store some of the instructions for the complete set of instructions. The complete set of instructions includes at least one of instructions for executing the method for obtaining a recognition model and instructions for executing the method for identifying a table structure. In other words, the combination of one or more computing devices 1300 can jointly execute the complete set of instructions.

[0159] It should be noted that the memory 1306 in different computing devices 1300 in the computing device cluster can store different instructions, each used to perform part of the functions of the apparatus provided in the embodiments of the present application. The apparatus provided in the embodiments of the present application includes at least one of a device for acquiring a recognition model or a device for recognizing a table structure. In other words, the instructions stored in the memory 1306 in different computing devices 1300 can implement the functions of one or more modules of the acquisition module 1001, the training module 1002, the acquisition module 1101, or the recognition module 1102.

[0160] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 14 A possible implementation is shown. Figure 14 As shown, two computing devices 1300A and 1300B are connected via a network. Specifically, the connection to the network is achieved through a communication interface in each computing device. In this type of possible implementation, a first instruction is stored in the memory 1306 of the computing device 1300A. The first instruction is used to execute the functions of one or more modules among the acquisition module 1001, the training module 1002, the acquisition module 1101, or the recognition module 1102. Correspondingly, a second instruction is stored in the memory 1306 of the computing device 1300B. The second instruction is used to execute the functions of some or all of the remaining modules.

[0161] For example, see Figure 14 , the memory 1306 in the computing device 1300A stores instructions for executing the functions of the acquisition module 1001, and the memory 1306 in the computing device 1300B stores instructions for executing the functions of the training module 1002. Figure 14 The connection method between the computing device clusters shown may be based on the consideration that the method for obtaining the recognition model provided in this application requires the storage of more data (such as training samples) and consumes more computing resources during the training process, so it is considered to entrust the functions implemented by the training module 1002 to the computing device 1300B for execution.

[0162] For another example, the memory 1306 in computing device 1300A stores instructions for executing the functions of acquisition module 1101, and the memory 1306 in computing device 1300B stores instructions for executing the functions of identification module 1102. This connection method between computing device clusters may be considered to consider that the method for identifying table structures provided in this application consumes a large amount of computing resources when running the recognition model to identify the table, so it is considered to transfer the functions implemented by identification module 1102 to computing device 1300B.

[0163] It should be understood that Figure 14 The functionality of the computing device 1300A shown in FIG. 1 may also be implemented by multiple computing devices 1300. Similarly, the functionality of the computing device 1300B may also be implemented by multiple computing devices 1300.

[0164] The present application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes Figure 3 The method for obtaining the recognition model shown, or, executing Figure 5 The method of identifying the table structure is shown.

[0165] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to Figure 3 The method for obtaining the recognition model shown, or, executing Figure 5 The method of identifying the table structure is shown.

[0166] In some embodiments, the present application provides a chip including a processor configured to call and execute instructions stored in a memory, so that a computer equipped with the chip executes Figure 3 The method for obtaining the recognition model shown, or, implementing Figure 5 The method of identifying the table structure is shown.

[0167] In other embodiments, the present application also provides another chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected through an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the computer equipped with the chip realizes Figure 3 The method for obtaining the recognition model shown, or, implementing Figure 5 The method of identifying the table structure is shown.

[0168] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0169] In this application, the terms "first," "second," and the like are used to distinguish between identical or similar items with substantially the same purpose and function. It should be understood that "first," "second," and "nth" do not have a logical or temporal dependency, nor do they limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," and the like to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another.

[0170] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0171] In this application, the term "at least one" means one or more, and the term "plurality" means two or more. For example, a plurality of test samples refers to two or more test samples. The terms "system" and "network" are often used interchangeably herein.

[0172] It should be understood that the terms used in the description of the various examples herein are for the purpose of describing the particular examples only and are not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0173] It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the listed items. The term "and / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this application generally indicates that the associated objects are in an "or" relationship.

[0174] It should also be understood that the terms “if” and “if” may be interpreted to mean “when” or “upon” or “in response to determining” or “in response to detecting.” Similarly, the phrases “if it is determined that ” or “if [stated condition or event] is detected” may be interpreted to mean “upon determining ” or “in response to determining ” or “upon detecting [stated condition or event]” or “in response to detecting [stated condition or event],” depending on the context.

[0175] The above are merely embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for obtaining a recognition model, characterized in that: The method comprises: Acquire a first training sample, the first training sample comprising a first cross-page table, the first cross-page table being obtained by merging at least two first regions, the at least two first regions being obtained by performing layout analysis on a first document; A recognition model is obtained by training according to the first training sample. The recognition model is used to perform structural recognition on the input table to obtain first structural data. The first structural data is used to represent the structure of the input table.

2. The method according to claim 1, characterized in that The step of training the recognition model according to the first training sample includes: Acquire an initial model, input the first training sample into the initial model, and obtain second structural data output by the initial model, where the second structural data is used to represent the structure of the first cross-page table; Acquire a second training sample according to the second structure data; The initial model is trained according to the second training sample to obtain the recognition model.

3. The method according to claim 2, characterized in that The acquiring a second training sample according to the second structure data includes: determining abnormal structural data in the second structural data, wherein the abnormal structural data represents a structure having an abnormality; The second training sample is obtained according to the abnormal structure data.

4. The method according to any one of claims 1 to 3, characterized in that: The first training sample also includes at least one of a first single-page table, a second cross-page table or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

5. A method for identifying a table structure, characterized in that: The method comprises: Get the table to be identified; The table to be identified is input into a recognition model to obtain first structural data output by the recognition model. The first structural data is used to characterize the structure of the table to be identified. The recognition model is trained based on a first training sample. The first training sample includes a first cross-page table. The first cross-page table is obtained by merging at least two first areas. The at least two first areas are obtained by performing layout analysis on a first document.

6. The method according to claim 5, characterized in that The obtaining of the table to be identified includes: Acquire a third document, perform layout analysis on the third document, and obtain a third region; The table to be identified is obtained according to the third area.

7. The method according to claim 6, characterized in that The acquiring the table to be identified according to the third area includes: In a case where the third region belongs to a cross-page table and the number of the third regions is at least two, merging at least two third regions to obtain the table to be identified; Alternatively, when the third area belongs to a single-page table, the third area is used as the table to be identified.

8. The method according to any one of claims 5 to 7, characterized in that: The method further comprises: Determining characters in the table to be recognized, where the table to be recognized is in a non-editable format; A table in an editable format is generated based on the characters and the first structure data.

9. The method according to any one of claims 5 to 8, characterized in that: The first training sample also includes at least one of a first single-page table, a second cross-page table or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

10. A device for obtaining a recognition model, characterized in that: The device comprises: An acquisition module is configured to acquire a first training sample, wherein the first training sample includes a first cross-page table, wherein the first cross-page table is obtained by merging at least two first regions, and wherein the at least two first regions are obtained by performing layout analysis on a first document; The training module is used to obtain a recognition model based on the first training sample training, and the recognition model is used to perform structural recognition on the input table to obtain first structural data, and the first structural data is used to represent the structure of the input table.

11. The device according to claim 10, characterized in that The training module is configured to obtain an initial model, input the first training sample into the initial model, and obtain second structural data output by the initial model, wherein the second structural data is used to represent the structure of the first cross-page table; Acquire a second training sample according to the second structure data; The initial model is trained according to the second training sample to obtain the recognition model.

12. The device according to claim 11, characterized in that The training module is configured to determine abnormal structural data in the second structural data, wherein the abnormal structural data represents a structure having an abnormality; The second training sample is obtained according to the abnormal structure data.

13. The device according to any one of claims 10 to 12, characterized in that: The first training sample also includes at least one of a first single-page table, a second cross-page table or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

14. A device for identifying a table structure, characterized in that: The device comprises: An acquisition module, used to obtain the table to be identified; The recognition module is used to input the table to be recognized into the recognition model to obtain the first structural data output by the recognition model, wherein the first structural data is used to characterize the structure of the table to be recognized, and the recognition model is trained based on a first training sample, wherein the first training sample includes a first cross-page table, and the first cross-page table is obtained by merging at least two first areas, and the at least two first areas are obtained by performing layout analysis on the first document.

15. The device according to claim 14, characterized in that The acquisition module is configured to acquire a third document, perform layout analysis on the third document to obtain a third region; and acquire the table to be identified based on the third region.

16. The device according to claim 15, characterized in that The acquisition module is used to merge at least two third areas to obtain the table to be identified when the third area belongs to a cross-page table and the number of the third areas is at least two; or, when the third area belongs to a single-page table, use the third area as the table to be identified.

17. The device according to any one of claims 14 to 16, characterized in that: The device further comprises: The generating module is configured to determine characters in the table to be identified, where the table to be identified is in a non-editable format; and generate a table in an editable format according to the characters and the first structural data.

18. The device according to any one of claims 14 to 17, characterized in that: The first training sample also includes at least one of a first single-page table, a second cross-page table or a second single-page table, wherein the first single-page table includes a second area, the second area is obtained by performing layout analysis on the second document, and the second cross-page table and the second single-page table are obtained by annotation.

19. A computing device, characterized in that The computing device includes a processor and a memory; The processor is configured to execute instructions stored in the memory, so as to enable the computing device to perform the operating steps of the method according to any one of claims 1 to 9.

20. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device including a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in a memory of the at least one computing device, so that the computing device cluster performs the operating steps of the method according to any one of claims 1 to 9.

21. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the operation steps of any one of the methods according to claims 1 to 9.

22. A computer-readable storage medium, characterized in that The computer-readable storage medium includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the operation steps of any one of the methods according to claims 1 to 9.

Images