Table generation method and device, equipment, storage medium and program product

Abstract

The invention provides a table generation method and device, equipment, a storage medium and a program product, and relates to the technical field of table processing, and the method comprises the steps: recognizing a first direction of a table image, and obtaining a first direction line; performing text recognition on the table image to obtain at least one initial textbox and texts corresponding to the initial textboxes; fusing the initial textboxes according to a second direction to obtain a target textbox; determining a second direction line according to each target textbox; determining cells according to the first direction lines and the second direction lines; determining a text corresponding to each cell according to the text corresponding to each initial textbox; and determining a table corresponding to the table image according to each cell and the text corresponding to each cell. According to the embodiment of the invention, the table in the image without the table line can be identified.

Description

Technical Field

[0001] The present application relates to the field of table processing technology, and in particular to a table generation method, device, equipment, storage medium and program product. Background Art

[0002] Tabular data can reflect the capabilities of an enterprise.

[0003] Currently, table lines in table images can be detected, and the table structure can be restored through the identified table lines.

[0004] However, this table line detection method is not suitable for restoring tables in images without table lines. Summary of the invention

[0005] The present application provides a table generation method, apparatus, device, storage medium and program product, which can identify a table in an image without table lines.

[0006] In a first aspect, the present application provides a table generation method, comprising:

[0007] Identify a first direction of the table image to obtain at least one first direction line, where the first direction includes a column direction or a row direction;

[0008] Performing text recognition on the table image to obtain at least one initial text box and text corresponding to each initial text box;

[0009] Merging the initial text frames according to a second direction to obtain at least one target text frame; the second direction is different from the first direction;

[0010] Merge the target text boxes according to the first direction to obtain at least one target text box;

[0011] Determining at least one second direction line according to each of the target text boxes;

[0012] Determine at least one cell according to each of the first direction lines and each of the second direction lines;

[0013] Determine the text corresponding to each of the cells according to the text corresponding to each of the initial text boxes;

[0014] The table corresponding to the table image is determined according to each of the cells and the text corresponding to each of the cells.

[0015] In a second aspect, the present application provides a table generating device, comprising:

[0016] A table image acquisition module, used for acquiring a table image;

[0017] A first direction recognition module, used to recognize a first direction of the table image to obtain at least one first direction line, where the first direction includes a column direction or a row direction;

[0018] A text box recognition module, used for performing text recognition on the table image to obtain at least one initial text box and text corresponding to each initial text box;

[0019] a target text box identification module, configured to merge the initial text boxes according to a second direction to obtain at least one target text box; the second direction is different from the first direction;

[0020] a target text box identification module, configured to merge the target text boxes according to the first direction to obtain at least one target text box;

[0021] A second direction identification module, used for determining at least one second direction line according to each of the target text boxes;

[0022] A cell determination module, used for determining at least one cell according to each of the first direction lines and each of the second direction lines;

[0023] A cell text determination module, used to determine the text corresponding to each of the cells according to the text corresponding to each of the initial text boxes;

[0024] The table generation module is used to determine the table corresponding to the table image according to the cells and the texts corresponding to the cells.

[0025] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0026] The memory stores computer-executable instructions;

[0027] The processor executes the computer-executable instructions stored in the memory to implement the method described in the present application.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in the present application.

[0029] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the present application when executed by a processor.

[0030] The present application provides a table generation method, apparatus, device, storage medium and program product, which first identifies lines in one direction, then identifies text boxes in an image, clusters each text box in a second direction, identifies lines in the second direction, determines cell coordinates based on the lines in the two directions, and uses the text content of the text box as the text content of the cell corresponding to the position. This can identify the table structure of a table image without table lines, solves the problem in the prior art that the table structure of table line detection cannot handle table restoration of images without table lines, improves the recognition accuracy of table results, and reduces the labor cost and improves the efficiency of table generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0032] Figure 1 A flowchart of a table generation method provided in an embodiment of the present application;

[0033] Figure 2 A flowchart of a table generation method provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a table generation scenario provided in an embodiment of the present application;

[0035] Figure 4 A flowchart of a table generation method provided in an embodiment of the present application;

[0036] Figure 5 A schematic diagram of a table generation scenario provided in an embodiment of the present application;

[0037] Figure 6 A schematic diagram of the structure of a table generating device provided in an embodiment of the present application;

[0038] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0039] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0040] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0041] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or must use the relevant content of the solution. The acquisition, storage, use, and processing of data in the technical solution of the present application are in compliance with the relevant provisions of national laws and regulations.

[0042] In this embodiment, the table structure in the image is identified by identifying table lines in the prior art, and a table of corresponding structure is generated. This method cannot detect and reconstruct a table without table lines. The present application proposes a table generation method, device, equipment, storage medium and program product, which first identifies lines in one direction, identifies text boxes in the image, clusters each text box in the second direction, identifies lines in the second direction, determines cell coordinates based on the lines in the two directions, and uses the text content of the text box as the text content of the cell corresponding to the position. The table structure can be identified for table images without table lines, which solves the problem that the table structure detected by table lines in the prior art cannot handle the restoration of images without table lines, and can improve the recognition accuracy of table results, while reducing the implementation labor cost and improving the efficiency of table generation.

[0043] Figure 1 The table generation method provided in the embodiment of the present application is a flow chart. The embodiment can be applied to the case of identifying a table structure in an image and generating a table. The method can be executed by a table generation device, which can be implemented in the form of hardware and / or software. Figure 1 As shown, the method comprises the following steps:

[0044] S101, obtaining a table image.

[0045] The table image includes the text content in the table. The table can be a table without table lines or a table with table lines. Optionally, the table image includes the text distributed in the table, but does not include the table lines.

[0046] S102: Identify a first direction of the table image to obtain at least one first direction line, where the first direction includes a column direction or a row direction.

[0047] The first direction may refer to the row direction or column direction of the table in the table image. Usually, the first direction of the table image is parallel or perpendicular to the boundary line of the table image. The column direction may refer to the direction of the table column; the row direction may refer to the direction of the table row. The first direction line may refer to a line in the first direction, and the line may be a straight line, a line segment, or a ray, etc. The first direction line is usually represented by the coordinates of a point on the first direction line. In one example, the first direction line is a line segment, and the two endpoints of the line segment are usually the intersection points between cells in the table, and the coordinates may include the coordinates of the two endpoints of the line segment in the first direction. The number of first direction lines is at least one. The coordinates of a first direction line are at least two. Recognizing the first direction of the table image is actually recognizing the text distribution in the first direction of the table image.

[0048] A pre-trained direction line detection model can be used to input the table image into the direction line detection model for processing, and the coordinates of at least one first direction line can be output, and at least two coordinates of the same first direction line determine the first direction line. The direction line detection model can be a deep learning model.

[0049] S103: Perform text recognition on the table image to obtain at least one initial text box and text corresponding to each initial text box.

[0050] In an embodiment of the present application, the text in the table image is usually related to the table to be recognized, for example, the text in the table image may be the text of a cell. Optionally, there is no text outside the table area in the table image. The initial text box may refer to a text box of continuous text. The text in the initial text box usually expresses the same meaning. The text corresponding to the initial text box may refer to the text in the initial text box.

[0051] Text distribution detection can be performed on the table image, and the texts that are clustered into one category are determined as the texts corresponding to the same initial text box, and the minimum circumscribed rectangle of the texts corresponding to the same initial text box is determined as the initial text box. Alternatively, a pre-trained text box detection model can be used for implementation. The table image is input into the text box detection model for processing, and the coordinates of at least one initial text box and the text corresponding to the initial text box are output, or only at least one initial text box is output, and the text within the range of each initial text box is determined according to the coordinates of each initial text box as the text corresponding to each initial text box.

[0052] S104. Merge the initial text boxes according to a second direction to obtain at least one target text box; the second direction is different from the first direction.

[0053] The second direction is different from the first direction. The first direction and the second direction are perpendicular to each other. If the first direction is the row direction, the second direction is the column direction. If the first direction is the column direction, the second direction is the row direction. The value ranges of the first direction and the second direction are both the row direction or the column direction. The second direction is parallel or perpendicular to the boundary line of the table image. The initial text boxes are merged according to the second direction, specifically, the text boxes on the straight line of the same second direction are merged into one or a small number of text boxes. Fusion may refer to merging or splicing, and the fused text boxes include all the text boxes before fusion. Optionally, the text boxes on the straight line of the same second direction are merged into one text box. The target text box may refer to the result of the fusion of the initial text boxes. The number of initial text boxes on the same straight line of the second direction is less than or equal to the number of target text boxes on the straight line of the second direction.

[0054] S105. Determine at least one second direction line according to each of the target text boxes.

[0055] At least one target text box on the same straight line in the second direction is obtained, and according to the boundaries of these target text boxes, the coordinates of the two endpoints of the line segment in the second direction are determined, thereby determining a second direction line. Target text boxes on different straight lines are used to determine different second direction lines. In one example, in at least one target text box on the same straight line in the second direction, at least one vertex pair on the same straight line in the second direction and the farthest distance is determined from the vertex of the first text box and the vertex of the last text box, and each vertex pair determines a second direction line. Exemplarily, the second direction is the row direction, the first text box is the leftmost text box, the last text box is the rightmost text box, and the vertex pairs on the same straight line in the second direction and the farthest distance include: a first vertex pair formed by the upper left vertex of the first text box and the upper right vertex of the last text box, and a second vertex pair formed by the lower left vertex of the first text box and the lower right vertex of the last text box. The second direction line determined by the first vertex pair is parallel to the second direction line determined by the second vertex pair. Or the first text box is the rightmost text box, and the last text box is the leftmost text box, and the vertex pairs on the same straight line in the second direction and the farthest apart include: a first vertex pair formed by the upper right vertex of the first text box and the upper left vertex of the last text box, and a second vertex pair formed by the lower right vertex of the first text box and the lower left vertex of the last text box.

[0056] S106. Determine at least one cell according to each of the first direction lines and each of the second direction lines.

[0057] The first direction line and the second direction line are combined to form a grid, wherein one grid is a cell, the boundary line of the cell is the first direction line or the second direction line, and the position of the cell can be represented by the first direction line and the second direction line. The cell can refer to a cell of a table in a table image.

[0058] S107. Determine the text corresponding to each of the cells according to the text corresponding to each of the initial text boxes.

[0059] The initial text box corresponding to the cell can be determined according to the position of each initial text box, the first direction line coordinates, the second direction line coordinates and the position of each cell. The corresponding relationship between the cell and the initial text box can mean that the position of the cell and the initial text box is consistent. The text corresponding to the initial text box is determined as the text corresponding to the corresponding cell.

[0060] S108. Determine the table corresponding to the table image according to the cells and the texts corresponding to the cells.

[0061] Based on the position of the cell and the text corresponding to the cell, you can determine the cells included in the table and the text content of the cells in the table. The position of the cell is used to determine the shape structure of the table. The text is used to determine the content of the table.

[0062] The present application provides a table generation method, apparatus, device, storage medium and program product, which first identifies lines in one direction, then identifies text boxes in an image, clusters each text box in a second direction, identifies lines in the second direction, determines cell coordinates based on the lines in the two directions, and uses the text content of the text box as the text content of the cell corresponding to the position. This can identify the table structure of a table image without table lines, solves the problem in the prior art that the table structure of table line detection cannot handle table restoration of images without table lines, improves the recognition accuracy of table results, and reduces the labor cost and improves the efficiency of table generation.

[0063] Figure 2 A flowchart of a table generation method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method comprises the following steps:

[0064] S201, obtaining a table image.

[0065] S202: Identify a first direction of the table image to obtain at least one first direction line, where the first direction includes a column direction or a row direction.

[0066] S203: Perform text recognition on the table image to obtain at least one initial text box and text corresponding to each initial text box.

[0067] S204, merging the initial text boxes according to a second direction to obtain at least one target text box; the second direction is different from the first direction.

[0068] S205. Detect whether a plurality of target text boxes adjacent to each other in the first direction meet a fusion condition according to the associated information of the table image; the associated information of the table image includes: key texts corresponding to each of the target text boxes and / or scene table structure information corresponding to the table image.

[0069] Before determining the second direction line, the target text box can also be optimized. The optimization method can be to detect whether there are at least two target text boxes in the first direction that can be further merged, and when it is determined that multiple adjacent target text boxes in the first direction can be further merged, merge the fused target text boxes. The fusion condition is used to detect whether the target text box can be merged. The fusion condition may include that the key text is consistent and / or the position of the text in the target text box corresponds to the scene table structure corresponding to the table image. Among them, the key text is consistent, that is, the associated information includes the key text, and it is judged from the dimension of the content included in the text box whether multiple target text boxes can be merged, and the position of the text in the target text box corresponds to the scene table structure corresponding to the table image, that is, the associated information of the table image includes the scene table structure information corresponding to the table image, and it is judged from the dimension of the position of the text box whether multiple target text boxes can be merged.

[0070] In an example, the first direction is the column direction, and the second direction is the row direction. After obtaining the target text box, the cells across the rows are equivalent to two target text boxes on the same column direction line, and the two target text boxes are adjacent in the column direction. The two target text boxes can be merged to realize the fusion operation of the target text boxes in the column direction.

[0071] S206: When it is detected that multiple adjacent target text boxes meet a fusion condition, the multiple adjacent target text boxes are merged.

[0072] When multiple adjacent target text boxes meet the fusion condition, the multiple target text boxes are merged and the target text boxes are updated, that is, the target text boxes before fusion are removed and the fused target text boxes are added. Meeting the fusion condition indicates that the multiple target text boxes belong to one cell. There can be multiple groups of target text boxes that meet the fusion condition. The target text boxes in the same group can be merged, and the target text boxes in different groups cannot be merged.

[0073] When it is detected that the adjacent multiple target text boxes do not meet the fusion condition, the adjacent multiple target text boxes are not merged.

[0074] S207: Determine at least one second direction line according to each of the target text boxes.

[0075] By adopting the target text box optimized by first direction fusion and determining the second direction line, the target text boxes belonging to the same cell can be merged, and then a second direction line is determined corresponding to the target text box of a cell, thereby avoiding a cell being divided by two second direction lines.

[0076] S208. Determine at least one cell according to each of the first direction lines and each of the second direction lines.

[0077] S209: Determine the text corresponding to each of the cells according to the text corresponding to each of the initial text boxes.

[0078] S210: Determine the table corresponding to the table image according to the cells and the texts corresponding to the cells.

[0079] It can be seen that by determining that multiple target text boxes adjacent to each other in the first direction meet the fusion conditions based on the associated information of the table image, the multiple target text boxes adjacent to each other in the first direction can be fused to form the target text boxes belonging to the same cell, and then a second direction line is determined corresponding to the target text box of a cell, thereby avoiding a cell being divided by two second direction lines and improving the accuracy of determining the second direction line.

[0080] Optionally, the associated information of the table image includes: key text corresponding to each of the target text boxes; detecting whether multiple adjacent target text boxes in the first direction meet the fusion condition based on the associated information of the table image includes: detecting whether the key texts corresponding to multiple adjacent target text boxes in the first direction belong to the same content; when the key texts corresponding to the multiple adjacent target text boxes belong to the same content, determining that the multiple adjacent target text boxes in the first direction meet the fusion condition.

[0081] Key text may refer to key information of text corresponding to the target text box. For example, key text may be text of a key text box corresponding to the target text box. A key text box may refer to a text box on the direction line where the target text box is located, which represents the content of cells on the direction line. Usually, the text corresponding to a key text box is the key text. A key text box may refer to the first initial text box on the direction line where the target text box is located. When the initial text box is detected, multiple initial text boxes may be sorted from left to right in the column direction and from top to bottom in the row direction. The first initial text box on the same row direction line may refer to the leftmost initial text box on the row direction line; the first initial text box on the same column direction line may refer to the topmost initial text box on the column direction line.

[0082] Multiple key texts belong to the same content, which can refer to the meaning of multiple key texts. Figure 1It is possible to detect whether the key texts corresponding to multiple target text boxes are the same or similar, so as to determine whether the key texts corresponding to the multiple target text boxes belong to the same content. When the key texts corresponding to multiple adjacent target text boxes belong to the same content, it is determined that the multiple adjacent target text boxes in the first direction meet the fusion condition; when the text contents of multiple adjacent target text boxes do not belong to the same content, it is determined that the multiple adjacent target text boxes in the first direction do not meet the fusion condition. For every two adjacent target text boxes, it is possible to detect whether the key texts corresponding to the two adjacent target text boxes belong to the same content, and then combine the two adjacent target text boxes whose key texts belong to the same content, and finally obtain multiple adjacent target text boxes belonging to the same content.

[0083] By detecting that the key texts corresponding to multiple adjacent target text boxes in the first direction belong to the same content, it is determined that the multiple target text boxes meet the fusion conditions. From the perspective of the content of the target text boxes, multiple target text boxes in the first direction with consistent contents can be detected and merged, thereby achieving the merging of target text boxes with the same content, streamlining the number of target text boxes, and reducing split cells.

[0084] Optionally, the associated information of the table image includes: scene table structure information corresponding to the table image; detecting whether multiple adjacent target text boxes in the first direction meet the fusion condition based on the associated information of the table image includes: detecting whether there is text in the multiple adjacent target text boxes located at a position corresponding to the scene table structure information; when there is text located at a position corresponding to the scene table structure, determining whether the multiple adjacent target text boxes in the first direction meet the fusion condition.

[0085] The scenario table structure information is used to determine the table structure, and the table structure is usually related to the usage scenario of the table. Exemplarily, the scenario table structure information may include a report in a financial scenario. The text at the position corresponding to the scenario table structure information in the adjacent multiple target text boxes may refer to the position of the text in the adjacent multiple target text boxes corresponding to the scenario table information structure. Exemplarily, the number of the adjacent multiple target text boxes is two, and accordingly, the text at the position corresponding to the scenario table structure information in the target text box may refer to the text being at the boundary position between the two target text boxes, that is, the middle position in the first direction. It can be understood that the position of the text of a cell is centered, so that when the cell is divided into two text boxes, the centered text is in the middle of the two divided text boxes, or at the boundary position between the two text boxes. Tables with such centered text are usually applied to financial scenarios.

[0086] By detecting that multiple target text boxes adjacent to each other in the first direction have text at positions corresponding to the scene table structure information, it is determined that the multiple target text boxes meet the fusion conditions. From the perspective of the text distribution position of the target text boxes, multiple target text boxes in the first direction at specific text distribution positions can be detected and merged, thereby achieving the merging of target text boxes at specific text distribution positions, streamlining the number of target text boxes, and reducing split cells.

[0087] Optionally, determining at least one cell according to each of the first direction lines and each of the second direction lines includes: determining at least one intersection coordinate according to each of the first direction lines and each of the second direction lines; and determining at least one cell according to each of the intersection coordinates.

[0088] The first direction line is equivalent to the table line of the first direction, and the second direction line is equivalent to the table line of the second direction. The intersection of the table line of the first direction and the table line of the second direction is the vertex of the cell. Therefore, the vertex coordinates of the cell can be determined according to the coordinates of the intersection of the first direction line and the second direction line. The diagonal fixed point coordinates of a cell can determine the position and size of the cell.

[0089] By determining the intersection points of each first direction line and each second direction line as the vertices of the cells, it is possible to determine the positions and sizes of the cells in the table based on the generated table lines, and accurately determine the distribution structure of the table cells.

[0090] Optionally, after the initial text boxes are merged in the second direction to obtain at least one target text box, the method further includes: adjusting the spacing of the target text boxes in the first direction.

[0091] The spacing in the first direction may refer to the distance in the first direction. Since the detected initial text boxes have size differences in the second direction, the spacings of the multiple target text boxes obtained by fusion in the first direction are uneven, which in turn causes the spacings between the second direction lines to be uneven, and finally causes the sizes of the cells in the table to differ greatly.

[0092] The target text boxes may be adjusted to have uniform spacing in the first direction, so that the target text boxes are evenly distributed in the first direction, and the distance between every two adjacent first target text boxes in the first direction is the same.

[0093] In one example, the first direction is a column direction, the second direction is a row direction, and the row spacings between the multiple target text boxes are adjusted to be the same.

[0094] After obtaining the target text boxes, the spacing between the target text boxes in the first direction is adjusted to adjust the position distribution of the target text boxes in the first direction, so that the structure of the table finally obtained is reasonable.

[0095] Optionally, after determining at least one cell according to each of the first direction lines and each of the second direction lines, it also includes: detecting multiple mergeable cells in each of the cells; when multiple mergeable cells exist, merging the multiple mergeable cells to obtain a merged cell, and updating each of the cells.

[0096] Among them, the merge detection of cells can be to optimize the merge situations that have not been detected in the early stage. The cells that can be merged can be: multiple cells with the same meaning. Usually, the cells that can be merged are multiple adjacent cells. The image area corresponding to the detected cells can be input into a pre-trained merge detection model for processing, and at least one group is output. Each group includes at least two cells. Cells in the same group can be merged, and cells in different groups cannot be merged. Alternatively, the table image and the position of the cells can be input into the merge detection model for processing, and at least one group is output.

[0097] For example, the multiple cells that can be merged may have the same meaning, or text with the same meaning may span multiple cells. For example, the multiple cells include characters of the same parameter; they may also be cells with text and at least one adjacent blank cell.

[0098] By merging and optimizing the cells after obtaining the cells, the cells can be streamlined, and the actual division of one cell into two cells can be avoided, thereby improving the detection accuracy of the table structure.

[0099] Optionally, each of the initial text boxes is merged in a second direction to obtain at least one target text box, including: merging multiple initial text boxes located in the same straight line in the second direction to obtain a target text box; and determining a single initial text box on the same straight line in the second direction as the target text box.

[0100] The target text box includes at least one initial text box on the same straight line in the second direction, and the target text box is also located on the same straight line in the second direction. The boundary line of the target text box can be determined according to the boundary lines of multiple initial text boxes on the same straight line in the second direction.

[0101] Specifically, the boundary line of the first direction of the target text frame is determined according to the boundary lines of the first direction of the first and last initial text frames in the second direction of the multiple initial text frames; the boundary line of the second direction of the target text frame is determined according to the boundary lines of the second direction of each initial text frame. The boundary line of the first direction may refer to a boundary line parallel to the first direction, and the boundary line of the second direction may refer to a boundary line parallel to the second direction. In an example, the first direction is the column direction, the second direction is the row direction, and the initial text frames arranged from left to right include initial text frame 1, initial text frame 2, and initial text frame 3. Among them, the first initial text frame is initial text frame 1, and the last initial text frame is initial text frame 3. The three initial text frames are merged to obtain the target text frame, wherein the boundary line of the first direction of the target text frame, that is, the left and right boundary lines, is determined according to the left and right boundary lines of initial text frame 1 and initial text frame 3, specifically, the left boundary line of the target text frame is the left boundary line of initial text frame 1, and the right boundary line of the target text frame is the right boundary line of initial text frame 3. The upper and lower boundary lines of the target text box are determined according to the upper and lower boundary lines of initial text box 1, initial text box 2 and initial text box 3. Usually, the upper boundary line of the topmost of the three initial text boxes is the upper boundary line of the target text box, and the lower boundary line of the bottommost of the three initial text boxes is the lower boundary line of the target text box.

[0102] In addition, after obtaining the target text box, the target text box can also be adjusted. For example, adaptive adjustment can be performed based on the distribution and boundary lines of the obtained multiple target text boxes. For another example, the sizes of target text boxes with similar sizes can be adjusted to the same size, and the spacing between the target text boxes in the first direction can be adjusted so that the target text boxes are evenly distributed in the first direction.

[0103] By fusing multiple initial text boxes located on the same straight line in the second direction to obtain a target text box, the initial text boxes can be merged in the second direction to determine the boundary of the table in the second direction, and then the table line of the table in the second direction can be determined, so as to generate table lines in the second direction in a scenario where table lines in the second direction do not exist, thereby improving the accuracy of structural detection of wireless tables.

[0104] Optionally, the lengths of the initial text boxes in the first direction are similar; and the number of the initial text boxes in the first direction is greater than the number of the initial text boxes in the second direction.

[0105] The lengths of each initial text box in the first direction are similar, which can indicate that the distances between the center lines of each initial text box are similar, and thus the distances between the second direction lines determined based on each initial text box are similar, that is, the degree of correlation between the second direction line and the initial text box is high, and thus the coordinates of the second direction line can be obtained by summarizing the initial text boxes to determine the position of the table line in the second direction.

[0106] The number of initial text boxes in the first direction may refer to the number of initial text boxes on the same straight line counted from the first direction. The number of initial text boxes in the second direction may refer to the number of initial text boxes on the same straight line counted from the second direction. The number of initial text boxes in the first direction is equivalent to the number of second direction lines, and the number of initial text boxes in the second direction is equivalent to the number of first direction lines. The number of initial text boxes in the first direction is greater than the number of initial text boxes in the second direction, indicating that the number of second direction lines is greater than the number of first direction lines. The detection difficulty of the second direction lines is higher than the detection height of the first direction.

[0107] By directly detecting the first direction lines of the table image and performing inductive aggregation detection on the second direction lines of the text boxes in the table image, table lines in the first direction are generated in a manner with low detection accuracy and low detection difficulty, and table lines in the second direction are generated in a manner with high detection accuracy and high detection difficulty. This allows the generation of table lines to be reasonably adapted to different scenarios, taking into account both the speed and accuracy of table line generation, thereby improving the recognition accuracy of the table structure.

[0108] Optionally, the first direction is a column direction, and the second direction is a row direction.

[0109] For most tables in most scenarios, the text height is basically the same, that is, the length of the text box in the column direction is basically the same, and the text width (number of words) varies, that is, the length of the text in the row direction is diverse. In addition, the number of rows is more than the number of columns. For example, the number of rows in a financial report is 1,000, and the number of columns is 40. The number of rows is much larger than the number of columns, and the number of columns is relatively fixed.

[0110] In the embodiment of the present application, a deep learning model is used to identify the first direction line, and a method of identifying and summarizing text boxes is used to identify the second direction line. For the deep learning model, the detection accuracy of using the model to identify a small number of columns is higher than the detection accuracy of using the model to identify a large number of rows. At the same time, the detection accuracy of the inductive method of text arrangement in the column direction is inferior to the detection accuracy of the inductive method of text arrangement in the row direction.

[0111] In a specific example, a deep learning model is used to identify the first direction line, and a text box induction method is used to identify the second table line.

[0112] By configuring the first direction as the column direction and the second direction as the row direction, the detection accuracy of the table lines in both directions of the table can be improved.

[0113] In an optional embodiment, the lengths of the initial text boxes in the second direction are similar, or the number of the initial text boxes in the first direction is less than the number of the initial text boxes in the second direction. Accordingly, text recognition is performed on the table image to obtain at least one initial text box and the text corresponding to each initial text box, including: obtaining the number of initial text boxes of at least one straight line in the second direction, obtaining a preset value range, and adjusting the initial text boxes corresponding to the straight lines that do not belong to the value range, and optimizing the initial text boxes so that the number of initial libraries of each straight line in the second direction belongs to the value range. Exemplarily, at least one pair of initial text boxes corresponding to the straight lines greater than the value range are merged, and at least one initial text box corresponding to the straight line less than the value range is split. In this way, a large number of initial text boxes in the second direction can be integrated into a fixed number range of initial text boxes, so that the initial text boxes can be better aggregated to obtain the second direction line later.

[0114] Optionally, determining the text corresponding to each of the cells based on the text corresponding to each of the initial text boxes includes: calculating the intersection-and-union ratio between each of the initial text boxes and each of the cells; for each of the cells, determining the initial text box corresponding to the cell based on the intersection-and-union ratio between the cell and each of the initial text boxes; for each of the cells, determining the text of the initial text box corresponding to the cell as the text corresponding to the cell.

[0115] The intersection-and-union ratio between the initial text box and the cell is used to determine the overlap rate and similarity between the initial text box and the cell, and the initial text box and the cell with the largest intersection-and-union ratio and greater than the intersection-and-union ratio threshold are determined to be the same object. The initial text box corresponding to the cell may refer to the initial text box that is the same object as the cell. The text of the initial text box is the text filled in the cell. The intersection-and-union ratio between the initial text box and the cell can be determined based on the vertex coordinates of the initial text box and the vertex coordinates of the cell.

[0116] By determining the initial text box corresponding to the cell according to the intersection-and-union ratio between the initial text box and the cell, and determining the text of the initial text box as the text of the corresponding cell, the content of the cell is determined, thereby improving the recognition accuracy of the table content.

[0117] In a specific example, Figure 3 As shown, the first direction is the column direction, and the second direction is the row direction. The table generation method includes:

[0118] S301, input the table image I2 into the table column recognition model M4, and obtain P table column line coordinates, namely, column direction line coordinates X1-XP.

[0119] S302, performing classic OCR (Optical Character Recognition) recognition on the table image I2, and obtaining K1 text box coordinates D1-DK1 and corresponding texts R1-RK1. Among them, D1-DK1 have been sorted from top to bottom and from left to right according to the horizontal and vertical coordinates.

[0120] S303 , classify and merge the text boxes D1 - DK according to the values ​​of the row coordinates to obtain K2 target text box coordinates D1 - DK2 and the corresponding first column texts R1 - RK2 , ie, key texts R1 - RK2 .

[0121] S304 , correct the coordinates of D1 to DK2 to obtain adjusted text box coordinates D1 to DK2 .

[0122] Adjust the distance between D1 and DK2 evenly.

[0123] S305. According to the text semantic features of the table and the structural characteristics of the financial statement, D1 to DK2 are merged to obtain Q table row line coordinates, namely, row direction line coordinates Y1 to YQ.

[0124] S306. According to the arrangement and combination of the table column line coordinates X1 to XP and the table row line coordinates Y1 to YQ, the (P-1)×(Q-1) cell coordinates B11 to B(P-1)(Q-1) can be obtained.

[0125] S307. Calculate the IOU of text boxes D1 to DK1 with cells B11 to B(P-1)(Q-1). Place the text contents in the cells with the largest IOU in order, and obtain the text C11 to C(P-1)(Q-1) of each cell. Thus, the cell coordinates and cell contents of the table are restored.

[0126] The embodiment of the present application realizes the restoration of the cell structure and content of a table from a table image. It can be applied to fields such as financial report intelligent recognition where the table style is complex and the table structure needs to be accurately recognized, making the application scenarios wider. It is also applicable to the situation where the original image has or does not have table lines, and the recognition accuracy is high.

[0127] Figure 4 A flowchart of a table generation method provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the method comprises the following steps:

[0128] S401, obtaining an initial image.

[0129] The initial image is the original image containing the table, and may be an image containing the table obtained by direct acquisition.

[0130] S402: Identify text boxes in the initial image to obtain at least one pre-processed text box.

[0131] The recognition method of the preprocessing text box and the recognition method of the initial text box can be the same.

[0132] S403: cropping the initial image according to each of the pre-processed text boxes to obtain text blocks corresponding to each of the pre-processed text boxes.

[0133] The text block may refer to an image region obtained by mapping a preprocessed text box to an initial image. A preprocessed text box may obtain a corresponding text block.

[0134] S404: Detect the flipping angle of each text block.

[0135] The flip angle of the text block can be determined according to the direction of the text in the text block. The flip angle can refer to the angle between the current direction of the text and the positive direction of the text, wherein the value range of the flip angle can be 0°, 90°, 180° and 270°.

[0136] S405: Determine a flip angle of the initial image according to the flip angle of each text block.

[0137] The flip angles of different text blocks may be different or the same. The flip angles of each text block may be counted to determine the flip angle of the initial image. For example, the flip angle with the largest number may be determined as the flip angle of the initial image, or the average flip angle of each text block may be determined as the flip angle of the initial image.

[0138] S406: Flip the initial image according to the flip angle of the initial image to obtain a table image.

[0139] Flipping the initial image to obtain a table image is equivalent to optimizing the angle of the initial image to obtain a table image, thereby reducing the table structure detection error introduced due to the inconsistent orientation of the table image. Flipping can refer to rotating the center point (e.g., the intersection of diagonal lines) of the table image by a flipping angle. Usually, the initial image after flipping, i.e., the orientation of the text in the table image, is a standard readable orientation.

[0140] S407: Identify a first direction of the table image to obtain at least one first direction line, where the first direction includes a column direction or a row direction.

[0141] By flipping the angle of the initial image to obtain a table image, the first direction in the table image can be made parallel or perpendicular to the boundary of the table image, and the second direction can be made parallel or perpendicular to the boundary of the table image, which can simplify the determination of the first direction line and the second direction line.

[0142] S408: Perform text recognition on the table image to obtain at least one initial text box and text corresponding to each initial text box.

[0143] S409: Merge the initial text boxes according to a second direction to obtain at least one target text box; the second direction is different from the first direction.

[0144] S410. Determine at least one second direction line according to each of the target text boxes.

[0145] S411. Determine at least one cell according to each of the first direction lines and each of the second direction lines.

[0146] S412. Determine the text corresponding to each of the cells according to the text corresponding to each of the initial text boxes.

[0147] S413: Determine the table corresponding to the table image according to the cells and the texts corresponding to the cells.

[0148] The embodiment of the present application recognizes the text in the initial image to obtain the flip angle of the text block, and flips the initial image according to the flip angle of the text block to obtain a table image, thereby optimizing the angle of the initial image, reducing the table structure detection error introduced due to the inconsistent orientation of the table image, and improving the accuracy of table structure detection.

[0149] Optionally, determining the flip angle of the initial image according to the flip angle of each text block includes: calculating the weight of each text block; and calculating the flip angle of the initial image according to the weight of each text block and the flip angle of each text block.

[0150] The weight of the text block is used to indicate the importance of the text block. The flip angles of the text blocks can be weighted and summed according to the weights of the text blocks to obtain the flip angle of the initial image. Alternatively, the sum of the weight values ​​of each flip angle can be counted, and the flip angle with the largest sum of weight values ​​can be determined as the flip angle of the initial image.

[0151] In one example, weight values ​​of 0°, 90°, 180° and 270° are counted, and for each flip angle, the sum of the weight values ​​is calculated, and the flip angle with the largest sum of weight values ​​is determined as the flip angle of the initial image.

[0152] By calculating the weight of the text block and determining the flip angle of the initial image based on the weight of the text block and the flip angle, the orientation of the initial image can be determined based on the orientation of most texts. This can avoid errors in text recognition or table orientation introduced by individual typos, and increase the fault tolerance of the table detection system.

[0153] Optionally, calculating the weight of each text block includes: calculating the weight of each text block according to distribution information of each text block; the distribution information of the text block includes: the aspect ratio of the text block and / or the density of surrounding text blocks of the text block.

[0154] The distribution information of the text block is used to determine the size and position of the text block in the initial image. The aspect ratio is used to determine the size, and the density of the surrounding text blocks is used to determine the position. The aspect ratio of the text block is used to describe the size ratio of the text block in the image, and the density of the surrounding text blocks is used to describe the density of the text block. The aspect ratio of the text block can determine the weight of the text block from the perspective of the text block itself, and the text block density of the surrounding text blocks can determine the weight of the text block from the perspective of the distribution of the text block. Among them, the density of the surrounding text blocks can be based on the number of surrounding text blocks within a region determined with the text block as the center. Among them, the surrounding text blocks within the region can refer to text blocks whose intersection and union ratio with the region is greater than a preset threshold. For example, a circle can be determined with the text block as the center and the preset length as the radius, and the number of text blocks within the circle is counted as the surrounding text block density. The surrounding text block densities of different text blocks can be normalized, and the normalized data updates the surrounding text block density of each text block.

[0155] Only the aspect ratio may be used as the weight of the text block; or only the density of surrounding text blocks may be used as the weight of the text block; or the weights of the aspect ratio and the density of surrounding text blocks may be preset, and the weighted sum of the aspect ratio and the density of surrounding text blocks may be calculated as the weight of the text block.

[0156] By using the aspect ratio of the text block and the density of the surrounding text blocks as the basis for calculating the text block weight, the weight of the text block can be determined from two dimensions: the size of the text block and the position distribution of the text block, so as to select the orientation of most text, determine the flip angle of the initial image, and improve the accuracy of determining the flip angle.

[0157] Optionally, the flip angle of the initial image is calculated based on the weight of each text block and the flip angle of each text block, including: obtaining a plurality of angle values; for each angle value, obtaining a text block with a flip angle of the angle value and the weight of each text block, and calculating the weight of the angle value; based on the weight of each angle value, selecting an angle value of a target weight as the flip angle of the initial image.

[0158] The multiple angle values ​​are the values ​​of the flip angle. The value range of the flip angle includes multiple angle values. When the flip angle actually detected by the text block is not in the multiple angle values, the angle value closest to the flip angle actually detected by the text block can be determined as the flip angle of the text block.

[0159] For an angle value, count the text blocks with a flip angle of the angle value, and calculate the weight of each text block to obtain the weight of the angle value. The target weight can refer to the most important weight. In addition, the target weight can also be a weight preset by the user. The angle value of the target weight is used as the flip angle of the initial image. Exemplarily, if the maximum weight is the most important weight, the target weight is the maximum weight value; if the minimum weight is the most important weight, the target weight is the minimum weight value.

[0160] By calculating the weight of each angle value based on a preset angle value, the flipping angle of the text block and the weight of the text block, and thus selecting the angle value of the target weight as the flipping angle of the initial image, the flipping angle of the text block can be mapped to a fixed and small number of angle values, and the weights of the angle values ​​can be statistically calculated to determine the flipping angle of the initial image, which can reduce the amount of calculation of the flipping angle and simplify the steps of determining the flipping angle.

[0161] In a specific example, Figure 5 As shown, the initial image is I1, the text detection model is M1, and the text direction judgment model is M2.

[0162] S501 , input the initial image I1 into the text detection model M1 , and obtain K text box coordinates D1 -DK .

[0163] S502 , cutting out text blocks I11 ˜ I1K obtained by cutting out the initial image I1 according to the text frame coordinates D1 ˜ DK .

[0164] S503, inputting I11-I1K into the text direction judgment model M2, and obtaining the flip angle O1-OK of each text block, wherein the value range of O is 0°, 90°, 180° and 270°.

[0165] S504 , calculating weights W11 ˜ W1K corresponding to I11 ˜ I1K according to the aspect ratio of the text block and the density of surrounding text blocks.

[0166] S505. The weights W of the text blocks belonging to each angle are accumulated to calculate the respective weight values ​​of 0°, 90°, 180° and 270°.

[0167] S506 , selecting the angle with the largest weight as , and using it as the rotation angle of the initial image I1 , and rotating the initial image I1 to obtain a table image I2 .

[0168] The accuracy of table orientation judgment can be improved by counting potential rotation angles.

[0169] Figure 6 This is a schematic diagram of the structure of the table generation device provided in the embodiment of the present application. Figure 6 As shown, the table generating device comprises:

[0170] The table image acquisition module 601 is used to acquire the table image;

[0171] A first direction recognition module 602, used to recognize a first direction of the table image to obtain at least one first direction line, where the first direction includes a column direction or a row direction;

[0172] A text box recognition module 603 is used to perform text recognition on the table image to obtain at least one initial text box and text corresponding to each initial text box;

[0173] A target text box identification module 604 is used to merge the initial text boxes according to a second direction to obtain at least one target text box; the second direction is different from the first direction;

[0174] A second direction identification module 605, configured to determine at least one second direction line according to each of the target text boxes;

[0175] A cell determination module 606, configured to determine at least one cell according to each of the first direction lines and each of the second direction lines;

[0176] A cell text determination module 607, used to determine the text corresponding to each of the cells according to the text corresponding to each of the initial text boxes;

[0177] The table generation module 608 is used to determine the table corresponding to the table image according to the cells and the texts corresponding to the cells.

[0178] Optionally, the table generating device further includes:

[0179] A fusion judgment module is used to detect whether a plurality of target text boxes adjacent to each other in the first direction meet a fusion condition according to the association information of the table image before determining at least one second direction line according to each target text box; the association information of the table image includes: key texts corresponding to each target text box and / or scene table structure information corresponding to the table image;

[0180] The text box fusion module is used to fuse the adjacent multiple target text boxes when it is detected that the adjacent multiple target text boxes meet the fusion condition.

[0181] Optionally, the associated information of the table image includes: key texts corresponding to each of the target text boxes;

[0182] The fusion judgment module is specifically used for:

[0183] Detecting whether key texts corresponding to a plurality of target text boxes adjacent to each other in the first direction belong to the same content;

[0184] When the key texts corresponding to the multiple adjacent target text boxes belong to the same content, it is determined that the multiple adjacent target text boxes in the first direction meet the fusion condition.

[0185] Optionally, the associated information of the table image includes: scene table structure information corresponding to the table image;

[0186] The fusion judgment module is specifically used for:

[0187] Detecting whether there are texts in a plurality of adjacent target text boxes at positions corresponding to the scenario table structure information;

[0188] When there is text located at a position corresponding to the scene table structure, it is determined that a plurality of target text boxes adjacent to each other in the first direction meet a fusion condition.

[0189] Optionally, the cell determination module 606 is specifically configured to:

[0190] Determine at least one intersection coordinate according to each of the first direction lines and each of the second direction lines;

[0191] At least one cell is determined according to the coordinates of each intersection point.

[0192] Optionally, the table generating device further includes:

[0193] The text frame spacing adjustment module is used to adjust the spacing of each target text frame in the first direction after merging each of the initial text frames in the second direction to obtain at least one target text frame.

[0194] Optionally, the table generating device further includes:

[0195] A merged cell detection module, configured to detect a plurality of cells that can be merged in each of the cells after determining at least one cell according to each of the first direction lines and each of the second direction lines;

[0196] The cell merging module is used to merge the multiple mergable cells when there are multiple mergable cells to obtain the merged cells, and update each of the cells.

[0197] Optionally, the target text box identification module 604 is specifically used for:

[0198] Merging a plurality of initial text boxes located on the same straight line in the second direction to obtain a target text box;

[0199] A single initial text box on the same straight line in the second direction is determined as a target text box.

[0200] Optionally, the lengths of the initial text boxes in the first direction are similar; and the number of the initial text boxes in the first direction is greater than the number of the initial text boxes in the second direction.

[0201] Optionally, the first direction is a column direction, and the second direction is a row direction.

[0202] Optionally, the cell text determination module 607 is specifically used to:

[0203] Calculating the intersection-and-union ratio between each of the initial text boxes and each of the cells;

[0204] For each of the cells, determining the initial text box corresponding to the cell according to the intersection-and-union ratio between the cell and each of the initial text boxes;

[0205] For each of the cells, the text of the initial text box corresponding to the cell is determined as the text corresponding to the cell.

[0206] Optionally, the table image acquisition module 601 is specifically used for:

[0207] Get the initial image;

[0208] Identify a text frame in the initial image to obtain at least one pre-processed text frame;

[0209] According to each of the pre-processed text boxes, the initial image is cropped to obtain text blocks corresponding to each of the pre-processed text boxes;

[0210] Detecting the flipping angle of each of the text blocks;

[0211] Determining a flip angle of the initial image according to the flip angle of each text block;

[0212] According to the flipping angle of the initial image, the initial image is flipped to obtain a table image.

[0213] Optionally, the table image acquisition module 601 is specifically used for:

[0214] Calculating the weight of each of the text blocks;

[0215] The flip angle of the initial image is calculated according to the weight of each text block and the flip angle of each text block.

[0216] Optionally, the table image acquisition module 601 is specifically used for:

[0217] The weight of each text block is calculated according to the distribution information of each text block; the distribution information of the text block includes: the aspect ratio of the text block and / or the density of surrounding text blocks of the text block.

[0218] Optionally, the table image acquisition module 601 is specifically used for:

[0219] Get multiple angle values;

[0220] For each angle value, obtain a text block with a flip angle equal to the angle value and a weight of each text block, and calculate the weight of the angle value;

[0221] According to the weights of the angle values, the angle value of the target weight is selected to be determined as the flip angle of the initial image.

[0222] The table generating device provided in the embodiment of the present invention can execute the table generating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0223] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0224] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0225] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0226] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a table generation method.

[0227] In some embodiments, the table generation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the table generation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the table generation method in any other appropriate manner (e.g., by means of firmware).

[0228] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0229] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0230] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0231] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0232] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0233] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0234] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the table generation method provided in any embodiment of the present application.

[0235] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0236] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A table generation method, characterized in that: include: Get the table image; Identify a first direction of the table image to obtain at least one first direction line, where the first direction includes a column direction or a row direction; Performing text recognition on the table image to obtain at least one initial text box and text corresponding to each initial text box; Merging the initial text frames according to a second direction to obtain at least one target text frame; the second direction is different from the first direction; Determining at least one second direction line according to each of the target text boxes; Determine at least one cell according to each of the first direction lines and each of the second direction lines; Determine the text corresponding to each of the cells according to the text corresponding to each of the initial text boxes; The table corresponding to the table image is determined according to each of the cells and the text corresponding to each of the cells.

2. The method according to claim 1, characterized in that Before determining at least one second direction line according to each of the target text boxes, the method further includes: According to the association information of the table image, detecting whether a plurality of target text boxes adjacent to each other in the first direction meet the fusion condition; the association information of the table image includes: key texts corresponding to each of the target text boxes and / or scene table structure information corresponding to the table image; When it is detected that multiple adjacent target text boxes meet a fusion condition, the multiple adjacent target text boxes are fused.

3. The method according to claim 2, characterized in that The associated information of the table image includes: key texts corresponding to each of the target text boxes; The detecting, based on the association information of the table image, whether a plurality of target text boxes adjacent to each other in the first direction meet a fusion condition comprises: Detecting whether key texts corresponding to a plurality of target text boxes adjacent to each other in the first direction belong to the same content; When the key texts corresponding to the multiple adjacent target text boxes belong to the same content, it is determined that the multiple adjacent target text boxes in the first direction meet the fusion condition.

4. The method according to claim 2, characterized in that: The associated information of the table image includes: scene table structure information corresponding to the table image; The detecting, based on the association information of the table image, whether a plurality of target text boxes adjacent to each other in the first direction meet a fusion condition comprises: Detecting whether there are texts in a plurality of adjacent target text boxes at positions corresponding to the scenario table structure information; When there is text located at a position corresponding to the scene table structure, it is determined that a plurality of target text boxes adjacent to each other in the first direction meet a fusion condition.

5. The method according to claim 1, characterized in that The determining at least one cell according to each of the first direction lines and each of the second direction lines comprises: Determine at least one intersection coordinate according to each of the first direction lines and each of the second direction lines; At least one cell is determined according to the coordinates of each intersection point.

6. The method according to claim 5, characterized in that After the initial text frames are merged according to the second direction to obtain at least one target text frame, the method further includes: The spacing of each target text box is adjusted in a first direction.

7. The method according to claim 1, characterized in that After determining at least one cell according to each of the first direction lines and each of the second direction lines, the method further includes: In each of the cells, detecting a plurality of cells that can be merged; When there are multiple cells that can be merged, the multiple cells that can be merged are merged to obtain merged cells, and each of the cells is updated.

8. The method according to claim 1, characterized in that The initial text frames are merged according to the second direction to obtain at least one target text frame, including: Merging a plurality of initial text boxes located on the same straight line in the second direction to obtain a target text box; A single initial text box on the same straight line in the second direction is determined as a target text box.

9. The method according to claim 1, characterized in that: The lengths of the initial text boxes in the first direction are similar; the number of the initial text boxes in the first direction is greater than the number of the initial text boxes in the second direction.

10. The method according to claim 9, characterized in that The first direction is a column direction, and the second direction is a row direction.

11. The method according to claim 1, characterized in that: The determining the text corresponding to each of the cells according to the text corresponding to each of the initial text boxes includes: Calculating the intersection-and-union ratio between each of the initial text boxes and each of the cells; For each of the cells, determining the initial text box corresponding to the cell according to the intersection-and-union ratio between the cell and each of the initial text boxes; For each of the cells, the text of the initial text box corresponding to the cell is determined as the text corresponding to the cell.

12. The method according to claim 1, characterized in that The step of obtaining a table image comprises: Get the initial image; Identify a text frame in the initial image to obtain at least one pre-processed text frame; According to each of the pre-processed text boxes, the initial image is cropped to obtain text blocks corresponding to each of the pre-processed text boxes; Detecting the flipping angle of each of the text blocks; Determining a flip angle of the initial image according to the flip angle of each text block; According to the flipping angle of the initial image, the initial image is flipped to obtain a table image.

13. The method according to claim 12, characterized in that Determining the flip angle of the initial image according to the flip angle of each text block includes: Calculating the weight of each of the text blocks; The flip angle of the initial image is calculated according to the weight of each text block and the flip angle of each text block.

14. The method according to claim 13, characterized in that The calculating the weight of each text block includes: The weight of each text block is calculated according to the distribution information of each text block; the distribution information of the text block includes: the aspect ratio of the text block and / or the density of surrounding text blocks of the text block.

15. The method according to claim 13, characterized in that The calculating the flip angle of the initial image according to the weight of each text block and the flip angle of each text block comprises: Get multiple angle values; For each angle value, obtain a text block with a flip angle equal to the angle value and a weight of each text block, and calculate the weight of the angle value; According to the weights of the angle values, the angle value of the target weight is selected to be determined as the flip angle of the initial image.

16. A table generating device, characterized in that: include: A table image acquisition module, used for acquiring a table image; A first direction recognition module, used to recognize a first direction of the table image to obtain at least one first direction line, where the first direction includes a column direction or a row direction; A text box recognition module, used for performing text recognition on the table image to obtain at least one initial text box and text corresponding to each initial text box; A target text box identification module, used for fusing the initial text boxes according to a second direction to obtain at least one target text box; the second direction is different from the first direction; A second direction identification module, used for determining at least one second direction line according to each of the target text boxes; A cell determination module, used for determining at least one cell according to each of the first direction lines and each of the second direction lines; A cell text determination module, used to determine the text corresponding to each of the cells according to the text corresponding to each of the initial text boxes; The table generation module is used to determine the table corresponding to the table image according to the cells and the texts corresponding to the cells.

17. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the table generation method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the table generation method according to any one of claims 1 to 15.

19. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the table generating method according to any one of claims 1 to 15.

Images