Table restoration method and device and electronic equipment

Abstract

The invention discloses a table restoration method and device and electronic equipment. The table restoration method comprises the steps that a first table picture is acquired; identifying the first table picture to obtain a first object, first position information of the first object, a second object and second position information of the second object; based on the first position information and the second position information, the first objects and the second objects are matched, a plurality of cells are obtained, and the number of the first objects included in the cells is 1; and based on the constraint condition, restoring the plurality of cells into a second table picture, the first table picture and the second table picture having the same structure, the table in the first table picture being non-editable, and the table in the second table picture being editable.

Description

Technical Field

[0001] This application relates to the technical field of table processing, and particularly relates to a table restoration method, apparatus, and electronic device. Background Art

[0002] In modern information processing and data management systems, tables, as an efficient data representation and storage form, are widely used in various fields, including financial statements, statistical data analysis, database management, and document processing. However, there are some tables that cannot be directly edited, such as tables in the form of pictures. In this case, it is necessary to perform structure recognition on the tables in order to restore editable tables.

[0003] Currently, table structure recognition algorithms based on machine learning technology can be roughly divided into three types: encoding-decoding, bottom-up paradigm, and splitting-merging. However, none of the above three algorithms can restore the table to an editable format table. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide a table restoration method, apparatus, and electronic device.

[0005] In a first aspect, the embodiments of this application provide a table restoration method, including: Obtain a first table picture; Recognize the first table picture to obtain a first object and its first position information, and a second object and its second position information; Based on the first position information and the second position information, match the first object with the second object to obtain multiple cells, where the number of first objects included in each cell is 1; Based on constraint conditions, restore multiple cells to a second table picture. The first table picture and the second table picture have the same structure. The table in the first table picture is not editable, and the table in the second table picture is editable.

[0006] In a possible implementation manner, the step of recognizing the first table picture to obtain a first object and its first position information, and a second object and its second position information includes: Recognize the first table picture to obtain the first object and the second object; Determine the coordinates of the detection frame of the first object as the first position information, and determine the coordinates of the detection frame of the second object as the second position information.

[0007] In a possible implementation manner, the step of matching the first object with the second object based on the first position information and the second position information to obtain multiple cells includes: Determine a first area corresponding to the first object based on the first position information, and determine a second area corresponding to the second object based on the second position information; For each of the first areas, calculate the first area and each of the second areas respectively according to a preset algorithm to obtain a second area that matches the first area; Determine the matching first area and second area as a cell.

[0008] In a possible implementation manner, before restoring multiple cells to a second table picture based on the constraint conditions, it includes: Based on the first position information and / or the second position information, determine the relative position information and attribute information of the cell.

[0009] In a possible implementation manner, the first position information at least includes a first coordinate of a specified point in the detection frame of the first object, and the second position information at least includes a second coordinate of a specified point in the detection frame of the second object; Restoring multiple cells to a second table picture based on the constraint conditions includes: Determine a table structure based on the first coordinate and / or the second coordinate; Based on the constraint conditions, the table structure, the relative position information, and the attribute information, determine label information of each cell, where the label information is used to characterize merge information of the cell in the row direction and / or merge information in the column direction; Based on the table structure and the label information, restore multiple cells to the second table picture.

[0010] In a possible implementation manner, the table restoration method further includes: The constraint conditions at least include that cells in the same row have the same height, cells in the same column have the same width, and the height difference and / or width difference between adjacent cells are less than a threshold.

[0011] In a possible implementation manner, When recognizing the first table picture, also obtain a third object and its third position information; Upload the third object to a target server; Obtain a uniform resource locator returned by the target server, where the uniform resource locator is used to link the third object.

[0012] In a possible implementation manner, the restoring multiple cells to the second table picture based on the table structure and the label information includes: Convert the table structure, the label information, the cell, the third position information, and the uniform resource locator to obtain a hypertext markup language string; Perform visualization processing on the hypertext markup language string to obtain the second table picture.

[0013] In a second aspect, an embodiment of the present application further provides a table restoration device, including: An acquisition module configured to acquire a first table picture; An identification module configured to identify the first table picture to obtain a first object and its first position information, and a second object and its second position information; A matching module configured to match the first object with the second object based on the first position information and the second position information to obtain a plurality of cells, where the number of first objects included in each cell is 1; A restoration module configured to restore a plurality of the cells to a second table picture based on constraint conditions, where the first table picture has the same structure as the second table picture, the table in the first table picture is not editable, and the table in the second table picture is editable.

[0014] In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor and a memory, where the memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through a bus. When the machine-readable instructions are executed by the processor, the steps of the table restoration method described in any one of the above are performed.

[0015] In the embodiment of the present application, the first object and the second object recognized are first matched to obtain a plurality of cells, where the first object is a table border line and the second object is text; then, a plurality of cells are restored to a second table picture based on the constraint conditions set for the table. Moreover, the table in the first table picture is not editable, and the table in the second table picture is editable. That is to say, the table restoration method in the embodiment of the present application can restore an uneditable table to an editable table, which is convenient for users to edit and use. In addition, compared with the existing table structure recognition algorithm, the table restoration method in the embodiment of the present application does not require a large amount of data training, effectively saving computing resources and reducing R & D costs. Description of the Drawings

[0016] To more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments described in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0017] Figure 1 The flowchart of a table restoration method provided by the present application is shown; Figure 2 The schematic diagram of a rectangular frame provided by the present application is shown; Figure 3 The schematic diagram of a cell provided by the present application is shown; Figure 4 The flowchart of matching a first object with a second object to obtain multiple cells based on the first position information and the second position information in a table restoration method provided by the present application is shown; Figure 5 The flowchart of restoring multiple cells to a second table picture in a table restoration method provided by the present application is shown; Figure 6 The schematic diagram of a table provided by the present application is shown; Figure 7A The schematic diagram of a first table picture provided by the present application is shown; Figure 7B Shows Figure 7A The hypertext markup language string corresponding to the first table picture in; Figure 7C Shows Figure 7A The schematic diagram of the second table picture corresponding to the first table picture in; Figure 8 The schematic diagram of the structure of a table restoration device provided by the present application is shown; Figure 9 The schematic diagram of the structure of an electronic device provided by the present application is shown. Detailed implementation manners

[0018] Reference is made herein to the various solutions and features of the present application with reference to the accompanying drawings.

[0019] It should be understood that various modifications can be made to the embodiments applied herein. Therefore, the above specification should not be regarded as a limitation, but only as an example of the embodiments. Those skilled in the art will think of other modifications within the scope and spirit of the present application.

[0020] The drawings included in and forming a part of the specification illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, are used to explain the principles of the present application.

[0021] These and other features of the present application will become apparent from the following description of the preferred forms of the embodiments given as non - limiting examples with reference to the drawings.

[0022] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art can surely implement many other equivalent forms of the present application, which have the features as described in the claims and thus are all within the scope of protection defined thereby.

[0023] When combined with the drawings, the above - mentioned and other aspects, features, and advantages of the present application will become more apparent in view of the following detailed description.

[0024] Specific embodiments of the present application will be described hereinafter with reference to the drawings; however, it should be understood that the claimed embodiments are merely examples of the present application and can be implemented in various ways. Well - known and / or repetitive functions and structures are not described in detail to avoid obscuring the present application with unnecessary or redundant details. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but are merely a basis and representative basis for the claims to teach those skilled in the art to use the present application in substantially any suitable detailed structure in a variety of ways.

[0025] This specification may use the phrases "in one embodiment", "in another embodiment", "in yet another embodiment", or "in other embodiments", which may each refer to one or more of the same or different embodiments according to the present application.

[0026] For the convenience of understanding the present application, a method for restoring a table provided by the present application will be introduced in detail first.

[0027] Figure 1 A flowchart of the table restoration method provided by the embodiment of the present application is shown, where the specific steps include S101 - S104.

[0028] S101, obtain a first table picture.

[0029] In a specific implementation, the first table picture includes at least a table, and the table is a framed and closed table. Of course, the first table picture may also include a title, a number, etc. of the table.

[0030] Among them, the first table picture can be directly intercepted. For example, when browsing documents such as literature and test papers, an interception operation is performed on a specific table to obtain the first table picture; the first table picture can also be obtained by recognizing and extracting the whole picture after obtaining the whole picture; the first table picture can also be collected by the user through an image acquisition device, etc.

[0031] S102. Recognize the first table picture to obtain the first object and its first position information, and the second object and its second position information.

[0032] Further, after obtaining the first table picture, recognize the first table picture. Among them, the first object obtained by recognizing the first table picture is the table frame line, and the second object is the text. Both the table frame line and the text are obtained by performing object detection on the picture by using object detection means.

[0033] Optionally, in the embodiment of the present application, the first table picture is recognized and processed by a detection model. Specifically, when recognizing the first table picture, the table frame line and the text are used as the targets for recognition, so as to obtain the first object and the second object. As an example, the table structure recognition technology included in the detection model, such as Cycle-CenterNet, is used to recognize the table frame line to obtain the first object; the table structure recognition model included in the detection model, such as optical character recognition (OCR) technology, is used to recognize the text to obtain the second object. Of course, the detection model can also use deep neural networks such as YOLO to perform the above recognition processing, etc. The embodiment of the present application does not make specific limitations on this.

[0034] After recognizing the first table picture, while obtaining the first object and the second object, the first position information of the first object and the second position information of the second object are also obtained. Specifically, when the detection target is a table, the coordinates of the detection frame of the detection target are determined as the first position information of the first object; when the detection target is text, the coordinates of the detection frame of the detection target are determined as the second position information of the second object. Correspondingly, the first position information of the first object can be the coordinates of the four corners of the table frame, or the coordinates of the center point and the coordinates of any two corners, or the coordinates of any two opposite angles, etc. The first position information in the embodiment of the present application includes the coordinates of the upper left corner and the lower right corner in the table frame. Similarly, the second position information includes the coordinates of the upper left corner and the lower right corner in the text frame. Among them, as an example Figure 2 A rectangular frame is shown, where angle A is the upper left corner of the rectangular frame and angle B is the lower right corner of the rectangular frame.

[0035] S103. Based on the first position information and the second position information, match the first object with the second object to obtain a plurality of cells, and the number of the first objects included in each cell is 1.

[0036] After obtaining the first object and its first position information, and the second object and its second position information, based on the first position information and the second position information, the first object is matched with the second object. Specifically, for each first object, it is respectively matched with each second object to obtain a second object that matches the first object, and the first object and the second object form a cell. At this time, referring to Figure 3 the schematic diagram of the cell shown, the number of first objects and the number of second objects included in the cell are both 1.

[0037] It should be noted that since there may be differences in the number of texts included and the means of text detection in a single cell of each table, therefore, the number of first objects included in each determined cell is 1, while the number of second objects included in each cell can be 0, 1, or multiple.

[0038] As an example, Figure 4 the flowchart showing the matching of the first object and the second object based on the first position information and the second position information to obtain multiple cells is shown, where the specific steps include S401 - S403.

[0039] S401, determining the first region corresponding to the first object based on the first position information, and determining the second region corresponding to the second object based on the second position information.

[0040] S402, for each first region, calculating the first region and each second region respectively according to a preset algorithm to obtain a second region that matches the first region.

[0041] S403, determining the matched first region and second region as a cell.

[0042] In a specific implementation, based on the first position information, the first region corresponding to the first object is determined. Referring to the above, the first position information represents the detection frame generated when detecting the first object. Based on this, the region corresponding to the detection frame is determined as the first region corresponding to the first object, that is, the first object is located within this first region. Similarly, when determining the second region corresponding to the second object based on the second position information, the region corresponding to the detection frame generated when detecting the second object is determined as the second region corresponding to the second object, and the second object is located within this second region.

[0043] Furthermore, for each first region, the first region and each second region are respectively calculated according to a preset algorithm to obtain a second region that matches the first region. Specifically, the preset algorithm refers to the following formula (1), and the second region that matches the first region is calculated through formula (1).

[0044] (1) in, Indicates the score of the first region matching the second region, represents the overlapping area between the first area and the second area, Indicates the second area.

[0045] In the embodiment of the present application, when the above calculation is performed for the first object, if If the value is greater than 0.9, it is determined that the corresponding second region matches the first region corresponding to the first object.

[0046] After determining a second region matching each first region, the matching first region and the second region are determined to be a unit cell.

[0047] S104, based on the constraint conditions, restoring the plurality of cells into a second table image, wherein the first table image and the second table image have the same structure, the table in the first table image is not editable, and the table in the second table image is editable.

[0048] Furthermore, after obtaining the plurality of cells, the plurality of cells are restored to the second table image based on the constraint conditions. Here, before restoring the plurality of cells to the second table image based on the constraint conditions, the relative position information and attribute information of the cells are first determined.

[0049] In the case where a cell includes a first object and a second object, the first position information and / or the second position information can both represent the position information of the cell. Based on this, the position information of each cell is determined based on the first position information and / or the second position information, and then the relative position information between multiple cells is determined, such as the layout of multiple cells in a first table image. At the same time, the attribute information of each cell, such as the height and width of the cell, can also be determined based on the first position information and / or the second position information.

[0050] It should be noted that, if the cell includes the first object but does not include the second object, the relative position information and attribute information of the cell are determined based on the first position information.

[0051] Optionally, refer to Figure 5 The flowchart shown restores multiple cells into a second table image, wherein the specific steps include S501-S503.

[0052] S501: Determine a table structure based on the first coordinate and / or the second coordinate.

[0053] S502 : Determine label information of each cell based on the constraint conditions, table structure, relative position information, and attribute information. The label information is used to represent the merge information of the cell in the row direction and / or the merge information in the column direction.

[0054] S503: Based on the table structure and label information, restore the multiple cells into a second table image.

[0055] Optionally, after obtaining multiple cells, the distances to the four vertices of the cell center are regressed from the cell center point to decode the cell bounding box (BBox), thereby decoding multiple discrete cells. Furthermore, based on the cell vertices, the distances to the center points of cells that share that vertex are regressed to decode the cell group box (GBox), thereby concatenating the discrete BBoxes to obtain a continuous cell GBox.

[0056] In this embodiment of the present application, the first position information includes at least the first coordinates of a specified point in the detection frame of the first object, and the second position information includes at least the second coordinates of a specified point in the detection frame of the second object. For example, the specified point is the vertex in the upper left corner of the detection frame. Based on this, the table structure is determined based on the first coordinates and / or the second coordinates.

[0057] As an example, a basic structure is initialized, which is an empty two-dimensional array, and each element in the array represents the row and column coordinates of a cell in the basic structure. Furthermore, for all cells, the table structure of the table in the first table image is determined based on the first coordinate and / or the second coordinate of each cell. Next, the table structure is determined based on the first coordinate as an example. Specifically, the first coordinate with the smallest y value is selected from all the first coordinates, and the cell corresponding to the first coordinate with the smallest y value forms the first row of the table structure; further, based on the y value in the first coordinate, the cell closer to the upper boundary is selected as the second row, wherein the cells in each row can be determined by referring to the following formula (2).

[0058] (2) in, Indicates the comparison value, represents the y value of cell 2, represents the y value of cell 1, Indicates the height of the cell.

[0059] Calculate the comparison value based on two cells Afterwards, confirm If the size between is less than or equal to the threshold, then it is determined that the two cells belong to the same row; if If the threshold is equal to the value of the two cells, it is determined that the two cells do not belong to the same row. The threshold in the embodiment of the present application is preferably 0.2.

[0060] After determining the cells included in each row, the cells in each row are sorted in ascending order according to the x value in the first coordinate of each cell, and cells with similar x values are deduplicated and compared to avoid the influence of differences in cell size on the sorting of cells in each row.

[0061] Afterwards, for each row, the first coordinate of the cell is added to the basic structure according to the sorting to obtain a table structure.

[0062] It is worth noting that, the above method may be used to first determine the cells in each column, and then sort each cell based on the y value of the first coordinate to obtain a table structure.

[0063] Furthermore, based on the constraints, table structure, relative position information and attribute information, the label information of each cell is determined. The constraints include at least that the cells in the same row have the same height, the cells in the same column have the same width, and the height difference and / or width difference between adjacent cells is less than a threshold. Specifically, each cell is traversed, and for each cell, it is compared with the adjacent cells to determine whether there is a merge between it and the adjacent cells, so as to determine the label information of the cell based on the obtained merge situation. The label information is used to characterize the merge information of the cell in the row direction and / or the merge information in the column direction.

[0064] Among them, adjacent cells are determined based on relative position relationships, and the relative position relationships of adjacent cells compared to the cell are up, down, left, and right. It should be known that due to the characteristics of the table, there are cells that correspond to two adjacent cells, three adjacent cells, or four adjacent cells. Figure 6 A schematic diagram of a table is shown, in which cell x corresponds to two adjacent cells, cell y corresponds to three adjacent cells, and cell z corresponds to four adjacent cells.

[0065] For example, for cell z, it is compared with its adjacent cells to determine that the adjacent cell to the right of cell z is merged. Moreover, the height of the adjacent cell is determined to be twice the height of cell z. Therefore, the label information of the adjacent cell is determined to be colspan2, which indicates that the adjacent cell is merged into two rows in the column direction. For another example, if the label information is rowspan3, it indicates that the cell is merged into three rows in the row direction.

[0066] It should be noted that the above tag information is obtained by traversing each cell. Therefore, for the same cell, different tag information may be obtained. For example, for cell w, the tag information of cell w is determined to be colspan3 by the adjacent cell whose relative position to cell w is to the left, and the tag information of cell w is determined to be colspan2 by the adjacent cell whose relative position to cell w is to the left. At this time, the maximum value is taken as the tag information of cell w, that is, colspan3.

[0067] In a specific implementation, for each cell, after comparing it with adjacent cells and obtaining the tag information of each cell, for each cell, based on the constraints that the heights of cells in the same row are the same and the widths of cells in the same column are the same, cell traversal is performed on the row to which the cell belongs and cell traversal is performed on the column to which the cell belongs, so as to determine whether the tag information of the cell is correct. If it is incorrect, it is modified, thereby ensuring the accuracy of the tag information of each cell.

[0068] Based on the table structure and tag information, multiple cells are restored to a second table picture. Optionally, the table structure, tag information, and cells are converted to obtain a Hypertext Markup Language string. Among them, the cell information in the Hypertext Markup Language string is arranged in the order from left to right and from top to bottom.

[0069] After that, the Hypertext Markup Language string is visually processed to obtain a second table picture. The table in the second table picture is editable, that is, the table restoration method of the embodiments of the present application can restore an uneditable table to an editable table, which is convenient for users to edit and use.

[0070] As an example, Figure 7A shows a schematic diagram of a first table picture, Figure 7B shows Figure 7A the Hypertext Markup Language string corresponding to the first table picture in Figure 7C shows Figure 7A the schematic diagram of the second table picture corresponding to the first table picture in

[0071] Considering that in addition to text, the table may also contain subgraphs. In this case, after recognizing the first table picture, a third object and its third position information are also obtained. The third object is the subgraph in the table.

[0072] In the embodiments of the present application, for the third object, the third object is uploaded to the target server. The target server will generate a corresponding Uniform Resource Locator for the third object, and the Uniform Resource Locator is used to link its corresponding third object.

[0073] After that, a uniform resource locator returned by the target server is obtained.

[0074] In a scenario where a sub - figure is included in a table, when restoring multiple cells to a second table picture based on the table structure and label information in the flowchart, the table structure, label information, cells, third - position information, and uniform resource locator are converted to obtain a hypertext markup language string. Similarly, the hypertext markup language string is visually processed to obtain the second table picture.

[0075] The table restoration method of the embodiments of the present application can restore an uneditable table to an editable table, which is convenient for users to edit and use. Moreover, compared with the existing table - structure recognition algorithms, the table restoration method of the embodiments of the present application does not require a large amount of data training and can be implemented using the current algorithms, effectively saving computing resources and reducing R & D costs.

[0076] Based on the same inventive concept, the second aspect of the present application also provides a table restoration device corresponding to the table restoration method. Since the principle of solving problems by the table restoration device in the present application is similar to that of the above - mentioned table restoration method of the present application, the implementation of the table restoration device can refer to the implementation of the method, and the repeated parts will not be elaborated.

[0077] Figure 8 The structural schematic diagram of the table restoration device provided by the embodiments of the present application is shown, which specifically includes: An acquisition module 801, configured to acquire a first table picture; An identification module 802, configured to identify the first table picture to obtain a first object and its first - position information, and a second object and its second - position information; A matching module 803, configured to match the first object with the second object based on the first - position information and the second - position information to obtain multiple cells, where the number of the first objects included in each cell is 1; A restoration module 804, configured to restore multiple cells to a second table picture based on constraint conditions, where the first table picture and the second table picture have the same structure, the table in the first table picture is uneditable, and the table in the second table picture is editable.

[0078] In another example, the identification module 802 is specifically configured to: Identify the first table picture to obtain the first object and the second object; Determine the coordinates of the detection frame of the first object as the first - position information, and determine the coordinates of the detection frame of the second object as the second - position information.

[0079] In yet another example, the matching module 803 is specifically configured to: Determine a first area corresponding to the first object based on the first position information, and determine a second area corresponding to the second object based on the second position information; For each of the first areas, calculate the first area and each of the second areas respectively according to a preset algorithm to obtain a second area that matches the first area; Determine the matched first area and second area as a cell.

[0080] In yet another example, the table restoration device further includes a determination module 805, which is configured to: Determine the relative position information and attribute information of the cell based on the first position information and / or the second position information.

[0081] In yet another example, the first position information at least includes the first coordinates of a specified point in the detection frame of the first object, and the second position information at least includes the second coordinates of a specified point in the detection frame of the second object; the restoration module 804 is specifically configured to: Determine the table structure based on the first coordinates and / or the second coordinates; Based on the constraint conditions, the table structure, the relative position information, and the attribute information, determine the label information of each cell, where the label information is used to represent the merge information of the cell in the row direction and / or the merge information in the column direction; Based on the table structure and the label information, restore multiple cells to the second table picture.

[0082] In yet another example, the constraint conditions at least include that the heights of the cells in the same row are the same, the widths of the cells in the same column are the same, and the height difference and / or width difference between adjacent cells are less than a threshold.

[0083] In yet another example, the recognition module 802 is further configured to: Recognize the first table picture and also obtain a third object and its third position information; Upload the third object to the target server; Obtain a uniform resource locator returned by the target server, where the uniform resource locator is used to link the third object.

[0084] In yet another example, the restoration module 804 is further configured to: Convert the table structure, the label information, the cells, the third position information, and the uniform resource locator to obtain a hypertext markup language string; Visualize the hypertext markup language string to obtain the second table picture.

[0085] In the embodiments of the present application, first, the identified first object is matched with the second object to obtain a plurality of cells, where the first object is a table border line and the second object is text; then, based on the constraint conditions set for the table, the plurality of cells are restored to the second table picture, and the table in the first table picture is not editable, while the table in the second table picture is editable. That is to say, the table restoration method in the embodiments of the present application can restore an uneditable table to an editable table, which is convenient for users to edit and use. Moreover, compared with the existing table structure recognition algorithms, the table restoration method in the embodiments of the present application does not require a large amount of data training, effectively saving computing resources and reducing R & D costs.

[0086] The embodiments of the present application provide a storage medium, which is a computer-readable medium and stores a computer program. When the computer program is executed by a processor, it implements the method provided in any embodiment of the present application, including the following steps S11 to S14: S11, obtain the first table picture; S12, identify the first table picture to obtain the first object and its first position information, and the second object and its second position information; S13, based on the first position information and the second position information, match the first object with the second object to obtain a plurality of cells, and the number of the first objects included in each cell is 1; S14, based on the constraint conditions, restore the plurality of cells to the second table picture, the first table picture and the second table picture have the same structure, the table in the first table picture is not editable, and the table in the second table picture is editable.

[0087] The embodiments of the present application provide an electronic device, and the structural schematic diagram of the electronic device can be as Figure 9 shown, at least including a memory 901 and a processor 902. A computer program is stored on the memory 901, and when the processor 902 executes the computer program on the memory 901, it implements the method provided in any embodiment of the present application. Exemplarily, the computer program steps of the electronic device are as follows S21 to S24: S21, obtain the first table picture; S22, identify the first table picture to obtain the first object and its first position information, and the second object and its second position information; S23, based on the first position information and the second position information, match the first object with the second object to obtain a plurality of cells, and the number of the first objects included in each cell is 1; S24. Based on the constraint conditions, restore the multiple cells to a second table picture. The first table picture has the same structure as the second table picture. The table in the first table picture is not editable, while the table in the second table picture is editable.

[0088] In the embodiment of the present application, first, match the recognized first object and second object to obtain multiple cells. Herein, the first object is a table frame line, and the second object is text. Then, based on the constraint conditions set for the table, restore the multiple cells to a second table picture. Moreover, the table in the first table picture is not editable, and the table in the second table picture is editable. That is to say, the table restoration method in the embodiment of the present application can restore an uneditable table to an editable table, which is convenient for users to edit and use. And, compared with the existing table structure recognition algorithms, the table restoration method in the embodiment of the present application does not require a large amount of data training, effectively saving computing resources and reducing R & D costs.

[0089] Optionally, in this embodiment, the above storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk, or optical disc, etc., various media that can store program codes. Optionally, in this embodiment, the processor executes the method steps described in the above embodiment according to the program codes stored in the storage medium. Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiment and optional implementation manners, and will not be elaborated herein. Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described may be executed in a different order than here, or they may be separately fabricated into individual integrated circuit modules, or multiple of them may be fabricated into a single integrated circuit module to be implemented. Thus, the present application is not limited to any specific combination of hardware and software.

[0090] Moreover, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application that have equivalent elements, modifications, omissions, combinations (e.g., schemes that cross various embodiments), adaptations, or alterations. The elements in the claims will be broadly interpreted based on the language employed in the claims and are not limited to the examples described in this specification or during the implementation of the present application, and the examples will be construed as non-exclusive. Thus, this specification and the examples are intended to be considered only as examples, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0091] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of their schemes) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. Additionally, in the above detailed description, various features can be grouped together to simplify the present application. This should not be construed as an intention that a disclosed feature not claimed is necessary for any claim. On the contrary, the subject matter of the present application can be less than all the features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the detailed description as examples or embodiments, where each claim independently serves as a separate embodiment, and it is contemplated that these embodiments can be combined with each other in various combinations or permutations. The scope of the present application should be determined with reference to the appended claims and the full scope of the equivalent forms empowered by these claims.

[0092] The above has described in detail multiple embodiments of the present application, but the present application is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications to the embodiments based on the concept of the present application, and these variations and modifications should all fall within the scope claimed by the present application.

Claims

1. A method for restoring a table, characterized in that, Including: Obtain a first table picture; Identify the first table picture to obtain a first object and its first position information, and a second object and its second position information; Based on the first position information and the second position information, match the first object with the second object to obtain a plurality of cells, where the number of first objects included in each cell is 1; Based on constraint conditions, restore the plurality of cells to a second table picture. The first table picture and the second table picture have the same structure. The table in the first table picture is not editable, and the table in the second table picture is editable.

2. The table restoration method according to claim 1, wherein The step of identifying the first table picture to obtain a first object and its first position information, and a second object and its second position information includes: Identify the first table picture to obtain the first object and the second object; Determine the coordinates of the detection frame of the first object as the first position information, and determine the coordinates of the detection frame of the second object as the second position information.

3. The table restoration method according to claim 1, characterized in that, The step of matching the first object with the second object based on the first position information and the second position information to obtain a plurality of cells includes: Based on the first position information, determine a first area corresponding to the first object, and based on the second position information, determine a second area corresponding to the second object; For each first area, calculate the first area and each second area respectively according to a preset algorithm to obtain a second area that matches the first area; Determine the matching first area and second area as a cell.

4. The table restoration method according to claim 3, wherein Before restoring the plurality of cells to the second table picture based on the constraint conditions, it includes: Based on the first position information and / or the second position information, determine the relative position information and attribute information of the cell.

5. The table restoration method according to claim 4, characterized in that, The first position information at least includes the first coordinates of a specified point in the detection frame of the first object, and the second position information at least includes the second coordinates of a specified point in the detection frame of the second object; The step of restoring the plurality of cells to the second table picture based on the constraint conditions includes: Based on the first coordinates and / or the second coordinates, determine the table structure; Based on the constraint conditions, the table structure, the relative position information, and the attribute information, determine the label information of each cell. The label information is used to represent the merging information of the cell in the row direction and / or the merging information in the column direction; Based on the table structure and the label information, restore the plurality of cells to the second table picture.

6. The table restoration method according to claim 5, characterized in that It also includes: The constraint conditions at least include that the heights of cells in the same row are the same, the widths of cells in the same column are the same, and the height difference and / or width difference between adjacent cells are less than a threshold.

7. The table restoration method according to claim 5, characterized in that When identifying the first table picture, a third object and its third position information are also obtained; Upload the third object to a target server; Obtain a uniform resource locator returned by the target server. The uniform resource locator is used to link the third object.

8. The table restoration method according to claim 7, wherein, Restoring a plurality of the cells to the second table picture based on the table structure and the tag information includes: Converting the table structure, the tag information, the cells, the third position information, and the uniform resource locator to obtain a hypertext markup language string; Performing a visualization process on the hypertext markup language string to obtain the second table picture.

9. A table restoration device, characterized in that, Including: An acquisition module configured to acquire a first table picture; An identification module configured to identify the first table picture to obtain a first object and its first position information and a second object and its second position information; A matching module configured to match the first object with the second object based on the first position information and the second position information to obtain a plurality of cells, where the number of the first objects included in each cell is 1; A restoration module configured to restore a plurality of the cells to a second table picture based on a constraint condition, where the first table picture has the same structure as the second table picture, the table in the first table picture is non-editable, and the table in the second table picture is editable.

10. An electronic device, characterized in that, Including: A processor and a memory, where the memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through a bus. When the machine-readable instructions are executed by the processor, the steps of the table restoration method according to any one of claims 1 to 8 are performed.

Images