Table Recognition and Reconstruction Method, Apparatus, Device, Medium and Program Product

Abstract

An embodiment of this specification discloses a method, apparatus, device, medium, and program product for table recognition and reconstruction. Among them, the method includes: reconstructing the table in the target image based on the target text in the target image obtained by performing text recognition on the target image including the table, the position of each target character in the target text corresponding to the target image, at least four target line segments obtained by performing line detection on the table in the target image, and the position corresponding to each of the target line segments.

Description

Technical Field

[0001] This specification relates to the technical field of data processing, and in particular, to a method, apparatus, device, medium, and program product for form recognition and reconstruction. Background Art

[0002] In various industries, there is a large amount of form extraction and entry work. For example, in the insurance industry, claims settlement requires extracting invoice form details and related information. If relying solely on manual extraction, not only does it require high labor costs, but it is also prone to extraction errors. Currently, in the actual process of machine invoice form recognition, the invoice form layouts are diverse, and both rule-based invoice form recognition methods or traditional image recognition methods need to continuously adjust the extraction rules or the thresholds set by traditional image algorithms to meet the needs of recognizing new invoice form types. Summary of the Invention

[0003] Embodiments of this specification provide a method, apparatus, device, medium, and program product for form recognition and reconstruction. By performing line detection on the form in the target image, the reconstruction of the form in the target image is achieved, which not only improves the robustness of form recognition and reconstruction, but also enables enterprises or individuals to achieve form recognition without adjusting thresholds and rules in different scenarios, reducing the cost of information extraction for enterprises or individuals and improving the efficiency of form recognition and reconstruction. The technical solutions are as follows:

[0004] In a first aspect, embodiments of this specification provide a method for form recognition and reconstruction, including:

[0005] Obtain a target image containing a form;

[0006] Perform optical character recognition on the above target image to obtain the target text in the above target image and the position of each target character in the above target text in the above target image; the above target text includes at least one of the above target characters;

[0007] Perform line detection on the form in the above target image to obtain a line detection result corresponding to the above form; the above line detection result includes at least four target line segments and the position corresponding to each of the above target line segments;

[0008] Reconstruct the form in the above target image according to the above target text, the position of each target character in the above target text in the above target image, and the above line detection result.

[0009] In a possible implementation manner, after obtaining the target image and before performing optical character recognition on the above target image to obtain the target text in the above target image and the position of each target character in the above target text in the above target image, the method further includes:

[0010] Preprocess the above target image to obtain a preprocessed target image;

[0011] Perform text recognition on the above target image to obtain the target text in the above target image and the position of each target character in the above target text corresponding to the above target image, including:

[0012] Perform text recognition on the preprocessed target image above to obtain the target text in the above target image and the position of each target character in the above target text corresponding to the preprocessed target image above;

[0013] Perform line detection on the table in the above target image to obtain the line detection result corresponding to the above table, including:

[0014] Perform line detection on the table in the preprocessed target image above to obtain the line detection result corresponding to the above table.

[0015] In a possible implementation, the above preprocessing includes performing quality enhancement processing on the above target image and / or performing image correction processing on the above target image.

[0016] In a possible implementation, the above reconstructing the table in the above target image according to the above target text and the position of each target character in the above target text corresponding to the above target image and the above line detection result includes:

[0017] Delete the duplicate target line segments in the above line detection result to obtain a set of non-duplicate target line segments; the set of non-duplicate target line segments includes at least four non-duplicate target line segments;

[0018] Reconstruct the table in the above target image according to the above target text and the position of each target character in the above target text corresponding to the above target image and the above set of non-duplicate target line segments and the position of each non-duplicate target line segment in the above set of non-duplicate target line segments.

[0019] In a possible implementation, before the above deleting the duplicate target line segments in the above line detection result, the above method further includes:

[0020] Determine the duplicate target line segments in the above line detection result according to the position of each target line segment in the above line detection result.

[0021] In a possible implementation, before the above deleting the duplicate target line segments in the above line detection result, the above method further includes:

[0022] Calculate the similarity between every two of the above target line segments in the above line detection result;

[0023] If the above similarity is greater than a preset threshold, then according to the first preset rule, one of the two target line segments corresponding to the above similarity is determined as the repeated target line segment.

[0024] In a possible implementation manner, the above-mentioned reconstruction of the table in the target image according to the above-mentioned target text, the positions corresponding to each target character in the target text in the target image, the above-mentioned non-repeated target line segment set, and the positions corresponding to each non-repeated target line segment in the non-repeated target line segment set includes:

[0025] Based on the positions corresponding to each non-repeated target line segment in the non-repeated target line segment set, the non-repeated target line segments in the non-repeated target line segment set are integrated according to the second preset rule to obtain at least one target cell and the coordinate information corresponding to the at least one target cell;

[0026] Based on the coordinate information corresponding to the at least one target cell and the positions corresponding to each target character in the target text in the target image, the target text and the at least one target cell are integrated according to the third preset rule to obtain the table in the target image.

[0027] In a possible implementation manner, the above-mentioned based on the positions corresponding to each non-repeated target line segment in the non-repeated target line segment set, the non-repeated target line segments in the non-repeated target line segment set are integrated according to the second preset rule to obtain at least one target cell and the coordinate information corresponding to the at least one target cell, including:

[0028] According to the positions corresponding to each non-repeated target line segment in the non-repeated target line segment set, the non-repeated target line segments in the non-repeated target line segment set are connected to obtain at least one target cell;

[0029] Determine the intersection coordinates between the non-repeated target line segments in the non-repeated target line segment set;

[0030] According to the intersection coordinates, determine the coordinate information corresponding to the at least one target cell.

[0031] In a possible implementation manner, the above-mentioned based on the coordinate information corresponding to the at least one target cell and the positions corresponding to each target character in the target text in the target image, the target text and the at least one target cell are integrated according to the third preset rule to obtain the table in the target image, including:

[0032] Determine the center coordinates corresponding to each target character in the target text based on the position of each target character in the target text in the target image;

[0033] If the center coordinates corresponding to the target character are within the range corresponding to the coordinate information of the target cell, fill the target character into the target cell to obtain the table in the target image.

[0034] In a possible implementation, after reconstructing the table in the target image according to the target text, the position of each target character in the target text in the target image, and the straight line detection result, the method further includes:

[0035] Output the table in a preset format.

[0036] In a second aspect, an embodiment of this specification provides a table recognition and reconstruction device, including:

[0037] An acquisition module, configured to acquire a target image containing a table;

[0038] A character recognition module, configured to perform character recognition on the target image to obtain the target text in the target image and the position of each target character in the target text in the target image; the target text includes at least one target character;

[0039] A straight line detection module, configured to perform straight line detection on the table in the target image to obtain the straight line detection result corresponding to the table; the straight line detection result includes at least four target line segments and the position corresponding to each target line segment;

[0040] A reconstruction module, configured to reconstruct the table in the target image according to the target text, the position of each target character in the target text in the target image, and the straight line detection result.

[0041] In a possible implementation, the device further includes:

[0042] A preprocessing module, configured to preprocess the target image to obtain a preprocessed target image;

[0043] The character recognition module is specifically configured to:

[0044] Perform character recognition on the preprocessed target image to obtain the target text in the target image and the position of each target character in the preprocessed target image;

[0045] The straight line detection module is specifically configured to:

[0046] Perform line detection on the table in the preprocessed target image to obtain the line detection result corresponding to the above table.

[0047] In a possible implementation, the above preprocessing includes performing quality enhancement processing on the above target image and / or performing image correction processing on the above target image.

[0048] In a possible implementation, the above reconstruction module includes:

[0049] A deletion unit for deleting duplicate target line segments in the above line detection result to obtain a set of non-duplicate target line segments; the set of non-duplicate target line segments includes at least four non-duplicate target line segments;

[0050] A reconstruction unit for reconstructing the table in the above target image according to the above target text, the positions of each target character in the above target text corresponding to the above target image, the set of non-duplicate target line segments, and the positions of each non-duplicate target line segment in the set of non-duplicate target line segments.

[0051] In a possible implementation, the above reconstruction module further includes:

[0052] A first determination unit for determining duplicate target line segments in the above line detection result according to the positions of each target line segment in the above line detection result.

[0053] In a possible implementation, the above reconstruction module further includes:

[0054] A calculation unit for calculating the similarity between every two of the above target line segments in the above line detection result;

[0055] A second determination unit for, if the above similarity is greater than a preset threshold, determining one of the two target line segments corresponding to the above similarity as a duplicate target line segment according to a first preset rule.

[0056] In a possible implementation, the above reconstruction unit includes:

[0057] A first integration subunit for integrating the non-duplicate target line segments in the set of non-duplicate target line segments according to the positions of each non-duplicate target line segment in the set of non-duplicate target line segments according to a second preset rule to obtain at least one target cell and the coordinate information corresponding to the at least one target cell;

[0058] A second integration subunit, configured to integrate the target text and the at least one target cell according to a third preset rule based on the coordinate information corresponding to the at least one target cell and the position of each target character in the target text in the target image, so as to obtain a table in the target image.

[0059] In a possible implementation manner, the first integration subunit is specifically configured to:

[0060] Connect the non-repetitive target line segments in the non-repetitive target line segment set according to the position of each non-repetitive target line segment in the non-repetitive target line segment set, so as to obtain at least one target cell;

[0061] Determine the intersection coordinates between the non-repetitive target line segments in the non-repetitive target line segment set;

[0062] Determine the coordinate information corresponding to the at least one target cell according to the intersection coordinates.

[0063] In a possible implementation manner, the second integration subunit is specifically configured to:

[0064] Determine the central coordinates of each target character in the target text according to the position of each target character in the target text in the target image;

[0065] If the central coordinates of the target character are within the range corresponding to the coordinate information of the target cell, fill the target character into the target cell to obtain a table in the target image.

[0066] In a possible implementation manner, the apparatus further includes:

[0067] An output module, configured to output the table in a preset format.

[0068] In a third aspect, an embodiment of this specification provides an electronic device, including: a processor and a memory;

[0069] The processor is connected to the memory;

[0070] The memory is configured to store executable program code;

[0071] The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method provided in the first aspect or any possible implementation manner of the first aspect of the embodiments of this specification.

[0072] Fourthly, an embodiment of this specification provides a computer storage medium. The computer storage medium stores multiple instructions, and the instructions are adapted to be loaded and executed by a processor to perform the method provided in the first aspect or any possible implementation manner of the first aspect of the embodiments of this specification.

[0073] Fifthly, an embodiment of this specification provides a computer program product containing instructions. When the computer program product runs on a computer or a processor, the computer or the processor is enabled to execute the table recognition and reconstruction method provided in the first aspect or any possible implementation manner of the first aspect of the embodiments of this specification.

[0074] In the embodiments of this specification, by performing text recognition on a target image containing a table, the target text in the target image and the position of each target character in the target text corresponding to the target image are obtained. And by performing straight line detection on the table in the target image, a straight line detection result corresponding to the table is obtained. The straight line detection result includes at least four target line segments and the position corresponding to each target line segment. Then, according to the target text, the position of each target character in the target text corresponding to the target image, and the straight line detection result, the table in the target image is reconstructed, so as to realize the reconstruction of the table in the target image, improve the robustness of table recognition and reconstruction, and help enterprises or individuals to realize the recognition of the table without adjusting thresholds and rules in different scenarios, reduce the cost of information extraction for enterprises or individuals, and improve the efficiency of table recognition and reconstruction. Description of the Drawings

[0075] To more clearly illustrate the technical solutions in the embodiments of this specification, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0076] Figure 1 It is a schematic diagram of the architecture of a table recognition and reconstruction system provided by an exemplary embodiment of this specification;

[0077] Figure 2 It is a schematic flowchart of a table recognition and reconstruction method provided by an exemplary embodiment of this specification;

[0078] Figure 3A It is a schematic diagram of a target image provided by an exemplary embodiment of this specification;

[0079] Figure 3B It is a schematic diagram of text recognition provided by an exemplary embodiment of this specification;

[0080] Figure 4A Another schematic diagram of a target image provided for an exemplary embodiment of this specification;

[0081] Figure 4B A schematic diagram of a table output in a preset format provided for an exemplary embodiment of this specification;

[0082] Figure 5 A schematic diagram of the implementation process of table reconstruction provided for an exemplary embodiment of this specification;

[0083] Figure 6A - Figure 6B A schematic diagram of a repeated target line segment provided for an exemplary embodiment of this specification;

[0084] Figure 7 Another schematic diagram of a repeated target line segment provided for an exemplary embodiment of this specification;

[0085] Figure 8 A schematic diagram of the implementation process of a second preset rule provided for an exemplary embodiment of this specification;

[0086] Figure 9 A schematic diagram of the process of obtaining a target cell provided for an exemplary embodiment of this specification;

[0087] Figure 10 A schematic diagram of the implementation process of a third preset rule provided for an exemplary embodiment of this specification;

[0088] Figure 11 A schematic diagram of the process of obtaining a reconstructed table provided for an exemplary embodiment of this specification;

[0089] Figure 12 Another schematic diagram of the process of a table recognition and reconstruction method provided for an exemplary embodiment of this specification;

[0090] Figure 13 A schematic diagram of the structure of a table recognition and reconstruction device provided for an exemplary embodiment of this specification;

[0091] Figure 14 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this specification. Detailed implementation manners

[0092] Next, the technical solutions in the embodiments of this specification will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this specification.

[0093] The terms "first", "second", "third", etc. in this specification, the claims and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that comprises a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or devices.

[0094] Please refer to Figure 1 , Figure 1 is a schematic diagram of the architecture of a form recognition and reconstruction system provided for an exemplary embodiment of this specification. As Figure 1 shown, the form recognition and reconstruction system may include: a first terminal cluster and a server 120. Among them:

[0095] The first terminal cluster may be a client side, specifically including one or more client sides, and the multiple client sides may include client side 110a, client side 110b, client side 110c... etc. A user version of software can be installed in the first terminal cluster to implement functions such as the user's online input of a target image including a form. Any one of the client sides in the first terminal cluster can establish a data relationship with the network and establish a data connection relationship with the server 120 through this network, such as sending a target image of a form and receiving the form after the target image is recognized and reconstructed. Among them, any one of the client sides in the first terminal cluster can be, but is not limited to, devices such as a mobile phone, a tablet computer, a laptop computer, etc. installed with the user version of software. It should be noted that any one of the client sides in the embodiments of this specification can be installed with a wide-angle camera. The wide-angle camera can be used to acquire a target image including a form. The client side can perform cropping processing on the target image and send the processed target image to other terminals or the server 120.

[0096] The server 120 can be a server capable of providing various table recognition and reconstruction. It can receive data such as a target image including a table sent from a network or a client, perform optical character recognition on the above target image to obtain the target text in the above target image and the position of each target character in the above target text corresponding to the above target image, and obtain the line detection result corresponding to the above table by performing line detection on the table in the above target image. The above line detection result includes at least four target line segments and the position corresponding to each of the above target line segments. Then, the table in the above target image is reconstructed according to the above target text, the position of each target character in the above target text corresponding to the above target image, and the above line detection result. The server 120 can also output the table in the reconstructed target image to any client in the first terminal cluster, etc. The server 120 can be, but is not limited to, a hardware server, a virtual server, a cloud server, etc.

[0097] The recognition and reconstruction of the table in the above target image are not limited to being executed by the above server 120, and can also be any client in the first terminal cluster. The above client can obtain a target image containing a table through a wide-angle camera installed on the client, or receive a target image containing a table sent by other clients through a network, etc., perform optical character recognition on the above target image, and perform line detection on the table in the above target image, so as to reconstruct the table in the above target image according to the target text obtained from the above optical character recognition, the position of each target character in the above target text corresponding to the above target image, and the above line detection result. The embodiments of this specification do not make specific limitations on this, and the following embodiments will all be described by taking the server 120 as an example to execute the recognition and reconstruction of the table in the above target image.

[0098] The network can be a medium providing a communication link between the server 120 and any client in the first terminal cluster, or can also be the Internet including network devices and transmission media, which is not limited thereto. The transmission medium can be a wired link (such as, but not limited to, coaxial cable, optical fiber, and digital subscriber line (DSL), etc.) or a wireless link (such as, but not limited to, wireless fidelity (WIFI), Bluetooth, and mobile device network, etc.).

[0099] It can be understood that Figure 1 The number of the first terminal cluster and the server 120 in the shown table recognition and reconstruction system is only for example. In specific implementation, the table recognition and reconstruction system can include any number of clients and servers. The embodiments of this specification do not make specific limitations on this. For example, but not limited to, the server 120 can be a server cluster composed of multiple servers.

[0100] Next, in combination with Figure 1 , the table recognition and reconstruction method provided in the embodiments of this specification will be introduced. Specifically, please refer to Figure 2 , which is a flowchart of a table recognition and reconstruction method provided in an exemplary embodiment of this specification. As Figure 2 shown, the table recognition and reconstruction method includes the following steps:

[0101] Step 202, obtain a target image containing a table.

[0102] Specifically, when the server 120 performs the recognition and reconstruction of the table in the target image, it can receive, through the network, a target image containing a table to be recognized and reconstructed sent by any client in the first terminal cluster; when a client in the first terminal cluster performs the recognition and reconstruction of the table in the target image, this client can also receive, through the network, a target image containing a table sent by other clients in the first terminal cluster, or obtain, through a wide-angle camera installed on this client, a target image containing a table to be recognized and reconstructed. The number of tables in the above target image can be one or multiple, and the embodiments of this specification do not limit this.

[0103] Step 204, perform text recognition on the target image to obtain the target text in the target image and the position of each target character in the target text corresponding to the target image.

[0104] Specifically, a preset text recognition algorithm can be used to perform text recognition on the target image, so as to obtain the target text in the above target image and the position of each target character in the above target text corresponding to the target image. The above target text includes at least one target character. The above target characters include but are not limited to letters, numbers, arithmetic symbols, punctuation marks, etc. The above preset text recognition algorithms include Optical Character Recognition (OCR) algorithms, deep learning text recognition algorithms, etc., and the embodiments of this specification do not limit this. The position of the above target characters corresponding to the target image includes but is not limited to the upper left vertex pixel coordinates and the lower right vertex pixel coordinates or the upper right vertex pixel coordinates and the lower left vertex pixel coordinates of the minimum bounding rectangle corresponding to each target character in the target image. That is, the target text in the target image can be recognized first, and then, according to the multiple pixel point coordinates corresponding to each target character in the above target text in the above target image, the minimum bounding rectangle corresponding to each target character in the target image is determined, and then, according to the upper left vertex pixel coordinates and the lower right vertex pixel coordinates or the upper right vertex pixel coordinates and the lower left vertex pixel coordinates of the above minimum bounding rectangle, the position of each target character in the above target image is determined.

[0105] Exemplarily, if it is recognized that Figure 3AThe minimum abscissa of the pixel coordinates corresponding to the target text 310 "fa" in the target image a shown is 88, the maximum abscissa is 600, the minimum ordinate is 80, and the maximum coordinate is 999. Then, as Figure 3B shown, the minimum bounding rectangle 320 corresponding to the target text 310 "fa" in the target image a can be determined, as well as the pixel coordinates of the upper left vertex A(3880, 180), the lower right vertex C(4600, 1999), the upper right vertex B(4600, 180), and the lower left vertex D(3880, 1999) of the minimum bounding rectangle 320 in the target image a. And the pixel coordinates of the upper left vertex A(3880, 180) and the lower right vertex C(4600, 1999) or the pixel coordinates of the upper right vertex B(4600, 180) and the lower left vertex D(3880, 1999) are determined as the position corresponding to the target text 310 "fa" in the target image a.

[0106] Optionally, the target image can be directly input into the text recognition model to output the target text in the target image and the position corresponding to each target text in the target text in the target image. The target text includes at least one target text. The target text includes, but is not limited to, letters, numbers, arithmetic symbols, punctuation marks, etc. The text recognition model is trained by known target text and images of the positions corresponding to each target text in the image.

[0107] Step 206, perform line detection on the table in the target image to obtain the line detection result corresponding to the table.

[0108] Specifically, a preset line detection algorithm can be used to detect lines in the target image of the table, so as to obtain the line detection result corresponding to the above table. The above line detection result includes at least four target line segments and the position corresponding to each of the above target line segments. The above at least four target line segments can be arranged and connected according to the positions corresponding to each of the above target line segments to form at least one closed cell. The position corresponding to the above target line segment includes, but is not limited to, the pixel coordinates of the two end pixel points corresponding to the above target line segment in the target image or the pixel coordinates of the central pixel point corresponding to the above target line segment in the target image and the length of the above target line segment. The above preset line detection algorithm can classify each pixel point in the image, determine the category of each point, and determine the object composed of multiple consecutive pixel points with the same category and the difference in their respective corresponding horizontal positions or vertical positions within a preset range and the number greater than the preset number as the target line segment, that is, the above target line segment is a horizontal line segment or a vertical line segment, and determine the pixel coordinates of the two end pixel points or the central pixel point of the above target line segment and the length of the above target line segment, that is, the number of pixel points included, as the position corresponding to the above target line segment. The above preset number can be 1000, 10000, etc., and the embodiments of this specification do not limit this. The above preset range can be 2, 10, 15, etc., and this specification does not limit this.

[0109] Optionally, the above target image can be directly input into the line detection model, so as to output at least four target line segments corresponding to the above table and the position corresponding to each of the above target line segments. The above at least four target line segments can be arranged and connected according to the positions corresponding to each of the above target line segments to form at least one closed rectangular cell. The above line detection model is trained based on the image of the above table including all the target line segments known to form the table and the positions corresponding to each of the above target line segments.

[0110] Step 208, reconstruct the table in the target image according to the target text and the positions corresponding to each target character in the target text in the target image and the line detection result.

[0111] Specifically, an information integration and reconstruction module can be used to integrate and adjust the target line segments in the line detection result to obtain at least one cell composed of the above target line segments and the coordinate information corresponding to the above at least one cell, and integrate each target character in the target text with the above at least one cell according to the positions corresponding to each target character in the target text in the target image and the coordinate information corresponding to the above at least one cell, so as to obtain the table in the reconstructed target image.

[0112] Optionally, to meet the different needs of different users for storing, viewing, editing, etc. of the recognized and reconstructed table in different formats, reduce the cost of enterprise information extraction, and improve efficiency, after reconstructing the table in the target image according to the above target text, the position of each target text in the target text in the target image, and the above straight line detection result, if the above table recognition and reconstruction is executed by the server 120, the server 120 can also output the above table to the user side in a preset format. If the above table recognition and reconstruction is executed by the user side, the user side can also output the above table in a preset format and display the above table on the page of the user side in the above preset format for the user to perform operations such as storing, viewing, editing, and information extraction on the above table. The above preset format includes but is not limited to file formats such as word, excel, and xml, and the user can also select and set according to needs.

[0113] Exemplarily, if the target image b as Figure 4A shown is obtained, and the output preset format is the excel format, then after the table in the target image b is recognized and reconstructed to obtain the table in the target image b, as Figure 4B shown, the table in the target image b can be output in the above excel format for the user to perform operations such as storing, viewing, editing, and information extraction on the content of the table in the target image b.

[0114] In the embodiments of this specification, by performing text recognition on the target image containing the table, the target text in the above target image and the position of each target text in the target text in the above target image are obtained, and by performing straight line detection on the table in the above target image, the straight line detection result corresponding to the above table is obtained. The above straight line detection result includes at least four target line segments and the position corresponding to each of the above target line segments. Then, according to the above target text, the position of each target text in the target text in the above target image, and the above straight line detection result, the table in the above target image is reconstructed, thereby realizing the reconstruction of the table in the target image, improving both the robustness of table recognition and reconstruction and enabling enterprises or individuals to realize table recognition without adjusting thresholds and rules in different scenarios, reducing the cost of enterprise or individual information extraction, and improving the efficiency of table recognition and reconstruction.

[0115] Due to factors such as the quality of the target image, the shooting angle of the table, and the performance of the straight line detection algorithm, when detecting straight lines in the table of the target image, there may be a situation where duplicate target line segments are detected. For example, for a long target line segment that makes up the above table, multiple short target line segments with duplicate pixel point coordinates are detected at the same time, or not only the above long target line segment is detected, but also target line segments close to the pixel point coordinates of the above long target line segment are detected, etc. As a result, when reconstructing the table in the target image using the target line segments in the straight line detection result obtained by the above straight line detection, problems such as low accuracy of table reconstruction may occur. To solve the above problems, next, in combination with Figure 1 - Figure 4B , a specific implementation process of step 208 in the table recognition and reconstruction method provided in the embodiments of this specification is introduced, which reconstructs the table in the target image according to the target text and the position of each target character in the target text in the target image and the straight line detection result. Specifically, please refer to Figure 5 , which is a schematic diagram of an implementation process of reconstructing a table provided in an exemplary embodiment of this specification. As Figure 5 shown, the implementation process of reconstructing the table includes the following steps:

[0116] Step 502, delete the duplicate target line segments in the straight line detection result to obtain a set of non-duplicate target line segments.

[0117] Specifically, in order to avoid the influence of duplicate target line segments on table reconstruction, the duplicate target line segments in the straight line detection result corresponding to the table obtained by detecting straight lines in the table of the target image can be deleted first, so as to obtain a set of non-duplicate target line segments. The above set of non-duplicate target line segments includes at least four non-duplicate target line segments. At least two of the above at least four non-duplicate target line segments are horizontal non-duplicate target line segments and at least two are vertical non-duplicate target line segments. The above duplicate target line segments can be used to represent target line segments whose pixel coordinates of all pixel points have been covered by the pixel coordinates of pixel points in other target line segments, or can be used to represent line segments composed of pixel points with the same pixel coordinates and the number greater than a preset number among at least four target line segments included in the above straight line detection result. This specification does not limit this. The above preset number can be 1000, 10000, etc., and the embodiments of this specification do not limit this.

[0118] Optionally, the duplicate target line segments in the above line detection result can be determined first according to the positions corresponding to each target line segment in the line detection result. That is, according to the pixel coordinates (starting point coordinates and ending point coordinates) of the two end pixel points corresponding to the target line segment or the pixel coordinates of the central pixel point, as well as the length of the above target line segment, the line segment formed by the pixel points corresponding to the duplicate pixel coordinates among at least four target line segments included in the above line detection result can be determined as the duplicate target line segment. Then, the determined duplicate target line segments in the above line detection result are deleted, so as to obtain a set of non-duplicate target line segments.

[0119] Exemplarily, as Figure 6A shown, if the position corresponding to the target line segment 610 is (100, 200), (900, 200), and the position corresponding to the target line segment 620 is (200, 200), (500, 200), then according to the positions corresponding to the above target line segment 610 and the above target line segment 620, it can be determined that the position corresponding to the line segment formed by the pixel points corresponding to the duplicate pixel coordinates in the above two target line segments is (200, 200), (500, 200). That is, the long target line segment 610 includes the short target line segment 620, so that the short target line segment 620 can be determined as the duplicate target line segment.

[0120] Exemplarily, as Figure 6B shown, if the position corresponding to the target line segment 630 is (100, 100), (100, 500), and the position corresponding to the target line segment 640 is (100, 300), (100, 900), then according to the positions corresponding to the above target line segment 630 and the above target line segment 640, it can be determined that the position corresponding to the line segment 650 formed by the pixel points corresponding to the duplicate pixel coordinates in the above two target line segments is (100, 300), (100, 500). That is, there is a duplicate part in the above target line segment 630 and the above target line segment 640, so that the duplicate part of the above target line segment 630 and the above target line segment 640, that is, the line segment 650, can be determined as the duplicate target line segment.

[0121] Optionally, in addition to determining the duplicate target line segments in the line detection result according to the positions corresponding to each target line segment in the line detection result as described above, it is also possible to calculate the similarity between every two target line segments in the line detection result. When the similarity is greater than a preset threshold, one of the two target line segments corresponding to the similarity is determined as a duplicate target line segment according to a first preset rule. The similarity is used to characterize the similarity between the multiple pixel coordinates corresponding to the two target line segments. When the lengths of the two target line segments are different, calculating the similarity between every two target line segments in the line detection result can be obtained by calculating the probability that the multiple pixel coordinates corresponding to the longer target line segment among the two target line segments are similar to the multiple pixel coordinates corresponding to the shorter target line segment. When the lengths of the two target line segments are different, the first preset rule may be to determine the shorter of the two target line segments as the duplicate target line segment; when the lengths of the two target line segments are the same, the first preset rule may be to determine any one of the two target line segments as the duplicate target line segment or to determine it according to the positions of other target line segments, etc., and the embodiments of the present specification do not limit this. The preset threshold may be 0.8, 0.9, etc., and the embodiments of the present specification do not limit this.

[0122] Exemplarily, when the lengths of the two target line segments for which the similarity is calculated are different, as Figure 7 shown, if the longer target line segment 710 among the two target line segments corresponds to 16,000 pixel coordinates, the shorter target line segment 720 corresponds to 10,000 pixel coordinates, and the number of similar pixel coordinates between the longer target line segment 710 and the shorter target line segment 720 is 9,900, then the similarity between the longer target line segment 710 and the shorter target line segment 720 can be calculated to be 0.99. If the preset threshold is 0.9, that is, the shorter target line segment 720 can be determined as the duplicate target line segment.

[0123] Step 504, reconstruct the table in the target image according to the target text, the positions corresponding to each target character in the target text in the target image, the set of non-duplicate target line segments, and the positions corresponding to each non-duplicate target line segment in the set of non-duplicate target line segments.

[0124] Specifically, first, based on the positions corresponding to each non-repetitive target line segment in the non-repetitive target line segment set, the non-repetitive target line segments in the non-repetitive target line segment set are integrated according to the second preset rule to obtain at least one target cell and the coordinate information corresponding to the at least one target cell. Then, based on the coordinate information corresponding to the at least one target cell and the positions corresponding to each target character in the target text in the target image, the target text and the at least one target cell are integrated according to the third preset rule, so as to obtain the table in the target image.

[0125] Further, as Figure 8 shown, the second preset rule includes the following steps:

[0126] Step 802, according to the positions corresponding to each non-repetitive target line segment in the non-repetitive target line segment set, connect the non-repetitive target line segments in the non-repetitive target line segment set to obtain at least one target cell.

[0127] Specifically, all the non-repetitive target line segments in the non-repetitive target line segment set can be connected according to the positions corresponding to each non-repetitive target line segment respectively, so as to obtain at least four intersection points, and at least one target cell is determined according to the at least four intersection points. The target cell is used to represent a rectangle formed by two adjacent intersection points on the same straight line among the at least four intersection points and two intersection points with the same horizontal coordinates or vertical coordinates corresponding to the two adjacent intersection points on the adjacent straight lines parallel to the straight line.

[0128] Exemplarily, as Figure 9 shown, if there are four non-repetitive target line segments in the non-repetitive target line segment set, where the position corresponding to the non-repetitive target line segment 910 is (1000, 1000), (9000, 1000), the position corresponding to the non-repetitive target line segment 920 is (800, 9000), (9500, 9000), the position corresponding to the non-repetitive target line segment 930 is (1000, 200), (1000, 12000), and the position corresponding to the non-repetitive target line segment 940 is (9000, 1000), (9000, 9000), then the four intersection point coordinates among the four non-repetitive target line segments can be determined as (1000, 1000), (9000, 1000), (9000, 9000), (1000, 9000), so as to obtain the target cell 950 as Figure 9 shown.

[0129] Step 804, determine the intersection point coordinates among the non-repetitive target line segments in the non-repetitive target line segment set.

[0130] Specifically, the same pixel coordinates in the non-repetitive target line segments in the non-repetitive target line segment set can be determined as the intersection coordinates between the non-repetitive target line segments in the non-repetitive target line segment set.

[0131] Step 806, determine the coordinate information corresponding to at least one target cell according to the intersection coordinates.

[0132] Specifically, the intersection coordinates of the upper left corner and the lower right corner or the lower left corner and the upper right corner in the target cell can be determined as the coordinate information corresponding to the target cell.

[0133] Further, as Figure 10 shown, the above-mentioned third preset rule includes the following steps:

[0134] Step 1002, determine the central coordinates corresponding to each target character in the target text according to the position of each target character in the target text in the target image.

[0135] Specifically, according to the position of each target character in the target text in the target image, that is, the pixel coordinates of the upper left vertex and the lower right vertex or the upper right vertex and the lower left vertex of the minimum bounding rectangle, the central coordinates of the minimum bounding rectangle corresponding to each target character in the target image can be calculated, and the above central coordinates are determined as the central coordinates corresponding to each target character in the target text.

[0136] Exemplarily, if the position of the target character "A" in the target image is (2000, 4000), (4000, 8000), then the central coordinates of the minimum bounding rectangle corresponding to the target character "A" in the target image can be calculated as (3000, 6000), and the above central coordinates (3000, 6000) are determined as the central coordinates corresponding to the target character "A".

[0137] Step 1004, if the central coordinates corresponding to the target character are within the range corresponding to the coordinate information of the target cell, fill the target character into the target cell to obtain the table in the target image.

[0138] Specifically, after determining the central coordinates corresponding to each target character in the target text and the coordinate information corresponding to the target cell, it is possible to first determine whether the central coordinates corresponding to the target character are within the range corresponding to the coordinate information of the target cell. If the central coordinates corresponding to the above target character are within the range corresponding to the coordinate information of the target cell, that is, the central coordinates corresponding to the above target character are within the rectangular range corresponding to the target cell, then the above target character can be directly filled into the corresponding target cell according to the fourth preset rule, thereby obtaining the table in the target image. The above fourth preset rule may include, but is not limited to, the preset target character font, the preset target character size, the preset target character spacing, etc. The above fourth preset rule may also be directly determining the size of the corresponding target character to be filled into the target cell according to the position of the recognized target character, and this specification does not limit this.

[0139] Exemplarily, as Figure 11 shown, if the central coordinate 1120 corresponding to the target character 1110 "□" is (2000, 4000), and the coordinate information corresponding to a certain target cell 1130 is E(1000, 2000), F(6000, 9000), then it can be determined that the horizontal range corresponding to the coordinate information of the above target cell 1130 is greater than 1000 and less than 6000, and the vertical range is greater than 2000 and less than 9000. Thus, it can be judged that the above target character 1110 "□" is within the range corresponding to the coordinate information of the above target cell 1130, that is, the above target character 1110 "□" can be filled into the target cell 1130, obtaining the table 1140 in the target image as Figure 11 shown.

[0140] Optionally, if the central coordinates corresponding to the target character are not within the range corresponding to the coordinate information of the target cell, such as the header, signature, etc. of the target text outside the table, the above target character can be filled in the corresponding position coordinates outside the target cell according to the central coordinates corresponding to the above target character, or the target character whose central coordinates are outside the range corresponding to the coordinate information of the target cell can be directly deleted, and this specification does not limit this.

[0141] In the embodiments of this specification, by deleting the repeated target line segments in the line detection result obtained by performing line detection on the table in the target image, a non-repeated target line segment set is obtained, and the table in the target image is reconstructed according to the target text and the position of each target character in the target text in the target image and the above non-repeated target line segment set and the position corresponding to each non-repeated target line segment in the non-repeated target line segment set, which can not only avoid the influence of the detected repeated target line segments on the table reconstruction, improve the robustness of the table recognition and reconstruction, but also improve the efficiency and accuracy of the table recognition and reconstruction.

[0142] Since there may be actual problems such as blurring and tilting in the table in the obtained target image, if the above-mentioned target image is directly used for table recognition and reconstruction, it will lead to problems such as poor robustness of table recognition and reconstruction, as well as low efficiency and accuracy of table recognition and reconstruction. To solve the above problems, next, in combination with Figure 1 - Figure 11 , a schematic flowchart of another table recognition and reconstruction method provided by the embodiments of this specification will be introduced. As Figure 12 shown, this table recognition and reconstruction method includes the following steps:

[0143] Step 1202: Obtain a target image containing a table.

[0144] Specifically, step 1202 is the same as step 202, which will not be elaborated here.

[0145] Step 1204: Preprocess the target image to obtain a preprocessed target image.

[0146] Specifically, in order to improve the efficiency and accuracy of table recognition and reconstruction, some preprocessing operations can be performed on the obtained target image containing a table first, so as to obtain a preprocessed target image. The above preprocessing operations include but are not limited to performing quality enhancement processing on the above target image and / or performing image correction processing on the above target image. When performing quality enhancement processing and image correction processing on the target image, the quality enhancement processing can be performed on the target image first, and then the image correction processing can be performed on the target image, or the image correction processing can be performed on the target image first, and then the quality enhancement processing can be performed on the target image. The embodiments of this specification do not limit this. The above quality enhancement processing includes but is not limited to image deblurring processing, image brightness enhancement processing, image contrast enhancement processing, image super-resolution reconstruction processing, etc. The above image correction processing includes but is not limited to image four-direction rotation technology, perspective correction technology, etc.

[0147] Step 1206: Perform character recognition on the preprocessed target image to obtain the target text in the preprocessed target image and the positions of each target character in the target text in the preprocessed target image.

[0148] Specifically, an optical character recognition module can be used to perform optical character recognition on the preprocessed target image, so as to obtain the target text in the preprocessed target image and the position corresponding to each target character in the target text in the preprocessed target image. The above-mentioned target text includes at least one target character. The above-mentioned optical character recognition module can include EasyOCR, tesseract-ocr, etc., or various preset optical character recognition algorithms can be used for substitution, which is not limited in this specification. The above-mentioned preset optical character recognition algorithms include optical character recognition (OCR) algorithms, deep learning neural network optical character recognition algorithms, etc., which are not limited in the embodiments of this specification. The above-mentioned deep learning neural network optical character recognition algorithms include, but are not limited to, optical character recognition models trained through images with known target text and the positions corresponding to each target character in the target text in the image.

[0149] Step 1208: Perform straight line detection on the table in the preprocessed target image to obtain the straight line detection result corresponding to the table.

[0150] Specifically, a straight line detection module can be used to perform straight line detection on the table in the preprocessed target image to obtain the straight line detection result corresponding to the table. The above-mentioned straight line detection result includes at least four target line segments and the position corresponding to each of the above-mentioned target line segments. The above-mentioned at least four target line segments can be arranged and connected according to the positions corresponding to each target line segment to form at least one closed rectangular cell. The above-mentioned straight line detection module can include a straight line detection model, or a preset semantic segmentation algorithm can be used for substitution, which is not limited in this specification. The above-mentioned straight line detection model is trained based on an image of the above-mentioned table containing all the target line segments that make up the table and the positions corresponding to each of the above-mentioned target line segments. The semantic segmentation algorithm is an algorithm that associates labels or categories with each pixel of an image, and is used to identify the set of pixels that make up distinguishable categories, that is, to classify each pixel point in the image, determine the category of each point, and thus perform region division. The above-mentioned preset semantic segmentation algorithms include semantic segmentation algorithms based on Texton Forest, semantic segmentation algorithms based on Random Forest, deep learning semantic segmentation algorithms, etc., which are not limited in this specification.

[0151] Step 1210: Reconstruct the table in the target image according to the target text, the position corresponding to each target character in the target text in the preprocessed target image, and the straight line detection result.

[0152] Specifically, according to the positions of each target character in the target text corresponding to the preprocessed target image, each target character in the above target text can be filled into the corresponding cells formed by the target line segments in the line detection result, so as to reconstruct the table in the target image.

[0153] In the embodiments of the present specification, by first performing preprocessing operations such as quality enhancement and / or image correction on the obtained target image containing a table, then performing character recognition and line detection on the target image after the above preprocessing operations, and reconstructing the table in the target image according to the target text obtained from the above character recognition, the positions of each target character in the preprocessed target image, and the line detection result obtained from the above line detection, it is possible to achieve quality enhancement and / or correction of the large direction and small angle of the target image containing a table, which not only enhances the robustness of table recognition and reconstruction, but also improves the efficiency and accuracy of table recognition and reconstruction.

[0154] Please refer to Figure 13 , Figure 13 A table recognition and reconstruction device provided for an exemplary embodiment of the present specification. The table recognition and reconstruction device 1300 includes:

[0155] An acquisition module 1310, configured to acquire a target image containing a table;

[0156] A character recognition module 1320, configured to perform character recognition on the above target image to obtain the target text in the above target image and the positions of each target character in the above target text corresponding to the above target image; the above target text includes at least one of the above target characters;

[0157] A line detection module 1330, configured to perform line detection on the table in the above target image to obtain the line detection result corresponding to the above table; the above line detection result includes at least four target line segments and the positions corresponding to each of the above target line segments;

[0158] A reconstruction module 1340, configured to reconstruct the table in the above target image according to the above target text, the positions of each target character in the above target text corresponding to the above target image, and the above line detection result.

[0159] In a possible implementation manner, the above table recognition and reconstruction device 1300 further includes:

[0160] A preprocessing module, configured to perform preprocessing on the above target image to obtain a preprocessed target image;

[0161] The above character recognition module 1320 is specifically configured to:

[0162] Perform character recognition on the preprocessed target image to obtain the target text in the target image and the positions of each target character in the target text corresponding to the preprocessed target image;

[0163] The above-mentioned straight line detection module 1330 is specifically used for:

[0164] Perform straight line detection on the table in the preprocessed target image to obtain the straight line detection result corresponding to the table.

[0165] In a possible implementation manner, the above-mentioned preprocessing includes performing quality enhancement processing on the target image and / or performing image correction processing on the target image.

[0166] In a possible implementation manner, the above-mentioned reconstruction module 1340 includes:

[0167] A deletion unit, configured to delete duplicate target line segments in the straight line detection result to obtain a set of non-duplicate target line segments; the set of non-duplicate target line segments includes at least four non-duplicate target line segments;

[0168] A reconstruction unit, configured to reconstruct the table in the target image according to the target text, the positions of each target character in the target text corresponding to the target image, the set of non-duplicate target line segments, and the positions of each non-duplicate target line segment in the set of non-duplicate target line segments.

[0169] In a possible implementation manner, the above-mentioned reconstruction module 1340 further includes:

[0170] A first determination unit, configured to determine the duplicate target line segments in the straight line detection result according to the positions of each target line segment in the straight line detection result.

[0171] In a possible implementation manner, the above-mentioned reconstruction module 1340 further includes:

[0172] A calculation unit, configured to calculate the similarity between every two of the target line segments in the straight line detection result;

[0173] A second determination unit, configured to, if the similarity is greater than a preset threshold, determine one of the two target line segments corresponding to the similarity as a duplicate target line segment according to a first preset rule.

[0174] In a possible implementation manner, the above-mentioned reconstruction unit includes:

[0175] The first integration subunit is configured to integrate the non-repetitive target line segments in the non-repetitive target line segment set according to a second preset rule based on the positions corresponding to each non-repetitive target line segment in the non-repetitive target line segment set, so as to obtain at least one target cell and the coordinate information corresponding to the at least one target cell;

[0176] The second integration subunit is configured to integrate the target text and the at least one target cell according to a third preset rule based on the coordinate information corresponding to the at least one target cell and the positions corresponding to each target character in the target text in the target image, so as to obtain the table in the target image.

[0177] In a possible implementation manner, the first integration subunit is specifically configured to:

[0178] Connect the non-repetitive target line segments in the non-repetitive target line segment set according to the positions corresponding to each non-repetitive target line segment in the non-repetitive target line segment set, so as to obtain at least one target cell;

[0179] Determine the intersection coordinates between the non-repetitive target line segments in the non-repetitive target line segment set;

[0180] Determine the coordinate information corresponding to the at least one target cell according to the intersection coordinates.

[0181] In a possible implementation manner, the second integration subunit is specifically configured to:

[0182] Determine the central coordinates corresponding to each target character in the target text according to the positions corresponding to each target character in the target text in the target image;

[0183] If the central coordinates corresponding to the target character are within the range corresponding to the coordinate information of the target cell, fill the target character into the target cell to obtain the table in the target image.

[0184] In a possible implementation manner, the apparatus 1300 further includes:

[0185] An output module, configured to output the table in a preset format.

[0186] The division of each module in the above table recognition and reconstruction device is only for illustrative purposes. In other embodiments, the table recognition and reconstruction device can be divided into different modules as needed to complete all or part of the functions of the above table recognition and reconstruction device. In the embodiments of this specification, the implementation of each module in the provided table recognition and reconstruction device can be in the form of a computer program. This computer program can run on a terminal or a server. The program module constituted by this computer program can be stored in the memory of the terminal or the server. When this computer program is executed by a processor, all or part of the steps of the table recognition method described in the embodiments of this specification are implemented.

[0187] Please refer to Figure 14 , Figure 14 , which is a schematic structural diagram of an electronic device provided by an exemplary embodiment of this specification. As Figure 14 shown, the electronic device 1400 may include: at least one processor 1410, at least one communication bus 1420, a user interface 1430, at least one network interface 1440, and a memory 1450.

[0188] Among them, the communication bus 1420 can be used to realize the connection and communication of the above-mentioned various components.

[0189] Among them, the user interface 1430 may include a display screen (Display) and a camera (Camera). Optionally, the user interface may further include a standard wired interface and a wireless interface.

[0190] Among them, the network interface 1440 may optionally include a Bluetooth module, a Near Field Communication (NFC) module, a Wireless Fidelity (Wi-Fi) module, etc.

[0191] Among them, the processor 1410 may include one or more processing cores. The processor 1410 connects various parts within the entire electronic device 1400 through various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 1450, and by calling the data stored in the memory 1450, it executes various functions of the routing electronic device 1400 and processes data. Optionally, the processor 1410 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 1410 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 1410 and may be implemented separately by a single chip.

[0192] Among them, the memory 1450 may include random access memory (RAM) and may also include read-only memory (ROM). Optionally, the memory 1450 includes a non-transitory computer-readable medium. The memory 1450 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1450 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as acquisition function, character recognition function, line detection function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store the data involved in the above-mentioned various method embodiments. Optionally, the memory 1450 may also be at least one storage device located far from the aforementioned processor 1410. As Figure 14 shown, the memory 1450, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0193] Specifically, the processor 1410 may be used to call the program instructions stored in the memory 1450 and specifically perform the following operations:

[0194] Obtain a target image containing a table.

[0195] Perform character recognition on the above target image to obtain the target text in the above target image and the position of each target character in the above target text corresponding to the above target image; the above target text includes at least one of the above target characters.

[0196] Perform straight-line detection on the table in the above target image to obtain the straight-line detection result corresponding to the above table; the above straight-line detection result includes at least four target line segments and the position corresponding to each of the above target line segments.

[0197] Reconstruct the table in the above target image according to the above target text, the position of each target character in the above target text corresponding to the above target image, and the above straight-line detection result.

[0198] In some possible embodiments, after the above processor 1410 executes to obtain the target image and before performing character recognition on the above target image to obtain the target text in the above target image and the position of each target character in the above target text corresponding to the above target image, it is further configured to execute:

[0199] Perform preprocessing on the above target image to obtain a preprocessed target image.

[0200] When performing character recognition on the above target image to obtain the target text in the above target image and the position of each target character in the above target text corresponding to the above target image, it is specifically configured to execute:

[0201] Perform character recognition on the preprocessed above target image to obtain the target text in the above target image and the position of each target character in the above target text corresponding to the preprocessed above target image.

[0202] When performing straight-line detection on the table in the above target image to obtain the straight-line detection result corresponding to the above table, it is specifically configured to execute:

[0203] Perform straight-line detection on the table in the preprocessed above target image to obtain the straight-line detection result corresponding to the above table.

[0204] In some possible embodiments, the above preprocessing includes performing quality enhancement processing on the above target image and / or performing image correction processing on the above target image.

[0205] In some possible embodiments, when the above processor 1410 executes to reconstruct the table in the above target image according to the above target text, the position of each target character in the above target text corresponding to the above target image, and the above straight-line detection result, it is specifically configured to execute:

[0206] Delete the duplicate target line segments in the above straight line detection results to obtain a set of non-duplicate target line segments; the above set of non-duplicate target line segments includes at least four non-duplicate target line segments.

[0207] Reconstruct the table in the above target image according to the above target text, the positions of each target character in the above target text corresponding in the above target image, the above set of non-duplicate target line segments, and the positions of each non-duplicate target line segment in the above set of non-duplicate target line segments.

[0208] In some possible embodiments, before the above processor 1410 executes to delete the duplicate target line segments in the above straight line detection results, it is further configured to execute:

[0209] Determine the duplicate target line segments in the above straight line detection results according to the positions of each target line segment in the above straight line detection results.

[0210] In some possible embodiments, before the above processor 1410 executes to delete the duplicate target line segments in the above straight line detection results, it is further configured to execute:

[0211] Calculate the similarity between every two of the above target line segments in the above straight line detection results.

[0212] If the above similarity is greater than a preset threshold, then determine one of the two target line segments corresponding to the above similarity as a duplicate target line segment according to the first preset rule.

[0213] In some possible embodiments, when the above processor 1410 executes to reconstruct the table in the above target image according to the above target text, the positions of each target character in the above target text corresponding in the above target image, the above set of non-duplicate target line segments, and the positions of each non-duplicate target line segment in the above set of non-duplicate target line segments, it is specifically configured to execute:

[0214] Based on the positions of each non-duplicate target line segment in the above set of non-duplicate target line segments, integrate the non-duplicate target line segments in the above set of non-duplicate target line segments according to the second preset rule to obtain at least one target cell and the coordinate information corresponding to the at least one target cell.

[0215] Based on the coordinate information corresponding to the at least one target cell and the positions of each target character in the above target text corresponding in the above target image, integrate the above target text and the at least one target cell according to the third preset rule to obtain the table in the above target image.

[0216] In some possible embodiments, the above-mentioned processor 1410 performs integration of the non-repetitive target line segments in the above-mentioned non-repetitive target line segment set according to the positions corresponding to each non-repetitive target line segment in the above-mentioned non-repetitive target line segment set according to a second preset rule to obtain at least one target cell and coordinate information corresponding to the at least one target cell, including:

[0217] Connect the non-repetitive target line segments in the above-mentioned non-repetitive target line segment set according to the positions corresponding to each non-repetitive target line segment in the above-mentioned non-repetitive target line segment set to obtain at least one target cell.

[0218] Determine the intersection coordinates between the non-repetitive target line segments in the above-mentioned non-repetitive target line segment set.

[0219] Determine the coordinate information corresponding to the at least one target cell according to the above intersection coordinates.

[0220] In some possible embodiments, when the above-mentioned processor 1410 performs integration of the above-mentioned target text and the above-mentioned at least one target cell according to the coordinate information corresponding to the above-mentioned at least one target cell and the positions corresponding to each target character in the above-mentioned target text in the above-mentioned target image according to a third preset rule to obtain the table in the above-mentioned target image, it is specifically used to perform:

[0221] Determine the central coordinates corresponding to each target character in the above-mentioned target text according to the positions corresponding to each target character in the above-mentioned target text in the above-mentioned target image.

[0222] If the central coordinates corresponding to the above-mentioned target character are within the range corresponding to the coordinate information of the above-mentioned target cell, fill the above-mentioned target character into the above-mentioned target cell to obtain the table in the above-mentioned target image.

[0223] In some possible embodiments, after the above-mentioned processor 1410 performs reconstruction of the table in the above-mentioned target image according to the above-mentioned target text and the positions corresponding to each target character in the above-mentioned target text in the above-mentioned target image and the above-mentioned straight line detection result, it is further used to perform:

[0224] Output the above table in a preset format.

[0225] This embodiment of the specification further provides a computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When it runs on a computer or a processor, the computer or the processor is enabled to execute one or more steps in the above embodiments. If each component module of the above table recognition and reconstruction device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in the above computer-readable storage medium.

[0226] In the above embodiments, they can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The above computer program product includes one or more computer instructions. When the above computer program instructions are loaded and executed on a computer, the processes or functions described above in the embodiments of this specification are generated in whole or in part. The above computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above computer instructions can be stored in a computer-readable storage medium or transmitted through the above computer-readable storage medium. The above computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The above computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The above available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, Digital Versatile Disc (DVD)), or a semiconductor medium (for example, Solid State Disk (SSD)), etc.

[0227] Those of ordinary skill in the art can understand that all or part of the processes in the above embodiments of the method can be completed by instructing relevant hardware through a computer program. This program can be stored in a computer-readable storage medium. When this program is executed, it can include the processes of the embodiments of the above various methods. The aforementioned storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disc that can store program codes. Without conflict, the technical features in this embodiment and the implementation solutions can be combined arbitrarily.

[0228] The above-described embodiments are merely described in terms of the preferred implementation modes of this specification, and do not limit the scope of this specification. Without departing from the design spirit of this specification, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of this specification should all fall within the protection scope determined by the claims.

[0229] The foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for table recognition and reconstruction, the method comprising: Obtaining a target image containing a table; Performing text recognition on the target image to obtain target text in the target image and the position of each target character in the target text corresponding to the target image; the target text includes at least one of the target characters; Performing line detection on the table in the target image to obtain a line detection result corresponding to the table; the line detection result includes at least four target line segments and the position of each target line segment; Deleting duplicate target line segments in the line detection result to obtain a set of non-duplicate target line segments; the set of non-duplicate target line segments includes at least four non-duplicate target line segments; Based on the position of each non-duplicate target line segment in the set of non-duplicate target line segments, integrating the non-duplicate target line segments in the set of non-duplicate target line segments according to a second preset rule to obtain at least one target cell and the coordinate information corresponding to the at least one target cell; Determining the center coordinate of each target character in the target text according to the position of each target character in the target text corresponding to the target image; Filling the target characters with center coordinates within the range corresponding to the coordinate information of the target cell into the target cell, and filling the target characters with center coordinates not within the range corresponding to the coordinate information of the target cell into the corresponding position coordinates outside the target cell to obtain the table in the target image.

2. The method according to claim 1, after obtaining the target image and before performing text recognition on the target image to obtain target text in the target image and the position of each target character in the target text corresponding to the target image, the method further comprises: Performing preprocessing on the target image to obtain a preprocessed target image; The performing text recognition on the target image to obtain target text in the target image and the position of each target character in the target text corresponding to the target image includes: Performing text recognition on the preprocessed target image to obtain target text in the target image and the position of each target character in the preprocessed target image corresponding to the target image; The performing line detection on the table in the target image to obtain a line detection result corresponding to the table includes: Performing line detection on the table in the preprocessed target image to obtain a line detection result corresponding to the table.

3. The method according to claim 2, wherein the preprocessing includes performing quality enhancement processing on the target image and / or performing image correction processing on the target image.

4. The method according to claim 1, before deleting duplicate target line segments in the line detection result, the method further comprises: Determining duplicate target line segments in the line detection result according to the position of each target line segment in the line detection result.

5. The method according to claim 1, before deleting duplicate target line segments in the line detection result, the method further comprises: Calculate the similarity between every two of the target line segments in the line detection result; If the similarity is greater than a preset threshold, determine one of the two target line segments corresponding to the similarity as a duplicate target line segment according to a first preset rule.

6. The method according to claim 1, wherein based on the positions corresponding to each non-duplicate target line segment in the non-duplicate target line segment set, integrating the non-duplicate target line segments in the non-duplicate target line segment set according to a second preset rule to obtain at least one target cell and the coordinate information corresponding to the at least one target cell, comprising: Connect the non-duplicate target line segments in the non-duplicate target line segment set according to the positions corresponding to each non-duplicate target line segment in the non-duplicate target line segment set to obtain at least one target cell; Determine the intersection coordinates between the non-duplicate target line segments in the non-duplicate target line segment set; Determine the coordinate information corresponding to the at least one target cell according to the intersection coordinates.

7. The method according to claim 1, after reconstructing the table in the target image according to the target text and the positions corresponding to each target character in the target text in the target image and the line detection result, the method further comprises: Output the table in a preset format.

8. A table recognition and reconstruction device, the device comprising: An acquisition module, configured to acquire a target image containing a table; A character recognition module, configured to perform character recognition on the target image to obtain the target text in the target image and the positions corresponding to each target character in the target text in the target image; the target text includes at least one of the target characters; A line detection module, configured to perform line detection on the table in the target image to obtain a line detection result corresponding to the table; the line detection result includes at least four target line segments and the positions corresponding to each target line segment; A reconstruction module, configured to delete the duplicate target line segments in the line detection result to obtain a non-duplicate target line segment set; the non-duplicate target line segment set includes at least four non-duplicate target line segments; based on the positions corresponding to each non-duplicate target line segment in the non-duplicate target line segment set, integrating the non-duplicate target line segments in the non-duplicate target line segment set according to a second preset rule to obtain at least one target cell and the coordinate information corresponding to the at least one target cell; Determine the center coordinates corresponding to each target character in the target text according to the positions corresponding to each target character in the target text in the target image; Fill the target characters with center coordinates within the range of the coordinate information corresponding to the target cell into the target cell, and fill the target characters with center coordinates not within the range of the coordinate information corresponding to the target cell into the corresponding position coordinates outside the target cell to obtain the table in the target image.

9. An electronic device, comprising: A processor and a memory; The processor is connected to the memory; The memory is configured to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method according to any one of claims 1-7.

10. A computer storage medium storing a plurality of instructions adapted to be loaded and executed by a processor to perform the method steps according to any one of claims 1-7.

11. A computer program product containing instructions that, when the computer program product runs on a computer or a processor, cause the computer or the processor to execute the form recognition and reconstruction method according to any one of claims 1-7.

Images