A table recognition method and system

Abstract

The present invention relates to the field of digital image processing technology, and its purpose is to provide a table recognition method and system. The table recognition method includes: obtaining a document image, and extracting an image area containing only a table from the document image to obtain a table image; preprocessing the table image to obtain a preprocessed table image; performing signal enhancement on the table lines in the preprocessed table image to obtain an enhanced table image; extracting a point set of each intersection in the table in the enhanced table image, and obtaining a table structure of the enhanced table image based on the point set; performing text detection on the enhanced table image to obtain a text detection result; performing text recognition on the text detection result to obtain a text recognition result; and generating an electronic table based on the table structure and the text recognition result. The present invention can detect, identify and restore tables in documents such as insurance policies with complex layouts, which is convenient for automatic table entry and archiving management of complex documents.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image processing, and in particular to a table recognition method and system. Background Art

[0002] Automated office is an indispensable process for modern enterprises, especially in the financial industry, where a large number of documents need to be sorted, entered and archived. Manual entry will consume a lot of manpower and time costs.

[0003] In the field of image processing, there has always been research related to document recognition. Especially with the development of deep learning in computer vision, document recognition technology has made great progress.

[0004] However, the layout of documents such as insurance documents is very complex, usually with multiple tables, and the layout of different insurance companies and insurance types may also be very different. In the process of using the existing technology to recognize such documents, the inventors found that the existing technology has at least the following problems, resulting in the automatic entry and filing of tables in the document cannot meet user requirements: 1) Documents with complex layouts cannot be analyzed; 2) Tables with complex structures are difficult to parse; 3) If the method is based on template matching, the generalization performance is extremely poor. Summary of the invention

[0005] The present invention aims to solve the above technical problems at least to a certain extent, and provides a table recognition method and system.

[0006] The technical solution adopted by the present invention is:

[0007] In a first aspect, the present invention provides a table recognition method, comprising:

[0008] Acquire a document image, and extract an image region containing only a table from the document image to obtain a table image;

[0009] Preprocessing the table image to obtain a preprocessed table image;

[0010] Performing signal enhancement on the table lines in the preprocessed table image to obtain an enhanced table image;

[0011] Extracting a point set of each intersection point in the table in the enhanced table image, and obtaining a table structure of the enhanced table image according to the point set;

[0012] Performing text detection on the enhanced table image to obtain a text detection result;

[0013] Performing text recognition on the text detection result to obtain a text recognition result;

[0014] An electronic table is generated according to the table structure and the text recognition result.

[0015] The present invention can detect, identify and restore tables in documents such as insurance policies with complex layouts, facilitate the automatic entry and archiving management of tables in complex documents such as insurance documents, and solve the problem of difficulty in table recognition caused by diverse types and complex layouts in documents such as insurance policies. At the same time, it can improve the accuracy of identifying the structure and content of tables in complex documents such as insurance policies. Compared with the existing technology, the present invention reduces the difficulty of development and training, improves the detection and recognition speed, and at the same time improves the versatility and stability of table recognition in documents.

[0016] In a possible design, an image region containing only a table is extracted from the document image to obtain a table image, including:

[0017] Build an object detection model;

[0018] Acquire a sample set, import the sample set into the target detection model, perform multi-scale training on the target detection model, and obtain a target detection model after initial training; wherein the samples in the sample set include document images containing tables;

[0019] Using pruning technology, the target detection model after initial training is simplified to obtain a simplified target detection model;

[0020] Based on transfer learning, the sample set is labeled to obtain labeling information;

[0021] Input the annotation information into the simplified target detection model, train the simplified target detection model again, and obtain the final trained target detection model;

[0022] The document image is input into the final trained object detection model to obtain a table image.

[0023] In a possible design, the target detection model adopts the FCOS model.

[0024] In a possible design, preprocessing the table image to obtain a preprocessed table image includes:

[0025] Converting the table image into a grayscale image, calculating a histogram of the grayscale image, and performing statistical analysis on the histogram to obtain a grayscale average value and a grayscale value range of background pixels in the grayscale image;

[0026] Filtering the background pixels of the histogram to obtain a filtered histogram, and then traversing the remaining target pixels and their grayscale values ​​in the histogram to obtain the grayscale value corresponding to the maximum inter-class variance between the target pixels;

[0027] The grayscale value corresponding to the maximum inter-class variance is used as a threshold, and the table image is binarized to obtain a preprocessed table image.

[0028] In a possible design, after filtering the background pixels of the histogram to obtain a filtered histogram, the table recognition method further includes:

[0029] The filtered histogram is imported into the trained ESRGAN model to perform super-resolution conversion on the grayscale image.

[0030] In a possible design, after the table image is binarized, the table recognition method further includes:

[0031] Based on Hough transform, the first line segment set in the table image after binarization processing is obtained;

[0032] Calculating an average value of the inclination angles of the first set of line segments;

[0033] Filtering the second line segments deviating from the average value of the inclination angles in the first line segment set to obtain a second line segment set;

[0034] Obtaining the average value of the inclination angles of the second line segment set to obtain the inclination angle of the table image after the binarization process;

[0035] According to the tilt angle of the binarized table image, the binarized table image is corrected based on a plane affine transformation.

[0036] In a possible design, extracting a point set of each intersection in the table in the enhanced table image, and obtaining a table structure of the enhanced table image according to the point set, includes:

[0037] By using a matrix intersection method, extracting a point set of each intersection point in the table in the enhanced table image;

[0038] Determine the structure of all cells in the enhanced table image according to the point set and the enhanced table image;

[0039] According to the structures of all cells, the table structure of the enhanced table image is obtained.

[0040] In one possible design, determining the structure of all cells in the enhanced table image according to the point set and the enhanced table image includes:

[0041] Construct a cell class, where the attributes of the cell class include the coordinates of the upper left corner of the cell, the coordinates of the lower right corner of the cell, and the belonging intersection;

[0042] Screening the point set of all intersections in the table in the enhanced table image, selecting a unique intersection from each point set, and then sorting the horizontal and vertical coordinates of all intersections respectively to obtain sorted intersections;

[0043] According to the horizontal and vertical coordinates of the sorted intersection points, all possible cell lengths and widths are clustered respectively;

[0044] Constructing a complete table according to the enhanced table image, wherein the complete table does not have merged cells;

[0045] For the upper left corner coordinates of any two adjacent cells in the complete table, a third line segment perpendicular to the line connecting the two intersection points and having a length of a preset value is constructed pixel by pixel between the two intersection points to obtain a third line segment set;

[0046] Taking the enhanced table image as a reference object, determining whether there is at least one third line segment in the third line segment set whose pixel value sum is 0, if so, determining that the two cells corresponding to the third line segment in the binarized table image are merged cells, and then modifying the class attribute of the latter cell of the two cells so that it belongs to the same intersection as the former cell, if not, determining that the two cells corresponding to the third line segment in the binarized table image are independent cells;

[0047] All cells in the enhanced table image are traversed until the structure of all cells in the enhanced table image is obtained.

[0048] In a possible design, text detection is performed on the enhanced table image to obtain a text detection result, including:

[0049] Based on the trained DB model, text detection is performed on the enhanced table image to obtain a text detection result.

[0050] In a second aspect, the present invention provides a table recognition system for implementing any of the table recognition methods described above; the table recognition system comprises:

[0051] A table image extraction module is used to obtain a document image and extract an image area containing only a table from the document image to obtain a table image;

[0052] A table image preprocessing module, which is communicatively connected to the table image extraction module and is used to preprocess the table image to obtain a preprocessed table image;

[0053] A table image enhancement module, communicatively connected to the image preprocessing module, for performing signal enhancement on table lines in the preprocessed table image to obtain an enhanced table image;

[0054] a table structure recognition module, communicatively connected to the table image enhancement module, for extracting a point set of each intersection in the table in the enhanced table image, and obtaining a table structure of the enhanced table image according to the point set;

[0055] A text detection module, communicatively connected to the table image enhancement module, for performing text detection on the enhanced table image to obtain a text detection result;

[0056] A text recognition module, communicatively connected to the text detection module, for performing text recognition on the text detection result to obtain a text recognition result;

[0057] The electronic table generation module is communicatively connected to the table structure recognition module and the text recognition module, and is used to generate an electronic table according to the table structure and the text recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a structural schematic diagram of a table recognition method in the present invention;

[0059] Figure 2 It is a structural schematic diagram of label matching for the target detection model in the present invention;

[0060] Figure 3 is an example diagram of an enhanced table image in the present invention;

[0061] Figure 4 It is a module block diagram of the table recognition system in the present invention. DETAILED DESCRIPTION

[0062] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0063] It should be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.

[0064] It should be understood that it should also be noted that in some alternative embodiments, the functions / actions that appear may be different from the order in which the drawings appear. For example, depending on the functions / actions involved, two figures shown in succession may actually be performed substantially concurrently, or may sometimes be performed in reverse order.

[0065] Embodiment 1:

[0066] A first aspect of this embodiment provides a form recognition method and system, which can be executed by, but is not limited to, a computer device or a virtual machine with certain computing resources, such as a personal computer (PC, which refers to a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants (PADs) or wearable devices, or a virtual machine Hypervisor, so as to automatically enter and archive complex documents such as insurance documents.

[0067] like Figure 1 As shown, a table recognition method and system may include but is not limited to the following steps:

[0068] S1. Obtain a document image, and extract an image region containing only a table from the document image to obtain a table image; in this embodiment, the document image may be, but is not limited to, a PDF, a photographed image, a scanned image, etc.;

[0069] In step S1, an image region containing only a table is extracted from the document image to obtain a table image, including:

[0070] S101. Construct a target detection model; it should be noted that the target detection model can be divided into a feature extraction module, a feature fusion module and a classification detection module, which are respectively used to perform feature extraction, feature fusion and classification detection operations on the received data.

[0071] In the prior art, for documents with complex layouts, there will be a large amount of redundant information and noise interference items. Using traditional image processing technology to obtain the table position in the document is very unstable, which is not conducive to obtaining the table image. To solve this technical problem, this embodiment makes the following improvements: the target detection model adopts the FCOS model (Fully Convolutional One-Stage Object Detection, full convolution single-stage target detection model). It should be noted that by adopting the FCOS model and using the FCOS detection method, the accuracy of the target position detection can be greatly improved; at the same time, the FCOS detection method is an anchor-free detection method, which greatly reduces the training time and development difficulty, and can also reduce the training parameters and speed up the reasoning speed, and greatly reduce the number of regression objects that need to be trained, while the generalization performance is stronger.

[0072] S102. Obtain a sample set, import the sample set into the target detection model, perform multi-scale training on the target detection model, and obtain a target detection model after initial training; wherein, the samples in the sample set include document images containing tables; in this embodiment, the sample set uses the open source TableBank dataset, specifically, the TableBank dataset is an image-based table detection and recognition dataset, which is based on Word and Latex documents on the Internet and constructed using a novel weak supervision mechanism, and contains a total of 417,234 high-quality labeled tables and their original documents in various fields. It should be understood that after the target detection model is trained at multiple scales, only a feature map of one scale is retained.

[0073] In this embodiment, when the target detection model is trained at multiple scales, it includes: obtaining feature maps of multiple scales, and then matching the correspondence between the regression object and the true label in the feature maps of different scales, so as to achieve feature extraction and integration, and obtain the final feature vector. In this embodiment, the feature maps of different scales are set to 5 types, and the feature maps of multiple scales are obtained by inputting the document image into the feature fusion module in the target detection model for processing. When matching the correspondence between the regression object and the true label in the feature maps of different scales, such as Figure 2 The figure shows the structure diagram of label matching. The middle figure is an 8*8 feature map, on which each pixel corresponds to a region on the original image in proportion; each small region on the original image is used as a regression object. Each regression object is compared with the true label frame. The regression object contained in the true label frame is retained, otherwise it is removed. Figure 2 In the left figure, a small area A in the second row and fourth column is included in a real label, while a small area in the third row and second column is not included in any real label, so A is retained as a valid regression object, and B is removed. More specifically, in this embodiment, a matrix of the corner point coordinates of the candidate regression object is constructed, a matrix of the corner point coordinates of the real label box is constructed, and matrix subtraction is used to quickly screen the valid regression object.

[0074] In this embodiment, an empirically based scale size can be set for each scale feature map; for a feature map of a single scale, if the ratio of the true label box to the original image does not match the scale size corresponding to the feature map, the regression object on the feature map does not need to pay attention to the true label box.

[0075] After matching the corresponding relationship between the regression object and the real label, in any single scale feature map, it is still possible that one regression object matches multiple real label frames. In this case, the regression object is called a fuzzy sample. This embodiment selects a real label frame with a smaller area and pairs it with the fuzzy sample as the regression target of the fuzzy sample, so that this embodiment can filter out most of the candidate regression objects, reduce the difficulty of training, and speed up the reasoning speed.

[0076] S103. Use pruning technology to simplify the target detection model after initial training to obtain a simplified target detection model; it should be noted that this embodiment uses pruning technology to simplify the model, which is suitable for situations where the overall size of input documents such as insurance policies is relatively small in actual scenarios.

[0077] S104. Based on transfer learning, the sample set is labeled to obtain labeling information; it should be noted that, in this embodiment, the sample set is labeled based on transfer learning, which can reduce the difficulty of manual labeling tasks in the early stage of model training. It should be understood that the labeling information of the sample set is obtained by manually labeling input documents such as insurance policies.

[0078] S105. Input the annotation information into the simplified target detection model, and retrain the simplified target detection model to obtain a final trained target detection model;

[0079] S106. Input the document image into the final trained object detection model to obtain a table image.

[0080] S2. Preprocessing the table image to obtain a preprocessed table image; it should be noted that the preprocessing is used to remove noise and interference items such as background watermarks and red seals in the table image, and to perform tilt angle correction and other processing on the table image to obtain a clear table image;

[0081] When the document background is complex (e.g., different background colors and many types of watermark colors), in the prior art, when preprocessing the table image, the foreground image and the background watermark are usually distinguished by image binarization, etc. However, when the difference between the background watermark and the background is large, the effect of directly preprocessing the table image by binarization is very poor, making it difficult to distinguish the foreground, background and background watermark at the same time. The reason is that the original algorithm will classify the background watermark into the category of the foreground, thereby failing to achieve the purpose of distinguishing the foreground. In order to solve the above problems of the prior art, the present embodiment further makes the following improvements: preprocessing the table image to obtain a preprocessed table image, including:

[0082] S201. Convert the table image into a grayscale image, and then calculate the histogram of the grayscale image. Since the background is the largest part of the image, the grayscale average value and grayscale value range of the background pixels in the grayscale image can be obtained by performing statistical analysis on the histogram;

[0083] S202. Filtering the background pixels of the histogram to obtain a filtered histogram;

[0084] In the prior art, when filtering the background and background watermark, it is inevitable that a slight information loss will be caused to the grayscale image. If the pixels of the input original image are too low, and in the subsequent inspection process, it is found that the confidence or accuracy of text recognition is generally low, the clarity of the filtered histogram will be too low. To solve this technical problem, in this embodiment, after filtering the background pixels of the histogram and obtaining the filtered histogram, the table recognition method further includes:

[0085] S203. Import the filtered histogram into the trained ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) model to perform super-resolution conversion on the low-resolution grayscale image. This can repair the table lines and the blurred areas of the table content in the table image to a certain extent, thereby improving the accuracy and stability of subsequent table recognition and text recognition.

[0086] In this embodiment, the process of obtaining the trained ESRGAN model is as follows:

[0087] Build the ESRGAN model;

[0088] Obtain a batch of table images, each table image includes a high-resolution image and a low-resolution image, wherein the low-resolution image is obtained by downsampling the high-resolution image;

[0089] The batch of table images is input into the ESRGAN model for training to obtain a trained ESRGAN model.

[0090] S204. Traverse the remaining target pixels and their grayscale values ​​in the histogram to obtain the grayscale value corresponding to the maximum inter-class variance between the target pixels; in this embodiment, the remaining target pixels and their grayscale values ​​in the histogram are the remaining target pixels and their grayscale values ​​after the histogram is removed from the filtered histogram;

[0091] S205. Using the gray value corresponding to the maximum inter-class variance as a threshold, binarizing the table image;

[0092] It should be noted that, during the binarization process, the horizontal lines of the table can be filtered out by using a morphological kernel of a corresponding scale according to the image size, and the vertical lines of the table can be filtered out through the corrosion and expansion operation of the binary image, and only the horizontal lines in the table are retained. In this embodiment, after the table image is binarized, the table recognition method further includes:

[0093] S206. Based on the Hough transform, obtain the first line segment set in the table image after binarization processing;

[0094] S207. Calculate the average inclination angle of the first line segment set;

[0095] S208. Filter the second line segments deviating from the average value of the inclination angle in the first line segment set to obtain a second line segment set;

[0096] S209. Obtaining the average value of the inclination angles of the second line segment set to obtain the inclination angle of the table image after the binarization process;

[0097] S210. According to the tilt angle of the binarized table image, based on the plane affine transformation, the binarized table image is corrected;

[0098] It should be noted that the above process can realize the tilt correction of the table image after the binarization process, thereby realizing the adjustment of the spatial domain of the table image.

[0099] S211. Obtaining a preprocessed table image. It should be noted that the preprocessed table image can effectively distinguish the foreground image from other images (other images include background, background watermark and other noise data).

[0100] The Otsu algorithm in the prior art usually calculates the threshold value that minimizes the intra-class variance and maximizes the inter-class variance through exhaustive search. This embodiment improves the Otsu threshold method to achieve adaptive calculation of the threshold value, which can adapt to documents such as insurance policies of different companies or types of insurance, avoid using preset templates, and stably filter out backgrounds and background watermarks (including red seals, etc.), making subsequent table line extraction more stable.

[0101] S3. Perform signal enhancement on the table lines in the preprocessed table image to obtain an enhanced table image; it should be noted that when performing signal enhancement on the table lines, conventional digital image processing techniques may be used, including but not limited to morphological image processing techniques, Hough transform processing techniques, etc., which are not limited here; in this embodiment, the enhanced table image is as follows Figure 3 As shown;

[0102] S4. Extracting the point set of each intersection in the table in the enhanced table image, and obtaining the table structure of the enhanced table image according to the point set;

[0103] In this embodiment, extracting a point set of each intersection in the table in the enhanced table image, and obtaining a table structure of the enhanced table image according to the point set includes:

[0104] S401. Extracting the point set of each intersection point in the table in the enhanced table image by means of matrix intersection method; compared with the traditional method of first calculating the fitting straight line and then calculating the intersection point of the horizontal and vertical lines by mathematical formula, in this embodiment, the matrix intersection method can greatly reduce the complexity of point set calculation and extraction;

[0105] S402. Determine the structure of all cells in the enhanced table image according to the point set and the enhanced table image;

[0106] In this embodiment, determining the structure of all cells in the enhanced table image according to the point set and the enhanced table image includes:

[0107] S4021. Construct a cell class, wherein the attributes of the cell class include the coordinates of the upper left corner of the cell, the coordinates of the lower right corner of the cell, and the intersection point (assuming that they belong to the same intersection point, they are the same cell);

[0108] S4022. Screening the point set of all intersections in the table in the enhanced table image, selecting a unique intersection from each point set, and then sorting the horizontal and vertical coordinates of all intersections to obtain sorted intersections;

[0109] S4023. Clustering all possible cell lengths and widths according to the horizontal and vertical coordinates of the sorted intersection points;

[0110] S4024. Construct a complete table according to the enhanced table image, wherein the complete table does not have merged cells. In this embodiment, the constructed complete table is 17 rows*11 columns;

[0111] S4025. For the upper left corner coordinates of any two adjacent cells in the complete table, a third line segment perpendicular to the line connecting the two intersection points and having a length of a preset value (the preset value is 10pix in this embodiment) is constructed pixel by pixel between the two intersection points to obtain a third line segment set;

[0112] S4026. Take the enhanced table image as a reference object, determine whether there is at least one third line segment in the third line segment set whose pixel value sum is 0, if so, determine that the two cells corresponding to the third line segment in the binarized table image are merged cells, and then modify the class attribute of the latter of the two cells so that it belongs to the same intersection as the former cell, if not, determine that the two cells corresponding to the third line segment in the binarized table image are independent cells;

[0113] All cells in the enhanced table image are traversed from left to right and from top to bottom, and the steps of obtaining the third line segment set to determine the cell type (i.e., steps S4025-S4026) are repeatedly executed until the structure of all cells in the enhanced table image is obtained.

[0114] In this embodiment, the step of obtaining the table structure of the enhanced table image is to construct a series of line sets between the vertices of any two adjacent cells, perform statistical analysis on each pixel point of the line set, so as to judge the structure of the cell, and finally obtain the table structure of the enhanced table image, and the result has a high fault tolerance rate.

[0115] S403. Obtain the table structure of the enhanced table image according to the structures of all cells.

[0116] S5. Perform text detection on the enhanced table image to obtain a text detection result;

[0117] In the prior art, methods for text detection can be divided into methods based on regression and methods based on segmentation. Among them, the method based on regression was developed earlier and can be subdivided into methods based on object boxes and methods based on pixel value regression. The method based on object box regression has a decent effect on regular-shaped text detection, but has a poor effect on irregular-shaped text detection; the method based on pixel value regression can process irregular text, but its real-time performance is extremely poor. The method based on segmentation is also aimed at pixel-level text detection, but the segmentation network only needs to classify the foreground and background for each pixel, and the training and inference time are greatly improved compared to the method based on pixel regression. Based on this, in this embodiment, text detection is performed on the enhanced table image to obtain text detection results, including:

[0118] Based on the trained DB (differentiable binarization) model, text detection is performed on the enhanced table image to obtain a text detection result. In this embodiment, by using an approximate binarization formula, the non-differentiable image binarization process is transformed into a differentiable process, and it becomes a part of the network training, so that the threshold can be adaptively trained, thereby improving the performance of text block segmentation and speeding up the training and text detection.

[0119] S6. Perform text recognition on the text detection result to obtain a text recognition result; it should be noted that, in this embodiment, the text recognition can be performed using, but is not limited to, the CRNN neural network model and the CTC loss function, which are not limited here;

[0120] S7. Generate an electronic form according to the table structure and the text recognition result. In this embodiment, the electronic form can be in, but is not limited to, JSON format or Excel format, which is not limited here.

[0121] This embodiment can detect, identify and restore tables in documents such as insurance policies with complex layouts, facilitate the automatic entry and archiving management of tables in complex documents such as insurance documents, and solve the problem of difficulty in table recognition caused by the diverse types and complex layouts of documents such as insurance policies. At the same time, it can improve the accuracy of identifying the table structure and content in complex documents such as insurance policies. Compared with the existing technology, this embodiment reduces the difficulty of development and training, improves the detection and recognition speed, and at the same time improves the versatility and stability of table recognition in documents.

[0122] Embodiment 2:

[0123] This embodiment provides a table recognition system for implementing the table recognition method in Embodiment 1; Figure 4 As shown, the table recognition system includes:

[0124] A table image extraction module is used to obtain a document image and extract an image area containing only a table from the document image to obtain a table image;

[0125] A table image preprocessing module, which is communicatively connected to the table image extraction module and is used to preprocess the table image to obtain a preprocessed table image;

[0126] A table image enhancement module, communicatively connected to the image preprocessing module, for performing signal enhancement on table lines in the preprocessed table image to obtain an enhanced table image;

[0127] a table structure recognition module, communicatively connected to the table image enhancement module, for extracting a point set of each intersection in the table in the enhanced table image, and obtaining a table structure of the enhanced table image according to the point set;

[0128] A text detection module, communicatively connected to the table image enhancement module, for performing text detection on the enhanced table image to obtain a text detection result;

[0129] A text recognition module, communicatively connected to the text detection module, for performing text recognition on the text detection result to obtain a text recognition result;

[0130] The electronic table generation module is communicatively connected to the table structure recognition module and the text recognition module, and is used to generate an electronic table according to the table structure and the text recognition result.

[0131] Embodiment 3:

[0132] Based on Embodiment 1 or 2, this embodiment discloses an electronic device, which may be a smart phone, a tablet computer, a laptop computer, or a desktop computer, etc. The electronic device may be referred to as a terminal, a portable terminal, a desktop terminal, etc., and the electronic device includes:

[0133] a memory for storing computer program instructions; and,

[0134] A processor is used to execute the computer program instructions to complete the operation of the table recognition method as described in any one of Embodiment 1.

[0135] Embodiment 4:

[0136] Based on any one of Embodiments 1 to 3, this embodiment discloses a computer-readable storage medium for storing computer-readable computer program instructions, wherein the computer program instructions are configured to execute the operations of the table recognition method described in Embodiment 1 when run.

[0137] It should be noted that if the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0138] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0139] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention is described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0140] Finally, it should be noted that the present invention is not limited to the above optional implementations, and anyone can derive other various forms of products under the enlightenment of the present invention. The above specific implementations should not be understood as limiting the scope of protection of the present invention. The scope of protection of the present invention should be based on the definition in the claims, and the description can be used to interpret the claims.

Claims

1. A table recognition method, characterized in that: include: Acquire a document image, and extract an image region containing only a table from the document image to obtain a table image; Preprocessing the table image to obtain a preprocessed table image; Performing signal enhancement on the table lines in the preprocessed table image to obtain an enhanced table image; Extracting a point set of each intersection point in the table in the enhanced table image, and obtaining a table structure of the enhanced table image according to the point set; Performing text detection on the enhanced table image to obtain a text detection result; Performing text recognition on the text detection result to obtain a text recognition result; Generate an electronic form according to the table structure and the text recognition result; Preprocessing the table image to obtain a preprocessed table image includes: Converting the table image into a grayscale image, calculating a histogram of the grayscale image, and performing statistical analysis on the histogram to obtain a grayscale average value and a grayscale value range of background pixels in the grayscale image; Filtering the background pixels of the histogram to obtain a filtered histogram, and then traversing the remaining target pixels and their grayscale values ​​in the histogram to obtain the grayscale value corresponding to the maximum inter-class variance between the target pixels; Using the gray value corresponding to the maximum inter-class variance as a threshold, performing binarization processing on the table image to obtain a preprocessed table image; After binarization processing is performed on the table image, the table recognition method further includes: Based on Hough transform, the first line segment set in the table image after binarization processing is obtained; Calculating an average value of the inclination angles of the first set of line segments; Filtering the second line segments deviating from the average value of the inclination angles in the first line segment set to obtain a second line segment set; Obtaining the average value of the inclination angles of the second line segment set to obtain the inclination angle of the table image after the binarization process; According to the tilt angle of the binarized table image, based on the plane affine transformation, the binarized table image is corrected; Extracting a point set of each intersection in the table in the enhanced table image, and obtaining a table structure of the enhanced table image according to the point set, including: By using a matrix intersection method, extracting a point set of each intersection point in the table in the enhanced table image; Determine the structure of all cells in the enhanced table image according to the point set and the enhanced table image; According to the structures of all cells, a table structure of the enhanced table image is obtained; Determining the structure of all cells in the enhanced table image according to the point set and the enhanced table image, including: Construct a cell class, where the attributes of the cell class include the coordinates of the upper left corner of the cell, the coordinates of the lower right corner of the cell, and the belonging intersection; Screening the point set of all intersections in the table in the enhanced table image, selecting a unique intersection from each point set, and then sorting the horizontal and vertical coordinates of all intersections respectively to obtain sorted intersections; According to the horizontal and vertical coordinates of the sorted intersection points, all possible cell lengths and widths are clustered respectively; Constructing a complete table according to the enhanced table image, wherein the complete table does not have merged cells; For the upper left corner coordinates of any two adjacent cells in the complete table, a third line segment perpendicular to the line connecting the two intersection points and having a length of a preset value is constructed pixel by pixel between the two intersection points to obtain a third line segment set; Taking the enhanced table image as a reference object, determining whether there is at least one third line segment in the third line segment set whose pixel value sum is 0, if so, determining that the two cells corresponding to the third line segment in the binarized table image are merged cells, and then modifying the class attribute of the latter cell of the two cells so that it belongs to the same intersection as the former cell, if not, determining that the two cells corresponding to the third line segment in the binarized table image are independent cells; All cells in the enhanced table image are traversed until the structure of all cells in the enhanced table image is obtained.

2. A table recognition method according to claim 1, characterized in that: Extracting an image region containing only a table from the document image to obtain a table image includes: Build an object detection model; Acquire a sample set, import the sample set into the target detection model, perform multi-scale training on the target detection model, and obtain a target detection model after initial training; wherein the samples in the sample set include document images containing tables; Using pruning technology, the target detection model after initial training is simplified to obtain a simplified target detection model; Based on transfer learning, the sample set is labeled to obtain labeling information; Input the annotation information into the simplified target detection model, train the simplified target detection model again, and obtain the final trained target detection model; The document image is input into the final trained object detection model to obtain a table image.

3. A table recognition method according to claim 2, characterized in that: The target detection model adopts the FCOS model.

4. A table recognition method according to claim 1, characterized in that: After filtering the background pixels of the histogram to obtain a filtered histogram, the table recognition method further includes: The filtered histogram is imported into the trained ESRGAN model to perform super-resolution conversion on the grayscale image.

5. A table recognition method according to claim 1, characterized in that: Performing text detection on the enhanced table image to obtain a text detection result includes: Based on the trained DB model, text detection is performed on the enhanced table image to obtain a text detection result.

6. A table recognition system, characterized in that: Used to implement the table recognition method according to any one of claims 1 to 5; the table recognition system comprises: A table image extraction module is used to obtain a document image and extract an image area containing only a table from the document image to obtain a table image; A table image preprocessing module, which is communicatively connected to the table image extraction module and is used to preprocess the table image to obtain a preprocessed table image; A table image enhancement module, communicatively connected to the image preprocessing module, for performing signal enhancement on table lines in the preprocessed table image to obtain an enhanced table image; a table structure recognition module, communicatively connected to the table image enhancement module, for extracting a point set of each intersection in the table in the enhanced table image, and obtaining a table structure of the enhanced table image according to the point set; A text detection module, communicatively connected to the table image enhancement module, for performing text detection on the enhanced table image to obtain a text detection result; A text recognition module, communicatively connected to the text detection module, for performing text recognition on the text detection result to obtain a text recognition result; The electronic table generation module is communicatively connected to the table structure recognition module and the text recognition module, and is used to generate an electronic table according to the table structure and the text recognition result.

Images