Unstructured document recognition method and system

Abstract

This invention relates to the field of document recognition technology, and more particularly to a method and system for recognizing unstructured documents. The method includes: acquiring a PDF document to be recognized; segmenting the PDF document to form a row dataset; dividing the row dataset into regions to obtain text regions and table regions; and using a recognition method to recognize data in the text regions and table regions respectively, obtaining data in the text regions and data in the table regions respectively. The purpose of this invention is to solve the problem of low accuracy in recognizing text and tables in unstructured documents using existing technologies.

Description

Technical Field

[0001] This invention relates to the field of document recognition technology, and in particular to a method and system for recognizing unstructured documents. Background Technology

[0002] Currently, Optical Character Recognition (OCR) document recognition based on computer vision has been widely used in the commercial field, such as the recognition of common ID cards, bank cards, and driver's licenses. It uses optical and computer technologies to read the text printed or written on paper and convert it into a format that can be accepted by computers and understood by humans.

[0003] In existing technologies, optical character recognition document recognition generally includes two steps: text detection and text recognition. Text detection mainly uses methods such as deep learning or machine learning to detect text regions in an image, providing an initial image for the subsequent text recognition module. Text recognition mainly involves recognizing and processing the text in the detection box.

[0004] However, most existing unstructured documents (such as unstructured archive documents) contain text paragraphs and tables. When processing documents containing tables, existing technologies directly recognize the tables as text and vertical lines (such as "|"), resulting in inaccurate data recognition results. Summary of the Invention

[0005] This invention provides a method and system for recognizing unstructured documents, which solves the problem of low recognition accuracy of text and tables in unstructured documents in existing technologies.

[0006] This invention provides a method for unstructured document recognition, comprising:

[0007] Obtain the PDF document of the document to be recognized;

[0008] The PDF document is segmented to form a row dataset;

[0009] The row dataset is divided into regions to obtain text regions and table regions;

[0010] A recognition method is used to identify the data in the text area and the table area respectively, and the data in the text area and the data in the table area are obtained respectively.

[0011] According to the present invention, an unstructured document recognition method is provided, wherein segmenting the PDF format document to form a line dataset includes:

[0012] Perform basic processing on each page of the PDF document to obtain all the basic pages;

[0013] Traverse the pixels in each row of each base page, remove the rows in the base page where the pixels are all of a single color, and merge the consecutive rows into one row to form a row dataset.

[0014] According to the unstructured document recognition method provided by the present invention, the line dataset is divided into regions to obtain text regions and table regions, including:

[0015] Using a projection method, each row in the row dataset is projected horizontally to obtain the row attribute of each row as a text line or a table line.

[0016] Based on the row attributes of all horizontal lines as either text lines or table lines, the row dataset is divided into regions to form text regions and table regions.

[0017] According to the present invention, an unstructured document recognition method is provided, wherein a projection method is used to perform horizontal projection on each row in the row dataset to obtain the row attribute of each row as a text line or a table line, including:

[0018] The projection method is used to project each row in the row dataset horizontally to obtain the projection line of each row.

[0019] If the projection lines of each row are continuous, then it is a row attribute of the table row;

[0020] If the projection lines of each row are not continuous, then it is a line attribute of the text row.

[0021] According to the unstructured document recognition method provided by the present invention, the method involves dividing the row dataset into text regions and table regions based on the row attributes of all horizontal rows as text lines or table lines, including:

[0022] Project each row of the text line's row attribute vertically. If the vertical height equals the row height, then it is a table content row.

[0023] After merging consecutive rows of table content into one row, each row of table content is represented as a table block;

[0024] The row above the first row in the table block is merged into the table block to obtain a new table block;

[0025] If the row attribute of the row below the last row in the new table block is a table line, then the row below the last row in the new table block is merged into the new table block, resulting in a complete table block and a non-table block. The complete table block is represented as a table area, and the non-table block is represented as a text area.

[0026] According to the unstructured document recognition method provided by the present invention, the line dataset is divided into regions to obtain text regions and table regions, including:

[0027] A preset region partitioning model is used to partition the row dataset into regions, resulting in text regions and table regions.

[0028] The preset region segmentation model is trained based on a neural network model, with each row dataset sample as input and text region samples and table region samples as output.

[0029] According to the unstructured document recognition method provided by the present invention, before the step of obtaining the PDF format document of the document to be recognized, the method further includes:

[0030] The document to be recognized is preprocessed, including brightness adjustment, binarization, and noise reduction, to obtain a preprocessed document;

[0031] The preprocessed document is converted to PDF format to obtain the PDF document of the document to be recognized.

[0032] According to the present invention, an unstructured document recognition method is provided, which uses a recognition method to perform data recognition on the text region and the table region respectively, and obtains the data in the text region and the data in the table region respectively, including:

[0033] Divide the table area into multiple cells;

[0034] A text recognition method is used to recognize data in the text area and each cell, respectively, to obtain the data in the text area and the data in the table area.

[0035] According to the present invention, an unstructured document recognition method is used to identify data in a table area to obtain data in the table area, including:

[0036] Distinguish between the header area and the data area of ​​the table region;

[0037] Convert the header area into XML format and create an XML data table;

[0038] Each row of the data area is stored as a record in the XML data table;

[0039] The XML data table is converted into the same format as the data in the text area to obtain the data in the table area.

[0040] The present invention also provides an unstructured document recognition system, comprising:

[0041] The acquisition module is used to acquire PDF documents of the document to be recognized;

[0042] The segmentation module is used to segment the PDF document to form a row dataset.

[0043] The segmentation module divides the row dataset into regions, resulting in text regions and table regions.

[0044] The recognition module is used to perform data recognition on the text area and the table area respectively using recognition methods, and obtain the data in the text area and the data in the table area respectively.

[0045] The present invention provides an unstructured document recognition method and system that segments PDF format documents to obtain a row dataset. After further dividing the row dataset, it can first distinguish between text regions and table regions, preventing direct recognition of table lines as vertical lines. Then, it classifies and recognizes the text regions and table regions, thereby improving the accuracy of data recognition in text regions and table regions. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating the unstructured document recognition method provided by the present invention.

[0048] Figure 2 This is a schematic diagram of the unstructured document recognition system provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] The following is combined with Figure 1 A method for recognizing unstructured documents according to the present invention includes:

[0051] S1. Obtain the PDF document of the document to be recognized. Specifically, if the document is a paper document, take a photo or scan the paper document to obtain the PDF document of the document to be recognized.

[0052] S2. Segment the PDF document to form a row dataset. This row dataset can be denoted as HList. The row dataset can also be numbered using an index, thus allowing each row in the dataset to be labeled.

[0053] S3. Divide the row dataset into regions to obtain text regions and table regions. The divided text regions and table regions are clearly distinguishable, thus avoiding the misidentification of vertical lines in the table as vertical lines and reducing recognition confusion. Both text regions and table regions can be numbered using an index to label each region.

[0054] S4. Using a recognition method, data recognition is performed on the text area and the table area respectively to obtain the data in the text area and the data in the table area respectively.

[0055] This invention segments PDF documents to obtain row datasets, then further divides these row datasets to distinguish between text and table regions, preventing confusion between them and avoiding the misidentification of vertical lines in tables as vertical lines. Simultaneously, it classifies and identifies text and table regions, thereby improving the accuracy of data recognition in both areas.

[0056] Based on the above embodiments, the step of segmenting the PDF format document to form a row dataset includes:

[0057] Perform basic processing on each page of the PDF document to obtain all the basic pages.

[0058] Traverse the pixels within each row of each base page, remove rows containing only monochrome pixels, and merge consecutive rows into a single row to form a row dataset. For example, monochrome pixels can be white or black. That is, if a row consists entirely of white pixels, it is considered invalid; otherwise, it is valid. Simultaneously, consecutive valid rows within each base page are treated as a single row to form a row dataset.

[0059] Based on the above embodiments, the row dataset is divided into regions to obtain text regions and table regions, including:

[0060] The projection method is used to project each row in the row dataset horizontally to obtain the row attribute of each row as a text line or a table line.

[0061] Based on the row attribute of all horizontal lines being either text lines or table lines, the row dataset is divided into text regions and table regions. This division of row attributes into text lines and table lines ensures that table lines are not intentionally ignored during recognition but are instead used as row separators within the table regions.

[0062] Based on the above embodiments, the method of using projection to project each row in the row dataset horizontally to obtain the row attribute of each row as a text line or a table line includes:

[0063] The projection method is used to project each row in the row dataset horizontally to obtain the projection line of each row.

[0064] If the projection lines of each row are continuous, then it is a row attribute of the table row.

[0065] If the projection lines of each row are not continuous, then it is a line attribute of the text row.

[0066] By judging the projection lines of each row, it can be determined whether each row is a table row or a text row.

[0067] Based on the above embodiments, the step of dividing the row dataset into text regions and table regions based on the row attributes of all horizontal rows as text lines or table lines includes:

[0068] Project each row of the text line's row attribute vertically. If the vertical height equals the row height, then it is a table content row.

[0069] After merging consecutive rows of table content into one row, each row of table content is represented as a table block.

[0070] The row above the first row in the table block is merged into the table block to obtain a new table block.

[0071] If the row attribute of the row following the last row in the new table block is a table line, then the row following the last row in the new table block is merged into the new table block, resulting in a complete table block and a non-table block. The complete table block is represented as a table area, and the non-table block is represented as a text area. The vertical height of each line of text is projected vertically and compared with the row height to determine whether the line is a table content line or a plain text line. The table line is then used as a row separator within the table area to avoid recognizing vertical lines in the table as vertical lines.

[0072] Based on the above embodiments, the row dataset is divided into regions to obtain text regions and table regions, including:

[0073] A preset region partitioning model is used to partition the row dataset into regions, resulting in text regions and table regions.

[0074] The preset region partitioning model is trained on a neural network model, using each row dataset sample as input and text region samples and table region samples as output. Specifically, during training, the preset region partitioning model is trained by forward and backward propagation based on the neural network model.

[0075] In this embodiment, the neural network model includes one input layer, three convolutional layers, three ReLU nonlinear activation layers, three pooling layers, one fully connected layer, and one output layer connected in sequence.

[0076] Meanwhile, forward propagation extracts feature images from each row of the row dataset samples through convolution and pooling, transforms these feature images into one-dimensional vectors, inputs them into fully connected layers, and the classifier derives the recognition result, i.e., the output vector. Each value in the output vector represents the probability that the input data corresponds to either a row of table content or a row of plain text. Backpropagation is used when the output of forward propagation does not distinguish between the corresponding row of table content and row of plain text in the expected output. Stochastic gradient descent is then used to optimize the backpropagation network training, updating the parameters of the convolutional layers. This achieves more accurate text and table region segmentation in the output of the predefined region segmentation model.

[0077] Based on the above embodiments, before the step of obtaining the PDF document of the document to be identified, the method further includes:

[0078] The document to be recognized is preprocessed, including brightness adjustment, binarization, and noise reduction, to obtain a preprocessed document.

[0079] The preprocessing steps for brightness adjustment are as follows: perform brightness correction on each page of the document to be recognized to obtain all the first base pages.

[0080] Specifically, each page in the document to be recognized is converted into a first grayscale image.

[0081] Gaussian filtering is applied to the first grayscale image to obtain the second grayscale image.

[0082] The second grayscale image is divided into several regions on an average basis.

[0083] Based on the aforementioned regions, an average grayscale matrix and a standard deviation matrix are constructed.

[0084] Based on the average grayscale matrix and standard deviation matrix, the background grayscale matrix is ​​obtained.

[0085] An ideal background gray value is generated based on the sum of the mean and standard deviation of the background gray matrix.

[0086] Based on the ideal background grayscale and the background grayscale matrix, the grayscale ratio matrix is ​​obtained.

[0087] Using bicubic interpolation, the grayscale ratio matrix is ​​interpolated and expanded to the same size as the image to be brightness corrected, resulting in a grayscale ratio expansion matrix.

[0088] Convert the image to be brightness corrected from the RGB color space to the YUV color space.

[0089] The Y component of the image to be brightness corrected is multiplied by the corresponding element in the gray-scale ratio expansion matrix to obtain the calculated Y component.

[0090] Determine whether the value of the Y component after the calculation exceeds the upper limit of the grayscale value, and set the Y component that exceeds the upper limit to the upper limit of the grayscale value.

[0091] The image to be luminance corrected is converted from the YUV color space after Y component correction back to the RGB color space to obtain the luminance corrected image as the first base page.

[0092] Remove the document background color from each of the first base pages to obtain all base pages.

[0093] Specifically, the first base page is multiplied by the grayscale ratio expansion matrix to obtain the grayscale image after grayscale multiplication correction.

[0094] Construct a first background distribution matrix with the same size as the image to be brightness corrected, with all initial values ​​set to 0.

[0095] Traverse all pixels in the grayscale image after grayscale multiplication correction. If the grayscale value of a pixel is greater than the grayscale threshold, the grayscale threshold is the mean of the background grayscale matrix.

[0096] Then, the corresponding element of the first background distribution matrix is ​​assigned the value 1 to obtain the second background distribution matrix.

[0097] Iterate through all elements in the second background distribution matrix. If an element with a value of 0 has all adjacent elements with a value of 1, then assign the value of that element to 1. This process yields the third background distribution matrix.

[0098] Traverse all elements in the third background distribution matrix. If an element with a value of 1 has more than three 0s in its adjacent elements, then assign the value -1 to that element to obtain the fourth background distribution matrix.

[0099] Traverse all elements in the fourth background distribution matrix, find the position of the element with a value of 1, and set the pixel at the corresponding position of the image after brightness correction to a discarded monochrome pixel to obtain all the base pages.

[0100] The preprocessed document is converted to PDF format to obtain the PDF document of the document to be recognized.

[0101] Based on the above embodiments, a recognition method is used to perform data recognition on the text area and the table area respectively, to obtain the data in the text area and the data in the table area respectively, including:

[0102] Divide the table area into multiple cells.

[0103] A text recognition method is used to recognize data in the text area and each cell, respectively, to obtain the data in the text area and the data in the table area.

[0104] Based on the above embodiments, an identification method is used to identify data in the table area to obtain the data in the table area, including:

[0105] Distinguish between the header area and the data area of ​​the table region. Specifically, iterate through each cell of the table region, match each cell with a pre-set header keyword database, and set the cells that match successfully as header cells.

[0106] Set the row containing the header cell as the header range, and set the other ranges as the data range.

[0107] Convert the header area into XML format and create an XML data table.

[0108] Each row of the data area is stored as a record in the XML data table.

[0109] The XML data table is converted into the same format as the data in the text area, resulting in the data in the table area. This enables object storage of the recognized data.

[0110] The unstructured document recognition system provided by the present invention is described below. The unstructured document recognition system described below can be referred to in correspondence with the unstructured document recognition method described above.

[0111] Please refer to the following: Figure 2 An unstructured document recognition system includes an acquisition module 210, a segmentation module 220, a partitioning module 230, and a recognition module 240.

[0112] The acquisition module 210 is used to acquire PDF format documents of the document to be identified.

[0113] The segmentation module 220 is used to segment the PDF format document to form a row dataset.

[0114] The segmentation module 230 divides the row dataset into regions, resulting in text regions and table regions.

[0115] The recognition module 240 is used to perform data recognition on the text area and the table area respectively using a recognition method, and obtain the data in the text area and the data in the table area respectively.

[0116] This invention acquires a PDF document via an acquisition module 210, segments the document into text and table regions via a segmentation module 220, divides the document into text and table regions via a partitioning module 230, and identifies the data in the text and table regions via a recognition module 240. This distinguishes between text and table regions, preventing direct recognition that might mistake table lines for vertical lines. By further classifying and recognizing the text and table regions, the accuracy of the recognition is improved.

[0117] The segmentation module 220 is used to segment the PDF document to form a row dataset, including:

[0118] Perform basic processing on each page of the PDF document to obtain all the basic pages.

[0119] Traverse the pixels in each row of each base page, remove the rows in the base page where the pixels are all of a single color, and merge the consecutive rows into one row to form a row dataset.

[0120] The segmentation module 230 is used to segment the row dataset into regions, resulting in text regions and table regions, including:

[0121] The projection method is used to project each row in the row dataset horizontally to obtain the row attribute of each row as a text line or a table line.

[0122] Based on the row attributes of all horizontal lines as either text lines or table lines, the row dataset is divided into regions to form text regions and table regions.

[0123] The segmentation module 230 is used to perform horizontal projection on each row in the row dataset using the projection method, to obtain the row attribute of each row as a text line or a table line, including:

[0124] The projection method is used to project each row in the row dataset horizontally to obtain the projection line of each row.

[0125] If the projection lines of each row are continuous, then it is a row attribute of the table row.

[0126] If the projection lines of each row are not continuous, then it is a line attribute of the text row.

[0127] The segmentation module 230 is used to segment the row dataset into text regions and table regions based on the row attributes of all rows being text lines or table lines, including:

[0128] Project each row of the text line's row attribute vertically. If the vertical height equals the row height, then it is a table content row.

[0129] After merging consecutive rows of table content into one row, each row of table content is represented as a table block.

[0130] The row above the first row in the table block is merged into the table block to obtain a new table block.

[0131] If the row attribute of the row below the last row in the new table block is a table line, then the row below the last row in the new table block is merged into the new table block, resulting in a complete table block and a non-table block. The complete table block is represented as a table area, and the non-table block is represented as a text area.

[0132] Furthermore, the segmentation module 230 is used to segment the row dataset into regions, resulting in text regions and table regions, including:

[0133] A preset region partitioning model is used to partition the row dataset into regions, resulting in text regions and table regions.

[0134] The preset region segmentation model is trained based on a neural network model, with each row dataset sample as input and text region samples and table region samples as output.

[0135] Before the step of obtaining the PDF document of the document to be recognized, the acquisition module 210 further includes:

[0136] The document to be recognized is preprocessed, including brightness adjustment, binarization, and noise reduction, to obtain a preprocessed document.

[0137] The preprocessed document is converted to PDF format to obtain the PDF document of the document to be recognized.

[0138] The recognition module 240 is used to perform data recognition on the text area and the table area respectively using a recognition method, and to obtain the data in the text area and the data in the table area respectively, including:

[0139] Divide the table area into multiple cells.

[0140] A text recognition method is used to recognize data in the text area and each cell, respectively, to obtain the data in the text area and the data in the table area.

[0141] Simultaneously, the recognition module 240 is used to perform data recognition on the table area using a recognition method to obtain the data in the table area, including:

[0142] Distinguish between the header area and the data area of ​​the table region.

[0143] Convert the header area into XML format and create an XML data table.

[0144] Each row of the data area is stored as a record in the XML data table.

[0145] The XML data table is converted into the same format as the data in the text area to obtain the data in the table area.

[0146] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recognizing unstructured documents, characterized in that, include: Obtain the PDF document of the document to be recognized; The PDF document is segmented to form a row dataset; The row dataset is divided into regions to obtain text regions and table regions; A recognition method is used to perform data recognition on the text area and the table area respectively, and the data in the text area and the data in the table area are obtained respectively. The row dataset is divided into regions to obtain text regions and table regions, including: Using a projection method, each row in the row dataset is projected horizontally to obtain the row attribute of each row as a text line or a table line. Based on the row attributes of all horizontal lines as text lines or table lines, the row dataset is divided into regions to form text regions and table regions. The method employs a projection approach to perform horizontal projection on each row in the row dataset, obtaining the row attribute of each row as either a text line or a table line, including: The projection method is used to project each row in the row dataset horizontally to obtain the projection line of each row. If the projection lines of each row are continuous, then it is a row attribute of the table row; If the projection lines of each row are not continuous, then it is a line attribute of the text row.

2. The unstructured document recognition method according to claim 1, characterized in that, The process of segmenting the PDF document to form a row dataset includes: Perform basic processing on each page of the PDF document to obtain all the basic pages; Traverse the pixels in each row of each base page, remove the rows in the base page where the pixels are all of a single color, and merge the consecutive rows into one row to form a row dataset.

3. The unstructured document recognition method according to claim 1 or 2, characterized in that, The process of dividing the row dataset into text regions and table regions based on the row attributes of all horizontal rows as text lines or table lines includes: Project each row of the text line's row attribute vertically. If the vertical height equals the row height, then it is a table content row. After merging consecutive rows of table content into one row, each row of table content is represented as a table block; The row above the first row in the table block is merged into the table block to obtain a new table block; If the row attribute of the row below the last row in the new table block is a table line, then the row below the last row in the new table block is merged into the new table block, resulting in a complete table block and a non-table block. The complete table block is represented as a table area, and the non-table block is represented as a text area.

4. The unstructured document recognition method according to claim 1 or 2, characterized in that, The row dataset is divided into regions to obtain text regions and table regions, including: A preset region partitioning model is used to partition the row dataset into regions, resulting in text regions and table regions. The preset region segmentation model is trained based on a neural network model, with each row dataset sample as input and text region samples and table region samples as output.

5. The unstructured document recognition method according to claim 1, characterized in that, Before the step of obtaining the PDF document of the document to be recognized, the method further includes: The document to be recognized is preprocessed, including brightness adjustment, binarization, and noise reduction, to obtain a preprocessed document; The preprocessed document is converted to PDF format to obtain the PDF document of the document to be recognized.

6. The unstructured document recognition method according to claim 1, characterized in that, Using a recognition method, data recognition is performed on the text area and the table area respectively to obtain the data in the text area and the data in the table area, including: Divide the table area into multiple cells; A text recognition method is used to recognize data in the text area and each cell, respectively, to obtain the data in the text area and the data in the table area.

7. The unstructured document recognition method according to claim 6, characterized in that, A recognition method is used to identify data in the table area to obtain the data in the table area, including: Distinguish between the header area and the data area of ​​the table region; Convert the header area into XML format and create an XML data table; Each row of the data area is stored as a record in the XML data table; The XML data table is converted into the same format as the data in the text area to obtain the data in the table area.

8. An unstructured document recognition system, characterized in that, include: The acquisition module is used to acquire PDF documents of the document to be recognized; The segmentation module is used to segment the PDF document to form a row dataset. The segmentation module divides the row dataset into regions, resulting in text regions and table regions. The recognition module is used to perform data recognition on the text area and the table area respectively using a recognition method, and obtain the data in the text area and the data in the table area respectively; The row dataset is divided into regions to obtain text regions and table regions, including: Using a projection method, each row in the row dataset is projected horizontally to obtain the row attribute of each row as a text line or a table line. Based on the row attributes of all horizontal lines as text lines or table lines, the row dataset is divided into regions to form text regions and table regions. The method employs a projection approach to perform horizontal projection on each row in the row dataset, obtaining the row attribute of each row as either a text line or a table line, including: The projection method is used to project each row in the row dataset horizontally to obtain the projection line of each row. If the projection lines of each row are continuous, then it is a row attribute of the table row; If the projection lines of each row are not continuous, then it is a line attribute of the text row.

Images