A document parsing method and system for intelligent text extraction and structured processing

Abstract

The present invention belongs to the field of document parsing technology, and relates to a document parsing method and system for intelligent text extraction and structural processing, the method comprising: 1) judging, converting and preprocessing input text; 2) performing angle correction on the preprocessed text in picture format; 3) performing layout analysis; 4) performing intelligent extraction of single-line text, and the intelligent extraction result of the single-line text includes line text recognition content and line text box; 5) comparing the line text box with the layout analysis box to obtain updated line text box and layout analysis box; 6) using recursive cutting and index tracking methods to perform layout sorting on the updated layout analysis box; 7) classifying the updated line text box according to the sorted layout analysis box, and performing structured output of the line text recognition content according to the layout category to obtain the final document parsing result. It can provide higher accuracy and efficiency when processing complex documents.

Description

Technical Field

[0001] The present invention belongs to the technical field of document parsing, and relates to a document parsing method and system, and in particular to a document parsing method and system for intelligent text extraction and structured processing. Background Art

[0002] With the rapid development of information technology, a large amount of text data is stored electronically. The effective extraction and utilization of this data has become increasingly important, especially in the fields of knowledge management and information retrieval. Among them, optical character recognition (OCR) technology has received significant attention due to its wide application in document digitization and automatic text analysis. However, in the face of document structures with charts, formulas and complex layouts, the existing OCR technology has not yet reached the ideal standards in terms of accuracy, robustness and processing speed.

[0003] At present, the challenges of extracting and utilizing text information mainly focus on the following points:

[0004] 1. When processing non-text elements such as tables, existing technologies may destroy the original layout and structure of the document, which brings additional difficulties to subsequent analysis and processing.

[0005] 2. Traditional OCR technology generally outputs text by line or by single character, which cannot effectively perform accurate extraction based on contextual semantics.

[0006] 3. For documents with complex layouts, existing technologies cannot effectively extract text content according to the way people read, but simply follow the order from top to bottom and from left to right.

[0007] 4. Existing technologies often find it difficult to effectively screen and remove useless information and scattered text, thus failing to ensure the high quality and accuracy of text extraction results.

[0008] 5. Existing optical character recognition (OCR) technology often does not work well when processing mathematical formulas, especially when the mathematical formulas are embedded in text, the recognition process often results in garbled results.

[0009] Therefore, in view of the defects existing in the above-mentioned prior art, it is necessary to develop a new document parsing method and system. Summary of the invention

[0010] In order to overcome the defects of the prior art, the present invention proposes a document parsing method and system for intelligent text extraction and structured processing, which can provide higher accuracy and efficiency when processing complex documents.

[0011] In order to achieve the above object, the present invention provides the following technical solutions:

[0012] A document parsing method for intelligent text extraction and structural processing, characterized by comprising the following steps:

[0013] 1) Determine the format of the input text. If the format of the input text is not a picture format, convert the input text into a picture format text and pre-process the picture format text;

[0014] 2) Performing text detection on the preprocessed text in the image format to obtain an OCR detection frame, determining whether to perform angle correction on the preprocessed text in the image format through the aspect ratio of the OCR detection frame and the text direction classifier, and performing angle correction when necessary to obtain the corrected text in the image format and the OCR detection frame;

[0015] 3) Performing layout analysis on the text in the corrected image format to obtain a layout analysis frame and its layout category;

[0016] 4) performing intelligent single-line text extraction on the corrected text in the picture format to obtain an intelligent single-line text extraction result, wherein the intelligent single-line text extraction result includes a line text recognition content and a line text box;

[0017] 5) Compare the line text box with the layout analysis box to improve the layout area and remove useless semantic information, thereby obtaining an updated line text box and layout analysis box;

[0018] 6) Using recursive cutting and index tracking methods to sort the updated layout analysis frame;

[0019] 7) The updated line text boxes are classified according to the sorted layout analysis boxes, and the line text recognition content is structured output according to the layout category to obtain the final document parsing result.

[0020] Preferably, the step 5) specifically includes:

[0021] 5.1) Compare the coordinates of the line text box and the layout analysis box and expand the layout analysis box so that it can completely cover the line text box contained in it. If there are line text boxes that are not contained in any layout analysis box, add a scattered layout analysis box to include all the line text boxes that are not contained in any layout analysis box in the scattered layout analysis box;

[0022] 5.2) Delete the row text boxes that do not constitute complete semantics;

[0023] 5.3) De-duplicate the layout analysis boxes and remove the layout analysis boxes with useless semantics, so as to obtain updated line text boxes and layout analysis boxes.

[0024] Preferably, the step 6) specifically includes:

[0025] 6.1) Constructing an index list to record the index of the updated layout analysis box in the input text, so as to track the initial position of the updated layout analysis box;

[0026] 6.2) Sort the updated layout analysis frames according to their positions in the X-axis direction, project them on the X-axis, and cut the input text along the X-axis direction based on the result of the X-axis projection to generate cut layout blocks;

[0027] 6.3) For each updated layout analysis frame in each layout block cut in the X-axis direction, sort them according to their positions in the Y-axis direction, and project them on the Y-axis. Based on the result of the Y-axis projection, each layout block cut in the X-axis direction is cut along the Y-axis direction to generate a cut layout block;

[0028] 6.4) If the layout blocks cut in the Y-axis direction can still be further cut, recursively execute steps 6.2) to 6.4) for these layout blocks until all layout blocks can no longer be cut;

[0029] 6.5) The layout blocks that can no longer be cut and their indexes in the index list are stored in the result list;

[0030] 6.6) Based on the index, trace back the original position of each layout block that can no longer be cut, and restore the original input text order of the cut layout blocks to obtain a sorted layout analysis frame.

[0031] Preferably, the step 7) specifically includes:

[0032] 7.1) Traverse the updated row text boxes and classify each updated row text box into the corresponding sorted layout analysis box;

[0033] 7.2) For the line text boxes in the layout analysis box classified as table, the table reconstruction algorithm is applied to reconstruct the table; for the line text boxes in the layout analysis box classified as non-table, the paragraph merging algorithm is applied to obtain the output text of the structured paragraph;

[0034] 7.3) Formatting the text recognition content in each layout analysis box;

[0035] 7.4) Output the structured text in each layout analysis box in the order of the layout analysis boxes to output the final document parsing result.

[0036] Preferably, the application of the table reconstruction algorithm to perform table reconstruction in step 7.2) is specifically as follows:

[0037] For a layout analysis frame whose layout category is table, create a virtual canvas, i.e., a two-dimensional matrix, each element of which corresponds to a character in the original table;

[0038] Calculate the canvas scaling factor according to the width of the row text box and the number of text characters in the layout analysis box whose layout type is table;

[0039] Sort the row text boxes in the layout analysis box whose layout category is table according to the Y-axis coordinates, determine the intersection of the sorted row text boxes on the Y-axis, and if the intersection between the row text boxes on the Y-axis and the ratio of the height of any row text box reach a preset threshold, it is considered that these row text boxes belong to the same row, and the row text boxes classified as the same row are sorted according to their positions on the X-axis;

[0040] On the virtual canvas, according to the sorting results of the positions on the X-axis and the Y-axis, the contents of the row text boxes classified as the same row are sequentially filled into the corresponding positions of the virtual canvas. If it is found during the filling process that the contents of the row text boxes classified as the same row exceed the size of the virtual canvas, the width of the virtual canvas is automatically adjusted;

[0041] Add spaces to the borders of each line text box that are classified as the same line to ensure that there is appropriate space between adjacent characters, avoid adhesion between characters, and form text output in a tabular form.

[0042] Preferably, the step 7.2) of applying the paragraph merging algorithm to obtain the output text of the structured paragraph specifically includes:

[0043] For a layout analysis frame whose layout category is non-tabular, the number of text characters in each row text frame in the layout analysis frame whose layout category is non-tabular is calculated, and the average text width of the row text frame is calculated according to the number of text characters;

[0044] The minimum and maximum values ​​of the X coordinates of the row text boxes in the layout analysis box with the layout category of non-tabular are used as the horizontal range of the layout analysis box with the layout category of non-tabular, and the number of text characters that each row text box should have is calculated by the horizontal range and the average text width;

[0045] The difference between the number of text characters that should be in each line text box and the actual number of text characters is used as the number of spaces to be filled. The number of spaces at the beginning and end is used to determine the start and end areas of the paragraphs in the layout analysis box whose layout category is non-tabular. The text in the line text box is wrapped or concatenated to form the output text of a structured paragraph.

[0046] In addition, the present invention also provides a document parsing system for intelligent text extraction and structural processing, which is characterized by comprising:

[0047] An input format determination and image preprocessing module, which is used to determine the format of the input text. If the format of the input text is not an image format, the input text is converted into an image format text and the image format text is preprocessed;

[0048] An automatic image correction module is used to perform text detection on the preprocessed image format text to obtain an OCR detection frame, determine whether to perform angle correction on the preprocessed image format text through the aspect ratio and text direction classifier of the OCR detection frame, and perform angle correction when angle correction is required to obtain the corrected image format text and OCR detection frame;

[0049] A layout analysis module, which is used to perform layout analysis on the text in the corrected image format to obtain a layout analysis frame and its layout category;

[0050] A single-line text intelligent extraction module, which is used to perform single-line text intelligent extraction on the text in the corrected image format to obtain an intelligent extraction result of the single-line text, wherein the intelligent extraction result of the single-line text includes a line text recognition content and a line text box;

[0051] A merging and comparing module is used to compare the line text box with the layout analysis box to improve the layout area and remove useless semantic information, thereby obtaining an updated line text box and layout analysis box;

[0052] A layout sorting module, which is used to sort the updated layout analysis frame using a recursive cutting and index tracking method;

[0053] The layout classification module is used to classify the updated line text boxes according to the sorted layout analysis boxes, and to perform structured output of the line text recognition content according to the layout categories to obtain the final document parsing results.

[0054] Compared with the prior art, the document parsing method and system for intelligent text extraction and structured processing of the present invention has one or more of the following beneficial technical effects:

[0055] 1. The present invention provides a full-process document or image content information parsing solution that combines multi-domain algorithms and models to achieve efficient and accurate parsing of complex documents.

[0056] 2. The present invention proposes a highly adaptable table reconstruction method, which uses character mapping to create a dynamically adjustable canvas, and fills the results of OCR recognition into the canvas according to their relative positions on the layout. It can recognize and extract the table structure in the document and reconstruct the table structure, no matter how complex its form or layout is.

[0057] 3. The present invention proposes a paragraph merging algorithm, which realizes the intelligent merging of text fragments after OCR processing by calculating the average width of characters and performing a paragraph reconstruction step of adaptive space insertion, thereby enhancing the semantic coherence of the text. At the same time, the text readability is improved through adaptive space insertion and line break judgment.

[0058] 4. The present invention proposes a layout sorting algorithm, which can realize layout sorting. Its characteristics are recursive cutting and index tracking, and sorting according to the original layout and design of the elements. Especially when processing documents such as academic papers and books, it fully considers the characteristic that the vertical layout has a higher priority than the horizontal layout, and efficiently reconstructs the layout structure of the original document, so as to obtain an accurate reading order. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of the document parsing method for intelligent text extraction and structured processing of the present invention.

[0060] Figure 2 It is a flow chart of the automatic picture correction method of the present invention.

[0061] Figure 3 It is a flow chart of the intelligent extraction method of a single-line text containing mathematical formulas of the present invention.

[0062] Figure 4 It is a flow chart of the embedded formula text splitting of the present invention.

[0063] Figure 5 It is a flow chart of the process of combining the formula recognition result and the text recognition result of the present invention.

[0064] Figure 6 It is a flow chart of the layout sorting method of the present invention.

[0065] Figure 7 It is a flow chart of the layout classification method of the present invention.

[0066] Figure 8 is a flow chart of the table reconstruction algorithm of the present invention.

[0067] Fig. 9 It is a flow chart of the paragraph merging algorithm of the present invention.

[0068] Fig.10 It is a schematic diagram of the structure of the document parsing system for intelligent text extraction and structured processing of the present invention. DETAILED DESCRIPTION

[0069] Before describing in detail any embodiment of the present invention, it should be understood that the present invention is not limited in its application to the construction and arrangement details of the components described below or illustrated in the following figures. The present invention can have other embodiments and can be practiced or carried out in various ways. In addition, it should be understood that the words and terms used here are for descriptive purposes and should not be considered restrictive. The use of "including" or "having" and its variations herein is intended to cover the items and their equivalents and additional items displayed below. Unless otherwise specified or limited, the terms "install", "connect", "support" and "couple" and their variations are widely used and cover direct installation and indirect installation, connection, support and connection. In addition, "connect" and "couple" are not limited to physical or mechanical connections or connections.

[0070] Furthermore, on the first aspect, in the disclosure of the present invention, the orientation or positional relationship indicated by terms such as "longitudinal", "transverse", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the orientation or positional relationship shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore the above terms cannot be understood as limitations on the present invention; on the second aspect, the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" cannot be understood as a limitation on the quantity.

[0071] In view of the defects existing in document parsing in the prior art, the present invention provides a document parsing method and system for intelligent text extraction and structured processing, which is a full-process technical solution for sequential extraction of text and formula content in books, PDF papers, and pictures, and can accurately identify and parse complex and diverse mathematical formulas in books, especially mathematical formulas embedded in texts, and effectively avoid the problem of garbled text recognition; when processing non-text elements such as tables, the original layout and structural output of the document can be maintained as much as possible, which is convenient for subsequent processing and analysis; moreover, the text is output in layout order and paragraph format, which improves the accuracy of the text semantic content; and it can effectively filter out useless information and scattered text, ensuring the high quality of the extracted text.

[0072] Figure 1 The flowchart of the document parsing method for intelligent text extraction and structural processing of the present invention is shown. Figure 1 As shown, the document parsing method for intelligent text extraction and structural processing of the present invention includes the following steps:

[0073] 1. Input format judgment and image preprocessing.

[0074] For the input text to be recognized, the format of the input text needs to be determined. If the format of the input text is not a picture format, the input text is converted into a picture format text and the picture format text is preprocessed.

[0075] Specifically, first, it is determined whether the input text is in a picture format. If the input text is not in a picture format, for example, in a PDF format, a PDF parser is used to convert the input text into a picture format.

[0076] Next, the text in the image format is dynamically extended to the background size, and the median filter algorithm is used to denoise the text in the image format. Dynamically extending the background size can meet the optimal input resolution requirements of multiple subsequent detection and recognition models. Through denoising, the image background can be processed to improve the image quality. The specific technologies of dynamically extending the background size and denoising are all existing technologies. For the sake of simplicity, they are not described in detail here.

[0077] 2. Automatic picture correction.

[0078] Considering that pages in some books or images may be rotated 90 degrees, automatic image correction is required.

[0079] When automatically correcting an image, text detection is performed on the text in the preprocessed image format to obtain an OCR detection frame. The aspect ratio of the OCR detection frame and the text direction classifier are used to determine whether to perform angle correction on the text in the preprocessed image format. When angle correction is required, angle correction is performed to obtain the corrected text in the image format and the OCR detection frame.

[0080] Specifically, Figure 2 As shown, the automatic correction of the picture specifically includes:

[0081] 1. Input the preprocessed text in the image format into the OCR detection model for text detection to obtain the OCR detection box.

[0082] 2. Calculate the aspect ratio of the OCR detection frame. If the proportion of OCR detection frames whose height is greater than their width exceeds the set threshold, the preprocessed image format text is rotated 90 degrees clockwise to obtain the updated image format text. Otherwise, the preprocessed image format text and OCR detection frame are directly output.

[0083] The threshold in this step can be set to 0.8.

[0084] 3. Input the updated text in the image format into the OCR detection model for text detection to obtain an updated OCR detection box.

[0085] 4. According to the updated OCR detection frame, the updated text in the picture format is cropped to obtain the OCR detection frame area and the OCR detection frame area is sent to the text direction classifier for direction determination. The output of the text direction classifier is 0 degrees or 180 degrees. If the proportion of the OCR detection frame area outputted as 180 degrees in the total OCR detection frame area is greater than the set threshold, the text in the updated picture format is rotated 180 degrees clockwise to obtain the text in the picture format that is updated again. Otherwise, the updated text in the picture format and the updated OCR detection frame are directly output.

[0086] The text direction classifier can be a text direction classifier of PaddleOCR. And, when cropping in the updated text in the image format according to the updated OCR detection frame, the text in the updated image format can be cropped according to the coordinates of the updated OCR detection frame to obtain multiple OCR detection frame areas and batch-feed the multiple OCR detection frame areas to the text direction classifier for direction determination. Moreover, the threshold in this step can also be set to 0.8.

[0087] 5. Input the text in the updated image format into the OCR detection model for text detection to obtain the updated OCR detection frame, and output the text in the updated image format and the updated OCR detection frame.

[0088] Among them, the OCR detection model can adopt the PP-OCRv4 detection model of PaddleOCR.

[0089] Through the above steps, text and OCR detection frames in the image format with the correct orientation can be obtained.

[0090] 3. Perform layout analysis on the text in the corrected image format to obtain a layout analysis frame and its layout category.

[0091] The layout categories include title, text, pictures, figure annotations, tables, table annotations, formulas, headers, footers and references.

[0092] Furthermore, when performing layout analysis, the text in the corrected picture format may be sent to a layout analysis model, for example, a PP-PicoDet model for layout analysis to obtain a layout analysis frame and its layout category.

[0093] Meanwhile, preferably, after obtaining the layout analysis frame, overlap judgment can be performed based on the coordinates of the layout analysis frame. If the layout analysis frame overlaps significantly, for example, the overlap rate exceeds 30%, deduplication processing is performed.

[0094] 4. Intelligent extraction of single-line text.

[0095] Single-line text intelligent extraction is performed on the corrected text in the picture format to obtain an intelligent extraction result of the single-line text, wherein the intelligent extraction result of the single-line text includes line text recognition content and a line text box.

[0096] In the present invention, the existing method can be used to intelligently extract the single-line text, for example, the existing OCR technology can be used to intelligently extract the single-line text.

[0097] Of course, considering that a single-line text may contain a data formula, in the present invention, different intelligent extraction methods can be used according to the different single-line texts to improve the accuracy of the single-line text extraction results. For a single-line text that does not contain a mathematical formula, the existing OCR technology is used to perform intelligent extraction of the single-line text; for a single-line text that contains a mathematical formula, such as Figure 3 The intelligent extraction method comprises:

[0098] 1. Perform formula detection on the text in the corrected image format to obtain the formula detection box and its formula category.

[0099] The formula categories include embedded formulas and interline formulas. In addition, when performing formula detection, the corrected text in the image format can be fed into the formula detection model to obtain a formula detection frame and its formula category.

[0100] In the present invention, different processing methods are used for embedded formulas and in-row formulas. For embedded formulas, the following processing steps 2 to 5 are performed, and for in-row formulas, the following processing steps 6 to 9 are performed.

[0101] 2. Embedded formula text splitting.

[0102] For OCR detection boxes containing embedded formulas, the formula and text are split to obtain formula detection results, thereby improving recognition accuracy.

[0103] Among them, it can be determined whether the OCR detection frame with the correct direction contains the embedded formula according to the formula detection frame whose formula category is the embedded formula, and the OCR detection frame containing the embedded formula is split to obtain the OCR detection frame of the plain text.

[0104] In the present invention, the embedded formula text splitting includes embedded formula and text matching and embedded formula and text cutting. Through the embedded formula and text matching, it can be judged whether the OCR detection box contains the embedded formula. Through the embedded formula and text cutting, the OCR detection box containing the embedded formula can be split to cut it into a formula detection box and a plain text OCR detection box.

[0105] The embedded formula and text matching is as follows: for each OCR detection box, find the embedded formula in it. The matching conditions are: (1) the center point coordinates of the formula detection box whose formula category is embedded formula are within the range of the OCR detection box; (2) the formula detection box whose formula category is embedded formula overlaps with the OCR detection box.

[0106] The specific algorithm for matching condition (1) is:

[0107] 1) Calculate the coordinates of the OCR detection frame and generate the upper and lower boundary lines and left and right boundary line equations of the OCR detection frame;

[0108] 2) Calculate the X coordinate of the center point of the formula detection box whose formula category is embedded formula and substitute it into the upper and lower boundary line equations of the OCR detection box to determine whether the Y coordinate of the center point of the formula detection box whose formula category is embedded formula is within the Y value range of the upper and lower boundary lines;

[0109] 3) Calculate the Y coordinate of the center point of the formula detection box whose formula category is embedded formula and substitute it into the left and right boundary line equations of the OCR detection box to determine whether the X coordinate of the center point of the formula detection box whose formula category is embedded formula is within the X value range of the left and right boundary lines.

[0110] The specific algorithm for matching condition (2) is:

[0111] 1) determining whether the formula detection box whose formula category is embedded formula and the OCR detection box have an intersection based on the coordinates of the formula detection box whose formula category is embedded formula and the coordinates of the OCR detection box;

[0112] 2) When matching condition (1) is met, the IOU (Intersection over Union, which indicates the ratio of the area of ​​the overlapping part of the two to the total area) of the formula detection box with the formula category of embedded formula and the OCR detection box is calculated. If the formula detection box with the formula category of embedded formula is within the range of the OCR detection box, and there is a certain overlap between the two, the formula detection box with the formula category of embedded formula is added to the corresponding formula list, and the OCR detection box is determined to be the OCR detection box containing the embedded formula.

[0113] The specific steps of embedding formula and text cutting are: cutting the OCR detection frame according to the coordinates of the formula detection frame whose formula category is embedded formula, adding the segmented formula detection frame to the corresponding list, and returning the result list. The specific algorithm for segmenting the OCR detection frame is:

[0114] 1) If the OCR detection box does not have an embedded formula, directly return to the original OCR detection box.

[0115] 2) If there is an embedded formula in the OCR detection frame, sort the formula detection frames whose formula category is embedded formula in the list obtained above according to the X-axis, process each formula detection frame whose formula category is embedded formula in turn, and segment it according to the position of its left boundary and the position of its right boundary.

[0116] 3) Make sure the OCR detection box is a rectangle after segmentation. If it is not a rectangle, segment out the extra parts. Finally, return a list of all segmented OCR detection boxes.

[0117] Based on the above principles, Figure 4 As shown, the embedded formula text splitting of the present invention specifically includes:

[0118] 1) For each OCR detection box, the corresponding first-order equations of the upper, lower, left and right boundary lines are obtained according to the coordinates of its upper left corner, lower left corner, upper right corner and lower right corner.

[0119] 2) selecting a formula detection frame whose formula category is an embedded formula from the formula detection frame, calculating the X-axis coordinate of the center point of the formula detection frame whose formula category is an embedded formula and substituting it into a linear equation of the upper and lower boundary lines of the OCR detection frame to determine whether the Y-axis coordinate of the center point of the formula detection frame whose formula category is an embedded formula is within the Y value interval of the upper and lower boundary lines of the OCR detection frame, calculating the Y-axis coordinate of the center point of the formula detection frame whose formula category is an embedded formula and substituting it into a linear equation of the left and right boundary lines of the OCR detection frame to determine whether the X-axis coordinate of the center point of the formula detection frame whose formula category is an embedded formula is within the X value interval of the left and right boundary lines of the OCR detection frame;

[0120] 3) If the Y-axis coordinate of the center point of the formula detection box with the formula category of embedded formula is within the Y value interval of the upper and lower boundary lines of the OCR detection box, and the X-axis coordinate of the center point of the formula detection box with the formula category of embedded formula is within the X value interval of the left and right boundary lines of the OCR detection box, then calculate the ratio of the overlapping area of ​​the OCR detection box and the formula detection box with the formula category of embedded formula to the total area of ​​the two, that is, IoU;

[0121] 4) Determine whether the IoU is greater than 0. If the IoU is greater than 0, add the formula detection box whose formula category is an embedded formula to the formula list, determine that the OCR detection box is an OCR detection box containing a mathematical formula, and split the OCR detection box.

[0122] If it is determined that the OCR detection box is an OCR detection box that does not contain an embedded formula, the OCR detection box is directly returned. If it is determined that the OCR detection box is an OCR detection box that contains an embedded formula, it is determined whether the OCR detection box has one or more embedded formulas. If there are multiple embedded formulas, the formula detection boxes whose formula categories are embedded formulas are sorted on the X-axis, and the OCR detection boxes are cut according to the left and right boundary positions of the formula detection boxes whose formula categories are embedded formulas.

[0123] Furthermore, if there is a non-rectangular OCR detection frame, during the cutting process, the redundant non-rectangular part of the OCR detection frame is cut and discarded, and finally the multiple split OCR detection frames are returned.

[0124] 3. Formula recognition.

[0125] The formula detection frame is cropped in the text in the corrected picture format to obtain a formula area, a formula recognition model is used to perform formula recognition on the formula area and the output LaTeX is format simplified to obtain a formula recognition result.

[0126] Specifically, first, the coordinates of the formula detection box are cropped in the text of the rectified picture format to obtain the formula area. In order to avoid the occurrence of garbled characters in the small picture formula recognition, the formula area is dynamically extended and background processed.

[0127] The processed formula area is sent to the formula recognition model, and the output result is in LaTeX format, that is, LaTeX. The formula recognition model can adopt the existing handwritten formula recognition model.

[0128] Then, the output LaTeX is sent to the rendering engine for rendering judgment. If the rendering fails (rendering failure is mainly due to the omission of some paired tokens), the format is repaired. If the rendering still fails after the format repair, additional error information is output to prevent the output of incorrect formulas. The rendering engine can use the KaTex rendering engine.

[0129] Finally, simplify the LaTeX and remove unnecessary formats to provide a clearer and more concise formula expression. The final output includes the formula detection box, formula category (embedded formula), and the LaTeX of the formula.

[0130] 4. Text recognition.

[0131] According to different scenarios, the corresponding OCR recognition model is used to perform text recognition on the OCR detection box of the plain text to obtain the text recognition result.

[0132] Specifically, firstly, the OCR detection frame of the split plain text is cropped in the text in the rectified image format to obtain the OCR detection frame area.

[0133] Then, the OCR detection box area is sent to the Chinese OCR recognition model for preliminary text recognition to obtain a preliminary text recognition result. The Chinese OCR recognition model can adopt the PP-OCRv4 Chinese recognition model of PaddleOCR.

[0134] Among them, in order to solve the problem of English and numerical space adhesion and improve the recognition accuracy of English and numerical content, in the present invention, if the preliminary text recognition result only contains English or numbers, the OCR detection box area is sent to the English OCR recognition model for recognition to obtain the final text recognition result and cover the preliminary text recognition result. Among them, the English OCR recognition model can adopt the PP-OCRv4 English recognition model of PaddleOCR. Therefore, by using different text recognition models in a targeted manner, the recognition is accurate and can efficiently process texts mixed in Chinese and English.

[0135] In the present invention, the final output result of text recognition includes an OCR detection frame and a final text recognition result.

[0136] 5. Merge processing.

[0137] Combined with the layout analysis frame and its layout category, the formula recognition result of step 3 and the text recognition result of step 4 are judged and merged in the same line detection frame to obtain the intelligent extraction result of the single line text containing the embedded formula.

[0138] In the present invention, after the formula recognition result is added to the text recognition result, the merging process continuously evaluates whether the formula recognition result and the text recognition result are located in the same row, and the formula recognition result and the text recognition result confirmed to be in the same row are merged.

[0139] The specific steps of the merge row algorithm are:

[0140] 1) Determine whether the formula detection box and the OCR detection box whose formula category is embedded formula are in the same row:

[0141] (1) Initial filtering: When the Y-axis intersection between the formula detection frame and the OCR detection frame of the formula category of embedded formula is greater than the threshold value of 0.5, and the X-axis intersection is less than the threshold value of 0.5 (screening out the formula detection frame and OCR detection frame that exist in different rows but have overlapping axes), the formula detection frame and OCR detection frame of the formula category of embedded formula are initially filtered out;

[0142] (2) Further confirmation, when the initial filtering is satisfied, the layout analysis frame and its layout category are combined to determine whether the formula detection frame and the OCR detection frame whose formula category is embedded formula are in the same layout;

[0143] (3) After the above two conditions are met, the X_cut algorithm is used to determine whether the formula detection box and the OCR detection box are segmented on the X-axis. If they cannot be segmented, they are determined to be in the same row; otherwise, they are not in the same row.

[0144] In the present invention, the X_cut algorithm is specifically:

[0145] (1) Detection box sorting: Sort the formula detection boxes and OCR detection boxes according to their X-axis coordinates.

[0146] (2) X-axis projection: Perform X-axis projection on the sorted formula detection boxes and OCR detection boxes whose formula categories are embedded formulas, and generate a projection array that reflects the distribution of formula detection boxes and OCR detection boxes whose formula categories are embedded formulas;

[0147] (3) Segmentation point confirmation: find the point with a low projection height (e.g., 0) according to the projection array and regard it as a potential segmentation point;

[0148] (4) Segmentation decision: If the potential segmentation point is 1, it means that the formula detection box and OCR detection box of the formula category "embedded formula" cannot be segmented on the X-axis. If there are multiple segmentation points, the formula detection box and OCR detection box of the formula category "embedded formula" are further segmented according to each segmentation point.

[0149] 2) Internal sorting within a row: internal sorting is performed on formula detection boxes and OCR detection boxes whose formula categories are confirmed to be embedded formulas in the same row.

[0150] The specific in-row sorting algorithm is as follows: a starting box is specified in the row, and its maximum allowed intersection range on the X-axis is calculated to filter and sort the adjacent left and right formula detection boxes and OCR detection boxes whose formula categories are embedded formulas, and add them to the result list one by one, and use them as new starting boxes in turn, and cycle until no new formula detection boxes and OCR detection boxes whose formula categories are embedded formulas that meet the conditions can be found.

[0151] 3) Merge detection frames: merge the formula detection frames and OCR detection frames whose formula categories are embedded formulas and are in the same row and have been correctly sorted. The merging methods include: merging text contents in order, merging into one detection frame, etc.

[0152] Based on the above principles, Figure 5 As shown, the merging process of the formula recognition result and the text recognition result of the present invention specifically includes:

[0153] 1) Add the formula recognition results to the text recognition results.

[0154] 2) Preliminarily determine whether the formula detection box and the OCR detection box of the formula category of embedded formula are in the same row, so as to preliminarily screen out the detection boxes belonging to the same row. When the Y-axis intersection of the formula detection box and the OCR detection box of the formula category of embedded formula is greater than the threshold value 0.5 and the X-axis intersection is less than the threshold value 0.5, it is preliminarily determined that the formula detection box and the OCR detection box of the formula category of embedded formula are in the same row.

[0155] 3) Classify the preliminarily screened detection frames in combination with the layout analysis frame and its layout category, and determine whether the preliminarily screened detection frames belong to the same layout category, so as to screen out the detection frames belonging to the same layout.

[0156] 4) Project the selected detection frames belonging to the same layout on the X-axis to determine the segmentation points, and judge whether the detection frames can be segmented on the X-axis according to the number of segmentation points. If they cannot be segmented, it is determined that the selected detection frames belonging to the same layout belong to the same line detection frames; if they can be segmented, it is determined that the selected detection frames belonging to the same layout do not belong to the same line detection frames.

[0157] 5) The determined detection boxes in the same row are sorted within the row and then merged, and the intelligent extraction result of the single-row text containing the embedded formula is output, which includes the row text recognition content and the row text box.

[0158] 6. For inter-line formulas, determine whether the formula detection frame overlaps the corrected OCR detection frame to a large extent.

[0159] In the present invention, it is possible to determine whether the coordinates of the formula detection frame of the formula category being the interline formula and the coordinates of the corrected OCR detection frame are substantially overlapped, for example, whether the overlap rate is greater than 90%. If there is a substantial overlap, it means that the corrected OCR detection frame is an OCR detection frame of a pure interline formula; if there is a substantial overlap, it means that the corrected OCR detection frame is not an OCR detection frame of a pure interline formula, but an ordinary text OCR detection frame, and text recognition can be performed on it using a text recognition model.

[0160] 7. Identify formulas between lines.

[0161] In this step, the same method as in step 3 is used to perform formula recognition on the formula detection box whose formula category is an inter-row formula to obtain a formula recognition result.

[0162] 8. Text recognition.

[0163] In this step, the same method as in step 4 is used to perform text recognition on the corrected OCR detection frame that substantially overlaps with the formula detection frame whose formula category is an interline formula, so as to obtain a text recognition result.

[0164] 9. Replacement.

[0165] The text recognition result obtained in step 8 is replaced with the formula recognition result obtained in step 7 to obtain an intelligent extraction result of a single line of text, which is a text extraction result of the inter-line formula, including line text recognition content and a line text box.

[0166] 5. Merge and compare.

[0167] The line text box is compared with the layout analysis box to improve the layout area and remove useless semantic information, thereby obtaining an updated line text box and layout analysis box.

[0168] Specifically, the combined comparison includes:

[0169] 1. Compare the coordinates of the line text box and the layout analysis box and expand the layout analysis box so that it can completely cover the line text box contained in it. If there are line text boxes not included in any layout analysis box, add a scattered layout analysis box and include all the line text boxes not included in any layout analysis box in the scattered layout analysis box.

[0170] 2. Delete the line text boxes that do not constitute complete semantics to prevent subsequent text from being classified into different layouts multiple times and remove duplicate boxes in the layout.

[0171] 3. De-duplicate the layout analysis frames and remove the layout analysis frames with useless semantics, that is, remove the layout contents such as headers, footers, and pictures, so as to obtain the updated merged line text detection frame and layout analysis frame.

[0172] 6. Page layout sorting.

[0173] In order to ensure that the identified layouts of different layout categories can be output in the correct order and format, the layouts need to be sorted. In the present invention, the updated layout analysis frame is sorted using a recursive cutting and index tracking method.

[0174] In the present invention, the recursive cutting and index tracking method is the YX_cut algorithm, which is similar to the aforementioned X_cut algorithm. Specifically, Figure 6 As shown, using the recursive cutting and index tracking method to sort the updated layout analysis frame specifically includes:

[0175] 1. Build an index list to record the index of the updated layout analysis box in the input text, so as to track the initial position of the updated layout analysis box during processing and sorting.

[0176] 2. Sort each updated layout analysis frame according to its position in the X-axis direction, project it on the X-axis, cut the input text along the X-axis direction based on the result of the X-axis projection, and generate the cut layout block. In this way, the cutting of the layout analysis frame in the X-axis direction is realized.

[0177] 3. For each updated layout analysis frame in each layout block cut in the X-axis direction, sort them according to their positions in the Y-axis direction and project them on the Y-axis. Based on the result of the Y-axis projection, each layout block cut in the X-axis direction is cut along the Y-axis direction to generate a cut layout block. In this way, the cutting of the layout analysis frame in the Y-axis direction is realized.

[0178] 4. If the layout blocks cut in the Y-axis direction can still be further cut, recursively perform steps 2 to 4 on these layout blocks until all layout blocks can no longer be cut. At this point, a layout block that can no longer be cut corresponds to a layout analysis frame.

[0179] 5. The layout blocks that can no longer be cut and their indexes in the index list are stored in the result list.

[0180] 6. Based on the index, trace back the original position of each layout block that can no longer be cut, and restore the original input text order of the cut layout blocks to obtain a sorted layout analysis frame.

[0181] 7. Page layout classification.

[0182] The updated line text boxes are classified according to the sorted layout analysis boxes, and the structured output of the line text recognition content is performed according to the layout category to obtain the final document parsing result.

[0183] In the present invention, if Figure 7 As shown, the layout classification specifically includes:

[0184] 1. Traverse the updated row text boxes and classify each updated row text box into the corresponding sorted layout analysis box.

[0185] 2. For the row text boxes in the layout analysis box classified as table, the table reconstruction algorithm is applied to rebuild the table. For the row text boxes in the layout analysis box classified as non-table, the paragraph merging algorithm is applied to obtain the output text of the structured paragraph.

[0186] 3. Format the text recognition content of each layout analysis box. For example, remove spaces at the beginning and end of a paragraph, add spaces after English numbers, etc.

[0187] 4. Output the structured text in each layout analysis box in the order of the layout analysis boxes to output the final document analysis results

[0188] In order to solve the problem that the original layout and structure of the document may be destroyed when processing tables in the existing document parsing process, the present invention proposes a highly adaptable table reconstruction method. The core idea of ​​the method is to use character mapping to create a dynamically adjustable virtual canvas (Canvas). By using character mapping to create a virtual canvas (Canvas), the contents of the row text box are filled into the virtual canvas according to their relative positions on the layout and the size of the virtual canvas is dynamically adjusted. It can identify and extract the table structure in the document and reconstruct the table structure, no matter how complex its form or layout is.

[0189] Specifically, Figure 8 As shown, the application of the table reconstruction algorithm to perform table reconstruction is specifically as follows:

[0190] (1) For a layout analysis frame whose layout category is table, create a virtual canvas, i.e., a two-dimensional matrix, each element of which corresponds to a character in the original table.

[0191] (2) Calculate a canvas scaling factor based on the width of the row text box and the number of text characters in the layout analysis box whose layout type is a table. The scaling factor is used to control the space occupied by each text character on the virtual canvas.

[0192] (3) Sort the row text boxes in the layout analysis box whose layout category is table according to the Y-axis coordinates, and determine the intersection of the sorted row text boxes on the Y-axis. If the intersection between the row text boxes on the Y-axis and the ratio of the height of any row text box reach a preset threshold, then these row text boxes are considered to belong to the same row, and the row text boxes classified as the same row are sorted according to their positions on the X-axis.

[0193] The preset threshold may be 0.5.

[0194] Through this step, the row text boxes in the layout analysis box with the layout category of table can be detected and merged in the same row.

[0195] (4) On the virtual canvas, according to the sorting results of the positions on the X-axis and the Y-axis, the contents of the row text boxes classified as the same row are sequentially filled into the corresponding positions of the virtual canvas. If it is found during the filling process that the contents of the row text boxes classified as the same row exceed the size of the virtual canvas, the width of the virtual canvas is automatically adjusted using the canvas scaling factor to ensure that all the contents can be correctly placed on the virtual canvas to avoid being cropped or overflowing.

[0196] (5) Adding spaces to the boundaries of each line text box that is classified as the same line to ensure that there is an appropriate gap between adjacent characters to avoid adhesion between characters and form a text output in a table format. When adding spaces, it is determined whether the virtual canvas boundary is reached. If so, the virtual canvas is expanded to prevent space overflow.

[0197] At the same time, unlike the line-by-line output adopted by traditional OCR, the present invention designs a structured output by paragraph and layout, that is, a paragraph merging algorithm is designed. This method retains the layout structure of the original document to the greatest extent, making the recognized content highly standardized and better readable.

[0198] Specifically, Fig. 9 As shown, the output text of the structured paragraph obtained by applying the paragraph merging algorithm specifically includes:

[0199] (1) For a layout analysis frame whose layout category is non-tabular, the number of text characters in each row text frame in the layout analysis frame whose layout category is non-tabular is calculated, and the average text width of the row text frame is calculated based on the number of text characters.

[0200] Among them, when calculating the number of text characters in each row text box in the layout analysis box whose layout category is non-table, the number of text characters is automatically adjusted according to different character categories (such as Chinese, English, numbers, punctuation marks, formulas LaTex, etc.) to dynamically approach the actual average character width. Specifically, if the row text box contains embedded formulas, the embedded formulas are not counted in the number of text characters, while English, numbers, and punctuation marks are counted as half the size of a text character, and Chinese is counted as one text character size. Therefore, the average text width of the row text box can be calculated by removing the embedded formula area from the width of the row text box and dividing it by the number of text characters.

[0201] (2) The minimum and maximum values ​​of the X coordinates of the row text boxes in the layout analysis box with the layout category of non-tabular are used as the horizontal range of the layout analysis box with the layout category of non-tabular. The number of text characters that each row text box should have is calculated based on the horizontal range and the average text width.

[0202] (3) The difference between the number of text characters that should be in each line text box and the actual number of text characters is used as the number of spaces to be filled. The start and end areas of the paragraphs in the layout analysis box whose layout category is non-tabular are determined by the number of spaces at the beginning and end. The text in the line text box is wrapped or concatenated to form the overall paragraph output in the layout.

[0203] Among them, for the line text box, check whether there is a space at the beginning or end of the recognized text. If there is, it means that the text of the line text box needs to be wrapped. Otherwise, the text of the line text box can be directly spliced ​​with the next line text box. Traverse the line text boxes and merge the text of all the line text boxes in the same layout into a whole paragraph output.

[0204] In addition, the present invention also relates to a document parsing system for intelligent text extraction and structured processing. Fig.10 As shown, the document parsing system for intelligent text extraction and structural processing of the present invention includes:

[0205] An input format determination and image preprocessing module, which is used to determine the format of the input text. If the format of the input text is not an image format, the input text is converted into an image format text and the image format text is preprocessed;

[0206] An automatic image correction module is used to perform text detection on the preprocessed image format text to obtain an OCR detection frame, determine whether to perform angle correction on the preprocessed image format text through the aspect ratio and text direction classifier of the OCR detection frame, and perform angle correction when angle correction is required to obtain the corrected image format text and OCR detection frame;

[0207] A layout analysis module, which is used to perform layout analysis on the text in the corrected image format to obtain a layout analysis frame and its layout category;

[0208] A single-line text intelligent extraction module, which is used to perform single-line text intelligent extraction on the text in the corrected image format to obtain an intelligent extraction result of the single-line text, wherein the intelligent extraction result of the single-line text includes a line text recognition content and a line text box;

[0209] A merging and comparing module is used to compare the line text box with the layout analysis box to improve the layout area and remove useless semantic information, thereby obtaining an updated line text box and layout analysis box;

[0210] A layout sorting module, which is used to sort the updated layout analysis frame using a recursive cutting and index tracking method;

[0211] The layout classification module is used to classify the updated line text boxes according to the sorted layout analysis boxes, and to perform structured output of the line text recognition content according to the layout categories to obtain the final document parsing results.

[0212] The overall solution of the present invention provides a comprehensive and efficient document parsing solution by combining cross-domain algorithms and the results of multiple models. In general, compared with existing technologies, the present invention can provide higher accuracy and efficiency when processing complex documents (such as academic papers with a large number of formulas and charts).

[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art can modify or replace the technical solution of the present invention according to the idea of ​​the present invention without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A document parsing method for intelligent text extraction and structural processing, characterized in that: The following steps are involved: 1) Determine the format of the input text. If the format of the input text is not a picture format, convert the input text into a picture format text and pre-process the picture format text; 2) Performing text detection on the preprocessed text in the image format to obtain an OCR detection frame, determining whether to perform angle correction on the preprocessed text in the image format through the aspect ratio of the OCR detection frame and the text direction classifier, and performing angle correction when necessary to obtain the corrected text in the image format and the OCR detection frame; 3) Performing layout analysis on the text in the corrected image format to obtain a layout analysis frame and its layout category; 4) performing intelligent single-line text extraction on the corrected text in the picture format to obtain an intelligent single-line text extraction result, wherein the intelligent single-line text extraction result includes a line text recognition content and a line text box; 5) Compare the line text box with the layout analysis box to improve the layout area and remove useless semantic information, thereby obtaining an updated line text box and layout analysis box; 6) Using recursive cutting and index tracking methods to sort the updated layout analysis frame; 7) Classifying the updated line text boxes according to the sorted layout analysis boxes, and outputting the structured line text recognition content according to the layout categories to obtain the final document parsing results; The step 7) specifically includes: 7.1) Traverse the updated row text boxes and classify each updated row text box into the corresponding sorted layout analysis box; 7.2) For the line text boxes in the layout analysis box classified as table, the table reconstruction algorithm is applied to reconstruct the table; for the line text boxes in the layout analysis box classified as non-table, the paragraph merging algorithm is applied to obtain the output text of the structured paragraph; 7.3) Formatting the text recognition content in each layout analysis box; 7.4) Output the structured text in each layout analysis frame in the order of the layout analysis frames to output the final document parsing result; The application of the table reconstruction algorithm in step 7.2) to perform table reconstruction is specifically as follows: For a layout analysis frame whose layout category is table, create a virtual canvas, i.e., a two-dimensional matrix, each element of which corresponds to a character in the original table; Calculate the canvas scaling factor according to the width of the row text box and the number of text characters in the layout analysis box whose layout type is table; Sort the row text boxes in the layout analysis box whose layout category is table according to the Y-axis coordinates, determine the intersection of the sorted row text boxes on the Y-axis, and if the intersection between the row text boxes on the Y-axis and the ratio of the height of any row text box reach a preset threshold, it is considered that these row text boxes belong to the same row, and the row text boxes classified as the same row are sorted according to their positions on the X-axis; On the virtual canvas, according to the sorting results of the positions on the X-axis and the Y-axis, the contents of the row text boxes classified as the same row are sequentially filled into the corresponding positions of the virtual canvas. If it is found during the filling process that the contents of the row text boxes classified as the same row exceed the size of the virtual canvas, the width of the virtual canvas is automatically adjusted; Add spaces to the borders of each line text box that are classified as the same line to ensure that there is appropriate space between adjacent characters, avoid adhesion between characters, and form text output in a tabular form.

2. The document parsing method for intelligent text extraction and structural processing according to claim 1, characterized in that: The step 5) specifically includes: 5.1) Compare the coordinates of the line text box and the layout analysis box and expand the layout analysis box so that it can completely cover the line text box contained in it. If there are line text boxes that are not contained in any layout analysis box, add a scattered layout analysis box to include all the line text boxes that are not contained in any layout analysis box in the scattered layout analysis box; 5.2) Delete the row text boxes that do not constitute complete semantics; 5.3) De-duplicate the layout analysis boxes and remove the layout analysis boxes with useless semantics, so as to obtain updated line text boxes and layout analysis boxes.

3. The document parsing method for intelligent text extraction and structural processing according to claim 2 is characterized in that: The step 6) specifically includes: 6.1) Constructing an index list to record the index of the updated layout analysis box in the input text, so as to track the initial position of the updated layout analysis box; 6.2) Sort the updated layout analysis frames according to their positions in the X-axis direction, project them on the X-axis, and cut the input text along the X-axis direction based on the result of the X-axis projection to generate cut layout blocks; 6.3) For each updated layout analysis frame in each layout block cut in the X-axis direction, sort them according to their positions in the Y-axis direction, and project them on the Y-axis. Based on the result of the Y-axis projection, each layout block cut in the X-axis direction is cut along the Y-axis direction to generate a cut layout block; 6.4) If the layout blocks cut in the Y-axis direction can still be further cut, recursively execute steps 6.2) to 6.4) for these layout blocks until all layout blocks can no longer be cut; 6.5) The layout blocks that can no longer be cut and their indexes in the index list are stored in the result list; 6.6) Based on the index, trace back the original position of each layout block that can no longer be cut, and restore the original input text order of the cut layout blocks to obtain a sorted layout analysis frame.

4. The document parsing method for intelligent text extraction and structural processing according to claim 3 is characterized in that: The step 7.2) of applying the paragraph merging algorithm to obtain the output text of the structured paragraph specifically includes: For a layout analysis frame whose layout category is non-tabular, the number of text characters in each row text frame in the layout analysis frame whose layout category is non-tabular is calculated, and the average text width of the row text frame is calculated according to the number of text characters; The minimum and maximum values ​​of the X coordinates of the row text boxes in the layout analysis box with the layout category of non-tabular are used as the horizontal range of the layout analysis box with the layout category of non-tabular, and the number of text characters that each row text box should have is calculated by the horizontal range and the average text width; The difference between the number of text characters that should be in each line text box and the actual number of text characters is used as the number of spaces to be filled. The number of spaces at the beginning and end is used to determine the start and end areas of the paragraphs in the layout analysis box whose layout category is non-tabular. The text in the line text box is wrapped or concatenated to form the output text of a structured paragraph.

5. A document parsing system for intelligent text extraction and structural processing, characterized in that: include: An input format determination and image preprocessing module, which is used to determine the format of the input text. If the format of the input text is not an image format, the input text is converted into an image format text and the image format text is preprocessed; An automatic image correction module is used to perform text detection on the preprocessed image format text to obtain an OCR detection frame, determine whether to perform angle correction on the preprocessed image format text through the aspect ratio and text direction classifier of the OCR detection frame, and perform angle correction when angle correction is required to obtain the corrected image format text and OCR detection frame; A layout analysis module, which is used to perform layout analysis on the text in the corrected image format to obtain a layout analysis frame and its layout category; A single-line text intelligent extraction module, which is used to perform single-line text intelligent extraction on the text in the corrected image format to obtain an intelligent extraction result of the single-line text, wherein the intelligent extraction result of the single-line text includes a line text recognition content and a line text box; A merging and comparing module is used to compare the line text box with the layout analysis box to improve the layout area and remove useless semantic information, thereby obtaining an updated line text box and layout analysis box; A layout sorting module, which is used to sort the updated layout analysis frame using a recursive cutting and index tracking method; The layout classification module is used to classify the updated line text boxes according to the sorted layout analysis boxes, and to output the structured line text recognition content according to the layout categories to obtain the final document parsing results, which specifically includes: 1) Traverse the updated row text boxes and classify each updated row text box into the corresponding sorted layout analysis box; 2) For the row text boxes in the layout analysis box classified as table, the table reconstruction algorithm is applied to reconstruct the table; for the row text boxes in the layout analysis box classified as non-table, the paragraph merging algorithm is applied to obtain the output text of the structured paragraph; 3) Formatting the recognized text content in each layout analysis box; 4) Output the structured text in each layout analysis frame in the order of the layout analysis frames to output the final document parsing result; The application of the table reconstruction algorithm in step 2) to perform table reconstruction is specifically as follows: For a layout analysis frame whose layout category is table, create a virtual canvas, i.e., a two-dimensional matrix, each element of which corresponds to a character in the original table; Calculate the canvas scaling factor according to the width of the row text box and the number of text characters in the layout analysis box whose layout type is table; Sort the row text boxes in the layout analysis box whose layout category is table according to the Y-axis coordinates, determine the intersection of the sorted row text boxes on the Y-axis, and if the intersection between the row text boxes on the Y-axis and the ratio of the height of any row text box reach a preset threshold, it is considered that these row text boxes belong to the same row, and the row text boxes classified as the same row are sorted according to their positions on the X-axis; On the virtual canvas, according to the sorting results of the positions on the X-axis and the Y-axis, the contents of the row text boxes classified as the same row are sequentially filled into the corresponding positions of the virtual canvas. If it is found during the filling process that the contents of the row text boxes classified as the same row exceed the size of the virtual canvas, the width of the virtual canvas is automatically adjusted; Add spaces to the borders of each line text box that are classified as the same line to ensure that there is appropriate space between adjacent characters, avoid adhesion between characters, and form text output in a tabular form.

Images