Method and system for document content recognition and intelligent paragraph and chapter decomposition

Abstract

The present invention belongs to the field of data processing technology, and specifically relates to a method and system for document content recognition and intelligent paragraph and chapter disassembly. The method comprises: S1, splitting a PDF document into pages and converting them into images; S2, performing DOM structure analysis on the split PDF pages to obtain a text character set; S3, performing layout analysis on the PDF page images to obtain a layout area set; S4, aggregating the obtained text character set and layout area set into character blocks; S5, performing a reading order recovery operation on the character blocks; S6, defining a title matching pattern set; S7, traversing the line sequence of the block sequence, and obtaining the hierarchical relationship between directories through a comparison algorithm; S8, outputting a structured content block result; S9, visually displaying the PDF page image and the content block coordinates, and the user draws a new coordinate area, and performs secondary disassembly according to the provided coordinate area information.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a method and system for document content recognition and intelligent paragraph and chapter decomposition. Background Art

[0002] In daily work and life, we usually need to extract the content of the documents we have and then use them, such as industry standards, operating manuals, papers and other documents. However, it is not enough to just extract the text content. It is often necessary to break the content into hierarchical paragraphs and then organize it into formatted content for better use.

[0003] When faced with unstructured documents such as PDF, OCR technology is usually needed to extract text information. However, traditional OCR can only obtain character fragments with coordinates. The results are disorganized and lack effective structural information. The latest OCR technology can obtain the entire content of the document end-to-end, but it cannot further decompose the elements and cannot separate titles, text paragraphs, tables, pictures and other content, making it difficult to obtain a hierarchical paragraph tree structure.

[0004] Through layout analysis technology, we can use the trained model to identify various elements of the page, and return the coordinates and tags of the identified area. The tags usually include text, titles, lists, graphics, tables, headers and footers, etc. Although it seems that a lot of useful information can be obtained, this method is closely related to the samples trained on the model. When actually identifying various complex document layouts in the real world, the generalization effect is difficult to be satisfactory, and it is difficult to expand and adapt the layout mode, which makes people helpless.

[0005] In addition, general file content extraction and structuring can only passively receive the output results of the system. If you want to adjust the recognition area, such as excluding comments and other areas, re-dividing and disassembling paragraphs, etc., there is basically a lack of adjustment methods. After all, no method can guarantee a 100% perfect result. Effective process intervention is an important way to supplement functions.

[0006] Therefore, it is very important to design a method and system for document content recognition and intelligent paragraph and chapter decomposition that is widely applicable to scenarios with different languages ​​and layout modes and can improve the accuracy of PDF document content recognition and decomposition. Summary of the invention

[0007] The present invention aims to overcome the problems in the prior art that the existing OCR technology and layout analysis technology are difficult to intelligently decompose paragraphs and chapters and have poor generalization effect in general scenarios. A method and system for document content recognition and intelligent paragraph and chapter decomposition is provided, which has wide applicability to scenarios with different languages ​​and different layout modes and can improve the accuracy of PDF document content recognition and decomposition.

[0008] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions:

[0009] The method for document content recognition and intelligent paragraph and chapter decomposition comprises the following steps:

[0010] S1, split the PDF document into pages and convert the content of each page into images;

[0011] S2, performing DOM structure analysis on the split PDF pages to obtain a text character set;

[0012] S3, performing layout analysis on the PDF page image to obtain a layout area set;

[0013] S4, aggregating the text character set obtained in step S2 and the layout area set obtained in step S3 into character blocks;

[0014] S5, performing a reading order recovery operation on the character blocks, specifically:

[0015] The character blocks are reorganized, split or merged; the character blocks are then sorted to obtain an ordered sequence of blocks; within each character block, the characters are aggregated into text lines to obtain an ordered sequence of lines;

[0016] S6, defines a set of title matching patterns for title matching and level generation of the directory tree;

[0017] S7, traverse the line sequence of the block sequence, compare and analyze the text line and the title matching pattern set, mark the matching line with a directory mark, and obtain the hierarchical relationship between the directories through a comparison algorithm;

[0018] S8, traverse the row sequence of the block sequence, merge the text in order, distinguish the image area, the table area and the text area according to the analysis result of step S3, generate three types of content blocks, divide the paragraphs according to the marking result of step S7, and output the structured content block result;

[0019] S9, visualize the coordinates of the PDF page image and content block, the user draws a new coordinate area, and performs secondary decomposition based on the provided coordinate area information; during the secondary decomposition, the context information of step S5 is restored, the text character set obtained in step S2 is divided into character blocks according to the coordinate area obtained in step S9, and then the process from step S5 to step S9 is repeated.

[0020] Preferably, step S2 comprises the following steps:

[0021] S21, using a parsing tool, scanning the DOM tree of the PDF page to collect text content of the text nodes; if the number of collected text characters is greater than a fixed threshold, the PDF document is processed as an editable document to extract text characters and character coordinates; otherwise, the image is processed by OCR to obtain the text content;

[0022] When applying OCR processing to an image, first select the document language, then use the image base64 data and language as parameters, input the open source model, and obtain the text characters and character coordinate information.

[0023] Preferably, step S3 comprises the following steps:

[0024] S31, using the open source framework and selecting the open source model for layout analysis, defines the coordinate set of the area with text, title, list, graphic, and table labels as the mark block set P1;

[0025] S32, separating the marking results of the header and footer for the header and footer cleaning process; separating the table marking results, using the OpenCV tool library to crop the page area, and using the open source model again to perform table recognition, obtaining the table area coordinates and the table HTML data set, which are used to initialize the block collection and the content splicing process of the result blocks;

[0026] S33, using the OpenCV tool library, first calls the binarization method to process the PDF page image to enhance the image contrast, and then calls the edge detection method and the contour search method to find the area with a height greater than 1.3 times the average search character height and a width greater than one thirtieth of the page width as the image candidate area.

[0027] Preferably, step S4 comprises the following steps:

[0028] S41, traverse all the text data information obtained in step S2, and convert them into structured text units one by one, and then use the text unit as the basic processing unit; make an overlap judgment between the text unit and the table and picture area, mark them as the initial attribute of the row, and calculate the average character height avgHeight of the document;

[0029] S42, aggregate all text units into blocks:

[0030] First, traverse the set of marked blocks P1 of layout analysis, generate a corresponding sub-block for each marked block, and each sub-block automatically contains corresponding marking information; the marking information includes regional coordinate information;

[0031] Check the overlap between the text unit and the area where the block is located. If there is overlap, the text unit will be classified into the corresponding block; if the layout analysis does not result in a block, all text units will be classified into an additional independent block;

[0032] If the text unit does not intersect with any block, there are three cases: if the position of the text unit is higher than the top block, a new block is created to collect the corresponding text unit as the head block; if the position of the text unit is lower than the bottom block, a new block is created to collect the corresponding text unit as the tail block; otherwise, the text unit is classified into the corresponding block according to the calculation result of the block collection algorithm A1;

[0033] The block aggregation algorithm A1 is defined as follows:

[0034] Let the current text unit be w, and the coordinates of the center point of the rectangle where the text unit w is located be w center (w x ,w y ), the y value of the upper boundary of the region is w_top;

[0035] Let block be b, and the center coordinates of the rectangle where block b is located be b center (b x , b y ), the y value of the upper boundary of the block is b_top;

[0036] Compare the text unit w with all candidate blocks b one by one; divide the block b into upper and lower halves with w_top as the center line, where blocks with b_top < w_top are placed in the upper half, otherwise they are placed in the lower half;

[0037] Find the blocks in the upper and lower halves that are closest to the current w: For the upper half, assume that the block currently used for comparison is b cur , the current minimum block in the upper half is b min , let w center and b center The square of the Euclidean distance is d, then b cur and b min The d values ​​of the text unit w are d cur and d min ; if d cur <d min , while b cur and bmin The regions where they are located have intersections in the horizontal direction, and b cur _top<b min _top, then skip the process, otherwise b min Set to b cur ; if d cur ≥d min , while b cur and b min The regions where they are located have intersections in the horizontal direction, and b cur _top>b min _top, then b min Set to b cur ; The screening process of the lower half is analogous to that of the upper half;

[0038] After iteration, select the respective blocks b from the upper and lower halves min , prioritize upward aggregation, first look at the upper half b min Does it exist? If it exists, return it. Otherwise, return to the lower half b. min , and finally get the block to which the text unit belongs.

[0039] Preferably, step S5 comprises the following steps:

[0040] S51, based on the blocks obtained in step S42, all blocks in each PDF page are compared in pairs, the intersection relationship between all blocks is calculated, and a set of block pairs with horizontal and vertical intersections is collected; based on the intersection relationship, the following linearization process is applied and the result is returned to implement the reading order recovery step:

[0041] Linearization: Apply the block cross-relation analysis algorithm A2 to the block set and obtain three results: no processing, horizontal splitting and vertical splitting. If no processing is required, the block set is directly returned. If horizontal splitting is required, the following horizontal splitting process is applied and the result is returned. If vertical splitting is required, the following vertical splitting process is applied and the result is returned.

[0042] Horizontal splitting: The three sets LS, RS and UDM obtained by the block cross-relationship analysis algorithm A2; first add each block corresponding to LS to the temporary group LeftSet, and obtain the blocks with a vertical relationship with LS through the UDM relationship table, add them to LeftSet, and sort the blocks in LeftSet in ascending order according to the y coordinate values ​​of the area where the blocks are located; then add the blocks corresponding to RS to the temporary group RightSet, obtain the blocks with a vertical relationship with RS through the UDM relationship table, add them to RightSet, and sort the blocks in RightSet in ascending order according to the y coordinate values ​​of the area where the blocks are located. The LeftSet and RightSet are spliced ​​into a list in order, which is the required result set;

[0043] Vertical splitting: first sort the blocks in ascending order according to the y-coordinate values ​​of the areas where the blocks are located, then traverse the blocks and group them. The grouping rule is: if a block belongs to the new set TempSet, a new group is created and marked I is added, and the block is added to the new group; otherwise, it is added to the previous group; traverse the grouping results, apply the following reorganization steps to the groups marked with I and save them to the result set, and apply the linearization steps to other groups and save them to the result set;

[0044] Reorganization: traverse the blocks to be processed, apply the horizontal binary algorithm A3 to each block, put the result set into a block set, and after the traversal is completed, apply the linearization step to the block set and return the result;

[0045] The block cross-relationship analysis algorithm A2 is defined as follows:

[0046] Define the left cross block set as LS, the right cross block set as RS, and the upper and lower cross block set as UD. Define the block upper and lower cross relationship table as UDM, which stores the mapping between blocks and blocks that have a vertical cross relationship with the blocks. Suppose there is a block a, through UDM, we can find all the block sets VS that have a vertical cross relationship with a. Define the query method as UDM(x), which can be described as UDM(a)=VS.

[0047] Assume that the horizontal intersection relationship is: there are block a and block b. If the areas where a and b are located intersect after horizontal extension, then a and b have a horizontal intersection relationship. If the x value of the left boundary of a is less than the x value of the left boundary of b, then a is considered to belong to LS and b to RS. Otherwise, a belongs to RS and b belongs to LS.

[0048] Assume that the vertical intersection relationship is: there are block a and block b. If the areas where a and b are located intersect after vertical extension, then a and b have a vertical intersection relationship, a and b belong to UD, and the mapping relationship between a and b is added to UDM. At this time, the result set of UDM (a) contains block b, and the result set of UDM (b) contains block a;

[0049] It is known that by comparing the cross-relationships between blocks, we can get the sets LS, RS, UD and UDM. If LS and RS are both empty, it means the single-column mode, no splitting is required, and the algorithm is exited. Traverse UD, if there is a block that does not belong to LS and RS, get VS through UDM (a). If VS intersects with LS and RS, it means that vertical splitting is required (single-column and double-column mixed mode, upper and lower splitting is required), and the corresponding block is added to a new set TempSet. Otherwise, horizontal splitting is required (double-column mode, splitting into two columns is required).

[0050] The horizontal bisection algorithm A3 is defined as follows:

[0051] Set the horizontal coordinate value of the left border of the text unit to x left , the horizontal median of the left and right boundaries is x center ; For the set of divided text units, calculate all text units x left The mean square error (mse) of the values ​​is calculated. When the mse is greater than the set threshold (which can be set to 85), the left and right blocks are divided. First, the x values ​​of all text units are calculated. center Mean x avg , all text units are marked with x left and mean x avg The comparison is divided into two sets, the left and right sets are then added to the result set and returned; if the division condition is not met, the original text unit set is directly returned.

[0052] Preferably, step S5 further comprises the following steps:

[0053] S52, header and footer cleaning based on a hybrid method of pattern recognition + cross-page text similarity comparison:

[0054] For cleaning headers, first traverse all pages, take out the first block of each page, and get the line sequence according to algorithm A4; merge the corresponding number of lines in the line sequence from most to least as candidate texts; apply the header and footer text similarity matching algorithm A5 to multiple candidate texts between different pages to obtain similarity calculation results. If the number of matching pages is greater than or equal to 1 / 3 of the total number of pages, the matching result is considered valid, and the corresponding text lines in the page block are cleared; the cleaning of footers is similar;

[0055] Finally, scan all blocks of the page and clear all blocks in the header and footer area identified by the model;

[0056] S53, sorting the lines in all blocks, dividing the lines based on the block text line division algorithm A4, and then merging the text units in the same line into one text unit. One text unit corresponds to one text line, and one block is a collection of multiple text lines. At this point, the line ordering process is completed, regardless of the single-column or double-column mode, they are finally uniformly arranged into an ordered line set;

[0057] The block text line division algorithm A4 is defined as follows:

[0058] Define a row container structure, which contains a set of text units in the same row; define the coordinate value y Point Refers to the average y-values ​​of all text unit coordinates in the current line container;

[0059] Traverse all the text units in the block, take the y value of the text unit coordinate, and compare it with the y value of all line containers PointCompare them one by one. If the difference is less than the average character height avgHeight of the whole document, they are classified into the same line, and the corresponding text unit is added to the line container. At the same time, y is updated. Point ; If no row container matches, a new row container is added and initialized with the current text unit; after traversing all text units in the block, the row container is initialized according to y Point Sort the text cells in the row container by their respective x-coordinate values;

[0060] The header and footer text similarity matching algorithm A5 is defined as follows: for the text content of a given page, special symbols are first cleaned, and then the remaining text is sorted by characters to obtain a character sequence; the Levenshtein distance d1 is calculated for the character sequences processed from two different pages. If the obtained distance d1 is less than 2, or the ratio of the distance d1 to the maximum length of the two texts is less than 0.2, the two text sequences are considered similar.

[0061] Preferably, step S6 comprises the following steps:

[0062] S61, traversing all text lines of the PDF document, maintaining a directory tree, and inserting a node in the same layer in order into the directory tree each time a directory sequence is identified, and the node can obtain the corresponding directory sequence by reading the corresponding path;

[0063] Define the root node of the directory tree as R; define the predecessor node as PN; define the current node, that is, the corresponding candidate text line as CN; define the left margin distance dmargin as the absolute value of the difference between the left boundary x value of the text line and the left boundary x value of the minimum text line on the page; define the priority distance between two nodes as dp, and the priority distance depends on the predefined priority relationship between different rules in the regular rule library;

[0064] Let m and n be two text nodes, and define the priority distance dp calculation formula f dp (m,n), the range is {dp gt ,dp similar ,dp lt}. The calculation method is as follows: If m has a higher priority than n, then return dp gt ; If the priorities of m and n are the same, return dp similar ; If the priority of m is less than n, return dp lt Preferably, step S7 comprises the following steps:

[0065] S71, pattern matching is performed between the line text and the regular rule library. If the pattern matching is successful, the process goes to branch 1);

[0066] 1) If the predecessor node is empty and the sequence number obtained by pattern matching is greater than the preset value, it means that the predecessor node is not the starting node and is directly discarded; otherwise, it is directly added to the root node and exits; if the predecessor node is not empty, enter branch 2);

[0067] 2) If the patterns of CN and PN are the same or the priority distance fdp(CN,PN)=dp similar , then go to branch 3), otherwise go to branch 4); if the patterns of CN and PN are inconsistent, go to branch 5);

[0068] 3) Based on the processing of sequence continuity, check the sequence consistency of CN and PN. If the sequence is approximately continuous, mark the row, insert CN into the parent node of PN, and exit; if the sequence is not continuous, enter branch 6);

[0069] 4) Check the sequence continuity of CN and PN. If they are continuous, go to branch 3); otherwise, calculate the priority distance f between CN and PN dp (CN, PN) gets dp, if dp=dp gt , it indicates that it is a new sub-level node, and determines whether the sequence number is 1. If it is, CN is inserted as a child node of PN; if dp=dp lt , take the upper node PN of PN parent Compared with CN, if f dp (CN,PN parent )=dp similar If the two conditions are continuous, then go to branch 3), otherwise exit directly;

[0070] 5) Calculate the CN and PN priority distance f dp (CN, PN) gets dp, check whether CN is the start sequence. When CN is the start sequence, if dp=dp similar , then CN is the new sequence, after CN is inserted into the parent node of PN, exit; if dp=dp lt , then PN is set as the parent node of PN and enters branch 1); if dp=dp gt , then CN is inserted as a child node of PN and exits. When CN is not the start sequence, if dp=dp lt , then PN is set as the parent node of PN and enters branch 1); otherwise, enters branch 6);

[0071] 6) Use parallel sequences for comparison, the parallel sequences include rule-based sequence s1, level-based sequence s2 and position-based sequence s3; compare CN with the sequence corresponding to the rule in s1, if the comparison result is continuous, CN is inserted into the child node list of the parent node of the last node of the corresponding sequence; otherwise, take out the sequence corresponding to the level where PN is located from s2, compare CN with the sequence for continuity, if continuous, CN is inserted into the child node list of the parent node of the last node of the sequence; otherwise, search for the corresponding sequence in s3 according to the dmargin value of CN, and compare it for continuity, if continuous, CN is inserted into the child node list of the parent node of the last node of the sequence; if none of the above is true, exit.

[0072] Preferably, step S8 comprises the following steps:

[0073] S81, traverse all rows; if a directory tag is scanned, a new target block is generated; if it is ordinary text, the content is directly spliced; if there is a table tag, the content of the corresponding area is replaced with table HTML, and then the content is spliced; if there is a picture tag, the text line in the picture area content is cleared, and the picture area is cropped, converted to base64 content, and then converted to HTML Tags, and splicing content;

[0074] S82, obtaining the decomposed paragraph blocks, each block including block area coordinates, directory sequence and content information.

[0075] The present invention also provides a system for document content recognition and intelligent paragraph and chapter decomposition, including:

[0076] Splitting module, used to split PDF documents into pages and convert the content of each page into images;

[0077] The character analysis module is used to perform DOM structure analysis on the split PDF pages to obtain a text character set;

[0078] The layout analysis module is used to analyze the layout of PDF page images and obtain a layout area set;

[0079] An aggregation module, used for aggregating the obtained text character set and the obtained layout area set into character blocks;

[0080] A recovery module, used for performing a reading order recovery operation on the character blocks;

[0081] Title matching definition module, used to define the title matching pattern set, complete the title matching and hierarchy generation of the directory tree;

[0082] The directory tree generation module is used to traverse the line sequence of the block sequence, compare and analyze the text line and the title matching pattern set, mark the matching line with a directory mark, and obtain the hierarchical relationship between directories through the comparison algorithm;

[0083] The result output module is used to traverse the line sequence of the block sequence, merge the text in order, distinguish the image area, table area and text area according to the analysis result of the layout analysis module, generate three types of content blocks, and then divide the paragraphs according to the marking result of the directory tree generation module, and output the structured content block results;

[0084] The visualization and secondary disassembly module is used to visualize the coordinates of PDF page images and content blocks. The user draws a new coordinate area and performs secondary disassembly based on the provided coordinate area information.

[0085] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention realizes the extraction of structured data from PDF document content and intelligently decomposes paragraphs and chapters, which is widely applicable to scenarios with different languages ​​and different layout modes. At the same time, based on the open source model and combined with the heuristic algorithm, the present invention has the advantages of both, because the model has a more generalized advantage in general scenarios, while the heuristic algorithm has a more precise advantage in specific scenarios. Improving the recognition and decomposition accuracy based on generalized scenarios is one of the characteristics of the present invention, while reducing the difficulty of adapting to small sample document formats. (2) The secondary decomposition method provided by the present invention makes it possible to manually intervene in the results and optimize them on the basis of intelligent analysis. (3) The system constructed by the present invention can be used independently, or as a pre-processing module for systems such as the regulations management platform and RAG tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 A flow chart of the method for document content recognition and intelligent paragraph and chapter decomposition in the present invention;

[0087] Figure 2 A schematic diagram of text content of document A provided in an embodiment of the present invention;

[0088] Figure 3 for Figure 2 A schematic diagram of the text area recognition result;

[0089] Figure 4 for Figure 2 A schematic diagram of the areas and labels obtained by performing layoutparser layout analysis;

[0090] Figure 5 for Figure 2A schematic diagram of the regions and labels obtained by performing PPStructure layout analysis;

[0091] Figure 6 A schematic diagram of the results of intelligent paragraph decomposition and directory tree generation provided by an embodiment of the present invention;

[0092] Figure 7 A schematic diagram of a paragraph decomposition block area provided by an embodiment of the present invention;

[0093] Figure 8 A schematic diagram of a secondary disassembly interface provided by an embodiment of the present invention;

[0094] Fig. 9 for Figure 8 A schematic diagram for performing block area adjustment;

[0095] Fig.10 for Fig. 9 A schematic diagram of the results of intelligent paragraph decomposition and directory tree generation after secondary decomposition;

[0096] Fig.11 The present invention is a flow chart of the directory tree generation process. DETAILED DESCRIPTION

[0097] In order to more clearly illustrate the embodiments of the present invention, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings and other implementation methods can be obtained based on these accompanying drawings without creative work.

[0098] like Figure 1 As shown, the present invention provides a method for document content recognition and intelligent paragraph and chapter decomposition, comprising the following steps:

[0099] 1. Split the PDF document into pages and convert the content of each page into images;

[0100] 2. Perform DOM structure analysis on the split PDF pages to obtain the text character set;

[0101] 3. Perform layout analysis on PDF page images to obtain a set of layout areas;

[0102] 4. Aggregate the text character set obtained in step S2 and the layout area set obtained in step S3 into character blocks;

[0103] 5. Performing reading order recovery operations on character blocks, specifically:

[0104] The character blocks are reorganized, split or merged; the character blocks are then sorted to obtain an ordered sequence of blocks; within each character block, the characters are aggregated into text lines to obtain an ordered sequence of lines;

[0105] 6. Define a set of title matching patterns for title matching and hierarchy generation in the directory tree;

[0106] 7. Traverse the line sequence of the block sequence, compare and analyze the text line and the title matching pattern set, mark the matching line with a directory mark, and obtain the hierarchical relationship between directories through the comparison algorithm;

[0107] 8. Traverse the row sequence of the block sequence, merge the text in order, distinguish the image area, table area and text area according to the analysis result of step S3, generate three types of content blocks, divide the paragraphs according to the marking result of step S7, and output the structured content block result;

[0108] 9. Visualize the coordinates of the PDF page image and content block. The user draws a new coordinate area and performs secondary decomposition based on the provided coordinate area information. During the secondary decomposition, the context information of step 5 is restored, the text character set obtained in step 2 is divided into character blocks according to the coordinate area obtained in step 9, and then the process from step 5 to step 9 is repeated.

[0109] For step 2, use parsing tools such as aspose.pdf or pdfbox to scan the DOM tree of the PDF page and collect the text content of the text nodes. If the number of collected characters is greater than a fixed threshold (here 20), it is processed as a native (editable) PDF to extract the text node characters and character coordinates; otherwise, OCR is performed on the image to obtain the text content.

[0110] When applying OCR processing to an image, first select the document language, then use the open source model PaddleOCR (the OCR engine is optional, the goal is just to get the characters and coordinates), input the image base64 data and language as parameters into the model, and get the characters and character coordinate information.

[0111] For step 3, use the open source framework layoutparser and select the open source model ppyolov2_r50vd_dcn_365e for layout analysis to obtain the layout analysis results: the set of area coordinates with text, title, list, graphic, and table labels is defined as the marker block set P1.

[0112] Use the open source framework PPStructure to analyze the layout and get the layout analysis results. Separate the header and footer tag results, and use this part of the data to clean the header and footer. Separate the table tag results, use OpenCV to crop the page area, and use the model again to recognize the table to get the table area coordinates and table html data set. This part of the data is used to initialize the block collection and the content splicing process of the result block.

[0113] Using the OpenCV tool library, we first call the binarization method (adaptiveThreshold) to process the image and enhance the image contrast. Then we call the edge detection method (canny) and the contour search method (findContours) to find the area whose height is greater than 1.3 times the average search character height and whose width is greater than one-thirtieth of the page width as the image candidate area.

[0114] Through the previous steps, no matter it is a text set extracted from the DOM tree or a text set extracted by OCR, the length of the obtained text is uncertain. It may be a single character, multiple characters, or a whole line of character blocks. Therefore, these irregular character blocks need to be aggregated and organized into an ordered line sequence.

[0115] For step 4, first traverse all the text data information obtained in the previous step, convert them one by one into structured text units, and then use the text unit as the basic processing unit. The overlapping areas of the text unit and the table and picture are judged and marked as the initial attributes of the row. At the same time, the average character height avgHeight of the document is calculated.

[0116] Then all text units are divided and aggregated. First, the set of marked blocks P1 of layout analysis is traversed, and a corresponding block is generated for each marked block. Each block automatically contains the corresponding marking information (including regional coordinate information). Check the overlap between the text unit and the area where the block is located. If there is overlap, the text unit is classified into the corresponding block. If the layout analysis does not obtain a regional block, all text content is classified into an independent block. If the text unit does not have a regional intersection with any group, there are three cases: if the position is higher than the top group, a new group is created to collect this part of the text unit as the head group; if it is lower than the bottom group, a new group is created to collect this part of the text unit as the tail group; otherwise, the text block and each block are classified into their respective blocks according to the calculation results of the block collection algorithm A1. After the above steps, the preliminary block processing of the entire content of the page is completed.

[0117] For the specific process of the block aggregation algorithm A1, please refer to the corresponding invention content part, which will not be elaborated here.

[0118] For step 5, based on the blocks obtained in the previous step, all blocks in each PDF page are compared pairwise, the cross relationship between all blocks is calculated, and the horizontally and vertically crossed block pairs are collected; based on the cross relationship, the following linearization process is applied and the result is returned to implement the reading order recovery step:

[0119] Linearization: Apply the block cross-relation analysis algorithm A2 to the block set and obtain three results: no processing, horizontal splitting and vertical splitting. If no processing is required, the block set is directly returned. If horizontal splitting is required, the following horizontal splitting process is applied and the result is returned. If vertical splitting is required, the following vertical splitting process is applied and the result is returned.

[0120] Horizontal splitting: three sets LS, RS and UDM obtained by block cross-relationship analysis algorithm A2. First, add each block corresponding to LS to the temporary group LeftSet, and obtain the blocks that have a vertical relationship with LS through the UDM relationship table, add them to LeftSet, and sort the blocks in LeftSet in ascending order according to the y-coordinate values ​​of the area where the blocks are located; then add the blocks corresponding to RS to the temporary group RightSet, obtain the blocks that have a vertical relationship with RS through the UDM relationship table, add them to RightSet, and sort the blocks in RightSet in ascending order according to the y-coordinate values ​​of the area where the blocks are located. Concatenate LeftSet and RightSet in order into a list, which is the required result set;

[0121] Vertical splitting: first sort the blocks in ascending order according to the y-coordinate values ​​of the areas where the blocks are located, then traverse the blocks and group them. The grouping rule is: if a block belongs to the new set TempSet, a new group is created and marked I is added, and the block is added to the new group; otherwise, it is added to the previous group; traverse the grouping results, apply the following reorganization steps to the groups marked with I and save them to the result set, and apply the linearization steps to other groups and save them to the result set;

[0122] Reorganization: traverse the blocks to be processed, apply the horizontal binary algorithm A3 to each block, put the result set into a block set, and after the traversal is completed, apply the linearization step to the block set and return the result;

[0123] Among them, the specific processes of the block cross-relationship analysis algorithm A2 and the horizontal binary division algorithm A3 can be found in the corresponding invention content part, which will not be elaborated here.

[0124] Step 5 also includes a header and footer cleaning process, which is based on a hybrid method of pattern recognition + cross-page text similarity comparison.

[0125] Taking header cleaning as an example, first traverse all pages, take out the first block of each page, and obtain the line sequence according to the block text line separation algorithm A4. The line sequence merges the corresponding number of lines from most to least as candidate texts. Here, the upper limit of the number is 5, so that at most 5 candidate texts consisting of 5 lines, 4 lines, and 1 line of text are obtained. Apply the header and footer text similarity matching algorithm A5 to multiple candidate texts between different pages to obtain similarity calculation results. If the number of matching pages is greater than or equal to 1 / 3 of the total number of pages (if the number of pages is 2, then 1 is taken), the matching result is considered valid, and the corresponding text lines in these page blocks are cleared. The footer cleaning is similar. Finally, scan all the blocks of the page, and clear all the blocks in the header and footer area identified by the model. The two methods complement each other to improve the reliability of header and footer cleaning.

[0126] Finally, all blocks are sorted within the blocks, mainly based on the A4 algorithm, and then the text units in the same row are merged into one text unit. One text unit corresponds to one text line, and one block is a collection of multiple text lines. At this point, the line sorting process is completed, regardless of single-column or double-column mode, it is finally uniformly arranged into an ordered line set.

[0127] Among them, the specific processes of the block text line division algorithm A4 and the header and footer text similarity matching algorithm A5 can be found in the corresponding invention content part, which will not be elaborated here.

[0128] For step 6, the generation of the directory tree mainly relies on identifying whether the text line header part meets the context-consistent sequence pattern, so as to mark the relevant lines. Heuristic rules are mainly used, and regular rule bases are defined for different languages. At the same time, there is a priority relationship between the rules.

[0129] First, all text lines are traversed while maintaining the directory tree. Every time a directory sequence is identified, an ordered node at the same level is inserted into the directory tree, so that the node can obtain its directory sequence by simply reading its path.

[0130] Define the root node R of the directory tree, define the predecessor node as PN (updated to the current node when recognition is successful, and serve as the predecessor node of the next node), and the current node (corresponding to the candidate text line) as CN (assuming that the rule matching result has been obtained). Define the left margin distance dmargin as the absolute value of the difference between the left boundary x value of the text line and the left boundary x value of the minimum text line on the page. Define the priority distance between two nodes as dp. The priority distance depends on the predefined priority relationship between different rules in the regular rule library. Let m and n be two text nodes, and define the priority distance dp calculation formula f dp (m,n), the range is {dp gt ,dp similar ,dplt}. The calculation method is as follows: If m has a higher priority than n, then return dp gt ; If the priorities of m and n are the same, return dp similar ; If the priority of m is less than n, return dp lt .

[0131] For step 7, Fig.11 As shown, the title recognition and hierarchical relationship generation are performed according to the following steps: first, the line text and the regular rule library are pattern matched, and if the match is successful, enter branch 1).

[0132] 1) If the predecessor node is empty and the sequence number obtained by pattern matching is greater than the preset value (for example, 5), it may be a non-starting node and is directly discarded; otherwise, it is directly added to the root node and exits. If the predecessor node is not empty, enter branch 2).

[0133] 2) If the patterns of CN and PN are the same or the priority distance fdp(CN,PN)=dp similar , then go to branch 3), otherwise go to branch 4). If the patterns of CN and PN are inconsistent, go to branch 5).

[0134] 3) This branch is mainly based on the processing of sequence continuity. It checks the sequence consistency of CN and PN. If the sequence is approximately continuous, the row is marked and CN is inserted after the parent node of PN (assuming PN * is the parent node of PN), exit. If the sequence is not continuous, enter branch 6).

[0135] 4) Check the sequence continuity of CN and PN. If they are continuous, go to branch 3); otherwise, calculate the priority distance f between CN and PN dp (CN, PN) gets dp, if dp=dp gt , it indicates that it is a new sub-level node, and determines whether the sequence number is 1. If it is, CN is inserted as a child node of PN; if dp=dp lt , take the upper node PN of PN parent Compared with CN, if f dp (CN,PN parent )=dp similar And it is continuous, then go to branch 3), otherwise exit directly.

[0136] 5) Calculate the CN and PN priority distance f dp (CN, PN) gets dp, check whether CN is the start sequence. When CN is the start sequence, if dp=dp similar , then CN is the new sequence, after CN is inserted into the parent node of PN, exit; if dp=dp lt, then PN is set as the parent node of PN and enters branch 1); if dp=dp gt , then CN is inserted as a child node of PN and exits. When CN is not the start sequence, if dp=dp lt , then PN is set as the parent node of PN and enters branch 1); otherwise, enters branch 6).

[0137] 6) This branch uses parallel sequences for comparison. Three parallel sequences are maintained in the context: rule-based sequence s1 (one sequence for each rule), level-based sequence s2 (one sequence for each tree level), and position-based sequence s3 (using the dmargin of the text line to correspond to different sequences). CN is compared with the sequence corresponding to the rule in s1. If the comparison result is continuous, CN is inserted into the child node list of the parent node of the last node in the sequence; otherwise, the sequence corresponding to the level where PN is located is taken out from s2, and CN is compared with the sequence for continuity. If it is continuous, CN is inserted into the child node list of the parent node of the last node in the sequence; otherwise, the corresponding sequence in s3 is searched according to the dmargin value of CN, and a continuity comparison is performed. If it is continuous, CN is inserted into the child node list of the parent node of the last node in the sequence; if none of the above is true, exit.

[0138] The above steps complete the directory marking of the text line.

[0139] For step 8, traverse all the lines. If a directory tag is scanned, a new target block is generated. If it is ordinary text, the content is directly spliced. If there is a table tag, the content of the area is replaced with the table html and the content is spliced. If there is an image tag, the text lines of part of the image area are cleared, and the image area is cropped using OpenCV, converted to base64 content, and then converted to html. Tags, splicing content. According to the above steps, you can get the disassembled paragraph blocks, each block contains block area coordinates, directory sequence, content and other information.

[0140] For step 9, the structured data generated above contains the regional coordinate information of all blocks, and then the regional coordinates are drawn with the PDF page image as the background, and the visual segmentation results can be obtained. Based on this, the drawing area is added, deleted, and modified on the page, and the regional coordinate information is resubmitted to the parsing process to complete the real-time dynamic parsing adjustment.

[0141] The second decomposition process can directly reuse the algorithm steps 5-9. The difference is that the aforementioned step 4 is based on a certain algorithm to classify characters into blocks. Here, characters are classified into divided areas by overlapping area matching, and a special unique identifier is added to each text area. In step 7, when a special identifier is recognized, it is directly divided into new paragraphs, and the rest is consistent with the original logic. In this way, the decomposition can be smooth and the directory tree structure can be restored.

[0142] The present invention also provides a system for document content recognition and intelligent paragraph and chapter decomposition, including:

[0143] Splitting module, used to split PDF documents into pages and convert the content of each page into images;

[0144] The character analysis module is used to perform DOM structure analysis on the split PDF pages to obtain a text character set;

[0145] The layout analysis module is used to analyze the layout of PDF page images and obtain a layout area set;

[0146] An aggregation module, used for aggregating the obtained text character set and the obtained layout area set into character blocks;

[0147] A recovery module, used for performing a reading order recovery operation on the character blocks;

[0148] Title matching definition module, used to define the title matching pattern set, complete the title matching and hierarchy generation of the directory tree;

[0149] The directory tree generation module is used to traverse the line sequence of the block sequence, compare and analyze the text line and the title matching pattern set, mark the matching line with a directory mark, and obtain the hierarchical relationship between directories through the comparison algorithm;

[0150] The result output module is used to traverse the line sequence of the block sequence, merge the text in order, distinguish the image area, table area and text area according to the analysis result of the layout analysis module, generate three types of content blocks, and then divide the paragraphs according to the marking result of the directory tree generation module, and output the structured content block results;

[0151] The visualization and secondary disassembly module is used to visualize the coordinates of PDF page images and content blocks. The user draws a new coordinate area and performs secondary disassembly based on the provided coordinate area information.

[0152] In order to more clearly illustrate the purpose and technical solution of the present invention, the specific implementation manner of the present invention will be described below with reference to the accompanying drawings.

[0153] by Figure 2One page of document A is shown to illustrate the workflow of the present invention. The layout of the page includes structures such as header, title, paragraph and sub-paragraph. There are identifiable serial numbers in the title and paragraph, and a title tree with a hierarchical structure can be constructed based on the serial numbers. The document is a scanned copy, and the text content extraction will rely on OCR processing.

[0154] The specific operation process is as follows:

[0155] 1. Use aspose.pdf to split document A into pages, and then convert each page into images to obtain image set P1.

[0156] 2. Use the aspose.pdf tool to traverse the DOM tree of the page. The collected text node list is empty, so the logical branch of OCR content extraction is executed.

[0157] 3. Call the OCR interface to parse the picture set P1, obtain the text and text coordinate set C1, and complete the text content extraction. The obtained text coordinates are as follows: Figure 3 As shown in , we can see that the boxed part identifies all the recognized texts, but the text units are somewhat fragmented.

[0158] 4. Call OpenCV to perform edge detection on the image set P1 to obtain the document image content candidate area set D1.

[0159] 5. Call layoutparser to perform layout analysis and obtain the layout result set L1. The layout analysis results are as follows: Figure 4 As shown, it can be seen that this page document only recognizes one area marked as a list.

[0160] 6. Call PPStructure to perform layout analysis. The layout analysis results are as follows: Figure 5 As shown, there are four rows marked as titles and one header row. The three types of layout analysis area sets are obtained by filtering: header and footer area, table area, and title area.

[0161] 7. Call the block grouping algorithm and use the L1 layout set to block the text result set C1 to obtain the character block set C2. Since L1 has only one area, only one block is obtained here.

[0162] 8. Perform the reading order recovery operation for C2. According to the algorithm result, there is no need to split the block, so the output result still has only one block, recorded as C3, but the line division has been completed within the block, and the line division is in order.

[0163] 9. Traverse the row sequence of C3 and match the title pattern line by line. In this way, the rows where [I. Purpose], [II. Glossary], [2.1 Management File Definition], etc. are located are identified as the rows where the title structure is located, and recorded in the row structure attribute. The automatic construction result of the title tree is as follows: Figure 6 shown.

[0164] 10. Execute the process of merging lines to generate paragraphs, and complete the content segmentation step. The segmentation result is as follows: Figure 7 The final visualization of the structured content is shown in Figure 6 shown.

[0165] 11. Next, a secondary disassembly operation is performed based on the above disassembly results.

[0166] 12. First, obtain the disassembled images and block area coordinates for interactive display on the front end, such as Figure 8 shown.

[0167] 12. In Figure 8 Based on the above, the block area is adjusted, the regional border of [3) Document Revocation] is removed, the regional border range of paragraph [2.2] is reduced, and the regional borders of paragraphs [1) Document Addition] and [2) Document Revision] are merged. The regional border adjustment is as follows: Fig. 9 shown.

[0168] 13. Perform the re-disassembly process and output the structured content and title tree, such as Fig.10 As shown in the figure. It can be seen that the title tree has removed the content of [3) File Revocation], but still maintains a valid tree structure, which meets the support for adding and deleting regions. The paragraph [2.2] only extracts the text content of the specified area, which meets the support for regional scaling. The paragraphs [1) File Addition][2) File Revision] are combined into one paragraph to return, which meets the support for regional merging.

[0169] The innovative features of the present invention are as follows:

[0170] 1. Effectively extract document content, adapt to various layout formats and language types, and output line sequences that conform to the reading order.

[0171] 2. Provide an efficient method for clearing headers and footers.

[0172] 3. The document content is intelligently broken down into chapters and paragraphs to generate a directory tree structure.

[0173] 4. The manual adjustment process is introduced, so that the document can be disassembled twice to maximize the reliability of the output results.

[0174] The above description is only a detailed description of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, according to the ideas provided by the present invention, there will be changes in the specific implementation methods, and these changes should also be regarded as the protection scope of the present invention.

Claims

1. A method for document content recognition and intelligent paragraph and chapter decomposition, characterized in that: The method comprises the following steps: S1, split the PDF document into pages and convert the content of each page into images; S2, performing DOM structure analysis on the split PDF pages to obtain a text character set; S3, performing layout analysis on the PDF page image to obtain a layout area set; S4, aggregating the text character set obtained in step S2 and the layout area set obtained in step S3 into character blocks; S5, performing a reading order recovery operation on the character blocks, specifically: The character blocks are reorganized, split or merged; the character blocks are then sorted to obtain an ordered sequence of blocks; within each character block, the characters are aggregated into text lines to obtain an ordered sequence of lines; S6, defines a set of title matching patterns for title matching and level generation of the directory tree; S7, traverse the line sequence of the block sequence, compare and analyze the text line and the title matching pattern set, mark the matching line with a directory mark, and obtain the hierarchical relationship between the directories through a comparison algorithm; S8, traverse the row sequence of the block sequence, merge the text in order, distinguish the image area, the table area and the text area according to the analysis result of step S3, generate three types of content blocks, divide the paragraphs according to the marking result of step S7, and output the structured content block result; S9, visualize the coordinates of the PDF page image and content block, the user draws a new coordinate area, and performs secondary decomposition based on the provided coordinate area information; during the secondary decomposition, the context information of step S5 is restored, the text character set obtained in step S2 is divided into character blocks according to the coordinate area obtained in step S9, and then the process from step S5 to step S9 is repeated.

2. The method for document content recognition and intelligent paragraph and chapter decomposition according to claim 1, characterized in that: Step S2 includes the following steps: S21, using a parsing tool, scanning the DOM tree of the PDF page to collect text content of the text nodes; if the number of collected text characters is greater than a fixed threshold, the PDF document is processed as an editable document to extract text characters and character coordinates; otherwise, the image is processed by OCR to obtain the text content; When applying OCR processing to an image, first select the document language, then use the image base64 data and language as parameters, input the open source model, and obtain the text characters and character coordinate information.

3. The method for document content recognition and intelligent paragraph and chapter decomposition according to claim 2, characterized in that: Step S3 includes the following steps: S31, using the open source framework and selecting the open source model for layout analysis, defines the coordinate set of the area with text, title, list, graphic, and table labels as the mark block set P1; S32, separating the marking results of the header and footer for the header and footer cleaning process; separating the table marking results, using the OpenCV tool library to crop the page area, and using the open source model again to perform table recognition, obtaining the table area coordinates and the table HTML data set, which are used to initialize the block collection and the content splicing process of the result blocks; S33, using the OpenCV tool library, first calls the binarization method to process the PDF page image to enhance the image contrast, and then calls the edge detection method and the contour search method to find the area with a height greater than 1.3 times the average search character height and a width greater than one thirtieth of the page width as the image candidate area.

4. The method for document content recognition and intelligent paragraph and chapter decomposition according to claim 3, characterized in that: Step S4 includes the following steps: S41, traverse all the text data information obtained in step S2, and convert them into structured text units one by one, and then use the text unit as the basic processing unit; make an overlap judgment between the text unit and the table and picture area, mark them as the initial attribute of the row, and calculate the average character height avgHeight of the document; S42, aggregate all text units into blocks: First, traverse the set of marked blocks P1 of layout analysis, generate a corresponding sub-block for each marked block, and each sub-block automatically contains corresponding marking information; the marking information includes regional coordinate information; Check the overlap between the text unit and the area where the block is located. If there is overlap, the text unit will be classified into the corresponding block; if the layout analysis does not result in a block, all text units will be classified into an additional independent block; If the text unit does not intersect with any block, there are three cases: if the text unit is higher than the top block, a new block is created to collect the corresponding text unit as the head block; if the text unit is lower than the bottom block, a new block is created to collect the corresponding text unit as the tail block; Otherwise, the text unit is classified into the corresponding block according to the calculation result of the block grouping algorithm A1; The block aggregation algorithm A1 is defined as follows: Let the current text unit be w, and the coordinates of the center point of the rectangle where the text unit w is located be w center (w x , w y ), the y value of the upper boundary of the region is w_top; Let block be b, and the center coordinates of the rectangle where block b is located be b center (b x , b y ), the y value of the upper boundary of the block is b_top; Compare the text unit w with all candidate blocks b one by one; divide the block b into two halves with w_top as the center line, where blocks with b_top < w_top are placed in the upper half, otherwise they are placed in the lower half; Find the blocks in the upper and lower halves that are closest to the current w: For the upper half, assume that the block currently used for comparison is b cur , the current minimum block in the upper half is b min , let w center and b center The square of the Euclidean distance is d, then b cur and b min The d values ​​of the text unit w are d cur and d min ; if d cur <d min , while b cur and b min The regions where they are located have intersections in the horizontal direction, and b cur _top<b min _top, then skip the process, otherwise b min Set to b cur ; if d cur ≥d min , while b cur and b min The regions where they are located have intersections in the horizontal direction, and b cur _top>b min _top, then b min Set to b cur ; The screening process of the lower half is analogous to that of the upper half; After iteration, select the respective blocks b from the upper and lower halves min , prioritize upward aggregation, first look at the upper half b min Does it exist? If it exists, return it. Otherwise, return to the lower half b. min , and finally get the block to which the text unit belongs.

5. The method for document content recognition and intelligent paragraph and chapter decomposition according to claim 4, characterized in that: Step S5 includes the following steps: S51, based on the blocks obtained in step S42, all blocks in each PDF page are compared in pairs, the intersection relationship between all blocks is calculated, and a set of block pairs with horizontal and vertical intersections is collected; based on the intersection relationship, the following linearization process is applied and the result is returned to implement the reading order recovery step: Linearization: Apply the block cross-relation analysis algorithm A2 to the block set and obtain three results: no processing, horizontal splitting and vertical splitting. If no processing is required, the block set is directly returned. If horizontal splitting is required, the following horizontal splitting process is applied and the result is returned. If vertical splitting is required, the following vertical splitting process is applied and the result is returned. Horizontal splitting: The three sets LS, RS and UDM obtained by the block cross relationship analysis algorithm A2; first, add each block corresponding to LS to the temporary group LeftSet, and obtain the block set with a vertical relationship with LS through the UDM relationship table, add it to LeftSet, and sort the blocks in LeftSet in ascending order according to the y coordinate values ​​of the area where the blocks are located; then add the block corresponding to RS to the temporary group RightSet, obtain the block with a vertical relationship with RS through the UDM relationship table, add it to RightSet, and sort the blocks in RightSet in ascending order according to the y coordinate values ​​of the area where the blocks are located; splice LeftSet and RightSet in order into a list, which is the required result set; Vertical splitting: first sort the blocks in ascending order according to the y-coordinate values ​​of the areas where the blocks are located, then traverse the blocks and group them. The grouping rule is: if a block belongs to the new set TempSet, a new group is created and marked I is added, and the block is added to the new group; otherwise, it is added to the previous group; traverse the grouping results, apply the following reorganization steps to the groups marked with I and save them to the result set, and apply the linearization steps to other groups and save them to the result set; Reorganization: traverse the blocks to be processed, apply the horizontal binary algorithm A3 to each block, put the result set into a block set, and after the traversal is completed, apply the linearization step to the block set and return the result; The block cross-relationship analysis algorithm A2 is defined as follows: Define the left cross block set as LS, the right cross block set as RS, and the upper and lower cross block set as UD; define the upper and lower cross relationship table of blocks as UDM, which stores the mapping between blocks and blocks that have a vertical cross relationship with the blocks. Assuming there is a block a, the UDM can be used to find all the block sets VS that have a vertical cross relationship with a. The query method is defined as UDM(x), and the corresponding description is UDM(a)=VS. Assume that the horizontal intersection relationship is: there are block a and block b. If the areas where a and b are located intersect after horizontal extension, then a and b have a horizontal intersection relationship; if the x value of the left boundary of a is less than the x value of the left boundary of b, then a is considered to belong to LS and b to RS, otherwise a belongs to RS and b belongs to LS; Assume that the vertical intersection relationship is: there are block a and block b. If the areas where a and b are located have an intersection after vertical extension, then a and b have a vertical intersection relationship, a and b belong to UD, and the mapping relationship between a and b is added to UDM. At this time, the result set of UDM (a) contains block b, and the result set of UDM (b) contains block a. It is known that by comparing the cross-relationships between blocks, we can get the sets LS, RS, UD and UDM. If LS and RS are both empty, it means the single-column mode, no splitting is required, and the algorithm is exited. Traverse UD, if there is a block a that does not belong to LS and RS, get VS through UDM (a). If VS has an intersection with LS and RS, it means that vertical splitting is required, and the corresponding block is added to a new set TempSet. Otherwise, horizontal splitting is required. The horizontal bisection algorithm A3 is defined as follows: Set the horizontal coordinate value of the left border of the text unit to x left , the horizontal median of the left and right boundaries is x center ; For the set of divided text units, calculate all text units x left The mean square error (mse) of the values ​​is calculated. When the mse is greater than the set threshold, the left and right blocks are divided. First, the x values ​​of all text units are calculated. center Mean x avg , all text units are marked with x left and mean x avg The comparison is divided into two sets, the left and right sets are then added to the result set and returned; if the division condition is not met, the original text unit set is directly returned.

6. The method for document content recognition and intelligent paragraph and chapter decomposition according to claim 5, characterized in that: Step S5 also includes the following steps: S52, header and footer cleaning based on a hybrid method of pattern recognition + cross-page text similarity comparison: To clean the header, first traverse all pages, take out the first block of each page, and get the row sequence according to Algorithm A4; The line sequence is from most to least, and the corresponding number of lines are merged as candidate texts; Apply the header and footer text similarity matching algorithm A5 to multiple candidate texts between different pages to obtain similarity calculation results. If the number of matching pages is greater than or equal to one third of the total number of pages, the matching result is considered valid, and the corresponding text lines in the page blocks are cleared; the footers are cleaned up in the same way; Finally, scan all blocks of the page and clear all blocks in the header and footer area identified by the model; S53, sorting the lines in all blocks, dividing the lines based on the block text line division algorithm A4, and then merging the text units in the same line into one text unit. One text unit corresponds to one text line, and one block is a collection of multiple text lines. At this point, the line ordering process is completed, regardless of the single-column or double-column mode, they are finally uniformly arranged into an ordered line set; The block text line division algorithm A4 is defined as follows: Define a row container structure, which contains a set of text units in the same row; define the coordinate value y Point Refers to the average y-values ​​of all text unit coordinates in the current line container; Traverse all the text units in the block, take the y value of the text unit coordinate, and compare it with the y value of all line containers Point Compare them one by one. If the difference is less than the average character height avgHeight of the whole document, they are classified into the same line, and the corresponding text unit is added to the line container. At the same time, y is updated. Point ; If no row container matches, a new row container is added and initialized with the current text unit; after traversing all text units in the block, the row container is initialized according to y Point Sort the text cells in the row container by their respective x-coordinate values; The header and footer text similarity matching algorithm A5 is defined as follows: for the text content of a given page, special symbols are first cleaned, and then the remaining text is sorted by characters to obtain a character sequence; the Levenshtein distance d1 is calculated for the character sequences processed from two different pages. If the obtained distance d1 is less than 2, or the ratio of the distance d1 to the maximum length of the two texts is less than 0.2, the two text sequences are considered similar.

7. The method for document content recognition and intelligent paragraph and chapter decomposition according to claim 6, characterized in that: Step S6 includes the following steps: S61, traversing all text lines of the PDF document, maintaining a directory tree, and inserting a node in the same layer in order into the directory tree each time a directory sequence is identified, and the node can obtain the corresponding directory sequence by reading the corresponding path; Define the root node of the directory tree as R; define the predecessor node as PN; define the current node, that is, the corresponding candidate text line as CN; define the left margin distance dmargin as the absolute value of the difference between the left boundary x value of the text line and the left boundary x value of the minimum text line on the page; define the priority distance between two nodes as dp, and the priority distance depends on the predefined priority relationship between different rules in the regular rule library; Let m and n be two text nodes, and define the priority distance dp calculation formula f dp (m,n), the range is {dp gt ,dp similar ,dp lt }; The calculation method is as follows: If m has a higher priority than n, then return dp gt ; If the priorities of m and n are the same, return dp similar ; If the priority of m is less than n, return dp lt .

8. The method for document content recognition and intelligent paragraph and chapter decomposition according to claim 7, characterized in that: Step S7 includes the following steps: S71, pattern matching is performed between the line text and the regular rule library. If the pattern matching is successful, the process goes to branch 1); 1) If the predecessor node is empty and the sequence number obtained by pattern matching is greater than the preset value, it means that the predecessor node is not the starting node and is directly discarded; Otherwise, add it directly to the root node and exit; If the predecessor node is not empty, go to branch 2); 2) If the patterns of CN and PN are the same or the priority distance is f dp (CN,PN)=dp similar , then go to branch 3), otherwise go to branch 4); if the patterns of CN and PN are inconsistent, go to branch 5); 3) Based on the processing of sequence continuity, check the sequence consistency of CN and PN. If the sequence is approximately continuous, mark the row, insert CN into the parent node of PN, and exit; If the sequence is not continuous, go to branch 6); 4) Check the sequence continuity of CN and PN. If they are continuous, go to branch 3); otherwise, calculate the priority distance f between CN and PN dp (CN, PN) gets dp, if dp=dp gt , it indicates that it is a new sub-level node, and determines whether the sequence number is 1. If so, CN is inserted as a child node of PN; If dp=dp lt , take the upper node PN of PN parent Compared with CN, if f dp (CN,PN parent )=dp similar And PN parent If the sequence is continuous with CN, then go to branch 3), otherwise exit directly; 5) Calculate the CN and PN priority distance f dp (CN, PN) gets dp, check whether CN is the starting sequence; when CN is the starting sequence, if dp=dp similar , then CN is the new sequence, and after CN is inserted into the parent node of PN, it exits; If dp=dp lt , then PN is set as the parent node of PN and enters branch 1); If dp=dp gt , then CN is inserted as a child node of PN and exits; When CN is not the start sequence, if dp=dp lt , then PN is set as the parent node of PN and enters branch 1); Otherwise enter branch 6); 6) using parallel sequences for comparison, the parallel sequences comprising a rule-based sequence s1, a level-based sequence s2, and a position-based sequence s3; CN is compared with the sequence of the corresponding rule in s1. If the comparison result is continuous, CN is inserted into the child node list of the parent node of the last node of the corresponding sequence; otherwise, the sequence corresponding to the level where PN is located is taken out from s2, and CN is compared with the sequence for continuity. If it is continuous, CN is inserted into the child node list of the parent node of the last node of the sequence; otherwise, the corresponding sequence in s3 is searched according to the dmargin value of CN, and a continuity comparison is performed. If it is continuous, CN is inserted into the child node list of the parent node of the last node of the sequence; if none of the above is true, exit.

9. The method for document content recognition and intelligent paragraph and chapter decomposition according to claim 8, characterized in that: Step S8 includes the following steps: S81, traverse all rows; if a directory tag is scanned, generate a new target block; if it is ordinary text, directly splice the content; if there is a table tag, replace all the content in the corresponding area with table HTML, and then splice the content; if there is an image tag, clear the text line in the image area content, crop the image area, convert it to base64 content, and then convert it to HTML tag, and splice the content; S82, obtaining the decomposed paragraph blocks, each block including block area coordinates, directory sequence and content information.

10. A system for document content recognition and intelligent paragraph and chapter separation, used to implement the method for document content recognition and intelligent paragraph and chapter separation as claimed in any one of claims 1 to 9, characterized in that: The system for document content recognition and intelligent paragraph and chapter decomposition comprises: Splitting module, used to split PDF documents into pages and convert the content of each page into images; The character analysis module is used to perform DOM structure analysis on the split PDF pages to obtain a text character set; The layout analysis module is used to analyze the layout of PDF page images and obtain a layout area set; An aggregation module, used for aggregating the obtained text character set and the obtained layout area set into character blocks; A recovery module, used for performing a reading order recovery operation on the character blocks; Title matching definition module, used to define the title matching pattern set, complete the title matching and hierarchy generation of the directory tree; The directory tree generation module is used to traverse the line sequence of the block sequence, compare and analyze the text line and the title matching pattern set, mark the matching line with a directory mark, and obtain the hierarchical relationship between directories through the comparison algorithm; The result output module is used to traverse the line sequence of the block sequence, merge the text in order, distinguish the image area, table area and text area according to the analysis result of the layout analysis module, generate three types of content blocks, and then divide the paragraphs according to the marking result of the directory tree generation module, and output the structured content block results; The visualization and secondary disassembly module is used to visualize the coordinates of PDF page images and content blocks. The user draws a new coordinate area and performs secondary disassembly based on the provided coordinate area information.

Images