Document data processing system and method based on lightweight model

Abstract

The invention provides a document data processing system and method based on a lightweight model, and the system comprises a document conversion module which is used for converting a to-be-processed document into a picture file; the layout analysis module is used for processing the picture file according to a pre-constructed layout analysis lightweight model, and extracting the category and area coordinates of each label in the picture file; the layout restoration module is used for respectively analyzing and processing different types of label areas according to the type and the area coordinate of each label to obtain the content corresponding to each label area; obtaining all contents of the to-be-processed document by combining the contents corresponding to each label area; and the text segmentation module is used for segmenting all contents of the to-be-processed document into text blocks with preset sizes through a divider, and completing format conversion and document data processing of the to-be-processed document. And rapid identification and separation of graphs, tables and characters in the PDF format document are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a document data processing system and method based on a lightweight model. Background Art

[0002] In the current digital environment, financial statements, corporate research reports, etc. contain a large amount of drawn chart information, which is often saved in PDF format. When extracting PDF content, it is critical to accurately identify and separate table and image information. This not only helps to avoid mixing chart text into the text, but also supports further analysis and processing of chart data. Document layout analysis is the core step of information extraction. How to quickly and accurately extract document information is a problem that needs to be solved urgently. Summary of the invention

[0003] In view of the above technical problems, the present invention provides a document data processing system and method based on a lightweight model, which completely identifies and separates the table and picture areas based on an open source lightweight layout analysis model in the Chinese field, and constructs independent data processing methods for different data types. The system includes: a document conversion module, a layout analysis module, a layout restoration module, and a text segmentation module;

[0004] A document conversion module, used to convert the document to be processed into an image file;

[0005] A layout analysis module, used to process the image file according to a pre-built layout analysis lightweight model, and extract the category and region coordinates of each label in the image file;

[0006] The layout restoration module is used to parse and process the label areas of different categories according to the category and area coordinates of each label, and obtain the content corresponding to each label area; by combining the content corresponding to each label area, the entire content of the document to be processed is obtained;

[0007] The text segmentation module is used to segment the entire content of the document to be processed into text blocks of a preset size through a segmenter, thereby completing the format conversion and document data processing of the document to be processed.

[0008] Furthermore, the document conversion module is used to convert the document to be processed into an image file, including:

[0009] Convert the PDF document to be processed into multiple image files through the component pdfplumber.

[0010] Furthermore, the categories of each tag include:

[0011] Main text Text, title Title, picture Figure, picture caption Figure caption, table Table, table caption Table caption, formula Equation.

[0012] Furthermore, the layout restoration module is used to parse and process the label areas of different categories according to the category and area coordinates of each label, and obtain the content corresponding to each label area, including:

[0013] Position each label according to its regional coordinates;

[0014] The area of ​​the label whose category is text is processed by the OCR component to obtain the content corresponding to the area of ​​text;

[0015] Convert the area with the tag category of Table to Markdown format and obtain the content corresponding to the area of ​​Table;

[0016] The region with the label category of "Figure" is analyzed by the multimodal model to obtain the content corresponding to the region of "Figure";

[0017] Convert the area with the label category of Equation to Latex format and obtain the content corresponding to the area of ​​Equation.

[0018] Furthermore, the text segmentation module is used to segment the entire content of the document to be processed into text blocks of a preset size through a segmenter to complete the format conversion and document data processing of the document to be processed, including:

[0019] Based on a recursive method, the entire content of the document to be processed is divided into a set of blocks equal to or smaller than a specified size by a splitter;

[0020] Any chunks smaller than the chunk size are re-merged until the chunk size is reached;

[0021] Reappend the non-whitespace character separator to the end of the block.

[0022] The present invention also provides a document data processing method based on a lightweight model, comprising:

[0023] Obtain the document to be processed, and convert the document to be processed into an image file;

[0024] Processing the image file according to a pre-built layout analysis lightweight model to extract the category and region coordinates of each label in the image file;

[0025] According to the category and area coordinates of each tag, the tag areas of different categories are parsed and processed respectively to obtain the content corresponding to each tag area; by combining the content corresponding to each tag area, the entire content of the document to be processed is obtained;

[0026] The entire content of the document to be processed is divided into text blocks of a preset size by a splitter, thereby completing the format conversion and document data processing of the document to be processed.

[0027] Furthermore, the document to be processed is converted into an image file, including:

[0028] Convert the PDF document to be processed into multiple image files through the component pdfplumber.

[0029] Furthermore, the categories of each tag include:

[0030] Main text Text, title Title, picture Figure, picture caption Figure caption, table Table, table caption Table caption, formula Equation.

[0031] Furthermore, according to the category and area coordinates of each label, the label areas of different categories are parsed and processed respectively to obtain the content corresponding to each label area, including:

[0032] Position each label according to its regional coordinates;

[0033] The area of ​​the label whose category is text is processed by the OCR component to obtain the content corresponding to the area of ​​text;

[0034] Convert the area with the tag category of Table to Markdown format and obtain the content corresponding to the area of ​​Table;

[0035] The region with the label category of "Figure" is analyzed by the multimodal model to obtain the content corresponding to the region of "Figure";

[0036] Convert the area with the label category of Equation to Latex format and obtain the content corresponding to the area of ​​Equation.

[0037] Furthermore, the entire content of the document to be processed is divided into text blocks of a preset size by a segmenter to complete the format conversion and document data processing of the document to be processed, including:

[0038] Based on a recursive method, the entire content of the document to be processed is divided into a set of blocks equal to or smaller than a specified size by a splitter;

[0039] Any chunks smaller than the chunk size are re-merged until the chunk size is reached;

[0040] Reappend the non-whitespace character separator to the end of the block.

[0041] The present invention provides a document data processing system and method based on a lightweight model. Based on an open source lightweight layout analysis model in the Chinese field, the system completely identifies and separates tables and image areas, effectively realizing the rapid identification and separation of graphics, tables and text in PDF format documents such as research reports and financial reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a structural schematic diagram of a document data processing system based on a lightweight model provided by an embodiment of the present invention;

[0043] Figure 2 It is a flowchart of a document data processing method based on a lightweight model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0045] The present invention provides a document data processing system based on a lightweight model, the system includes: a document conversion module, a layout analysis module, a layout restoration module and a text segmentation module; the document conversion module is responsible for converting PDF files into images, the layout analysis module identifies and separates tables, images and texts, the layout restoration reassembles the processed data together, and the text segmentation module uses a segmenter with as much semantic meaning as possible to achieve text segmentation in a recursive manner. The detailed data processing flow is as follows: Figure 1 As shown:

[0046] A document conversion module, used to convert the document to be processed into an image file;

[0047] A layout analysis module, used to process the image file according to a pre-built layout analysis lightweight model, and extract the category and region coordinates of each label in the image file;

[0048] The layout restoration module is used to parse and process the label areas of different categories according to the category and area coordinates of each label, and obtain the content corresponding to each label area; by combining the content corresponding to each label area, the entire content of the document to be processed is obtained;

[0049] The text segmentation module is used to segment the entire content of the document to be processed into text blocks of a preset size through a segmenter, thereby completing the format conversion and document data processing of the document to be processed.

[0050] Furthermore, the document to be processed specifically refers to a PDF document. The text segmentation module converts PDF into images, which is an important prerequisite for layout analysis. Given a PDF file, based on pdfplumber, it is necessary to cut the PDF according to each page and convert it into a set of images with a certain clarity.

[0051] In the layout analysis module, the categories of each tag involved include: Text, Title, Figure, Figure caption, Table, Table caption, and Equation. The details are as follows:

[0052] element name Text Main text (paragraph) Title title Figure picture Figure caption Image caption Table sheet Table caption Table Title Equation formula

[0053] This module is based on an open source lightweight model for layout analysis in research report scenarios. It inputs the path of the image collection to be analyzed formed in the first step and the path of the model weight file to be loaded, and outputs all detected area coordinates and bounding boxes.

[0054] The layout restoration module locates each label according to its area coordinates; processes the area where the label category is text through the OCR component to obtain the content corresponding to the area of ​​the text; converts the area where the label category is table into Markdown format to obtain the content corresponding to the area of ​​the table; parses the area where the label category is picture through a multimodal model to obtain the content corresponding to the area of ​​the picture; converts the area where the label category is equation into Latex format to obtain the content corresponding to the area of ​​the equation.

[0055] The text segmentation module recursively generates a set of blocks equal to or less than a specified size by using a segmenter to segment the entire content of the document to be processed; re-merges any blocks smaller than the block size until the block size is reached; and re-attaches the non-whitespace character segmenter to the end of the block.

[0056] Text segmentation module: works by recursively segmenting the text until all generated chunks are equal to or smaller than the specified chunk size, with more acceleration:

[0057] a. Use segmenters that make as much semantic sense as possible to segment the text. To ensure that the chunks are as semantically meaningful as possible, use the following segmenters in the following order:

[0058] 1. Maximum sequence of newline characters (\n) and / or carriage return characters (\r);

[0059] 2. Tabulation of the maximum sequence;

[0060] 3. A maximum sequence of whitespace characters (defined by the \s character class of the regular expression);

[0061] 4. Sentence terminators (., ?, ! and *);

[0062] 5. Sentence separators (;,,,(,),[,], ",",',',','," and `);

[0063] b. Recursively split the generated blocks until a set of blocks equal to or smaller than the specified block size is generated;

[0064] c. Re-merge any chunks smaller than the chunk size until the chunk size is reached;

[0065] d. Reappend the non-whitespace separators to the end of the chunk (except the last chunk) if doing so would not make the chunk exceed the chunk size, otherwise add the non-whitespace separators as their own chunk.

[0066] Based on the same inventive concept, the present invention also provides a document data processing method based on a lightweight model. Figure 2 As shown, including:

[0067] Step S101, obtaining a document to be processed, and converting the document to be processed into an image file;

[0068] Step S102, processing the image file according to the pre-built layout analysis lightweight model, and extracting the category and region coordinates of each label in the image file;

[0069] Step S103, according to the category and area coordinates of each tag, the tag areas of different categories are parsed respectively to obtain the content corresponding to each tag area; by combining the content corresponding to each tag area, the entire content of the document to be processed is obtained;

[0070] Step S104, dividing the entire content of the document to be processed into text blocks of a preset size by a segmenter, thereby completing the format conversion and document data processing of the document to be processed.

[0071] Furthermore, the document to be processed is converted into an image file, including:

[0072] Convert the PDF document to be processed into multiple image files through the component pdfplumber.

[0073] Furthermore, the categories of each tag include:

[0074] Main text Text, title Title, picture Figure, picture caption Figure caption, table Table, table caption Table caption, formula Equation.

[0075] Furthermore, according to the category and area coordinates of each label, the label areas of different categories are parsed and processed respectively to obtain the content corresponding to each label area, including:

[0076] Position each label according to its regional coordinates;

[0077] The area of ​​the label whose category is text is processed by the OCR component to obtain the content corresponding to the area of ​​text;

[0078] Convert the area with the tag category of Table to Markdown format and obtain the content corresponding to the area of ​​Table;

[0079] The region with the label category of "Figure" is analyzed by the multimodal model to obtain the content corresponding to the region of "Figure";

[0080] Convert the area with the label category of Equation to Latex format and obtain the content corresponding to the area of ​​Equation.

[0081] Furthermore, the entire content of the document to be processed is divided into text blocks of a preset size by a segmenter to complete the format conversion and document data processing of the document to be processed, including:

[0082] Based on a recursive method, the entire content of the document to be processed is divided into a set of blocks equal to or smaller than a specified size by a splitter;

[0083] Any chunks smaller than the chunk size are re-merged until the chunk size is reached;

[0084] Reappend the non-whitespace character separator to the end of the block.

[0085] The present invention provides a document data processing system and method based on a lightweight model, which uses an open source lightweight model in the field of research reports to implement layout analysis, so as to improve the model's ability to understand the document structure and avoid the problem of traditional models that it is difficult to accurately distinguish paragraphs and other layout elements in the document. The method uses a semantically meaningful segmenter to recursively implement text segmentation. Although this method does not use advanced semantic segmentation, the key point is to accelerate document chunck segmentation. It effectively realizes the rapid recognition and separation of graphics, tables and text in PDF format documents such as research reports and financial reports.

[0086] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalents that do not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A document data processing system based on a lightweight model, characterized in that: include: Document conversion module, layout analysis module, layout restoration module and text segmentation module; A document conversion module, used to convert the document to be processed into an image file; A layout analysis module, used to process the image file according to a pre-built layout analysis lightweight model, and extract the category and region coordinates of each label in the image file; The layout restoration module is used to parse and process the label areas of different categories according to the category and area coordinates of each label, and obtain the content corresponding to each label area; By combining the contents corresponding to each label area, the entire contents of the document to be processed are obtained; The text segmentation module is used to segment the entire content of the document to be processed into text blocks of a preset size through a segmenter, thereby completing the format conversion and document data processing of the document to be processed.

2. The system according to claim 1, characterized in that The document conversion module is used to convert the document to be processed into an image file, including: Convert the PDF document to be processed into multiple image files through the component pdfplumber.

3. The system according to claim 1, characterized in that Categories for each tag, including: Main text Text, title Title, picture Figure, picture caption Figure caption, table Table, table caption Table caption, formula Equation.

4. The system according to claim 1, characterized in that The layout restoration module is used to parse and process the label areas of different categories according to the category and area coordinates of each label, and obtain the content corresponding to each label area, including: Position each label according to its regional coordinates; The area of ​​the label whose category is text is processed by the OCR component to obtain the content corresponding to the area of ​​text; Convert the area with the tag category of Table to Markdown format and obtain the content corresponding to the area of ​​Table; The region with the label category of "Figure" is analyzed by the multimodal model to obtain the content corresponding to the region of "Figure"; Convert the area with the label category of Equation to Latex format and obtain the content corresponding to the area of ​​Equation.

5. The system according to claim 1, characterized in that The text segmentation module is used to segment the entire content of the document to be processed into text blocks of a preset size through a segmenter, and complete the format conversion and document data processing of the document to be processed, including: Based on a recursive method, the entire content of the document to be processed is divided into a set of blocks equal to or smaller than a specified size by a splitter; Any chunks smaller than the chunk size are re-merged until the chunk size is reached; Reappend the non-whitespace character separator to the end of the block.

6. A document data processing method based on a lightweight model, characterized in that: include: Obtain the document to be processed, and convert the document to be processed into an image file; Processing the image file according to a pre-built layout analysis lightweight model to extract the category and region coordinates of each label in the image file; According to the category and area coordinates of each label, the label areas of different categories are parsed and processed respectively to obtain the content corresponding to each label area; By combining the contents corresponding to each label area, the entire contents of the document to be processed are obtained; The entire content of the document to be processed is divided into text blocks of a preset size by a splitter, thereby completing the format conversion and document data processing of the document to be processed.

7. The method according to claim 6, characterized in that Convert the documents to be processed into image files, including: Convert the PDF document to be processed into multiple image files through the component pdfplumber.

8. The method according to claim 6, characterized in that Categories for each tag, including: Main text Text, title Title, picture Figure, picture caption Figure caption, table Table, table caption Table caption, formula Equation.

9. The method according to claim 6, characterized in that According to the category and area coordinates of each label, the label areas of different categories are parsed and processed respectively to obtain the content corresponding to each label area, including: Position each label according to its regional coordinates; The area of ​​the label whose category is text is processed by the OCR component to obtain the content corresponding to the area of ​​text; Convert the area with the tag category of Table to Markdown format and obtain the content corresponding to the area of ​​Table; The region with the label category of "Figure" is analyzed by the multimodal model to obtain the content corresponding to the region of "Figure"; Convert the area with the label category of Equation to Latex format and obtain the content corresponding to the area of ​​Equation.

10. The method according to claim 6, characterized in that The entire content of the document to be processed is divided into text blocks of a preset size by a splitter to complete the format conversion and document data processing of the document to be processed, including: Based on a recursive method, the entire content of the document to be processed is divided into a set of blocks equal to or smaller than a specified size by a splitter; Any chunks smaller than the chunk size are re-merged until the chunk size is reached; Reappend the non-whitespace character separator to the end of the block.

Images