Method for extracting and processing text data of PDF (Portable Document Format) document

Abstract

The invention discloses a method for extracting and processing text data of a PDF (Portable Document Format) document, which comprises the following steps: word processing: dividing the data content of a word object in the PDF document into three attributes of characters, paper coordinates and printing size, and carrying out differentiated processing according to the three attributes so as to obtain standard data of the word in an article; extracting character objects from the PDF document, extracting characters with coordinates in a table range region, and storing the characters into a character object set in sequence; table processing is carried out, data content of a table in the PDF document is divided into table structure objects, paper coordinates and printing sizes, and the table structure objects are divided into cell characters, cell sizes and row and column positions; extracting table objects in each page, and storing the table objects in a table object set in sequence; the problems that in the prior art, tables and characters in bidding document PDF documents are not accurately extracted, and the tables cannot be accurately recognized during page crossing are solved, common announcements, papers and scheme type PDF documents can be processed, and accurate article structure data can be restored.

Description

Technical Field

[0001] The invention belongs to the technical field of table content extraction, and in particular relates to a method for extracting and processing text data from a PDF document. Background Art

[0002] In current bidding activities, some bidders directly modify and write other bids in the process of preparing bids, which may lead to great similarity. Such illegal operations not only evade routine electronic information detection, but also make it difficult for judges to accurately find problems due to a large number of bid reviews. Moreover, the data storage structure of the PDF document content of the bid does not have article formats such as text, paragraphs, titles, and tables, resulting in the extracted text being inconsistent with the text specifications when writing the article. Due to the paper size limit, text content including text and tables will be split across pages. Therefore, it is necessary to design a method for extracting and processing text data from PDF documents to solve the above problems. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method for extracting and processing text data from PDF documents, which is used to solve the problems of inaccurate extraction of tables and texts in tender PDF documents in the prior art, and inability to accurately identify tables when they cross pages. The method has the characteristics of being able to process common announcements, papers, and scheme-type PDF documents and restore accurate article structure data.

[0004] In order to achieve the above technical effects, the technical solution adopted by the present invention is: A method for extracting and processing text data from a PDF document comprises the following steps: S1, perform text processing, divide the data content of the text object in the PDF document into three attributes: character, paper coordinates and print size, and perform differentiated processing according to the three attributes to obtain the standard data of the text in the article; Use open source components PDFBox and IText to extract text objects from PDF documents, extract the text in the table range, and store them in the text object collection in order; S2, performing table processing: dividing the data content of the table in the PDF document into a table structure object, paper coordinates and print size, and the table structure object is divided into cell text, cell size and row and column positions; The open source component Tabula is used to extract the table objects in each page and store them in the table object collection in sequence.

[0005] Preferably, in step S1, the specific method for performing text processing is: S101, division of regions: Take the upper left corner of the paper as the origin of the coordinate system, extend to the right as the X axis, and extend downward as the Y axis to construct the paper coordinate area; Divide the text entered in the standard article format into header, body and footer areas in the paper coordinate area; The positions of regions with different coordinate ranges are preset according to the paper size, so as to identify whether the text is located in the header or footer area or the body area; S102, merge text paragraphs: The selected text area is traversed in order, and the line break behavior is determined based on whether the Y-axis coordinates of two consecutive characters exceed one character height. If they exceed, line break is performed; After the line break, the paragraph is determined based on whether the X-axis coordinate of the first character is more than one character width away from the leftmost end of the text area. If so, the paragraph is ended. S103, identify the title: The title and body text styles are distinguished by the length and width of the text's own printing area and the character set. The size of the paper printing area occupied by the text of each character set in different font styles is pre-enumerated and counted. The character set and the length and width of the printing area are compared with the text extracted from PDF. The matching character set and the length and width of the printing area are used as style attributes to identify the title and title level.

[0006] Preferably, in step S2, the table processing includes merging tables across pages, and the specific method is: Traverse the table object collection in sequence according to the page number. When the table page number changes, determine whether the table spans pages based on the last and first table objects of two consecutive pages. The determination method is: Determine whether the maximum Y-axis coordinate of the bottom of the table on the previous page is less than one row height from the bottom of the paper body area; Determine whether the minimum Y-axis coordinate of the top of the table on the next page is less than one row height from the top of the paper body area; Determine whether the width of each column of the two tables is the same; If all three judgments are yes, it means that the tables on the two pages belong to the same table, and the partial tables separated by the cross-page spread are merged into a complete table.

[0007] The beneficial effects of the present invention are as follows: Through rigorous business logic design and processing, the present invention can process and restore accurate article structure data for common announcements, papers, and scheme-type PDF documents, and is widely used in content duplication checking, difference comparison, semantic analysis, and intelligent filling through information system integration. It solves the problems of inaccurate extraction of tables and texts in tender PDF documents in the prior art, and inability to accurately identify tables when they cross pages. It has the characteristics of being able to process common announcements, papers, and scheme-type PDF documents and restore accurate article structure data. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 It is a flowchart of this application. DETAILED DESCRIPTION

[0009] Embodiment 1: like Figure 1 As shown, a method for extracting and processing text data from a PDF document comprises the following steps: S1, perform text processing, divide the data content of the text object in the PDF document into three attributes: character, paper coordinates and print size, and perform differentiated processing according to the three attributes to obtain the standard data of the text in the article; Use open source components PDFBox and IText to extract text objects from PDF documents, extract the text in the table range, and store them in the text object collection in order; S2, performing table processing: dividing the data content of the table in the PDF document into a table structure object, paper coordinates and print size, and the table structure object is divided into cell text, cell size and row and column positions; The open source component Tabula is used to extract the table objects in each page and store them in the table object collection in sequence.

[0010] Preferably, in step S1, the specific method for performing text processing is: S101, division of regions: Take the upper left corner of the paper as the origin of the coordinate system, extend to the right as the X axis, and extend downward as the Y axis to construct the paper coordinate area; Divide the text entered in the standard article format into header, body and footer areas in the paper coordinate area; The positions of regions with different coordinate ranges are preset according to the paper size, so as to identify whether the text is located in the header or footer area or the body area; S102, merge text paragraphs: The selected text area is traversed in order, and the line break behavior is determined based on whether the Y-axis coordinates of two consecutive characters exceed one character height. If they exceed, line break is performed; After the line break, the paragraph is determined based on whether the X-axis coordinate of the first character is more than one character width away from the leftmost end of the text area. If so, the paragraph is ended. S103, identify the title: The title and body text styles are distinguished by the length and width of the text's own printing area and the character set. The size of the paper printing area occupied by the text of each character set in different font styles is pre-enumerated and counted. The character set and the length and width of the printing area are compared with the text extracted from PDF. The matching character set and the length and width of the printing area are used as style attributes to identify the title and title level.

[0011] Preferably, in step S2, the table processing includes merging tables across pages.

[0012] Embodiment 2: The method for merging tables across pages is: Traverse the table object collection in sequence according to the page number. When the table page number changes, determine whether the table spans pages based on the last and first table objects of two consecutive pages. The determination method is: Determine whether the maximum Y-axis coordinate of the bottom of the table on the previous page is less than one row height from the bottom of the paper body area; Determine whether the minimum Y-axis coordinate of the top of the table on the next page is less than one row height from the top of the paper body area; Determine whether the width of each column of the two tables is the same; If all three judgments are yes, it means that the tables on the two pages belong to the same table, and the partial tables separated by the cross-page spread are merged into a complete table.

[0013] Apply to use open source components to extract text, tables, pictures and other data content from PDF documents, optimize the data through custom algorithm combinations, splice single words into paragraphs and titles of the article, merge table pages that appear due to paper size limitations, and restore the true and complete structure of the input article.

Claims

1. A method for extracting and processing text data from a PDF document, characterized in that: The following steps are involved: S1, perform text processing, divide the data content of the text object in the PDF document into three attributes: character, paper coordinates and print size, and perform differentiated processing according to the three attributes to obtain the standard data of the text in the article; Extract text objects from the PDF document, extract the text whose coordinates are in the table range, and store them in the text object set in order; S2, performing table processing, dividing the data content of the table in the PDF document into a table structure object, paper coordinates and print size, and the table structure object is divided into cell text, cell size and row and column positions; Extract the table objects in each page and store them into the table object collection in order.

2. A method for extracting and processing text data from a PDF document according to claim 1, characterized in that: In step S1, the specific method for text processing is: S101, division of regions: Take the upper left corner of the paper as the origin of the coordinate system, extend to the right as the X axis, and extend downward as the Y axis to construct the paper coordinate area; Divide the text entered in the standard article format into header, body and footer areas in the paper coordinate area; The positions of regions with different coordinate ranges are preset according to the paper size, so as to identify whether the text is located in the header or footer area or the body area; S102, merge text paragraphs: The selected text area is traversed in order, and the line break behavior is determined based on whether the Y-axis coordinates of two consecutive characters exceed one character height. If they exceed, line break is performed; After the line break, the paragraph is determined based on whether the X-axis coordinate of the first character is more than one character width away from the leftmost end of the text area. If so, the paragraph is ended. S103, identify the title: The title and body text styles are distinguished by the length and width of the text's own printing area and the character set. The size of the paper printing area occupied by the text of each character set in different font styles is pre-enumerated and counted. The character set and the length and width of the printing area are compared with the text extracted from PDF. The matching character set and the length and width of the printing area are used as style attributes to identify the title and title level.

3. A method for extracting and processing text data from a PDF document according to claim 2, characterized in that: In step S2, table processing includes merging tables across pages, and the specific method is as follows: Traverse the table object collection in sequence according to the page number. When the table page number changes, determine whether the table spans pages based on the last and first table objects of two consecutive pages. The determination method is: Determine whether the maximum Y-axis coordinate of the bottom of the table on the previous page is less than one row height from the bottom of the paper body area; Determine whether the minimum Y-axis coordinate of the top of the table on the next page is less than one row height from the top of the paper body area; Determine whether the width of each column of the two tables is the same; If all three judgments are yes, it means that the tables on the two pages belong to the same table, and the partial tables separated by the cross-page spread are merged into a complete table.

Images