Structured extraction method and device for PDF document data, equipment and storage medium

Abstract

The invention relates to the technical field of data processing and the field of financial science and technology, and discloses a structured extraction method of PDF (Portable Document Format) document data, which is used for analyzing a PDF document and identifying a content type by configuring parameters of a coordinate system, and determining boundaries of text and table contents. And positioning the text content through a coordinate system, carrying out coherence check, ensuring the continuity and integrity of the text, and extracting the text content. For the table content, judging whether a table boundary exists or not, if so, identifying an intersection point of table lines, determining a cell position and extracting the content; and if the clear boundary does not exist, determining column and row segmentation lines of the table by using a coordinate overlap analysis method, and extracting the content of the boundless table. According to the method, the structured data in the financial report can be efficiently and accurately extracted, the defect of poor processing of complex tables in the prior art is overcome, and the method is suitable for processing the complex tables and text contents in the financial report.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method, device, equipment and storage medium for structured extraction of PDF document data. Background Art

[0002] In the modern financial and accounting fields, financial institutions and enterprises often rely on a large number of financial reports for business analysis, compliance review and decision making. These financial reports are stored and shared in the form of PDF documents, containing a large number of complex tables, charts and text content. These reports are often complex in structure, including a large number of text paragraphs, as well as a large number of nested tables and graphics, which are of great value for the financial analysis and report review of enterprises.

[0003] However, traditional PDF document processing methods face many challenges. In the financial and accounting fields, manually processing and extracting data from reports is time-consuming and error-prone. The unstructured data characteristics of the PDF format make it difficult to extract key information from documents, especially when processing financial reports containing large numbers of tables, text annotations, and complex multi-level indexes.

[0004] Most current financial and accounting systems rely on manual operations for data extraction and classification. Although there are some automated solutions on the market, such as extraction tools based on OCR (optical character recognition), these tools often perform poorly when dealing with complex tables, tables without clear boundaries, merged cells or cross-page tables, and cannot provide high-precision structured data extraction. Especially when it comes to multiple titles and multi-level indexes in financial reports, existing tools are difficult to accurately identify and process, which affects the efficiency and accuracy of data extraction.

[0005] In order to solve these problems, an efficient and automated technical solution is urgently needed, which can accurately identify and extract the text and table content in financial reports without relying on human intervention, and structure these data and provide them to subsequent analysis and audit systems. This can not only greatly improve the efficiency of financial data processing, but also reduce errors caused by manual operations and improve the overall level of financial data management. Summary of the invention

[0006] The main purpose of the present invention is to provide a structured extraction method, device, equipment and storage medium for PDF document data, aiming to solve the technical problem that the prior art cannot accurately identify and extract structured data when processing complex table structures and borderless table contents in financial reports, resulting in low data processing efficiency and prone to errors.

[0007] To achieve the above object, the present invention provides a structured extraction method for PDF document data, comprising:

[0008] Configure parameters of a coordinate system of a PDF document, parse the PDF document and identify content types, and determine the boundaries of the PDF document content, wherein the content types include text content and table content;

[0009] Locating the text content by using the coordinate system to determine the boundary coordinates of the text content;

[0010] Performing a coherence check to identify the continuity and integrity of the text content and performing a text content extraction operation;

[0011] Determine whether the table content has a table boundary;

[0012] If so, identify the intersection of the table lines, determine the location of each cell, and extract the text content of each cell from the table;

[0013] If not, the coordinate overlap analysis method is used to determine the position of the column and row dividing lines of the table content, and the position and boundary coordinates of the cell are determined based on the determined column and row dividing lines, and the text content of each cell is extracted from the table.

[0014] In one embodiment, a coherence check is performed to identify the continuity and integrity of the text content, and a text content extraction operation is performed, including:

[0015] Check the ending punctuation of the text content to determine the text type of the text content, which includes a complete sentence or paragraph and text to be verified;

[0016] If the text content is a complete sentence or paragraph, directly extract the text content;

[0017] If the text content is text to be verified, check whether the width of the text content is greater than a preset ratio threshold of the width of the PDF document;

[0018] If yes, merge the text content with the text content of the next line, update the boundary coordinates of the merged text content, and extract the merged text content;

[0019] If not, the text content is determined to be title content, the title content is extracted as an independent unit, and the boundary coordinates of the title content are retained.

[0020] In one embodiment, checking the ending punctuation mark of the text content to determine the text type of the text content includes:

[0021] According to the language setting of the PDF document, a corresponding punctuation mark library is set, wherein the punctuation mark library includes punctuation marks that serve as the end of a sentence or a paragraph;

[0022] Detecting the language used by the PDF document and determining a corresponding target punctuation library;

[0023] Match the ending punctuation marks of each line or paragraph to verify whether they belong to the target punctuation mark library;

[0024] If the match is successful, the text type of the text content is determined to be a complete sentence or paragraph;

[0025] If the match fails, the text type of the text content is determined to be text to be verified.

[0026] In one embodiment, the coordinate overlap analysis method is used to determine the position of the column and row dividing lines of the table content, including:

[0027] Determine the boundary coordinates of each table content, including left coordinates, right coordinates, top coordinates, and bottom coordinates;

[0028] Compare the left and right coordinates of the adjacent table contents. If there is overlap, determine that the adjacent table contents belong to the same column, group the adjacent table contents into one column, and determine the minimum left coordinate and the maximum right coordinate of each column.

[0029] Compare the upper coordinates and lower coordinates of the table contents in each column. If there is overlap, determine that the overlapping table contents belong to the same row and group them into the same row, and determine the minimum upper coordinate and maximum lower coordinate of each row.

[0030] The position of the column dividing line is determined according to the minimum left coordinate and the maximum right coordinate of the adjacent columns, and the position of the row dividing line is determined according to the minimum upper coordinate and the maximum lower coordinate of the adjacent rows.

[0031] In one embodiment, after extracting the text content of each cell from the table, the method further includes:

[0032] Identify and extract header and index information from tables;

[0033] Based on the text content of each cell, the default character is used as a separator to concatenate the index, title, and cell content to generate tabular data in a logical text format.

[0034] In one embodiment, generating tabular data in a logical text format includes:

[0035] Analyze the intersection of column and row dividers in the table content to determine whether there are multiple indexes or headers;

[0036] If there is a column splitter that does not intersect any row splitter, mark it as a merged column;

[0037] If there is a row splitter that does not intersect any column splitter, mark it as a merged row;

[0038] Determine the title and index with the intersection of the column dividing line and the row dividing line as the title and index of the bottom layer;

[0039] The hierarchical structure of multiple titles or indexes is processed in layers. If the coordinate range of a title or index contains the coordinate range of other titles or indexes, it will be regarded as the upper title or index, and the contained title or index will be regarded as the lower title or index;

[0040] Convert table contents with multiple headers into logical text format and generate logical text containing multi-level indexes.

[0041] In one embodiment, determining whether the table content has a table boundary includes:

[0042] Use image processing algorithms to identify horizontal and vertical lines in PDF documents;

[0043] If horizontal and vertical lines are identified and they form a closed geometric shape, then it is determined that a table boundary exists;

[0044] If multiple breakpoints are identified on the horizontal or vertical line, and the length of each breakpoint exceeds the first threshold, it is impossible to clearly determine that there is a table boundary;

[0045] When it is not possible to clearly identify the existence of table boundaries, the existence of table boundaries is determined based on content density analysis;

[0046] If, in a certain area of ​​the PDF document, the distribution density of characters or content blocks exceeds a second threshold, and the arrangement of the characters or content blocks has a feature of row and column alignment, it is determined that a table boundary exists in the area.

[0047] Furthermore, to achieve the above object, the present invention also provides a structured extraction device for PDF document data, comprising:

[0048] A content parsing module, used to configure the parameters of the coordinate system of the PDF document, parse the PDF document and identify the content type, and determine the boundary of the PDF document content, wherein the content type includes text content and table content;

[0049] A text positioning module, used to locate the text content through the coordinate system and determine the boundary coordinates of the text content;

[0050] A text extraction module, used to perform a coherence check to identify the continuity and integrity of the text content, and perform a text content extraction operation;

[0051] A table boundary judgment module is used to judge whether the table content has a table boundary;

[0052] The cell extraction module is used to identify the intersection of the table lines, determine the position of each cell, and extract the text content of each cell from the table;

[0053] The borderless table processing module is used to determine the position of the column and row dividing lines of the table content using the coordinate overlap analysis method, determine the position and boundary coordinates of the cell based on the determined column and row dividing lines, and extract the text content of each cell from the table.

[0054] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a structured extraction device for PDF document data, the structured extraction device for PDF document data comprising a memory, a processor, and a structured extraction program for PDF document data stored in the memory and executable on the processor, wherein the structured extraction program for PDF document data implements the steps of the structured extraction method for PDF document data as described above when executed by the processor.

[0055] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer storage medium, on which a structured extraction program for PDF document data is stored, and when the structured extraction program for PDF document data is executed by a processor, the steps of the structured extraction method for PDF document data as described above are implemented.

[0056] Beneficial effects: The present invention relates to a structured extraction method for PDF document data. By configuring the parameters of the coordinate system, the PDF document is parsed and the content type is identified, and the boundaries of the text and table content are determined. The text content is located by the coordinate system, and a coherence check is performed to ensure the continuity and integrity of the text, and the text content is extracted. For the table content, it is determined whether there is a table boundary. If so, the intersection of the table lines is identified, the cell position is determined and the content is extracted; if there is no clear boundary, the coordinate overlap analysis method is used to determine the column and row dividing lines of the table, and the content of the borderless table is extracted. The present invention can efficiently and accurately extract structured data in financial reports, overcomes the shortcomings of the prior art in poor processing of complex tables, and is suitable for processing complex tables and text content in financial reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0058] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for structured extraction of PDF document data according to the present invention;

[0059] Figure 2A schematic diagram of functional modules of a preferred embodiment of a structured extraction device for PDF document data of the present invention;

[0060] Figure 3 It is a schematic diagram of the structure of the device hardware operating environment involved in the embodiment of the device for structured extraction of PDF document data of the present invention. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0062] It should be noted that in the modern financial and accounting fields, financial institutions and enterprises usually rely on a large number of financial reports for business analysis, compliance review and decision making. These financial reports are stored and shared in the form of PDF documents, containing a large number of complex tables, charts and text content. These reports are often complex in structure, including a large number of text paragraphs, as well as a large number of nested tables and graphics. This information is of great value for the financial analysis and report review of enterprises.

[0063] However, traditional PDF document processing methods face many challenges. In the financial and accounting fields, manually processing and extracting data from reports is time-consuming and error-prone. The unstructured data characteristics of the PDF format make it difficult to extract key information from documents, especially when processing financial reports containing large numbers of tables, text annotations, and complex multi-level indexes.

[0064] Most current financial and accounting systems rely on manual operations for data extraction and classification. Although there are some automated solutions on the market, such as extraction tools based on OCR (optical character recognition), these tools often perform poorly when dealing with complex tables, tables without clear boundaries, merged cells or cross-page tables, and cannot provide high-precision structured data extraction. Especially when it comes to multiple titles and multi-level indexes in financial reports, existing tools are difficult to accurately identify and process, which affects the efficiency and accuracy of data extraction.

[0065] In order to solve these problems, an efficient and automated technical solution is urgently needed, which can accurately identify and extract the text and table content in financial reports without relying on human intervention, and structure these data and provide them to subsequent analysis and audit systems. This can not only greatly improve the efficiency of financial data processing, but also reduce errors caused by manual operations and improve the overall level of financial data management.

[0066] See also Figure 1 , Figure 1This is a flow chart of an embodiment of a structured extraction method for PDF document data provided by the present invention. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0067] like Figure 1 As shown, the structured extraction method of PDF document data proposed by the present invention includes the following steps:

[0068] S10, configuring parameters of a coordinate system of a PDF document, parsing the PDF document and identifying a content type, and determining a boundary of the PDF document content, wherein the content type includes text content and table content;

[0069] In this embodiment, configuring the coordinate system parameters of a PDF document refers to setting and adjusting the relevant parameters of the coordinate system used to parse the PDF document to ensure that the document content can be accurately located and identified. The configuration includes defining the units of the coordinate system (such as pixels, millimeters), adapting to PDF pages of different sizes, adjusting the rotation angle to cope with document layouts of different formats, etc. For example, for a financial report containing rotated pages, the system needs to adjust the coordinate system according to the rotation angle so that it can correctly parse the text and table content on the page.

[0070] Parsing a PDF document means analyzing the structure of a PDF file through an algorithm and extracting the text, images, tables, and other contents. Identifying content types means classifying different contents into text or table contents during the parsing process. For example, when parsing a financial statement, the system needs to identify the income statement as table content and the notes as text content.

[0071] Determining the boundaries of PDF document content means analyzing the coordinate information of each content block to determine its position and range on the page. This step ensures that the boundaries of text paragraphs, tables, images, etc. can be accurately identified. For example, in a financial report containing multiple tables, the system needs to determine the boundaries of each table through coordinate information for subsequent table data extraction.

[0072] In one embodiment, the system first configures the coordinate system parameters to adapt to the page by analyzing the page size and layout of the financial report. For annual financial reports that contain rotated pages, the system automatically detects the rotation angle of the page and adjusts the coordinate system. Next, the system parses the entire PDF document, identifies the text content and table content, identifies the income statement and expense statement as tables, and identifies the management comments as text. When determining the boundaries of each part, the system ensures that the boundaries of text paragraphs and tables are clearly defined by detecting the coordinates of the content blocks, for example, the top, bottom, left and right boundaries of each table are clearly demarcated to avoid confusion with adjacent content.

[0073] In other specific embodiments, the system is suitable for processing multilingual financial reports. When configuring the coordinate system, the system adjusts the coordinate system parameters according to the language settings and font size of the document. For example, for a financial report containing both Chinese and English, the system adjusts the coordinate accuracy for Chinese and English texts respectively to ensure the accuracy of content recognition. When parsing, the system is able to recognize and distinguish text content in different languages, and when determining boundaries, it takes into account the differences in the layout of multilingual texts to ensure that the boundaries of bilingual text paragraphs and tables are accurate.

[0074] Through the above steps, it is possible to accurately identify and extract text and table content in financial reports of different formats and languages, significantly improving the efficiency and accuracy of data processing. Especially when processing multi-page, multi-language financial reports, it can flexibly adapt to the various layouts and content complexities of the documents, ensuring that all key data can be extracted in a structured form, providing reliable data support for subsequent financial analysis and auditing.

[0075] S20, detecting the logical relationship between the data table in the SAS platform and the corresponding table in the Hive platform, and generating a table corresponding relationship mapping;

[0076] In this embodiment, the text content in the PDF document is accurately positioned using a configured coordinate system, and the boundary coordinates of each text content are determined. Through the coordinate system, the system can identify the exact location of the text content in the document (i.e., the coordinate points of the upper left corner and the lower right corner), thereby defining the boundaries of the text content. This technology is particularly important when processing financial reports containing a large number of text paragraphs, because the boundaries of these paragraphs must be accurately identified to ensure the integrity of the content and the consistency of the context. For example, when parsing the management discussion section in a financial report, the system uses the coordinate system to accurately determine the boundaries of each paragraph to avoid confusion between paragraphs.

[0077] In one specific implementation, the system first scans the pages of the PDF document through a configured coordinate system to identify the starting and ending positions of each text block. For a financial analysis report containing multiple paragraphs of text, the system detects the coordinates of the upper left corner and lower right corner of each paragraph of text and records them as the boundary coordinates of the paragraph of text. For example, in a quarterly financial report, the system identifies the starting coordinates of the "Market Environment Analysis" section as (100,500) and the ending coordinates as (400,700), which define the boundaries of the text block. In this way, the system can ensure that each text block is accurately positioned in the document and kept separate from adjacent content.

[0078] In another specific embodiment, for annual financial reports containing mixed text and images, the system uses a coordinate system to accurately locate each text paragraph. In this embodiment, the system first divides the page content into blocks, distinguishes between image and text areas, and then locates and determines the boundaries of each text area. For example, when the system parses a financial forecast section containing chart illustrations, the system will first locate the boundary coordinates of the chart, then identify the text portion adjacent to the chart, and record the boundary coordinates of these texts separately to ensure that there is no overlap or misalignment between the text and the chart. In this way, the system can handle documents with complex layouts and ensure that the boundaries of each text block are clear and separated from other content.

[0079] In other specific embodiments, when processing a cross-page financial report, the system uses a coordinate system to ensure the integrity of each paragraph of text, even if the text content spans multiple pages. In this embodiment, the system not only determines the boundary coordinates of each text block, but also merges the text blocks on different pages into a whole in the case of cross-page. For example, when processing the "Risk Management" chapter, if the chapter spans two pages, the system will first locate the text boundary coordinates on the first page, and then continue to locate the starting coordinates of the text on the second page, and merge the two parts to generate a whole boundary coordinate to ensure that the chapter can remain coherent after parsing.

[0080] By using the coordinate system to locate text content and determine its boundary coordinates, the system can accurately identify and extract text content in financial reports, ensuring the integrity and accuracy of the content regardless of whether the text content is in a single page or a cross-page layout. This not only improves the efficiency of document processing, but also reduces content confusion and data extraction errors caused by inaccurate text positioning, providing a more reliable basis for subsequent analysis of financial data.

[0081] S30, performing a coherence check to identify the continuity and integrity of the text content, and performing a text content extraction operation;

[0082] In this embodiment, before extracting the text content in the PDF document, the system first performs a coherence check on the text content to ensure the continuity and integrity of the text content. Coherence check means that the system uses an algorithm to identify whether the text is contextually continuous and has not been accidentally interrupted or misplaced. For example, when processing financial reports, the system needs to confirm that each paragraph of text is logically complete and is not interrupted by other content such as pictures and tables, so as to ensure that the extracted text can accurately reflect the original information of the report.

[0083] In one specific implementation, when processing the notes to the financial statements, the system first scans the entire text of the notes and detects the line spacing and paragraph identifiers (such as punctuation, indentation, etc.). Through the coherence check, the system can identify whether there is any text that is misplaced or broken due to typesetting or other factors. If the system detects that there is no period or other end symbol at the end of a line of text, and the beginning of the next line of text belongs to the same paragraph, the system merges the two lines of text to ensure the coherence of the content of the notes. After the coherence is confirmed, the system performs a text extraction operation to extract the complete note text for subsequent use.

[0084] In another specific embodiment, when analyzing the Management Discussion and Analysis (MD&A) section included in the annual financial report, the system first performs a coherence check on the text content. Since the MD&A section usually contains complex financial terms and analytical paragraphs, the system identifies the coherence between paragraphs by detecting the sentence structure of each paragraph. If it is detected that the beginning of a paragraph is directly connected to the end of the previous paragraph and the two paragraphs are logically coherent, the system determines that the two paragraphs may be incorrectly separated and merges them into one paragraph. After confirming the integrity of the text content, the system performs a text extraction operation to extract the text content of the entire MD&A section in a complete and coherent form.

[0085] In other specific embodiments, when processing financial reports containing chart explanations, the system first performs a coherence check on each explanatory text paragraph. For example, in a report, a certain paragraph of text explains the data in a chart on the previous page, but due to typesetting reasons, the text is separated between different pages or charts. The system determines whether these separations are reasonable by analyzing the context of the text content. If the system recognizes that these separations will destroy the coherence of the explanation, the relevant text will be reassembled into a complete paragraph. After the check is passed, the system performs a text extraction operation to extract the coherent explanatory text content and associate it with the corresponding chart data.

[0086] By performing coherence checks and identifying the continuity and integrity of text content, the system can ensure that the text content extracted from financial reports is logically coherent and complete. It effectively avoids content fragmentation or dislocation caused by typesetting or other document structure problems, improves the accuracy and reliability of text data extraction, and ensures the consistency and integrity of the data used in subsequent analysis.

[0087] S40, determining whether the table content has a table boundary;

[0088] In this embodiment, by analyzing the table content in the PDF document, it is determined whether the table has clear border lines (such as horizontal lines and vertical lines). Table borders are usually used to separate cell contents in a table, and clear borders help the system to accurately identify and extract data in the table. In financial reports, tables are usually used to display financial data, so determining whether a table border exists is a key step in extracting table data. For example, the system may determine the existence of a border by detecting whether there are continuous lines in the table. If a complete line frame is detected, it is determined that the table has a clear border.

[0089] In one specific implementation, when processing financial statements containing complex structures (such as cross-page tables), the system determines the table boundaries through multiple steps. First, the system detects the lines in the page. If only some of the lines exist, the system will use an algorithm to complete the possible boundary lines; secondly, the system checks whether the rows and columns of the table content are arranged regularly, and determines whether there are merged cells or cross-row and cross-column situations. If the system can determine the integrity of the table through these two steps, it is determined that the table has boundaries. When processing the comprehensive income statement distributed on multiple pages in the annual financial report, the system can accurately determine the table boundaries and complete data extraction by detecting and completing the boundary lines.

[0090] By judging whether there are table boundaries in the table content, the system can adapt to different forms of table structures, whether it is a table with clear boundaries or a table without boundaries or a complex structure, and can effectively identify and process it. It significantly improves the accuracy and adaptability of extracting table data from financial reports, ensuring that the extracted data can accurately reflect the financial information in the report, and providing reliable data support for subsequent financial analysis and data processing.

[0091] S50, if yes, identify the intersection of the table lines, determine the position of each cell, and extract the text content of each cell from the table;

[0092] In this embodiment, after determining that the table has clear boundaries, the system determines the position of each cell by identifying the intersections of horizontal and vertical lines in the table. The identification of intersections is a key step because each intersection represents one of the four corners of a cell in the table. By determining these intersections, the system can clearly identify the boundaries and positions of each cell, and then extract the corresponding text content from each cell. For example, in a balance sheet, the system first identifies all intersections in the table, then determines the range of each cell through these intersections, and extracts the financial data in each cell.

[0093] In one specific implementation, when processing a balance sheet containing multi-level classifications, the system first uses image processing technology to identify the horizontal and vertical lines of the table and find all intersections. The system uses these intersections to divide the position of the cells and records the boundary coordinates of each cell. For each identified cell, the system extracts the text content inside it one by one, such as asset category, amount, etc., and saves this data in a structured manner for subsequent analysis. For example, after the system identifies the intersection, it determines that the range of a certain cell is (100,200) to (300,400), and then extracts the text content "Accounts receivable: 5 million" in the cell.

[0094] In another specific embodiment, when the system processes the income statement of the annual financial report, it first detects the boundary lines of the table and identifies the intersections inside the table. Since the income statement may contain merged cells, the system pays special attention to the distance between these intersections and determines whether any cells are merged by calculation. When it is determined that certain cells are merged, the system adjusts the boundaries of these cells according to the intersection positions and treats the text content in these cells as a whole when extracting the content. For example, if the system detects that the range of the merged cell is larger than that of the ordinary cell, when extracting the content of the cell, the system will extract "Sales Revenue: 10 million" at one time instead of splitting it into multiple parts.

[0095] In other specific embodiments, when processing a cross-page financial statement, the system determines the location of the cell by identifying the intersection of the table lines on each page. For a cross-page table, the system first identifies the intersection points on each page, and then merges these cross-page intersection points through an algorithm to form a complete table structure. The system uses these merged intersection information to accurately locate the position of each cell and extract the text content. For example, when processing a cross-page "Accounts Payable Statement", the system identifies the intersection points on each page and then merges these intersection points to ensure that detailed information in the cross-page cells can be accurately extracted, such as "Supplier A: 2 million".

[0096] By identifying the intersection of table lines and determining the location of each cell, the system can efficiently and accurately extract data from tables, regardless of whether the table is regular, contains merged cells, or spans pages. This ensures the integrity and accuracy of data extraction, and can significantly improve the efficiency of data parsing, especially when dealing with complex financial statements, providing a reliable foundation for subsequent data analysis and processing.

[0097] S60, if not, use a coordinate overlap analysis method to determine the position of the column and row dividing lines of the table content, determine the position and boundary coordinates of the cell according to the determined column and row dividing lines, and extract the text content of each cell from the table.

[0098] In this embodiment, when no clear table boundaries are detected, a coordinate overlap analysis method is used to determine the column and row dividing lines of the table. Coordinate overlap analysis is to compare the coordinate positions of adjacent content blocks to determine whether they belong to the same column or the same row, and then infer the position of the column and row dividing lines. By determining the position of these dividing lines, the system can accurately divide the boundaries of each cell and extract the corresponding text content from it. For example, when processing a borderless financial table, the system will determine the boundaries of the columns and rows based on the horizontal and vertical overlapping relationship between the characters and the content blocks, thereby extracting the data in each cell.

[0099] In one specific implementation, when processing borderless tables in annual financial statements, the system first analyzes the left and right and top and bottom coordinates of each text content block, and determines the column and row dividing lines by calculating the horizontal and vertical overlaps of adjacent content blocks. For example, in a borderless "Cash Flow Statement", the system compares the upper and lower coordinates of adjacent rows to identify their overlapping areas, thereby determining the position of the row dividing lines. Subsequently, the system determines the position of the column dividing lines by comparing the left and right coordinates of adjacent columns. Through these analyses, the system can accurately divide each cell and extract text content such as "Cash inflow from operating activities: 20 million".

[0100] In another specific implementation, when processing financial tables with merged cells, the system uses a coordinate overlap analysis method to determine the position of the dividing line by analyzing the spatial relationship between content blocks. Since merged cells may make the dividing lines of certain columns or rows irregular, the system will perform a more detailed analysis of the coordinates of each cell to ensure that the boundaries of each cell are accurately divided. For example, when processing the "Balance Sheet", the system detects that multiple content blocks in a column have the same upper and lower coordinates, but different widths. The system determines that these content blocks belong to merged cells. On this basis, the system determines the dividing lines between columns and rows and extracts the complete content in the merged cell, such as "Total fixed assets: 10 million".

[0101] By using the coordinate overlap analysis method to determine the column and row dividing lines of the table content, the system can effectively handle tables without borders or unclear boundaries, accurately divide the position of each cell, and extract the text content therein. It improves the ability to handle complex or irregular tables, especially in financial statements, and can effectively deal with tables of various formats to ensure the integrity and accuracy of data extraction.

[0102] The present invention relates to a structured extraction method for PDF document data. By configuring the parameters of a coordinate system, the PDF document is parsed and the content type is identified, and the boundaries of text and table content are determined. The text content is located by the coordinate system, a coherence check is performed, the continuity and integrity of the text are ensured, and the text content is extracted. For the table content, it is determined whether there is a table boundary. If so, the intersection of the table lines is identified, the cell position is determined, and the content is extracted; if there is no clear boundary, the column and row dividing lines of the table are determined using a coordinate overlap analysis method, and the content of the borderless table is extracted. The present invention can efficiently and accurately extract structured data in financial reports, overcomes the shortcomings of the prior art in poor processing of complex tables, and is suitable for processing complex tables and text content in financial reports.

[0103] In one embodiment, the above S30 includes:

[0104] S301, checking the ending punctuation of the text content, and determining the text type of the text content, wherein the text type includes a complete sentence or paragraph and a text to be verified;

[0105] S302, if the text content is a complete sentence or paragraph, directly extract the text content;

[0106] S303, if the text content is text to be verified, checking whether the width of the text content is greater than a preset ratio threshold of the width of the PDF document;

[0107] S304, if yes, merge the text content with the text content of the next line, update the boundary coordinates of the merged text content, and extract the merged text content;

[0108] S305: If not, determine that the text content is title content, extract the title content as an independent unit, and retain the boundary coordinates of the title content.

[0109] In this embodiment, the text content in the PDF document is checked for coherence to ensure that the text is logically and physically continuous and complete. By checking the punctuation at the end of the text, the system determines whether the text is a complete sentence or paragraph, or a text that needs further verification. If the text is a complete sentence or paragraph, the system extracts it directly; if the text is to be verified, it will check the ratio of its width to the page width to determine whether it needs to be merged with the next line or processed as title content. This process ensures that the extracted text content is accurate and structured, and is particularly suitable for processing financial reports with complex formats.

[0110] In one specific implementation, when processing the Management Discussion and Analysis (MD&A) section in a quarterly financial report, the system first scans the punctuation at the end of each paragraph of text to determine the text type. For complete sentences or paragraphs, the system directly extracts the text content; for text paragraphs that do not end with punctuation, the system identifies whether their width is close to a certain proportion of the page width (such as 90%). If so, the system merges it with the next line of text and updates the boundary coordinates of the merged text to ensure the coherence and completeness of the extracted content. For example, when the system detects that the width of a paragraph is close to the page width, it merges the paragraph with the next line to ensure that the paragraph content is complete and correct.

[0111] In another specific embodiment, when analyzing the financial summary section of the annual financial report, the system also first checks the punctuation at the end of each paragraph of text to determine its completeness. For text content with a smaller width, the system will further determine whether it is a title. For example, after identifying a short text paragraph such as "Total Revenue", the system determines that it is title content based on its width and position, and extracts it as an independent unit while retaining its boundary coordinates so that the title can be accurately located in subsequent analysis. For example, the system recognizes that the width of the short sentence "Total Revenue" is only 50% of the page width. Based on its position and content characteristics, it is determined to be a title and extracted independently.

[0112] This embodiment checks the text content for coherence and determines the type of the text based on its width, punctuation marks and other features. The system can ensure that the extracted text content is complete and accurate when processing complex financial reports. It effectively avoids incoherent content caused by broken or misplaced text paragraphs, improves the quality of text extraction, and can better preserve the structure and content of the original document, especially when processing reports with complex layouts.

[0113] In one embodiment, in the above S301, checking the ending punctuation mark of the text content and determining the text type of the text content includes:

[0114] S3011, according to the language setting of the PDF document, setting a corresponding punctuation mark library, wherein the punctuation mark library includes punctuation marks that serve as the end of a sentence or paragraph;

[0115] S3012, detecting the language used by the PDF document and determining a corresponding target punctuation library;

[0116] S3013, matching the ending punctuation marks of each line or paragraph to verify whether they belong to the target punctuation mark library;

[0117] S3014, if the match is successful, the text type of the text content is determined to be a complete sentence or paragraph;

[0118] S3015: If the match is unsuccessful, the text type of the text content is determined to be text to be verified.

[0119] In this embodiment, according to the language setting of the PDF document, a corresponding punctuation library is established, which contains punctuation marks commonly used as the end of sentences or paragraphs in the language. Then, the system detects the language used in the document and selects the corresponding punctuation library. During the text content processing, the system matches the punctuation marks at the end of each line or paragraph to verify whether it belongs to the punctuation library. If the match is successful, the system identifies the text content as a complete sentence or paragraph; otherwise, it is identified as text to be verified. This method can accurately distinguish different text types in the document and ensure the coherence and integrity of the text content in subsequent processing.

[0120] In one specific implementation, when processing the English part of the annual financial report of a financial institution, the system first loads an extended English punctuation library according to the language setting of the document. The library includes not only conventional periods (.), question marks (?), and exclamation marks (!), but also some special punctuation combinations such as "?!". The system identifies the language type of the document through a language detection algorithm, and selects the most appropriate punctuation library according to the content context of the paragraph. The system then matches the punctuation at the end of each line or paragraph. If the match is successful, the system not only determines that the text is a complete sentence or paragraph, but also further analyzes whether the paragraph contains quotations or content enclosed in brackets. If the match is unsuccessful, the system marks the text as text to be verified, and automatically calls the natural language processing algorithm to try to reparse the text to determine whether there are any abnormalities in the grammatical structure. This implementation is particularly suitable for financial reports containing complex grammar and symbol usage, and can more comprehensively ensure the integrity and coherence of the text content.

[0121] In another specific embodiment, when processing multilingual (such as Chinese and English) financial reports, the system uses a dynamic punctuation library, which automatically expands or reduces the available punctuation set according to the language characteristics of the paragraph. For example, for Chinese paragraphs, the system adds Chinese colons (:) and semicolons (,) to the punctuation library, and for English paragraphs, semicolons (;) and hyphens (-). After the system detects the paragraph language in the document, it dynamically generates a punctuation library suitable for the paragraph and performs a matching check. If the match is successful, the system will not only identify the text as complete, but also identify the format of the paragraph (such as whether it is centered, bold, etc.) to determine whether the text may be a title or comment. If the match fails, the system further analyzes whether the paragraph is an annotation text with a table description or a picture description, and extracts it separately. This implementation method can more flexibly handle multilingual financial reports with complex formats and diverse contents, ensuring the integrity of the text and additional content.

[0122] In other specific embodiments, when processing chapters with chart annotations in financial statements, the system not only checks the ending punctuation of the text content, but also adds checks for punctuation inside the text. For example, the system will identify long sentences with multiple punctuation marks and determine whether they contain ellipsis (…) or quotation marks (""). The system associates these symbols with the context based on the semantic structure of the text to confirm the integrity and logical coherence of the text. In the case of a successful match, the system will also associate the text with its corresponding chart data to ensure the logical coherence between the annotations and the charts. If the match is unsuccessful, the system will further determine whether the paragraph belongs to a poorly segmented text block (such as being incorrectly separated during the page conversion process), and try to reassemble the text block, and finally perform content extraction. This implementation method is particularly suitable for financial reports containing a large number of charts and complex annotations, and can better ensure the relevance and accuracy of the annotation content and chart data.

[0123] By setting and using a multilingual punctuation library, this embodiment enables the system to accurately identify and process the end punctuation of texts in different languages ​​and determine the integrity of the text. This is particularly effective when processing complex financial reports in multiple languages, and can avoid text processing errors caused by punctuation mismatches, ensure accurate extraction and coherence of text content, and improve the reliability and accuracy of document parsing.

[0124] In one embodiment, in the above S60, using a coordinate overlap analysis method to determine the position of the column and row dividing lines of the table content includes:

[0125] S601, determining the boundary coordinates of each table content, including left coordinates, right coordinates, top coordinates and bottom coordinates;

[0126] S602, comparing the left coordinates and the right coordinates of the adjacent table contents, if there is overlap, determining that the adjacent table contents belong to the same column, grouping the adjacent table contents into one column, and determining the minimum left coordinate and the maximum right coordinate of each column;

[0127] S603, comparing the upper coordinates and lower coordinates of the table contents in each column, and if there is overlap, determining that the overlapping table contents belong to the same row and grouping them into the same row, and determining the minimum upper coordinate and the maximum lower coordinate of each row;

[0128] S604, determining the position of the column dividing line according to the minimum left coordinate and the maximum right coordinate of the adjacent columns, and determining the position of the row dividing line according to the minimum upper coordinate and the maximum lower coordinate of the adjacent rows.

[0129] In this embodiment, the method of coordinate overlap analysis is used to determine the position of the column and row dividing lines of the table content in the PDF document. Specifically, the system first determines the boundary coordinates (left coordinates, right coordinates, upper coordinates and lower coordinates) of each table content block, and then determines whether they belong to the same column by comparing the left and right coordinates of adjacent content blocks, and determines whether they belong to the same row by comparing the upper and lower coordinates in each column. Finally, the system determines the position of the column dividing lines and row dividing lines of the table based on this information. This process ensures that the cell positions in the table can be accurately divided without clear boundary lines.

[0130] In one specific implementation, the system first processes a borderless financial table, such as an income statement containing multi-level classifications. The system analyzes the boundary coordinates of each text content block and first determines the left and right coordinates. For adjacent content blocks, the system compares their left and right coordinates. If these coordinates overlap, it is determined that these content blocks belong to the same column. The system then groups these content blocks into a column and calculates the minimum left coordinate and the maximum right coordinate of the content blocks in the column. The system then determines whether there is overlap in each column by comparing the upper and lower coordinates. If there is overlap, it is determined that these content blocks belong to the same row and calculates the minimum upper coordinate and the maximum lower coordinate of the row. Finally, the system determines the dividing line between columns and rows based on these coordinate data to accurately divide the cell positions in the table.

[0131] In other specific embodiments, the system uses a more sophisticated coordinate overlap analysis method when processing complex tables containing merged cells. The system first determines the boundary coordinates of each content block, and determines the ownership of adjacent content blocks by calculating the coordinate overlap area. If the overlapping area between adjacent content blocks is large, the system determines that they may belong to the same column or row, and further analyzes their boundary coordinates. For merged cells, the system determines which cells need to be merged by analyzing the range of left and right and top and bottom coordinates. After determining the dividing lines between columns and rows, the system extracts the text content of merged cells and regular cells separately to ensure that all data can be accurately captured.

[0132] In this embodiment, by using the coordinate overlap analysis method, the system can accurately determine the position of the column and row dividing lines in the table in the absence of clear boundaries, which is particularly suitable for complex tables without borders or containing merged cells. It ensures that the position of each cell can be accurately divided, thereby achieving high-precision data extraction, and ensuring the integrity and accuracy of the data even when processing complex financial report tables.

[0133] In one embodiment, in the above S50 or S60, after extracting the text content of each cell from the table, the process further includes:

[0134] S701, identifying and extracting the title and index information in the table;

[0135] S702, based on the text content of each cell, using a preset character as a separator, connecting the index, the title and the cell content, and generating table data in a logical text format.

[0136] In this embodiment, after extracting the text content of each cell in the table, the system needs to perform subsequent processing steps. These steps include identifying and extracting the title and index information in the table, and converting the extracted text content into a logical text format. By using preset characters (such as @ or #) as delimiters, the index, title, and cell content are connected together to generate data in a logical text format that can be used for subsequent analysis. This process ensures structured processing of the table content, which is particularly suitable for financial scenarios where further analysis or calculation of the table content is required.

[0137] In one specific implementation, the system first extracts the text content of each cell in the annual financial statement. After the extraction is completed, the system identifies the title and index information at the top and left side of the table, such as identifying "asset category" as the title and "amount" as the column title. After identifying this information, the system uses the preset character "@" as a separator to connect the index, title and cell content in the table to generate data in a logical text format. For example, for a cell content "10 million", if its corresponding row index is "current assets" and the column title is "amount", the system will format it as "current assets@amount@10 million" and store or output the generated logical text for subsequent analysis.

[0138] In another specific embodiment, when processing a multi-level classified financial table (such as an income statement), the system identifies multiple titles and hierarchical indexes in the table after extracting the cell content. For example, the system identifies "main business income" as the first-level title and "domestic sales" as the second-level title. After extracting these titles, the system uses the separator "#" to connect the title, index and cell content, and generates data in a logical text format. For example, the system formats the cell content of "5 million" as "main business income#domestic sales#5 million", ensuring that the data can clearly reflect the hierarchical structure of the table and facilitate subsequent classification and analysis processing.

[0139] This embodiment further identifies and extracts the title and index information in the table after extracting the text content of each cell, and converts it into a logical text format, so that the system can perform more detailed structural processing on the table data. Not only does it retain the hierarchical structure of the table, but it also provides high-quality structured data for subsequent data analysis and calculation, especially when processing complex financial report tables, which significantly improves the readability of the data and the accuracy of the analysis.

[0140] In one embodiment, in the above S702, generating table data in a logical text format includes:

[0141] S7021, analyzing the intersection of the column and row dividing lines of the table content to determine whether there are multiple indexes or headers;

[0142] S7022, if there is a column dividing line that does not intersect with any row dividing line, mark it as a merged column;

[0143] S7023, if there is a row splitting line that does not intersect with any column splitting line, mark it as a merged row;

[0144] S7024, determine the title and index with the intersection of the column dividing line and the row dividing line as the title and index of the bottom layer;

[0145] S7025, hierarchical processing is performed on the hierarchical structure of multiple titles or indexes. If the coordinate range of a title or index contains the coordinate range of other titles or indexes, it is regarded as the upper title or index, and the contained title or index is regarded as the lower title or index;

[0146] S7026, convert the table content containing multiple titles into a logical text format, and generate a logical text containing a multi-level index.

[0147] In this embodiment, the system first determines whether the table contains multiple indexes or headers by analyzing the intersections of the table's column and row dividing lines. Then, the system identifies and marks possible merged columns or merged rows, determines the headers and indexes in the table, and divides them into different levels. Finally, the system associates these multiple headers and indexes through a logical text format to generate structured table data containing multi-level indexes. This process ensures that the complex structure of the table can be clearly and systematically converted into a logical text format that can be used for further analysis and processing.

[0148] In one specific embodiment, when processing an annual financial report table with multiple indexes, the intersections of the columns and row dividers of the table content are first analyzed to determine whether there are multiple indexes or titles. The system not only checks the existence of intersections, but also identifies possible multiple titles and indexes by measuring the distance between the intersections and analyzing the arrangement pattern of the intersections. For example, when the system finds that multiple intersections have the same horizontal distance and are arranged vertically, it may indicate that these columns belong to the same level of titles. The system marks these columns as the same title level and further layers them according to their contextual information. For columns that do not intersect with any row dividers, the system marks them as merged columns, and combines the content and format information of these columns to infer their actual meaning (such as summary data in merged columns). The system finally converts this information into logical text format data, uses preset characters to connect titles, indexes, and content at different levels, and generates structured output data.

[0149] In another specific embodiment, when processing financial statements containing nested structures, not only the intersections of the columns and row dividing lines of the table are analyzed, but also the style and font of the table content are detected. For example, the system identifies titles in the table that use bold fonts, and combines these bold titles with the intersection information to determine whether there are multiple indexes or nested titles. For the detected merged columns, the system further analyzes the alignment of the content in these columns to distinguish different levels of titles in the same column. After integrating this information, the system generates a data structure in a logical text format in hierarchical order. For example, for a table with a nested structure, the system may generate structured data such as "Total Assets @ Current Assets @ Cash and Cash Equivalents @ 2023 @ 10 million", in which the titles and indexes of each level are clearly distinguished and structured.

[0150] In other specific embodiments, when processing multi-page or cross-page financial statements, the system not only relies on the intersection of column and row dividing lines to determine multiple titles or indexes, but also analyzes the coherence of cross-page content. The system first identifies the intersection of column and row dividing lines on each page, and then determines whether there is a title or index that continues to the next page by comparing the content and layout of the cross-page. For those columns that do not intersect with any row dividing lines, the system not only marks them as merged columns, but also considers whether the content of these columns is coherent with the content of the previous page when crossing pages. When processing row dividing lines, the system also determines whether the cross-page rows belong to the same row visually or semantically. When generating data in a logical text format, the system structures the cross-page data and uses specific characters to represent the cross-page relationship. For example, the generated data may be "2022|Profit and Loss Statement|Main Business Income|Domestic Sales|Spanning Pages@30 Million" to clearly represent the cross-page relationship and hierarchical structure.

[0151] Through the above steps, this embodiment ensures that the system can accurately identify and structure multiple titles and indexes when processing different types of financial statements, and generate data in a logical text format, thereby improving data availability and analysis efficiency. Whether processing nested tables, merged columns, or cross-page content, the system can respond flexibly and generate high-quality structured data to meet the needs of complex financial scenarios.

[0152] In one embodiment, the above S40 includes:

[0153] S401, identifying horizontal lines and vertical lines in the PDF document using an image processing algorithm;

[0154] S402, if horizontal lines and vertical lines are identified, and the horizontal lines and vertical lines form a closed geometric shape, it is determined that a table boundary exists;

[0155] S403: If it is identified that there are multiple breakpoints on the horizontal line or the vertical line, and the length of each breakpoint exceeds the first threshold, it is impossible to clearly determine that there is a table boundary;

[0156] S404, when it is not possible to clearly determine whether there is a table boundary, determine whether there is a table boundary based on content density analysis;

[0157] S405: If, in a certain area of ​​the PDF document, the distribution density of characters or content blocks exceeds a second threshold, and the arrangement of characters or content blocks has a feature of row-column alignment, it is determined that a table boundary exists in the area.

[0158] In this embodiment, first, the system uses an image processing algorithm to identify horizontal and vertical lines in the table. If these lines form a closed geometric shape (such as a rectangle), the system determines that the table has a boundary. If there are multiple breakpoints in the horizontal or vertical line and the length of each breakpoint exceeds a preset threshold, the system cannot clearly determine that the table has a boundary. In the case where the table boundary cannot be clearly determined, the system will further analyze the density of the content to determine whether the arrangement of characters or content blocks has the characteristics of row and column alignment, so as to determine whether there is a table boundary in the area.

[0159] In one specific embodiment, when processing financial statements with partially broken borders, the system first uses an image processing algorithm to identify horizontal and vertical lines in a PDF document. The system not only identifies the existence of these lines, but also determines the integrity of the boundaries by analyzing the continuity of each line segment and the distribution of the break points. The system finds that some horizontal and vertical lines are broken, but the length of the break points is less than the first threshold, so the system attempts to automatically connect these broken line segments to reconstruct the complete table boundary. If the connected lines form a closed rectangle or other geometric shape, the system determines that the table has a boundary. If the reconstruction fails, the system further determines the existence of the table through density analysis. This method expands the basic image processing method and improves the recognition ability of irregular tables by automatically repairing broken boundaries.

[0160] In another specific embodiment, when processing an annual report containing nested tables, the system uses an image processing algorithm to identify horizontal and vertical lines, and further analyzes the arrangement and nesting relationship of these lines. The system detects that multiple table boundaries overlap or intersect to form a nested structure. In this case, the system first identifies the boundaries of each nested table through an image processing algorithm, and analyzes its internal structure layer by layer. If the boundaries of the nested table are incomplete, the system will use a recursive density analysis method to perform density detection on each nested layer one by one, and determine whether the distribution of characters conforms to the row and column alignment characteristics of the table. After the system identifies the complete structure of the nested table, it extracts it in layers and generates logical text format data. This method expands the processing capabilities of complex table structures, especially for the accurate identification and extraction of multi-layer nested tables.

[0161] In other specific embodiments, when processing complex financial statements with charts and annotations, the system first uses image processing algorithms to identify horizontal and vertical lines, but due to the presence of charts and annotations, the boundaries of some tables may be obscured or interfered with. The system not only identifies conventional horizontal and vertical lines, but also determines whether these areas are part of the table by analyzing the boundaries of charts and annotation areas. If the system detects an obstructed or interfered area, it infers the possible table boundaries by calculating the content density and row and column alignment features of the obstructed area. For example, if it is detected that the character density and arrangement of a certain area conform to the row and column alignment mode of the table, the system marks it as a table area and attempts to restore the obstructed boundary lines. This implementation method effectively expands the system's ability to recognize table boundaries in complex scenarios by combining image processing and density analysis technology, especially when charts, annotations and tables are mixed, and the table content can still be accurately extracted.

[0162] Through the above steps, this embodiment can flexibly deal with bordered and unbordered tables when processing financial statements of various complex formats. Even when the table boundaries are not obvious or there are breaks, the system can still accurately identify and infer the existence and scope of the table through technologies such as content density analysis, thereby ensuring the integrity and accuracy of data extraction. The adaptability of the system in various application scenarios is significantly improved, especially when processing financial statements with diverse formats and high complexity, it can provide efficient and reliable table data extraction and structuring capabilities.

[0163] The present invention also provides a structured extraction device for PDF document data, referring to Figure 2 , Figure 2 The functional module diagram of a preferred embodiment of a structured extraction device for PDF document data of the present invention is shown in FIG. The structured extraction device for PDF document data comprises:

[0164] A content parsing module 10, configured to configure the parameters of the coordinate system of the PDF document, parse the PDF document and identify the content type, and determine the boundary of the PDF document content, wherein the content type includes text content and table content;

[0165] A text positioning module 20, used to locate the text content through the coordinate system and determine the boundary coordinates of the text content;

[0166] A text extraction module 30, for performing a coherence check to identify the continuity and integrity of the text content, and performing a text content extraction operation;

[0167] A table boundary determination module 40, used to determine whether the table content has a table boundary;

[0168] A cell extraction module 50 is used to identify the intersection of the table lines, determine the position of each cell, and extract the text content of each cell from the table;

[0169] The borderless table processing module 60 is used to determine the position of the column and row dividing lines of the table content using a coordinate overlap analysis method, determine the position and boundary coordinates of the cell based on the determined column and row dividing lines, and extract the text content of each cell from the table.

[0170] The specific implementation of the structured extraction device for PDF document data of the present invention is basically the same as the embodiments of the structured extraction method for PDF document data described above, and will not be described in detail herein.

[0171] The present invention also provides a structured extraction device for PDF document data, such as Figure 3 As shown, the structured extraction device of the PDF document data may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0172] Those skilled in the art will understand that Figure 3 The hardware structure of the device for extracting structured PDF document data shown in the figure does not constitute a limitation on the device for extracting structured PDF document data, and may include more or less components than those shown in the figure, or combine some components, or arrange the components differently.

[0173] like Figure 3 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a structured extraction program for PDF document data. The operating system is a program for managing and controlling the structured extraction device and software resources for PDF document data, and supports the operation of the network communication module, the user interface module, the structured extraction program for PDF document data, and other programs or software; the network communication module is used to manage and control the network interface 1004; and the user interface module is used to manage and control the user interface 1003.

[0174] exist Figure 3 In the hardware structure of the structured extraction device for PDF document data shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; the processor 1001 can call the structured extraction program of PDF document data stored in the memory 1005, and perform the same operation as the structured extraction method of PDF document data.

[0175] The specific implementation of the structured extraction device for PDF document data of the present invention is substantially the same as the embodiments of the structured extraction method for PDF document data described above, and will not be described in detail herein.

[0176] In addition, an embodiment of the present invention further provides a computer storage medium, on which a structured extraction program for PDF document data is stored. When the structured extraction program for PDF document data is executed by a processor, the steps of the structured extraction method for PDF document data as described above are implemented.

[0177] The specific implementation of the computer storage medium of the present invention is basically the same as the embodiments of the structured extraction method of PDF document data described above, and will not be described in detail here.

[0178] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.

[0179] It should be noted that if software tools or components other than those of our company appear in the embodiments of the present application, they are only used for illustrative purposes and do not represent actual use.

Claims

1. A structured extraction method for PDF document data, characterized in that: The following steps are involved: Configure parameters of a coordinate system of a PDF document, parse the PDF document and identify content types, and determine the boundaries of the PDF document content, wherein the content types include text content and table content; Locating the text content by using the coordinate system to determine the boundary coordinates of the text content; Performing a coherence check to identify the continuity and integrity of the text content and performing a text content extraction operation; Determine whether the table content has a table boundary; If so, identify the intersection of the table lines, determine the location of each cell, and extract the text content of each cell from the table; If not, the coordinate overlap analysis method is used to determine the position of the column and row dividing lines of the table content, and the position and boundary coordinates of the cell are determined based on the determined column and row dividing lines, and the text content of each cell is extracted from the table.

2. The structured extraction method of PDF document data according to claim 1, characterized in that: A coherence check is performed to identify the continuity and integrity of the text content, and a text content extraction operation is performed, including: Check the ending punctuation of the text content to determine the text type of the text content, which includes a complete sentence or paragraph and text to be verified; If the text content is a complete sentence or paragraph, directly extract the text content; If the text content is text to be verified, check whether the width of the text content is greater than a preset ratio threshold of the width of the PDF document; If yes, merge the text content with the text content of the next line, update the boundary coordinates of the merged text content, and extract the merged text content; If not, the text content is determined to be title content, the title content is extracted as an independent unit, and the boundary coordinates of the title content are retained.

3. The structured extraction method of PDF document data according to claim 2, characterized in that: Check the ending punctuation of the text content to determine the text type of the text content, including: According to the language setting of the PDF document, a corresponding punctuation mark library is set, wherein the punctuation mark library includes punctuation marks that serve as the end of a sentence or a paragraph; Detecting the language used by the PDF document and determining a corresponding target punctuation library; Match the ending punctuation marks of each line or paragraph to verify whether they belong to the target punctuation mark library; If the match is successful, the text type of the text content is determined to be a complete sentence or paragraph; If the match fails, the text type of the text content is determined to be text to be verified.

4. The structured extraction method of PDF document data according to claim 1, characterized in that: Use the coordinate overlap analysis method to determine the location of the column and row separators of the table content, including: Determine the boundary coordinates of each table content, including left coordinates, right coordinates, top coordinates, and bottom coordinates; Compare the left and right coordinates of the adjacent table contents. If there is overlap, determine that the adjacent table contents belong to the same column, group the adjacent table contents into one column, and determine the minimum left coordinate and the maximum right coordinate of each column. Compare the upper coordinates and lower coordinates of the table contents in each column. If there is overlap, determine that the overlapping table contents belong to the same row and group them into the same row, and determine the minimum upper coordinate and maximum lower coordinate of each row. The position of the column dividing line is determined according to the minimum left coordinate and the maximum right coordinate of the adjacent columns, and the position of the row dividing line is determined according to the minimum upper coordinate and the maximum lower coordinate of the adjacent rows.

5. The structured extraction method of PDF document data according to claim 1, characterized in that: After extracting the text content of each cell from the table, also include: Identify and extract header and index information from tables; Based on the text content of each cell, the default character is used as a separator to concatenate the index, title, and cell content to generate tabular data in a logical text format.

6. The structured extraction method of PDF document data according to claim 5, characterized in that: Generates tabular data in logical text format, including: Analyze the intersection of column and row dividers in the table content to determine whether there are multiple indexes or headers; If there is a column splitter that does not intersect any row splitter, mark it as a merged column; If there is a row splitter that does not intersect any column splitter, mark it as a merged row; Determine the title and index with the intersection of the column dividing line and the row dividing line as the title and index of the bottom layer; The hierarchical structure of multiple titles or indexes is processed in layers. If the coordinate range of a title or index contains the coordinate range of other titles or indexes, it will be regarded as the upper title or index, and the contained title or index will be regarded as the lower title or index; Convert table contents with multiple headers into logical text format and generate logical text containing multi-level indexes.

7. The structured extraction method of PDF document data as claimed in claim 1, characterized in that: Determining whether the table content has a table boundary includes: Use image processing algorithms to identify horizontal and vertical lines in PDF documents; If horizontal and vertical lines are identified and they form a closed geometric shape, then it is determined that a table boundary exists; If multiple breakpoints are identified on the horizontal or vertical line, and the length of each breakpoint exceeds the first threshold, it is impossible to clearly determine that there is a table boundary; When it is not possible to clearly identify the existence of table boundaries, the existence of table boundaries is determined based on content density analysis; If, in a certain area of ​​the PDF document, the distribution density of characters or content blocks exceeds a second threshold, and the arrangement of the characters or content blocks has a feature of row and column alignment, it is determined that a table boundary exists in the area.

8. A structured extraction device for PDF document data, characterized in that: The structured extraction device of the PDF document data comprises: A content parsing module, used to configure the parameters of the coordinate system of the PDF document, parse the PDF document and identify the content type, and determine the boundary of the PDF document content, wherein the content type includes text content and table content; A text positioning module, used to locate the text content through the coordinate system and determine the boundary coordinates of the text content; A text extraction module, used to perform a coherence check to identify the continuity and integrity of the text content, and perform a text content extraction operation; A table boundary judgment module is used to judge whether the table content has a table boundary; The cell extraction module is used to identify the intersection of the table lines, determine the position of each cell, and extract the text content of each cell from the table; The borderless table processing module is used to determine the position of the column and row dividing lines of the table content using the coordinate overlap analysis method, determine the position and boundary coordinates of the cell based on the determined column and row dividing lines, and extract the text content of each cell from the table.

9. A structured extraction device for PDF document data, characterized in that: The structured extraction device for PDF document data includes a memory, a processor, and a structured extraction program for PDF document data stored in the memory and executable on the processor. When the structured extraction program for PDF document data is executed by the processor, the steps of the structured extraction method for PDF document data according to any one of claims 1 to 7 are implemented.

10. A computer storage medium, characterized in that: The storage medium stores a structured extraction program for PDF document data, and when the structured extraction program for PDF document data is executed by a processor, the steps of the structured extraction method for PDF document data according to any one of claims 1 to 7 are implemented.

Images