PDF (Portable Document Format) document content extraction method and device

Abstract

The invention provides a PDF (Portable Document Format) document content extraction method and device. The method comprises the steps of determining whether a PDF document is an image-based PDF document or not; under the condition that the PDF document is an image-based PDF document, converting the PDF document into a target image; the target image is cut according to a target object in the target image obtained through recognition, a cut image only containing the target object is obtained, blank filling is conducted on an area corresponding to the target object of the cut target image, a filled target image is obtained, and the target object is at least one of a table and an embedded picture; performing analysis processing on the cut image based on a deep learning model to obtain structured information corresponding to the cut image; recognizing the filling target image based on an optical character recognition technology to obtain text information corresponding to the filling target image; and integrating the structured information and the text information to obtain page content information corresponding to the PDF document.

Description

Technical Field

[0001] This application relates to the field of document processing technology, and in particular to a method and apparatus for extracting the content of a PDF document. Background Technology

[0002] Currently, AI (Artificial Intelligence) has become an indispensable and important assistant, serving as a convenient channel for knowledge acquisition in both work and life. However, acquiring knowledge through AI often faces challenges such as inaccurate content and a lack of niche knowledge. To enable employees to quickly access accurate internal company knowledge, such as company regulations and technical parameters, establishing a corporate knowledge base is crucial. By uploading processed internal knowledge documents and leveraging the semantic understanding and retrieval capabilities of AI models to quickly find and answer the required questions in the knowledge base, a private knowledge assistant for the enterprise can be built.

[0003] When building enterprise knowledge management and intelligent assistant applications, constructing an efficient knowledge base system requires processing massive amounts of technical documents, of which PDF (Portable Document Format) files account for a significant proportion. These PDF documents typically contain important information such as technical specifications, product manuals, and research reports, and are one of the core data sources for enterprise knowledge bases. However, PDF documents have a complex structure, mainly divided into two categories: text-based PDFs and image-based PDFs.

[0004] For text-based PDFs, the content can be directly extracted and used for model training or question-answering systems. However, image-based PDFs (such as scanned documents or documents in certain special formats) cannot have their text extracted directly and must rely on OCR (Optical Character Recognition) technology for recognition. Furthermore, tabular data in PDF documents is particularly difficult to process because tables may be embedded as images, and traditional text extraction methods cannot correctly parse their structure and semantic content. Summary of the Invention

[0005] This application provides a method and apparatus for extracting PDF document content, enabling efficient and accurate extraction of document content from image-based PDF documents.

[0006] In a first aspect, embodiments of this application provide a method for extracting content from a PDF document, the method comprising:

[0007] Determine if the PDF document is an image-based PDF document;

[0008] If the PDF document is an image-based PDF document, the PDF document is converted into a target image;

[0009] Based on the identified target object within the target image, the target image is cropped to obtain a cropped image containing only the target object. The area corresponding to the target object in the cropped target image is then filled with blanks to obtain a filled target image. The target object is at least one of a table and an embedded image.

[0010] The cropped image is parsed and processed based on a deep learning model to obtain the structured information corresponding to the cropped image;

[0011] The target image is identified using optical character recognition technology to obtain the text information corresponding to the target image.

[0012] The structured information and the text information are integrated to obtain the page content information corresponding to the PDF document.

[0013] Secondly, embodiments of this application provide a device for extracting the content of a PDF document, the device comprising:

[0014] The document determination module is used to determine whether a PDF document is an image-based PDF document;

[0015] A document conversion module is used to convert the PDF document into a target image when the PDF document is an image-based PDF document;

[0016] The image processing module is used to crop the target image based on the identified target object in the target image to obtain a cropped image containing only the target object, and to fill the area corresponding to the target object in the cropped target image with blanks to obtain a filled target image. The target object is at least one of a table and an embedded image.

[0017] The information acquisition module is used to parse and process the cropped image based on a deep learning model to obtain the structured information corresponding to the cropped image;

[0018] The text acquisition module is used to identify the target image to be filled based on optical character recognition technology, and obtain the text information corresponding to the target image to be filled;

[0019] The content acquisition module is used to integrate the structured information and the text information to obtain the page content information corresponding to the PDF document.

[0020] Thirdly, embodiments of this application provide an electronic device, including:

[0021] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the PDF document content extraction method described in any of the preceding claims.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium that, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the PDF document content extraction method described in any of the preceding claims.

[0023] Compared with the prior art, the embodiments of this application have the following advantages:

[0024] In this embodiment, after converting the image-based PDF document into a target image, the target objects (embedded images and / or tables) in the target image are cropped out. The structured information of the cropped image is then analyzed using a deep learning model, and the text information corresponding to the target image is analyzed and filled using optical character recognition technology. Finally, the analyzed information is integrated. This method can efficiently solve the problem of table loss during PDF parsing, thereby enabling efficient and accurate extraction of document content from image-based PDF documents.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0026] Figure 1 A flowchart illustrating the steps of a method for extracting content from a PDF document, as provided in this application embodiment;

[0027] Figure 2 A schematic diagram illustrating the conversion of a PDF document into an image, as provided in an embodiment of this application;

[0028] Figure 3 A schematic diagram illustrating an extracted table and an embedded image provided in an embodiment of this application;

[0029] Figure 4 A schematic diagram of a target image filled according to an embodiment of this application;

[0030] Figure 5 A schematic diagram of the structure of a PDF document content extraction device provided in this application embodiment.

[0031] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0034] Reference Figure 1 The diagram illustrates a flowchart of the steps involved in extracting content from a PDF document according to an embodiment of this application. Figure 1 As shown, the method for extracting the content of the PDF document may include steps 101 to 106.

[0035] Step 101: Determine if the PDF document is an image-based PDF document.

[0036] This application embodiment can be applied to scenarios where content extraction from image-based PDF documents is performed using deep learning models.

[0037] When extracting content from a PDF document, you can first determine whether the PDF document is an image-based PDF document. Specifically, this can be done in the following three ways:

[0038] 1. Parse the internal object structure of the PDF document to obtain the text proportion of text objects within the PDF document. Based on the relationship between the text proportion and a pre-set quantity threshold, determine whether the PDF document is an image-based PDF document.

[0039] In this embodiment, the internal structure of the PDF document can be parsed. This involves using a PDF parsing library (such as PyMuPDF or PDFMiner) to read the PDF file, traversing the page content stream, and identifying text objects (LTTextContainer) and image objects (LTRasterImage). Then, the text object percentage can be calculated, which is the ratio of the number of text objects (N_text) to the total number of objects (N_total). Specifically, this can be calculated as the ratio of the amount of text object data (such as font and character encoding data) to the total file size. Finally, the calculated ratio can be compared with a preset threshold (e.g., 10%). If the text percentage is less than the threshold, the PDF is classified as an image-based PDF; otherwise, it is classified as a text-based PDF.

[0040] For example, taking a scanned PDF as an example, the parsing result is: a total of 100 objects, including 5 text objects (a small number of titles recognized by OCR) and 95 image objects (newspaper layout images). The text percentage = 5 / 100 = 5% < 10%, so it is judged as an image-type PDF, etc.

[0041] 2. Parse the PDF document and determine the object distribution characteristics within the PDF document based on the parsing results. Determine whether the PDF document is an image-based PDF document based on the ratio of text-based to image-based features in the object distribution characteristics.

[0042] In this embodiment, object distribution features can be extracted first, i.e., the PDF document is parsed, and the object distribution features within the PDF document are determined based on the parsing results. Specifically, for text-type features, the following can be statistically analyzed: text box density (number of text boxes per unit page), text line continuity (whether the vertical spacing between adjacent text lines conforms to layout rules, such as 0.5-1.5 times the font height), and font features (whether there are multiple fonts or font size variations). For image-type features, the following can be statistically analyzed: image resolution (pixel density of a single image), and image area (the percentage of pixel area of ​​an image object on the page).

[0043] Then, the feature ratio can be calculated, that is, the text features (F_text) and image features (F_image) are scored separately (e.g., 0-100 points), and the ratio is calculated as follows: Feature ratio = F_image / (F_image + F_text).

[0044] Determine the PDF document type: If the feature ratio is greater than 10%, the PDF document is determined to be an image-based PDF document; otherwise, it is a text-based PDF document.

[0045] For example, taking a scanned handwritten note PDF as an example, feature analysis: Text features: Text box density 0.1 per page (only a small number of handwritten OCR results), text line spacing is irregular. Image features: Image resolution 300 DPI, image area occupancy 95%, compressed format JPEG. Feature ratio = 95 / (5+95) = 95% > 10%, therefore the PDF document is determined to be an image-based PDF document.

[0046] 3. Parse the PDF document and determine its multi-dimensional document features based on the parsing results. Weight the multi-dimensional document features according to pre-set feature weights to obtain a feature-weighted score. Determine whether the PDF document is an image-based PDF document based on the feature-weighted score.

[0047] In this embodiment, multi-dimensional features can be predefined, including text-related features such as text proportion (weight 0.3), OCR confidence (weight 0.2), and text reproducibility (weight 0.1). Image-related features include image resolution (weight 0.2), image compression rate (weight 0.1), and image-text overlap rate (weight 0.1).

[0048] Feature extraction and scoring: Example feature scoring: Text percentage: 30% → Score 60 (out of 100, corresponding to a threshold of 30% for 60 points). Image resolution: 300 DPI → Score 90 (≥300 DPI gets full marks), OCR confidence: 40% → Score 40 (low OCR confidence). Weighted calculation of total score: e.g., Text percentage 60 × 0.3 + Resolution 90 × 0.2 + OCR confidence 40 × 0.2 + ... = 62 points.

[0049] Based on the total score: A preset threshold (e.g., 70 points) is used. If the total score is less than 70 points, it is determined to be an image-based PDF document; otherwise, it is determined to be a text-based PDF document.

[0050] If the PDF document is determined to be a text-based PDF document, the text and tables within the PDF document can be directly extracted, and the format can be corrected using a rule engine. That is, for PDF text containing text and tables, the extracted information, such as titles and tables, is separated. In this case, the format can be corrected using a rule engine.

[0051] After determining that the PDF text is an image-based PDF document, proceed to step 102.

[0052] Step 102: If the PDF document is an image-based PDF document, convert the PDF document into a target image.

[0053] If a PDF document is determined to be an image-based PDF document, it can be converted into a target image.

[0054] In practice, online conversion tools can be used to convert PDF documents into target images. Alternatively, professional PDF conversion software can be used. The pdf2image library can also be used to convert each page of a PDF into a high-resolution target image. This embodiment does not limit the method of converting PDF documents into target images.

[0055] After converting the PDF document, the resulting target images can be stored page by page to avoid loading the entire file. Each page can be named... <page>.png, etc.

[0056] In one specific implementation of this application, a PDF document can be converted page by page into a target image of a specified format based on a preset conversion method, and the target image can be stored page by page. This allows for subsequent content extraction using a page-by-page loading and dynamic memory management approach. Specifically, a generator-based page-by-page loading technique can be used to achieve on-demand data retrieval through a "lazy computation" mechanism, avoiding memory overflow caused by loading the entire dataset at once. Dynamic memory management: Dynamic memory management uses an "on-demand allocation, timely release" strategy to ensure the stability of memory usage during streaming processing. Processing progress saving and restoration mechanism: Interrupted resume processing is achieved through "state checkpointing," ensuring the reliability of large task processing.

[0057] After converting the PDF document into the target image, proceed to step 103.

[0058] Step 103: Based on the identified target object within the target image, crop the target image to obtain a cropped image containing only the target object. Fill the area corresponding to the target object in the cropped target image with blanks to obtain a filled target image. The target object is at least one of a table and an embedded image.

[0059] After converting a PDF document into a target image, target objects within the target image can be identified. These target objects can be at least one of the following: tables and embedded images.

[0060] After identifying the target object within the target image, the target image can be cropped based on the identified target object to obtain a cropped image containing only the target object. Specifically, this can include the following three scenarios:

[0061] 1. If the target image contains only a table, then only the table area within the target image can be cropped to obtain a cropped image.

[0062] 2. If the target image contains only the embedded image, then only the embedded image area within the target image can be cropped to obtain the cropped image.

[0063] 3. If the target image contains both a table and an embedded image, then a cropped image containing only the table and a cropped image containing only the embedded image can be extracted from the target image. For example... Figure 2 As shown, a PDF document contains both text (i.e., ...) Figure 2 (Paragraphs shown) and tables ( Figure 2 (as shown in Tables 1 and 2) and embedded images ( Figure 2 shown" Figure 1 At this point, you can crop the image from the converted PDF document. Figure 3 Image. For example... Figure 3 As shown, the cropped images include: an image containing only Table 1, i.e., 1_table_1.png; an image containing only Table 2, i.e., 1_table_2.png; and an image containing only embedded image 1, i.e., 1_img_1.png, etc.

[0064] It is understood that the above examples are merely examples listed to better understand the technical solutions of the embodiments of this application, and are not intended to be the only limitation on the embodiments.

[0065] After obtaining the cropped image, blank pixels can be filled into the region corresponding to the target object in the cropped target image to obtain the filled target image. This involves filling the cropped region from the target image with blank pixels; the resulting image is the filled target image. The above implementation process can be described in detail below with reference to specific implementation methods.

[0066] In one specific implementation of this application, step 103 may include:

[0067] Sub-step A1: Detect the target image based on a lightweight neural network model to obtain the target object within the target image.

[0068] In this embodiment, after the target image is obtained through conversion, it can be detected based on a lightweight neural network model (such as YOLO-Tiny) to obtain the target object within the target image.

[0069] Sub-step A2: Crop the target image according to the region where the target object is located to obtain a cropped image that only contains the target object.

[0070] After identifying the target object within the target image, the target image can be cropped based on the area where the identified target object is located, to obtain a cropped image containing only the target object.

[0071] Sub-step A3: Fill the area where the target object is located in the cropped target image with blanks to obtain a filled target image, and add the image identifier corresponding to the cropped image to the filled area of ​​the filled target image.

[0072] Then, blanking can be applied to the area containing the target object in the cropped target image to obtain a filled target image. An image identifier corresponding to the cropped image can then be added to the filled area of ​​the filled target image. In this example, the image identifier could be the filename of the cropped image, etc.

[0073] In practical implementation, a lightweight CNN model (such as YOLO-Tiny) can be used to detect table regions, output table bounding boxes, crop the table regions, and save them as independent images, named as follows: <page>_table_<table_index> The image is saved as a .png file. White space is used to cover the original table area, and the filename is written at the covered location. A lightweight CNN model (such as YOLO-Tiny) is used to detect image regions, output image bounding boxes, crop the image region, and save it as a separate image, named... <page>img<img_index> The .png file will fill the original table area with white space, and the filename will be written at the overwritten location. For example... Figure 4 As shown, the area corresponding to Table 1 after backfilling can be filled with the filename "1_table_1.png" of the cropped image corresponding to Table 1; the area corresponding to Table 2 after backfilling can be filled with the filename "1_table_2.png" of the cropped image corresponding to Table 2; the area corresponding to Embedded Image 1 after backfilling can be filled with the filename "1_img_1.png", etc.

[0074] The backfilled image (i.e., the filled target image) can be saved as " <page>filled.png".

[0075] Step 104: The cropped image is parsed and processed based on a deep learning model to obtain the structured information corresponding to the cropped image.

[0076] After obtaining the cropped image, it can be parsed using a deep learning model to extract the corresponding structured information. This process will be described in detail below.

[0077] In one specific implementation of this application, step 104 may include:

[0078] Sub-step B1: When the target object in the cropped image is a table, the cropped image is parsed using a Transformer-based table recognition model to obtain the table structured information corresponding to the cropped image.

[0079] In this embodiment, after obtaining the cropped image, if the target object within the cropped image is a table, a Transformer-based table recognition model can be used to parse the cropped image and obtain the corresponding table structured information. Specifically, a Transformer-based table recognition model (such as TableFormer) is used to parse the table image into structured data. The table header, cell relationships, and data types (such as numbers and dates) are extracted to generate semantically annotated table content.

[0080] The model inference process can be summarized as follows: I. Input Image Preprocessing: 1. Image Standardization: Adjust the cropped image to the model input size (e.g., 512×512 pixels), maintaining the aspect ratio and filling edges to ensure a consistent input format. 2. Image Enhancement (Optional): Denoise, binarize, or enhance the contrast of the standardized image to improve the clarity of table lines and text. 3. OCR Text Extraction: Use OCR tools (e.g., Tesseract, PaddleOCR) to extract the text content and its coordinate positions (x1, y1, x2, y2) from the image, forming "text-position" pairs. II. Multimodal Feature Fusion: 1. Visual Feature Extraction: Extract visual features of the image using CNN or ViT (Visual Transformer) to capture spatial structural information such as table lines and regions. 2. Text and Layout Feature Encoding: The text extracted by OCR is converted into token embeddings, combined with the semantic information of the text content, and position encoding is performed on the text coordinates. Two-dimensional spatial positions (such as relative row and column positions, border coordinates) are converted into numerical vectors that the model can understand, reflecting the layout relationship of the text in the table. 3. Feature Fusion: Visual features, text embeddings, and position encoding are linked through the Transformer's multimodal fusion layer (such as a cross-attention mechanism), allowing the model to understand the correspondence between "the position of the text in the image" and "the visual structure of the table." III. Transformer Model Forward Inference: 1. Input Encoding: The fused multimodal feature sequence is used as input to the Transformer encoder. Each feature vector includes positional information, facilitating the model's capture of global dependencies. 2. Self-attention mechanism processing: Through a multi-layer Transformer encoder, the self-attention mechanism is used to analyze the correlation between different features (including: capturing the dependencies between rows and columns in the table (such as horizontal alignment of text in the same line, vertical correlation within the same column), and identifying cross-regional relationships of merged cells (such as determining which cells belong to the same merged block through attention weights)). 3. Decoder generates structural sequence: For the end-to-end model, the decoder generates serialized labels for the table structure (such as "start of row," "cell," "merged cell," etc.) based on the features output by the encoder, gradually constructing the row and column structure of the table.

[0081] Sub-step B2: When the target object within the cropped image is an embedded image, the cropped image is parsed using a Transformer-based image recognition model to obtain the image structure information corresponding to the cropped image.

[0082] When the target object within a cropped image is an embedded image, a Transformer-based image recognition model can be used to parse the cropped image and obtain its corresponding structured information. In other words, a Transformer-based image recognition model (such as ImageFormer) can be used to parse the image content into structured data.

[0083] The model inference process can be summarized as follows: I. Input Image Preprocessing: 1. Image Standardization: Adjust the cropped image to the model-specified size (e.g., 224×224 or 384×384 pixels), maintaining proportions and filling edges to avoid content distortion. Normalize pixel values ​​(e.g., subtract the mean, divide by the standard deviation) to match the input format during model training. 2. Multimodal Information Extraction (Optional): If the image contains text (e.g., trademarks, slogans), use OCR tools (e.g., PaddleOCR, Tesseract) to extract the text content and coordinates, forming "text-location" pairs to prepare for subsequent multimodal fusion. II. Visual Feature Extraction and Block Encoding: 1. Image Blocking: Divide the preprocessed image into fixed-size patches (e.g., 16×16 pixels), with each patch considered a "visual token." Convert each patch into a feature vector (e.g., 768-dimensional embedding) through a linear projection layer, preserving local spatial features. 2. Position Encoding: Add two-dimensional position encoding to each patch feature to reflect its relative position in the original image (e.g., row / column index), helping the Transformer understand spatial structure. 3. Multimodal Feature Fusion (if text is included): Fuse the text embedding extracted by OCR with the corresponding visual patch features through a cross-attention mechanism to enhance the model's understanding of "image-text association" (e.g., text-annotated object regions). III. Transformer Model Forward Inference: 1. Decoder Processing: Input the segmented visual feature sequence (including position encoding) into the Transformer encoder. A multi-layer self-attention mechanism captures global dependencies (analyzing semantic associations between different patches (e.g., the spatial co-occurrence of "sky" and "clouds"), and identifies cross-regional features of objects (e.g., the front and body of a car are associated through attention weights)). Simultaneously, a multi-layer perceptron (MLP) is combined to enhance feature representation capabilities, generating feature vectors containing global semantics. 2. Multi-scale feature fusion (optional): For complex scenes, feature fusion of different scales can be achieved through a hierarchical Transformer structure (such as ViT-L / 16) or Feature Pyramid Network (FPN), taking into account both details (small objects) and the overall scene layout. 3. Task-specific output layer: Depending on the model design, the output layer may include: an object detection head (predicting the category, bounding box coordinates, and confidence score of each candidate region), a semantic segmentation head (predicting the category label for each pixel and generating a mask), and a relationship prediction head (inferring spatial relationships between objects (e.g., "a car is next to a tree") or attributes (e.g., "the sky is blue")).IV. Post-processing: Structured Information Generation: 1. Object Detection Result Parsing: Non-maximum suppression (NMS) is applied to the candidate bounding boxes output by the model to filter overlapping boxes and retain high-confidence object detection results, forming an "object list" containing: category labels, bounding box coordinates, confidence scores, etc. 2. Semantic and Relationship Structuring: Attribute Extraction: Based on the model's output of object features, attributes (such as color, size, action) are inferred. Scene Relationship Modeling: Spatial position (such as "above" or "to the right") or logical association (such as "person riding a bicycle") between objects is analyzed through attention weights to generate relation triples (subject-verb-object). 3. Multimodal Information Fusion and Integration: The text information extracted by OCR is associated with the detected objects (such as the text "No Parking" corresponding to the detected "Warning Sign" object), supplementing the semantic details of the structured information. 4. Structured Format Output: The above information is organized into JSON, XML, or graph structures.

[0084] Step 105: Recognize the target image based on optical character recognition technology to obtain the text information corresponding to the target image.

[0085] After obtaining the target image for filling, optical character recognition (OCR) technology can be used to identify the text information corresponding to the target image. Specifically, OCR (such as PaddleOCR) can be performed on the marked full-page image to recognize non-table text. The OCR results can then locate tables and image filename markers (such as 1_table_1.png, 1_img_1.png) and replace them with the corresponding semantic table data.

[0086] Step 106: Integrate the structured information and the text information to obtain the page content information corresponding to the PDF document.

[0087] After obtaining the structured and textual information, they can be integrated to obtain the page content information corresponding to the PDF document. Specifically, image identifiers and coordinate information contained in the text information can be acquired. Based on the coordinate and image identifiers, the positional information of the structured and textual information within the target image is identified. Then, based on the positional information, the structured and textual information are integrated to obtain the page content information corresponding to the PDF document, that is, the final output coherent page content, including text and structured tables.

[0088] Of course, when there are paginated target images (i.e., the original PDF document is large and contains paginated images), the structured information of each page's target image and the aforementioned text information can be integrated to obtain the page content information for each page. Furthermore, a context-based approach is used to integrate the paginated page content information to obtain the page content information of the PDF document.

[0089] Simultaneously, a semantic consistency verification mechanism can be employed to check the semantic consistency of the final output page content. If the verification passes, it indicates that the page information integration is successful, and the final content is output. Otherwise, the context-based integration method is used again to integrate the paginated content.

[0090] The PDF document content extraction method provided in this application converts the image-based PDF document into a target image, then crops out the target objects (embedded images and / or tables) from the target image. It then parses the structured information of the cropped image using a deep learning model, and uses optical character recognition (OCR) technology to parse and fill in the corresponding text information in the target image. Finally, the parsed information is integrated. This method efficiently solves the problem of table loss during PDF parsing, thus enabling efficient and accurate extraction of document content from image-based PDF documents.

[0091] Reference Figure 5 The diagram shows a schematic representation of a PDF document content extraction device provided in an embodiment of this application. Figure 5 As shown, the PDF document content extraction device 500 may include the following modules:

[0092] Document determination module 510 is used to determine whether a PDF document is an image-based PDF document;

[0093] The document conversion module 520 is used to convert the PDF document into a target image when the PDF document is an image-based PDF document;

[0094] Image processing module 530 is used to crop the target image based on the identified target object in the target image to obtain a cropped image containing only the target object, and to fill the area corresponding to the target object in the cropped target image with blanks to obtain a filled target image, wherein the target object is at least one of a table and an embedded image;

[0095] The information acquisition module 540 is used to parse and process the cropped image based on a deep learning model to obtain the structured information corresponding to the cropped image;

[0096] The text acquisition module 550 is used to identify the filling target image based on optical character recognition technology to obtain the text information corresponding to the filling target image;

[0097] The content acquisition module 560 is used to integrate the structured information and the text information to obtain the page content information corresponding to the PDF document.

[0098] Optionally, the document determination module includes:

[0099] The text percentage acquisition unit is used to parse the internal object structure of the PDF document and obtain the text percentage of the text objects in the PDF document.

[0100] The first document determination unit is used to determine whether a PDF document is an image-based PDF document based on the relationship between the text proportion and a preset quantity threshold.

[0101] Optionally, the document determination module includes:

[0102] A distribution feature determination unit is used to parse the PDF document and determine the object distribution features within the PDF document based on the parsing results.

[0103] The second document determination unit is used to determine whether a PDF document is an image-based PDF document based on the ratio of text-type features to image-type features in the object distribution characteristics.

[0104] Optionally, the document determination module includes:

[0105] A document feature determination unit is used to parse the PDF document and determine the multi-dimensional document features corresponding to the PDF document based on the parsing results.

[0106] The weighted scoring acquisition unit is used to weight the multi-dimensional document features according to the feature weights corresponding to the pre-set multi-dimensional document features to obtain a feature weighted score;

[0107] The third document determination unit is used to determine whether a PDF document is an image-based PDF document based on the feature-weighted score.

[0108] Optionally, the image processing module includes:

[0109] An object detection unit is used to detect the target image based on a lightweight neural network model to obtain the target object within the target image;

[0110] An image cropping unit is used to crop the target image according to the region where the target object is located, so as to obtain a cropped image containing only the target object;

[0111] An image filling unit is used to fill the blank area of ​​the target object in the cropped target image to obtain a filled target image, and to add the image identifier corresponding to the cropped image to the filled area of ​​the filled target image.

[0112] Optionally, the information acquisition module includes:

[0113] The table information acquisition unit is used to parse the cropped image using a Transformer-based table recognition model when the target object in the cropped image is a table, so as to obtain the table structured information corresponding to the cropped image.

[0114] The image information acquisition unit is used to parse and process the cropped image using a Transformer-based image recognition model when the target object in the cropped image is an embedded image, so as to obtain the image structure information corresponding to the cropped image.

[0115] Optionally, the content acquisition module includes:

[0116] The information acquisition unit is used to acquire the image identifiers and coordinate information contained in the text information;

[0117] A location recognition unit is used to recognize the location information of the structured information and the text information within the target image based on the coordinate information and the image identifier;

[0118] The page content acquisition unit is used to integrate the structured information and the text information based on the location information to obtain the page content information corresponding to the PDF document.

[0119] Optionally, the document conversion module includes:

[0120] An image conversion unit is used to convert the PDF document page by page into the target image in a specified format based on a preset conversion method;

[0121] An image storage unit is used to store the target image page by page, so that when extracting content later, the target image can be extracted using a page-by-page loading and dynamic memory management method.

[0122] Optionally, the content acquisition module includes:

[0123] The content acquisition unit is used to integrate the structured information and text information of the target image on each page when there are paginated target images to obtain the page content information of each page;

[0124] The content information acquisition unit is used to integrate the page content information of the pagination using a context-based approach to obtain the page content information of the PDF document.

[0125] The PDF document content extraction device provided in this application converts the image-based PDF document into a target image, then crops out the target objects (embedded images and / or tables) from the target image, analyzes the structured information of the cropped image based on a deep learning model, analyzes the text information corresponding to the target image based on optical character recognition technology, and then integrates the analyzed information. This method can efficiently solve the problem of table loss in PDF parsing, thereby enabling efficient and accurate extraction of document content from image-based PDF documents.

[0126] This application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described method for extracting PDF document content.

[0127] Figure 6 A schematic diagram of the structure of an electronic device 600 according to an embodiment of the present invention is shown. Figure 6 As shown, the electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 602 or loaded from storage unit 608 into random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0128] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, microphone, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0129] The various processes and handling described above can be executed by processing unit 601. For example, the methods of any of the above embodiments can be implemented as computer software programs tangibly contained in a computer-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by CPU 601, one or more actions of the methods described above can be performed.

[0130] Additionally, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for extracting PDF document content.

[0131] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0132] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminals (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0134] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal, causing a series of operational steps to be executed on the computer or other programmable terminal to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0136] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0137] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes said element.

[0138] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.< / page> < / page> < / page> < / page>

Claims

1. A method for extracting content from a PDF document, characterized in that, The method includes: Determine if the PDF document is an image-based PDF document; If the PDF document is an image-based PDF document, the PDF document is converted into a target image; Based on the identified target object within the target image, the target image is cropped to obtain a cropped image containing only the target object. The area corresponding to the target object in the cropped target image is then filled with blanks to obtain a filled target image. The target object is at least one of a table and an embedded image. The cropped image is parsed and processed based on a deep learning model to obtain the structured information corresponding to the cropped image; The target image is identified using optical character recognition technology to obtain the text information corresponding to the target image. The structured information and the text information are integrated to obtain the page content information corresponding to the PDF document.

2. The method according to claim 1, characterized in that, Determining whether a PDF document is an image-based PDF document includes: Parse the internal object structure of the PDF document to obtain the text proportion of the text objects in the PDF document; Based on the relationship between the text percentage and a pre-set quantity threshold, it is determined whether the PDF document is an image-based PDF document.

3. The method according to claim 1, characterized in that, Determining whether a PDF document is an image-based PDF document includes: The PDF document is parsed, and the object distribution characteristics within the PDF document are determined based on the parsing results; Based on the ratio of textual features to image features in the object distribution characteristics, determine whether the PDF document is an image-based PDF document.

4. The method according to claim 1, characterized in that, Determining whether a PDF document is an image-based PDF document includes: The PDF document is parsed, and the multi-dimensional document features corresponding to the PDF document are determined based on the parsing results; The multi-dimensional document features are weighted according to the pre-set feature weights corresponding to the multi-dimensional document features to obtain a feature weighted score; Whether a PDF document is an image-based PDF document is determined based on the feature-weighted score.

5. The method according to claim 1, characterized in that, The step of cropping the target image based on the identified target object within the target image to obtain a cropped image containing only the target object, and then filling the area corresponding to the target object in the cropped target image with blanks to obtain a filled target image, includes: The target image is detected using a lightweight neural network model to obtain the target object within the target image; The target image is cropped based on the region where the target object is located, resulting in a cropped image that contains only the target object. The area where the target object is located in the cropped target image is filled with blank space to obtain a filled target image, and the image identifier corresponding to the cropped image is added to the filled area of ​​the filled target image.

6. The method according to claim 1, characterized in that, The deep learning model is used to parse and process the cropped image to obtain the structured information corresponding to the cropped image, including: When the target object in the cropped image is a table, a Transformer-based table recognition model is used to parse the cropped image to obtain the table structure information corresponding to the cropped image. When the target object within the cropped image is an embedded image, a Transformer-based image recognition model is used to parse and process the cropped image to obtain the image structure information corresponding to the cropped image.

7. The method according to claim 5, characterized in that, The process of integrating the structured information and the text information to obtain the page content information corresponding to the PDF document includes: Obtain the image identifier and coordinate information contained in the text information; Based on the coordinate information and the image identifier, identify the position information of the structured information and the text information within the target image; The structured information and text information are integrated based on the location information to obtain the page content information corresponding to the PDF document.

8. The method according to claim 1, characterized in that, The step of converting the PDF document into a target image includes: The PDF document is converted page by page into the target image in the specified format based on a preset conversion method; The target image is stored page by page so that content extraction can be performed on the target image using a page-by-page loading and dynamic memory management method during subsequent content extraction.

9. The method according to claim 1, characterized in that, The process of integrating the structured information and the text information to obtain the page content information corresponding to the PDF document includes: When the target image exists in pagination, the structured information and text information of the target image on each page are integrated to obtain the page content information of each page; By using contextual association, the page content information of the paginated document is integrated to obtain the page content information of the PDF document.

10. A device for extracting content from a PDF document, characterized in that, The device includes: The document determination module is used to determine whether a PDF document is an image-based PDF document; A document conversion module is used to convert the PDF document into a target image when the PDF document is an image-based PDF document; The image processing module is used to crop the target image based on the identified target object in the target image to obtain a cropped image containing only the target object, and to fill the area corresponding to the target object in the cropped target image with blanks to obtain a filled target image. The target object is at least one of a table and an embedded image. The information acquisition module is used to parse and process the cropped image based on a deep learning model to obtain the structured information corresponding to the cropped image; The text acquisition module is used to identify the target image to be filled based on optical character recognition technology, and obtain the text information corresponding to the target image to be filled; The content acquisition module is used to integrate the structured information and the text information to obtain the page content information corresponding to the PDF document.

Images