PDF (Portable Document Format) document conversion device and method, storage medium and computer equipment

Abstract

The invention discloses a PDF document conversion device and method, a storage medium and computer equipment. The device comprises a data processing module which converts a PDF document into a to-be-processed image and obtains original image data of an embedded image; the layout analysis module performs region division on the to-be-processed image, and determines a text region, a formula region and a table region of the PDF document; the first recognition module performs content recognition on the text region, the formula region and the table region to generate a text recognition result, a formula recognition result and a table recognition result; the second recognition module generates an image recognition result based on the original image data; the semantic analysis module generates semantic representations of a text recognition result, a formula recognition result, a table recognition result and an image recognition result through a large language model; and the PowerPoint generation module maps the obtained recognition result and semantic representation to a PowerPoint template to generate a final PowerPoint. The accuracy and content richness of generating the presentation file based on the PDF document can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a conversion device and method for a PDF document, a storage medium and a computer device. BACKGROUND

[0002] At current academic conferences or academic reports, it is often necessary to convert a PDF version of a research document into a PPT presentation for reporting. The existing method of converting a PDF into a presentation usually first reads the PDF document through a PDF reader, captures a screenshot image of the PDF document using a screenshot tool, then processes the screenshot image to extract page elements such as titles, paragraphs or images in the screenshot image, and finally directly maps the extracted content to the PPT according to a fixed module, for example, mapping the first-level title in the PDF document to the title page of the PPT, and mapping the second-level title in the PDF document to the content page title of the PPT.

[0003] The existing method relies on a screenshot image of the PDF document for data extraction. When the quality of the original PDF document is poor, the screenshot operation will exacerbate the data distortion problem of the document, which leads to an increase in the error rate of subsequent image processing steps, affecting the accuracy of the extracted key content. In addition, the conversion method based on preset rules lacks understanding of the content of the document, and there is a problem of content emptiness in the generated presentation. SUMMARY

[0004] To this end, embodiments of the present application provide a conversion device and method for a PDF document, a storage medium and a computer device, which realize improving the content accuracy of the generated presentation and performing semantic understanding on the obtained data, so that the content of the finally generated presentation is more rich and the logic is more clear.

[0005] In a first aspect, the present application provides a conversion device for a PDF document.

[0006] The present application is realized by the following technical solutions:

[0007] A conversion device for a PDF document, the conversion device comprising:

[0008] a data processing module configured to receive a PDF document, convert the PDF document into a to-be-processed image page by page, and obtain original image data of an embedded image in the PDF document from a memory;

[0009] a layout analysis module configured to perform region division on the to-be-processed image using a pre-trained layout analysis model, and determine a target detection region of the PDF document, the target detection region including a text region, a formula region and a table region;

[0010] The first recognition module is configured to perform text recognition on the text region to generate a text recognition result, perform formula recognition on the formula region to generate a formula recognition result, and perform data recognition on the table region to generate a table recognition result.

[0011] The second recognition module is configured to perform content recognition on the original image data to generate an image recognition result.

[0012] The semantic analysis module is configured to generate semantic representations of the text recognition result, the formula recognition result, the table recognition result, and the image recognition result by using a large language model.

[0013] The presentation generation module is configured to combine the text recognition result, the formula recognition result, the table recognition result, and the image recognition result with the semantic representations and map them into a template of a presentation to generate a final presentation.

[0014] In a preferred example of the present application, the data processing module can be further configured to: locate an image object in the PDF document, read image encoding data in an original encoding format corresponding to the image object from the memory, and temporarily store the image encoding data in a memory buffer as the original image data.

[0015] In a preferred example of the present application, the layout analysis model can be further configured to be constructed based on a YOLOv10 model.

[0016] In a preferred example of the present application, the layout analysis module can be further configured to:

[0017] preprocess the image to be processed into an input image with a preset resolution;

[0018] extract multi-scale feature maps of the input image by using a pre-trained YOLOv10 model;

[0019] perform feature fusion and enhancement on the multi-scale feature maps to output aggregated feature maps;

[0020] perform region detection and classification based on the aggregated feature maps to output content region images, wherein the content region images include text regions, formula regions, and table regions;

[0021] establish a hierarchical relationship topology of the content region images, wherein the hierarchical relationship includes parent-child relationship, reference relationship, inclusion relationship, and sibling relationship.

[0022] In a preferred example of the present application, the first recognition module can further include:

[0023] a text recognition unit configured to perform text recognition on the text region by using an OCR algorithm to obtain a structured text recognition result;

[0024] The formula recognition unit is configured to generate an explanation text of a formula in a formula region by using a multimodal large language model.

[0025] The table recognition unit is configured to identify a table type and table elements of a table region by using a multimodal large language model, and obtain a data relationship of the elements in the table.

[0026] In a preferred example of the present application, the text recognition unit is specifically configured to:

[0027] perform geometric correction processing on the text region to obtain a corrected image;

[0028] call two OCR engines in parallel to perform character recognition on the corrected image;

[0029] perform weighted fusion on the character recognition results output by the two OCR engines, and output a structured text recognition result.

[0030] In a preferred example of the present application, the formula recognition unit is specifically configured to:

[0031] recognize and extract formula content of the formula region;

[0032] convert the formula content into a LaTeX representation vector of the formula;

[0033] input the LaTeX representation vector into a multimodal large language model to generate an explanation text under a preset prompt word.

[0034] In a preferred example of the present application, the table recognition unit is specifically configured to:

[0035] extract a table of the table region by using a multimodal large language model, and identify a table type;

[0036] analyze elements in the table and relationships between the elements based on the table type, and generate a data relationship of the table.

[0037] In a preferred example of the present application, the second recognition module is specifically configured to:

[0038] call a decoder corresponding to an original encoding format to output an RGB pixel matrix, and save the RGB pixel matrix as a lossless format image;

[0039] identify a target region containing characters in the lossless format image by using a DB character detection algorithm;

[0040] perform character recognition on the target region by using an OCR algorithm, and determine an image recognition result.

[0041] In a preferred example of the present application, the conversion device can be further configured to:

[0042] The feedback module is configured to receive a user's modification suggestion on the generated presentation, convert the modification suggestion into a modification instruction, and send the modification instruction to the presentation generation module to enable the presentation generation module to adjust the presentation based on the modification instruction.

[0043] In a preferred example of the present application, the data processing module can be further configured to:

[0044] The to-be-processed image is preprocessed by removing the background and adjusting the contrast.

[0045] In a second aspect, the present application provides a conversion method for a PDF document.

[0046] The present application is achieved by the following technical solutions:

[0047] A conversion method for a PDF document, by the conversion device for a PDF document according to the first aspect, the conversion method comprises:

[0048] Receiving a PDF document, converting the PDF document into a to-be-processed image page by page, and obtaining original image data of an embedded image in the PDF document from the memory;

[0049] Dividing the to-be-processed image into regions, determining a target detection region, and the target detection region includes a text region, a formula region and a table region;

[0050] Performing content recognition on the target detection region, and outputting a text recognition result, a formula recognition result and a table recognition result;

[0051] Performing content recognition on the original image data to generate an image recognition result;

[0052] Generating semantic representations of the text recognition result, the formula recognition result, the table recognition result and the image recognition result by a large language model;

[0053] Mapping the text recognition result, the formula recognition result, the table recognition result and the image recognition result combined with the semantic representations into a template of a presentation to generate a final presentation.

[0054] In a preferred example of the present application, the conversion device can be further configured to:

[0055] The original image data of the embedded image in the PDF document obtained from the memory comprises:

[0056] Locate an image object in a PDF document, read image encoding data in a raw encoding format corresponding to the image object from memory, and temporarily store the image encoding data in a memory buffer as raw image data.

[0057] In a third aspect, the present application provides a computer readable storage medium.

[0058] The present application is achieved by the following technical solutions:

[0059] A computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of any of the conversion methods for a PDF document described above.

[0060] In a fourth aspect, the present application is achieved by the following technical solutions:

[0061] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of any of the conversion methods for a PDF document described above when executing the computer program.

[0062] In summary, compared with the prior art, the technical solutions provided in the embodiments of the present application have at least the following beneficial effects:

[0063] The PDF document conversion device proposed in the present application can convert a PDF document into a to-be-processed image based on a data processing module, and obtain raw image data of an embedded image in the PDF document from memory, divide a to-be-processed image into execution regions using a layout analysis module, determine a text region, a formula region, and a table region contained in the to-be-processed image, further identify the text region, the formula region, and the table region using a first recognition model to generate a text recognition result, a formula recognition result, and a table recognition result, identify the content of the raw image data using a second recognition module to generate an image recognition result, and generate semantic representations of the text recognition result, the formula recognition result, the table recognition result, and the image recognition result using a semantic analysis module. A presentation generation module is configured to map the text recognition result, the formula recognition result, the table recognition result, and the image recognition result combined with the semantic representations into a template of a presentation to generate a final presentation. The PDF document conversion device proposed in the present application can automatically extract text, formulas, tables, and images from a PDF document, perform semantic analysis on the recognition results, integrate the recognition results and the semantic representations, and automatically convert them into a final presentation to improve the conversion efficiency of the presentation. When extracting the content in the PDF document, the device obtains raw image data corresponding to the embedded image from memory, processes the raw image data, reduces data distortion, and improves the accuracy of the recognized data. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 A structure diagram of a conversion device for a PDF document provided by an embodiment of the present application is shown in the figure.

[0065] Figure 2 A flow diagram of a conversion method for a PDF document provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0066] The specific embodiments are merely used to explain the present application, and are not intended to limit the present application. Those skilled in the art can make modifications to the embodiments without creative effort, and the modifications shall fall within the scope of the present application.

[0067] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort shall fall within the scope of the present application.

[0068] In addition, the term “and / or” in the present application is merely used to describe the association relationship of the associated objects, and indicates that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character “ / ” in the present application generally represents an “or” relationship between the associated objects, unless otherwise specified.

[0069] The terms “first”, “second”, and the like in the present application are used to distinguish the same items or similar items with basically the same function and purpose, and it should be understood that there is no logical or time sequence relationship between “first”, “second”, and “n”, and the quantity and execution order are not limited.

[0070] In the embodiments of the present application, the words “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design solution described as “exemplary” or “for example” in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of “exemplary” or “for example” is intended to present the relevant concept in a specific manner.

[0071] Before the embodiments of the present application are further described in detail, the terms and names involved in the embodiments of the present application are explained, and the terms and names involved in the embodiments of the present application are applicable to the following explanations:

[0072] PDF (Portable Document Format) is a universal file format developed by Adobe, which accurately preserves text, images, fonts, and layout, ensuring consistent display across different applications, operating systems, and devices.

[0073] PPT (PowerPoint Presentation) is a presentation software and its file format developed by Microsoft, which integrates text, images, tables, and multimedia through slides, widely used in speeches, teaching, and business presentations.

[0074] OCR (Optical Character Recognition) is a technology that converts printed, handwritten, or typed text images into machine-editable text.

[0075] LLM (Large Language Model) is a deep learning-based natural language processing model that acquires language understanding and generation capabilities through large-scale pre-training.

[0076] MLLM (Multimodal Model Large Language Model) refers to an AI model that can process and understand multiple types of data such as text, images, and audio.

[0077] API (Application Programming Interface) is a set of predefined rules, protocols, and tools that allow different software components to communicate, exchange data, or call functions.

[0078] NLP (Natural Language Processing) is a branch of computer science and artificial intelligence that enables computers to understand, interpret, and generate human language.

[0079] The following further describes the embodiments of the present application in conjunction with the accompanying drawings.

[0080] Referring to Figure 1 The first exemplary embodiment of the present application provides a conversion device for a PDF document, which specifically includes:

[0081] The data processing module 10 is configured to receive a PDF document, convert the PDF document into a to-be-processed image page by page, and obtain original image data of an embedded image in the PDF document from a memory. Generally, the data processing module provides an HTTP API endpoint through which the data processing module receives a to-be-converted PDF document uploaded by a user through a webpage or a mobile APP.

[0082] In a possible implementation, when the data processing module 10 receives a to-be-converted PDF document, it is necessary to convert the to-be-converted PDF document into a to-be-processed image page by page, store the to-be-processed image corresponding to each PDF document in a storage device in a page code order, and call the to-be-processed image in the page code order for subsequent module processing. In actual implementation, a PDF rendering engine such as PDFium can be used to convert the PDF document into a to-be-processed image page by page.

[0083] The layout analysis module 20 is configured to perform region division on the to-be-processed image by using a pre-trained layout analysis model to divide a target detection region, wherein the target detection region includes a text region, a formula region, and a table region.

[0084] In actual implementation, a trained layout analysis model is loaded, the to-be-processed image is input into the layout analysis model, and a model output result of the to-be-processed image is obtained. The model output result includes a plurality of detection regions, a bounding box of each detection region, and a category label corresponding to each detection region. The category label of the detection region includes text, formula, and table. The text region, the formula region, and the table region are determined according to the category label.

[0085] The first recognition module 30 is configured to perform text recognition on the text region of the target detection region, output a text recognition result, perform formula recognition on the formula region to generate a formula recognition result, and perform data recognition on the table region to generate a table recognition result. Specifically, the first recognition module 30 is connected to an output end of the layout analysis module 20, receives the divided text region, formula region, and table region from the layout analysis module 20, processes different types of regions, performs text content recognition on the text region to obtain a text recognition result, recognizes a formula contained in the formula region to obtain a formula recognition result, and performs data recognition on the table region to obtain a table recognition result. It should be noted that the text recognition result is structured text data obtained by performing text recognition on the text region, the formula recognition result is a machine-parsable mathematical representation and natural language explanation, and the table recognition result is a structured data and relationship description in a table.

[0086] The second recognition module 40 is configured to perform content recognition on the original image data to generate an image recognition result.

[0087] Specifically, the second recognition module 40 is connected with the output end of the data processing module 10, obtains the original image data from the data processing module 10, performs content recognition on the original image data, and generates an image recognition result. It should be noted that the content recognition on the original image data refers to recognizing the text content in the original image data to generate the text data contained in the original image data.

[0088] The semantic analysis module 50 is configured to generate semantic representations of the text recognition result, the formula recognition result, the table recognition result and the image recognition result by the large language model.

[0089] In actual implementation, the text recognition result, the formula recognition result, the table recognition result and the image recognition result are input into the multi-modal large language model for semantic understanding, relationship reasoning and knowledge fusion to generate a JSON format semantic representation.

[0090] It should be noted that the semantic representation is to convert the original recognition result into structured knowledge understandable by machines. For example, the semantic representation of the text recognition result is used to indicate the language logic in the text, including paragraph structure, keywords, entities and topics; the semantic representation of the formula recognition result can be used to indicate the mathematical structure in the formula, including symbol hierarchy, operator relationship, LaTeX / MathML coding, etc.; the semantic representation of the table recognition result can be used to indicate the two-dimensional relationship in the table, including row-column topology, cell semantics, header association; and the semantic representation of the image recognition result is used to indicate the recognized text content in the image.

[0091] The presentation generation module 60 is configured to map the text recognition result, the formula recognition result, the table recognition result and the image recognition result combined with the semantic representation into a template of a presentation to generate a final presentation.

[0092] The conversion device of the PDF document can automatically extract text, formula, table and image from the PDF document, perform semantic analysis on the recognition result, integrate the recognition result and the semantic representation, and automatically convert the integrated result into a final presentation, thereby improving the conversion efficiency of the presentation; and when extracting the content in the PDF document, the device obtains the original image data corresponding to the embedded image from the memory, processes the original image data, and reduces data distortion, so that the accuracy of the recognized data is higher.

[0093] In one possible implementation, the data processing module 10 is specifically configured to:

[0094] The image object in the PDF document is located, the original image coding data in the original coding format corresponding to the image object is read from the memory, and the image coding data is temporarily stored in the memory buffer as the original image data.

[0095] Specifically, each page of the PDF document is traversed, page.get_images() is called to obtain a list of image objects in the page; for each image object, doc.extract_image(xref) is called to extract image encoding data in the original encoding format of the image object, the image encoding data being binary data of the image, including image width, image height, color space, channel bit number, image format of the image object, and the binary data is stored in a memory buffer as original image data.

[0096] In one possible implementation, the layout analysis model is constructed based on a YOLOv10 model.

[0097] In some embodiments, the layout analysis module is specifically configured to:

[0098] preprocess the image to be processed into an input image of a preset resolution;

[0099] extract a multi-scale feature map of the input image through a pre-trained YOLOv10 model;

[0100] perform feature fusion and enhancement on the multi-scale feature map to output an aggregated feature map;

[0101] perform region detection and classification based on the aggregated feature map to output a target detection region, wherein the target detection region includes a text region, a formula region, and a table region;

[0102] establish a hierarchical relationship topology of each target detection region, the hierarchical relationship including a parent-child relationship, a reference relationship, a containing relationship, and a peer relationship.

[0103] In actual implementation, the training process of the YOLOv10 model includes construction of training data, model training, and model testing.

[0104] It needs to be explained that the YOLOv10 model includes a local network Backbone, an aggregation network, a region detection network and a region level network, wherein the local network Backbone is used to extract multi-scale features of the input PDF document, and outputs four different scale feature maps, which are 64x64 feature map, 32x32 feature map, 16x16 feature map and 8x8 feature map; the aggregation network is used to perform feature fusion and enhancement on the different scale feature maps output by the local network Backbone, and outputs an aggregated feature map; the region detection network is used to perform region detection and classification based on the aggregated feature map output by the aggregation network, and outputs a region classification result and a region bounding box, wherein the region classification result includes a text region, a formula region and a table region, and a region bounding box of each region, and the region bounding box is represented as (x, y, w, h); the region level network is used to output the hierarchical relationship of each detected region, including parent-child relationship, reference relationship, containing relationship and peer relationship.

[0105] First, the public PDF document is obtained, and PDFium is used to convert the PDF document into a 512x512 page image according to the page; the LabelImg labeling software is used to label the boundary box coordinates of each region in the page image, the category of each region and the hierarchical relationship between the regions. Correspondingly, the text region, formula region, table region and image region are labeled, as well as the boundary box coordinates of each region and the hierarchical relationship between the regions.

[0106] The labeled data is divided into a training set, a validation set and a test set according to a ratio of 7:2:1. The training set is input into the YOLOv10 model for training, a loss function is constructed based on the model output result and the labeled data, and the model hyperparameters are optimized using the Adam optimizer. When the preset performance of the model reaches the preset requirement, the hyperparameters of the model are fixed, and the model is used as a layout analysis model.

[0107] It needs to be explained that the layout analysis model in the present application has the ability to identify image regions, but the present application does not use the YOLOv10 model to predict image regions, but only uses the YOLOv10 model to extract regions containing text, formulas and tables.

[0108] In one possible implementation, the first identification module 30 includes a text identification unit, a formula identification unit and a table identification unit.

[0109] The text identification unit is configured to perform text recognition on the text region by using an OCR algorithm to obtain a structured text recognition result.

[0110] The formula identification unit is configured to generate an explanatory text of the formula in the formula region by using a multi-modal large language model.

[0111] The table recognition unit is configured to perform type and element recognition on the table region by using a multi-modal large language model, to obtain data relationships of elements in the table.

[0112] In some embodiments, the text recognition unit is specifically configured to:

[0113] perform geometric correction processing on the text region to obtain a corrected image;

[0114] call two OCR engines in parallel to perform character recognition on the corrected image;

[0115] weight and fuse the character recognition results output by the two OCR engines to output a structured text recognition result.

[0116] Specifically, the divided text region can have problems of inclination and distortion, and therefore geometric correction processing is performed on the text region, a straight line and an average angle are detected by using a Hough transform, and the image is rotated to obtain a corrected image.

[0117] The two OCR engines are called in parallel, one of which uses PaddleOCR and the other uses pytesseract. The two OCR engines are called to perform character recognition on the corrected image to obtain PaddleOCR recognition results and pytesseract recognition results; the two recognition results are aligned in text lines, the weights of the two recognition results are assigned, and character-level fusion is performed. The weights of the two recognition results can be set according to actual requirements, which are not limited herein.

[0118] Illustratively, first, an image of a text region to be processed is read, the image of the text region is converted into a gray-scale image, a cv2.Canny function is called to perform boundary detection on the gray-scale image, and a Hough transform is used to detect straight lines within the boundary. The average angle of all straight lines within the boundary is calculated, the gray-scale image is corrected based on the calculated average angle, and a corrected image is output.

[0119] PaddleOCR is called to recognize the corrected image to obtain a plurality of text lines and a confidence degree corresponding to each text line; pytesseract is called to recognize the corrected image to obtain a plurality of text lines and a confidence degree corresponding to each text line. The text lines obtained by the recognition results of the two approaches are aligned, the two recognition results are fused character by character, the result with a high confidence degree is selected, and a final text recognition result is output.

[0120] In the actual processing process, after obtaining the text recognition result, the text recognition unit is also used for correcting and optimizing the text recognition result, including character correction on the text recognition result, hyphen splitting on the text recognition result, such as splitting "fi" into "fi", special Unicode character replacement; optimizing the text structure of the text recognition result, such as processing the end-of-line hyphen to ensure the coherence of the text content; in addition, advanced text correction can be performed by inputting the text recognition result and prompt into a large language model (LLM) to correct spelling and semantic errors.

[0121] In some embodiments, the formula recognition unit is specifically used for:

[0122] identifying and extracting formula content of the formula region;

[0123] converting the formula content into a LaTeX representation vector of the formula;

[0124] inputting the LaTeX representation vector into a multi-modal large language model to generate an explanatory text of the formula under a preset prompt.

[0125] Specifically, a Mathpix formula recognition engine is called to identify and extract formula content in the formula region; the identification result is mapped into a LaTeX command sequence, in which basic symbols are directly converted and complex structures generate nested commands.

[0126] A LaTeX representation vector is established based on the LaTeX command sequence, and it should be noted that the LaTeX representation vector is in the form of a LaTeX syntax tree, in which the root node is the entire formula, the child nodes represent sub-expressions, and the leaf nodes correspond to basic symbols.

[0127] The LaTeX syntax tree is input into a multi-modal large language model to generate an explanatory text of the formula under a preset prompt.

[0128] In a feasible implementation, the multi-modal large language model is obtained by fine-tuning a qwen-2.5-32B base model, a large number of LaTeX formulas of digital formulas, and explanatory texts distilled by a large language model (such as DeepSeek) are collected as labeled data to form training data; the training data also contains part of the explanatory texts of the LaTeX formulas manually annotated.

[0129] The LaTeX representation vector is input into the trained multi-modal large language model to obtain the basic definition and composition analysis of the formula output by the multi-modal large language model.

[0130] In some embodiments, the table recognition unit is specifically used for:

[0131] extract a table in a table region through a multi-modal large language model, and identify a table type;

[0132] analyze elements in the table and relationships between the elements based on the table type, and generate a data relationship of the table.

[0133] In actual implementation, a PDF document of the table region is input into the multi-modal large language model to obtain a target table obtained by a structure layout component of the multi-modal large language model, and the type of the target table is identified; data elements in the target table and relationships between the elements are analyzed through the multi-modal large language model, and a data relationship expressed by the table is determined based on the above information. In a feasible implementation, the multi-modal large language model is specifically obtained by fine-tuning training on the basis of a qwen-2.5-32B model.

[0134] In a possible implementation, the second identification module is specifically configured to:

[0135] call a decoder corresponding to the original encoding format to output an RGB pixel matrix, and save the RGB pixel matrix as a lossless format image;

[0136] identify a target region containing text in the lossless format image through a DB text detection algorithm;

[0137] perform text recognition on the target region by using an OCR algorithm to determine an image recognition result.

[0138] In actual implementation, a decoder is automatically matched according to an original encoding format of the embedded image, and is uniformly converted into an 8-bit depth RGB format and saved as a lossless PNG format. A target region containing text in the image is identified through a DB (Differentiable Binarization) text detection algorithm, an OCR algorithm is called for text recognition for the target region, and an image recognition result containing text content is determined.

[0139] Specifically, the presentation generation module includes a framework note generation unit, a chart formula generation unit, and a slide multi-modal unit. The framework note generation unit is configured to integrate the text recognition result and the corresponding semantic representation to generate the overall structure and content of the presentation. The chart formula generation unit is configured to integrate the formula recognition result, the table recognition result, the image recognition result, and the corresponding semantic representation to generate images, tables, and formulas for presentation in the presentation. The slide multi-modal unit is configured to design the layout, color, and font of the slides for different parts of the presentation to ensure overall style consistency, integrate different types of content such as text, images, tables, and formulas into the presentation, and optimize the presentation order and manner of the content in the presentation through the multi-modal large language model.

[0140] In some embodiments, the conversion device also includes a knowledge enhancement module for constructing a domain knowledge graph based on the keywords of the PDF document to be processed, using the domain knowledge graph to understand the content of the presentation in depth, and identifying professional terms appearing in the presentation and representing the professional terms in a concept-method-theory relationship graph structure.

[0141] In some embodiments, the conversion device also includes a user preference adaptation module for constructing a user preference model to record the generation preferences of different users for the presentation, including the style, content density, and professional term usage frequency of the presentation, color preference, and font preference, so that the presentation generation module generates the presentation based on the user preferences.

[0142] In actual implementation, the user preference adaptation module can also obtain the role type selected by the user when converting the document, and adjust the content generation strategy in the presentation generation module according to different role types. For example, the user with the role type of teacher is provided with more detailed explanation notes, and the user with the role type of enterprise speaker is provided with business guidance content.

[0143] The introduction of the user preference adaptation module enables the device to dynamically adjust the generation strategy of the presentation based on the role type of the user. The device can provide customized services for different users based on their usage habits. This adaptive closed-loop design and vertical domain knowledge support make the application have high practical value in professional scenarios such as scientific research reports and academic exchanges, and improve the efficiency of generating presentations based on academic literature for researchers.

[0144] An exemplary embodiment of the present application provides a conversion device for PDF documents, which is further improved based on the first exemplary embodiment, and the specific improvements are as follows:

[0145] The feedback module is configured to receive user modification opinions for the generated presentation, convert the modification opinions into modification instructions, and send the modification instructions to the presentation generation module based on the modification instructions to enable the presentation generation module to adjust the presentation based on the modification instructions.

[0146] Specifically, the feedback module is configured to receive user input modification opinions for the presentation, interpret the user's modification opinions through a large language model, including understanding the user's modification opinions, evaluating the impact of the user's feedback modification opinions on the structure of the presentation, and converting the user's modification opinions into structured JSON modification instructions. The modification instructions are fed back to the presentation generation module, and the presentation generation module adjusts the presentation based on the above modification instructions.

[0147] The feedback module can update the presentation according to the modification instruction, and can continuously learn and optimize the generation capability according to the user feedback. In actual implementation, the presentation generation module also maintains the version of the generated presentation, so as to facilitate the user to compare and select different versions of the presentation.

[0148] Yet another exemplary embodiment of the present application provides a conversion device for a PDF document, which is further improved on the basis of the first exemplary embodiment, and the specific improvements are as follows:

[0149] The pre-processing of the to-be-processed image includes removing the background and adjusting the contrast.

[0150] The removing the background includes removing the watermark, the letterhead or the decorative frame in the to-be-processed PDF document.

[0151] The adjusting the contrast includes converting the to-be-processed PDF document to the LAB color space, including the L channel, the A channel and the B channel.

[0152] The adaptive histogram equalization is performed on each channel of the LAB color space, the L channel, the A channel and the B channel are merged, and the BGR color space is converted back.

[0153] The pre-processing can effectively improve the processing accuracy of the subsequent modules.

[0154] Referring to Figure 2 Another embodiment of the present application also provides a conversion method for a PDF document, which is executed by using the conversion device for a PDF document mentioned in the above embodiments. The steps of the conversion method specifically include:

[0155] S1: receiving a PDF document, converting the PDF document into a to-be-processed image page by page, and obtaining original image data of an embedded image in the PDF document from the memory;

[0156] S2: dividing the to-be-processed image into regions, and determining a target detection region, the target detection region including a text region, a formula region and a table region;

[0157] S3: performing content recognition on the target detection region, and outputting a text recognition result, a formula recognition result and a table recognition result;

[0158] S4: performing content recognition on the original image data, and generating an image recognition result;

[0159] S5: generating semantic representations of the text recognition result, the formula recognition result, the table recognition result and the image recognition result by using a large language model;

[0160] S6: mapping the text recognition result, the formula recognition result, the table recognition result and the image recognition result combined with semantic representation into a template of the presentation to generate a final presentation.

[0161] In some embodiments, the original image data of the embedded image in the PDF document is obtained from the memory, comprising:

[0162] The image object in the PDF document is located, the image coding data in the original coding format corresponding to the image object is read from the memory, and the image coding data is temporarily stored in the memory buffer as the original image data.

[0163] Further, after generating the presentation, the application further comprises:

[0164] The modification opinion of the user for the generated presentation is received, the modification opinion is converted into a modification instruction, and the modification instruction is sent to the presentation generation module to make the presentation generation module adjust the presentation based on the modification instruction.

[0165] The specific limitations of the conversion method for the PDF document provided in the embodiment can be referred to the embodiments of the conversion device for the PDF document in the foregoing, which will not be repeated here.

[0166] The computer device provided in the embodiment of the application can comprise a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by the processor to make the processor execute the steps of the conversion method for the PDF document according to any one of the above embodiments.

[0167] The working process, working details and technical effects of the computer device provided in the embodiment can be referred to the embodiments of the conversion method for the PDF document in the foregoing, which will not be repeated here.

[0168] The computer readable storage medium provided in the embodiment of the application stores a computer program, and the computer program is executed by the processor to implement the steps of the conversion method for the PDF document according to any one of the above embodiments. The computer readable storage medium refers to a carrier for storing data, which can include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash disk and / or a memory stick, etc. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0169] The working process, working details and technical effects of the computer readable storage medium provided by the embodiment can be referred to the above-mentioned embodiment of the conversion method of the PDF document, and will not be described here.

[0170] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments of the methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0171] The technical features of the above embodiments can be combined in any manner. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present disclosure.

[0172] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is taken as an example for illustration. In actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the system is divided into different functional units or modules to complete all or part of the above-described functions.

Claims

1. A conversion device for PDF documents, characterized in that, The conversion device includes: The data processing module is configured to receive a PDF document, convert the PDF document into images to be processed page by page, and obtain the original image data of the embedded images in the PDF document from memory. The layout parsing module is configured to perform region segmentation on the image to be processed using a pre-trained layout parsing model, and determine the target detection region of the PDF document, wherein the target detection region includes text region, formula region and table region; The first recognition module is configured to perform text recognition on text areas to generate text recognition results, perform formula recognition on formula areas to generate formula recognition results, and perform data recognition on table areas to generate table recognition results. The second recognition module is configured to perform content recognition on the original image data and generate an image recognition result. The semantic parsing module is configured to generate semantic representations of text recognition results, formula recognition results, table recognition results, and image recognition results through a large language model; The presentation generation module is configured to combine text recognition results, formula recognition results, table recognition results, and image recognition results with semantic representation and map them into a presentation template to generate the final presentation.

2. The PDF document conversion device according to claim 1, characterized in that, The data processing module is specifically used to: locate image objects in a PDF document, read image encoding data in the original encoding format corresponding to the image object from memory, and temporarily store the image encoding data in a memory buffer as the original image data.

3. The PDF document conversion device according to claim 1, characterized in that, The layout parsing model is built based on the YOLOv10 model.

4. The PDF document conversion apparatus according to claim 3, characterized in that, The layout parsing module is specifically used for: The image to be processed is preprocessed into an input image with a preset resolution; Multi-scale feature maps of the input image are extracted using a pre-trained YOLOv10 model; Multi-scale feature maps are fused and enhanced to output an aggregated feature map; Based on the aggregated feature map, region detection and classification are performed to output the target detection region, which includes text region, formula region and table region. Establish a hierarchical relationship topology for each target detection region, including parent-child relationships, reference relationships, containment relationships, and sibling relationships.

5. The PDF document conversion apparatus according to claim 1, characterized in that, The first identification module includes: The text recognition unit is used to perform text recognition on text regions using the OCR algorithm to obtain structured text recognition results; The formula recognition unit is used to generate explanatory text for formulas in the formula area using a multimodal large language model; The table recognition unit is used to identify the table type and table elements of a table region using a multimodal large language model, and to obtain the data relationships between the elements in the table.

6. The PDF document conversion apparatus according to claim 5, characterized in that, The text recognition unit is specifically used for: Geometric correction is performed on the text region to obtain the corrected image; Two OCR engines are invoked in parallel to perform text recognition on the corrected image; The text recognition results from the two OCR engines are weighted and fused to output a structured text recognition result.

7. The PDF document conversion apparatus according to claim 5, characterized in that, The formula recognition unit is specifically used for: Identify and extract the formula content of the formula region; Convert the formula content into a LaTeX representation vector of the formula; The LaTeX representation vector is input into a multimodal large language model to generate explanatory text under preset prompts.

8. The PDF document conversion apparatus according to claim 5, characterized in that, The table recognition unit is specifically used for: The table in the table region is extracted using a multimodal large language model, and the table type is identified. Based on the table type analysis, the elements in the table and the relationships between the elements are generated to produce the data relationships of the table.

9. The PDF document conversion apparatus according to claim 2, characterized in that, The second identification module is specifically used for: Call the decoder corresponding to the original encoding format, output the RGB pixel matrix, and save the RGB pixel matrix as a lossless image; The DB text detection algorithm is used to identify target regions containing text in lossless format images. The target region is used to perform text recognition using an OCR algorithm to determine the image recognition result.

10. The conversion apparatus for PDF documents according to any one of claims 1 to 9, characterized in that, The conversion device further includes: The feedback module is used to receive user feedback on the generated presentation, convert the feedback into modification instructions, and send the modification instructions to the presentation generation module so that the presentation generation module can adjust the presentation based on the modification instructions.

11. The conversion apparatus for PDF documents according to any one of claims 1 to 9, characterized in that, The data processing module is also used for: Preprocessing is performed on the image to be processed, including background removal and contrast adjustment.

12. A method for converting PDF documents, characterized in that, The conversion method includes: Receive a PDF document, convert the PDF document page by page into images to be processed, and retrieve the original image data of the embedded images in the PDF document from memory; The image to be processed is divided into regions to determine the target detection region, which includes text region, formula region and table region; Perform content recognition on the target detection area and output text recognition results, formula recognition results, and table recognition results; Content recognition is performed on the original image data to generate image recognition results; Semantic representations of text recognition results, formula recognition results, table recognition results, and image recognition results are generated using a large language model; The text recognition results, formula recognition results, table recognition results, and image recognition results are combined with semantic representation and mapped to the presentation template to generate the final presentation.

13. The method for converting PDF documents according to claim 12, characterized in that, The step of retrieving the original image data of the embedded images in the PDF document from memory includes: Locate the image object in the PDF document, read the image encoding data in the original encoding format corresponding to the image object from memory, and temporarily store the image encoding data in the memory buffer as the original image data.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method of claim 12 or 13.

15. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the steps of the method of claim 12 or 13.

Images