PDF (Portable Document Format) conversion multiplexing method and device, computer equipment and storage medium

Abstract

The invention discloses a PDF (Portable Document Format) conversion multiplexing method and device, computer equipment and a storage medium. The method comprises the following steps: extracting metadata features, logic structure features and visual layout features of a PDF document; performing dynamic partitioning according to the type of the PDF document to obtain a plurality of document blocks, and performing hash processing on each document block to generate a corresponding hash value; performing first-level comparison on the PDF document and a PDF document in a historical record according to the metadata features; performing second-level comparison on the PDF document and a PDF document in a historical record according to the logic structure feature; performing third-level comparison on the hash value of each document block in the PDF document and the hash value of each document block in the PDF document in a historical record, and determining whether different document blocks exist in the PDF document or not; if yes, incremental conversion processing is conducted on the difference blocks, the difference blocks obtained after incremental conversion are spliced with the converted parts of the unchanged content in the corresponding PDF document in the historical record, and a complete conversion result is formed. The method greatly improves the conversion efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of document conversion processing, and in particular, to a PDF conversion and reuse method, apparatus, computer device, and storage medium. Background Art

[0002] There are some problems in the current PDF document conversion processing technology, mainly reflected in the limitations of the global hashing technology. Specifically, the global hashing technology has two major limitations: First, it is too sensitive to the modification of PDF document metadata (such as author information, creation time) or minor changes (such as adding blank pages). Even if the content of the document has not changed substantially, its hash value will still change completely, which makes it impossible to effectively reuse the converted parts of the unchanged content of the corresponding PDF document in the historical record; Second, different format versions of the same document (such as PDFs exported from Adobe InDesign and PDFs saved as from Word) will generate different hash values due to encoding differences. Although the content of the document is exactly the same, the hashing algorithm only calculates based on byte-level data and lacks the ability to recognize the semantic content of the document. Summary of the Invention

[0003] The purpose of the present invention is to provide a PDF conversion and reuse method, apparatus, computer device, and storage medium, aiming to solve the problem that the existing document conversion solutions cannot reuse the historical conversion results.

[0004] In a first aspect, an embodiment of the present invention provides a PDF conversion and reuse method based on feature comparison, including:

[0005] When a PDF document needs to be format-converted, extract the metadata features, logical structure features, and visual layout features of the PDF document;

[0006] Dynamically divide the PDF document into multiple document blocks according to the type of the PDF document, and perform hashing processing on each document block to generate corresponding hash values;

[0007] Perform a first-level comparison between the PDF document and the PDF documents in the historical record according to the metadata features;

[0008] After the first-level comparison passes, perform a second-level comparison between the PDF document and the PDF documents in the historical record according to the logical structure features;

[0009] After the second-level comparison passes, compare the hash values of the document blocks in the PDF document with the hash values of the document blocks in the PDF documents in the historical record at a third level to determine whether there are different document blocks in the PDF document;

[0010] If the document blocks with differences in the PDF document are used as differential blocks, perform incremental conversion processing on the differential blocks, and splice the differentially converted differential blocks with the converted parts of the unchanged content in the corresponding PDF document in the historical record to form a complete conversion result.

[0011] In a second aspect, an embodiment of the present invention further provides a PDF conversion reuse device based on feature comparison, which includes:

[0012] An extraction unit, configured to extract the metadata features, logical structure features, and visual layout features of the PDF document when format conversion of the PDF document is required;

[0013] A chunking unit, configured to dynamically chunk the PDF document according to the type of the PDF document to obtain a plurality of document chunks, and perform hashing processing on each of the document chunks to generate corresponding hash values;

[0014] A first-level comparison unit, configured to perform a first-level comparison of the PDF document with the PDF documents in the historical record according to the metadata features;

[0015] A second-level comparison unit, configured to perform a second-level comparison of the PDF document with the PDF documents in the historical record according to the logical structure features after the first-level comparison passes;

[0016] A third-level comparison unit, configured to, after the second-level comparison passes, compare the hash values of the document chunks in the PDF document with the hash values of the document chunks in the PDF documents in the historical record to determine whether there are document chunks with differences in the PDF document;

[0017] A conversion unit, configured to, if the document blocks with differences in the PDF document are used as differential blocks, perform incremental conversion processing on the differential blocks, and splice the differentially converted differential blocks with the converted parts of the unchanged content in the corresponding PDF document in the historical record to form a complete conversion result.

[0018] In a third aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for PDF conversion reuse based on feature comparison as described in the first aspect is implemented.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the processor is caused to execute the method for PDF conversion reuse based on feature comparison as described in the first aspect.

[0020] An embodiment of the present invention provides a PDF conversion and reuse method, apparatus, computer device, and storage medium. Through three-level comparison, this method can accurately identify the unchanged parts and different parts in a PDF document. For the unchanged parts, the converted results of the corresponding PDF document in the historical record are directly reused, avoiding repeated conversion, greatly improving the conversion efficiency. At the same time, only incremental conversion processing is performed on the different blocks, reducing the amount of data to be processed and further accelerating the conversion speed. Further, considering metadata features, logical structure features, and the hash values of document blocks comprehensively, the PDF document is compared from multiple dimensions, which can more accurately judge the similarity and difference of the document, thus ensuring the consistency of the converted document with the original document in content, structure, and format. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 It is a schematic flowchart of the PDF conversion and reuse method based on feature comparison provided by the embodiment of the present invention;

[0023] Figure 2 It is a schematic sub-flowchart of the PDF conversion and reuse method based on feature comparison provided by the embodiment of the present invention;

[0024] Figure 3 It is another schematic sub-flowchart of the PDF conversion and reuse method based on feature comparison provided by the embodiment of the present invention;

[0025] Figure 4 It is another schematic sub-flowchart of the PDF conversion and reuse method based on feature comparison provided by the embodiment of the present invention;

[0026] Figure 5 It is another schematic sub-flowchart of the PDF conversion and reuse method based on feature comparison provided by the embodiment of the present invention;

[0027] Figure 6 It is a schematic diagram of the PDF conversion and reuse apparatus based on feature comparison provided by the embodiment of the present invention;

[0028] Figure 7 It is a schematic diagram of the computer device provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0030] It should be understood that when used in this specification and the appended claims, the terms "comprises" and "comprising" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0031] It should also be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0032] It should be further understood that the term " / and" as used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0033] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a PDF conversion and reuse method based on feature comparison provided by an embodiment of the present invention. The method includes steps S101 to S106:

[0034] S101. When it is necessary to perform format conversion on a PDF document, extract the metadata features, logical structure features, and visual layout features of the PDF document;

[0035] In this embodiment, metadata is information describing the basic attributes of a document, including title, author, creation time, modification time, keywords, etc. Through metadata, the type, source, and purpose of a document can be quickly identified, facilitating subsequent processing. Recording the creation and modification times of a document helps to track the version history of the document. The keywords and descriptive information in metadata can be used for indexing and retrieval of the document, improving the search efficiency.

[0036] The logical structure refers to the organization method of the content in a PDF document, such as chapters, paragraphs, lists, tables, etc. During the format conversion process, the logical structure features can keep the logical structure of the document unchanged, ensuring that the converted document is easy to read and understand. Based on the logical structure, the content of the document can be automatically extracted, reorganized, and formatted, improving the processing efficiency.

[0037] Furthermore, visual layout refers to the arrangement of elements (such as text, images, tables, etc.) in a PDF document on a page. During the format conversion process, extracting visual layout features can keep the visual layout of the original document as unchanged as possible, ensuring that the converted document is visually consistent with the original document. Reasonable visual layout can improve the readability of the document and make it easier for readers to understand the document content.

[0038] It should be noted that after extracting the metadata features, logical structure features, and visual layout features of the PDF document, it is also necessary to remove the redundant information in the metadata, correct the errors in the logical structure, and optimize the unreasonableness in the visual layout. According to the requirements of the target format, the extracted feature information is converted to the corresponding format. The converted document is quality checked to ensure the accuracy and consistency of the metadata, logical structure, and visual layout.

[0039] In one embodiment, if Figure 2 As shown, in the step S101, the metadata features, logical structure features and visual layout features of the PDF document are extracted, including steps S201 to S203:

[0040] S201, extracting metadata features in the PDF document using a PDF parsing tool, wherein the metadata features include a generation tool, a creation software version, and a custom field;

[0041] S202, analyzing the title level, paragraph order, and cross-page content association of the PDF document, and constructing a document structure tree according to the title level, paragraph order, and cross-page content association as a logical structure feature;

[0042] S203: extracting spatial distribution histograms of the text area, image area, and blank area of the PDF document and using them as visual layout features.

[0043] In this embodiment, in step S201, when parsing a PDF document, a PDF parsing library is used to extract key information. For Python developers, libraries such as PyPDF2, pdfplumber, or PyMuPDF (fitz) can be selected; Java developers can use Apache PDFBox (an open-source Java library developed and maintained by the Apache Software Foundation, specifically for processing PDF documents); and C# developers have iTextSharp available. Through these parsing libraries, the metadata of the PDF document can be easily obtained, and the key fields include: generation tool (which can be obtained through the producer field, such as Adobe Acrobat Pro 2023), software version at creation (obtained through the creator field, such as Microsoft Word), and custom fields (i.e., non-standard fields, such as Author, Company, etc.).

[0044] Furthermore, in step S202, the extraction of logical structure features aims to deeply analyze the internal structure of the document. The specific objectives include analyzing the title hierarchy, clarifying the paragraph order, and handling content across pages, so as to construct a structure tree of the document. Specifically, first, the overall structure of the document needs to be parsed. For the title hierarchy, with the help of the PDF parsing library, extract the text with title features (such as H1, H2, etc. styles), and further infer its hierarchical attribution based on visual styles such as font size and boldness; second, when determining the paragraph order, parse the text blocks page by page, and record in detail the starting position (coordinate information) and specific content of each paragraph; finally, for content across pages, it is necessary to accurately detect whether a paragraph is truncated by pagination (that is, whether the paragraph text is distributed across two pages), and construct a complete document structure tree based on these analysis results.

[0045] Furthermore, in step S203, the extraction of visual layout features is mainly to extract the spatial distribution information of text regions, image regions, and blank regions in the document. After generating the spatial distribution information of each region, continue to generate the spatial distribution histogram of these elements. Specifically, first is the region division. For the text region, with the help of the PDF parsing library, extract the specific position of the text block; for the image region, extract the position and size information of the image object; at the same time, it is also necessary to calculate the blank region in the page, that is, the part not covered by text or images. Second is to construct the spatial distribution histogram. Use a specific algorithm to divide the page into grids (for example, 10x10 cells), and count the proportion of text, images, and blanks in each cell, so as to visually display the visual layout features of the document.

[0046] In one embodiment, as Figure 3 shown, step S202 includes steps S301 to S303:

[0047] S301. Detect the coordinates, font size, and indentation of text blocks in the PDF document, and determine the title level of the PDF document based on a rule engine;

[0048] S302. Construct a document structure tree, recording the chapter nesting relationship and the association index of cross-page content;

[0049] S303. Generate a structure description file in JSON format.

[0050] In this embodiment, step S301 uses an improved DOC algorithm (Document Order Construction algorithm). In the improved DOC algorithm, the Detect stage is the basis of the entire process, aiming to extract key structured information from the PDF document to provide data support for subsequent document structure sorting (Order) and document content construction (Construct). Specifically, the goal of the Detect stage is to extract the following information from the PDF document, including the coordinates, font size, and indentation of text blocks in the PDF document. The text block coordinates determine the position of each text block on the page (such as the coordinates of the upper left corner and the lower right corner); the indentation refers to the measured indentation distance of the text block relative to the starting position of the page or paragraph. Then, based on predefined rules, it is judged whether the text block is a title and its title level is determined. That is, for each text block, a rule engine is applied for judgment: check whether the font size meets the requirements of the title level. Check the alignment method (centered, left-aligned, right-aligned). Check the indentation (for example, H3 may have an indent). If the text block meets the rules of a certain level, it is marked as a title of that level. If the text block does not meet any title rules, it is marked as ordinary text.

[0051] Furthermore, in step S302, based on the analysis results of step S301, the structure tree of the document will be further constructed. This process not only involves recording the nesting relationship between chapters in the document to clearly show the hierarchical structure of the document, but also needs to specifically handle the association problem of cross-page content. By establishing an association index for cross-page content, the continuity and integrity of the document content are ensured.

[0052] Finally, in step S303, based on the information extracted and constructed in the previous two stages, a structure description file in JSON format will be generated. This file not only details the structured information such as the title level and chapter nesting relationship of the document, but may also contain key data such as the association index of cross-page content, providing great convenience for subsequent document processing, retrieval, or display.

[0053] In one embodiment, as Figure 4 shown, step S203 includes steps S401 - S402:

[0054] S401. Use OpenCV technology to perform layout segmentation on the PDF page, and identify the text area, image area, and blank area according to the segmentation results.

[0055] S402. Calculate the spatial distribution histograms of the identified regions to generate an n-dimensional feature vector.

[0056] In this embodiment, step S401 uses OpenCV technology to perform layout segmentation on the PDF page. First, the page is converted into an image format for analysis through image preprocessing techniques (such as binarization, edge detection, etc.), and then a layout segmentation algorithm based on geometric features (such as connected component analysis, projection contour analysis, etc.) is used to partition the page elements. It is divided into a text area (including editable text content such as paragraphs and headings, usually represented as a set of regularly arranged characters), an image area (including rasterized graphics such as illustrations and photos, with specific texture features), and a blank area (the blank part without valid information, which can be identified by pixel density).

[0057] In step S402, a feature vector is generated, and quantitative feature extraction is performed based on the layout information obtained in step S401. For the spatial distribution characteristics of all identified regions, a histogram statistical method is used for calculation. Specifically, first, a feature space including dimensions such as region type, area ratio, and relative position is established, and then through region clustering and spatial grid division, the distribution frequencies of different types of regions in each spatial unit of the page are statistically calculated. For example, the grid can be divided along the horizontal and vertical directions of the page, and the ratios of the text area, image area, and blank area in each grid are calculated respectively. Finally, these statistics are normalized and concatenated into an n-dimensional feature vector (such as a 16-dimensional feature vector). This spatial distribution histogram feature can not only retain the macroscopic layout characteristics of the page (such as the text and image mixing pattern) but also reflect the microscopic distribution law (such as the density gradient of text blocks).

[0058] It should be noted that when performing the operations of steps S401 and S402, the scanned PDF can be binarized to eliminate noise interference. Specifically, a hybrid strategy that comprehensively selects the global threshold (such as the Otsu algorithm) and the adaptive threshold (such as the Niblack or Sauvola algorithm) is adopted. For documents with uneven brightness, local adaptive threshold processing is used. At the same time, a multi-scale binarization method is combined to generate multiple candidate results, and the stability is improved through intersection or weighted averaging. After binarization, morphological operations (such as opening and closing operations) are used to remove isolated noise and fill text holes. At the same time, non-text regions with small areas are removed through connected component analysis, and edge detection (such as the Canny algorithm) is combined to ensure clear text edges. If the background has complex textures, the background can be smoothed first using guided filtering or bilateral filtering, and then the text skeleton is extracted through a morphological thinning algorithm (such as the Zhang-Suen algorithm), so as to reduce noise interference while retaining structural information. Secondly, local sensitive hashing (LSH) is used to compress the feature dimension. First, local features (such as SIFT, SURF, or ORB descriptors) or global features (such as HOG, color histogram) of the image need to be extracted, and the dimension is reduced to 50-100 dimensions through PCA or LDA to reduce redundant information. In the LSH implementation, a random projection method is used to generate hash functions. The conflict rate and computational efficiency are balanced by adjusting the number of projection vectors and the size of the hash table buckets. At the same time, a multi-hash table strategy (such as 10 independent hash tables) is combined to improve the retrieval robustness, and the Hamming distance or cosine similarity is used to measure the similarity of hash keys, so as to achieve efficient processing of the scanned PDF.

[0059] S102. Dynamically divide the PDF document into multiple document blocks according to the type of the PDF document, and perform hashing processing on each document block to generate corresponding hash values;

[0060] In this embodiment, a dynamic block division rule is used to perform dynamic block division processing on the PDF document. The core is to adaptively adjust the block division strategy according to the semantic structure and content type of the document to ensure the logical integrity and independence of each document block, thereby improving the accuracy and reusability of subsequent hashing processing.

[0061] Specifically, we first analyze the layout features (such as title hierarchy, paragraph distribution, proportion of tables / charts, etc.) and content features (such as keywords, term frequency, etc.) of the PDF document, and automatically determine its type (such as contracts, papers, or financial reports, etc.). For example, contract documents: identify legal text structures such as "clauses" and "chaps", and divide them into blocks according to the boundaries of legal clauses (such as "Article 1 Definitions" and "Article 2 Rights and Obligations" each in one block), ensuring that each block contains the complete content of the clause. Academic papers: Detect chapter titles such as "Abstract", "Introduction", and "Experimental Methods", divide them into blocks according to academic chapters, and retain the coherent semantics within the chapters. Financial statements: Locate the table area and the associated explanatory text, and treat each table and its context description as an independent block to avoid the separation of table data and explanatory text.

[0062] Furthermore, according to the matching block strategy, the block boundaries are precisely delineated by combining layout analysis (such as title position, paragraph indentation) and semantic analysis (such as punctuation rules, term continuity). For example, contract terms locate the block cutting point through iconic text such as "Article X"; paper chapters rely on title styles (such as bold fonts, numbering levels) to determine the block range; financial report tables associate adjacent table titles and annotation texts by identifying border lines or cell alignment features.

[0063] Furthermore, for each document block (which may be a text paragraph, table or image area), extract its content byte stream (text encoding or pixel data), and use a cryptographic hash algorithm (such as SHA-256) to generate a hash value for each document block. This process normalizes irrelevant differences (such as spaces and line breaks) to ensure that the hash values of the same content are consistent in different format versions, while retaining sensitivity to substantive modifications (such as changes in the text of the terms).

[0064] In summary, for different format versions of the same document (such as PDF to Word and original Word) with the same content, the block hash values remain consistent; when a single clause or chapter is modified, only the hash value of the corresponding document block is affected, and other unmodified document blocks can still reuse the historical processing results.

[0065] It should be noted that when dealing with dynamic chunking operations, problems with chunk exception handling are encountered, that is, the content fragmentation problem common in traditional document chunking, especially the semantic integrity damage caused by the mechanical splitting of large cross-page elements (such as tables, charts, multi-column texts). This embodiment will detect such situations and automatically merge the chunks to ensure the integrity of the content. Specifically, after initial chunking, potential abnormal chunks will be detected through the following rules. Cross-page element recognition: Analyze the content continuity at the end of the chunk and the beginning of the next page, such as whether a table is truncated by a page break (detect features such as repeated table headers and unclosed cells); Semantic fault analysis: Check whether the chunk boundary cuts off a complete semantic unit (such as half a sentence, an unfinished list item); Layout contradiction verification: Compare the physical splitting result with the document logical structure (such as the starting position of the chapter pointed to by the table of contents being misaligned with the actual chunk).

[0066] Furthermore, once an anomaly is detected, the following correction operations will be automatically triggered. Specifically, the upper and lower halves of the originally split table will be remerged into a single document chunk, and all the explanatory texts across pages will be associated with it; if the text chunk is interrupted in the middle of a sentence, the chunk boundary will be extended backward to the end punctuation of the sentence or the end of the paragraph; for multi-column layout documents, vertical chunking by column is preferred rather than horizontal cutting to avoid confusion of content between columns. The corrected document chunk will regenerate the hash value and mark the association relationship of the original chunks (such as "merged from chunk A, chunk B").

[0067] Specifically, as Figure 5 shown, step S102 includes steps S501 to S503:

[0068] S501. Pre-configure corresponding dynamic chunking rules according to different types of preset PDF documents;

[0069] In this embodiment, for different types of preset PDF documents (such as contracts, reports, papers, tables, etc.), different dynamic chunking rules need to be formulated according to their content structure characteristics. For example, contract documents usually contain fixed areas such as titles, clauses, and signatures, and the chunk boundaries can be defined based on text positions (such as headers, footers), keywords (such as "contract number", "signing date"), and paragraph length thresholds; table documents rely on row and column detection algorithms (such as table line recognition based on the projection method) combined with text alignment methods (such as left alignment, center alignment) to divide cell or sub-table areas; for academic papers, the format characteristics (font, font size, indentation) of chapter titles (such as "abstract", "introduction") can be used to construct a hierarchical chunking model. All rules need to be stored in the form of configurable parameters and support dynamic adjustment through a visual interface or configuration file to adapt to minor changes in the document.

[0070] S502. Determine the type of the PDF document through the NLP model, and call the corresponding dynamic chunking rule to perform content segmentation on the PDF document to obtain multiple document chunks;

[0071] In this embodiment, first, text extraction and semantic analysis are performed on the PDF document through a pre-trained NLP model. A BERT classifier can be used to determine the type of the document. The NLP model needs to be fine-tuned on a multi-type document corpus to learn the lexical features of different types of documents (such as "liability for breach of contract" in a contract, "research methods" in a paper), syntactic structures (such as the "if-then" sentence pattern in a clause), and discourse patterns (such as the "problem - analysis - suggestion" structure in a report). After identifying the document type, the pre-configured dynamic chunking rule is called to perform content segmentation on the document. For example, a contract document can be automatically divided into clause chunks according to clause numbers (such as "Article 1", "Article 2"), and a table document generates structured data chunks through row and column parsing. The chunking result needs to retain the logical relationships between chunks (such as parent-child nesting, sequential dependence), and output structured data containing chunk coordinates, type labels, and content summaries.

[0072] S503. Calculate the literal fingerprint hash value, structural fingerprint hash value, and layout fingerprint hash value of each document chunk respectively, and splice the literal fingerprint hash value, structural fingerprint hash value, and layout fingerprint hash value of each document chunk to generate a corresponding composite hash value.

[0073] In this embodiment, for each document chunk, the hash values of its literal fingerprint, structural fingerprint, and layout fingerprint are calculated respectively. Specifically, the literal fingerprint identifies whether the content has been deleted (for example, whether a paragraph has been deleted). The text in the document chunk is transformed into a feature vector through text vectorization technology, and then locality-sensitive hashing (LSH) is used to generate a fixed-length hash value to capture semantic similarity.

[0074] The structural fingerprint identifies whether the order has changed (for example, whether the chapter order has been swapped). By extracting the hierarchical structure of the document chunk (such as title - subtitle - body text), paragraph distribution (such as the number of paragraphs, average length), and the proportion of lists / tables, a unique identifier is generated using SHA-256 after feature encoding.

[0075] The layout fingerprint identifies whether the format has been adjusted (for example, whether the font has changed from Song typeface to Kai typeface). By analyzing visual features such as font type, font size, color, line spacing, etc. of the document chunk, combined with coordinate position information, a layout feature hash value is generated through a hash algorithm (such as MD5).

[0076] Finally, the above three types of hash values are concatenated in a preset order (such as "text fingerprint + structural fingerprint + layout fingerprint"), and a composite hash value is generated through secondary hashing (such as SHA-1) to form a globally unique identifier, ensuring uniqueness and irreversibility, and improving the conversion efficiency.

[0077] S103. Perform a first-level comparison between the PDF document and the PDF documents in the historical record according to the metadata features;

[0078] In this embodiment, the metadata features are equivalent to the "identity information" of the document, such as who created it, what software was used, the internal number, etc. Even if the content of the document remains unchanged, if the file name or save time is changed, the traditional method will have problems recognizing the document. However, this embodiment will additionally record these "identity tags" to avoid misjudgment.

[0079] Furthermore, the first-level comparison strategy is to perform hierarchical comparison based on the metadata features, and filter the historical records with the same producer field and matching document types. Specifically, first compare the unique identifier fields such as the document number and version number. If they match exactly, it is directly determined as a duplicate document. If the strong match fails, then compare the text fields such as the title and keywords. Calculate the similarity between the fields through the text similarity algorithm, and set a threshold (such as similarity ≥ 0.8) to determine as a potential duplicate. Generate a list of candidate duplicate documents according to the comparison rules and sort them according to the following priorities. The documents with exactly the same strong match fields are ranked at the top, and the weakly matched documents are arranged in descending order of similarity.

[0080] S104. After the first-level comparison passes, perform a second-level comparison between the PDF document and the PDF documents in the historical record according to the logical structure features;

[0081] In this embodiment, the second-level comparison strategy is based on the node similarity of the comparison document structure tree. Specifically, the node types include headings (such as H1 - H6), paragraphs, lists, tables, pictures, etc. Each node records its type, text content, hierarchical depth (such as chapter, sub-chapter), and relative position (such as page number, coordinates). Construct a structure tree through the hierarchical relationship of the headings. For example, "Chapter 1" contains "1.1 Background" and "1.2 Methods" to form node relationships. Use the Tree Edit Distance (TED) algorithm to compare the similarity of the two structure trees. If the TED of the two structure trees ≤ 2 (the preset tree edit distance threshold), it is determined that the logical structures are highly similar, and the document is marked as a duplicate document. If TED > 2, it is directly excluded and no further comparison is performed.

[0082] S105. After the second-level comparison passes, compare the hash values of each document block in the PDF document with the hash values of each document block in the PDF document in the historical record for a third-level comparison to determine whether there are different document blocks in the PDF document;

[0083] In this embodiment, the hash value (text + structure + layout) of each document block of the current PDF is compared one by one with the hash values of each document block of the PDF document in the historical record. If the hash values are the same, it is marked as "no difference block"; if the hash values are different, it is marked as "difference block".

[0084] S106. If there are different document blocks in the PDF document, use the different document blocks as differential blocks, perform incremental conversion processing on the differential blocks, and splice the incrementally converted differential blocks with the converted parts of the unchanged content in the corresponding PDF document in the historical record to form a complete conversion result.

[0085] In this embodiment, only the identified differential blocks are converted, rather than reprocessing the entire document. Specifically, start the Docker containerized conversion service (such as libreoffice-headless) for the differential blocks to obtain the incrementally converted differential blocks, and extract the converted parts of the unchanged content from the historical record. These parts have been processed in the previous conversion and do not need to be operated repeatedly. Determine the insertion position of the differential blocks in the conversion result according to the positions of the differential blocks in the PDF document to be converted, and insert the converted differential blocks into the corresponding positions in the historical record to obtain a complete converted document.

[0086] In summary, through the three-level comparison mechanism, incremental conversion, and result splicing, the efficient conversion and update of PDF documents are achieved. It avoids repeated processing of unchanged parts, improves processing efficiency, and ensures the consistency and integrity of the conversion result.

[0087] In one embodiment, the PDF conversion reuse method based on feature comparison further includes step S107:

[0088] S107. If there are no different document blocks in the PDF document, directly call the converted content of the PDF document in the historical record and use it as the conversion result.

[0089] In this embodiment, when the PDF document passes the three-level comparison mechanism (i.e., the first-level comparison, the second-level comparison, and the third-level comparison) and it is confirmed that all document blocks have not changed, the converted content of the PDF document in the historical record can be directly reused as the conversion result to avoid repeated processing and improve efficiency.

[0090] Specifically, confirm that the metadata features of the document and the logical structure features of the document have not changed, and the hash values of all document blocks are exactly the same as those of the corresponding document blocks in the PDF document in the historical record, indicating that the content is identical. Then it shows that the current document is exactly the same as the document in the historical record, and there is no need to re-convert. This step directly reuses the historical conversion result when it is determined that the documents are identical, achieving an efficient incremental processing logic and greatly improving the response speed.

[0091] In one embodiment, the PDF conversion reuse method based on feature comparison further includes step S108:

[0092] S108. Determine whether the differential block exists in other positions of the PDF document in the historical record according to the visual layout feature. If so, swap the converted content at the corresponding position in the corresponding PDF document in the historical record to obtain the conversion result.

[0093] In this embodiment, when converting a scanned PDF document (such as a contract) to Word, the method in the prior art may mess up the signature position. However, this embodiment will determine whether the differential block has appeared in other positions in the historical record (even if the content is different, but the visual layout features are similar). If it exists, reuse the conversion result at the corresponding position in the historical record to avoid repeated conversion.

[0094] Specifically, perform feature extraction on the differential block to extract the spatial layout feature, size ratio feature, and style feature of the differential block. The spatial layout feature represents the position of the differential block in the page of the PDF document (such as the upper left corner, bottom, etc.); the size ratio represents the width, height, and length ratio of the differential block; the style feature represents visual attributes such as font, color, and border. Search in the conversion result of the historical record to see if there are other blocks with similar visual layout features. If a matching block is found, swap the converted content of this block in the historical record to the position of the current differential block. After swapping, adjust the context association (such as links, references, etc.) between the block content and the current document, and splice the swapped block with the conversion result of the unchanged part to form a complete document.

[0095] It should be noted that if it is determined according to the visual layout feature that the differential block does not exist in other positions of the PDF document in the historical record, that is, no matching block is found, then incremental conversion processing needs to be performed on the differential block, and splice the incrementally converted differential block with the part of the unchanged content converted in the corresponding PDF document in the historical record to form a complete conversion result.

[0096] In summary, through the cross-position matching mechanism of the visual layout feature, efficient reuse and position swapping of the differential block are achieved, improving the document conversion efficiency and reducing resource consumption.

[0097] In one embodiment, the PDF conversion and reuse method based on feature comparison further includes step S109:

[0098] S109. When there are character recognition errors in the scanned PDF document, automatically correct them through context semantic checking.

[0099] In this embodiment, when encountering a document with dense charts (such as a product manual), an intelligent chunking strategy needs to be adopted: when a dense chart area is detected, automatically switch to the "chunk by chart" mode; if it is found that the user frequently modifies the content of a certain area, then separate this area into a single chunk for subsequent rapid update. For possible errors in the character recognition of the scanned PDF, automatic correction can be performed through the context semantic checking mechanism (for example, intelligently correct "3D drawing machine" to "3D printer"); when the chunk comparison fails, the system will degrade to only compare the text and layout features, and still maintain a content reuse rate of more than 70%.

[0100] As Figure 6 shown, an embodiment of the present invention also provides a PDF conversion and reuse device based on feature comparison, including: an extraction unit 601, a chunking unit 602, a first-level comparison unit 603, a second-level comparison unit 604; a third-level comparison unit 605; a conversion unit 606.

[0101] The extraction unit is used to extract the metadata features, logical structure features, and visual layout features of the PDF document when format conversion of the PDF document is required;

[0102] The chunking unit is used to dynamically chunk the PDF document according to the type of the PDF document to obtain a plurality of document chunks, and perform hashing processing on each document chunk to generate a corresponding hash value;

[0103] The first-level comparison unit is used to perform a first-level comparison of the PDF document with the PDF documents in the historical record according to the metadata features;

[0104] The second-level comparison unit is used to perform a second-level comparison of the PDF document with the PDF documents in the historical record according to the logical structure features after the first-level comparison passes;

[0105] The third-level comparison unit is used to perform a third-level comparison of the hash values of each document chunk in the PDF document with the hash values of each document chunk in the PDF documents in the historical record after the second-level comparison passes, and determine whether there are different document chunks in the PDF document;

[0106] A conversion unit, which, if the document blocks with differences in the PDF document are used as differential blocks, performs incremental conversion processing on the differential blocks, and splices the differentially converted differential blocks with the converted parts of the unchanged content in the corresponding PDF document in the historical record to form a complete conversion result.

[0107] The PDF conversion and reuse device based on feature comparison provided by the embodiments of the present invention can accurately identify the unchanged parts and the different parts in the PDF document through three-level comparison. For the unchanged parts, the converted results of the corresponding PDF documents in the historical record are directly reused, avoiding repeated conversion, greatly improving the conversion efficiency. At the same time, only incremental conversion processing is performed on the differential blocks, reducing the amount of data to be processed and further accelerating the conversion speed. Further, considering the metadata features, logical structure features, and hash values of the document blocks comprehensively, the PDF document is compared from multiple dimensions, and the similarity and difference of the document can be judged more accurately, so as to ensure the consistency of the converted document with the original document in content, structure, and format.

[0108] The embodiments of the present invention also provide a PDF conversion and reuse device based on feature comparison, which can be implemented in the form of a computer program, and the computer program can run on a computer device as shown in Figure 7 shown.

[0109] Please refer to Figure 7 , Figure 7 which is a schematic block diagram of the computer device provided by the embodiments of the present invention. The computer device 700 is a server, and the server can be an independent server or a server cluster composed of multiple servers.

[0110] Refer to Figure 7 , the computer device 700 includes a processor 702, a memory, and a network interface 705 connected through a system bus 701. Among them, the memory can include a non-volatile storage medium 703 and an internal memory 704.

[0111] The non-volatile storage medium 703 can store an operating system 7031 and a computer program 7032. When the computer program 7032 is executed, the processor 702 can be made to execute the PDF conversion and reuse method based on feature comparison.

[0112] The processor 702 is used to provide computing and control capabilities to support the operation of the entire computer device 700.

[0113] The internal memory 704 provides an environment for the operation of the computer program 7032 in the non-volatile storage medium 703. When the computer program 7032 is executed by the processor 702, the processor 702 can be made to execute the PDF conversion and reuse method based on feature comparison.

[0114] The network interface 705 is used for network communication, such as providing data information transmission, etc. Those skilled in the art can understand that Figure 7 the structure shown in Figure 7 is only a block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the computer device 700 to which the solution of the present invention is applied. The specific computer device 700 may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0115] Those skilled in the art can understand that Figure 7 the embodiments of the computer device shown in Figure 7 do not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements. For example, in some embodiments, the computer device may only include a memory and a processor. In such an embodiment, the structures and functions of the memory and the processor are the same as those in Figure 1 the embodiment shown, and will not be described in detail here.

[0116] It should be understood that in the embodiments of the present invention, the processor 702 may be a central processing unit (CPU), and the processor 702 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0117] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be described in detail here. Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the compositions and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0118] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods, or units with the same function can be aggregated into one unit. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0119] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present invention.

[0120] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0121] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), magnetic disks, or optical discs that can store program codes.

[0122] As described above, the above are only specific implementation manners of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A PDF conversion and reuse method based on feature comparison, characterized in that Including: When it is necessary to perform format conversion on a PDF document, extract the metadata features, logical structure features, and visual layout features of the PDF document; Dynamically divide the PDF document into multiple document blocks according to the type of the PDF document, and perform hashing processing on each document block to generate corresponding hash values; Perform a first-level comparison between the PDF document and the PDF documents in the historical record according to the metadata features; After passing the first-level comparison, perform a second-level comparison between the PDF document and the PDF documents in the historical record according to the logical structure features; After passing the second-level comparison, compare the hash values of each document block in the PDF document with the hash values of each document block in the PDF documents in the historical record to determine whether there are different document blocks in the PDF document; If so, use the different document blocks in the PDF document as differential blocks, perform incremental conversion processing on the differential blocks, and splice the incrementally converted differential blocks with the unchanged parts of the corresponding PDF documents in the historical record after conversion to form a complete conversion result.

2. The method for PDF conversion and reuse based on feature comparison according to claim 1, wherein Also including: If there are no different document blocks in the PDF document, directly call the content after conversion of the PDF document in the historical record and use it as the conversion result.

3. The method for PDF conversion and reuse based on feature comparison according to claim 1, wherein Also including: Judge whether the differential blocks exist in other positions of the PDF documents in the historical record according to the visual layout features. If so, swap the corresponding positions of the content after conversion of the corresponding PDF documents in the historical record to obtain the conversion result.

4. The method for PDF conversion and reuse based on feature comparison according to claim 1, wherein The extraction of the metadata features, logical structure features, and visual layout features of the PDF document includes: Use a PDF parsing tool to extract the metadata features in the PDF document, where the metadata features include the generation tool, the created software version, and custom fields; Analyze the title hierarchy, paragraph order, and cross-page content association of the PDF document, and construct a document structure tree according to the title hierarchy, paragraph order, and cross-page content association and use it as the logical structure feature; Extract the spatial distribution histograms of the text regions, image regions, and blank regions of the PDF document and use them as visual layout features.

5. The method for PDF conversion and reuse based on feature comparison according to claim 4, wherein The analysis of the title hierarchy, paragraph order, and cross-page content association of the PDF document, and the construction of a document structure tree according to the title hierarchy, paragraph order, and cross-page content association and use it as the logical structure feature includes: Detect the coordinates, font sizes, and indentation amounts of the text blocks in the PDF document, and judge the title level of the PDF document based on a rule engine; Construct a document structure tree, record the chapter nesting relationship and the association index of cross-page content; Generate a structure description file in JSON format.

6. The method for PDF conversion and reuse based on feature comparison according to claim 4, wherein The extraction of the spatial distribution histograms of the text regions, image regions, and blank regions of the PDF document and use them as visual layout features includes: Use OpenCV technology to perform layout segmentation on the PDF pages, and identify text regions, image regions, and blank regions according to the segmentation results; Calculate the spatial distribution histograms of the identified regions respectively, and generate an n-dimensional feature vector.

7. The method for PDF conversion and reuse based on feature comparison according to claim 1, wherein Dynamically partitioning the PDF document according to the type of the PDF document to obtain a plurality of document blocks, and performing hashing processing on each of the document blocks to generate corresponding hash values, including: Pre-configuring corresponding dynamic partitioning rules according to the types of different preset PDF documents; Judging the type of the PDF document through an NLP model, and calling the corresponding dynamic partitioning rule to perform content segmentation on the PDF document to obtain a plurality of document blocks; Calculating the text fingerprint hash value, the structure fingerprint hash value, and the layout fingerprint hash value of each of the document blocks respectively, and splicing the text fingerprint hash value, the structure fingerprint hash value, and the layout fingerprint hash value of each of the document blocks to generate a corresponding composite hash value.

8. A PDF conversion and reuse device based on feature comparison, characterized in that Including: An extraction unit, configured to extract the metadata features, logical structure features, and visual layout features of the PDF document when format conversion of the PDF document is required; A partitioning unit, configured to dynamically partition the PDF document according to the type of the PDF document to obtain a plurality of document blocks, and perform hashing processing on each of the document blocks to generate corresponding hash values; A first-level comparison unit, configured to perform a first-level comparison between the PDF document and the PDF documents in the historical record according to the metadata features; A second-level comparison unit, configured to perform a second-level comparison between the PDF document and the PDF documents in the historical record according to the logical structure features after the first-level comparison passes; A third-level comparison unit, configured to, after the second-level comparison passes, compare the hash values of the document blocks in the PDF document with the hash values of the document blocks in the PDF documents in the historical record at the third level to determine whether there are different document blocks in the PDF document; A conversion unit, configured to, if so, use the different document blocks in the PDF document as differential blocks, perform incremental conversion processing on the differential blocks, and splice the incrementally converted differential blocks with the unchanged content-converted parts in the corresponding PDF documents in the historical record to form a complete conversion result.

9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the PDF conversion reuse method based on feature comparison according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the PDF conversion reuse method based on feature comparison according to any one of claims 1 to 7.

Images