Method for segmenting and complementing complete semantic paragraphs from PDF file based on article structure tree

By generating article structure trees and performing semantic similarity pruning technology, complete semantic paragraphs are extracted from PDF files, and the difficulty of paragraph boundary recognition and redundant information caused by neglecting structural information in the existing technology is solved, and high-quality paragraphs are achieved extract.

CN118966236APending Publication Date: 2024-11-15WEICHENG INTELLIGENT POWER TECH (HANGZHOU) CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411151858.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The prior art ignores the structural information of the document when extracting paragraphs from PDF files, resulting in the inability to accurately identify paragraph boundaries, lose subtitle content, and generate redundant or incorrect paragraph content.

Method used

By generating an article structure tree, combining context analysis, structure analysis and text content analysis, the missing parent nodes in the structure tree are completed, and semantic similarity pruning technology uses the subtitle as the root node to traverse the child nodes to synthesize a complete semantic paragraph.

Benefits of technology

Ensure the integrity and structural clarity of paragraphs, avoid the problem of incoherence of text content, shorten the search time, filter out redundant information, and obtain clear semantic and complete paragraphs, suitable for large-scale model retrieval enhancement generation and model training High-quality background knowledge of data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966236A_ABST
    Figure CN118966236A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of large model retrieval enhancement generation, and solves the problems that paragraph boundaries are difficult to accurately identify and the generation efficiency is high due to neglecting text structure information, simple segmentation rules and neglecting small titles in the prior art. The method for segmenting and complementing the complete semantic paragraph from the PDF file based on the article structure tree comprises the steps that the article structure tree is generated according to the PDF file, parent nodes missing in the article structure tree are complemented, subtitles in the PDF file are searched, and the subtitles are segmented and complemented according to the subtitles. According to the method, the integrity of the paragraph and the clarity of the structure can be guaranteed, meanwhile, the search time is shortened, and redundant information generated due to errors of an article structure tree and the like is filtered out.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of large model retrieval enhancement generation, and in particular to a method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree. Background Art

[0002] In the large-model retrieval-augmented generation (RAG) scenario, extracting text paragraphs from PDF files can provide an external knowledge source for large models, enabling them to generate accurate and contextual answers while reducing model hallucinations.

[0003] There are two main methods for generating paragraphs from PDF files in the prior art:

[0004] 1. Divide the paragraphs according to the order of the article content:

[0005] (1) PDF parsing: First, use a PDF parsing tool (such as PyPDF2, PDFMiner, Tika, etc.) to extract text information from the PDF file and convert it into a processable format such as plain text, HTML, or XML.

[0006] (2) Sentence segmentation: Then, the text is divided into separate sentences by finding the boundaries of the sentences.

[0007] (3) Text segmentation: Finally, the text is divided into paragraphs according to certain rules. This includes division according to word count or frequency of keyword occurrence. For example, if a paragraph contains a large number of keywords, then this paragraph may contain important information.

[0008] 2. Parse PDF and divide paragraphs into hierarchical layout information (the most advanced method in the prior art, such as LayoutPDFReader, etc.):

[0009] (1) Realize the recognition of tables, charts and texts in PDF.

[0010] (2) Realize the division of chapters, subsections and paragraphs, and realize the linking of chapters and paragraphs.

[0011] (3) Form lists and nested lists for the text content to realize the structure tree generated by the text content.

[0012] The defects of the method for generating paragraphs from PDF files in the prior art are mainly:

[0013] 1. The traditional article content sequential segmentation method mainly relies on the text content and some rules, but ignores the structural information of the text. PDF files usually contain rich structural information, such as titles, subtitles, lists, tables, etc. This structural information can help us identify the logical structure and paragraph boundaries of the document more accurately.

[0014] 2. Unable to accurately identify paragraph boundaries: Traditional segmentation methods usually rely on some simple rules, such as word limit or keyword frequency. However, these rules cannot accurately identify paragraph boundaries.

[0015] 3. Loss of subheading content: In PDF files, subheadings are usually used to mark the beginning of a new topic or subtopic. However, traditional segmentation methods may ignore these subheadings, resulting in the loss of important information.

[0016] 4. The current methods of parsing PDF to generate document structure tree, such as LayoutPDFReader technology, etc., although they realize the division of paragraphs in the way of article structure tree, only generate levels and paragraphs. The generation of paragraphs does not consider the semantic structure, so that the generation of paragraphs depends only on the level of the structure tree, which will make the paragraph content redundant and contain some erroneous information. Summary of the invention

[0017] The purpose of this application is to overcome the problems in the prior art that it is difficult to accurately identify paragraph boundaries and cannot identify redundant and erroneous information in paragraph content due to ignoring text structure information, simple segmentation rules, and ignoring subheadings, and to provide a method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree.

[0018] In a first aspect, a method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree is provided, comprising:

[0019] Get the PDF file of the article to be processed;

[0020] Generate an article structure tree according to the PDF file to mark the content of the PDF file in at least one form of directory, table, picture, page number, title, parent node, child node, and root node;

[0021] Completing the missing parent nodes in the article structure tree through context analysis, structure analysis, text content analysis and predetermined rules;

[0022] Search the subheadings in the PDF file;

[0023] Taking the subheading as the root node, all child nodes of each subheading are traversed, and nodes are pruned according to semantic similarity to synthesize a complete semantic paragraph.

[0024] Furthermore, the PDF file is generated into an article structure tree through the LayoutPDFReader method.

[0025] Further, the missing parent node in the article structure tree is completed by context analysis, structure analysis, text content analysis and predetermined rules, including at least one of the following methods:

[0026] According to the coherence, known information and context of the parent nodes, the missing parent nodes are completed using the majority interpolation method;

[0027] If the previous and next parent nodes of a missing parent node are inconsistent, the missing value is filled based on the nesting of the parent node values ​​and a sequence or sequential filling method;

[0028] Complete the missing parent nodes according to the predetermined rules and keywords.

[0029] Further, searching for subheadings in the article to be processed includes at least one of the following methods:

[0030] Determine subheadings based on titles in the article tree structure;

[0031] Determine subheadings based on preset keywords and node hierarchical positions;

[0032] Determine the subheadings based on the hierarchical results in the article structure tree.

[0033] Furthermore, all child nodes of each subheading are traversed with the subheading as the root node, and nodes are pruned according to semantic similarity to synthesize a complete semantic paragraph, including:

[0034] Taking each subheading as the root node, traverse all the child nodes of each subheading through the breadth-first search or depth-first search method, and prune them according to the semantic similarity;

[0035] The semantic similarity is obtained by calculating the embeddings of the sentences of each layer of nodes and the previous layer of nodes;

[0036] Set a similarity threshold. If the semantic similarity between nodes is higher than the set similarity threshold, the node will be retained. If the semantic similarity between nodes is lower than or equal to the set similarity threshold, the corresponding node will be pruned.

[0037] The article structure tree obtained after pruning is merged into a complete semantic paragraph using a recursive algorithm.

[0038] Furthermore, during the expansion of each level node in the depth-first search, the nodes are screened and pruned according to the set similarity threshold. Nodes with similarity lower than the similarity threshold are pruned and no longer enter the search queue. For the retained nodes, the depth-first search continues and the pruning operation is repeated until there are no more child nodes.

[0039] In a second aspect, a device for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree is provided, comprising:

[0040] The acquisition module is used to obtain the PDF file of the article to be processed;

[0041] A structure tree generation module, used to generate an article structure tree according to the PDF file, so as to mark the content of the PDF file in at least one form of directory, table, picture, page number, title, parent node, child node, and root node;

[0042] A parent node completion module, used to complete the missing parent nodes in the article structure tree through context analysis, structure analysis, text content analysis and predetermined rules;

[0043] A subtitle search module, used to search for subtitles in the PDF file;

[0044] The node pruning module is used to traverse all child nodes of each subheading with the subheading as the root node, and prune the nodes according to the semantic similarity to synthesize a complete semantic paragraph.

[0045] According to a third aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0046] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable medium stores a program code for execution by a device, wherein the program code includes steps for executing the method in any one of the implementations of the first aspect described above.

[0047] In a fifth aspect, an electronic device is provided, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements a method as in any one of the implementations in the first aspect above.

[0048] The present application has the following beneficial effects: the present application generates an article structure tree based on a pdf file, completes the parent nodes of the tree structure through context analysis and structural analysis in the article structure tree, and obtains complete semantic paragraphs by traversing and searching the subheading parent nodes and pruning them based on semantic similarity, which can ensure the integrity of the paragraphs and the clarity of the structure. The data completion processing based on the article structure tree can avoid the incoherence of the text content due to the missing data in the article structure tree; searching with the subheading as the root node and pruning based on semantic similarity at the same time limits the clear topic for the paragraph content, shortens the search time, filters out redundant information generated due to article structure tree errors and other reasons, and obtains paragraphs with clear semantics and completeness, which provides high-quality and professional background knowledge for the generation of large model retrieval enhanced generation (RAG) and model training data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application.

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 It is a flow chart of a method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree according to Example 1 of the present application;

[0052] Figure 2 It is a structural block diagram of a device for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree according to Embodiment 2 of the present application;

[0053] Figure 3 It is a schematic diagram of the internal structure of the electronic device of Example 5 of the present application.

[0054] Reference numerals:

[0055] 100, acquisition module; 200, structure tree generation module; 300, parent node completion module; 400, subtitle search module; 500, node pruning module. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Example 1

[0058] Embodiment 1 of the present application involves a method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree, comprising: obtaining a PDF file of an article to be processed; generating an article structure tree based on the PDF file to mark the content of the PDF file in at least one form of a directory, a table, a picture, a page number, a title, a parent node, a child node, and a root node; completing missing parent nodes in the article structure tree through context analysis, structure analysis, text content analysis, and predetermined rules; searching for subheadings in the PDF file; traversing all child nodes of each subheading with the subheading as a root node, and pruning nodes according to semantic similarity to synthesize complete semantic paragraphs. The method generates an article structure tree based on a PDF file, completes parent nodes of the tree structure from the article structure tree through context analysis and structure analysis, and obtains complete semantic paragraphs by traversing and searching the subheading parent nodes and pruning them according to semantic similarity, thereby ensuring the integrity of the paragraph and the clarity of the structure.

[0059] Specifically, Figure 1 A flow chart of a method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree in application embodiment 1 is shown, including:

[0060] S10, obtaining the PDF file of the article to be processed;

[0061] S20, generating an article structure tree according to the PDF file, so as to mark the content of the PDF file in at least one form of a directory, a table, a picture, a page number, a title, a parent node, a child node, and a root node;

[0062] Specifically, the LayoutPDFReader method is used to generate a document tree structure from the PDF file, and the article content is marked in the form of directory, table, picture, page number, title, parent node, child node, root node, etc. Among them, the LayoutPDFReader method is used to read the layout information of the PDF file and convert it into a structured form such as: DocTree (i.e., article structure tree).

[0063] S30, completing the missing parent node in the article structure tree through context analysis, structure analysis, text content analysis and predetermined rules, specifically including at least one of the following methods:

[0064] (1) According to the coherence of the parent node, known information and context, the majority interpolation method is used to complete the missing parent nodes. For example, as shown in Table 1 below, in 10 consecutive sentences, if the parent node is the same before and after, and only the parent node is missing in the middle 2-3 contents, the missing parent node (i.e., parent_id) is filled with the same parent node: ac2b8.

[0065] Table 1:

[0066]

[0067] (2) If the parent nodes before and after the missing value of the parent node are inconsistent, the missing value is filled based on the nesting of the parent node values ​​and the sequence or sequential filling method to ensure the continuity and logic of the sequence. For example, as shown in Table 2 below, if the parent nodes before and after the missing value have a logical relationship, the missing value is filled based on the next level parent node.

[0068] Table 2:

[0069]

[0070]

[0071] (3) Fill in the form according to page numbers, formulas and other type rules and keywords. For example, as shown in Table 3 below, if a statement begins with keywords such as "the above formula" or "where", the parent node of the formula is supplemented with the parent node of the current statement: 5c071.

[0072] Table 3:

[0073] text id parent_id Px≥P·f b7b In the above formula, Px is the nominal input power of the reducer in kW. 9b67e 5c071

[0074] In the above embodiment, the incomplete parts in the structure tree are supplemented by context analysis, structure analysis, text content analysis, and rule and pattern matching methods, which can ensure the integrity of the paragraphs and the clarity of the structure. The data completion processing based on the article structure tree can avoid the incoherence of the text content due to missing data in the article structure tree.

[0075] S40, searching for subtitles in the PDF file, specifically including at least one of the following methods:

[0076] (1) Determine the subheadings based on the tags (titles, etc.) in the article tree structure.

[0077] (2) Determine subheadings based on keywords (Chapter *, Section *, etc.) and the hierarchical position of the node.

[0078] (3) In the article tree structure, find the subheading according to the hierarchical structure. For example, if the current node is the same as the parent node of other subheadings that have been determined, the current node is considered to be a subheading as well.

[0079] S50, taking the sub-heading as the root node, traversing all the child nodes of each sub-heading, and pruning the nodes according to the semantic similarity to synthesize a complete semantic paragraph, specifically including the following steps:

[0080] (1) Taking each subheading as the root node, use the breadth-first search (BFS) or depth-first search (DFS) method to traverse all child nodes of each subheading, and prune them according to semantic similarity.

[0081] (2) The semantic similarity is obtained by calculating the embeddings of the sentences of each layer of nodes and the previous layer of nodes. Embeddings (i.e., embedded representations) play an important role in fields such as deep learning and natural language processing. They convert raw data (such as text, images, etc.) into vectors of fixed dimensions, which facilitates mathematical calculations and model training.

[0082] (3) Set a similarity threshold to determine whether the semantic similarity between nodes is high enough to determine whether to retain or prune the node. Only when the semantic similarity between nodes is higher than the set similarity threshold will the node be retained, otherwise pruning will be performed.

[0083] (4) During the depth-first search, during the expansion of nodes at each level, nodes are screened and pruned according to a pre-set similarity threshold. Nodes whose semantic similarity is lower than the similarity threshold can be pruned and no longer enter the search queue.

[0084] (5) For the retained nodes, continue the depth-first search and repeat the above pruning operation until there are no more child nodes.

[0085] (6) Depending on the actual situation and search results, it may be necessary to adjust the similarity threshold to balance the search efficiency and the quality of the search results.

[0086] (7) The obtained article structure tree is merged into text paragraphs using a recursive algorithm to obtain a complete semantic paragraph.

[0087] An example of a complete semantic paragraph is as follows:

[0088] The selection of fuses and fuses

[0089] Only when the fuse and the cutout are correctly selected can they play a protective role. Generally, according to the needs of the protected circuit, the specifications of the fuse are selected first, and then the specifications of the fuse are determined according to the specifications of the fuse. 1. Selection of the rated current of the fuse (1) For short-circuit protection of resistive load circuits such as lighting and electric heating equipment, the rated current of the fuse should be slightly greater than or equal to the rated current of the load. (2) Since the starting current of the motor is very large, it must be considered that the fuse cannot be broken during starting, so the rated current of the fuse is selected to be larger. Single motor: Fuse rated current = (1.5~2.5) × motor rated current. Multiple motors: Fuse rated current = (1.5~2.5) × rated current of the motor with the largest capacity + the sum of the rated currents of the remaining motors. Step-down starting motor: Fuse rated current = (1.5~2.0) × motor rated current. DC motor and wound-rotor motor: Fuse rated current = (1.2~1.5) × motor rated current. 2. Fuse selection The rated voltage and rated current of the fuse should be greater than or equal to the rated voltage of the circuit and the rated current of the installed fuse. The type of fuse is determined according to the circuit requirements and installation conditions.

[0090] In the above embodiment, the search is performed with the subheading as the root node and the branches are pruned according to the semantic similarity, which defines a clear theme for the paragraph content, shortens the search time, filters out redundant information generated due to errors in the article structure tree, and obtains a complete paragraph with clear semantics.

[0091] Example 2

[0092] like Figure 2 As shown, a device for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree according to Embodiment 2 of the present application includes:

[0093] An acquisition module 100 is used to acquire a PDF file of an article to be processed;

[0094] A structure tree generation module 200 is used to generate an article structure tree according to the PDF file, so as to mark the content of the PDF file in at least one form of directory, table, picture, page number, title, parent node, child node, and root node;

[0095] A parent node completion module 300 is used to complete the missing parent nodes in the article structure tree through context analysis, structure analysis, text content analysis and predetermined rules;

[0096] A subtitle search module 400, used to search for subtitles in the PDF file;

[0097] The node pruning module 500 is used to traverse all the child nodes of each subheading with the subheading as the root node, and prune the nodes according to the semantic similarity to synthesize a complete semantic paragraph.

[0098] It should be noted that other specific implementations of the device for segmenting and completing complete semantic paragraphs from a PDF file based on the article structure tree in the embodiment of the present invention can refer to the specific implementation of the method for segmenting and completing complete semantic paragraphs from a PDF file based on the article structure tree mentioned above. To avoid redundancy, they will not be repeated here.

[0099] Example 3

[0100] A computer program product involved in Example 3 of the present application is used to store a computer program. When the computer program runs on a computer, the method in any one of the implementation modes in Example 1 of the present application is implemented.

[0101] Example 4

[0102] A computer-readable storage medium according to Embodiment 4 of the present application, wherein the computer-readable storage medium stores a program code for execution by a device, wherein the program code includes steps for executing a method in any one of the implementations in Embodiment 1 of the present application;

[0103] Among them, the computer-readable storage medium can be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM); the computer-readable storage medium can store program code, and when the program stored in the computer-readable storage medium is executed by the processor, the processor is used to execute the steps of the method in any one of the implementation methods in Example 1 of the present application.

[0104] Example 5

[0105] like Figure 3 As shown, an electronic device involved in Embodiment 5 of the present application includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, a method in any one of the implementations in Embodiment 1 of the present application is implemented;

[0106] Among them, the processor can adopt a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the method in any one of the implementation methods in Example 1 of the present application.

[0107] The processor may also be an integrated circuit electronic device with signal processing capability. In the implementation process, each step of the method in any implementation of Embodiment 1 of the present application may be completed by an integrated logic circuit of hardware in the processor or by instructions in software form.

[0108] The above-mentioned processor may also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the functions required to be performed by the unit included in the data processing device of the embodiment of the present application in combination with its hardware, or executes the method in any one of the implementation modes in the embodiment 1 of the present application.

[0109] The above are only preferred specific implementations of the present application; however, the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and its improved ideas of the present application within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree, characterized in that: include: Get the PDF file of the article to be processed; Generate an article structure tree according to the PDF file to mark the content of the PDF file in at least one form of directory, table, picture, page number, title, parent node, child node, and root node; Completing the missing parent nodes in the article structure tree through context analysis, structure analysis, text content analysis and predetermined rules; Search the subheadings in the PDF file; Taking the subheading as the root node, all child nodes of each subheading are traversed, and nodes are pruned according to semantic similarity to synthesize a complete semantic paragraph.

2. The method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree according to claim 1, characterized in that: Generate an article structure tree from a PDF file using the LayoutPDFReader method.

3. The method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree according to claim 1, characterized in that: Completing the missing parent node in the article structure tree by context analysis, structure analysis, text content analysis and predetermined rules includes at least one of the following methods: According to the coherence, known information and context of the parent nodes, the missing parent nodes are completed using the majority interpolation method; If the previous and next parent nodes of a missing parent node are inconsistent, the missing value is filled based on the nesting of the parent node values ​​and a sequence or sequential filling method; Complete the missing parent nodes according to the predetermined rules and keywords.

4. The method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree according to claim 1, characterized in that: Searching for subheadings in the article to be processed includes at least one of the following methods: Determine subheadings based on titles in the article tree structure; Determine subheadings based on preset keywords and node hierarchical positions; Determine the subheadings based on the hierarchical results in the article structure tree.

5. The method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree according to claim 1, characterized in that: Taking the subheading as the root node, traverse all the child nodes of each subheading, and prune the nodes according to the semantic similarity to synthesize a complete semantic paragraph, including: Taking each subheading as the root node, traverse all the child nodes of each subheading through the breadth-first search or depth-first search method, and prune them according to the semantic similarity; The semantic similarity is obtained by calculating the embeddings of the sentences of each layer of nodes and the previous layer of nodes; Set a similarity threshold. If the semantic similarity between nodes is higher than the set similarity threshold, the node will be retained. If the semantic similarity between nodes is lower than or equal to the set similarity threshold, the corresponding node will be pruned. The article structure tree obtained after pruning is merged into a complete semantic paragraph using a recursive algorithm.

6. The method for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree according to claim 5, characterized in that: During the expansion process of each level node in the depth-first search, the nodes are screened and pruned according to the set similarity threshold. Nodes with similarity lower than the similarity threshold are pruned and no longer enter the search queue. For the retained nodes, the depth-first search continues and the pruning operation is repeated until there are no more child nodes.

7. A device for segmenting and completing complete semantic paragraphs from a PDF file based on an article structure tree, characterized in that: include: The acquisition module is used to obtain the PDF file of the article to be processed; A structure tree generation module, used to generate an article structure tree according to the PDF file, so as to mark the content of the PDF file in at least one form of directory, table, picture, page number, title, parent node, child node, and root node; A parent node completion module, used to complete the missing parent nodes in the article structure tree through context analysis, structure analysis, text content analysis and predetermined rules; A subtitle search module, used to search for subtitles in the PDF file; The node pruning module is used to traverse all child nodes of each subheading with the subheading as the root node, and prune the nodes according to the semantic similarity to synthesize a complete semantic paragraph.

8. A computer program product, characterized in that The computer program product stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code for execution by a device, wherein the program code includes steps for executing the method according to any one of claims 1 to 6.

10. An electronic device, characterized in that: The electronic device includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction implements the method according to any one of claims 1 to 6 when executed by the processor.

Citation Information

Cited By

  • Text segmentation method and device for retrieval enhancement generation

    CN119311723A

  • Log processing method and device, computer equipment, readable storage medium and program product

    CN119647470A

  • Document paragraph content merging and segmenting method based on combined tree structure

    CN119918505A