Document processing method and related device

Abstract

The invention discloses a document processing method and a related device, and relates to the technical field of data processing. Comprising the following steps: analyzing an original document to obtain a text sequence corresponding to the original document; processing the text sequence through a large language model to generate a directory list corresponding to the text sequence; segmenting the text sequence to obtain a plurality of text blocks; based on the plurality of text blocks, a preset fusion rule and a large language model, determining a plurality of target semantic text segments and titles, starting point positions and ending point positions of the target semantic text segments; constructing a hierarchical knowledge structure based on the target list, the plurality of target semantic text segments, and titles, starting point positions and ending point positions of the target semantic text segments; the hierarchical knowledge structure has the characteristics of structuralization, traceability and semantic coherence, and the knowledge retrieval accuracy and efficiency can be improved by storing the hierarchical knowledge structure into a knowledge base.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a document processing method and related apparatus. Background Technology

[0002] The RAG knowledge base is a structured database used to store specialized data in the Retrieval-Augmented Generation architecture. It can provide accurate and scenario-based external information support for large models, avoiding answer bias or information lag caused by large models relying on general training data.

[0003] In the insurance industry, original documents (such as insurance terms, claims guidelines, product manuals, regulatory policy documents, etc.) are generally characterized by their large size, complex structure, and dense use of technical terms. They not only contain a large amount of unstructured text, but also often include semi-structured and unstructured elements such as tables, diagrams, and annotations. Furthermore, key information is often scattered across multiple levels of layout.

[0004] In the insurance industry, common RAG knowledge base construction methods often rely on fixed-length text chunks or simple chapter splitting, lacking an understanding of the inherent logical structure of insurance documents. This crude approach easily leads to knowledge fragmentation, semantic breaks, or loss of key context; severely impacting the retrieval accuracy of large models and the quality of generated reference statements. For example, when a human agent needs to quickly obtain an explanation of a specific insurance clause, a fragmented knowledge base may generate incorrect responses due to incomplete context.

[0005] Therefore, how to efficiently and accurately transform original documents into a knowledge organization form with a table of contents, section headings, and a complete hierarchy, and store it in a knowledge base to improve the accuracy and efficiency of knowledge retrieval, has become one of the urgent technical problems to be solved in the field of data processing technology. Summary of the Invention

[0006] To address the aforementioned issues, this application provides a document processing method that efficiently and accurately transforms original documents into a knowledge organization form with a table of contents, section headings, and a complete hierarchy, constructing a hierarchical knowledge structure for storage in a knowledge base, thereby improving the accuracy and efficiency of knowledge retrieval.

[0007] The embodiments of this application disclose the following technical solutions:

[0008] The first aspect of this application provides a document processing method, including:

[0009] The original document is parsed to obtain the text sequence corresponding to the original document; the original document includes text and images.

[0010] The text sequence is processed using a large language model to generate a directory list corresponding to the text sequence;

[0011] The text sequence is segmented to obtain multiple text blocks; adjacent text blocks have overlapping portions;

[0012] Based on the multiple text blocks, the preset fusion rules, and the large language model, multiple target semantic text segments are determined, as well as the title, start position, and end position of the target semantic text segments;

[0013] Based on the target list, the multiple target semantic text segments, and the titles, starting positions, and ending positions of the target semantic text segments, a hierarchical knowledge structure is constructed.

[0014] In one optional implementation, determining multiple target semantic text segments, and the titles, start positions, and end positions of the target semantic text segments based on the multiple text blocks, preset fusion rules, and the large language model, includes:

[0015] For each text block, multiple semantic text segments corresponding to the text block are generated using the text block, multiple prompt words, and the large language model, as well as the title, start position, and end position of each semantic text segment;

[0016] Based on preset fusion rules, adjacent semantic text segments, as well as the titles, start positions, and end positions of adjacent semantic text segments, are fused to obtain multiple target semantic text segments, as well as the titles, start positions, and end positions of the target semantic text segments.

[0017] In one optional implementation, for each text block, generating multiple semantic text segments corresponding to that text block using the text block, multiple prompt words, and the large language model, as well as the title, start position, and end position of each semantic text segment, includes:

[0018] For each text block, the text block and the first prompt word are input into the large language model to generate multiple semantic text segments, as well as the start and end positions of each semantic text segment;

[0019] The multiple semantic text segments and the second prompt word are input into the large language model to generate a title for each semantic text segment.

[0020] In one optional implementation, the fusion of adjacent semantic text segments, along with their titles, start positions, and end positions, based on preset fusion rules, yields multiple target semantic text segments, along with their titles, start positions, and end positions, including:

[0021] Obtain the semantic similarity of adjacent semantic text segments;

[0022] If the semantic similarity of adjacent semantic text segments exceeds a preset first threshold, the adjacent semantic text segments are merged to obtain the target semantic text segment; at the same time, the titles, start positions and end positions of the adjacent semantic text segments are merged to obtain the title, start position and end position of the target semantic text segment.

[0023] In one optional implementation, the fusion of adjacent semantic text segments, along with their titles, start positions, and end positions, based on preset fusion rules, yields multiple target semantic text segments, along with their titles, start positions, and end positions, including:

[0024] Determine the character length of each of the semantic text segments;

[0025] For each semantic text segment, if the character length of the semantic text segment is less than a preset second threshold, the semantic text segment is merged with the adjacent semantic text segments to obtain the target semantic text paragraph; at the same time, the title, start position and end position of the semantic text segment are merged with the title, start position and end position of the adjacent semantic text segments to obtain the title, start position and end position of the target semantic text paragraph.

[0026] In one optional implementation, parsing the original document to obtain the text sequence corresponding to the original document includes:

[0027] The text in the original document is processed by optical character recognition to obtain a first text sequence;

[0028] The images in the original document are analyzed using the visual language model to obtain a second text sequence;

[0029] The set of the first text sequence and the second text sequence is taken as the text sequence.

[0030] In one optional implementation, the step of processing the text sequence using a large language model to generate a directory list corresponding to the text sequence includes:

[0031] After determining that the total number of characters in the text sequence exceeds a preset third threshold, the text sequence and the third prompt word are input into the large language model to generate a directory list corresponding to the text sequence.

[0032] A second aspect of this application provides a document processing apparatus, comprising:

[0033] The text sequence acquisition module is used to parse the original document and obtain the text sequence corresponding to the original document; the original document includes text and images;

[0034] The directory list acquisition module is used to process the text sequence through a large language model and generate a directory list corresponding to the text sequence.

[0035] The text block determination module is used to perform a segmentation operation on the text sequence to obtain multiple text blocks; adjacent text blocks have overlapping parts;

[0036] The structural information acquisition module is used to determine multiple target semantic text segments, as well as the title, start position and end position of the target semantic text segments, based on the multiple text blocks, the preset fusion rules and the large language model.

[0037] The hierarchical knowledge structure determination module is used to construct a hierarchical knowledge structure based on the target list, the multiple target semantic text segments, and the titles, start positions, and end positions of the target semantic text segments.

[0038] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any implementation of the first aspect.

[0039] A fourth aspect of this application provides an electronic device, comprising:

[0040] A memory on which computer programs are stored;

[0041] A processor for executing the computer program in the memory to implement the steps of the method described in any implementation of the first aspect.

[0042] Compared with the prior art, this application has the following beneficial effects:

[0043] This application discloses a document processing method, including: parsing the original document to obtain the text sequence corresponding to the original document; processing the text sequence through a large language model to generate a table of contents corresponding to the text sequence; segmenting the text sequence to obtain multiple text blocks; determining multiple target semantic text segments, as well as the titles, start positions, and end positions of the target semantic text segments, based on the multiple text blocks, preset fusion rules, and the large language model; and constructing a hierarchical knowledge structure based on the target list, multiple target semantic text segments, and the titles, start positions, and end positions of the target semantic text segments. Because this method deeply mines the semantic relationships and logical hierarchy of the text sequence through a large language model, anchors the overall knowledge structure with a table of contents as a framework, and precisely divides the target semantic text segments and clarifies their titles and position information, it avoids the fragmentation and semantic breaks caused by traditional simple segmentation, while fully preserving the contextual logic and hierarchical relationships between information. This results in a structured, traceable, and semantically coherent knowledge structure. Therefore, storing this hierarchical knowledge structure in a knowledge base can improve the accuracy and efficiency of knowledge retrieval. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart illustrating a document processing method provided in an embodiment of this application;

[0046] Figure 2 A schematic diagram of target semantic text segment information provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the structure of the document processing apparatus provided in the embodiments of this application. Detailed Implementation

[0048] The RAG knowledge base is a structured database used to store specialized data in the Retrieval-Augmented Generation architecture. It can provide accurate and scenario-based external information support for large models, avoiding answer bias or information lag caused by large models relying on general training data.

[0049] In the insurance industry, original documents (such as insurance terms, claims guidelines, product manuals, regulatory policy documents, etc.) are generally characterized by their large size, complex structure, and dense use of technical terms. They not only contain a large amount of unstructured text, but also often include semi-structured and unstructured elements such as tables, diagrams, and annotations. Furthermore, key information is often scattered across multiple levels of layout.

[0050] In the insurance industry, common RAG (Related Information Groups) knowledge base construction methods often rely on fixed-length text chunks or simple chapter splitting, lacking an understanding of the inherent logical structure of insurance documents. This crude approach easily leads to knowledge fragmentation, semantic breaks, or loss of key context. For example, the applicable scenarios of "exclusions" in insurance clauses may be strongly related to the preceding "scope of insurance liability" or "underwriting conditions," and simple splitting will sever this logical connection. The process steps and material requirements in claims guidelines are often tied to specific case types and amount thresholds; fragmented storage may lead to the omission of key constraints. This directly affects retrieval accuracy and search result quality. When human agents need to quickly retrieve key information such as explanations of specific clauses, claims process details, or product liability boundaries when serving customers, fragmented knowledge with incomplete context may cause misunderstandings, leading to incorrect responses, increased risk of claims disputes, and even affecting customer trust and corporate compliance.

[0051] Therefore, how to efficiently and accurately transform original documents into a knowledge organization form with a table of contents, section headings, and a complete hierarchy, and store it in a knowledge base to improve the accuracy and efficiency of knowledge retrieval, has become one of the urgent technical problems to be solved in the field of data processing technology.

[0052] To address the aforementioned issues, this application discloses a document processing method, comprising: parsing the original document to obtain a text sequence corresponding to the original document; processing the text sequence using a large language model to generate a table of contents corresponding to the text sequence; segmenting the text sequence to obtain multiple text blocks; determining multiple target semantic text segments, as well as the titles, start positions, and end positions of the target semantic text segments, based on the multiple text blocks, preset fusion rules, and the large language model; and constructing a hierarchical knowledge structure based on the target list, the multiple target semantic text segments, and the titles, start positions, and end positions of the target semantic text segments.

[0053] Because this method deeply mines the semantic relationships and logical hierarchy of text sequences through a large language model, anchors the overall knowledge structure with a directory list as a framework, and precisely divides the target semantic text segments and clarifies their titles and location information, it avoids the fragmentation and semantic breaks caused by traditional simple segmentation, while fully preserving the contextual logic and hierarchical relationships between information. This makes the constructed knowledge structure structured, traceable, and semantically coherent. Therefore, storing this hierarchical knowledge structure in a knowledge base can improve the accuracy and efficiency of knowledge retrieval.

[0054] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0055] Figure 1 This is a flowchart illustrating a document processing method provided in an embodiment of this application. (In conjunction with...) Figure 1 As shown, the document processing method disclosed in this application includes:

[0056] S101, parse the original document to obtain the text sequence corresponding to the original document.

[0057] The original documents in this application include text and images. In the insurance field, original documents may include insurance product terms and conditions manuals, claims service guides, application forms and health declarations, insurance regulatory policy documents, and customer service manuals, etc.

[0058] The original documents in this application may be in the format of PDF, Word, or image files, etc.

[0059] In one optional implementation, the original document is parsed to obtain the text sequence corresponding to the original document, specifically as follows:

[0060] The text in the original document is processed by optical character recognition (OCR), that is, the text content in the original document is extracted to obtain the first text sequence;

[0061] By analyzing the images in the original document using visual language models (such as CLIP, Contrastive Language-Image Pre-training), the meaning of the text, tables, flowcharts, and text in the images is understood, resulting in a second text sequence.

[0062] The set of the first text sequence and the second text sequence is taken as the text sequence.

[0063] S102, The text sequence is processed using a large language model to generate a directory list corresponding to the text sequence.

[0064] Specifically, the total length of the text sequence can be calculated to obtain the total number of characters in the text. After the total number of characters in the text sequence exceeds a preset third threshold, the text sequence and the third prompt word are input into the large language model to generate a directory list corresponding to the text sequence.

[0065] For example, the third threshold is 10,000 characters. The third prompt template could be: Based on the following text content, automatically identify its core theme, chapter logic, and hierarchical structure to generate a standardized and clear table of contents.

[0066] Among them, the large language model can be a single sentence of the text or a thousand questions on the general meaning.

[0067] It should be noted that if the total length of the calculated text does not exceed the preset third threshold, the target list, target semantic text segment, and the title, start position and end position of the target semantic text segment are directly extracted from the text sequence.

[0068] S103, the text sequence is segmented to obtain multiple text blocks.

[0069] In this application, any two adjacent text blocks obtained from the segmentation operation have overlapping portions.

[0070] For example, after obtaining the text sequence, the text sequence can be divided into multiple overlapping text blocks according to a preset segmentation rule.

[0071] For example, the preset segmentation rule could be: set a single text block length threshold (e.g., 512 tokens), and configure a 20% text overlap ratio, that is, the later text block contains 20% of the content at the end of the previous text block, and the previous text block also covers 20% of the information at the beginning of the later text block.

[0072] The core value of this overlapping design lies in effectively maintaining the logical coherence and contextual relevance of the text content by preserving the semantic overlap area of ​​adjacent text blocks, avoiding semantic breaks and information gaps caused by mechanical segmentation, and providing basic support for the accurate identification and fusion of subsequent target semantic text segments.

[0073] S104. Based on the multiple text blocks, the preset fusion rules, and the large language model, determine multiple target semantic text segments, as well as the title, start position, and end position of the target semantic text segments.

[0074] First, for each text block, multiple semantic text segments corresponding to the text block are generated using the text block, multiple prompt words, and the large language model, as well as the title, start position, and end position of each semantic text segment.

[0075] Specifically, for each text block, the text block and the first prompt word are input into the large language model to generate multiple semantic text segments, as well as the start and end positions of each semantic text segment; then, the multiple semantic text segments and the second prompt word are input into the large language model to generate the title of each semantic text segment.

[0076] For example, the first prompt could be: "Based on the following text block content, please accurately split it into independent and complete semantic text segments (each semantic text segment must revolve around a single core theme, avoiding cross-theme splitting). Requirements: 1. Clearly mark the starting and ending character positions of each semantic text segment in the original text block (based on the character index of the original text block, starting from index 0); 2. The split semantic text segments must completely retain the core information, without omitting key details or severing semantic logic; 3. If there are nested expressions in the text block (such as sub-items in clauses, supplementary content in descriptions), they must be split into independent text segments according to semantic level to ensure that the semantic boundaries of each text segment are clear. The text block content is as follows: {text block}."

[0077] For example, the second prompt could be: "Please generate an exact matching title for each of the following semantic text segments (the title must be extracted from the core content of the text segment and should not include any extra irrelevant information). Requirements: 1. The title language should be concise and formal, conforming to the expression standards of professional documents in the insurance field (such as terms and guidelines), and the number of characters should be controlled within 15; 2. The title should accurately summarize the core theme of the text segment, avoiding vague or generalized expressions, and ensuring that the core information of the text segment can be quickly identified through the title; 3. If multiple semantic text segments have a hierarchical relationship (such as general overview and specific details), the title should reflect the corresponding hierarchical logic (this can be presented through "main title - subtitle" or keyword echoing). The semantic text segments are as follows: {List of Semantic Text Segments}."

[0078] Secondly, based on the preset fusion rules, adjacent semantic text segments, as well as the titles, start positions, and end positions of adjacent semantic text segments, are fused to obtain multiple target semantic text segments, as well as the titles, start positions, and end positions of the target semantic text segments.

[0079] The preset fusion rules in this application can be based on the similarity of semantic text paragraphs to fuse adjacent semantic text paragraphs; they can be based on the number of characters in the semantic text paragraphs to fuse adjacent semantic text paragraphs; or they can be based on the semantic coherence of the context to fuse adjacent semantic text paragraphs.

[0080] In one optional implementation, adjacent semantic text segments are fused based on their similarity. Specifically, the semantic similarity of adjacent semantic text segments is obtained. If the semantic similarity of adjacent semantic text segments exceeds a preset first threshold, the adjacent semantic text segments are fused to obtain the target semantic text segment. At the same time, the title, start position, and end position of the adjacent semantic text segments are fused to obtain the title, start position, and end position of the target semantic text segment.

[0081] In one optional implementation, adjacent semantic text segments are merged based on the number of characters in the semantic text segment. Specifically, the character length of each semantic text segment is determined. For each semantic text segment, if the character length of the semantic text segment is less than a preset second threshold (e.g., 200 characters), the semantic text segment is merged with adjacent semantic text segments to obtain the target semantic text segment. At the same time, the title, start position, and end position of the semantic text segment are merged with the titles, start positions, and end positions of adjacent semantic text segments to obtain the title, start position, and end position of the target semantic text segment.

[0082] S105. Based on the target list, the multiple target semantic text segments, and the titles, start positions, and end positions of the target semantic text segments, a hierarchical knowledge structure is constructed.

[0083] It should be noted that the semantic text paragraphs described in this application are also called summaries, the titles of the semantic text paragraphs are also called summary titles, the starting point of the semantic text paragraphs is also called the summary starting point, and the ending point of the semantic text paragraphs is also called the summary ending point.

[0084] Figure 2 This is a schematic diagram illustrating target semantic text segment information provided in an embodiment of this application. (Combined with...) Figure 2 As shown, the target semantic text information includes the title of each semantic text paragraph (summary), the starting position of the semantic text paragraph (starting line and starting point text), and the ending position of the semantic text paragraph (ending line and ending point text).

[0085] Based on the same inventive concept, this application also discloses a document processing device. Figure 3 This is a schematic diagram of the document processing apparatus provided in an embodiment of this application. (In conjunction with...) Figure 3 As shown, the document processing apparatus 300 disclosed in this application includes:

[0086] The text sequence acquisition module 301 is used to parse the original document and acquire the text sequence corresponding to the original document; the original document includes text and images;

[0087] The directory list acquisition module 302 is used to process the text sequence through a large language model to generate a directory list corresponding to the text sequence;

[0088] The text block determination module 303 is used to perform a segmentation operation on the text sequence to obtain multiple text blocks; adjacent text blocks have overlapping parts;

[0089] The structural information acquisition module 304 is used to determine multiple target semantic text segments, as well as the title, start position and end position of the target semantic text segments, based on the multiple text blocks, the preset fusion rules and the large language model.

[0090] The hierarchical knowledge structure determination module 305 is used to construct a hierarchical knowledge structure based on the target list, the multiple target semantic text segments, and the title, start position and end position of the target semantic text segments.

[0091] In one alternative implementation, the structure information acquisition module 304 includes:

[0092] The first data acquisition unit is used to generate, for each text block, multiple semantic text segments corresponding to the text block, as well as the title, start position and end position of each semantic text segment, through the text block, multiple prompt words and the large language model;

[0093] The second data acquisition unit is used to fuse adjacent semantic text segments, as well as the titles, start positions and end positions of adjacent semantic text segments, based on preset fusion rules, to obtain multiple target semantic text segments, as well as the titles, start positions and end positions of the target semantic text segments.

[0094] In one optional implementation, the first data acquisition unit includes:

[0095] A semantic text segment generation subunit is used to input the text block and the first prompt word into the large language model for each text block to generate multiple semantic text segments, as well as the start and end positions of each semantic text segment;

[0096] The text segment title generation subunit is used to input the multiple semantic text segments and the second prompt word into the large language model to generate a title for each semantic text segment.

[0097] In one alternative implementation, the second data acquisition unit includes:

[0098] The similarity determination subunit is used to obtain the semantic similarity between adjacent semantic text segments;

[0099] The first text segment determination subunit is used to merge adjacent semantic text segments to obtain the target semantic text segment if the semantic similarity of adjacent semantic text segments exceeds a preset first threshold; at the same time, the title, start position and end position of the adjacent semantic text segments are merged to obtain the title, start position and end position of the target semantic text segment.

[0100] In one alternative implementation, the second data acquisition unit includes:

[0101] The character length determination subunit is used to determine the character length of each semantic text segment;

[0102] The second text segment determination subunit is used to, for each semantic text segment, if the character length of the semantic text segment is less than a preset second threshold, merge the semantic text segment with the adjacent semantic text segments to obtain the target semantic text segment; at the same time, merge the title, start position and end position of the semantic text segment with the title, start position and end position of the adjacent semantic text segments to obtain the title, start position and end position of the target semantic text segment.

[0103] In one alternative implementation, the text sequence acquisition module 301 includes:

[0104] The first text sequence determination unit is used to process the text in the original document through optical character recognition to obtain a first text sequence;

[0105] The second text sequence determination unit is used to analyze the images in the original document through the visual language model to obtain the second text sequence;

[0106] A text sequence determination unit is used to take the set of the first text sequence and the second text sequence as the text sequence.

[0107] In one alternative implementation, the directory listing acquisition module 302 includes:

[0108] The directory list acquisition unit is used to input the text sequence and the third prompt word into the large language model after determining that the total number of characters in the text sequence exceeds a preset third threshold, and generate a directory list corresponding to the text sequence.

[0109] Based on the document processing method and apparatus provided in the foregoing embodiments, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the document processing method mentioned above.

[0110] Based on the document processing method and apparatus provided in the foregoing embodiments, this application also provides an electronic device, including:

[0111] A memory on which computer programs are stored;

[0112] A processor is configured to execute the computer program in the memory to implement some or all of the steps in the document processing method provided in the foregoing embodiments.

[0113] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0114] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A document processing method, characterized in that, The method includes: The original document is parsed to obtain the text sequence corresponding to the original document; the original document includes text and images. The text sequence is processed using a large language model to generate a directory list corresponding to the text sequence; The text sequence is segmented to obtain multiple text blocks; adjacent text blocks have overlapping portions; Based on the multiple text blocks, the preset fusion rules, and the large language model, multiple target semantic text segments are determined, as well as the title, start position, and end position of the target semantic text segments; Based on the target list, the multiple target semantic text segments, and the titles, starting positions, and ending positions of the target semantic text segments, a hierarchical knowledge structure is constructed.

2. The method according to claim 1, characterized in that, The process of determining multiple target semantic text segments, along with their titles, start positions, and end positions, based on the multiple text blocks, preset fusion rules, and the large language model, includes: For each text block, multiple semantic text segments corresponding to the text block are generated using the text block, multiple prompt words, and the large language model, as well as the title, start position, and end position of each semantic text segment; Based on preset fusion rules, adjacent semantic text segments, as well as the titles, start positions, and end positions of adjacent semantic text segments, are fused to obtain multiple target semantic text segments, as well as the titles, start positions, and end positions of the target semantic text segments.

3. The method according to claim 2, characterized in that, For each text block, multiple semantic text segments corresponding to that text block are generated using the text block, multiple prompt words, and the large language model, along with the title, start position, and end position of each semantic text segment, including: For each text block, the text block and the first prompt word are input into the large language model to generate multiple semantic text segments, as well as the start and end positions of each semantic text segment; The multiple semantic text segments and the second prompt word are input into the large language model to generate a title for each semantic text segment.

4. The method according to claim 2, characterized in that, The method, based on preset fusion rules, fuses adjacent semantic text segments, as well as the titles, start positions, and end positions of adjacent semantic text segments, to obtain multiple target semantic text segments, and the titles, start positions, and end positions of the target semantic text segments, including: Obtain the semantic similarity of adjacent semantic text segments; If the semantic similarity of adjacent semantic text segments exceeds a preset first threshold, the adjacent semantic text segments are merged to obtain the target semantic text segment; at the same time, the titles, start positions and end positions of the adjacent semantic text segments are merged to obtain the title, start position and end position of the target semantic text segment.

5. The method according to claim 2, characterized in that, The method, based on preset fusion rules, fuses adjacent semantic text segments, as well as the titles, start positions, and end positions of adjacent semantic text segments, to obtain multiple target semantic text segments, and the titles, start positions, and end positions of the target semantic text segments, including: Determine the character length of each of the semantic text segments; For each semantic text segment, if the character length of the semantic text segment is less than a preset second threshold, the semantic text segment is merged with the adjacent semantic text segments to obtain the target semantic text paragraph; at the same time, the title, start position and end position of the semantic text segment are merged with the title, start position and end position of the adjacent semantic text segments to obtain the title, start position and end position of the target semantic text paragraph.

6. The method according to claim 1, characterized in that, The step of parsing the original document to obtain the text sequence corresponding to the original document includes: The text in the original document is processed by optical character recognition to obtain a first text sequence; The images in the original document are analyzed using the visual language model to obtain a second text sequence; The set of the first text sequence and the second text sequence is taken as the text sequence.

7. The method according to claim 1, characterized in that, The step of processing the text sequence using a large language model to generate a directory list corresponding to the text sequence includes: After determining that the total number of characters in the text sequence exceeds a preset third threshold, the text sequence and the third prompt word are input into the large language model to generate a directory list corresponding to the text sequence.

8. A document processing apparatus, characterized in that, The device includes: The text sequence acquisition module is used to parse the original document and obtain the text sequence corresponding to the original document; the original document includes text and images; The directory list acquisition module is used to process the text sequence through a large language model and generate a directory list corresponding to the text sequence. The text block determination module is used to perform a segmentation operation on the text sequence to obtain multiple text blocks; adjacent text blocks have overlapping parts; The structural information acquisition module is used to determine multiple target semantic text segments, as well as the title, start position and end position of the target semantic text segments, based on the multiple text blocks, the preset fusion rules and the large language model. The hierarchical knowledge structure determination module is used to construct a hierarchical knowledge structure based on the target list, the multiple target semantic text segments, and the titles, start positions, and end positions of the target semantic text segments.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-7.

10. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.

Images