Construction method of text knowledge base, computer equipment and program product
By extracting first-level headings using a large language model and constructing a multi-level text structure tree, the problem of chaotic structure in existing document segmentation methods is solved, achieving clear and controllable text structure and accurate information organization.
Patent Information
- Application Number
- CN202511296914.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-11
- Publication Date
- 2025-10-17
- Estimated Expiration
- 2045-09-11
AI Technical Summary
Existing document segmentation methods are based on fixed rules, which results in single chapters being cut off or multiple chapters being mixed, making it difficult to accurately restore the original document structure and affecting the accuracy of information organization.
The first-level headings of the original text are extracted using a large language model. Based on the first-level headings, the text is segmented. A parallel and progressive multi-layer structure segmentation strategy is adopted to construct a multi-level text structure tree. By identifying keywords at each level and determining the preset termination conditions, a multi-level text structure tree is formed.
It achieves clear and controllable text structure, reduces semantic fragmentation and structural chaos, improves the rigor and completeness of knowledge organization, and enhances the accuracy and computational efficiency of information organization.
Smart Images

Figure CN120804233A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of natural language processing, and particularly relates to a text knowledge base construction method, computer equipment and a program product. BACKGROUND
[0002] With the development of intelligent agents, the ability to acquire and utilize external knowledge is a very key technology in the construction of intelligent agents. The most typical one is the RAG (Retrieval-augmented Generation) technology, which provides the intelligent agent with corresponding knowledge by retrieving document slices.
[0003] In the existing technologies such as RAG, the original document is usually first divided and stored in a vector database, and then when a query request is received, a number of most relevant paragraphs are retrieved and sorted from the vector database, and finally the part with the highest confidence is delivered as a prompt input to a large model. However, the current mainstream document division method is mostly based on fixed rules, such as division by word count, paragraph or punctuation. This static and coarse-grained division method is prone to cause a single chapter to be cut off or multiple chapters to be mixed, making it difficult to accurately restore the original document structure and affecting the accuracy of information organization. SUMMARY In order to solve the problem that the existing document division method is difficult to accurately restore the original document structure and affects the accuracy of information organization, the present application provides a text knowledge base construction method, computer equipment and a program product.
[0004] The technical problem of the present application is solved by providing a method for constructing a text knowledge base, comprising the following steps: obtaining an original text, extracting a first-level title of the original text through a large language model; based on the first-level title, the original text is divided to obtain a first-level text corresponding to the number of first-level titles; based on the first-level title, the original text is divided to obtain a first-level text corresponding to the number of first-level titles, specifically including: locating the first position of the first occurrence of the first first-level title in the original text; determining the second position of the first occurrence of the second first-level title based on the first occurrence of the first first-level title; dividing the content between the first position and the second position as the first first-level text with the first position and the second position as boundaries; continue to locate the first-level title in the original text, and sequentially take the second first-level title located in the previous time as the starting first-level title of the current time division, until the last first-level title is located, and the content between the adjacent two first-level titles is divided to obtain the first-level text corresponding to the number of first-level titles; the first-level text that does not meet the preset termination division condition is divided in parallel and progressively until the subtext after division meets the preset termination division condition, and the division is stopped; based on the main title of the original text and all first-level texts and subtexts after division, a multi-level text structure tree is constructed; the text structure tree includes all original contents of the original text.
[0005] Preferably, after obtaining the original text and extracting the first-level title of the original text through the large language model, the method further comprises: determining whether each extracted first-level title exists in the original text through the large language model; if the content of the extracted first-level title does not exist in the original text, the extraction fails, and the extraction operation of the first-level title is performed again.
[0006] Preferably, the preset termination division condition includes judging whether the text length is greater than a preset text length threshold, and / or determining whether the text includes a substructure or a nested chapter through the large language model according to a preset prompt word.
[0007] Preferably, after the original text is divided based on the first-level title to obtain the first-level text corresponding to the number of first-level titles, the method further comprises: if all first-level texts meet the preset termination division condition of first-level texts, the division is stopped; and a text structure tree is constructed based on the main title of the original text and all first-level texts after division.
[0008] Preferably, the first-level texts that do not meet the preset termination segmentation condition are subjected to parallel progressive segmentation until the segmented subtexts meet the preset termination segmentation condition, and the segmentation is stopped, specifically including: all the first-level texts that do not meet the preset termination segmentation condition are subjected to parallel segmentation processing to obtain at least two second-level texts corresponding to each first-level text; it is judged whether all the second-level texts under all the first-level texts meet the preset termination segmentation condition; all the second-level texts that do not meet the preset termination segmentation condition are subjected to subsequent rounds of parallel progressive segmentation until all the subtexts segmented in the subsequent rounds meet the preset termination condition and the segmentation is stopped.
[0009] Preferably, after all the second-level texts that do not meet the preset termination segmentation condition are subjected to subsequent rounds of parallel progressive segmentation until all the subtexts segmented in the subsequent rounds meet the preset termination condition and the segmentation is stopped, the method further includes: taking all the first-level texts, second-level texts and subtexts that meet the preset termination segmentation condition after being segmented for multiple rounds as leaf nodes; and performing information enhancement on the leaf nodes to obtain information-enhanced leaf nodes.
[0010] Preferably, the information enhancement on the leaf nodes to obtain information-enhanced leaf nodes specifically includes: identifying whether there is missing target information in the leaf nodes according to a preset prompt instruction through a large language model; the target information includes reference information; if there is missing target information in the leaf nodes, performing level-by-level tracing on the leaf nodes with missing target information according to the segmentation path of the original text to determine the missing target information in the leaf nodes from the previous layer of nodes of the leaf nodes with missing target information; and supplementing the determined target information from a layer of nodes to the leaf nodes with missing target information to obtain information-enhanced leaf nodes.
[0011] Preferably, a multi-level text structure tree is constructed based on the main title of the original text and all the first-level texts and subtexts segmented, specifically including: determining the level of the text structure tree to be constructed according to the segmentation rounds; obtaining the main title of the original text and taking the main title as a root node; generating corresponding structure nodes from the first-level texts, second-level texts and subtexts formed after multiple rounds of segmentation and giving each structure node a unique identifier; constructing the mapping relationship between the parent nodes and the child node list to which each structure node belongs; the structure nodes include title content, level, parent node, child node list and original text segment; and constructing the multi-level text structure tree corresponding to the original text based on the root node, the structure nodes and the mapping relationship.
[0012] To solve the above technical problems, the present application provides another technical solution as follows: a computer device including a storage, a processor and a computer program stored on the storage, the processor executing the computer program to implement the method for constructing a text knowledge base according to any one of the above.
[0013] To solve the above technical problems, the present application provides another technical solution as follows: a computer program product having a computer program stored thereon, the computer program being executed by a processor to implement the text knowledge base construction method according to any one of the above.
[0014] Compared with the prior art, the text knowledge base construction method, computer device and program product provided by the present application have the following beneficial effects: 1. The text knowledge base construction method provided by the present application first extracts a first-level title of an original text through a large language model, then locates a first position where a first first-level title first appears in the original text and a second position where a second first-level title first appears, and divides a first first-level text according to the positions of the two; the division of subsequent first-level texts is performed in the same way to obtain first-level texts corresponding to the number of first-level titles; and the first-level texts that do not meet the preset termination division condition are continuously divided in parallel and progressively until the subtexts after division meet the preset termination division condition, and the division is stopped. Finally, a multi-level text structure tree is constructed based on the main title of the original text and all the first-level texts and subtexts after division. Compared with the traditional division method, the present application adopts a document parallel and progressive multi-level structure division strategy, recognizes the text structure layer by layer and recursively constructs a multi-level text structure tree, which not only preserves the original logical hierarchy of the original text, but also reduces the problems of semantic fragmentation and structural disorder, makes the hierarchical relationship clear and controllable, effectively improves the rigor and integrity of knowledge organization, and makes the information organization of the text structure tree more accurate.
[0015] 2. In the present application, after each round of division, the divided texts are judged one by one for the preset termination division condition, that is, whether the text length is greater than the preset text length threshold is judged, and / or the semantic judgment of the text is performed by the large language model according to the preset prompt word to determine whether the text includes a substructure or a nested chapter, so as to determine whether the original text and the divided texts continue to perform the structure extraction and division operation of the next level. Through the judgment of the text length and / or the large language model, the fragments that may not need to be processed can be quickly filtered, the overall processing efficiency is improved, and the calculation efficiency and adaptability are taken into account while maintaining the accuracy of the text structure, thereby providing high-quality node boundary division basis for the construction of the subsequent hierarchical text structure tree.
[0016] 3、The application realizes parallel progressive segmentation of each document after segmentation, that is, after each round of text structure segmentation is completed, each subtext generated after segmentation is judged one by one for termination segmentation to determine whether to continue to execute the structure extraction and segmentation operation of the next level. The termination segmentation judgment is performed in parallel in each round of segmentation, and each subtext after each round of segmentation is independently evaluated and judged, ensuring the flexibility and efficiency of segmentation.
[0017] 4、The application can improve the integrity and semantic accuracy of the content included in all leaf nodes by taking all first-level texts, second-level texts and subtexts that meet the preset termination segmentation condition after multiple rounds of segmentation as leaf nodes and enhancing the information of the leaf nodes, thereby better supporting subsequent knowledge base construction and downstream task calling.
[0018] 5、When the large language model determines that there is missing target information of a leaf node, the application can trace the missing target information of the leaf node level by level according to the segmentation path of the original text to determine the missing target information in the leaf node from the previous node of the leaf node with missing target information, and supplement the determined target information from the previous node to the leaf node with missing target information to obtain the information-enhanced leaf node. In this way, the missing information is identified, and the missing information is found and determined and completed through level-by-level tracing, so as to enhance the semantic integrity of the content included in each leaf node.
[0019] 6、The application can first determine the level of the text structure tree to be constructed according to the segmentation round, take the main title of the original text as the root node, then generate corresponding structure nodes according to the first-level texts, second-level texts and subtexts after multiple rounds of segmentation, and assign a unique identifier to each structure node, and construct the mapping relationship between the parent node and the child node list to which each structure node belongs, so that each structure node includes title content, level, parent node, child node list and original text segment. Finally, based on the root node, structure node and mapping relationship, a multi-level text structure tree corresponding to the original text is constructed, so that the text structure tree presents a multi-level knowledge organization tree with clear parent-child levels, complete content mounting and rich semantic extraction in form. The text structure tree not only significantly improves the visibility and controllability of the original text structure, but also provides a solid data foundation and structural guarantee for downstream tasks such as knowledge retrieval, content generation and intelligent question answering.
[0020] 7. The present invention also provides a computer device having the same beneficial effects as the above-mentioned method for constructing a text knowledge base, which will not be described in detail here.
[0021] 8. The present invention also provides a computer program product having the same beneficial effects as the above-mentioned method for constructing a text knowledge base, which will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0023] Figure 1 This is a flowchart of the steps of a method for constructing a text knowledge base provided by the first embodiment of the present invention.
[0024] Figure 2 This is an example diagram of the document structure tree of the original text in the method for constructing a text knowledge base provided by the first embodiment of the present invention.
[0025] Figure 3 This is a flowchart of steps S201 to S204 of the method for constructing a text knowledge base provided by the first embodiment of the present invention.
[0026] Figure 4 This is a flowchart of steps S3051 to S3053 of the method for constructing a text knowledge base provided by the first embodiment of the present invention.
[0027] Figure 5 This is a flowchart of steps S401 to S405 of the method for constructing a text knowledge base provided by the first embodiment of the present invention.
[0028] Figure 6 It is a schematic diagram of the framework of a computer device provided by the second embodiment of the present invention.
[0029] Figure 7 It is a schematic diagram of the framework of a computer program product provided by the third embodiment of the present invention.
[0030] Description of the accompanying drawings: 1. Computer equipment; 2. Computer program products; 10. Memory; 11. Processor; 12. Computer program; 20. Computer program. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and implementation examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0032] See also Figure 1 and Figure 2 The first embodiment of the present invention provides a method for constructing a text knowledge base, comprising the following steps: S10: Obtaining the original text, and extracting the first-level title of the original text using a large language model; Specifically, the original text may be the unsegmented original document corresponding to the document slices stored in the vector retrieval library and returned by the agent for retrieval. The large language model may be a general-purpose large language model, such as Qwen3 or DeepSeek. Users can select a large language model based on their actual needs, and this implementation does not impose any restrictions on this.
[0033] In step S10, after the complete source text is input into a large language model with structural understanding capabilities, the large language model can be guided to extract the source text's first-level headings and output them in the corresponding format by explicitly specifying the output format, such as JSON (JavaScript Object Notation), in a preset output format prompt. Using the large language model to extract the source text's first-level headings improves its robustness. Even if the source text has inconsistent formats, mixed text, or is converted from a scanned document, the large language model's language understanding capabilities can still identify potential structural divisions. Furthermore, compared to directly extracting multi-level headings, the recognition of first-level headings is more accurate and stable. Because the number of heading levels in the source text varies, directly extracting a multi-level structure can easily lead to inconsistent results, confusing formatting, and even problems such as rewriting or missing headings. Different models also perform significantly differently in multi-level extraction, further impacting reliability. Focusing on first-level headings significantly improves the consistency and accuracy of extraction.
[0034] It's important to note that the titles extracted by the large language model must be consistent with the original text, including details like spaces and symbols, to ensure accurate formatting. During the extraction process, the large language model utilizes a combination of explicit markup and implicit semantic features to determine title boundaries and their logical attribution, accurately locating and determining the first-level titles in the original text. Explicit markup includes numbering, punctuation, and typesetting. Implicit semantic features include topic content and paragraph aggregation.
[0035] Understandably, to ensure the extraction quality, after the large language model extracts the first-level title, it can also check all first-level titles one by one to see if they actually appear in the original document. If there is a missing or mismatching case, it is determined that the extraction fails, and the extraction operation is performed again to further ensure the integrity and usability of the extraction result.
[0036] It should be noted that when the original text document length is short and there is no division or chapter description in the original text, the original text does not need to be divided. When there is no clear section division in the original text, but the content of the original text is composed of different module contents, the first-level title of this type of original text can be obtained from the subheadings of different modules. Specifically, it can be determined by the large language model whether there is a first-level title in the original text; if there is no first-level title in the original text, it is determined whether there is a module theme in the original text; if there is a module theme in the original text, the module theme is extracted as the first-level title of the original text. Thus, the first-level title is extracted for different document lengths and different document characteristics, thereby providing the application range of the embodiment.
[0037] The following gives a text example of an original text; Development history of large language models Chapter 1: Origin and early exploration of language models Statistical language models (N-gram): basic principles, application fields and limitations… Rule-based dialogue systems: ELIZA, SHRDLU, and other early attempts… The emergence of word embeddings (Word2Vec, GloVe) and simple neural network language models… Chapter 2: Turning point in the era of neural networks (2014-2017) Language modeling capabilities of RNN and LSTM:… Seq2Seq and attention mechanism (Attention): introducing stronger context processing… The birth of the Transformer architecture (2017): Self-Attention, parallel computing capabilities, long-distance dependency modeling… Chapter 3: Rise of pre-trained language models (2018-2020) BERT and bidirectional context modeling: innovation of Masked Language Modeling… Initial attempts of the GPT series (GPT-1 / 2): causal language modeling and improved generation capabilities… Unified task modeling philosophy of models such as T5 and BART: transforming NLP tasks into "text-to-text"… Chapter 4: Large-scale Models and General Language Intelligence (2020–2023) GPT-3: Parameter Explosion and Few-shot Capabilities: … Prompt Learning and Fine-tuning Method Evolution: In-context learning, Prompt Tuning, LoRA, and other techniques… Initial Integration of Multimodal Models: CLIP, DALL·E, Flamingo, and other visual-language models… Emergent Abilities of Large Models: Abilities beyond scale… Chapter 5: Agents, Multimodal, and Future Trends (2023–Present) GPT-4 and Multimodal Integration (GPT-4V): … Language Models as Agents: Tool Use, Task Planning, Autonomous Execution (e.g., AutoGPT, ReAct)… Open-source Large Models and Community-driven Innovation: LLaMA, Mistral, Gemma, and others… Future Challenges and Directions: Safety Alignment, Cost Optimization, Path to General Intelligence (AGI)… In the above text example, the development history of large language models is the main title. Chapter 1: Origins and Early Exploration of Language Models; Chapter 2: Turning Point in the Era of Neural Networks (2014–2017); Chapter 3: Rise of Pre-trained Language Models (2018–2020); Chapter 4: Large-scale Models and General Language Intelligence (2020–2023); Chapter 5: Agents, Multimodal, and Future Trends (2023–Present), the content of the five chapters is the first-level title that can be extracted.
[0038] S20: Based on the first-level title, the original text is divided into first-level texts corresponding to the number of first-level titles; Specifically, after identifying all the first-level titles in the original text, the large language model can divide the original text based on the first-level titles, so that each first-level title corresponds to a first-level text, that is, each first-level title corresponds to the content of a chapter, so that the first divided text can maintain the integrity of the semantics, thereby facilitating the integrity of the semantics of each subtext in subsequent division.
[0039] For example, after dividing the above text example of the original text based on the first-level title, the specific example of the first-level text is as follows: The first first-level text: Chapter 1: Origins and Early Exploration of Language Models Statistical Language Model (N-gram): Basic Principles, Applications, and Limitations… Rule-based dialogue systems: early attempts such as ELIZA and SHRDLU… The beginnings of neural networks: the emergence of word embeddings (Word2Vec, GloVe) and simple neural network language models… Second level text: Chapter 2: Turning Points in the Neural Network Era (2014–2017) Language modeling capabilities of RNN and LSTM:… Seq2Seq and Attention: Introducing Stronger Contextual Processing… The birth of the Transformer architecture (2017): Self-Attention, parallel computing capabilities, long-distance dependency modeling… The subsequent first-level texts are segmented as described above, and each first-level text includes a chapter title and corresponding chapter content, making the content of the first-level text more complete.
[0040] S30: segmenting the primary text that does not meet the preset termination segmentation condition in parallel and progressively until the segmented sub-text meets the preset termination segmentation condition, and then stopping the segmentation; Specifically, the preset termination segmentation condition is a segmentation condition set for whether the text needs to be segmented, which is used to automatically determine whether the text needs to continue to be segmented or stop segmenting during the text segmentation process. Specifically, when the preset termination segmentation condition is met, the segmentation is stopped; when the preset termination segmentation condition is not met, parallel progressive segmentation continues.
[0041] If the original text, or the first-level text or sub-text after segmentation, does not meet the preset termination conditions, the original text, or the first-level text or sub-text after segmentation, can continue to the next round of segmentation. If the original text, or the first-level text or sub-text after segmentation, meets the preset termination conditions, the segmentation of the original text, or the first-level text or sub-text after segmentation, stops. By using the preset termination conditions to determine whether a document needs to be segmented, random segmentation can be avoided, thereby ensuring the semantic integrity of the text after multiple rounds of segmentation.
[0042] More specifically, parallel progressive segmentation involves processing all texts after each round of segmentation in parallel, treating all texts that meet the termination criteria as independent inputs for continued progressive segmentation. This parallel processing involves simultaneously applying the same pre-set termination criteria to multiple texts after each round of segmentation. Progressive segmentation involves progressively splitting the text according to the splitting rounds, forming a multi-level, coarse-to-fine, and overall-to-partial structured representation, such as from chapter to subsection, and then to paragraph or sentence structure.
[0043] Parallel processing, for example, the original text is divided into 5 primary texts, 5 primary texts do not meet the preset termination splitting condition, 5 primary texts will need to continue to split, 5 primary texts are parallel to split, the parallel number is 5, assuming that each primary text is split into 4 subtexts, then there are 5*4=20 subtexts in total, and the 20 subtexts will be parallel to continue to split, and the parallel number is 20. Each subtext is processed independently and does not interfere with each other. Through each round of parallel processing of the split document, the processing efficiency can be improved, while avoiding mutual interference between subtexts.
[0044] More specifically, the specific process of step S30 is as follows: When 5 primary titles are identified in the original text, the original text is first divided into 5 subtexts according to the 5 primary titles. Then, the judgment of the preset termination splitting condition is made for each subtext. If the judgment result is that the subtext can continue to be split, the subtext is taken as an independent input, and the primary title extraction and document splitting process is executed again. At this time, for the structural titles identified in the subtext, such as secondary titles, the system regards them as the "primary titles" of the subtext, and accordingly performs the next round of splitting operation. The above process can be recursively executed as needed until all subtexts meet the termination splitting condition.
[0045] Based on the example of the primary text given in step 20, if there is a primary text in step S20 that does not meet the preset termination splitting condition, parallel progressive splitting of all primary texts that do not meet the preset termination splitting condition is required.
[0046] Taking the second primary text as an example, the second primary text does not meet the preset termination splitting condition and can be parallelly and progressively split: Second primary text: Chapter 2: Turning point in the era of neural networks (2014-2017) Language modeling capabilities of RNN and LSTM:… Seq2Seq and Attention: Introducing stronger context processing… The birth of the Transformer architecture (2017): Self-Attention, parallel computing capabilities, long-distance dependency modeling… First, extract all secondary titles in the second primary text; First secondary title under the second primary text: Language modeling capabilities of RNN and LSTM; Second secondary title under the second primary text: Seq2Seq and Attention; The third subheading under the third primary text: The birth of the Transformer architecture (2017).
[0047] Secondly, the primary text is segmented based on the identified first subheading, and the segmented subtext is as follows: The first subtext under the second primary text: RNN and LSTM language modeling capabilities:… The second subtext under the second primary text: Seq2Seq and attention mechanism (Attention): Introducing stronger context processing… The third subtext under the third primary text: The birth of the Transformer architecture (2017): Self-Attention, parallel computing capabilities, long-distance dependency modeling… At this time, the remaining primary texts that do not meet the preset termination segmentation condition are segmented in the same way as the second primary text, and the primary text that meets the preset termination segmentation condition stops segmentation; and the subtext segmented by all primary texts is judged whether it can continue to be segmented in the subsequent round, if the subtext can continue to be segmented, it continues to be segmented in the same way, until the subsequent segmented subtext meets the preset termination segmentation condition, and the segmentation stops. This reflects the parallel processing of all texts at the same level, and progressive segmentation is performed for each segmented text.
[0048] S40: Construct a multi-level text structure tree based on the main title of the original text and all segmented primary texts and subtexts.
[0049] Specifically, in addition to the main title, each segmented text includes corresponding title content, so a multi-level text structure tree can be constructed in turn based on the main title of the original text and all segmented primary texts and subtexts. Please refer to Figure 2 , Figure 2 The text structure tree generated based on the above text example is given. Compared with the traditional segmentation method, the present application adopts a document parallel progressive multi-layer structure segmentation strategy, which identifies the text structure layer by layer and recursively constructs a multi-level text structure tree. Not only does it retain the original logical hierarchy of the original text, but it also reduces the problem of semantic fragmentation and structural disorder, making the relationship between each level clear and controllable, effectively improving the rigor and completeness of knowledge organization, and making the information organization of the text structure tree more accurate.
[0050] It should be noted that in this embodiment, the level of the constructed text structure tree is determined according to the splitting round.
[0051] Understandably, ensuring the accurate positioning of first-level headings is crucial for effective segmentation. Because the same heading may appear multiple times in the original text, a large language model must identify the actual structural location of each heading. Existing techniques generally consider the first occurrence of a structural heading in the text as its true structural demarcation point. For example, numbered headings, such as "Chapter 1" or "2.1," are less likely to appear repeatedly, and their structural boundaries are clear. Unnumbered headings, such as "Summary" or "Background Introduction," may appear multiple times within the text, particularly when quoting or reviewing context. In such cases, directly basing the decision on the first matching position may result in incorrect structural segmentation. For example, in a document, the title of Chapter 4 appears between the first and second headings in the preceding text, but its true structural location should be later, after the third heading. Segmenting the content based solely on the first occurrence of the first-level heading may result in errors. As described above, the first occurrence of Chapter 4 is between the first and second headings, which clearly does not conform to the normal order of chapters. Therefore, step S20 effectively avoids this problem of using the first matching position as the basis for incorrect structural segmentation.
[0052] See also Figure 3 Furthermore, step S20 specifically includes: S201: Locate the first position where the first level one heading first appears in the original text; S202: Determine a second position where a second first-level title first appears based on the first first-level title first appearing; S203: dividing the content between the first position and the second position into a first level text, with the first position and the second position as a boundary; S204: Continue locating the first-level titles in the original text, and use the second first-level title located in the previous time as the starting first-level title of the current segmentation, until the last first-level title is located, and segment the content between two adjacent first-level titles in turn to obtain first-level texts corresponding to the number of first-level titles.
[0053] Specifically, the position where the first primary title in the document first appears is usually correct and does not occur out of order. Therefore, after the large language model identifies the primary title in the original text, the first position where the first primary title appears is first located, and then the second position where the second primary title first appears is located based on the first position where the first primary title appears; the third position of the third primary title is located according to the second position where the second primary title first appears, and the positions of subsequent primary titles are located in the same way as described above, thereby realizing the positioning of the multiple primary titles appearing in the original text in turn. The division order between the multiple primary titles positioned is that the content between the first position of the first primary title first appearing and the second position of the second primary title first appearing is divided into the first primary text corresponding to the first primary title. The division of subsequent primary texts is performed in the same way, that is, the content between the directly adjacent two primary titles after the multiple primary titles are all positioned is divided, and the division of the last primary title is to the end of the original text.
[0054] Understandably, by steps S201-S204, the division of the primary text corresponding to the primary title is realized, the division of each paragraph of content according to the real structure position of the primary title is realized, the primary title appearing in an abnormal position is included in the content of the previous round of division, and the boundary of the divided text is clear and the semantics is complete. In addition, this division method can effectively avoid the structure disorder caused by title mismatch, the title appearing in the previous text will be automatically excluded from the interference, and the clarity and stability of the subsequent structure are ensured.
[0055] Understandably, after each round of text division is completed, each subtext generated in the new round is judged for termination division, that is, whether all subtexts after each round of division meet the termination division condition is judged to determine whether to continue to perform the title extraction and division operation of the next level. In this embodiment, by parallel processing of the text after each round of division, independent evaluation and parallel execution of each subdocument in each round are realized, and flexibility and efficiency are ensured.
[0056] In an alternative embodiment, the preset termination division condition includes judging whether the text length is greater than a preset text length threshold.
[0057] Specifically, a preset maximum text length threshold can be set for each text segment, for example, in the unit of word count, paragraph count, or Token quantity. If the length of the segmented primary text or a certain subtext is equal to or greater than the threshold, i.e., the preset termination splitting condition is not met, it is assumed by default that the subtext has the possibility of further structural division, thereby triggering a new round of primary title extraction and splitting process. If it is less than the threshold, i.e., the preset termination splitting condition is met, the primary text or the certain subtext cannot be further structurally divided, and the splitting of the primary text or the certain subtext is stopped. The preset text length threshold can be flexibly configured according to actual application scenarios, for example, it can be appropriately relaxed when processing long policy documents, technical manuals, etc., and a smaller upper limit can be set in the context of short articles, bulletins, etc.
[0058] Optionally, for multiple rounds of splitting, different preset text length thresholds can also be set according to the required splitting rounds to adapt to the length of the text corresponding to the multiple rounds of splitting, so as to avoid that the preset text length threshold is set too large at the beginning, and the subtext that can be split in the subsequent process is judged as unsplittable.
[0059] In an optional implementation, the preset termination splitting condition includes determining whether the text includes a substructure or a nested chapter by a large language model according to a preset prompt word.
[0060] Specifically, due to the irregularity of document structure and the difference in content, only text length judgment may not be able to comprehensively cover all splitting requirements. Therefore, a large language model with structure understanding and semantic analysis capabilities can also be introduced to evaluate the structural complexity of each subtext. The system analyzes whether the text still contains potential substructures or nested chapters through a preset specific prompt word. If the large language model analyzes that the text contains potential substructures or nested chapters, i.e., the preset termination splitting condition is not met, the text can continue to be recursively processed. If the large language model determines that the content structure of the text is basically complete and difficult to be further split in terms of semantics, i.e., the preset termination splitting condition is met, the text no longer continues to be recursively processed.
[0061] In an optional implementation, the preset termination splitting condition includes determining whether the text length is greater than a preset text length threshold, and determining whether the text includes a substructure or a nested chapter by a large language model according to a preset prompt word.
[0062] Specifically, in the present embodiment, two judgment conditions can be used together to determine whether the text needs to be segmented. In the preliminary screening stage, the text that may not need to be processed can be quickly filtered by the preset text length threshold to improve the overall processing efficiency; for the boundary case or the segment with ambiguous structure, the semantic auxiliary judgment is performed by the large language model to enhance the robustness and accuracy. The termination judgment mechanism not only maintains the structural accuracy, but also considers the computational efficiency and adaptability, providing high-quality node boundary division basis for the construction of subsequent hierarchical text structure tree.
[0063] It should be noted that the text that does not need to be processed can be the text that meets the judgment of the preset text length threshold, and the text that does not need to be determined by the large language model; the text with ambiguous structure or boundary condition can be the text that does not meet the judgment of the preset text length threshold.
[0064] It can be understood that in the present embodiment, the user can select any one of the above-mentioned embodiments to determine the preset termination segmentation condition of the segmented text. Any one of the above-mentioned termination segmentation conditions can determine whether to continue to perform the structure extraction and splitting operation of the next level.
[0065] Optionally, the parameter settings of the termination segmentation condition can also be dynamically adjusted according to specific task scenarios. For example, different tasks such as question and answer, extraction, and writing can correspond to different text length thresholds or large model judgment confidence thresholds to realize customized control strategy and improve the user's use experience.
[0066] Further, step S30 specifically comprises: S301: performing parallel segmentation processing on all first-level texts that do not meet the preset termination segmentation condition to obtain at least two second-level texts corresponding to each first-level text; Specifically, the parallel segmentation processing can be to separately segment each first-level text in the same way as the segmentation text applied in steps S201-S204, so as to obtain at least two second-level texts corresponding to the next level of each first-level text that can be segmented.
[0067] S302: determining whether all second-level texts under all first-level texts meet the preset termination segmentation condition; Specifically, whether it is the first-level text segmented from the original text, or the second-level text segmented from the first-level text, or the subtext segmented from the second-level text, or the text of the next level segmented from the subtext, after the segmentation is completed, each text needs to be judged by the preset termination segmentation condition. That is, each round of text segmentation needs to judge the segmented text by the preset termination segmentation condition to determine whether each segmented text can be segmented in the subsequent round.
[0068] S303: continue parallel progressive segmentation of all secondary texts that do not meet the preset termination segmentation condition to the next round until all subtexts after segmentation of the next round meet the preset termination condition and stop segmentation.
[0069] Specifically, in this embodiment, the overall stop segmentation is that after multiple rounds of segmentation, when all subtexts in this round will meet the preset termination condition, i.e., all subtexts cannot be segmented, the overall segmentation process of the original text is completed. If after segmentation, there is a text that does not meet the preset termination condition, parallel progressive segmentation will continue until after a round of segmentation, all texts meet the preset termination condition.
[0070] Understandably, by performing parallel progressive segmentation on each text after each round of segmentation, i.e., after each round of text structure segmentation is completed, termination segmentation judgment is performed on each subtext newly generated after segmentation to determine whether to continue to perform the next level of structure extraction and segmentation operation, the structure of the original text is identified layer by layer and the document tree is recursively constructed, not only the original logical level of the original text is preserved, but also the problem of semantic fragmentation and structure disorder is reduced, effectively improving the rigor and integrity of knowledge organization. And termination segmentation judgment is performed in parallel in each round of segmentation, and each subtext after each round of segmentation is independently evaluated and judged, ensuring the flexibility and efficiency of segmentation.
[0071] Further, after step S303, the method further comprises: S304: all primary texts, secondary texts and subtexts that meet the preset termination segmentation condition after multiple rounds of segmentation are taken as leaf nodes; Specifically, the leaf nodes are composed of texts that cannot be segmented after the segmentation process of the original text is completed, for example, an original text has 2 large chapters, and each chapter has 5 small sections, then it is finally segmented into 10 subtexts. The 10 subtexts are leaf nodes. The leaf nodes can be used for retrieval.
[0072] S305: information enhancement is performed on the leaf nodes to obtain information-enhanced leaf nodes.
[0073] Specifically, information enhancement is to complete the missing information of the leaf nodes, so that the information of the leaf nodes is complete. Since each leaf node is the text corresponding to the last level on each segmentation path, after segmentation, fine information enhancement processing can be performed on the content of each leaf node to improve the completeness and semantic accuracy of the information, thereby better supporting subsequent knowledge base construction and downstream task calling, such as question answering, summary generation, information matching, etc.
[0074] Please refer to Figure 4Further, step S305 specifically includes: S3051: identifying, by the large language model, whether there is missing target information in the leaf node according to the preset prompt instruction; the target information includes reference information; Specifically, the target information missing is due to the structural segmentation, which causes the context information in some content segments to be incomplete, and the manifestations include but are not limited to: “today” appears, but the specific date cannot be determined; “he” or “she” appears, but there is no clear information about the corresponding reference object; “this matter” appears, but the specific content of the event cannot be determined; The table field appears, but its semantic definition is located in other parts of the document, and is not included in the current slice, making it difficult to understand the field.
[0075] The target information in the embodiment can include reference information, which can include reference information in four dimensions of time, person, event, and table. The large language model can identify and judge the missing information one by one for the information in four dimensions of time, person, event, and table. If the content of the text is complete, no modification is made to the leaf node; if the target information is indeed missing, the missing type is explicitly marked so that the missing type information can be directly searched later.
[0076] More specifically, the large language model can identify whether there is a situation of target information missing in the input text content based on the preset information recognition prompt word.
[0077] The following gives an example of the large language model identifying whether there is a situation of target information missing in the input text content based on the preset information recognition prompt word: For example, the two adjacent leaf nodes are: Leaf node 1: Today, someone has been 62 years old, with a head full of gray hair telling the vicissitudes of life. He takes the bus and subway, and rides a bicycle. He is so simple that he doesn't even want to dye his hair, but just wears a hat to cover it up when he goes out.
[0078] Leaf node 2: In the eyes of outsiders, he has nothing, but he seems to have nothing. Friends around him one after another leave him, and he always appears alone in front of the camera.
[0079] The most painful thing is that his loneliness is rarely understood by others. In this complex environment, he chooses to be lonely for art and perfection. He adheres to his artistic belief, which is also the root of his loneliness.
[0080] The preset information recognition prompt word in the large language model is: Please determine if there is a case of target information missing in the following text content. Target information missing mainly refers to the incomplete context information in some content fragments due to structural segmentation. The forms include but are not limited to: "today" appears, but the specific date cannot be determined; "he" or "she" appears, but there is a lack of clear information about the corresponding referent; "this matter" appears, but the specific content of the event is not clear; The table field appears, but its semantic definition is located in other parts of the document, and is not included in the current slice, making it difficult to understand.
[0081] If so, output yes and the missing question. If not, output no.
[0082] Enter leaf node 2 into the large language model. The large language model can identify the missing information of "he" in leaf node 2 according to the above preset information.
[0083] S3052: If there is missing target information in the leaf node, trace back to the previous node of the leaf node with missing target information according to the segmentation path of the original text, and determine the missing target information in the leaf node from the previous node of the leaf node. S3053: Supplement the target information determined from a node to the leaf node missing the target information to obtain the information-enhanced leaf node.
[0084] Specifically, the segmentation in this embodiment is a parallel progressive segmentation, and multiple multi-level segmentation paths are formed with the main title of the original text as the origin. Each segmentation path includes multiple segmented texts that progress in turn according to the segmentation round. A text corresponds to a node at a level on the segmentation path. Therefore, the segmentation path of the leaf node with missing target information can be determined first, and the node at the previous level to which the leaf node belongs is determined based on the segmentation path, so that the content of the missing target information of the leaf node is determined from the node at the previous level. If the content of the missing target information of the leaf node is not identified in the node at the previous level, it needs to continue to identify along the node at the next level of the node at the previous level.
[0085] For example: Assuming that the current leaf node with missing target information is numbered 4.3.1.1, the large language model will first trace back to its previous node 4.3.1 to determine whether it contains the missing information; if not, continue to trace back to 4.3, and then to 4, until the node containing the required information is found. After finding, the relevant content is extracted and supplemented to the current leaf node content to achieve semantic integrity enhancement.
[0086] Thus, the step S3051-S3053 realizes the tracing of the split path to which the leaf node belongs, so as to realize the finding and completion of the missing target information in a step-by-step manner, thereby completing the information enhancement of the leaf node, making the ambiguous expression clear and the missing content complete, greatly improving the content integrity and clarity, and realizing the enhancement of the semantic integrity of the content included in each leaf node. In addition, compared with the split mode based on fixed rules in the prior art, through the parallel progressive split in the embodiment, after the split is completed, the information enhancement processing is performed on the leaf node, the missing information is completed, so as to avoid the problem that the text segment with semantic association may be split based on the existing fixed rules, causing the pronoun mentioned in the subsequent text to lose the explicit reference, and the problems of context semantic loss and local key information loss.
[0087] Please refer to Figure 5 Further, step S40 specifically includes: S401: determining the level of the text structure tree to be constructed according to the split round; S402: obtaining the main title of the original text, and taking the main title as the root node; Specifically, the level of the text structure tree to be constructed can be determined according to the total split round of the original text. The first round of split will form two levels, specifically, taking the main title of the original text as the root node of the first level, and taking each first-level text after the split as a structure node of the second level; the second round of split will form three levels, specifically, taking the main title of the original text as the root node of the first level, taking each first-level text after the first round of split as a structure node of the second level, and taking each second-level text after the second round of split as a structure node of the third level. The subsequent rounds of split and levels are similar, and the total level of the final document structure tree is the total split round of the original text plus one.
[0088] S403: generating the corresponding structure node according to the first-level text, the second-level text and the subtext formed after the multiple rounds of split, and giving each structure node a unique identifier; Specifically, all the first-level texts, the second-level texts and the subtexts formed after the split are structure nodes of the corresponding level. The structure node includes a node corresponding to the text that can be split and a leaf node corresponding to the text that cannot be split. And each structure node not only includes the corresponding level information, i.e., the level, but also includes the title content, the parent node, the child node list and the original text segment. The lower the level, the closer to the root node, for example, the first-level title is Level 1, the second-level title is Level 2, and so on.
[0089] More specifically, each structural node is given a unique identifier, which facilitates locating the corresponding structural node in the document structure tree, and improves retrieval efficiency.
[0090] Understandably, the leaf nodes in the structural nodes are leaf nodes after information enhancement processing, and the splitting path information of the leaf nodes in the original text can also be added to the content, effectively improving the accuracy of retrieval.
[0091] S404: Construct a mapping relationship between the parent node and the child node list to which each structural node belongs; the structural node includes title content, hierarchical level, parent node, child node list, and original text segment; Specifically, the mapping relationship is a parent-child mapping relationship between a structural node and its parent node and child node. The parent node of each structural node is the splitting source node of the previous level; the child node is the next level structural node further identified in the node text segment. By establishing this parent-child mapping, the complete restoration of the tree structure can be achieved, ensuring accurate positioning of each text in the overall structure of the document. For example, taking the subtext of the second section of the first chapter in the text as a structural node, the adjacent nodes are the first section of the first chapter and the third section of the first chapter. The first chapter directly connected is its parent node, and the first paragraph and the second paragraph under the first section of the first chapter directly connected to the second section of the first chapter are its child nodes. The parent node of each structural node has only one, and the child node can have none or at least one.
[0092] S405: Construct a multi-level text structure tree corresponding to the original text based on the root node, the structural node, and the mapping relationship.
[0093] Specifically, based on the root node, all structural nodes, and corresponding hierarchical information, and the mapping relationship between all structural nodes, a multi-level document structure tree corresponding to the original text can be constructed. This document structure tree not only includes the original content of the original text, but also makes the text structure tree formally present as a multi-level knowledge organization tree with clear parent-child levels, complete content mounting, and rich semantic extraction. Moreover, this text structure tree structure not only significantly improves the visibility and controllability of the original text structure, but also provides a solid data foundation and structural guarantee for downstream tasks such as knowledge retrieval, content generation, and intelligent question answering. In addition, the final generated document structure tree clearly presents the parent-child mapping relationship of each chapter and its substructure, and the content information of any specified chapter can be obtained, so that the original text is no longer a flat text collection, but a knowledge graph input source with clear levels, complete structure, and rich semantics, greatly improving the retrieval efficiency and generation quality of the intelligent agent system.
[0094] It can be understood that the embodiment realizes building a document structure tree based on the actual hierarchy of the original text, and adds hierarchy information into the structure node to enhance the global information of the slice; and locally, for the leaf node, the time, character, event, table and other information of each leaf node are enhanced to further enhance the information of the leaf node. In the retrieval, the global information and the local information of the text slice are enhanced, so that the retrieval accuracy and comprehensiveness of different granularity query requests can be improved.
[0095] Further, after step S10, the method further comprises: S11: if all first-level texts meet the preset termination segmentation condition, stop segmentation; S12: constructing a text structure tree based on the main title of the original text and all segmented first-level texts.
[0096] It can be understood that if all first-level texts formed after the first segmentation of the original text meet the preset termination segmentation condition, the segmentation process is stopped, and a two-layer text structure tree is constructed directly with the main title of the original text and all segmented first-level texts. The first layer is the root node corresponding to the main title, and the second layer is the structure node corresponding to all first-level texts under the main title. Each structure node includes a first-level title and corresponding text content. The two-layer text structure tree is more suitable for short documents, and the structure of the document is simpler and clearer.
[0097] It should be noted that in the embodiment, segmentation is performed according to the levels present in the original text, that is, the original text has several levels, and segmentation can be performed for several levels. When there is no level in the original text, segmentation can not be performed. The specific segmentation condition is determined according to the actual structure of the original text, and the embodiment does not limit the specific segmentation level of each text.
[0098] Referring to Figure 6 The second embodiment of the present application provides a computer device 1, which comprises a storage 10, a processor 11 and a computer program 12 stored in the storage 10. The processor 11 executes the computer program 12 to realize the above-mentioned text knowledge base construction method.
[0099] It can be understood that the computer device 2 of the embodiment has the same beneficial effects as the above-mentioned text knowledge base construction method, and will not be described here.
[0100] Referring to Figure 7 The third embodiment of the present application provides a computer program product 2, which stores a computer program 20. The computer program 20 is executed by a processor to realize the above-mentioned text knowledge base construction method.
[0101] It can be understood that the computer program product 2 of the embodiment has the same beneficial effects as the method for constructing a text knowledge base described above, and will not be described here.
[0102] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that the determination of B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information.
[0103] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least two embodiments of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should also be understood by those skilled in the art that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.
[0104] In various embodiments of the present application, it should be understood that the size of the serial number of the above processes does not mean the inevitable sequence of execution, and the execution sequence of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0105] In the flowcharts and block diagrams of the drawings of the present application, the possible implementation architecture, function and operation of the system, method and computer program product according to various embodiments of the present application are illustrated. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can also occur in different order from that marked in the drawings. For example, two blocks represented in succession can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, based on the functions involved. It should be particularly noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for executing the specified function or operation, or can be implemented by a combination of special-purpose hardware and computer instructions.
[0106] The text knowledge base construction method, the computer device, and the program product are described in detail above, and the principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges can be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application. Any modification, equivalent replacement, improvement, and the like within the principles of the present application should be included in the protection scope of the present application.
Claims
1. A method for constructing a text knowledge base, characterized by: The following steps are involved: Obtaining an original text, and extracting a first-level title of the original text using a large language model; Segment the original text based on the first-level headings to obtain first-level texts corresponding to the number of the first-level headings; The original text is segmented based on the first-level headings to obtain first-level texts corresponding to the number of first-level headings, specifically including: Locate the first position where the first level one heading first appears in the original text; Determine a second position of the first occurrence of the second first level heading based on the first occurrence of the first first level heading; Taking the first position and the second position as the boundary, the content between the first position and the second position is divided into the first level one text; Continue locating the first-level headings in the original text, and use the second first-level heading located in the previous segmentation as the starting first-level heading of the current segmentation, until the last first-level heading is located, and segment the content between two adjacent first-level headings in turn to obtain first-level text corresponding to the number of first-level headings; The first-level text that does not meet the preset termination segmentation condition is segmented in parallel and progressively until the segmented sub-text meets the preset termination segmentation condition, and then the segmentation stops; A multi-level text structure tree is constructed based on the main title of the original text and all the segmented primary texts and sub-texts; the text structure tree includes all the original contents of the original text.
2. The method for constructing a text knowledge base according to claim 1, wherein: After obtaining the original text and extracting the first-level title of the original text using the large language model, the method further includes: Use the large language model to determine whether each extracted first-level title appears in the original text; If the extracted first-level title content does not exist in the original text, the extraction fails and the first-level title extraction operation is performed again.
3. The method for constructing a text knowledge base according to claim 1, wherein: The preset termination segmentation condition includes determining whether the text length is greater than a preset text length threshold, and / or performing semantic judgment on the text based on preset prompt words through a large language model to determine whether the text includes substructures or nested chapters.
4. The method for constructing a text knowledge base according to claim 3, wherein: After segmenting the original text based on the first-level headings to obtain first-level texts corresponding to the number of the first-level headings, the method further includes: If all the first-level texts meet the preset termination conditions, the segmentation will stop; Construct a text structure tree based on the main title of the original text and all the first-level texts after segmentation.
5. The method for constructing a text knowledge base according to claim 4, wherein: The first-level text that does not meet the preset termination segmentation conditions is segmented in parallel and progressively until the segmented sub-text meets the preset termination segmentation conditions, and then the segmentation stops. Specifically, it includes: Segment all the first-level texts that do not meet the preset termination segmentation condition in parallel to obtain at least two second-level texts corresponding to each first-level text; Determine whether all the second-level texts under all the first-level texts meet the preset termination segmentation condition; All secondary texts that do not meet the preset termination segmentation condition will continue to be segmented in parallel and progressively in subsequent rounds until all sub-texts after segmentation in subsequent rounds meet the preset termination condition and the segmentation is stopped.
6. The method for constructing a text knowledge base according to claim 5, wherein: Continuing to segment all secondary texts that do not meet the preset termination condition in parallel and progressively in subsequent rounds until all subtexts segmented in subsequent rounds meet the preset termination condition, and then stopping the segmentation, the method further includes: All first-level texts, second-level texts, and sub-texts that meet the preset termination conditions after multiple rounds of segmentation are regarded as leaf nodes; Information enhancement is performed on the leaf nodes to obtain information-enhanced leaf nodes.
7. The method for constructing a text knowledge base according to claim 6, wherein: Performing information enhancement on the leaf node to obtain an information-enhanced leaf node specifically includes: Identify whether there is missing target information in the leaf node according to the preset prompt instruction through the large language model; the target information includes reference information; If there is a leaf node missing the target information, then trace the leaf node missing the target information step by step according to the segmentation path of the original text, and determine the target information missing in the leaf node from the upper layer node of the leaf node missing the target information; The target information determined from the nodes of the first layer is supplemented to the leaf nodes that lack the target information to obtain the leaf nodes after information enhancement.
8. The method for constructing a text knowledge base according to claim 7, wherein: Construct a multi-level text structure tree based on the main title of the original text and all the first-level texts and sub-texts after segmentation, including: Determine the level of the text structure tree to be constructed according to the segmentation rounds; Get the main title of the original text and use the main title as the root node; After multiple rounds of segmentation, the primary text, secondary text, and subtext are formed into corresponding structural nodes, and each structural node is given a unique identifier; Constructing a mapping relationship between the parent node and the child node list of each structural node; the structural node includes title content, hierarchical level, parent node, child node list and original text segment; A multi-level text structure tree corresponding to the original text is constructed based on the root node, the structure nodes and the mapping relationship.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the method for constructing a text knowledge base according to any one of claims 1 to 8.
10. A computer program product having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for constructing a text knowledge base according to any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Paragraph analysis method and device, equipment and storage medium
CN114154479A
Text segmentation method and device for retrieval enhancement generation
CN119311723A
Knowledge retrieval-oriented government affair document enhanced generation method, electronic equipment and medium
CN119416776A
Document segmentation method and device, computer equipment and storage medium
CN119474250A
Document title hierarchy extraction method and device, equipment and medium
CN119808770A