Method for constructing text knowledge base based on large language model, computer device and program product

Abstract

The present application relates to the technical field of natural language processing, and particularly relates to a text knowledge base construction method, computer equipment and program product. The present application provides a text knowledge base construction method, which extracts a first-level title of an original text through a large language model; splits the original text based on the first-level title to obtain first-level texts corresponding to the number of first-level titles; performs parallel progressive splitting on the first-level texts that do not meet a preset termination splitting condition until the split subtexts meet the preset termination splitting condition, and then stops the splitting; and constructs a multi-level text structure tree based on the main title of the original text and all the split first-level texts and subtexts. The method realizes layer-by-layer identification of the text structure and recursive construction of the multi-level text structure tree, not only retains the original logical hierarchy of the original text, but also reduces the problems of semantic fragmentation and structural disorder, so that the hierarchical relationship is clear and controllable, and the rigor and integrity of knowledge organization are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a method for constructing a text knowledge base based on a large language model, a computer device and a program product. BACKGROUND

[0002] With the development of intelligent agents, the ability to acquire and utilize external knowledge is a very key technology in the construction of intelligent agents. The most typical one is the RAG (Retrieval-augmented Generation) technology, which provides the intelligent agent with corresponding knowledge by retrieving document slices.

[0003] In the existing technologies such as RAG, the original document is usually first divided and stored in a vector database, and then when a query request is received, a number of most relevant paragraphs are retrieved and sorted from the vector database, and finally the part with the highest confidence is delivered as a prompt input to the large model. However, the current mainstream document division method is mostly based on fixed rules, such as division by word count, paragraph or punctuation. This static and coarse-grained division method is prone to cause a single chapter to be cut off or multiple chapters to be mixed, making it difficult to accurately restore the original document structure and affecting the accuracy of information organization. SUMMARY

[0004] In order to solve the problem that the existing document division method is difficult to accurately restore the original document structure and affects the accuracy of information organization, the present application provides a method for constructing a text knowledge base based on a large language model, a computer device and a program product.

[0005] The technical problem solving scheme of the present application provides a method for constructing a text knowledge base based on a large language model, comprising the following steps: obtaining an original text, extracting a first-level title of the original text based on explicit markers and implicit semantic features through a large language model; dividing the original text based on the first-level title to obtain a first-level text corresponding to the number of first-level titles; dividing the original text based on the first-level title to obtain a first-level text corresponding to the number of first-level titles, specifically including: locating the first position of the first occurrence of the first first-level title in the original text; determining the second position of the first occurrence of the second first-level title based on the first occurrence of the first first-level title; dividing the content between the first position and the second position as the first first-level text with the first position and the second position as boundaries; continue to locate the first-level title in the original text, and sequentially take the second first-level title located in the previous time as the starting first-level title for the current time cutting, until the last first-level title is located, and the content between the adjacent two first-level titles is cut to obtain the first-level text corresponding to the number of first-level titles; the first-level text that does not meet the preset termination cutting condition is cut in parallel and progressively until the subtext after cutting meets the preset termination cutting condition, and the cutting is stopped; the preset termination cutting condition includes determining whether the text includes a substructure or a nested chapter through a large language model according to a preset prompt word; the parallel progressive cutting is parallel processing for all texts after each cutting, and all texts meeting the termination cutting condition are taken as independent inputs to continue progressive cutting; the progressive splitting is to split the text according to the cutting round; a multi-level text structure tree is constructed based on the main title of the original text and all first-level texts and subtexts after cutting; the text structure tree includes all original contents of the original text; the text structure tree takes the main title of the original text as the root node of the first level, takes each first-level text after the first round of cutting as a structure node of the second level, and takes each subtext after the current round of cutting as a structure node for the next round of cutting in the subsequent multiple rounds of cutting; the structure node includes title content, level, parent node, child node list, original text segment, and mapping relationship between the parent node and the child node list.

[0006] Preferably, after obtaining the original text and extracting the first-level title of the original text through the large language model, the method further comprises: determining whether each extracted first-level title appears in the original text through the large language model; if the content of the extracted first-level title does not exist in the original text, the extraction fails, and the extraction operation of the first-level title is performed again.

[0007] Preferably, the preset termination cutting condition includes determining whether the text length is greater than a preset text length threshold.

[0008] Preferably, after the original text is segmented based on the primary titles to obtain primary texts corresponding to the number of primary titles, the method further comprises: if all primary texts meet a preset termination segmentation condition, stopping segmentation; and constructing a text structure tree based on the main title of the original text and all segmented primary texts.

[0009] Preferably, the primary texts that do not meet the preset termination segmentation condition are subjected to parallel progressive segmentation until the segmented subtexts meet the preset termination segmentation condition, and the segmentation is stopped, specifically comprising: parallel segmentation processing all primary texts that do not meet the preset termination segmentation condition to obtain at least two secondary texts corresponding to each primary text; determining whether all secondary texts under all primary texts meet the preset termination segmentation condition; and continuing parallel progressive segmentation of all secondary texts that do not meet the preset termination segmentation condition in subsequent rounds until all subtexts segmented in the subsequent rounds meet the preset termination condition and the segmentation is stopped.

[0010] Preferably, after the primary texts, secondary texts and subtexts that meet the preset termination segmentation condition after multiple rounds of segmentation are stopped, the method further comprises: taking all primary texts, secondary texts and subtexts that meet the preset termination segmentation condition after multiple rounds of segmentation as leaf nodes; and performing information enhancement on the leaf nodes to obtain information-enhanced leaf nodes.

[0011] Preferably, the information enhancement on the leaf nodes to obtain information-enhanced leaf nodes specifically comprises: identifying whether there is missing target information in the leaf nodes according to a preset prompt instruction through a large language model; the target information includes reference information; if there is missing target information in the leaf nodes, performing level-by-level tracing on the leaf nodes with missing target information according to a segmentation path of the original text to determine the missing target information in the leaf nodes from a previous layer of the leaf nodes with missing target information; and supplementing the determined target information from a layer to the leaf nodes with missing target information to obtain information-enhanced leaf nodes.

[0012] Preferably, a multi-level text structure tree is constructed based on the main title of the original text and all the first-level texts and subtexts after segmentation, specifically including: determining the level of the text structure tree to be constructed according to the segmentation round; obtaining the main title of the original text, taking the main title as the root node; generating corresponding structure nodes according to the first-level texts, second-level texts and subtexts formed after multiple rounds of segmentation, and giving each structure node a unique identifier; constructing the mapping relationship between the parent node and the child node list to which each structure node belongs; the structure node includes title content, level, parent node, child node list and original text segment; and constructing the multi-level text structure tree corresponding to the original text based on the root node, the structure node and the mapping relationship.

[0013] To solve the above technical problems, the present application provides another technical solution as follows: a computer device comprising a storage, a processor and a computer program stored on the storage, wherein the processor executes the computer program to implement the construction method of a text knowledge base based on a large language model according to any one of the above.

[0014] To solve the above technical problems, the present application provides another technical solution as follows: a computer program product having a computer program stored thereon, wherein the computer program is executed by a processor to implement the construction method of a text knowledge base based on a large language model according to any one of the above.

[0015] Compared with the prior art, the construction method of a text knowledge base based on a large language model, the computer device and the program product provided by the present application have the following beneficial effects:

[0016] 1. The construction method of a text knowledge base based on a large language model provided by the present application first extracts a first-level title of an original text through a large language model, then locates a first position where a first first-level title first appears and a second position where a second first-level title first appears in the original text, and divides a first first-level text according to the positions of the two; the division of subsequent first-level texts is carried out in the same way to obtain first-level texts corresponding to the number of first-level titles; and the first-level texts that do not meet the preset termination segmentation condition are continuously subjected to parallel progressive segmentation until the subtexts after segmentation meet the preset termination segmentation condition, and the segmentation is stopped; finally, a multi-level text structure tree is constructed based on the main title of the original text and all the first-level texts and subtexts after segmentation. Compared with the traditional segmentation method, the present application adopts a document parallel progressive multi-level structure segmentation strategy, identifies the text structure layer by layer and recursively constructs a multi-level text structure tree, not only retains the original logical level of the original text, but also reduces the problems of semantic fragmentation and structural disorder, makes the hierarchical relationship clear and controllable, effectively improves the rigor and integrity of knowledge organization, and makes the information organization of the text structure tree more accurate.

[0017] 2、In the present application, after each round of cutting, the cut texts are judged one by one for preset termination cutting conditions, that is, by judging whether the text length is greater than the preset text length threshold, and / or by using a large language model to determine whether the text includes substructures or nested chapters according to the preset prompt words, to determine whether the original text and the cut texts continue to perform the next level of structure extraction and cutting operation. Through the judgment of text length and / or large language model, the fragments that may not need to be processed can be quickly filtered, the overall processing efficiency is improved, and the computational efficiency and adaptability are considered while maintaining the accuracy of the text structure, providing high-quality node boundary division basis for the construction of subsequent hierarchical text structure tree.

[0018] 3、In the present application, all first-level texts that do not meet the preset termination cutting conditions are processed in parallel to obtain at least two second-level texts under each first-level text; and whether all second-level texts under all first-level texts meet the preset termination cutting conditions is judged to continue the subsequent round of parallel progressive cutting for all second-level texts that do not meet the preset termination cutting conditions, until all subtexts after the subsequent round of cutting meet the preset termination conditions to stop cutting. Each document after cutting is processed in parallel and progressively, that is, after each round of text structure cutting, each subtext generated after cutting is judged one by one for termination cutting to determine whether to continue the next level of structure extraction and cutting operation. The termination cutting judgment is performed in parallel in each round of cutting, and each subtext after each round of cutting is independently evaluated and judged to ensure the flexibility and efficiency of the cutting.

[0019] 4、In the present application, all first-level texts, second-level texts and subtexts that meet the preset termination cutting conditions after multiple rounds of cutting are used as leaf nodes; and the leaf nodes are enhanced for information, so that the integrity and semantic accuracy of the content included in all leaf nodes are improved, thereby better supporting the subsequent knowledge base construction and downstream task calling.

[0020] 5、In the present application, when the large language model determines that there is missing target information in the leaf node, the missing target information in the leaf node is traced back level by level according to the cutting path of the original text to determine the missing target information in the leaf node from the previous layer node of the leaf node with missing target information; and the target information determined from the previous layer node is supplemented to the leaf node with missing target information to obtain the information-enhanced leaf node. Thus, the missing information is identified, and the missing information is found and determined and completed through the method of tracing back level by level, so as to enhance the semantic integrity of the content included in each leaf node.

[0021] 6、The text structure tree to be constructed can be determined according to the segmentation round in the application; the main title of the original text is taken as the root node; the corresponding structure nodes are generated according to the first-level text, the second-level text and the subtext formed after multiple rounds of segmentation, and each structure node is given a unique identifier; the mapping relationship between the parent node and the child node list to which each structure node belongs is constructed; so that each structure node includes title content, hierarchical level, parent node, child node list and original text segment; finally, the multi-level text structure tree corresponding to the original text is constructed based on the root node, the structure node and the mapping relationship, so that the text structure tree presents a multi-level knowledge organization tree with clear parent-child hierarchy, complete content mounting and rich semantic extraction in form. The text structure tree structure not only significantly improves the visibility and controllability of the original text structure, but also provides a solid data foundation and structural guarantee for downstream tasks such as knowledge retrieval, content generation and intelligent question answering.

[0022] 7、The application also provides a computer device having the same beneficial effects as the above-mentioned method for constructing a text knowledge base based on a large language model, which will not be repeated here.

[0023] 8、The application also provides a computer program product having the same beneficial effects as the above-mentioned method for constructing a text knowledge base based on a large language model, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0025] Figure 1 is a step flow chart of the method for constructing a text knowledge base based on a large language model provided by the first embodiment of the application.

[0026] Figure 2 is an example diagram of the document structure tree of the original text of the method for constructing a text knowledge base based on a large language model provided by the first embodiment of the application.

[0027] Figure 3 is a step flow chart of steps S201 to S204 of the method for constructing a text knowledge base based on a large language model provided by the first embodiment of the application.

[0028] Figure 4 is a step flow chart of steps S3051 to S3053 of the method for constructing a text knowledge base based on a large language model provided by the first embodiment of the application.

[0029] Figure 5 is a step flow chart of steps S401 to S405 of the method for constructing a text knowledge base based on a large language model provided by the first embodiment of the present application.

[0030] Figure 6 is a frame schematic diagram of a computer device provided by the second embodiment of the present application.

[0031] Figure 7 is a frame schematic diagram of a computer program product provided by the third embodiment of the present application.

[0032] Explanation of the drawings:

[0033] 1, computer device; 2, computer program product;

[0034] 10, memory; 11, processor; 12, computer program; 20, computer program. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0036] Please refer to Figure 1 and Figure 2 The first embodiment of the present application provides a method for constructing a text knowledge base based on a large language model, comprising the following steps:

[0037] S10: obtaining an original text and extracting a first-level title of the original text by a large language model;

[0038] Specifically, the original text can be an original document corresponding to a document slice stored in a vector retrieval library and returned by an intelligent agent for retrieval without being processed by slicing. The large language model can be a general large language model, such as qwen3, deepseek, etc. The user can select according to the actual situation, and the present embodiment does not limit this.

[0039] In step S10, after the complete original text is input into the large language model with structure understanding capability, the output format can be specified in the preset output format prompt word, such as JSON (JavaScript Object Notation), to guide the large language model to extract the first-level title of the original text and output the first-level title in the corresponding format. By extracting the first-level title of the original text through the large language model, the robustness of the large language model can be improved. Even if the original text has inconsistent formats, mixed arrangements, or is converted from a scanned copy, the large language model can still recognize the potential structure division through its language understanding capability. In addition, compared with directly extracting multi-level titles, the identification of first-level titles is more accurate and stable. Since the number of title levels in the original text is not fixed, directly extracting multi-level structures may lead to inconsistent results, chaotic formats, and even title rewriting or omission. Different models have different performances in multi-level extraction, which further affects the reliability. By focusing on the first-level title, the consistency and accuracy of extraction can be significantly improved.

[0040] It should be noted that the title extracted by the large language model should be consistent with the content of the original text, including spaces, symbols, and other details, to ensure accurate format restoration. The large language model utilizes explicit markers and implicit semantic features in the extraction process to determine title boundaries and their logical attribution, thereby accurately locating and determining the first-level title of the original text. Explicit markers include numbers, punctuation, and formatting styles. Implicit semantic features include topic content and paragraph aggregation.

[0041] It can be understood that to ensure extraction quality, after the large language model extracts the first-level title, all first-level titles can be checked one by one to determine whether they actually appear in the original document. If there is a missing or mismatching case, it is determined that the extraction fails, and the extraction operation is performed again to further ensure the integrity and usability of the extraction result.

[0042] It should be noted that when the original text document is short in length and there is no division or chapter description in the original text, the original text does not need to be divided. When there is no explicit chapter division in the original text, but the content of the original text is composed of different module contents, the first-level title of such original text can be obtained from the sub-titles of different modules. Specifically, the large language model can be used to determine whether there is a first-level title in the original text. If there is no first-level title in the original text, it is determined whether there is a module topic in the original text. If there is a module topic in the original text, the module topic is extracted as the first-level title of the original text. In this way, the first-level title is extracted for different document lengths and different document characteristics, thereby providing the application range of the present embodiment.

[0043] The following gives a text example of an original text;

[0044] The Evolution of Large Language Models

[0045] Chapter 1: Origins and Early Exploration of Language Models

[0046] Statistical Language Models (N-gram): Basic Principles, Application Areas, and Limitations…

[0047] Rule-based Dialogue Systems: Early Attempts like ELIZA, SHRDLU…

[0048] The Emergence of Neural Networks: Word Embeddings (Word2Vec, GloVe) and Simple Neural Network Language Models…

[0049] Chapter 2: The Turning Point of the Neural Network Era (2014-2017)

[0050] RNNs and LSTMs' Language Modeling Capabilities:…

[0051] Seq2Seq and Attention: Introducing Stronger Contextual Processing…

[0052] The Birth of the Transformer Architecture (2017): Self-Attention, Parallel Computing Capabilities, Long-Distance Dependency Modeling…

[0053] Chapter 3: The Rise of Pretrained Language Models (2018-2020)

[0054] BERT and Bidirectional Contextual Modeling: The Innovation of Masked Language Modeling…

[0055] The Initial Attempts of the GPT Series (GPT-1 / 2): Causal Language Modeling and Enhanced Generation Capabilities…

[0056] The Unified Task Modeling Concept of Models like T5, BART: Converting NLP Tasks into "Text-to-Text"…

[0057] Chapter 4: Large-Scale Models and General-Purpose Language Intelligence (2020-2023)

[0058] GPT-3: Parameter Explosion and Few-shot Capabilities:…

[0059] The Evolution of Prompt Learning and Fine-Tuning Methods: In-context Learning, Prompt Tuning, LoRA, and Other Techniques…

[0060] The Initial Integration of Multimodal Models: Visual-Language Models like CLIP, DALL·E, Flamingo…

[0061] Emergent Abilities: Scaling beyond size…

[0062] Chapter 5: Agents, Multimodality, and Future Trends (2023–Present)

[0063] GPT-4 and Multimodal Capabilities (GPT-4V):…

[0064] Language Models as Agents: Tool Use, Task Planning, Autonomous Execution (e.g., AutoGPT, ReAct)…

[0065] Open-Source Large Language Models and Community-Driven Innovation: LLaMA, Mistral, Gemma, and More…

[0066] Future Challenges and Directions: Safety Alignment, Cost Optimization, Pathways to General Intelligence (AGI)…

[0067] In the above text example, the development history of large language models is the main title. Chapter 1: Origins and Early Exploration of Language Models; Chapter 2: Turning Point in the Era of Neural Networks (2014–2017); Chapter 3: Rise of Pretrained Language Models (2018–2020); Chapter 4: Large-Scale Models and General Language Intelligence (2020–2023); Chapter 5: Agents, Multimodality, and Future Trends (2023–Present), the content of the five chapters is the first-level title that can be extracted.

[0068] S20: Based on the first-level title, the original text is divided into first-level texts corresponding to the number of first-level titles;

[0069] Specifically, after identifying all the first-level titles in the original text, the large language model can divide the original text based on the first-level titles, so that each first-level title corresponds to a first-level text, that is, each first-level title corresponds to the content of a chapter, so that the first divided text can maintain the integrity of the semantics, thereby facilitating the integrity of the semantics of each subtext in subsequent division.

[0070] For example, after dividing the above text example of the original text based on the first-level title, the specific example of the first-level text is as follows:

[0071] First first-level text:

[0072] Chapter 1: Origins and Early Exploration of Language Models

[0073] Statistical Language Models (N-gram): Basic Principles, Application Areas, and Limitations…

[0074] Rule-Based Dialogue Systems: Early Attempts by ELIZA, SHRDLU, and More…

[0075] The sprouting of neural networks: the advent of word embeddings (Word2Vec, GloVe) and simple neural network language models…

[0076] Second level text:

[0077] Chapter 2: Turning point in the era of neural networks (2014-2017)

[0078] Language modeling capabilities of RNNs and LSTMs:…

[0079] Seq2Seq and Attention: Introducing stronger context processing…

[0080] The advent of the Transformer architecture (2017): Self-Attention, parallel computing capabilities, modeling of long-range dependencies…

[0081] Subsequent level texts are divided as described above, and each level text includes a chapter title and corresponding chapter content, making the content of the level text more complete.

[0082] S30: Perform parallel progressive cutting on the level text that does not meet the preset termination cutting condition until the cut subtext meets the preset termination cutting condition, and stop cutting;

[0083] Specifically, the preset termination cutting condition is a cutting condition set for whether the text needs to be cut, which is used to automatically determine whether the text needs to be cut or stopped during the text segmentation process. Specifically, when the preset termination cutting condition is met, the cutting is stopped; when the preset termination cutting condition is not met, the parallel progressive cutting is continued.

[0084] When the original text or the cut level text or subtext does not meet the preset termination cutting condition, the original text or the cut level text or subtext can continue to be cut in the next round; when the original text or the cut level text or subtext meets the preset termination cutting condition, the original text or the cut level text or subtext stops cutting. By determining whether the document needs to be cut based on the preset termination cutting condition, random cutting that destroys the text semantics is avoided, so as to ensure the integrity of the semantics of the text after multiple rounds of splitting as much as possible.

[0085] More specifically, the parallel progressive segmentation is to process all the texts after each round of segmentation in parallel, and all the texts that meet the termination segmentation condition are taken as independent inputs to continue the progressive segmentation. The parallel processing is to judge the same preset termination segmentation condition for multiple texts after each round of segmentation at the same time. The progressive segmentation is to split the texts step by step according to the splitting round, and split the segmented texts into a structured representation forming a multi-level structure from coarse to fine and from whole to part, such as from chapter to section, and then to paragraph or sentence.

[0086] The parallel processing is, for example, the original text is split into 5 primary texts, none of the 5 primary texts meets the preset termination segmentation condition, and the 5 primary texts all need to continue to be split, so the 5 primary texts are split in parallel, the parallel number is 5, and assuming that each primary text is split into 4 subtexts, there are now a total of 5*4=20 subtexts, and the 20 subtexts will be split in parallel, and the parallel number is 20. The subtexts are processed independently and do not interfere with each other in parallel. Through parallel processing of the segmented texts in each round, the processing efficiency can be improved, and mutual interference between subtexts can be avoided.

[0087] More specifically, the specific process of step S30 is as follows: when 5 primary titles are identified in the original text, the original text is first divided into 5 subtexts according to the 5 primary titles. Then, the preset termination segmentation condition is judged for each subtext. If the judgment result is to continue to split, the subtext is taken as an independent input, and the primary title extraction and document segmentation process is executed again. At this time, the structural title such as the secondary title identified in the subtext is regarded by the system as the “primary title” of the subtext, and the next round of segmentation operation is performed accordingly. The above process can be recursively executed as needed until all subtexts meet the termination splitting condition.

[0088] Based on the example of the primary text given in step 20, if there is a primary text in step S20 that does not meet the preset termination segmentation condition, parallel progressive segmentation is performed on all the primary texts that do not meet the preset termination segmentation condition.

[0089] Taking the second primary text as an example, the second primary text does not meet the preset termination segmentation condition, and parallel progressive segmentation can be performed:

[0090] The second primary text:

[0091] Chapter 2: Turning point in the era of neural networks (2014-2017)

[0092] Language modeling capabilities of RNN and LSTM:…

[0093] Seq2Seq and attention mechanism (Attention): Introduce stronger context processing…

[0094] The birth of the Transformer architecture (2017): Self-Attention, parallel computing capabilities, long-distance dependency modeling…

[0095] First, extract all the second-level titles in the second first-level text;

[0096] The first second-level title under the second first-level text: RNN and LSTM language modeling capabilities;

[0097] The second second-level title under the second first-level text: Seq2Seq and attention mechanism (Attention);

[0098] The third second-level title under the third first-level text: The birth of the Transformer architecture (2017).

[0099] Second, based on the identified first second-level title, the first-level text is divided, and the divided subtext is as follows:

[0100] The first subtext under the second first-level text: RNN and LSTM language modeling capabilities:…

[0101] The second subtext under the second first-level text: Seq2Seq and attention mechanism (Attention): Introduce stronger context processing…

[0102] The third subtext under the third first-level text: The birth of the Transformer architecture (2017): Self-Attention, parallel computing capabilities, long-distance dependency modeling…

[0103] At this time, the remaining first-level texts that do not meet the preset termination segmentation conditions are simultaneously segmented in the same way as the second first-level text, and the first-level texts that meet the preset termination segmentation conditions are stopped from segmentation; and the subtexts segmented from all first-level texts are judged for the preset termination segmentation conditions, whether the segmented subtexts can continue to be segmented in subsequent rounds, if the subtexts can continue to be segmented, continue to be segmented in the same way, until the subtexts segmented in subsequent rounds meet the preset termination segmentation conditions, and stop segmentation. This reflects parallel processing of all texts at the same level, and progressive segmentation of each segmented text.

[0104] S40: Based on the main title of the original text and all first-level texts and subtexts segmented, a multi-level text structure tree is constructed.

[0105] Specifically, each segmented text in this embodiment will include the corresponding title content in addition to the main title, so a multi-level text structure tree can be directly constructed according to the main title of the original text and all the segmented first-level texts and subtexts in turn. Please refer to Figure 2 , Figure 2 The text structure tree generated based on the above text example is given. Compared with the traditional segmentation method, the present application adopts a document parallel progressive multi-level structure segmentation strategy, recognizes the text structure layer by layer and recursively constructs a multi-level text structure tree, which not only retains the original logical hierarchy of the original text, but also reduces the problems of semantic fragmentation and structural disorder, makes the hierarchical relationship clear and controllable, effectively improves the rigor and integrity of knowledge organization, and makes the information organization of the text structure tree more accurate.

[0106] It should be noted that in this embodiment, the level of the constructed text structure tree is determined according to the splitting round.

[0107] It can be understood that ensuring the accuracy of the positioning of the first-level title is the key to the segmentation effect. Since there may be multiple occurrences of the same title in the original text, the large language model needs to identify the actual structure positioning of each title. The prior art generally regards the position of the first occurrence of a certain structure title in the text as the true structure boundary point. For example, for numbered titles such as “Chapter 1” or “2.1”, the possibility of repeated occurrence is relatively low, and the structure boundary is clear; while for titles without numbers such as “Summary” or “Background Introduction”, they may appear multiple times in the text, especially when referring or reviewing the context content. If the first matching position is directly used as the standard, it may lead to incorrect structure division. For example, in a document, the title of the fourth chapter appears between the first and second titles in the previous text, but its true structure position should be after the third title later. If the content is only split according to the position of the first occurrence of the first-level title, an error may occur. As mentioned above, the position of the first occurrence of the title of the fourth chapter is between the first and second titles, which obviously does not conform to the normal order of chapters. Based on this, through step S20, the problem that the first matching position may lead to incorrect structure division can be effectively avoided.

[0108] Please refer to Figure 3 , further, step S20 specifically comprises:

[0109] S201: locating the first position of the first occurrence of the first first-level title in the original text;

[0110] S202: determining the second position of the first occurrence of the second first-level title based on the first occurrence of the first first-level title;

[0111] S203: divide the content between the first position and the second position into a first primary text;

[0112] S204: continue to locate the primary headings in the original text, and sequentially take the second primary heading located in the previous round as the starting primary heading for the current round of division, until the last primary heading is located, and the content between the adjacent two primary headings is divided to obtain the primary texts corresponding to the number of primary headings.

[0113] Specifically, the position where the first primary heading appears in the document is usually correct and has not been disordered. Therefore, after the large language model identifies the primary headings in the original text, the first position where the first primary heading appears is first located, and then the second position where the second primary heading appears is located based on the first position where the first primary heading appears; the third position where the third primary heading appears is located according to the second position where the second primary heading appears, and the positions of the subsequent primary headings are located in the same way as above, thereby realizing the sequential location of the multiple primary headings appearing in the original text. The division order between the multiple primary headings located is to divide the content between the first position where the first primary heading appears and the second position where the second primary heading appears into the first primary text corresponding to the first primary heading. The division of the subsequent primary texts is performed in the same way, that is, the content between the directly adjacent two primary headings after the multiple primary headings are all located is divided, and the division of the last primary heading is to the end of the original text.

[0114] It can be understood that by steps S201-S204, the primary text corresponding to the primary headings is divided, the division of each section of content according to the real structure position of the primary headings is realized, the primary headings appearing in abnormal positions are included in the content of the previous round of division, and the boundaries of the divided texts are clear and the semantics are complete. In addition, this division method can effectively avoid the structure disorder caused by incorrect matching of the headings, and the headings incorrectly appearing in the previous text will be automatically excluded from the text content of the previous round of division, ensuring the clarity and stability of the subsequent structure.

[0115] It can be understood that after each round of text division is completed, each subtext generated in the round is judged for termination division, that is, whether all subtexts after each round of division meet the termination division condition is judged to determine whether to continue the next level of heading extraction and division operation. In this embodiment, by processing the text after each round of division in parallel, independent evaluation and parallel execution of each subdocument in each round are realized, ensuring flexibility and efficiency.

[0116] In an alternative embodiment, the preset termination division condition includes judging whether the text length is greater than a preset text length threshold.

[0117] Specifically, a preset maximum text length threshold can be set for each text segment, for example, in the unit of word count, paragraph count, or Token number. If the length of the segmented primary text or a certain subtext is equal to or greater than the threshold, i.e., the preset termination splitting condition is not met, it is assumed that the subtext has the possibility of further structural division, thereby triggering a new round of primary title extraction and splitting process. If it is less than the threshold, i.e., the preset termination splitting condition is met, the primary text or the certain subtext cannot be further structurally divided, and the splitting of the primary text or the certain subtext is stopped. The preset text length threshold can be flexibly configured according to the actual application scenario, for example, it can be appropriately relaxed when processing long policy documents, technical manuals, etc., and a smaller upper limit can be set in the short text, report, and other scenarios.

[0118] Optionally, for multiple rounds of splitting, different preset text length thresholds can also be set according to the required splitting rounds to adapt to the length of the text corresponding to the multiple rounds of splitting, so as to avoid that the preset text length threshold is set too large at the beginning, and the subtext that can be split in the subsequent process is judged as unsplittable.

[0119] In an optional implementation, the preset termination splitting condition includes determining whether the text includes a substructure or a nested chapter by a large language model according to a preset prompt word.

[0120] Specifically, due to the irregularity of the document structure and the difference in content, only text length judgment may not be able to comprehensively cover all splitting requirements. Therefore, the present embodiment can also introduce a large language model with structure understanding and semantic analysis capability to evaluate the structural complexity of each subtext. The system analyzes whether the text still contains potential substructures or nested chapters through a preset specific prompt word. If the large language model analyzes that the text contains potential substructures or nested chapters, i.e., the preset termination splitting condition is not met, the text can continue to be recursively processed. If the large language model determines that the content structure of the text is basically complete and difficult to be further split in semantics, i.e., the preset termination splitting condition is met, the text no longer continues to be recursively processed.

[0121] In an optional implementation, the preset termination splitting condition includes determining whether the text length is greater than a preset text length threshold, and determining whether the text includes a substructure or a nested chapter by a large language model according to a preset prompt word.

[0122] Specifically, in the embodiment, two judgment conditions can be used together to determine whether the text needs to be segmented. In the preliminary screening stage, the text that may not need to be processed can be quickly filtered through the preset text length threshold to improve the overall processing efficiency. For the boundary case or the segment with ambiguous structure, the semantic auxiliary judgment is performed by the large language model to enhance the robustness and accuracy. The termination judgment mechanism considers the calculation efficiency and adaptability while maintaining the structure accuracy, and provides high-quality node boundary division basis for the construction of the subsequent hierarchical text structure tree.

[0123] It should be noted that the text that does not need to be processed can be the text that meets the judgment of the preset text length threshold, and the text that does not need to be determined by the large language model; the text with boundary condition or ambiguous structure can be the text that does not meet the judgment of the preset text length threshold.

[0124] It can be understood that in the embodiment, the user can select any one of the above-mentioned embodiments to determine the preset termination segmentation condition of the segmented text. Any one of the above-mentioned termination segmentation conditions can determine whether to continue to perform the structure extraction and splitting operation of the next level.

[0125] Optionally, the parameter settings of the termination segmentation condition can also be dynamically adjusted according to specific task scenarios. For example, different tasks such as question and answer, extraction, writing, etc. can correspondingly set different text length thresholds or large model judgment confidence thresholds to realize customized control strategy and improve the user's use feeling.

[0126] Further, step S30 specifically comprises:

[0127] S301: performing parallel segmentation processing on all first-level texts that do not meet the preset termination segmentation condition to obtain at least two second-level texts corresponding to each first-level text;

[0128] Specifically, the parallel segmentation processing can be to separately segment each first-level text in the same way as the segmentation text applied in steps S201-S204, so as to obtain at least two second-level texts corresponding to the next level of each first-level text that can be segmented.

[0129] S302: determining whether all second-level texts under all first-level texts meet the preset termination segmentation condition;

[0130] Specifically, whether it is the first-level text segmented from the original text, or the second-level text segmented from the first-level text, and the subtext segmented from the second-level text, and the next level of text segmented from the subtext, after the segmentation is completed, each text needs to be judged for the preset termination segmentation condition. That is, each round of text segmentation needs to judge the segmented text for the preset termination segmentation condition to determine whether each segmented text can be segmented in the subsequent round.

[0131] S303: Continue to perform subsequent rounds of parallel progressive segmentation on all second-level texts that do not meet the preset termination segmentation condition until all subtexts after the subsequent round of segmentation meet the preset termination condition and stop segmentation.

[0132] Specifically, in this embodiment, the overall stop segmentation is performed when all subtexts in the round meet the preset termination condition, that is, all subtexts cannot be segmented, and the overall segmentation process of the original text is completed. If after segmentation, there is a text that does not meet the preset termination condition, parallel progressive segmentation will continue until all texts meet the preset termination condition after a round of segmentation.

[0133] It can be understood that by performing parallel progressive segmentation on each text after each round of segmentation, that is, after each round of text structure segmentation is completed, each subtext newly generated after segmentation is judged for termination segmentation one by one to determine whether to continue to perform the next level of structure extraction and segmentation operation, the structure of the original text is identified and the document tree is recursively constructed layer by layer, not only the original logical level of the original text is preserved, but also the problem of semantic fragmentation and structural disorder is reduced, and the rigor and integrity of knowledge organization are effectively improved. The termination segmentation judgment is performed in parallel in each round of segmentation, and each subtext after each round of segmentation is independently evaluated and judged to ensure the flexibility and efficiency of the segmentation.

[0134] Further, after step S303, the method further comprises:

[0135] S304: All first-level texts, second-level texts and subtexts that meet the preset termination segmentation condition after multiple rounds of segmentation are taken as leaf nodes;

[0136] Specifically, the leaf node is composed of text that cannot be segmented after the overall segmentation process of the original text is completed, for example, an original text has 2 large chapters, and each chapter has 5 small sections, and then 10 subtexts are finally segmented. The 10 subtexts are leaf nodes. The leaf node can be used for retrieval.

[0137] S305: Information enhancement is performed on the leaf node to obtain an information-enhanced leaf node.

[0138] Specifically, the information enhancement is to complete the missing information of the leaf node, so that the information of the leaf node is complete. Since each leaf node is the text corresponding to the last level on each split path, after the splitting is completed, the information enhancement processing of each leaf node content can also be carried out in detail to improve the completeness and semantic accuracy of the information, so as to better support the subsequent knowledge base construction and downstream task calling, such as question and answer, abstract generation, information matching and the like.

[0139] Please refer to Figure 4 Further, step S305 specifically includes:

[0140] S3051: identifying whether there is missing target information in the leaf node according to the preset prompt instruction through the large language model; the target information includes reference information;

[0141] Specifically, the target information missing is due to the structural splitting, which causes the context information in some content fragments to be incomplete, and the manifestations include but are not limited to:

[0142] "Today" appears, but the specific date cannot be determined;

[0143] "He" or "she" appears, but there is no clear information about the corresponding reference object;

[0144] "This matter" appears, but the specific content of the event cannot be determined;

[0145] The table field appears, but its semantic definition is located in other parts of the document, which is not included in the current slice, so that the field is difficult to understand.

[0146] The target information in the embodiment can include reference information, which can include reference information in four dimensions of time, person, event and table. The large language model can identify and judge the missing information one by one for the information in four dimensions of time, person, event and table. If the content of the text is complete, no modification is made to the leaf node; if the target information is indeed missing, the missing type is explicitly marked, so that the missing type information can be directly searched for later.

[0147] More specifically, the large language model can identify whether there is a situation of target information missing in the input text content based on the preset information identification prompt word.

[0148] The following gives an example of the large language model identifying whether there is a situation of target information missing in the input text content based on the preset information identification prompt word:

[0149] For example, the two adjacent leaf nodes are:

[0150] Leaf node 1:

[0151] Now 62 years old, he has a head of silver hair that speaks of the vicissitudes of time. He no longer travels in a luxury car with bodyguards, but takes the bus, subway, and even rides a bicycle. He is so simple that he doesn't even want to dye his white hair, but just wears a hat to cover it up when he goes out.

[0152] Leaf node 2:

[0153] In the eyes of outsiders, he has nothing, but he seems to have nothing. One after another, his friends around him left him, and he always appeared alone in front of the camera. He has no partner and no close friends.

[0154] The most painful thing is that his loneliness is rarely understood. In the complex environment of the entertainment industry, he chose to offend people for art and be alone for perfection. He said, "Acting is more important than people!" This sentence is both his artistic belief and the source of his loneliness.

[0155] The preset information recognition prompt word in the large language model is:

[0156] Please judge whether there is a situation of target information missing in the following text content. Target information missing mainly refers to the incomplete context information in some content fragments due to structural segmentation, which includes but is not limited to:

[0157] "Today" appears, but the specific date cannot be determined;

[0158] "he" or "she" appears, but the corresponding referent information is missing;

[0159] "this matter" appears, but the specific content of the event cannot be determined;

[0160] The table field appears, but its semantic definition is located in other parts of the document, and is not included in the current slice, making it difficult to understand.

[0161] If so, output yes and the missing question. If not, output no.

[0162] Enter leaf node 2 into the large language model. The large language model can identify that the information of "he" in leaf node 2 is missing according to the above preset information recognition prompt word.

[0163] S3052: If there is missing target information in the leaf node, trace back to the missing target information in the leaf node according to the segmentation path of the original text, and determine the missing target information in the leaf node from the previous node of the leaf node.

[0164] S3053: supplement the target information determined from a layer node to a leaf node missing the target information, to obtain a leaf node with enhanced information.

[0165] Specifically, the segmentation in this embodiment is a parallel progressive segmentation, and a plurality of multi-level segmentation paths are formed with the main title of the original text as the origin. Each segmentation path includes a plurality of segmented texts progressively segmented according to segmentation rounds, and one text corresponds to one node at one level on the segmentation path. Therefore, the segmentation path of the leaf node missing the target information can be determined first, and the node at the upper level to which the leaf node belongs is determined based on the segmentation path, so that the content of the target information missing from the leaf node is determined from the node at the upper level. If the content of the target information missing from the leaf node is not identified in the node at the upper level to which the leaf node belongs, identification needs to be continued along the node at the further upper level of the node at the upper level.

[0166] For example, assuming that the current leaf node missing the target information is numbered 4.3.1.1, the large language model will first backtrack to the node 4.3.1 at the upper level thereof, determine whether the node 4.3.1 contains the missing information, and if not, continue to backtrack to 4.3 and then to 4, until a node containing the required information is found. After the node containing the required information is found, the relevant content is extracted and supplemented to the content of the current leaf node, so as to achieve semantic integrity enhancement.

[0167] Thus, steps S3051-S3053 realize level-by-level tracing of the segmentation path to which the leaf node belongs, so as to realize the determination and completion of the missing target information in a level-by-level tracing manner, thereby completing the information enhancement of the leaf node, making the ambiguous expression clear and the missing content complete, greatly improving the content integrity and clarity, and realizing the semantic integrity enhancement of the content included in each leaf node. In addition, compared with the segmentation mode based on fixed rules in the prior art, through the parallel progressive segmentation in this embodiment, the information enhancement processing of the leaf node after segmentation can complete the missing information, so as to avoid the problem that the segmentation based on the existing fixed rules may split the text segments with semantic association, resulting in the loss of clear reference of pronouns mentioned in the subsequent text, and the loss of context semantics and the missing of local key information.

[0168] Please refer to Figure 5 Further, step S40 specifically includes:

[0169] S401: determining the level of the text structure tree to be constructed according to the segmentation round;

[0170] S402: obtaining the main title of the original text, and taking the main title as the root node;

[0171] Specifically, the level of the text structure tree to be constructed can be determined by the total number of splitting rounds of the original text. The first round of splitting will form two levels, specifically, the main title of the original text is the root node of the first level, and each first-level text after splitting corresponds to a structure node of the second level; the second round of splitting will form three levels, specifically, the main title of the original text is the root node of the first level, each first-level text after the first round of splitting corresponds to a structure node of the second level, and each second-level text after the second round of splitting corresponds to a structure node of the third level. The subsequent rounds of splitting and levels are similar. The total level of the final document structure tree is the total number of splitting rounds of the original text plus one.

[0172] S403: generating corresponding structure nodes according to the first-level texts, second-level texts and subtexts formed after multiple rounds of splitting, and assigning a unique identifier to each structure node;

[0173] Specifically, all first-level texts, second-level texts and subtexts formed after splitting are structure nodes of the corresponding level. The structure node includes nodes corresponding to texts that can be split and leaf nodes corresponding to texts that cannot be split. Each structure node not only includes corresponding level information, i.e., level level, but also includes title content, parent node, child node list and original text segment. The lower the level level, the closer to the root node, for example, the first-level title is Level 1, the second-level title is Level 2, and so on.

[0174] More specifically, each structure node is assigned a unique identifier, which facilitates locating the corresponding structure node in the document structure tree and improves search efficiency.

[0175] It can be understood that the leaf node in the structure node is a leaf node after information enhancement processing, and the splitting path information of the leaf node in the original text can also be added to its content, effectively improving the accuracy of the search.

[0176] S404: constructing the mapping relationship between the parent node and the child node list to which each structure node belongs; the structure node includes title content, level level, parent node, child node list and original text segment;

[0177] Specifically, the mapping relationship is specifically a parent-child mapping relationship between a structural node and a parent node and a child node. The parent node of each structural node is a segmentation source node of a previous level; and the child node is a next level structural node further identified in the node text segment. By establishing the parent-child mapping, complete restoration of the tree structure can be achieved, and accurate positioning of each text in the overall structure of the document is ensured. For example, a subtext of a second section of a first chapter in the text is a structural node, and the adjacent nodes are a first section of the first chapter and a third section of the first chapter. The first chapter directly connected thereto is the parent node, and the first and second paragraphs under the subtext of the second section of the first chapter are the child nodes. The parent node of each structural node is only one, and the child node can be none or at least one.

[0178] S405: constructing a multi-level text structure tree corresponding to the original text based on the root node, the structural nodes, and the mapping relationship.

[0179] Specifically, based on the root node, all structural nodes, and corresponding level information, and the mapping relationship between all structural nodes, a multi-level document structure tree corresponding to the original text can be constructed. The document structure tree not only includes the original content of the original text, but also makes the text structure tree present as a multi-level knowledge organization tree with clear parent-child levels, complete content mounting, and rich semantic extraction in form. Moreover, the text structure tree structure not only significantly improves the visibility and controllability of the original text structure, but also provides a solid data foundation and structural guarantee for downstream tasks such as knowledge retrieval, content generation, and intelligent question answering. In addition, the final generated document structure tree clearly presents the parent-child mapping relationship of each chapter and its substructure, and the content information of any specified chapter can be obtained, so that the original text is no longer a flat text collection, but a knowledge graph input source with clear levels, complete structure, and rich semantics, greatly improving the retrieval efficiency and generation quality of the intelligent agent system.

[0180] Understandably, the embodiment realizes building a document structure tree based on the actual hierarchical structure of the original text, and adds hierarchical information into the structural nodes to enhance the global information of the slices; and locally, for the leaf nodes, the information of time, person, event, table, etc. of each leaf node is enhanced to further enhance the information of the leaf nodes. In the retrieval, the global information of the text slices is enhanced, and the local information of the text slices is also enhanced, so as to improve the retrieval accuracy and comprehensiveness of different granularity query requests.

[0181] Further, after step S10, the method further comprises:

[0182] S11: if all first-level texts satisfy the preset termination segmentation condition, stopping segmentation;

[0183] S12: constructing a text structure tree based on the main title of the original text and all the first-level texts after segmentation.

[0184] It can be understood that if all the first-level texts formed after the first segmentation of the original text meet the preset termination segmentation condition, the segmentation process is stopped, and a two-layer text structure tree is constructed directly with the main title of the original text and all the first-level texts after segmentation. The first layer is the root node corresponding to the main title, and the second layer is the structure nodes corresponding to all the first-level texts under the main title. Each structure node includes a first-level title and corresponding text content. The two-layer text structure tree is more suitable for short documents, making the structure of the document simpler and clearer.

[0185] It should be noted that in the embodiment, segmentation is performed according to the levels present in the original text, that is, the original text has several levels, and segmentation can be performed on several levels. When there is no level in the original text, segmentation can not be performed. The specific segmentation condition is determined according to the actual structure of the original text, and the embodiment does not limit the specific segmentation level of each text.

[0186] Referring to Figure 6 The second embodiment of the present application provides a computer device 1, which comprises a storage 10, a processor 11 and a computer program 12 stored in the storage 10. The processor 11 executes the computer program 12 to implement the above-mentioned method for constructing a text knowledge base based on a large language model.

[0187] It can be understood that the computer device 2 of the embodiment has the same beneficial effects as the above-mentioned method for constructing a text knowledge base based on a large language model, and will not be described here.

[0188] Referring to Figure 7 The third embodiment of the present application provides a computer program product 2, which stores a computer program 20. The computer program 20 is executed by a processor to implement the above-mentioned method for constructing a text knowledge base based on a large language model.

[0189] It can be understood that the computer program product 2 of the embodiment has the same beneficial effects as the above-mentioned method for constructing a text knowledge base based on a large language model, and will not be described here.

[0190] In the embodiments provided by the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information.

[0191] It should be understood that every feature, structure, or characteristic described in relation to one embodiment is applicable to at least one other embodiment, unless the context clearly dictates otherwise. Thus, appearances of the phrase "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily referring to the same embodiment. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner on one or more embodiments. It will also be appreciated by those of skill in the art that references to a structure or feature that is disposed "adjacent" to another feature can encompass instances in which the structures grow integrated, and that the relative configuration of the structures can be reversed.

[0192] In various embodiments of the present application, it should be understood that the magnitude of the serial number of the above-mentioned processes does not mean the inevitable sequence of execution, and the execution sequence of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0193] The flow diagrams and block diagrams in the drawings are illustrations of architectures, functional processes, and operational processes, according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the drawings. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0194] The above discloses a method for constructing a text knowledge base based on a large language model, a computer device and a program product, and the principle and implementation manner of the present application are described by applying specific examples. The above embodiment description is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range can be changed, and the above description should not be understood as limiting the present application. Any modification, equivalent replacement and improvement within the principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for constructing a text knowledge base based on a large language model, characterized in that: The method comprises the following steps: obtaining an original text, and extracting a first-level title of the original text based on explicit marks and implicit semantic features by using a large language model; segmenting the original text based on the first-level title to obtain first-level texts corresponding to the number of first-level titles; wherein the segmentation of the original text based on the first-level title to obtain the first-level texts corresponding to the number of first-level titles comprises: locating a first position where a first first-level title first appears in the original text; determining a second position where a second first-level title first appears based on the first first-level title; dividing the content between the first position and the second position into a first first-level text based on the first position and the second position as boundaries; continuing to locate the first-level titles in the original text, and sequentially taking the second first-level title located in the previous step as a starting first-level title for the current segmentation, until the last first-level title is located, and sequentially segmenting the content between adjacent first-level titles to obtain the first-level texts corresponding to the number of first-level titles; performing parallel progressive segmentation on the first-level texts that do not meet the preset termination segmentation condition until the segmented subtexts meet the preset termination segmentation condition, and stopping the segmentation; the preset termination segmentation condition comprises determining whether the text includes a substructure or a nested chapter by performing semantic judgment on the text according to a preset prompt word by using the large language model; the parallel progressive segmentation is parallel processing of all texts after each round of segmentation, and all texts meeting the termination segmentation condition are taken as independent inputs to continue the progressive segmentation; the progressive segmentation is gradually segmenting the texts according to the segmentation rounds; constructing a multi-level text structure tree based on the main title of the original text and all the segmented first-level texts and subtexts; the text structure tree includes all original contents of the original text; the text structure tree takes the main title of the original text as a root node of the first level, takes each first-level text after the first round of segmentation as a structure node of the second level, and takes each subtext after the subsequent rounds of segmentation as a structure node of the next round of segmentation; the structure node includes title content, level, parent node, child node list, and original text segment; there is a mapping relationship between the parent node and the child node list of each structure node. 2.The method of claim 1, wherein: After obtaining the original text and extracting the first-level title of the original text by using the large language model, the method further comprises: judging whether each extracted first-level title appears in the original text one by one by using the large language model; if the content of the extracted first-level title does not exist in the original text, the extraction fails, and the extraction operation of the first-level title is performed again. 3.The method of claim 1, wherein: The preset termination segmentation condition comprises judging whether the length of the text is greater than a preset text length threshold. 4.The method of claim 3, wherein the constructing of the text knowledge base based on the large language model comprises: obtaining a plurality of text data from a plurality of data sources; and training the large language model based on the plurality of text data. After segmenting the original text based on the first-level title to obtain the first-level texts corresponding to the number of first-level titles, the method further comprises: if all the first-level texts meet the preset termination segmentation condition, stopping the segmentation; constructing a text structure tree based on the main title of the original text and all the segmented first-level texts.

5. The method for constructing a text knowledge base based on a large language model as described in claim 4, characterized in that: The first-level texts not satisfying the preset termination segmentation condition are subjected to parallel progressive segmentation until the segmented subtexts satisfy the preset termination segmentation condition, and the segmentation is stopped, specifically including: all the first-level texts not satisfying the preset termination segmentation condition are subjected to parallel segmentation to obtain at least two second-level texts corresponding to each first-level text; whether all the second-level texts under all the first-level texts satisfy the preset termination segmentation condition is judged; all the second-level texts not satisfying the preset termination segmentation condition are subjected to subsequent rounds of parallel progressive segmentation until all the subtexts segmented in the subsequent rounds satisfy the preset termination condition and the segmentation is stopped.

6. The method for constructing a text knowledge base based on a large language model as described in claim 5, characterized in that: After the first-level texts, second-level texts and subtexts satisfying the preset termination segmentation condition are segmented in multiple rounds and the segmentation is stopped, the method further includes: the first-level texts, second-level texts and subtexts satisfying the preset termination segmentation condition segmented in multiple rounds are taken as leaf nodes; the leaf nodes are subjected to information enhancement to obtain information-enhanced leaf nodes.

7. The method for constructing a text knowledge base based on a large language model as described in claim 6, characterized in that: The leaf nodes are subjected to information enhancement to obtain information-enhanced leaf nodes, specifically including: whether there is missing target information in the leaf nodes is identified according to a preset prompt instruction through a large language model; the target information includes referential information; if there is missing target information in the leaf nodes, the leaf nodes with missing target information are subjected to level-by-level tracing according to the segmentation path of the original text to determine the missing target information in the leaf nodes from the previous layer of the leaf nodes with missing target information; the target information determined from the layer is supplemented to the leaf nodes with missing target information to obtain information-enhanced leaf nodes.

8. The method for constructing a text knowledge base based on a large language model as described in claim 7, characterized in that: A multi-level text structure tree is constructed based on the main title of the original text and all the first-level texts and subtexts segmented, specifically including: the level of the text structure tree to be constructed is determined according to the segmentation round; the main title of the original text is obtained and taken as a root node; corresponding structure nodes are generated according to the first-level texts, second-level texts and subtexts segmented in multiple rounds, and each structure node is given a unique identifier; a mapping relationship between the parent node and the child node list to which each structure node belongs is constructed; the structure node includes title content, level, parent node, child node list and original text segment; a multi-level text structure tree corresponding to the original text is constructed based on the root node, the structure node and the mapping relationship.

9. A computer device comprising a storage, a processor and a computer program stored on the storage, characterized in that: The processor executes the computer program to implement the method for constructing a text knowledge base based on a large language model according to any one of claims 1-8.

10. A computer program product having stored thereon a computer program, characterized in that: The computer program is executed by the processor to implement the method for constructing a text knowledge base based on a large language model according to any one of claims 1-8.

Images