A dynamic directory tree-based supervised fine-tuning corpus automatic generation method and medium

Abstract

This invention relates to an automated method and medium for generating supervised fine-tuning corpora based on dynamic directory trees. The method includes: acquiring the parsed original document, dividing it using a multi-granularity semantic segmentation method to obtain multiple location blocks, and assigning a globally unique identifier to each location block; based on the location blocks and their identifiers, using a large language model to parse the content of each location block, generating a multi-level directory structure mapping its inherent knowledge logic, and constructing a dynamic directory tree; acquiring a preset questioning strategy, selecting target nodes from the dynamic directory tree, and progressively generating questions and answers based on the target nodes to form question-answer pairs, which serve as supervised fine-tuning corpora. Simultaneously, during the answer generation process, a dynamic extended context strategy and a structured thought chain are used to guide answer generation, completing the automatic corpus generation process. Compared with existing technologies, this invention has the advantages of efficiently and automatically producing high-quality supervised fine-tuning corpora supporting multi-hop reasoning.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method and medium for the automated generation of supervised fine-tuning corpora based on dynamic directory trees. Background Technology

[0002] Automating the generation of supervised fine-tuning (SFT) corpora from specialized documents using large-scale language models (LLM) is an important way to improve the specialization capabilities of models in specific domains. However, when the documents are characterized by their length, complex knowledge structure, and rich internal logical hierarchy (such as academic papers, technical standards, encyclopedias, etc.), existing automated generation methods have significant limitations in technical implementation, resulting in question-answer pairs whose quality is insufficient to meet the training requirements of higher-order cognitive tasks. The shortcomings of existing technologies are specifically reflected in the following aspects: 1. The document segmentation method is mismatched with the semantic structure, resulting in damage to knowledge integrity. Current methods often employ mechanical segmentation strategies based on fixed lengths or simple paragraph markers. This strategy ignores the inherent knowledge organization logic of the document itself, easily severing originally coherent semantic units at segmentation boundaries. As a result, the local contextual information upon which the generated question depends is incomplete, forcing the model to generate answers based on incomplete context. This not only makes it difficult to guarantee the completeness of the answer but also easily induces factual errors or fabricated content.

[0003] 2. The problem generation strategy is too flat and lacks the ability to respond to the macro-level of documents. Because existing methods cannot perceive the macro-level organizational structure of documents (such as chapters-sub-chapter-key points), the questions generated exhibit a "flat" characteristic. There is a lack of logical connections between questions, failing to form a hierarchical question system that progresses from simple to complex, from overview to detail, and from concept to application. This method struggles to generate "multi-hop questions" that require integrating information from multiple chapters and possess deep reasoning value, significantly limiting the complex reasoning and knowledge connection capabilities of the trained model.

[0004] 3. The definition of the search scope for answering questions faces a contradiction between efficiency and effectiveness. Expanding the scope of context accessible to the model when generating answers (e.g., providing an entire chapter or text) to ensure completeness will lead to a surge in token usage, significantly increasing processing time and costs. Simultaneously, an excessively large input window will introduce a large amount of irrelevant noise, interfering with the model's accurate location and extraction of key knowledge points, thus reducing the accuracy and relevance of the answers. Conversely, strictly limiting answer generation to small pre-segmented text fragments will fail to handle comprehensive questions requiring the integration of information from multiple disparate paragraphs, significantly increasing the difficulty of generating high-quality comprehensive question answers.

[0005] In summary, existing technologies struggle to automatically produce SFT training corpora that combine deep reasoning value, high accuracy, and good hierarchical structure. Developing a new technological solution capable of intelligently understanding the inherent knowledge structure of documents and thereby achieving efficient and accurate knowledge localization and multi-hop question-answer generation has become an urgent need in this field. Summary of the Invention

[0006] The purpose of this invention is to provide an automated method and medium for generating supervised fine-tuning corpora based on dynamic directory trees that supports multi-hop reasoning and is highly content-aligned.

[0007] The objective of this invention can be achieved through the following technical solutions: An automated method for generating supervised fine-tuning corpora based on dynamic directory trees includes the following steps: The parsed original document is obtained and divided using a multi-granularity semantic segmentation method to obtain multiple location blocks, and each location block is assigned a globally unique identifier. Based on the location blocks and their identifiers, a large language model is used to parse the content of each location block, generate a multi-level directory structure that maps its inherent knowledge logic, and construct a dynamic directory tree. The system acquires a preset questioning strategy, selects target nodes from the dynamic directory tree, and generates questions and answers step by step based on the target nodes to form question-answer pairs, which serve as supervised fine-tuning corpus. At the same time, during the answer generation process, the system uses a dynamic extended context strategy and a structured thought chain to guide the answer generation, thus completing the automatic corpus generation process.

[0008] Furthermore, the step of obtaining multiple positioning blocks includes: Coarse-grained segmentation: For the original document, it is initially divided according to a pre-set fixed length threshold to obtain multiple directory blocks; Fine-grained segmentation: For each directory block, a semantic-based recursive segmentation algorithm is used for fine-grained division to obtain multiple location blocks corresponding to each directory block.

[0009] Furthermore, the step of performing fine division includes: The semantic-based recursive segmentation algorithm first identifies the natural language boundaries of each directory block as the segmentation points, wherein the natural language boundaries include paragraph endings, newline characters, and periods. Based on the aforementioned split points, each directory block is recursively divided into multiple location blocks.

[0010] Furthermore, the step of constructing the dynamic directory tree includes: All location blocks and identifiers under the same directory block are input into the large language model. The large language model reads and understands the content of all location blocks according to specific instructions, and generates a multi-level directory structure that maps its inherent knowledge logic. Each directory entry in the multi-level directory structure is marked with an identifier of one or more location blocks that support the content of the directory entry, and the multi-level directory structure covers the core theme and division logic of the original document. The multi-level directory structure is automatically parsed and structured to transform it into a tree-like data structure, forming a dynamic directory tree. Each node in the dynamic directory tree includes the following key fields: level: indicates the level depth of the node in the dynamic directory tree, where the node represents a directory entry; title: Indicates the name of the directory entry; chunk_ids: An array used to store the identifiers of all location blocks associated with the corresponding directory entry; children: This is an array used to store all the child nodes of the corresponding node.

[0011] Furthermore, the questioning strategy includes the following: Questioning Strategy 1: Select the leaf node in the dynamic directory tree as the target node; Questioning Strategy Two: Select the parent leaf node in the dynamic directory tree as the target node.

[0012] Furthermore, the question-answer pair generation step includes: Question generation: Use the key field "title" in the target node as the question topic, where "title" represents the name of the directory entry; Input the question topic and the original text information corresponding to the target node into the large language model to generate a question; Answer generation: Based on the key fields title and chunk_ids in the target node, all location blocks associated with the generated question are extracted. The associated location blocks are used as the initial context, and chunk_ids is used as an array to store the identifiers of all location blocks associated with the corresponding directory entry. Based on all the associated location blocks and their identifiers, the dynamic context expansion strategy takes each associated location block as the core and automatically includes the location blocks adjacent to it before and after in the context scope, which together with the initial context constitute an expanded context window, wherein the expanded context window is located in the same directory block; The question and the extended context window are input into the large language model. By injecting prompt words into the large language model, one or more answers corresponding to the question are generated, forming one or more question-answer pairs, with the key field title in the target node as the core. The prompt words limit the use of a structured thinking chain reasoning mode to first generate the reasoning process, and then combine the reasoning process to generate the answer.

[0013] Furthermore, the steps in generating the reasoning process using the structured thought chain reasoning model include: The problem is analyzed and categorized by professional field to determine its professional field and core examination points; Based on the relevant professional field and core examination points, precise positioning is performed in the extended context window to identify all relevant positioning as relevant evidence fragments; Analyze and sort out the inherent logical relationships between the relevant evidence fragments, including one or more of the following: causality, parallelism, progression, and contrast; By integrating the relevant evidence fragments based on the inherent logical relationships, a reasoning path supporting the answer can be constructed.

[0014] Furthermore, it also includes quality inspection steps: During the question generation process, based on the actual professional field and the pre-set question rejection rules of the type system, the large language model is instructed to exclude questions that do not conform to the question rejection rules when generating questions; During the answer generation process, it is first determined whether the question can be answered based on the extended context window. If it can, the answer is generated directly; otherwise, the question is discarded and no answer needs to be generated. A large language model is used to perform multi-dimensional quantitative evaluation of the generated answers to obtain corresponding scores. These scores are then compared with a preset threshold. If the score is greater than or equal to the threshold, the generated answer is retained; otherwise, the answer and its corresponding question are discarded. The dimensions are as follows: Completeness and Clarity: Assess whether the subject of the question is clear, and whether the answer comprehensively covers the core information and is logically clear; Technical depth: Assess whether the questions and answers involve in-depth content, including technical parameters, applicable conditions, and principle analysis; Knowledge-based and analytical: Assess whether the questions and answers require or demonstrate deep cognitive abilities, including induction, summarization, comparison, and analysis; Accuracy and Citation Transparency: Assess whether the answer is absolutely faithful to the provided extended context window, whether all arguments are supported by clear and traceable original texts, and whether the expression is self-consistent; Illusion Index: This assesses whether any factual statement in the answer cannot be supported by direct or reasonable inference within the provided extended context window. If an illusion is found, the score will be 0. Difficulty and Depth of Reasoning: Assess the complexity of the reasoning required for the problem, whether it is a simple restatement or requires multiple steps of derivation and synthesis.

[0015] Furthermore, it also includes a multi-round question-and-answer generation process: Metadata analysis is performed on the question-answer pairs generated in a single round. Preliminary grouping is performed based on the key field chunk_id, question strategy, and keywords in the dynamic directory tree. The chunk_ids is used as an array to store the identifiers of all location blocks associated with the corresponding directory entry. Based on a variety of preset synthesis modes, select one or more synthesis modes, including deep questioning mode and breadth expansion mode. The deep questioning mode forms a progressive question-and-answer chain by going deeper into the parent directory level to the subdirectory level. The breadth expansion mode generates question-and-answer pairs for multiple parallel subdirectory levels under the same parent directory level and combines them into a comprehensive question and answer that covers multiple aspects of the topic. Select single-turn or multi-turn question-answer pairs and their corresponding extended context windows and input them into the large language model. Instruct it to reconstruct multi-turn question-answer pairs using independent questions and answers according to the selected synthesis mode. Each round of question-answer pairs needs to be automatically validated by the set validation criteria, which include: Theme consistency: Determine whether each round of question and answer revolves around the same core theme. If so, keep it; otherwise, discard it. Round validity: Check whether the number of rounds of the question-and-answer pair has reached the preset minimum requirement. If so, continue to the next round. If not, stop or continue according to actual needs, and filter out question-and-answer pairs that contain meaningless or short rounds. Logical coherence: The logical coherence of the question-and-answer pairs is evaluated to eliminate those that lack logical connection between rounds or are out of order.

[0016] The present invention also provides a storage medium storing a program thereon, which, when executed, implements the above-described method for automatically generating supervised fine-tuning corpus based on a dynamic directory tree.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention, from semantic segmentation to the construction of the directory tree, and then through the directory tree to guide the generation of question-answer pairs, can efficiently and automatically produce high-quality supervised fine-tuning corpus samples that support multi-hop reasoning and are highly consistent with the inherent knowledge structure of the document.

[0018] (2) The multi-granularity semantic segmentation method in this invention adopts a two-stage segmentation strategy from coarse to fine. First, the document is segmented into larger directory blocks. These directory blocks ensure that when the directory tree is built later, the large language model can focus on processing a content block with a relatively concentrated theme within a controllable context window, thereby avoiding computational redundancy and understanding interference caused by processing the entire text at once. Then, each directory block is segmented into multiple location blocks through a semantic-based recursive segmentation algorithm. These location blocks can provide the original text location for the subsequently generated directory entries as accurately as possible, ensuring that each location block carries one or more semantically complete semantic groups, thereby avoiding the risk of being forcibly truncated at the point of the sentence or logic at the physical level. Therefore, this multi-granularity semantic segmentation method solves the dilemma of semantic fragmentation and retrieval scope definition caused by "blind segmentation" in the prior art.

[0019] (3) This invention uses a big language model to intelligently analyze the segmented text content and autonomously generate a multi-level directory structure that maps its inherent knowledge logic. This transforms the unstructured original text into a structured directory tree, which reveals the macro knowledge system of the document and establishes a precise index relationship between knowledge nodes and text fragments. This makes the generated questions no longer isolated and flat, but naturally form a question system with hierarchical logic (such as from macro overview to micro details, from concept to application) consistent with the document structure. The question-answer pairs generated in this way not only have higher accuracy, but more importantly, they themselves contain rich logical relationships, which greatly enhances the value of the corpus in training the deep reasoning and knowledge connection capabilities of the model, and fundamentally overcomes the defects of the flat generation of existing technologies.

[0020] (4) This invention generates answers by dynamically constructing extended context windows and a structured thought chain mechanism. The dynamically constructed extended context window not only ensures that there is sufficient background information when generating answers to avoid misinterpretation, but also effectively reconstructs the local semantic coherence temporarily severed by the segmentation operation, laying a solid foundation for the model to accurately understand and deeply reason. The structured thought chain mechanism ensures that the reasoning path of the answer is clear and conforms to logical relationships through the thought chain reasoning mode, thereby solving the problem of missing key information that may be caused by text segmentation in the existing answer generation steps, and ensuring that the generated answers have high accuracy, completeness and logical rigor.

[0021] (5) The directory tree of this invention clearly reveals the hierarchy and relationships between knowledge points. The system can flexibly select different nodes as question anchors according to preset strategies, thereby generating diverse question-and-answer instances with controllable complexity. For example, it can easily generate comparative and extensible questions for parallel child nodes under the same parent node, or generate progressive, multi-step reasoning (multi-hop) deep questions from top to bottom along the path of the directory tree. This flexible questioning mechanism based on knowledge structure is the key to the automatic production of high-value supervised fine-tuning corpora that support complex reasoning training, breaking through the limitation of traditional methods that can only generate shallow question-and-answer.

[0022] (6) This invention cleverly solves the dilemma of defining the search scope by combining the strategies of "multi-granular semantic segmentation" and "extended context answer generation". Fine-grained positioning blocks provide "coordinates" for accurate knowledge positioning, while the dynamic expansion strategy reconstructs the semantic coherence required to generate answers. This avoids the computational burden and noise interference caused by excessive context, and also prevents inaccurate answers or illusions caused by information fragmentation, thus achieving the best balance between efficiency and effectiveness.

[0023] (7) In the multi-turn dialogue synthesis stage of this invention, through policy-driven intelligent combination, discrete high-quality single-turn question-and-answer pairs can be synthesized into multi-turn dialogue data with consistent themes and logical coherence based on different interaction modes (such as deep probing and breadth expansion). This provides valuable data resources for training next-generation models with complex dialogue, deep question answering, and continuous topic management capabilities, demonstrating the good scalability and forward-looking nature of the method of this invention. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is an example of a parsed dynamic directory tree in an embodiment of the present invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0026] This embodiment provides an automated method for generating supervised fine-tuning corpora based on dynamic directory trees. The core of this method lies in creating a new paradigm "from semantic segmentation to directory tree construction, and then to tree-guided question-and-answer generation" by first constructing a macro-level knowledge framework using a large language model and then delving into the details. This enables the efficient and automated production of high-quality SFT corpora that support multi-hop reasoning and are highly consistent with the document's inherent knowledge structure. Specifically, such as... Figure 1As shown, the method includes the following steps: S1. Multi-granularity semantic segmentation steps This step aims to resolve the dilemma of semantic fragmentation and retrieval scope definition caused by "blind segmentation" in the background technology. Its core lies in adopting a two-level segmentation strategy from coarse to fine to construct text units that have both semantic integrity and operational feasibility for subsequent processing.

[0027] coarse-grained slitting The original, parsed long document is initially divided according to a pre-defined, relatively large fixed length threshold. The resulting text units are called "directory chunks." This threshold is typically set to the tens of thousands of words, aiming to divide the long document into multiple large modules with moderate content spans that are easy to process independently. Each directory chunk covers a continuous portion of the document. Its size design ensures that when subsequently building the directory tree, the large language model can focus on processing a relatively topic-focused content block within a controllable context window, thereby avoiding computational redundancy and comprehension interference caused by processing the entire text at once.

[0028] Fine-grained slitting For each directory block obtained in the first level, a semantic-based recursive segmentation algorithm is used for fine-grained partitioning. This algorithm prioritizes identifying and utilizing natural language boundaries such as paragraph ends, line breaks, and periods as segmentation points, recursively dividing a single directory block into multiple significantly reduced text units, called "location chunks." The word count of each location chunk is typically controlled to be in the hundreds. The core purpose of this level of segmentation is to provide the most accurate original text location for the subsequently generated directory entries, ensuring that each location chunk carries one or more relatively complete semantic groups, such as an explanation of a complete technical point, a description of an object's characteristics, or a narration of an event process, thereby physically avoiding the risk of forced truncation at sentence or logical midpoints.

[0029] Identifier assignment Each generated location block is assigned a globally unique identifier. This identifier serves as the unique digital identity of the location block throughout the entire process and will become the core basis for subsequently associating directory tree nodes with specific text content and achieving accurate knowledge location.

[0030] S2. Steps for constructing a dynamic directory tree The dynamic directory tree construction step aims to address the core shortcomings of the background technology, namely the flat nature and lack of hierarchy in problem generation. The core of this step lies in the intelligent analysis of the segmented text content by the instruction-based large language model, which autonomously generates a multi-level directory structure that maps to its inherent knowledge logic. This transforms the unstructured raw text into a structured knowledge tree, providing a global blueprint for subsequent hierarchical and systematic problem generation.

[0031] Table of Contents Generation After multi-granular semantic segmentation, all location blocks belonging to the same directory block and their unique identifiers are submitted as complete input information to the large language model. Simultaneously, the large language model is given explicit instructions, requiring it to read and understand the entire text, and not simply summarize it, but generate a directory structure with a clear hierarchical relationship. This directory must comprehensively cover the core themes and logical divisions of the input text. A specific requirement is that each entry in the directory must be clearly labeled with the identifier (ID) of one or more location blocks supporting the content of that entry.

[0032] In this way, an explicit mapping from abstract knowledge nodes to specific text locations is achieved, resulting in a multi-level directory structure. For example, for a document about the discovery of ancient pottery, the large language model might generate directory entries with associated localization block IDs as follows: * I. Archaeological Discoveries [0,1,2] ** 1.1 Relevant Discoveries of Miaodigou Culture [0,1,2] *** 1.1.1 Characteristics, Functions, and Ritual Speculations of the Openwork Human-Face Inverted Basin-Shaped Pottery [0,1,2] **** 1.1.1.1 Background and initial questions surrounding the discovery of the openwork human-faced inverted basin-shaped pottery [0] **** 1.1.1.2 Analysis of the characteristics of openwork human face inverted basin-shaped pottery[1] **** 1.1.1.3 Functional speculation and ritual association of openwork human face inverted basin-shaped pottery[2] In this example, the content of the catalog entry “1.1.1.2 Feature analysis of openwork human face inverted basin-shaped pottery” is entirely supported by the positioning block with ID [1].

[0033] Directory Tree Parsing The multi-level directory structure generated by the large language model is automatically parsed and structured, transforming it into a machine-readable and operable tree data structure, forming a dynamic directory tree (knowledge tree). Each directory tree generation is performed by model extraction, not rule parsing. Therefore, different prompt word templates can be used for different document types (e.g., laws and regulations vs. books and textbooks) to achieve better directory generation results. Each node in the directory tree contains the following key fields: "level" represents the node's level depth in the tree, "title" represents the name of the directory entry, "chunk_ids" is an array storing all the location block IDs associated with the entry, and "children" is an array storing all its child nodes. Thus, a portion of the above example will be converted into a JSON object (e.g., ...). Figure 2 (As shown).

[0034] In this way, unstructured text content is transformed into a structured knowledge tree. This reveals the document's macro-level knowledge system and establishes a precise index relationship between knowledge nodes and text fragments, laying the foundation for subsequent accurate knowledge location and hierarchical question generation.

[0035] S3, Problem Generation Steps By utilizing the dynamic directory tree constructed in the preceding steps, any node in the directory tree can be flexibly selected as the anchor point for questions according to the training objective, rather than being limited to leaf nodes. This enables the generation of a comprehensive question system that ranges from macro-level overviews to micro-level details, and from single-point knowledge to multi-hop connections.

[0036] Questioning strategies First, the structured directory tree generated by the dynamic directory tree construction step is received and parsed. Then, a target node is selected from the directory tree according to a preset or user-defined question strategy. The selection of the target node is not fixed but can be dynamically adjusted according to the needs of the training corpus.

[0037] Strategy 1: Select leaf nodes (i.e., the lowest level nodes) as target nodes. This strategy is suitable for generating factual or analytical questions that focus on the finest-grained knowledge points and have relatively concentrated answers.

[0038] Strategy 2: Select the parent leaf node (such as a second-level or third-level node) as the target node. This strategy is suitable for generating inductive and comprehensive questions that require summarizing the content of all its child nodes, such as requiring a large language model to summarize the key points of a chapter or compare the similarities and differences between multiple subtopics.

[0039] Problem generation After identifying the target node, the title field (i.e., the entry title) of that node is used as the question topic. The question topic and the original text information corresponding to the target node are input into the large language model to generate the question.

[0040] S4. Answer Generation Steps After identifying the target node, the original text content of all the location blocks associated with the node's `title` field (i.e., the entry title) and its `chunk_ids` field is submitted as input to the large language model. The large language model uses the entry title as the core theme and, strictly based on the specific content of the provided location blocks, generates one or more corresponding question-answer pairs with clear subject statements and complete structures. By combining the entry title with the specific original text, the generated questions are ensured to be highly topic-specific and can be accurately answered entirely based on the given context, effectively enhancing the independence and factual accuracy of the questions.

[0041] To address the potential loss of key information due to text segmentation and ensure the generated answers possess high accuracy, completeness, and logical rigor, this step achieves this goal by dynamically constructing an expanded context window and introducing a Chain of Thought (CoT) mechanism. This process significantly differs from simple restatement or summarization of the original text.

[0042] Context configuration First, the question generated during the question generation step and its directly associated location block identifiers (e.g., the location block ID associated with the question is [3,4]) are received. To ensure sufficient background information when generating the answer to avoid misinterpretation, these associated blocks are not used as the context alone. Instead, a dynamic context expansion strategy is executed: taking the directly associated location block as the core, its adjacent preceding and following location blocks (e.g., blocks with IDs [2] and [5]) are automatically included in the processing scope, thus forming a more complete and semantically coherent extended context window (i.e., [2,3,4,5]), which is located in the same directory block. This strategy effectively reconstructs the local semantic coherence temporarily severed by the segmentation operation, laying a solid foundation for accurate understanding and deep reasoning of large language models.

[0043] Subsequently, the user's question and the expanded context window are submitted to the large language model, and prompt words are injected to guide answer generation, resulting in answer pairs, which serve as supervised fine-tuning corpus. The core requirement is that the answer must be generated strictly based on and solely on the provided expanded context content, prohibiting the introduction of external knowledge or subjective assumptions, thereby eliminating factual illusions at the source. To improve the reliability and logic of the answer, the prompt words explicitly require the large language model to use the Chain of Reasoning (CoT) reasoning model to generate the reasoning process before deriving the final answer. This CoT template requires the large language model to systematically perform the following operations: First, analyze and categorize the question to clarify its professional field and core examination points; then, accurately locate the original text within the provided context to identify all relevant evidence fragments; next, analyze and organize the inherent logical relationships between these evidence fragments, such as causality, parallelism, progression, or contrast; finally, integrate the evidence based on this logical relationship to construct a clear reasoning path supporting the final answer.

[0044] Ultimately, the large language model needs to output two parts: first, a complete, step-by-step deductive thought process; and second, a final answer strictly based on the aforementioned reasoning. This answer must meet multiple requirements, including semantic independence, comprehensive information, accurate expression, and linguistic standardization, ensuring that it can independently and completely answer the posed question.

[0045] This design makes the questioning ability of this method extremely powerful. For example, for the same document on ancient pottery, different levels of questions can be generated flexibly: selecting the leaf node “Characteristic Analysis of Hollow Human Face Inverted Basin Pottery”[1] can generate detailed questions such as “What are the specific morphological features of hollow human face inverted basin pottery?”; if its parent node “Characteristics, Functions and Ritual Speculations of Hollow Human Face Inverted Basin Pottery”[0,1,2] is selected, then multi-hop reasoning questions such as “Try to comprehensively discuss the characteristics, possible functions and potential role of hollow human face inverted basin pottery in rituals” can be generated, which require the integration of information from multiple positioning blocks to answer.

[0046] S5. Multi-round question-and-answer generation steps Preferably, this embodiment also provides an optional step S5 for generating multi-turn question-and-answer pairs. The multi-turn dialogue (multi-turn question-and-answer pair) generation stage, as an advanced optional function of this invention, aims to solve the problem that a single question-and-answer pair cannot simulate continuous and in-depth dialogue interaction. Its core purpose is to intelligently combine multiple semantically related and logically progressive independent question-and-answer pairs into multi-turn dialogue data with a unified theme and logical coherence. A significant feature of this stage is the diversity of its synthesis modes, supporting the flexible generation of dialogues with different structures according to different training objectives, greatly enhancing the richness and practicality of the generated corpus.

[0047] Multi-round strategy This phase relies on a flexible and configurable multi-turn synthesis strategy library. First, metadata analysis is performed on all generated single-turn question-answer pairs, and they are initially grouped based on their source directory node ID (chunk_id), question type, and keywords in the directory tree. The strategy library pre-configures various synthesis modes, which can be selected or combined as needed. The main modes include, but are not limited to: The "deep inquiry" mode involves asking questions about a core entity or concept, moving from broad concepts to specific details or underlying reasons, forming a progressive dialogue chain by delving into the parent directory level and then to the sub-directory level.

[0048] The "breadth expansion" mode generates question-and-answer pairs for multiple parallel subdirectories under the same parent directory level and combines them into a comprehensive dialogue that covers multiple aspects of the topic.

[0049] Model transcription After determining the synthesis strategy, the process of constructing multi-turn dialogue instances begins. Multiple selected single-turn question-and-answer pairs and their corresponding original context information are submitted to the large language model, which is then instructed to reconstruct these independent question-and-answer pairs into a natural and fluent multi-turn dialogue according to the specified strategy pattern. The reconstruction process must strictly adhere to the instructions, including but not limited to: adding reasonable and strategy-compliant introductory phrases and transition words to the dialogue (e.g., "We just learned about X, so what about Y?" or "Besides A, what factors need to be considered in B?"); ensuring clear referencing in each turn of dialogue to avoid ambiguity; and keeping the entire dialogue centered around the core theme and staying on track.

[0050] Quality control A preliminary quality filtering mechanism is built-in. Each generated multi-turn dialogue undergoes automated verification using a set of basic rules. The verification criteria mainly include: topic consistency, determining whether each turn of the dialogue consistently revolves around the same core topic; discarding dialogues with topic drift detected; turn validity, checking whether the number of dialogue turns meets the preset minimum requirement (e.g., no less than 2 turns), and filtering out dialogues containing meaningless or excessively short turns; and logical coherence, performing a simple logical coherence assessment of the dialogue, eliminating generated results that clearly lack logical connection between turns or have disordered order. This preliminary filtering significantly reduces the workload of subsequent unified quality check steps.

[0051] Output format The multi-turn dialogues, which undergo initial quality control, will be combined with the original single-turn question-and-answer pairs to form a corpus set for quality inspection, which will then enter a unified quality check and filtering process. These multi-turn dialogues are output in a structured format, with each dialogue accompanied by metadata information such as the synthesis strategy used and the source question-and-answer pair IDs, to facilitate subsequent tracking and analysis.

[0052] S6. Quality Inspection Procedures This embodiment also includes a quality inspection stage, which establishes an automated, multi-dimensional, and quantifiable evaluation and filtering pipeline. This stage consists of two steps: preliminary screening and final screening, ensuring the high quality and high usability of the final output corpus.

[0053] Pre-screening In the question generation sub-step of the question-answering generation phase, a first filtering mechanism is embedded. Clearly defined question rejection rules are pre-defined, instructing the large language model to exclude low-value or unsuitable questions during question construction, reducing invalid computation at the source. Specific rejection conditions should be customized based on the actual domain and type system.

[0054] Post-screening For the question-answer pairs that have passed the initial screening and been generated, a multi-dimensional quantitative evaluation model based on a large language model is used for further screening. This large language model independently scores each question-answer pair across the following six core dimensions (e.g., using a scoring scale of 0-5): 1. Completeness and Clarity: Assess whether the subject of the question is clear and whether the answer fully covers the core information and is logically clear.

[0055] 2. Technical depth: Assess whether the questions and answers involve in-depth content such as technical parameters, applicable conditions, and principle analysis.

[0056] 3. Knowledge and Analytical Ability: Assess whether the questions and answers require or demonstrate deep cognitive abilities such as induction, summarization, comparison, and analysis.

[0057] 4. Accuracy and Citation Transparency: Assess whether the answer is absolutely faithful to the provided context, whether all arguments are supported by clear and traceable original texts, and whether the expression is self-consistent.

[0058] 5. Illusion Index: This is a veto item. It assesses whether any factual statement in the answer cannot be supported by direct or reasonable inference in the provided context. The presence of illusion results in a score of 0.

[0059] 6. Difficulty and Depth of Reasoning: Assess the complexity of the reasoning required for the problem, whether it is a simple restatement or requires multiple steps of derivation and synthesis.

[0060] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0061] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0062] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0066] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for automatically generating supervised fine-tuning corpora based on dynamic directory trees, characterized in that, Includes the following steps: The parsed original document is obtained and divided using a multi-granularity semantic segmentation method to obtain multiple location blocks, and each location block is assigned a globally unique identifier. Based on the location blocks and their identifiers, a large language model is used to parse the content of each location block, generate a multi-level directory structure that maps its inherent knowledge logic, and construct a dynamic directory tree. The system acquires a preset questioning strategy, selects target nodes from the dynamic directory tree, and generates questions and answers step by step based on the target nodes to form question-answer pairs, which serve as supervised fine-tuning corpus. At the same time, during the answer generation process, the system uses a dynamic extended context strategy and a structured thought chain to guide the answer generation, thus completing the automatic corpus generation process.

2. The method for automatically generating supervised fine-tuning corpora based on dynamic directory trees according to claim 1, characterized in that, The step of obtaining multiple positioning blocks includes: Coarse-grained segmentation: For the original document, it is initially divided according to a pre-set fixed length threshold to obtain multiple directory blocks; Fine-grained segmentation: For each directory block, a semantic-based recursive segmentation algorithm is used for fine-grained division to obtain multiple location blocks corresponding to each directory block.

3. The method for automatically generating supervised fine-tuning corpora based on dynamic directory trees according to claim 2, characterized in that, The steps for fine division include: The semantic-based recursive segmentation algorithm first identifies the natural language boundaries of each directory block as the segmentation points, wherein the natural language boundaries include paragraph endings, newline characters, and periods. Based on the aforementioned split points, each directory block is recursively divided into multiple location blocks.

4. The method for automatically generating supervised fine-tuning corpora based on dynamic directory trees according to claim 2 or 3, characterized in that, The steps for constructing the dynamic directory tree include: All location blocks and identifiers under the same directory block are input into the large language model. The large language model reads and understands the content of all location blocks according to specific instructions, and generates a multi-level directory structure that maps its inherent knowledge logic. Each directory entry in the multi-level directory structure is marked with an identifier of one or more location blocks that support the content of the directory entry, and the multi-level directory structure covers the core theme and division logic of the original document. The multi-level directory structure is automatically parsed and structured to transform it into a tree-like data structure, forming a dynamic directory tree. Each node in the dynamic directory tree includes the following key fields: level: indicates the level depth of the node in the dynamic directory tree, where the node represents a directory entry; title: Indicates the name of the directory entry; chunk_ids: An array used to store the identifiers of all location blocks associated with the corresponding directory entry; children: This is an array used to store all the child nodes of the corresponding node.

5. The method for automatically generating supervised fine-tuning corpus based on dynamic directory tree according to claim 1, characterized in that, The questioning strategies include the following: Questioning Strategy 1: Select the leaf node in the dynamic directory tree as the target node; Questioning Strategy Two: Select the parent leaf node in the dynamic directory tree as the target node.

6. The method for automatically generating supervised fine-tuning corpora based on dynamic directory trees according to claim 1, characterized in that, The steps for generating the question-answer pairs include: Question generation: Use the key field "title" in the target node as the question topic, where "title" represents the name of the directory entry; Input the question topic and the original text information corresponding to the target node into the large language model to generate a question; Answer generation: Based on the key fields title and chunk_ids in the target node, all location blocks associated with the generated question are extracted. The associated location blocks are used as the initial context, and chunk_ids is used as an array to store the identifiers of all location blocks associated with the corresponding directory entry. Based on all the associated location blocks and their identifiers, the dynamic context expansion strategy takes each associated location block as the core and automatically includes the location blocks adjacent to it before and after in the context scope, which together with the initial context constitute an expanded context window, wherein the expanded context window is located in the same directory block; The question and the extended context window are input into the large language model. By injecting prompt words into the large language model, one or more answers corresponding to the question are generated, forming one or more question-answer pairs, with the key field title in the target node as the core. The prompt words limit the use of a structured thinking chain reasoning mode to first generate the reasoning process, and then combine the reasoning process to generate the answer.

7. The method for automatically generating supervised fine-tuning corpora based on dynamic directory trees according to claim 6, characterized in that, The steps involved in generating the reasoning process using the structured thinking chain reasoning model include: The problem is analyzed and categorized by professional field to determine its professional field and core examination points; Based on the relevant professional field and core examination points, precise positioning is performed in the extended context window to identify all relevant positioning as relevant evidence fragments; Analyze and sort out the inherent logical relationships between the relevant evidence fragments, including one or more of the following: causality, parallelism, progression, and contrast; By integrating the relevant evidence fragments based on the inherent logical relationships, a reasoning path supporting the answer can be constructed.

8. The method for automatically generating supervised fine-tuning corpus based on dynamic directory tree according to claim 1, characterized in that, It also includes quality inspection steps: During the question generation process, based on the actual professional field and the pre-set question rejection rules of the type system, the large language model is instructed to exclude questions that do not conform to the question rejection rules when generating questions; During the answer generation process, it is first determined whether the question can be answered based on the extended context window. If it can, the answer is generated directly; otherwise, the question is discarded and no answer needs to be generated. A large language model is used to perform multi-dimensional quantitative evaluation of the generated answers to obtain corresponding scores. These scores are then compared with a preset threshold. If the score is greater than or equal to the threshold, the generated answer is retained; otherwise, the answer and its corresponding question are discarded. The dimensions are as follows: Completeness and Clarity: Assess whether the subject of the question is clear, and whether the answer comprehensively covers the core information and is logically clear; Technical depth: Assess whether the questions and answers involve in-depth content, including technical parameters, applicable conditions, and principle analysis; Knowledge-based and analytical: Assess whether the questions and answers require or demonstrate deep cognitive abilities, including induction, summarization, comparison, and analysis; Accuracy and Citation Transparency: Assess whether the answer is absolutely faithful to the provided extended context window, whether all arguments are supported by clear and traceable original texts, and whether the expression is self-consistent; Illusion Index: This assesses whether any factual statement in the answer cannot be supported by direct or reasonable inference within the provided extended context window. If an illusion is found, the score will be 0. Difficulty and Depth of Reasoning: Assess the reasoning complexity required for the problem, whether it is a simple restatement or requires multiple steps of derivation and synthesis.

9. The method for automatically generating supervised fine-tuning corpora based on dynamic directory trees according to claim 1, characterized in that, It also includes a multi-round question-and-answer generation process: Metadata analysis is performed on the question-answer pairs generated in a single round. Preliminary grouping is performed based on the key field chunk_id, question strategy, and keywords in the dynamic directory tree. The chunk_ids is used as an array to store the identifiers of all location blocks associated with the corresponding directory entry. Based on a variety of preset synthesis modes, select one or more synthesis modes, including deep questioning mode and breadth expansion mode. The deep questioning mode forms a progressive question-and-answer chain by going deeper into the parent directory level to the subdirectory level. The breadth expansion mode generates question-and-answer pairs for multiple parallel subdirectory levels under the same parent directory level and combines them into a comprehensive question and answer that covers multiple aspects of the topic. Select single-turn or multi-turn question-answer pairs and their corresponding extended context windows and input them into the large language model. Instruct it to reconstruct multi-turn question-answer pairs using independent questions and answers according to the selected synthesis mode. Each round of question-answer pairs needs to be automatically validated by the set validation criteria, which include: Theme consistency: Determine whether each round of question and answer revolves around the same core theme. If so, keep it; otherwise, discard it. Round validity: Check whether the number of rounds of the question-and-answer pair has reached the preset minimum requirement. If so, continue to the next round. If not, stop or continue according to actual needs, and filter out question-and-answer pairs that contain meaningless or short rounds. Logical coherence: The logical coherence of the question-and-answer pairs is evaluated to eliminate those that lack logical connection between rounds or are out of order.

10. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements a method for automatically generating supervised fine-tuning corpora based on a dynamic directory tree as described in any one of claims 1-9.

Images