Language model can explain reasoning track processing method, device, equipment and storage medium
By fine-tuning the segmented question-and-answer process and extracting reasoning trajectories from the large language model, the problem of insufficient interpretability of the large language model in professional fields is solved, and the interpretability and traceability of the model output are improved.
Patent Information
- Application Number
- CN202610246884.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-02
- Publication Date
- 2026-06-12
AI Technical Summary
Large language models lack interpretability and traceability in professional applications, making it difficult for users to trust model results and diagnose problems.
By segmenting documents in the target domain, structured question-and-answer data is constructed to guide the large language model in generating intermediate inference steps. The inference trajectory is then used to analyze the model output, thereby improving the interpretability and traceability of the model.
This improves the interpretability and traceability of large language models, enabling users to understand and verify the model's decision-making process, thereby enhancing the model's reliability and diagnostic efficiency.
Smart Images
Figure CN122198164A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and storage medium for processing language model-interpretable reasoning trajectories. Background Technology
[0002] Large Language Models (LLMs) have made significant progress in various natural language processing tasks, but their "black box" nature becomes a major obstacle when applied to specialized fields such as materials science. In these fields, simply providing correct answers is insufficient; users often need to understand how the model arrives at those answers in order to validate, debug, or build trust. Traditional LLM fine-tuning methods typically focus only on the accuracy of the final output, neglecting the model's internal reasoning process, resulting in insufficient interpretability and traceability in key application scenarios.
[0003] For example, in materials science research, large language models may be used to extract the properties of specific materials or predict their behavior from a large amount of literature. If the model gives a prediction but cannot explain its reasoning path, researchers will find it difficult to trust the result and will be unable to discover potential errors or biases. Furthermore, when the model errs, the lack of a reasoning path makes diagnosing problems and improving the model exceptionally difficult. While techniques such as "Chain-of-Thought" (CoT) prompts can induce the model to generate reasoning steps, these steps are usually generated ex-post, and their quality and consistency are difficult to guarantee.
[0004] Therefore, there is an urgent need for a method to process the interpretable reasoning trajectory of language models, which can improve the interpretability and traceability of large language models. Summary of the Invention
[0005] The main objective of this invention is to provide a method, apparatus, device, and storage medium for processing interpretable reasoning trajectories of language models, aiming to solve the technical problem that the intermediate reasoning process of large language models in task processing is not visible in the prior art, resulting in insufficient interpretability and traceability of large language models.
[0006] To achieve the above objectives, the present invention provides a method for processing interpretable reasoning trajectories of language models, the method comprising the following steps: The target domain document is segmented to obtain multiple text units; Structured question-and-answer data is constructed based on the text units, and the structured question-and-answer data includes instructions that guide the generation of reasoning steps; The structured question-answering data is used to fine-tune the large language model in a segmented question-answering manner, and the intermediate reasoning steps generated by the large language model are extracted during the fine-tuning process. The reasoning trajectory is obtained based on the intermediate reasoning steps. The inference trajectory is used to perform model output analysis of the large language model to obtain model output analysis results.
[0007] Optionally, the step of segmenting the target domain document to obtain multiple text units includes: The target domain document is preprocessed to obtain a preprocessed target domain document. Semantic boundary recognition is performed on the preprocessed target domain document, and the thematic transition positions between adjacent paragraphs are determined based on the recognition results. The preprocessed target domain document is segmented according to the topic transition position to obtain multiple text units.
[0008] Optionally, the step of constructing structured question-and-answer data based on the text units includes: Information is extracted from the text units to obtain key information, and a set of candidate questions is generated based on the key information; Target questions are selected from the candidate question set using relevance scoring, and corresponding reference answers are generated for the target questions; The instructions that guide the generation of reasoning steps are inserted between the target question and the reference answer in the form of a chain-like thinking template to generate structured question-and-answer data corresponding to the text unit.
[0009] Optionally, the step of using the structured question-answering data to perform segmented question-answering fine-tuning on the large language model, extracting intermediate inference steps generated by the large language model during the fine-tuning process, and obtaining the inference trajectory based on the intermediate inference steps includes: Each text unit and its corresponding structured question-and-answer data is input into the large language model section by section; The large language model is guided to generate corresponding intermediate reasoning steps before generating the final answer, and to generate the final answer corresponding to the structured question-and-answer data after generating the intermediate reasoning steps. Based on the structured question-and-answer data and the corresponding final answers, the large language model is fine-tuned in a segmented question-and-answer manner; The timestamp information and token location index information of the intermediate reasoning steps are obtained, and the semantic correspondence between the timestamp information and the token location index information and the reference answer is aligned to obtain a reasoning trajectory with step numbers and logical dependencies.
[0010] Optionally, after the step of obtaining the reasoning trajectory based on the intermediate reasoning steps, the method further includes: The text unit, the structured question-and-answer data, and the reasoning trajectory are associated to obtain the association result, and the association result is stored in the target database.
[0011] Optionally, the step of using the inference trajectory to perform model output analysis of the large language model and obtaining the model output analysis results includes: The corresponding reasoning trajectory is retrieved from the target database based on the user's query information; An explanation of the decision-making process of the large language model is generated based on the inference trajectory, and the inference trajectory is visualized. The inference trajectory is compared with the final answer output by the corresponding model to obtain the verification result; When the verification result indicates the existence of a logical break, the text unit that caused the logical break and the corresponding structured question and answer data are located, and the location result is used as the model output analysis result.
[0012] Optionally, the method further includes: Based on the positioning results, the text units and / or the structured question-and-answer data are corrected to generate corrected data; The large language model is trained using the corrected data, and the model is optimized based on the training results.
[0013] Furthermore, to achieve the above objectives, the present invention also proposes a processing device for interpretable reasoning trajectories based on language models, the device comprising: The document segmentation and processing module is used to segment documents in the target domain to obtain multiple text units; A question-and-answer data construction module is used to construct structured question-and-answer data based on the text units, wherein the structured question-and-answer data includes instructions that guide the generation of reasoning steps; The reasoning trajectory extraction module is used to perform segmented question-and-answer fine-tuning on the large language model using the structured question-and-answer data, and extract the intermediate reasoning steps generated by the large language model during the fine-tuning process, and obtain the reasoning trajectory based on the intermediate reasoning steps; The reasoning trajectory utilization module is used to perform model output analysis of the large language model using the reasoning trajectory, and obtain model output analysis results.
[0014] Furthermore, to achieve the above objectives, the present invention also proposes a processing device for language model interpretable reasoning trajectories, the device comprising: a memory, a processor, and a processing program for language model interpretable reasoning trajectories stored in the memory and executable on the processor, the processing program for language model interpretable reasoning trajectories being configured to implement the steps of the processing method for language model interpretable reasoning trajectories as described above.
[0015] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a processing program for language model interpretable inference trajectories, wherein when the processing program for language model interpretable inference trajectories is executed by a processor, it implements the steps of the language model interpretable inference trajectory processing method described above.
[0016] This invention discloses a method for segmenting a target domain document to obtain multiple text units; constructing structured question-and-answer data based on these text units, the structured question-and-answer data including instructions to guide the generation of inference steps; fine-tuning a large language model using the structured question-and-answer data in a segmented manner, extracting intermediate inference steps generated by the large language model during the fine-tuning process, and obtaining an inference trajectory based on these intermediate inference steps; and using the inference trajectory to analyze the model output of the large language model to obtain model output analysis results. Because this invention utilizes structured question-and-answer data to fine-tune a large language model in a segmented manner, extracts intermediate inference steps generated by the large language model during the fine-tuning process, obtains an inference trajectory based on these intermediate inference steps, and then uses the inference trajectory to analyze the model output of the large language model, compared to existing technologies, this invention improves the interpretability and traceability of the large language model. Attached Figure Description
[0017] Figure 1 This is a flowchart illustrating the first embodiment of the method for processing interpretable reasoning trajectories using the language model of the present invention. Figure 2 This is a flowchart illustrating the second embodiment of the method for processing interpretable reasoning trajectories using the language model of the present invention. Figure 3 This is a flowchart illustrating the third embodiment of the method for processing interpretable reasoning trajectories using the language model of the present invention. Figure 4 This is a structural block diagram of the first embodiment of the processing device for interpretable reasoning trajectories based on the language model of the present invention; Figure 5 This is a schematic diagram of the structure of a processing device for interpretable inference trajectories of a language model in the hardware operating environment involved in the embodiments of the present invention.
[0018] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0019] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.
[0020] This invention provides a method for processing interpretable reasoning trajectories of language models, referring to... Figure 1 , Figure 1This is a flowchart illustrating the first embodiment of the method for processing interpretable reasoning trajectories using the language model of the present invention.
[0021] In this embodiment, the method for processing the interpretable reasoning trajectory of the language model includes steps S10 to S40: Step S10: Segment the target domain document to obtain multiple text units.
[0022] It should be noted that the execution subject of this embodiment can be a computer server device with data processing, network communication, and program execution functions applied in the scenario of extracting and utilizing interpretable inference trajectories of language models, such as a server, tablet computer, or personal computer, or an electronic device capable of realizing the above functions (such as a processing device for interpretable inference trajectories of language models). The following uses a system containing a processing device for interpretable inference trajectories of language models (hereinafter referred to as the system) as an example to illustrate this embodiment and the following embodiments.
[0023] Understandably, the aforementioned target domain documents can refer to technical documents, academic papers, patent documents, standards and specifications, or reports in a specific professional field (such as materials science, medicine, finance, or law). These documents contain structured professional knowledge and logically organized content, serving to provide a domain-specific knowledge base for the large language model.
[0024] It should be understood that segmentation processing can refer to the process of dividing a target domain document into relatively independent and semantically complete sections, chapters, paragraphs, or logical blocks based on the logical structure of the document content in the target domain document.
[0025] It should be noted that segmentation can be performed in at least one of the following ways: dividing based on the original chapter titles of the target domain document, dividing based on the semantic coherence and thematic consistency between paragraphs, or dividing based on the identification of content boundaries through natural language processing technology.
[0026] It should be understood that a text unit refers to the smallest processing unit with independent semantics and complete logical content, obtained through segmentation. Each text unit satisfies the following characteristics: it contains a relatively complete knowledge fragment or topic content; and it can be understood and processed semantically independently of other text units.
[0027] In a specific implementation, logical paragraph boundaries in the target domain document can be identified, and the target domain document can be segmented according to the logical paragraph boundaries to divide the target domain document into multiple semantically complete text units.
[0028] Step S20: Construct structured question-and-answer data based on the text units, wherein the structured question-and-answer data includes instructions that guide the generation of reasoning steps.
[0029] It should be noted that structured question-answering data can refer to a set of "question Q-answer A-additional field" triples that are encapsulated in a unified format after information is extracted from text units; the additional field at least includes the unit identifier, the instruction to guide the generation of reasoning steps, and the offset of the source of the answer, which can be directly parsed by the program without the need for natural language understanding.
[0030] Furthermore, the answer source offset can be a location marker in structured question-and-answer data used to locate the basis for the answer, enabling a fast and accurate mapping from the answer to the original text.
[0031] It should be explained that the instruction that guides the generation of reasoning steps can be a control signal embedded between question Q and answer A in the form of a template or tag, which forces the large language model to output parsable intermediate reasoning steps before outputting the final answer.
[0032] In a specific implementation, key information can be extracted from the text unit to obtain key information, and a set of candidate questions can be generated based on the key information; a target question can be selected from the set of candidate questions through relevance scoring, and a corresponding reference answer can be generated for the target question; the instruction to guide the generation of reasoning steps can be inserted between the target question and the reference answer in the form of a chain thinking template to generate structured question and answer data corresponding to the text unit.
[0033] It should be noted that key information can be elements within a text unit that carry core knowledge and semantics. The candidate question set can be a list of all candidate questions generated in batches using the same set of key information and preset question templates (such as "What is…", "Why…", or "How to calculate…").
[0034] It should be explained that the chain-like thinking template can include start and end markers for reasoning steps, step type labels, and logical dependency order identifiers.
[0035] Understandably, selecting target questions from the candidate question set through relevance scoring can be achieved by quantifying the semantic similarity between candidate questions and the original text: first, use a sentence encoder to obtain question vectors and text unit vectors; calculate cosine similarity and sort them in descending order; take questions that are higher than a preset threshold as target questions, and discard the rest.
[0036] It should be understood that the target questions can be the final questions retained after relevance scoring and about to enter the fine-tuning corpus. Each text unit typically retains 1-3 target questions. The reference answer can be a textualized expression of key information tied to the target question, serving as the "standard answer" in the fine-tuning stage; its character offset is also recorded for subsequent evidence tracing.
[0037] Step S30: Use the structured question-answering data to perform segmented question-answering fine-tuning on the large language model, and extract the intermediate reasoning steps generated by the large language model during the fine-tuning process, and obtain the reasoning trajectory based on the intermediate reasoning steps.
[0038] It should be explained that segmented question-answering fine-tuning can be done by using text units as the smallest sample granularity, and inputting the corresponding structured question-answering data into the large language model one by one for fine-tuning; each parameter update is based only on the structured question-answering data of the current text unit, ensuring one sample per segment and one gradient per segment, thereby realizing local knowledge injection and solidification of reasoning format.
[0039] It should be added that intermediate reasoning steps generated by the large language model can be extracted during the fine-tuning process. The steps to obtain the reasoning trajectory based on the intermediate reasoning steps may include: extracting intermediate reasoning steps generated by the large language model during the fine-tuning process; performing structured parsing on the intermediate reasoning steps to form a reasoning trajectory containing step numbers, logical relationships and evidence correspondence information.
[0040] It should be noted that intermediate reasoning steps refer to natural language text fragments explicitly generated by the large language model under the constraints of a chain-like thinking template before outputting the final answer. Each fragment may be accompanied by a timestamp or sequence number and can be automatically extracted by the program. Intermediate reasoning steps can include at least: concept recognition (the large language model identifies and interprets key concepts in the text unit), information extraction (the large language model extracts relevant facts and data from the text unit), logical connection (the large language model explains the logical relationships between different information points), computation process (the large language model demonstrates any calculations or derivations involved), and evidence citation (the large language model points out the text fragments on which its answer or reasoning is based). These generated intermediate reasoning steps are recorded in real time, forming a reasoning trajectory.
[0041] Understandably, a complete sequence can be formed by splicing together all intermediate inference steps generated on the same text unit in the order of their generation.
[0042] In a specific implementation, each text unit and its corresponding structured question-and-answer data can be input into a large language model section by section; the large language model is guided to generate corresponding intermediate reasoning steps before generating the final answer, and to generate the final answer corresponding to the structured question-and-answer data after generating the intermediate reasoning steps; the large language model is fine-tuned in a segmented question-and-answer manner based on each structured question-and-answer data and its corresponding final answer; the timestamp information of the intermediate reasoning steps is obtained, and the intermediate reasoning steps are adjusted according to the timestamp information, with the adjustment result serving as the reasoning trajectory.
[0043] It should be added that the generation order (i.e., timestamp information) and token position index information of intermediate reasoning steps can also be obtained, and the semantic correspondence between the generation order and token position index information of each intermediate reasoning step and the reference answer can be aligned to form a reasoning trajectory with step number and logical dependency.
[0044] To facilitate users in querying the reasoning process of specific answers or analyzing the thinking patterns of large language models in different contexts, after the step of obtaining the reasoning trajectory based on the intermediate reasoning steps, the method further includes: associating the text unit, the structured question-and-answer data, and the reasoning trajectory to obtain an association result, and storing the association result in a target database. The target database can be a searchable database.
[0045] It should be noted that the association results can be stored in the target database in the form of structured fields. The structured fields may include text unit ID, question-answer pair ID, inference step sequence number, logical dependency index, and consistency result label, etc.
[0046] Step S40: Analyze the model output of the large language model using the inference trajectory to obtain the model output analysis results.
[0047] Understandably, obtaining the reasoning trajectory based on intermediate reasoning steps can visualize the internal reasoning process of the "black box" of the large language model, enhance users' understanding and trust in the decision-making of the large language model, and significantly improve the interpretability of the large language model.
[0048] It should be understood that by recording the complete reasoning chain through reasoning trajectory, each model output of the large language model can be traced back to its basis and logic, thereby improving the traceability of the large language model.
[0049] It should be explained that the model output analysis results can be structured conclusions and diagnostic information derived from evaluating the reliability, correctness, logic, and interpretability of the model output (i.e., the final answer) of the large language model based on the reasoning trajectory.
[0050] In practical implementation, the inference trajectory can be used to perform consistency checks and logical coherence checks to obtain the model output analysis results.
[0051] This embodiment discloses a method for segmenting a target domain document to obtain multiple text units; constructing structured question-and-answer data based on these text units, the structured question-and-answer data including instructions to guide the generation of inference steps; fine-tuning a large language model using the structured question-and-answer data in a segmented manner, extracting intermediate inference steps generated by the large language model during the fine-tuning process, and obtaining an inference trajectory based on these intermediate inference steps; and using the inference trajectory to analyze the model output of the large language model to obtain model output analysis results. Because this embodiment uses structured question-and-answer data to fine-tune a large language model in a segmented manner, extracts intermediate inference steps generated by the large language model during the fine-tuning process, obtains an inference trajectory based on these intermediate inference steps, and then uses the inference trajectory to analyze the model output of the large language model, compared to existing technologies, this embodiment improves the interpretability and traceability of the large language model.
[0052] refer to Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the method for processing interpretable reasoning trajectories using the language model of the present invention.
[0053] Based on the first embodiment described above, in this embodiment, step S10 includes steps S101 to S103: Step S101: Preprocess the target domain document to obtain the preprocessed target domain document.
[0054] Step S102: Perform semantic boundary recognition on the preprocessed target domain document, and determine the thematic transition position between adjacent paragraphs based on the recognition result.
[0055] Step S103: The preprocessed target domain document is segmented according to the topic transition position to obtain multiple text units.
[0056] It should be understood that preprocessing a target domain document can involve removing formatting tags from the target domain document to obtain standardized text, i.e., the preprocessed target domain document.
[0057] It should be noted that semantic boundary recognition can refer to calculating the semantic similarity or topic distribution distance between adjacent paragraphs in a preprocessed target domain document, with paragraphs as the smallest granularity; when the semantic similarity or topic distribution distance exceeds a predefined threshold, it is determined that there is a semantic break, thereby marking paragraph-level boundary candidate points.
[0058] It should be explained that the topic transition point can refer to the position between paragraphs corresponding to the maximum local distance among the above paragraph-level boundary candidate points, as the precise point where the document topic changes significantly; this point is used as the physical segmentation coordinate for subsequent sectioning processing to ensure the topic continuity within each text unit and the topic heterogeneity between units.
[0059] Understandably, by recognizing semantic boundaries and identifying thematic transition points, paragraph themes within the same text unit are made continuous, cross-thematic splicing is eliminated, fact drift in subsequent structured question-and-answer data is reduced, over-segmentation or under-segmentation is avoided, and the segmentation accuracy and the reliability and rationality of segmentation processing are improved.
[0060] This embodiment discloses a method for preprocessing a target domain document to obtain a preprocessed target domain document; performing semantic boundary recognition on the preprocessed target domain document to determine the thematic transition positions between adjacent paragraphs based on the recognition results; and segmenting the preprocessed target domain document according to the thematic transition positions to obtain multiple text units. Because this embodiment performs semantic boundary recognition on the preprocessed target domain document, determines the thematic transition positions between adjacent paragraphs based on the recognition results, and segments the preprocessed target domain document according to the thematic transition positions, compared to existing technologies, this embodiment improves the segmentation accuracy and the reliability and rationality of the segmentation process.
[0061] refer to Figure 3 , Figure 3 This is a flowchart illustrating the third embodiment of the method for processing interpretable reasoning trajectories using the language model of the present invention.
[0062] Based on the above embodiments, in this embodiment, step S40 includes steps S401 to S404: Step S401: Retrieve the corresponding reasoning trajectory from the target database based on the user query information.
[0063] Step S402: Generate an explanation of the decision-making process of the large language model based on the inference trajectory, and visualize the inference trajectory.
[0064] Step S403: Perform a consistency check between the inference trajectory and the final answer output by the corresponding model to obtain the check result.
[0065] Step S404: When the verification result indicates that there is a logical break, locate the text unit that caused the logical break and the corresponding structured question and answer data, and use the location result as the model output analysis result.
[0066] Understandably, in the specific implementation, the corresponding reasoning trajectory can be retrieved from the target database based on the user's query information, so that the user can not only obtain the final answer output by the large language model, but also understand the source and derivation process of the answer through the reasoning trajectory, thus improving the user interaction experience.
[0067] It should be noted that the explanation of the decision-making process of the large language model generated based on the reasoning trajectory can be a natural language explanation of the decision-making process of the large language model generated based on the reasoning trajectory. Therefore, users do not need to have professional algorithm knowledge to verify the basis of the answer, which significantly reduces the understanding threshold.
[0068] It should be understood that the explanation of the decision-making process of the large language model is generated based on the reasoning trajectory, and by visualizing the reasoning trajectory, the system can provide both natural language description and structured diagram in one go, which can meet the needs of different review scenarios, reduce the number of repeated queries, and make the rationality of the output of the large language model quickly evaluated by using the reasoning trajectory.
[0069] It should be noted that when the model gives an incorrect or unsatisfactory final answer, analyzing its reasoning trajectory can help to quickly locate the problem, such as insufficient knowledge, misunderstanding, or logical reasoning error. Based on the analysis of the reasoning trajectory, the training data can be adjusted, the model structure optimized, or the fine-tuning strategy improved, thereby enhancing the performance and robustness of the large language model.
[0070] In a practical implementation, consistency verification can be performed by checking the logical dependency between the sequence of intermediate inference steps contained in the inference trajectory and the final answer output by the model: verifying one by one whether each key assertion in the final answer can be found in the inference trajectory and is semantically consistent, and checking whether the causal order between the steps is broken; if all pass, it is determined to be consistent, otherwise it is broken.
[0071] Understandably, if the verification result indicates a logical break, the breakpoint number (the sequence number of the first mismatched or out-of-order step in the reasoning trajectory) and the break type (missing, conflicting, or reversed order) are provided. Then, based on the breakpoint number and break type, the text unit causing the logical break and its corresponding structured question-and-answer data are located, and the location result is used as the model's output analysis result. The model's output analysis result can be a quadruple including the breakpoint number, text unit identifier, structured question-and-answer data identifier, and break type. This quadruple can uniquely determine the source data location and cause of the logical break, and serves as the input index for the correction process.
[0072] It should be explained that consistency verification can also include multi-indicator detection based on step sequence constraints and evidence matching degree; when any indicator is lower than the preset indicator threshold, it is determined that there is a logical break and the break type and corresponding text unit are output.
[0073] In a specific implementation, the text unit and / or the structured question-and-answer data can be corrected based on the positioning result to generate corrected data; the corrected data can be used to train the large language model, and the large language model can be optimized based on the training result.
[0074] Understandably, the corrected data can be a new set of data formed by partially replacing or supplementing text units and / or structured question-and-answer data based on the location results.
[0075] This embodiment discloses a method for retrieving a corresponding inference trajectory from a target database based on user query information; generating an explanation of the decision-making process of a large language model based on the inference trajectory, and visualizing the inference trajectory; performing consistency verification between the inference trajectory and the final answer output by the corresponding model to obtain a verification result; when the verification result indicates a logical break, locating the text unit causing the logical break and the corresponding structured question-and-answer data, and using the location result as the model output analysis result; correcting the text unit and / or the structured question-and-answer data according to the location result to generate corrected data; training the large language model using the corrected data, and optimizing the large language model based on the training result. Because this embodiment performs consistency verification between the inference trajectory and the final answer output by the corresponding model, locates the text unit causing the logical break and the corresponding structured question-and-answer data when the verification result indicates a logical break, and then corrects the text unit and / or structured question-and-answer data according to the location result, and uses the corrected data to train and optimize the large language model, compared to existing technologies, this embodiment promotes the diagnosis and improvement of large language models, and enhances the performance and robustness of large language models.
[0076] Furthermore, this embodiment of the invention also proposes a storage medium storing a processing program for language model interpretable inference trajectories. When the processing program for language model interpretable inference trajectories is executed by a processor, it implements the steps of the language model interpretable inference trajectory processing method described above.
[0077] Reference Figure 4 , Figure 4 This is a structural block diagram of the first embodiment of the processing device for interpretable reasoning trajectories based on the language model of the present invention.
[0078] like Figure 4 As shown, the language model interpretable reasoning trajectory processing device proposed in this embodiment of the invention includes: a document segmentation processing module 501, a question-and-answer data construction module 502, a reasoning trajectory extraction module 503, and a reasoning trajectory utilization module 504.
[0079] The document segmentation and processing module 501 is used to segment the target domain document to obtain multiple text units.
[0080] The question-and-answer data construction module 502 is used to construct structured question-and-answer data based on the text units, and the structured question-and-answer data includes instructions that guide the generation of reasoning steps.
[0081] The reasoning trajectory extraction module 503 is used to perform segmented question-and-answer fine-tuning on the large language model using the structured question-and-answer data, and extract the intermediate reasoning steps generated by the large language model during the fine-tuning process, and obtain the reasoning trajectory based on the intermediate reasoning steps.
[0082] The reasoning trajectory utilization module 504 is used to perform model output analysis of the large language model using the reasoning trajectory, and obtain model output analysis results.
[0083] The question-and-answer data construction module 502 is further configured to extract key information from the text unit and generate a candidate question set based on the key information; select a target question from the candidate question set through relevance scoring and generate a corresponding reference answer for the target question; insert instructions that guide the generation of reasoning steps between the target question and the reference answer in the form of a chain thinking template to generate structured question-and-answer data corresponding to the text unit.
[0084] The reasoning trajectory extraction module 503 is further configured to input each text unit and its corresponding structured question-and-answer data into the large language model section by section; guide the large language model to generate corresponding intermediate reasoning steps before generating the final answer, and generate the final answer corresponding to the structured question-and-answer data after generating the intermediate reasoning steps; perform segmented question-and-answer fine-tuning on the large language model based on each structured question-and-answer data and its corresponding final answer; obtain the timestamp information and token position index information of the intermediate reasoning steps, and align them with the semantic correspondence between the timestamp information and the token position index information and the reference answer to obtain a reasoning trajectory with step numbers and logical dependencies.
[0085] The reasoning trajectory extraction module 503 is further configured to associate the text unit, the structured question-and-answer data and the reasoning trajectory to obtain the association result, and store the association result in the target database.
[0086] This device embodiment discloses a method for segmenting a target domain document to obtain multiple text units; constructing structured question-and-answer data based on the text units, the structured question-and-answer data including instructions to guide the generation of inference steps; using the structured question-and-answer data to fine-tune a large language model through segmented question-and-answer processing, and extracting intermediate inference steps generated by the large language model during the fine-tuning process, obtaining an inference trajectory based on the intermediate inference steps; and using the inference trajectory to analyze the model output of the large language model to obtain model output analysis results. Because this device embodiment uses structured question-and-answer data to fine-tune a large language model through segmented question-and-answer processing, extracts intermediate inference steps generated by the large language model during the fine-tuning process, obtains an inference trajectory based on the intermediate inference steps, and then uses the inference trajectory to analyze the model output of the large language model, compared to existing technologies, this device embodiment improves the interpretability and traceability of the large language model.
[0087] Based on the first embodiment of the processing device for interpretable reasoning trajectories of the language model described above, a second embodiment of the processing device for interpretable reasoning trajectories of the language model described above is proposed.
[0088] In this embodiment, the document segmentation processing module 501 is further configured to preprocess the target domain document to obtain a preprocessed target domain document; perform semantic boundary recognition on the preprocessed target domain document, determine the topic transition position between adjacent paragraphs based on the recognition result; and perform segmentation processing on the preprocessed target domain document based on the topic transition position to obtain multiple text units.
[0089] Other embodiments or specific implementations of the processing device for interpreting reasoning trajectories using the language model of the present invention can be found in the above-described method embodiments, and will not be repeated here.
[0090] This application provides a processing device for interpretable reasoning trajectories of a language model. The processing device for interpretable reasoning trajectories of a language model includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the processing method for interpretable reasoning trajectories of the language model in Embodiment 1 described above.
[0091] The following is for reference. Figure 5This document illustrates a schematic diagram of a processing device suitable for implementing the language model-interpretable inference trajectory in the embodiments of this application. The processing device for the language model-interpretable inference trajectory in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The processing device shown that can interpret reasoning trajectories using language models is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0092] like Figure 5 As shown, the processing device for interpretable reasoning trajectories of the language model may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory 1002 or a program loaded from storage device 1003 into random access memory 1004. Random access memory 1004 also stores various programs and data required for the operation of the processing device for interpretable reasoning trajectories of the language model. The processing unit 1001, read-only memory 1002, and random access memory 1004 are interconnected via bus 1005. Input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to input / output interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the processing device for interpretable inference trajectories of the language model to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows processing devices for interpretable inference trajectories of the language model with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.
[0093] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0094] The language model interpretable reasoning trajectory processing device provided in this application, employing the language model interpretable reasoning trajectory processing method described in the above embodiments, can solve the technical problem in the prior art where the intermediate reasoning process of a large language model in task processing is not visible, resulting in insufficient interpretability and traceability of the large language model. Compared with the prior art, the beneficial effects of the language model interpretable reasoning trajectory processing device provided in this application are the same as those of the language model interpretable reasoning trajectory processing method provided in the above embodiments, and other technical features in this language model interpretable reasoning trajectory processing device are the same as those disclosed in the previous embodiment method, and will not be repeated here.
[0095] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0096] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0097] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0098] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0100] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the protection scope of the present invention.
Claims
1. A method for processing interpretable reasoning trajectories of a language model, characterized in that, The method includes: The target domain document is segmented to obtain multiple text units; Structured question-and-answer data is constructed based on the text units, and the structured question-and-answer data includes instructions that guide the generation of reasoning steps; The structured question-answering data is used to fine-tune the large language model in a segmented question-answering manner, and the intermediate reasoning steps generated by the large language model are extracted during the fine-tuning process. The reasoning trajectory is obtained based on the intermediate reasoning steps. The inference trajectory is used to perform model output analysis of the large language model to obtain model output analysis results.
2. The method for processing interpretable reasoning trajectories based on a language model as described in claim 1, characterized in that, The step of segmenting the target domain document to obtain multiple text units includes: The target domain document is preprocessed to obtain a preprocessed target domain document. Semantic boundary recognition is performed on the preprocessed target domain document, and the thematic transition positions between adjacent paragraphs are determined based on the recognition results. The preprocessed target domain document is segmented according to the topic transition position to obtain multiple text units.
3. The method for processing interpretable reasoning trajectories based on a language model as described in claim 1, characterized in that, The step of constructing structured question-answering data based on the text units includes: Information is extracted from the text units to obtain key information, and a set of candidate questions is generated based on the key information; Target questions are selected from the candidate question set using relevance scoring, and corresponding reference answers are generated for the target questions; The instructions that guide the generation of reasoning steps are inserted between the target question and the reference answer in the form of a chain-like thinking template to generate structured question-and-answer data corresponding to the text unit.
4. The method for processing interpretable reasoning trajectories based on a language model as described in claim 1, characterized in that, The step of using the structured question-answering data to perform segmented question-answering fine-tuning on the large language model, extracting intermediate inference steps generated by the large language model during the fine-tuning process, and obtaining the inference trajectory based on the intermediate inference steps includes: Each text unit and its corresponding structured question-and-answer data is input into the large language model section by section; The large language model is guided to generate corresponding intermediate reasoning steps before generating the final answer, and to generate the final answer corresponding to the structured question-and-answer data after generating the intermediate reasoning steps. Based on the structured question-and-answer data and the corresponding final answers, the large language model is fine-tuned in a segmented question-and-answer manner; The timestamp information and token location index information of the intermediate reasoning steps are obtained, and the semantic correspondence between the timestamp information and the token location index information and the reference answer is aligned to obtain a reasoning trajectory with step numbers and logical dependencies.
5. The method for processing interpretable reasoning trajectories based on a language model as described in claim 4, characterized in that, After the step of obtaining the reasoning trajectory based on the intermediate reasoning steps, the method further includes: The text unit, the structured question-and-answer data, and the reasoning trajectory are associated to obtain the association result, and the association result is stored in the target database.
6. The method for processing interpretable reasoning trajectories based on a language model as described in claim 5, characterized in that, The step of using the inference trajectory to perform model output analysis of the large language model and obtaining the model output analysis results includes: The corresponding reasoning trajectory is retrieved from the target database based on the user's query information; An explanation of the decision-making process of the large language model is generated based on the inference trajectory, and the inference trajectory is visualized. The inference trajectory is compared with the final answer output by the corresponding model to obtain the verification result; When the verification result indicates the existence of a logical break, the text unit that caused the logical break and the corresponding structured question and answer data are located, and the location result is used as the model output analysis result.
7. The method for processing interpretable reasoning trajectories of a language model as described in claim 6, characterized in that, The method further includes: Based on the positioning results, the text units and / or the structured question-and-answer data are corrected to generate corrected data; The large language model is trained using the corrected data, and the model is optimized based on the training results.
8. A processing device for interpretable reasoning trajectories based on a language model, characterized in that, The device includes: The document segmentation and processing module is used to segment documents in the target domain to obtain multiple text units; A question-and-answer data construction module is used to construct structured question-and-answer data based on the text units, wherein the structured question-and-answer data includes instructions that guide the generation of reasoning steps; The reasoning trajectory extraction module is used to perform segmented question-and-answer fine-tuning on the large language model using the structured question-and-answer data, and extract the intermediate reasoning steps generated by the large language model during the fine-tuning process, and obtain the reasoning trajectory based on the intermediate reasoning steps; The reasoning trajectory utilization module is used to perform model output analysis of the large language model using the reasoning trajectory, and obtain model output analysis results.
9. A processing device for interpretable reasoning trajectories based on a language model, characterized in that, The device includes: a memory, a processor, and a language model interpretable reasoning trajectory processor stored in the memory and executable on the processor, the language model interpretable reasoning trajectory processor being configured to implement the steps of the language model interpretable reasoning trajectory processing method as described in any one of claims 1 to 7.
10. A storage medium, characterized in that, The storage medium stores a processing program for interpretable reasoning trajectories of a language model, which, when executed by a processor, implements the steps of the processing method for interpretable reasoning trajectories of a language model as described in any one of claims 1 to 7.