A vertical domain large model inference ability fine-tuning training method and system and a storage medium

Abstract

The application discloses a vertical domain large model inference ability fine-tuning training method and system and a storage medium, relates to the technical field of artificial intelligence and large language model fine-tuning, and comprises the following steps: acquiring inference data for language model inference, wherein the inference data comprises a structured subjective thinking chain sample, and the structured subjective thinking chain sample decomposes an inference process into inference steps; and controlling a language model based on the inference data, so that the language model generates an inference sequence comprising a step type mark and a subjective cognitive feature field and a decision conclusion when receiving an input question in a target field. The method disclosed by the application collects and structures expert implicit experience, combines multi-dimensional evaluation and confidence decay to filter high-quality data, cooperates with three-modal knowledge injection and semantic consistency fuse check, effectively suppresses knowledge illusion, and improves the rigor and reliability of professional inference of the vertical domain large model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and large language model fine-tuning technology, specifically to a method, system, and storage medium for fine-tuning training of reasoning ability of a large vertical model. Background Technology

[0002] With the increasing application of large language models in vertical fields such as healthcare, finance, industrial diagnostics, and law, how to endow these models with deep professional reasoning and complex decision-making capabilities similar to domain experts has become a research hotspot in the field of artificial intelligence. Traditional fine-tuning training methods typically input binary-aligned data of "question-standard answer" or "question-simple reasoning steps-answer" directly into the model. However, the professional decision-making process in vertical fields often relies heavily on the implicit experience accumulated by experts in long-term practice, such as intuitive inspiration, thought process backtracking, parallel verification of multiple hypotheses, and risk control. Existing training paradigms lack explicit extraction and structured modeling of the subjective thought process of experts, making it difficult for language models to reproduce the deep thinking logic of domain experts when facing complex and unknown situations. This easily leads to logical gaps or reasoning that remains superficial.

[0003] In the process of constructing and screening professional reasoning data, existing technologies also face severe challenges in data quality control. Reasoning data obtained from expert oral accounts or transcriptions of historical cases often contains a large number of colloquial expressions, information redundancy, and even logical flaws in individual steps. Traditional text cleaning and evaluation methods mostly rely on simple word overlap or overall semantic similarity indicators, which are difficult to accurately quantify the step-by-step logical rationality in a long chain of reasoning. Once a deviation occurs in a preceding reasoning step, the error will accumulate and amplify in the later stages of the reasoning chain. Existing data screening mechanisms lack effective perception of this inter-step error propagation mechanism, allowing samples with hidden logical defects to enter model training, directly reducing the effectiveness of model parameter updates and convergence stability.

[0004] Furthermore, in terms of model fine-tuning control and inference output, existing fine-tuning control mechanisms struggle to accurately control the inference structure generated by the model. The model is prone to generating lengthy and ineffective repetitive steps, or the pattern degenerates into a single type of mechanical deduction. At the same time, the "knowledge illusion" problem inherent in general large models is further amplified in professional vertical fields. When generating subjective inference sequences, the model frequently exhibits phenomena that deviate from real-world physical mechanisms or violate industry standards and norms. Existing output control lacks real-time semantic logic alignment and hard verification blocking mechanisms between the subjective inference path within the model and the external authoritative knowledge system. It cannot promptly circuit breakers non-compliant inference conclusions, which severely restricts the practical application of vertical domain large models in professional scenarios such as industry and medicine, where security requirements are extremely high. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by this invention is that existing vertical domain large model fine-tuning training methods have insufficient depth of model reasoning thinking, poor model convergence stability, and a single and degenerate reasoning mode, as well as the problem of how to perform real-time semantic alignment during model reasoning.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a method for fine-tuning the reasoning ability of a large-scale vertical domain model, comprising: acquiring unstructured reasoning data generated by experts when reasoning about professional tasks in a target domain; performing structured encoding on the unstructured reasoning data to generate structured subjective thought chain samples, wherein the structured subjective thought chain samples contain at least step type markers and subjective cognitive feature fields; the step type markers are used to indicate the cognitive stage type to which the current reasoning step belongs; the subjective cognitive feature fields are used to express at least one of the heuristic markers, confidence values, and alternative solution information used by the expert during the reasoning process; and fine-tuning the language model based on the structured subjective thought chain samples, so that when the language model receives an input question from the target domain, it generates a reasoning sequence and a decision conclusion containing step type markers and subjective cognitive feature fields.

[0008] As a preferred embodiment of the vertical domain large model reasoning ability fine-tuning training method described in this invention, the method for obtaining structured subjective thought chain samples includes obtaining unstructured expert reasoning texts through thought expression, structured interviews, historical case retrospection, and multi-subject collaborative reasoning; extracting heuristic strategy markers and alternative probability hypotheses from thought expression and structured interviews, extracting logical retrospection correction markers and interaction divergence markers from historical case retrospection and multi-subject collaborative reasoning, and normalizing and encapsulating them into the subjective cognitive feature fields.

[0009] As a preferred embodiment of the vertical domain large model reasoning ability fine-tuning training method described in this invention, the following steps are included before fine-tuning the language model using structured subjective thought chain samples: quality screening of the structured subjective thought chain samples; initial confidence of each reasoning step in the structured subjective thought chain samples is calculated using evaluation indicators; it is determined whether the initial confidence of the current reasoning step is lower than a preset confidence threshold; if it is lower, the transit confidence of subsequent reasoning steps is calculated by attenuating it according to a preset attenuation factor, and the overall confidence of the sample is calculated based on the transit confidence; structured subjective thought chain samples with an overall confidence of lower than a preset qualified threshold are removed from the reasoning data.

[0010] As a preferred embodiment of the vertical domain large model reasoning ability fine-tuning training method described in this invention, the fine-tuning training of the language model based on structured subjective thought chain samples includes: obtaining the number of causal reasoning layers and the number of associated knowledge items contained in the structured subjective thought chain samples; classifying the structured subjective thought chain samples into different difficulty levels according to the number of causal reasoning layers and the number of associated knowledge items; and controlling the language model to sequentially input the corresponding category of structured subjective thought chain samples according to the difficulty level from low to high for phased parameter updates.

[0011] As a preferred embodiment of the vertical domain large model reasoning ability fine-tuning training method described in this invention, the fine-tuning training of the language model based on structured subjective thought chain samples also includes supervised fine-tuning. When controlling the language model to calculate the loss function, a step length constraint term and a step diversity constraint term are introduced. The step length constraint term calculates the penalty loss based on the difference between the number of generated reasoning steps and the preset number of steps range. The step diversity constraint term calculates the penalty loss based on the distribution negative entropy of each step type label in the generated reasoning steps, and combines it with the basic prediction loss to constrain the parameters of the language model.

[0012] As a preferred embodiment of the vertical domain large model reasoning ability fine-tuning training method described in this invention, it further includes: preference optimization, which combines structured subjective thought chain samples of different qualities into a preference dataset containing preference sequences and non-preference sequences based on the score difference; using the updated language model as a policy model based on the preference dataset for preference alignment training, and calculating the relative entropy of the probability distribution between the policy model and the preset reference model in real time during the training process; terminating preference alignment training when the relative entropy exceeds a preset upper limit threshold or the preference loss value does not decrease for a preset number of consecutive steps.

[0013] As a preferred embodiment of the vertical domain large model reasoning ability fine-tuning training method described in this invention, it further includes: trimodal knowledge enhancement control, binding explicit knowledge entity reference tags in structured subjective thought chain samples; during language model training, controlling the intermediate layer representation distribution of the language model to fit the representation distribution of the preset knowledge teacher model; during the process of the language model receiving input questions and generating reasoning sequences, retrieving complementary knowledge entries from the external knowledge base based on the semantic features of the current reasoning step and integrating them into subsequent reasoning steps.

[0014] As a preferred embodiment of the vertical domain large model reasoning ability fine-tuning training method described in this invention, it further includes: semantic consistency verification, extracting subjective deduction path conclusions and external knowledge association path conclusions from the reasoning sequence; calling the semantic logic verification model to determine the implication and contradiction relationships between subjective deduction path conclusions and external knowledge association path conclusions, and calculating the semantic consistency rate; when the semantic consistency rate reaches a preset consistency threshold, outputting a decision conclusion; when the semantic consistency rate is lower than the preset consistency threshold, triggering an alarm or blocking the output of the decision conclusion.

[0015] Another objective of this invention is to provide a vertical domain large model reasoning ability fine-tuning training system, which can acquire reasoning data for language model reasoning. The reasoning data includes structured subjective thought chain samples, which deconstruct the reasoning process into reasoning steps, thus solving the problem of insufficient depth of model reasoning thinking in current vertical domain large model fine-tuning training methods.

[0016] As a preferred embodiment of the vertical domain large model reasoning ability fine-tuning training system described in this invention, it includes: a data acquisition and encoding module, a data evaluation and screening module, a model training and optimization module, and an application service and knowledge verification module; the data acquisition and encoding module is used to acquire unstructured reasoning text, parse and normalize it into structured subjective thought chain samples, the structured subjective thought chain samples contain multiple step units arranged in reasoning order, each step unit is explicitly associated with a step type label, reasoning content, and subjective cognitive feature fields used to express expert decision-making experience; the data evaluation and screening module is used to quantitatively score the structured subjective thought chain samples using multi-dimensional evaluation indicators, perform confidence decay propagation calculation, and classify the screened samples into different difficulty levels according to the number of causal reasoning layers and the number of associated knowledge items contained in the samples; the model training and optimization module is used for The subjective thought chain dataset is input into the target language model in ascending order of difficulty for phased parameter updates. Step length and step diversity constraints are introduced to penalize deviations in the number of reasoning steps and uniformity in step type distribution. A preference dataset is constructed based on the filtered dataset, and the updated language model is used as the policy model for preference alignment training. The application service and knowledge verification module is used to inject trimodal knowledge when the vertical domain reasoning model receives input questions from the target domain. It jointly executes explicit knowledge citation, implicit knowledge distillation, and inference-time knowledge retrieval to guide the vertical domain reasoning model to generate a reasoning sequence. Before outputting the decision conclusion, the subjective derivation path conclusion and the external knowledge association path conclusion in the reasoning sequence are extracted. The semantic logic verification model is called to calculate the semantic consistency rate between the two, and a professional decision conclusion is output or an alarm prompt or output blocking is triggered.

[0017] Another object of the present invention is to provide a vertical domain large model inference capability fine-tuning training storage medium, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the vertical domain large model inference capability fine-tuning training method.

[0018] The beneficial effects of this invention are: The vertical domain large-scale model reasoning ability fine-tuning training method provided by this invention acquires unstructured text and extracts heuristic strategies, alternative hypotheses, logical backtracking, and interaction divergence normalization, encapsulating them into subjective cognitive feature fields. This achieves explicit and structured modeling of domain experts' deep intuitive judgments and implicit decision-making experience, improving the professionalism and depth of thinking in the model's reasoning sequence. It calculates the initial confidence of each step through evaluation metrics, and calculates the confidence of subsequent steps based on a preset decay factor when the initial confidence is lower than a preset threshold. This eliminates samples with overall confidence below the acceptable threshold, effectively curbing the accumulation and amplification of local logical flaws in later stages, ensuring the high purity and logical rigor of the fine-tuning training dataset. Based on the number of causal reasoning layers and the number of associated knowledge items, samples are divided into different difficulty levels. The model is controlled to input corresponding samples in stages according to difficulty from low to high for phased parameter updates, establishing a progressive capability ladder, significantly improving the convergence efficiency and stability of large-scale model fine-tuning training. The loss function introduces a step length constraint term calculated based on the deviation difference of the number of steps, and a step multiplication factor calculated based on the negative entropy of the step type distribution. Diversity constraints precisely control the generated reasoning structure, effectively preventing the phenomena of lengthy sequences or degenerate step types. A dataset containing both preferred and non-preferred sequences is constructed to control the training of the strategy model. Training is terminated when the relative entropy calculated in real-time exceeds the upper limit threshold or the preference loss fails to decrease continuously. This provides reliable early stopping protection while achieving efficient preference alignment, avoiding mode collapse of the strategy model. A three-modal knowledge enhancement control is employed, which involves binding explicit knowledge entity reference tags to samples, controlling the distribution of intermediate layer representations and fitting the knowledge teacher model during training, and retrieving external knowledge based on the semantic features of steps during reasoning and integrating it into subsequent steps. This achieves deep integration of explicit, implicit, and dynamically retrieved knowledge, internalizing authoritative domain knowledge into model parameters and the derivation context. Subjective derivation path conclusions and external knowledge association path conclusions are extracted from the reasoning sequence. A semantic logic verification model is called to calculate the semantic consistency rate between the two, and an alarm or output blocking is triggered when the consistency rate is lower than a threshold. This establishes a rigid semantic logic circuit breaker mechanism, fundamentally curbing the generation of illusions and greatly ensuring the security and traceability of professional decision-making conclusions. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an overall flowchart of the vertical domain large model reasoning ability fine-tuning training method in this invention.

[0021] Figure 2 This is a flowchart illustrating the subjective thought chain data construction and quality assessment process for the vertical domain large model reasoning ability fine-tuning training method in this invention.

[0022] Figure 3 This is a two-stage fine-tuning training framework diagram for the vertical domain large model reasoning ability fine-tuning training method in this invention.

[0023] Figure 4 This is a schematic diagram illustrating an application scenario of the vertical domain large model inference capability fine-tuning training method in the present invention for industrial equipment fault diagnosis.

[0024] Figure 5 This is a schematic diagram of the knowledge enhancement thinking chain injection mechanism in the vertical domain large model reasoning ability fine-tuning training method of the present invention.

[0025] Figure 6 This is a schematic diagram of the overall system for fine-tuning training of vertical domain large model reasoning ability in the present invention. Detailed Implementation

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0027] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0028] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0029] Example 1

[0030] Reference Figure 1 This is the first embodiment of the present invention, which provides a method for fine-tuning the reasoning ability of a large vertical model.

[0031] Specifically, the first step is to acquire unstructured reasoning data generated by experts when reasoning about professional tasks in the target domain. In the actual collection and acquisition process, the target domain can be a specific vertical professional field. Unstructured reasoning data includes handwritten notes, transcripts of voice diagnosis, mind maps, historical judgment logs, on-site analysis reports, or expert oral recordings left by experts when answering or handling professional questions in the target domain. These are original thought trajectory texts that are not subject to standardized formats and uniform grammatical constraints, and truly reflect the experts' leaps in thinking, experience-based insights, and uncertain judgments when solving complex problems.

[0032] Unstructured reasoning data is structured and semantically mapped to generate a structured subjective thought chain sample 101. During the encoding process, the system utilizes text parsing, regular expression extraction techniques, and large language model-assisted annotation tools, combined with the target domain's knowledge graph and professional dictionary, to segment the unstructured text according to the thought evolution sequence, identify entities, and extract relationships, transforming it into a data structure with strict nested relationships and a standard grammatical format. The structured subjective thought chain sample 101 includes at least a step type marker 1011 and a subjective cognitive feature field 1012. The step type marker 1011 indicates the cognitive stage type of the current reasoning step, including but not limited to the fact extraction stage, hypothesis formulation stage, logical deduction stage, evidence verification stage, and decision output stage. In actual encoding, specific predefined structured marker symbols (e.g., using...) are inserted at the beginning of each stage of the text sequence.<Type:Fact_Extraction> Mark the fact extraction stage and adopt<Type:Hypothesis_Generation> Mark the hypothesis proposal stage, adopt <type:verification>The model explicitly indicates the specific cognitive behavior performed in the current step by marking evidence verification stages, etc. The subjective cognitive feature field 1012 is used to express at least one of the heuristic markings, confidence values, and alternative solution information used by experts in the reasoning process. Among them, heuristic markings are used to record the non-deterministic quick inference rules or intuitive experiences summarized by experts based on long-term industry experience (such as "when indicator A and indicator B are abnormal at the same time, a certain cause should be given priority"). Confidence values ​​are used to quantify the degree of certainty of experts regarding the credibility of the current reasoning step, intermediate reasoning results, or data by numerical values ​​or predefined levels (such as [Confidence: 0.85] or [Confidence: High]). Alternative solution information records the alternative diagnostic paths retained by experts when facing multiple possibilities or uncertainties, the logic for excluding alternative solutions, and the reasons for elimination (such as "alternative solution B is postponed due to lack of key verification evidence"). These features are seamlessly embedded into each reasoning step through structured key-value pairs, forming high-density thought chain data.

[0033] The pre-trained language model is fine-tuned based on structured subjective thought chain samples 101. In the specific training process, the input question from the target domain is used as the input sequence, and the structured subjective thought chain samples 101, containing corresponding step type labels 1011 and subjective cognitive feature fields 1012, are used as the target output sequence to construct a high-quality supervised fine-tuning training dataset 301. This training dataset is then input into the pre-trained language model. The difference between the generated sequence and the target sequence is calculated using the cross-entropy loss function, and the weight parameters of the language model are updated based on the backpropagation algorithm to adjust the conditional probability distribution of the model predicting the next character or label in a given context. During training, parameters are iteratively optimized batch by batch until the model's loss function converges and the prediction accuracy reaches a preset threshold. This allows the trained language model to not only master domain knowledge but also deeply learn to imitate the implicit cognitive logic and heuristic thinking paths of professional experts when solving complex tasks. As a result, when the language model receives unknown input questions from the target domain in subsequent practical application scenarios, it can follow the thinking framework obtained during training to automatically generate a complete reasoning sequence 402 containing step type labels 1011 and subjective cognitive feature fields 1012, as well as the final credible decision conclusion. This achieves accurate replication, explicit expression, and high-quality output of the implicit reasoning experience of experts.

[0034] Example 2

[0035] Reference Figure 1 and Figure 2 This is the second embodiment of the present invention. Unlike the previous embodiment, this embodiment provides a method for explicitly collecting and structurally encoding the implicit reasoning experience of experts, thus solving the problem of lacking explicit extraction and structured modeling of the subjective thinking links of experts.

[0036] Specifically, the process involves collecting expert reasoning data using four progressive methods to obtain unstructured expert reasoning texts. This forms a progressive data collection process from "individual independent reasoning" to "collaborative reasoning." First, individual experts' independent reasoning processes are collected through verbal thinking, with each expert completing the process independently to ensure diverse individual reasoning paths are captured. Second, structured interviews delve into the implicit knowledge and intuitive judgments that experts may not have fully expressed in verbal thinking. Third, case studies focus on key decision points and reflections in real historical cases, supplementing complex reasoning scenarios that may have been missed in verbal thinking and interviews. Finally, collaborative reasoning records the knowledge transfer and judgment convergence processes in multi-expert collaborative scenarios. The data collected using these four methods are then aggregated and processed in the second stage (unstructured data processing and structured coding). The advantage of this progressive data collection design is that verbal thinking acquires the basic "explicit reasoning process," structured interviews supplement "implicit knowledge elements," case studies focus on "key decision moments," and collaborative reasoning captures "knowledge transfer mechanisms." These four methods complement each other to provide a complete coverage of the expert reasoning process.

[0037] Think-Aloud Protocol invites at least three domain experts to independently complete the same task, recording and transcribing their verbal reasoning process throughout. Extract the complete reasoning content of the five cognitive links of "target identification - hypothesis generation - evidence collection - hypothesis verification - conclusion formation". Each expert completes the same task independently to ensure that the diverse reasoning paths of different experts on the same problem are collected; the structured interview (Cognitive Task Analysis, CTA) designs a five-stage interview outline, including task overview, knowledge requirements, situation awareness, decision inspiration, and error identification. This method is particularly suitable for collecting tacit knowledge - that is, intuitive judgments that experts find difficult to express directly. In the interview, hypothetical questions guide experts to disclose their decision-making basis in different situations. Each expert conducts 2-3 in-depth interviews (1.5-2 hours each time); the case retrospective (Protocol Analysis, PA) selects real historical cases (such as typical failure cases in the past 5 years in the industrial field) and invites experts to review their reasoning process, focusing on collecting three key points: (1) decision point - the moment when the expert makes a choice among multiple possible solutions, (2) divergence point - the moment when the reasoning path forks and multiple hypotheses need to be evaluated in parallel, (3) reflection point - the moment when the expert doubts the previous judgment and actively retrospectively corrects it; collaborative reasoning (Cognitive Apprenticeship, Cognitive Apprenticeship (CA) invites multiple experts to collaborate on complex tasks, recording the processes of "knowledge transfer" (expert A explaining a judgment basis to expert B) and "judgment convergence" (how multiple experts reach a consensus on different views of the same issue).

[0038] In the 1012 stage of extracting features from unstructured text and normalizing and encapsulating them into the subjective cognitive feature field, the unstructured text first undergoes preprocessing to remove sensitivity. Regular expressions + LLM are used to desensitize the text, replacing sensitive information such as device number and operator name with code. Secondly, the format is normalized, and the inference chain format is unified as a six-tuple of (number, step type, inference content, subjective component, knowledge reference, confidence level). The steps include six standard types: target identification, hypothesis generation, evidence collection, hypothesis verification, conclusion formation, and reflection. Then, duplicate detection and deduplication are performed using SimHash (a locality-sensitive hashing algorithm that maps similar text to similar hash values), removing duplicate samples with a similarity greater than 0.95. Next, the difficulty is automatically graded (difficulty 1-5) based on "reasoning depth" and "number of knowledge entries involved." The grading is based on a combined score of "reasoning depth" and "number of knowledge entries involved." Reasoning depth is defined as the number of causal reasoning layers in the reasoning chain (the number of reasoning levels from observation to root cause), and the number of knowledge entries is defined as the number of domain knowledge entries required to complete the reasoning.

[0039] The specific grading standards are as follows: Difficulty 1 (Simple): Reasoning depth 1-2 layers, involving ≤3 knowledge items, a single symptom directly corresponds to the fault (e.g., "vibration exceeding limits → poor dynamic balance"), serving as basic reasoning ability training for the model, ensuring the model grasps the most direct causal mapping; Difficulty 2 (Moderately Simple): Reasoning depth 2-3 layers, involving 4-6 knowledge items, requiring comprehensive analysis based on 2-3 symptoms, cultivating the model's ability to comprehensively judge multiple signals; Difficulty 3 (Medium): Reasoning depth 3-4 layers, involving 7-10 knowledge items, requiring the elimination of multiple possible fault causes, training the model's fault diagnosis and multi-hypothesis reasoning abilities; Difficulty 4 (Moderately Difficult): Reasoning depth 4-5 layers, involving 11-15 knowledge items, complex fault localization across systems, improving the model's ability to handle complex faults across systems; Difficulty 5 (Difficult): Reasoning depth ≥5 layers, involving ≥16 knowledge items, novel and rare faults, requiring creative reasoning, challenging the model's extreme reasoning ability, improving the model's performance in boundary conditions. This five-level classification method, compared to simple binary (easy / hard) or three-level (easy / medium / hard) classification, offers a finer granularity that allows the training process to proceed with a progressive strategy from easy to difficult, avoiding instability caused by the model jumping directly from easy to difficult tasks. The five-level system directly corresponds to the progressive sample strategy of SFT and DPO training (adding difficulty 1→5 step by step), achieving a smooth increase in difficulty. The five-level system also facilitates subsequent evaluation by stratifying accuracy according to difficulty level, accurately identifying at which difficulty level the model performs poorly.

[0040] Furthermore, structured coding was performed using a two-stage pipeline based on LLM-assisted coding and expert review. First, GPT-4 was used to provide structured prompts and guidance to the transcribed text, extracting fields such as reasoning steps, subjective components, knowledge references, and confidence levels. Then, two coders independently reviewed the transcription, with disputed parts decided by a third senior expert. The Cohen's Kappa score for coding consistency reached 0.78.

[0041] The transcribed text was parsed into SCoT structured fields. First, "text preprocessing" was performed, which involved segmenting the transcribed text of expert oral reasoning into sentences (using sentence boundary markers such as periods, question marks, and exclamation marks) to break long paragraphs into individual reasoning sentence units (average 15-30 words per sentence). Then, "reasoning step type labeling" was performed, using GPT-4 to classify the reasoning step type of each sentence unit. The prompt template was "Please determine which step type the following reasoning statement belongs to (target identification / hypothesis generation / evidence collection / hypothesis verification / conclusion formation / reflection), and give the confidence level." Each step type corresponds to specific language pattern features—for example, "the possible reason is..." corresponds to hypothesis generation, "based on... data" corresponds to evidence collection, and "therefore it can be determined" corresponds to conclusion formation, etc. Next, "subjective component extraction" was performed. For the classified reasoning statements, subjective component fields were further extracted: (a) experience pattern—using five predefined heuristic labels (anchoring and adjustment, representativeness heuristic, availability heuristic, target-means analysis, work orientation). (a) Matching – matching is performed before the search, and unmatched samples are manually labeled by the coders; (b) Confidence – extracting the degree of certainty expressed in the statement (such as keywords like "high probability" / "possible" / "uncertainty" mapped to confidence values ​​of 0-1); (c) Alternatives – identifying whether the reasoning statement contains expressions such as "or", "alternative", "another possibility", etc., and extracting alternative hypotheses; then, "knowledge reference extraction" is performed again, using domain dictionary matching + LLM to help identify domain professional terms and knowledge entries referenced in the reasoning statement, and automatically associating them with the corresponding entities and relational identifiers in the domain knowledge graph; finally, "expert review and correction" is performed, and the above automatic extraction results are independently reviewed by two coders. If the coders and experts disagree, a "three-level adjudication mechanism" is adopted. First, the two coders negotiate. If they disagree, the sample is submitted to a third senior expert for adjudication. If the senior expert still cannot determine the result, the sample is marked as "disputed" and temporarily shelved. In subsequent data accumulation, cross-validation is performed using more similar samples. Finally, the Cohen's coding consistency is determined. A Kappa score of 0.78 (above the acceptable threshold of 0.7) indicates good consistency in the encoding results.

[0042] The core fields of the SCoT data structure include: Top-level fields: Unique identifier (id), domain, task type, difficulty level (difficulty 1-5), problem description, context data, subjective inference chain list, final answer, and quality metadata; Inference step fields: Step identifier (step_id), Step type, Reasoning content, Subjective factors (experience_pattern, heuristic, confidence[0,1], alternatives), Knowledge refs, and Expert identifier (expert_id); Quality metadata: Expert consistency (Kappa coefficient), Reasoning depth (reasoning_depth 1-5), and Knowledge coverage. 0-1); Heuristic coding tags: five categories of standard tags, including anchoring and adjustment, representativeness heuristic, availability heuristic, goal-means analysis, and work-forward search.

[0043] Furthermore, the Knowledge Enhancement Thinking Chain (KA-SCoT) is extended by adding three knowledge-related fields in each SCoT reasoning step: (1) explicit knowledge refs – referencing entity or relation identifiers in the domain knowledge graph; (2) implicit kd source – recording the knowledge distillation source for this step; and (3) inference retrieval query – recording the retrieval query that can be used during reasoning.

[0044] SCoT Sample Size: 5000-50000 samples per domain are recommended, with an average of 5-15 inference steps per sample. The recommended range of "5000-50000 samples per domain" is based on the following considerations: The lower limit of 5000 samples reflects the data requirements for fine-tuning deep learning models. For large vertical domain models (taking 7-byte parameters as an example), SFT fine-tuning requires at least several thousand high-quality training data points to effectively learn domain inference patterns. 5000 samples is an empirically estimated minimum data volume for achieving stable convergence—training curves show that when the data volume is below 3000 samples, the validation loss oscillates in the later stages of training (fluctuation amplitude > 0.15). The initial convergence is unstable; after reaching 5000 samples, the validation loss tends to smooth out (fluctuation amplitude <0.05). Each sample averages 5-15 inference steps, and 5000 samples can provide training signals for approximately 25000-75000 inference steps, which is sufficient to cover the core inference patterns in the domain. The upper limit of 50000 samples is based on the law of diminishing marginal returns; when the data volume exceeds 50000 samples, the accuracy improvement brought by each additional 10,000 samples is less than 0.5 percentage points (experiments show that the improvement from 5000 to 10000 samples is approximately...). The accuracy improvement is 4.2 percentage points, with an increase of approximately 2.1 percentage points from 10,000 to 20,000 data points, and approximately 1.3 percentage points from 30,000 to 50,000 data points. However, the improvement is less than 0.5 percentage points after exceeding 50,000 data points. Furthermore, the cost of data collection and encoding increases linearly with the amount of data, and the return on investment decreases significantly after exceeding 50,000 data points. Compared with other settings, if a smaller scale (e.g., 1,000-3,000 data points) is used, the inference accuracy decreases significantly (approximately 8-12 percentage points), and the diversity of the inference chain is insufficient, making the model prone to overfitting to limited data. If a larger scale (e.g., more than 100,000 data points) is used, the data collection cost surges (estimated at approximately 500-800 people), but the accuracy improvement is limited (<2 percentage points), and the risk of low-quality data contamination increases. A large amount of data means that it is difficult to conduct rigorous manual quality review. A data scale of 5,000-50,000 data points achieves the optimal balance between inference accuracy, data cost, and quality controllability, making the method of this invention both high-performance and engineering-applicable.

[0045] Example 3

[0046] Reference Figure 1 and Figure 2 This is the third embodiment of the present invention. Unlike the previous embodiment, this embodiment provides a method for multi-dimensional quality assessment and confidence decay screening of structured subjective thought chain samples 101, which solves the problem that the data screening mechanism lacks effective perception of the inter-step error propagation mechanism, resulting in samples with implicit logical defects entering the training.

[0047] Specifically, in the stage of quality screening and confidence decay calculation of the 101 structured subjective thought chain samples, the SCoT data was first systematically evaluated using the "four-dimensional twelve-indicator" system, which includes professionalism, logic, interpretability and consistency.

[0048] Professionalism includes three indicators: E1 Mechanism correctness (0-1 points, expert scoring); E2 Accuracy of domain knowledge citation (0-1 points, knowledge graph matching + expert review); E3 Standardization of professional terminology (0-1 points, terminology dictionary matching + expert sampling review); Logicality includes three indicators: L1 Coherence of reasoning steps (0-1 points, LLM scoring); L2 Completeness of reasoning chain (0-1 points, step coverage calculation); L3 Reasonableness of reasoning branches (0-1 points, branch coverage and depth comprehensive scoring); Explainability includes three indicators: I1 Clarity of reasoning intent (1-5 points, LLM scoring); I2 Sufficiency of evidence citation (0-1 points, evidence density calculation); I3 Decision traceability (0-1 points, reverse tracing success rate); Consistency includes three indicators: C1 Internal consistency of reasoning chain (0-1 points, contradiction detection); C2 Inter-expert consistency (0-1 points, Kappa coefficient ≥ 0.7 is acceptable); C3 Consistency with domain consensus (0-1 points, comparison with authoritative literature / standards).

[0049] The system employs twelve quantitative indicators across four dimensions, utilizing three LLMs as evaluators. A dedicated prompt template is designed for each indicator. Bias is eliminated through a combination of "position swapping and majority voting," with the average of the two evaluation swaps for each pair of samples taken. Multiple LLMs are used for voting. A referee model different from the target model is employed, along with at least 10% of the data undergoing manual review (requiring Cohen's Kappa ≥ 0.7). The initial confidence level for each inference step in the sample is calculated. A confidence decay propagation mechanism is introduced: it determines whether the initial confidence level of the current inference step is lower than a preset confidence threshold (the first preset threshold is set to 0.6). If the initial confidence level of the current inference step is lower than 0.6, the propagated confidence level of subsequent inference steps is calculated by decaying the initial confidence level by a preset decay factor of 0.8. Subsequently, the system compares the calculated overall confidence score of the sample with a preset qualified threshold (the second preset threshold is set to 0.5): if the overall confidence score of the sample is lower than 0.5, the sample is identified as a low-quality sample and removed from the training set (returned for relabeling or discarded); only samples with a total quality assessment score of 0.8, a score of 0.7 in each dimension, and an overall confidence score of 0.5 are retained in the high-quality training set. The original unstructured data is processed to generate 5558 structured samples, and after quality screening, 3612 high-quality samples are retained (the screening retention rate is 65%). Further data augmentation strategies such as rewriting, counterfactual augmentation, and cross-device migration are combined to finally expand to 8200 high-quality training samples.

[0050] Example 4

[0051] Reference Figure 1 and Figure 3 This is the fourth embodiment of the present invention. Unlike the previous embodiment, this embodiment provides a way to introduce constraints in the supervision and fine-tuning 301 stage to solve the problem that the fine-tuning control mechanism is difficult to accurately control the deductive deconstruction, and is prone to generating lengthy and invalid steps or degenerating into a single type of mechanical deduction.

[0052] Specifically, based on the inference data-controlled language model, a supervised fine-tuning 301 training phase is performed. Using a high-quality SCoT dataset that has been evaluated and selected, the model learns an end-to-end mapping from "input question → SCoT inference chain → final output." The loss function consists of three parts: the main loss term, the inference chain length regularization term, and the inference pattern diversity regularization term. The negative log-likelihood of the main loss term, and the supervised model generating the correct inference steps and final answer, are expressed as follows: , in, For the first The inference chain of the sample, For the final answer, For input questions, For model parameters, This represents the total number of training samples.

[0053] The regularization term for the reasoning chain length, which constrains the length of the reasoning chain to a reasonable range, is expressed as: , in, For the regularization term of the thought chain length, For the first The actual number of steps in the inference chain of each sample. The maximum allowed number of steps is set to 20. The minimum allowed number of steps is set to 3. A positive penalty is applied when the reasoning chain exceeds 20 steps or is shorter than 3 steps.

[0054] Reasoning pattern diversity regularization term Encourage the use of diverse step types, defined as six reasoning pattern distributions. negative entropy When a model uses only a few types of steps, the entropy value is low, the negative entropy is high, and a positive penalty is generated.

[0055] Diversity regularization term The expansion is defined as: , , in, For the first The frequency of occurrence of various reasoning patterns in the reasoning chain (6 in total: target identification, hypothesis generation, evidence collection, hypothesis testing, conclusion formation, and reflection). The information entropy is distributed across six reasoning modes. When the model uses only a few step types, the entropy value is low, the negative entropy is high, and a positive penalty is generated.

[0056] The loss function is specifically expressed as follows: , in, The total SFT loss is obtained by weighted summation of the three sub-items. The total number of training samples, Given the input problem and model parameters Below, the model generates the inference chain. and the final answer The conditional probability, This is the length regularization weight coefficient, set to 0.1, which controls the strength of the inference chain length penalty term. For the first The inference chain length regularization term for each sample penalizes inference chains that are too long or too short. This is the diversity regularization weight coefficient, set to 0.05, which controls the strength of the reasoning pattern diversity penalty term. For the first The regularization term for the diversity of reasoning patterns in a sample penalizes reasoning patterns that are too uniform.

[0057] Furthermore, key technical parameters are configured: Efficient parameter fine-tuning is performed using the QLoRA (Quantized Low-Rank Adaptation) scheme—the base model is loaded using NF4 (4-bit Normal Float) quantization format, with LoRA (Low-Rank Adaptation) rank r=64, scaling factor α=128, dropout=0.05, and the target module is... Only about 0.5% of the parameters are trained.

[0058] Optimizer: AdamW (Adam with Weight Decay, an adaptive moment estimation optimizer with weight decay) Peak learning rate The first 5% steps are linear warm-up, followed by cosine annealing. .

[0059] Batch strategy: Equivalent batch size 32, training for 3 epochs, loss mask: loss is calculated only for the inference chain + final answer, and the problem part uses a mask of -100.

[0060] Hardware configuration: 4 NVIDIA A100-80G GPUs (Graphics Processing Units), using the ZeRO-2 (Zero Redundancy Optimizer Stage 2) deep learning training optimization library developed by Microsoft for distributed training. The training time is about 18 seconds per step and the total training time is about 72 hours.

[0061] Base model: Qwen2.5-7B-Instruct (natively supports 32K context windows), alternative models include Llama-3-8B-Instruct and ChatGLM3-6B.

[0062] SFT training results: The training loss decreased from the initial 2.85 to 0.62, and the validation loss decreased from 2.92 to 0.71 (optimal epoch=2.5). After training, the model's validation set achieved a "final answer accuracy" of 78.4%, a "reasoning step accuracy" of 72.6%, and a "reasoning interpretability" of 0.74.

[0063] Example 5

[0064] Reference Figures 1-4 This is the fifth embodiment of the present invention. Unlike the previous embodiment, this embodiment provides a method for training preference alignment based on preference dataset 3021 and preventing policy model mode collapse through KL divergence monitoring and early stopping mechanism. This solves the problem that the preference optimization 302 method lacks effective training stability control, causing the model to deviate from the preset distribution.

[0065] Specifically, in the preference optimization 302 and preference dataset construction 3021 stages, three complementary methods are adopted: Method A (automatic construction), Method B (expert annotation), and Method C (rule-based construction). Method A (automatic construction) uses the SFT model to generate multiple SCoT inference chains at different temperatures (0.5 / 0.7 / 0.9), and scores them using a multidimensional evaluation system. Pairs with a score difference > 0.5 are considered obvious preference pairs (1500 pairs), and those with a score difference between 0.3 and 0.5 are considered subtle preference pairs (1200 pairs). Method B (expert annotation) invites at least 3 domain experts to conduct pairwise comparisons of the inference chains, requiring a Kappa ≥ 0.7 for acceptance (500 pairs). Method C (rule-based construction) automatically filters based on rules such as "includes hypothesis verification steps," "inference depth ≥ 3," and "includes explicit knowledge references 4011."

[0066] First, parallel construction was performed, with three methods running independently and synchronously. Method A involved automatic batch generation and scoring on a GPU cluster (approximately 4 hours). Method B involved independent annotation by a team of domain experts (3 experts annotating separately, approximately 3 working days). Method C involved filtering existing data according to preset rules (approximately 30 minutes). Second, independent quality reviews were conducted. Preference pairs built using the three methods underwent internal quality reviews. The automated construction results of Method A required manual review (sampling rate ≥ 5%) to confirm the reasonableness of the scoring. The expert annotation results of Method B required Kappa coefficient calculation to ensure consistency (≥ 0.7 for adoption). The rule-based construction results of Method C were reviewed and filtered by a senior expert. Finally, the results from the three methods were aggregated and merged. After the obtained preference pairs are aggregated, duplicate pairs (the same comparison combination for the same question) are removed and merged into a unified DPO training dataset. If different preference pairs are generated for the same question under different methods, the final preference is determined by the "majority consensus" principle (i.e., the conclusions of at least two methods are consistent). This "parallel independent construction + aggregation fusion" strategy automatically constructs a wide-coverage dataset (can be generated in large batches), has the highest quality expert annotation (but is costly and has a small sample size), and has high efficiency in rule-based construction (suitable for scenarios with clear quality standards). The three complement each other to form the optimal combination of "quantity + quality + efficiency". Parallel execution shortens the construction cycle (reducing the time consumption by about 60% compared to performing it one by one); independent review and fusion avoid the systematic bias of a single construction method.

[0067] Furthermore, a multi-dimensional evaluation system was used to define scoring indicators and set scoring standards for the SCoT inference chain. For twelve indicators across four dimensions, each indicator had a clearly defined scoring standard and anchoring example. Taking indicator E1 (mechanism correctness) as an example: 1 point = completely conforms to the domain mechanism and the reasoning is flawless; 0.8 points = basically conforms but with minor deviations; 0.6 points = partially conforms but with significant deviations; 0.4 points = multiple errors exist; 0 points = completely wrong. LLM automatic scoring was performed. A dedicated Prompt template was used to call the LLM for evaluation of each indicator. Taking E1 as an example, the Prompt structure was: "You are a {domain} expert. Please evaluate the mechanism correctness of each step in the following inference chain. Inference chain: {SCoT content}. Please score according to the following criteria (0-1 points): ... Please output the score and reasoning." To eliminate LLM bias, each indicator was evaluated three times independently (using different LLMs), and the median was taken as the original score for that indicator. Then, dimensional aggregation was performed, and the weighted sum of the scores of the three indicators in each dimension was used to obtain the original score. Dimensional scores (weights can be adjusted according to domain characteristics; the default equal weight is 0.33:0.33:0.33), and the four dimension scores are then weighted and summed to obtain the total score (the weighted configuration of professionalism 0.3, logicality 0.3, interpretability 0.2, and consistency 0.2 achieved the best results in the experiment). Further manual review and calibration are performed. The automatic scoring results are stratified at a rate of 10% and reviewed by domain experts. The difference between the automatic and manual scores is calculated. If the mean absolute error is >0.1, calibration is triggered—linear regression calibration of the LLM score is performed using the manually reviewed data (calibration formula: ...). ,in and The final score criteria were determined through a manual-automatic paired regression analysis. Data with a final score ≥ 0.8 and each dimension ≥ 0.7 was considered high-quality data (entered into the training set); data with a total score between 0.6 and 0.8 was considered data requiring correction (returned to expert correction or entered into the relabeling set); and data with a total score < 0.6 was considered low-quality data (discarded directly). This multi-threshold method avoids the "one-size-fits-all" problem of a single total score threshold. An inference chain might have a high total score but a low score in a certain dimension (e.g., professionalism 0.9 but consistency 0.5). The dual threshold of "total score + lowest score in each dimension" ensures the overall quality of the data.

[0068] A total of 3200 pairs of DPO training data were constructed using the three methods described above. The DPO training objective is represented as follows: , in, This represents the DPO loss function value. The smaller the value, the greater the degree to which the model rewards the preferred chain more than the undesired chain. For the dataset All triples Take the expected value (average). Where... For input questions, The preferred inference chain (winner) is selected. This is a loser reasoning chain. This involves taking the sigmoid function and then the logarithm of the value within the parentheses. The sigmoid function maps any real number to the interval (0,1). The temperature parameter (with a value of 0.3) controls the preference intensity. The larger the value, the more sensitive the model is to preferences, but the more easily it deviates from the pre-training distribution. For the current policy model (to be trained) at the input Output generated below The probability of. Initialize the optimal checkpoint from the SFT model. As a reference model (parameter freeze) at input Output generated below The probability of. The weights are fixed for the SFT model and are not updated during training. When the probability ratio of the policy model to the preferred chain is higher than that to the undesired chain, the value in parentheses is positive, and the loss decreases. The DPO loss function implicitly includes a KL divergence constraint to prevent the policy model from... Excessive deviation from the reference model .

[0069] Furthermore, the two-stage progressive linkage mechanism consists of five collaborative mechanisms, including mechanism 1 - model initialization, mechanism 2 - sample progressiveness, mechanism 3 - early stop protection, mechanism 4 - KL divergence monitoring, and mechanism 5 - knowledge distillation collaboration.

[0070] Mechanism 1 - Model Initialization: The final weights of the SFT model are simultaneously used as the initial strategy for the DPO. and reference strategy To ensure continuity.

[0071] Mechanism 2 - Sample asymptotically: SFT samples are added gradually from difficulty 1 to 5, and DPO preference pairs transition from obvious superiority / inferiority (score difference > 0.5) to subtle superiority / inferiority (score difference 0.3-0.5).

[0072] Mechanism 3 - Early Stop Protection: Training stops when the DPO loss does not decrease for 500 consecutive steps. In the DPO training experiment, different early stop steps were compared (250 steps / 500 steps / 750 steps / 1000 steps / no early stop). The results showed that: training was insufficient at 250 steps (validation loss was about 5.2% higher), and early stop at 500 steps achieved the optimal balance between validation loss and final accuracy. Slight overfitting was already evident at 750 and 1000 steps (validation loss started to rise, but training loss continued to decrease). 500 steps corresponded precisely to the time point when the model reached its optimal performance on the validation set. From the training curve, in the total training steps of about 1500-2000, the loss decreased rapidly in the first 1000 steps, entered a stable decline period from 1000-1500 steps, and the loss tended to level off after 1500 steps. Setting a continuous window of no decrease of 500 steps effectively avoided the noise fluctuations in the early stage and only triggered the stop after actual convergence. Compared to other step counts, setting 250 steps makes it easier to be falsely triggered by loss fluctuations in the early stages of training (DPO training has a loss fluctuation of about 5.3% in the first 500 steps), leading to premature stopping and undertraining (accuracy drops by about 6.8 percentage points). Setting 750 or 1000 steps reduces the risk of false triggers but increases unnecessary training time (training about 30-50% more steps), and the model may already be overfitting on the validation set (validation loss starts to rise by about 3-5%). Without early stopping, the model shows obvious signs of overfitting after 2000 steps (training loss continues to decrease but validation loss increases), and accuracy drops by about 3.2 percentage points. 500 steps is a proven robust value that can adapt to DPO training on datasets of different sizes (7B-13B parameters) and in different domains. The total number of steps in DPO training is generally in the range of 1500-3000 steps. A continuous non-decreasing window of 500 steps (accounting for about 15-30% of the total steps) is sufficient to determine whether the model has converged, while leaving enough margin to avoid interference from short-term fluctuations. The 500-step early stopping mechanism achieves the optimal balance between training sufficiency and preventing overfitting, avoiding undertraining caused by stopping too early and the decline in generalization performance caused by stopping too late, while saving about 20-40% of ineffective training time.

[0073] Mechanism 4 - KL divergence monitoring: Real-time monitoring of the KL divergence between the policy model and the reference model; if it exceeds a threshold... Training should be stopped at this time.

[0074] Mechanism 5 - Knowledge Distillation Collaboration: This mechanism utilizes the teacher model in parallel, aligning the inference step distributions of students and teachers through KL divergence loss, and distilling the weights. It increases linearly from 0.2 to 0.4.

[0075] In a practical application stage, such as Figure 4 The system first collects and integrates multi-source heterogeneous data as the reasoning context. This multi-source heterogeneous data includes real-time equipment sensor data, operation logs, and historical maintenance records. The system inputs this multi-source heterogeneous data into a finely tuned, large-scale vertical domain model, triggering a SCoT (Structured CoT) reasoning process comprising seven standard cognitive stages: The first step is target identification, accurately identifying "abnormal increase in bearing temperature" as the key fault phenomenon and automatically associating it with reference knowledge base marker K12; the second step is hypothesis generation, proposing candidate root causes based on historical experience, including poor lubrication, bearing wear, cooling system failure, and overload, and associating them with reference knowledge markers K5 and K7; the third step is evidence collection. The first step extracts and analyzes vibration spectrum, temperature evolution trend, and oil analysis data, generating a confidence value of 0.78 for the current step. The fourth step performs hypothesis verification, successfully eliminating "cooling system failure" and "overload" based on the chain of evidence, raising the confidence of the verification step to 0.85. The fifth step performs root cause determination, ultimately locking the core root cause of the failure to "poor lubrication," and outputting a high confidence value of 0.92. The sixth step executes treatment recommendations, generating standardized actions including "immediately stop the machine, check the lubrication circuit, replenish lubricating oil, and replace the damaged bearing," and attaching the reference knowledge tag K23. The seventh step performs reflective verification, performing a self-check on the completeness and consistency of the global reasoning logic, resulting in an overall step confidence value of 0.87.

[0076] Furthermore, after the SCoT inference chain is executed, the system automatically organizes and formats a structured diagnostic report. The report clearly includes diagnostic conclusions, treatment suggestions, a knowledge reference list, and a comprehensive confidence assessment value. To ensure absolute safety and inference reliability in industrial scenarios, the structured diagnostic report is connected to the expert review module in real time as a feedback node. Domain experts conduct online evaluation of the inference steps, root causes, and treatment suggestions generated by the model. When inference deviations or logical flaws are found, experts manually correct and annotate the model output directly on the interface. The corrected high-quality diagnostic data is automatically absorbed into the offline training data pool as new high-value samples of structured subjective thinking chains for incremental retraining and version iteration updates of the large-scale model in the vertical domain. Simultaneously, the expert-corrected final decision and maintenance plan are re-accumulated into the historical maintenance record database through a feedback loop, thus realizing the integration of multi-source data collection. SCoT Structured Reasoning Expert-human collaborative review A self-evolutionary mechanism for the entire process of "data accumulation and model closed-loop iteration".

[0077] Example 6

[0078] Reference Figure 1 and Figure 5 This is the sixth embodiment of the present invention. Unlike the previous embodiment, this embodiment provides a specific method for three-modal knowledge enhancement control, which solves the problem that knowledge illusion is prone to occur when generating subjective reasoning sequence 402 and that there is a lack of a real-time calibration mechanism for an external authoritative knowledge system.

[0079] Specifically, in the implementation phase of trimodal knowledge enhancement control, trimodal knowledge injection 401 is adopted, and the concept of "Knowledge Enhancement Thinking Chain (KA-SCoT)" is proposed, which deeply integrates knowledge injection with thinking chain reasoning. It includes three complementary methods: explicit knowledge citation 4011, implicit knowledge distillation 4012, and inference-time knowledge retrieval 4013.

[0080] Explicit Knowledge Reference 4011—Enhancing Interpretability: The `explicit_knowledge_refs` field is defined in the SCoT data structure. Each inference step explicitly references at least one knowledge entry from the domain knowledge graph. Each knowledge entry contains five elements: entity, relation, attribute, source, and confidence level. During inference, the model explicitly outputs an interpretable process: "Based on knowledge K1...K3, we arrive at...conclusion".

[0081] The knowledge citation accuracy reward function is expressed as: , in, This is a reward value for the accuracy of knowledge citation, ranging from [0,1]. A higher value indicates that the knowledge cited by the model is more accurate and comprehensive. This is the set of knowledge references predicted by the model, containing the IDs of all knowledge entries that the model considers relevant. This is a collection of authentic knowledge references (annotated by domain experts), containing the knowledge entry IDs that the standard answer should cite. This is an indicator function. It refers to the set of knowledge references predicted by the model. With real knowledge reference set The value is 1 if the information is completely identical, and 0 otherwise, ensuring that the referenced knowledge entries are completely correct. The knowledge reference coverage function is defined as follows: This is calculated by dividing the number of knowledge entries correctly referenced by the model by the total number of knowledge entries that should be referenced. It measures the completeness of the references.

[0082] Implicit Knowledge Distillation 4012—Deeply internalizes domain knowledge, selecting a domain-based model (e.g., Qwen2.5-72B-Instruct for industrial applications) as the teacher model to generate high-quality SCoT inference chains for the training problem. The distillation loss aligns with both the final output and the distribution of intermediate inference steps, expressed as: , in, The total knowledge distillation loss value is obtained by weighted summation of the distillation alignment term and the SFT retention term. The distillation weight coefficients are linearly increased from 0.2 to 0.4 during training. Initially, α is small, allowing the model to learn the SFT well first; later, α is increased to strengthen distillation alignment. The total number of reasoning steps. For the teacher model and student model in the reasoning steps The output shows the KL divergence of the probability distribution. The KL divergence measures the difference between two distributions; the smaller the value, the closer the distributions are. For the teacher model in input The next generation Step-by-step reasoning and the final answer The probability distribution, This defines the probability distribution for generating corresponding outputs for the student model under the same input. The base SFT loss term is retained, while the distillation process ensures that the model's learned SFT capabilities are not destroyed.

[0083] Knowledge retrieval during inference (4013) ensures the timeliness of knowledge. It uses the FAISS (Facebook AI SimilaritySearch, a high-dimensional vector similarity search engine open-sourced by Meta) vector database to store entity and triple embeddings of the knowledge graph (based on the BGE-large-zh-v1.5 Chinese embedding model of the BGE (BAAI General Embedding, an open-source general semantic embedding) series, with a vector dimension of 1024). During inference, each inference step generates a retrieval query, retrieves the top-5 knowledge items, and performs secondary filtering (threshold 0.6) through the BGE-reranker-large re-ranking model, retaining the top-3 as the inference context. It supports "dynamic knowledge retrieval", and different queries can be executed in different steps.

[0084] The training phase uses a combination of "explicit knowledge reference 4011 + implicit knowledge distillation 4012"; the inference phase uses a combination of "explicit knowledge reference 4011 + inference-time retrieval". The outputs of these three methods are fused through a "knowledge consistency voter" (based on an NLI (Natural Language Inference) model to determine three semantic relationships: implication, contradiction, and neutrality). A unified conclusion is output when the consistency rate is ≥0.7; otherwise, a low-confidence alarm is triggered. Evaluation shows that the voter improves the "inter-expert consistency" C2 index by 5.8 percentage points.

[0085] Example 7

[0086] Reference Figure 6 The seventh embodiment of the present invention provides a vertical domain large model reasoning ability fine-tuning training system, including a data acquisition and encoding module 100, a data evaluation and screening module 200, a model training and optimization module 300, and an application service and knowledge verification module 400.

[0087] The data acquisition and encoding module 100 is used to acquire unstructured reasoning text and parse and normalize it into structured subjective thought chain samples 101. The structured subjective thought chain samples 101 contain multiple step units arranged in the order of reasoning. Each step unit is explicitly associated with a step type label 1011, reasoning content, and a subjective cognitive feature field 1012 used to express the expert's decision-making experience.

[0088] The data evaluation and screening module 200 is used to quantitatively score the structured subjective thinking chain sample 101 using multidimensional evaluation indicators, perform confidence decay propagation calculation, and classify the screened samples into different difficulty levels according to the number of causal reasoning layers and the number of related knowledge items contained in the sample.

[0089] The model training and optimization module 300 is used to input the subjective thinking chain dataset into the target language model in stages according to the difficulty level from low to high for phased parameter updates. By introducing step length constraints and step diversity constraints, loss penalties are imposed on the deviation of the number of reasoning steps and the uniformity of step type distribution. Based on the filtered dataset, a preference dataset 3021 is constructed, and the updated language model is used as a policy model for preference alignment training.

[0090] The application service and knowledge verification module 400 is used to guide the vertical domain inference model to generate an inference sequence 402 when the vertical domain inference model receives an input question from the target domain. This is achieved by using trimodal knowledge injection 401, jointly executing explicit knowledge citation 4011, implicit knowledge distillation 4012, and inference-time knowledge retrieval 4013. Before outputting the decision conclusion, the module extracts the subjective deduction path conclusion and the external knowledge association path conclusion from the inference sequence 402, calls the semantic logic verification model to calculate the semantic consistency rate between the two, and outputs a professional decision conclusion or triggers an alarm prompt and blocks the output.

[0091] This embodiment also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the fine-tuning training method for vertical domain large model inference capabilities proposed in the above embodiments.

[0092] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0093] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0094] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.< / type:verification>

Claims

1. A method for fine-tuning the reasoning ability of a large vertical model, characterized in that, include: Acquire unstructured reasoning data generated by experts when they reason about professional tasks in a target domain; The unstructured reasoning data is structured and encoded to generate a structured subjective thought chain sample (101). The structured subjective thought chain sample (101) includes at least a step type label (1011) and a subjective cognitive feature field (1012). Obtaining the structured subjective thought chain sample (101) includes obtaining unstructured expert reasoning text through thought aloud, structured interviews, historical case retrospection and multi-subject collaborative reasoning; Heuristic strategy markers and alternative probability hypotheses are extracted from the thought process and structured interviews, and logical backtracking correction markers and interaction divergence markers are extracted from the historical case retrospective and multi-subject collaborative reasoning, and then normalized and encapsulated into the subjective cognitive feature field (1012). The fine-tuning training of the language model using the structured subjective thought chain sample (101) also includes quality screening of the structured subjective thought chain sample (101) and calculating the initial confidence of each reasoning step in the structured subjective thought chain sample (101) through evaluation indicators. Determine whether the initial confidence level of the current inference step is lower than the preset confidence threshold. If it is lower, calculate the transit confidence level of each subsequent inference step by attenuating it according to the preset attenuation factor, and calculate the overall confidence level of the sample based on the transit confidence level. Structured subjective thought chain samples (101) with an overall confidence level lower than a preset qualified threshold are removed from the inference data; The step type marker (1011) is used to indicate the cognitive stage type to which the current reasoning step belongs; The subjective cognitive feature field (1012) is used to express at least one of the heuristic labels, confidence values, and alternative solution information used by experts in the reasoning process; The language model is fine-tuned based on the structured subjective thought chain sample (101) so that when the language model receives an input question from the target domain, it generates a reasoning sequence (402) containing a step type label (1011) and a subjective cognitive feature field (1012) as well as a decision conclusion.

2. The method for fine-tuning the inference ability of a large vertical model as described in claim 1, characterized in that: The fine-tuning training of the language model based on the structured subjective thought chain sample (101) includes, Obtain the number of causal reasoning layers and the number of related knowledge items contained in the structured subjective thinking chain sample (101); Based on the number of causal reasoning layers and the number of related knowledge items, the structured subjective thinking chain sample (101) is classified into different difficulty levels. The language model is controlled to be input into the corresponding category of structured subjective thought chain samples (101) in order of increasing difficulty level for phased parameter updates.

3. The method for fine-tuning the inference ability of a large vertical model as described in claim 2, characterized in that: The fine-tuning training of the language model based on the structured subjective thought chain sample (101) also includes supervised fine-tuning (301). When calculating the loss function for the controlled language model, a step length constraint term and a step diversity constraint term are introduced. The step length constraint term calculates the penalty loss based on the difference between the number of generated inference steps and the preset number of steps range; The step diversity constraint term calculates the penalty loss based on the distribution negative entropy of each step type label (1011) in the generated inference steps, and combines it with the basic prediction loss to constrain the parameters of the language model.

4. The method for fine-tuning the inference ability of a large vertical model as described in any one of claims 1 to 3, characterized in that: It also includes preference optimization (302), which combines structured subjective thought chain samples (101) of different qualities into a preference dataset (3021) containing preference sequences and non-preference sequences based on the score difference. Based on the preference dataset (3021), the updated language model is used as the policy model for preference alignment training, and the relative entropy of the probability distribution between the policy model and the preset reference model is calculated in real time during the training process. When the relative entropy exceeds a preset upper limit threshold, or when the preference loss value fails to decrease for a preset number of consecutive steps, the preference alignment training is terminated.

5. The method for fine-tuning the inference ability of a large vertical model as described in claim 4, characterized in that: It also includes trimodal knowledge enhancement control, which binds explicit knowledge entity reference tags in the structured subjective thought chain sample (101); During the language model training process, the intermediate layer representation distribution of the language model is fitted to the representation distribution of the preset knowledge teacher model; During the process of the language model receiving the input question and generating the reasoning sequence (402), complementary knowledge items are retrieved from the external knowledge base based on the semantic features of the current reasoning step and incorporated into subsequent reasoning steps.

6. The method for fine-tuning the reasoning ability of a large vertical model as described in any one of claims 1 to 3 and 5, characterized in that: It also includes semantic consistency verification, extracting the subjective deduction path conclusions and external knowledge association path conclusions in the reasoning sequence (402); The semantic logic verification model is invoked to determine the implication and contradiction relationships between the conclusions of the subjective deduction path and the conclusions of the external knowledge association path, and the semantic consistency rate is calculated. When the semantic consistency rate reaches a preset consistency threshold, a decision conclusion is output. When the semantic consistency rate is lower than a preset consistency threshold, an alarm is triggered or the output of the decision conclusion is blocked.

7. A vertical domain large model reasoning ability fine-tuning training system, employing the vertical domain large model reasoning ability fine-tuning training method as described in any one of claims 1 to 6, characterized in that: It includes a data acquisition and encoding module (100), a data evaluation and screening module (200), a model training and optimization module (300), and an application service and knowledge verification module (400). The data acquisition and encoding module (100) is used to acquire unstructured reasoning text and parse and normalize it into structured subjective thought chain samples (101). The structured subjective thought chain samples (101) contain multiple step units arranged in the order of reasoning. Each step unit is explicitly associated with a step type label (1011), reasoning content, and a subjective cognitive feature field (1012) used to express the expert's decision-making experience. The data evaluation and screening module (200) is used to quantify and score the structured subjective thinking chain sample (101) using multidimensional evaluation indicators, perform confidence decay propagation calculation, and divide the screened sample into different difficulty levels according to the number of causal reasoning layers and the number of related knowledge items contained in the sample. The model training and optimization module (300) is used to input the subjective thinking chain dataset into the target language model in order of difficulty from low to high for phased parameter updates. By introducing step length constraints and step diversity constraints, loss penalties are imposed on the deviation of the number of reasoning steps and the uniformity of step type distribution. Based on the filtered dataset, a preference dataset (3021) is constructed, and the updated language model is used as a policy model for preference alignment training. The application service and knowledge verification module (400) is used to, when the vertical domain reasoning model receives an input question from the target domain, employ trimodal knowledge injection (401), jointly execute explicit knowledge reference (4011), implicit knowledge distillation (4012), and inference-time knowledge retrieval (4013), guide the vertical domain reasoning model to generate a reasoning sequence (402), and before outputting a decision conclusion, extract the subjective derivation path conclusion and the external knowledge association path conclusion in the reasoning sequence (402), call the semantic logic verification model to calculate the semantic consistency rate between the two, output a professional decision conclusion or trigger an alarm prompt and block the output.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the vertical domain large model inference capability fine-tuning training method as described in any one of claims 1 to 6.

Images