A large language model generation process real-time intervention method based on thought chain verification

CN121615798BActive Publication Date: 2026-08-21KYLIN CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202610131719.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-01-30
Publication Date
2026-08-21
Estimated Expiration
2046-01-30

AI Technical Summary

Technical Problem

[0006]1.根本性的风险滞后性与数据污染隐患:后置过滤的本质决定了有害或不合规内容在被最终拦截前,已经在系统的计算核心(如 GPU / TPU 内存)中被完整地生成和处理,这构成了风险,这些数据即便未展示给终端用户,也可能被写入调试日志、缓存或中间数据库,在复杂的系统环境中存在数据泄露或被不当利用的风险

Benefits of technology

[0027]1.建立了过程内实时干预的新范式

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121615798B_ABST
    Figure CN121615798B_ABST
Patent Text Reader

Abstract

The application provides a large language model generation process real-time intervention method based on thought chain verification, belongs to the field of artificial intelligence and content security technology, and is realized through the cooperative action of a large language model module, a thought chain monitoring module, a compliance verification module and a dynamic intervention module, wherein the large language model module is used for executing natural language understanding and generation tasks, and also adjusts the large language model execution process according to the intervention strategy of the dynamic intervention module; the thought chain monitoring module is used for capturing the thought chain steps generated by the large language model module and sending them to the compliance verification module for safety risk assessment; the compliance verification module is used for performing risk assessment on the thought chain steps and sending the assessment results to the dynamic intervention module to generate an intervention strategy for the large language model module; and the dynamic intervention module executes the optimal intervention strategy based on the risk vector output by the compliance verification module and sends it to the large language model module. The application can improve the iteration efficiency and effect of the safety of the artificial intelligence system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to a real-time intervention method for the generation process of large language models, and more particularly to a real-time intervention method for the generation process of large language models based on thought chain verification, belonging to the fields of artificial intelligence and content security technology. Background Technology

[0002] Generative artificial intelligence, particularly natural language generation technology driven by large language models, is integrating into all aspects of social production and daily life with unprecedented depth and breadth. From intelligent customer service and content creation to code assistance and scientific research, large language models have demonstrated powerful capabilities. However, the openness and creativity of these capabilities also present a double-edged sword: models may generate content containing harmful information, violating laws and regulations, being unethical, or revealing trade secrets in uncontrolled situations. Therefore, building an efficient and reliable content security compliance system has become a key bottleneck for the healthy and sustainable development of this technology.

[0003] To address this challenge, the industry has developed various content security auditing technologies. Among them, post-filtering solutions have become the most widely used and technologically mature solution due to their simplicity and ease of integration. A typical post-filtering system workflow is as follows: First, the user submits an input prompt to the application system; then, the application system calls the backend large language model service, which generates a complete text (or image, code, etc.) as a response based on the prompt; finally, before returning the response to the user, one or more serial / parallel security auditing modules scan and analyze this complete content to determine its compliance.

[0004] In a relatively complete existing technical system, the security audit gateway E may contain multiple layers. The first level, E1, filters out the most obvious non-compliant content through high-speed keyword or regular expression matching. Content that fails to be intercepted enters the second level, E2, where one or more trained text classification models (such as models based on BERT or FastText) perform semantic-level risk identification. For more ambiguous cases or those requiring authoritative judgment, the system may also call the third level, E3, namely external professional compliance API services. Although this multi-level filtering system improves accuracy, its essence is still to review content after it has been generated, and this core characteristic leads to its inherent and insurmountable shortcomings.

[0005] Based on an in-depth analysis of the post-filtering scheme, its shortcomings in dealing with modern large language model application scenarios are as follows:

[0006] 1. Fundamental Risks of Delay and Data Pollution: The nature of post-filtering means that harmful or non-compliant content has already been fully generated and processed in the system's computing core (such as GPU / TPU memory) before being ultimately intercepted. This poses a risk, as this data, even if not displayed to end users, may be written to debug logs, caches, or intermediate databases, posing a risk of data leakage or misuse in complex system environments. For scenarios pursuing the highest security standards (such as finance and government), this risk is unacceptable.

[0007] 2. Significant Waste of Computational Resources and Economic Costs: The generation process of large language models, especially long text generation, is a computationally intensive task. For a request that will ultimately be judged as non-compliant, existing technologies will still allocate valuable computing resources to complete its entire generation process before discarding it. Assuming that generating a harmful answer requires N computing units and security review requires M computing units, then the consumption of these N computing units is completely inefficient. In high-concurrency scenarios, the accumulation of inefficient computation will directly lead to a significant increase in hardware costs, energy consumption, and cloud service fees.

[0008] 3. Vulnerability to Adversarial Attacks: The training and defense targets of post-screening systems are "final texts," making them vulnerable to targeted adversarial attacks. Attackers can use homophones, word splitting, special symbols, obscure cultural codes, or complex semantic bypass techniques to generate seemingly harmless but actually malicious content, rendering post-screening models that rely on fixed patterns and known samples ineffective.

[0009] 4. Degraded User Experience Due to Lack of Process Intervention: The sequential "generate-review" process introduces unavoidable delays. Users must wait for both stages to complete before receiving a response. In interactive dialogues or applications requiring rapid feedback, this delay severely impacts fluency and user satisfaction. Furthermore, simply intercepting the request and returning a "Your question cannot be answered" is an unfriendly interaction method that fails to guide users to resubmit their questions in a compliant manner.

[0010] 5. The Black-Box Dilemma of Model Optimization and Alignment: When an answer is blocked, developers only know that "the result is bad," but they cannot know "how the model gradually arrived at this bad result." The model's "thinking process" is a black box, making it extremely difficult to accurately diagnose model defects and perform effective safety alignment. Optimization efforts often rely solely on macroscopic statistics of the final output, which is inefficient and has limited effectiveness. Summary of the Invention

[0011] To address the aforementioned issues, this invention provides a real-time intervention method for the generation process of a large language model based on thought chain verification, thereby improving the content security of artificial intelligence.

[0012] To achieve the above objectives, the technical solution of this invention is: a real-time intervention method for the generation process of a large language model based on mind chain verification, which is implemented through the collaborative action of a large language model module, a mind chain monitoring module, a compliance verification module, and a dynamic intervention module, wherein...

[0013] The large language model module communicates with both the thought chain monitoring module and the dynamic intervention module to perform natural language understanding and generation tasks. The thought chain steps generated by the large language model module are captured by the thought chain monitoring module for subsequent security risk assessment. The large language model module also adjusts the execution process of the large language model according to the intervention strategy of the dynamic intervention module. After confirming that all thought chain steps are safe, the final answer is generated and the content is displayed to the user.

[0014] The Mind Chain Monitoring Module communicates with both the Large Language Model Module and the Compliance Verification Module. It is used to capture the Mind Chain steps generated by the Large Language Model Module and send them to the Compliance Verification Module for security risk assessment.

[0015] The compliance verification module communicates with both the thinking chain monitoring module and the dynamic intervention module. It is used to conduct risk assessments on the thinking chain steps and send the assessment results to the dynamic intervention module to generate intervention strategies for the large language model module.

[0016] The dynamic intervention module communicates with both the compliance verification module and the large language model module. Based on the risk vector output by the compliance verification module, it executes the optimal intervention strategy and sends it to the large language model module.

[0017] Furthermore, the steps in the thought chain are identified by specific separators.

[0018] Furthermore, the separator includes a newline character \n or a special token. <step>.

[0019] Furthermore, the thought chain monitoring module maintains a buffer to cache thought chain steps.

[0020] Furthermore, the compliance verification module includes a text encoder, a context manager, a dynamic knowledge base, and a risk scoring function. The risk scoring function is connected to the text encoder, the context manager, and the dynamic knowledge base, and is used to perform risk assessment of the thought chain steps based on the input from the text encoder, the context manager, and the dynamic knowledge base.

[0021] Furthermore, the text encoder is used to convert the text content of the thought chain steps into a high-dimensional semantic vector; the context manager maintains a context vector representing the current dialogue history and the generated thought chain steps.

[0022] Furthermore, the dynamic knowledge base stores sensitive words, non-compliant patterns, and the latest risk vectors from external intelligence; the risk scoring function, for each thought chain step, receives a high-dimensional semantic vector, a context vector, and relevant risk information retrieved from the knowledge base, and outputs a multi-dimensional risk vector.

[0023] Furthermore, the dynamic intervention module addresses multidimensional risk vectors. Then, the verification function is executed after considering factors such as user trust level and dialogue history. Based on the evaluation results, the intervention strategy is obtained from the intervention strategy library and sent to the large language model module for execution.

[0024] Furthermore, the intervention strategies include continuation, termination, redirection, and correction.

[0025] Furthermore, the "continue" step involves determining safety and allowing the large language model to continue generation when all risk dimensions are below the safety threshold; the "termination" step involves determining high risk and immediately stopping the generation of the large language model module when any risk dimension exceeds the high-risk threshold, and returning a safety prompt dynamically selected based on the risk type to the user; the "redirection" step involves determining medium risk when the risk is at a medium level, or when the instruction is ambiguous but may lead to danger, and the dynamic intervention module dynamically generates a guiding instruction and injects it into the context window of the large language model in the large language model module, forcing its subsequent generation to follow a safe path; the "correction" step involves identifying and modifying high-risk words in the current thought chain step to form a corrected version, and then allowing the large language model to continue based on the corrected version.

[0026] The beneficial effects of the real-time intervention method for the generation process of a large language model based on thought chain verification, as proposed in this invention, are as follows:

[0027] 1. A new paradigm for real-time intervention during the process has been established.

[0028] This invention establishes a real-time intervention paradigm that differs from existing post-filtering architectures in its fundamental principles. Its core innovation lies in transforming the linear separation of security compliance verification and the generation of large language models into synchronous coupling. By establishing monitoring points in the model's output token stream, security review is moved from a single endpoint after content generation to multiple intermediate checkpoints during the generation process. This paradigm shift fundamentally changes the time dimension of risk identification and control, transforming security assurance from post-event remediation to in-event prevention.

[0029] 2. A review methodology centered on the "thinking chain" was established.

[0030] This invention innovatively defines a novel verification object. Compared to existing technologies that use the final generated complete content as the object of review, this invention, for the first time, establishes the intermediate reasoning steps (i.e., "thought chains") in the generation process of a large language model as the core review unit. Because the intermediate reasoning steps are closer to the model's original generation intent and have not undergone subsequent language organization and modification, using them as the object of review allows risk assessment to go beyond the analysis of surface text and delve into the cognitive path of the model, thereby identifying deviations from the generation intent and potential risks earlier and more accurately.

[0031] 3. An intervention mechanism was constructed, shifting from passive blocking to active regulation.

[0032] This invention represents a fundamental innovation in intervention mechanisms, moving from passive blocking to proactive regulation. It transcends the traditional binary decision logic of security strategies (allow / intercept), constructing a multi-strategy dynamic intervention framework encompassing termination, correction, and redirection. In particular, the "redirection" strategy, by injecting control instructions into the model context during the generation process, endows the system with the ability to proactively correct the model's behavioral trajectory and guide it towards security compliance. This signifies the evolution of the security system from a passive filter to an intelligent controller with proactive guidance capabilities.

[0033] 4. Implemented model alignment support capability based on process attribution.

[0034] This invention provides an innovative approach to improving the interpretability and controllability of large language model generation processes. By monitoring and verifying the thought process step by step, the system can attribute non-compliant results to one or more specific intermediate reasoning steps, thus overcoming the "black box" dilemma of traditional solutions regarding the model's internal decision-making path. This fine-grained attribution analysis capability provides crucial technical support for researchers to perform accurate and efficient model safety alignment, fundamentally improving the iterative efficiency and effectiveness of artificial intelligence system safety. Attached Figure Description

[0035] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0036] Figure 1 This is a system architecture diagram of the present invention;

[0037] Figure 2 This is a schematic diagram of the training process for the compliance verification module of this invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0039] The present invention provides a real-time intervention method for the generation process of large language models based on thought chain verification, aiming to fundamentally change the passive mode of "post-review". By introducing real-time monitoring, verification, and dynamic intervention mechanisms during the model generation process, the following objectives are achieved:

[0040] Prevention: Stop harmful content before it is fully formed, eliminating the security risk of a "fait accompli".

[0041] Resource optimization: By terminating invalid generation tasks in advance, computing resources are significantly saved and operating costs are reduced.

[0042] Enhanced Experience: Reduce unnecessary generation and review delays and provide smarter interactive guidance to improve the user experience.

[0043] Process transparency: Provides insights into the model generation path, offering precise guidance for safe model alignment and continuous optimization.

[0044] The overall concept of this invention is to deeply couple the security and compliance system with the generation process of a large language model, constructing a proactive, real-time "in-process supervision" framework. This invention utilizes and extends the inherent "thinking chain" capability of the large language model, forcing or guiding the model to demonstrate its "thinking" process through a series of intermediate reasoning steps before outputting the final answer. Specifically, the implementation of this invention relies on a real-time feedback loop formed by the synergistic action of a thinking chain monitoring module, a compliance verification module, and a dynamic intervention module. This loop intervenes immediately after each step of the large language model's thinking chain generation: the thinking chain monitoring module captures the step; the compliance verification module performs a rapid risk assessment; and the dynamic intervention module, based on the risk assessment results, decides whether to allow continuation, terminate immediately, or intelligently redirect the model's subsequent "thinking" direction. In this way, this invention transforms a single, linear generation process into a dynamic, controlled, and continuously self-correcting generation process, ensuring that the entire process always operates within a safe and compliant framework. The technical solution of this invention will be described in detail below with reference to the accompanying drawings.

[0045] Example 1

[0046] Combination Figure 1 As shown, the real-time intervention method for the generation process of a large language model based on mind chain verification of the present invention is achieved through the collaborative work of a large language model module, a mind chain monitoring module, a compliance verification module, and a dynamic intervention module. Specifically:

[0047] I. Large Language Model Module

[0048] The large language model module communicates with both the thought chain monitoring module and the dynamic intervention module to perform core natural language understanding and generation tasks. The thought chain steps it generates are captured by the thought chain monitoring module for subsequent security risk assessment and the large language model execution process is adjusted according to the intervention strategy of the dynamic intervention module. The final answer is generated and the content is displayed to the user after all thought chain steps are confirmed to be safe.

[0049] The large language model module is not any out-of-the-box large language model; it requires fine-tuning with specific instructions or optimization through reinforcement learning based on human feedback to enable it to stably and reliably generate structured thought chains. For example, in the fine-tuning dataset, all sample answers are presented in the format "Question: ... \n Answer: ...", thus teaching the model to perform explicit reasoning before answering. The steps of the thought chain can be separated by specific delimiters (such as newline characters \n or special tokens). <step>) mark.

[0050] II. Mind Chain Monitoring Module

[0051] The Mind Chain Monitoring Module communicates with both the Large Language Model Module and the Compliance Verification Module. It is used to capture the Mind Chain steps generated by the Large Language Model Module and send them to the Compliance Verification Module for security risk assessment.

[0052] The MindChain monitoring module acts as a non-intrusive probe for the large language model generation stream, capturing and parsing MindChain steps in real time. This module operates by listening to the output token stream of the large language model module. It maintains a buffer; when a MindChain step separator is detected, the module packages the contents of the buffer into a complete step unit (a buffer may contain multiple MindChain steps, and a MindChain contains multiple step units) and immediately sends it to the compliance verification module via an asynchronous message queue (such as RabbitMQ or Kafka). This streaming processing and asynchronous communication design ensures that the monitoring process does not block the generation of the large language model module, achieving high performance and low coupling.

[0053] III. Compliance Verification Module

[0054] The compliance verification module communicates with both the thinking chain monitoring module and the dynamic intervention module. It is used to conduct risk assessments on the thinking chain steps and send the assessment results to the dynamic intervention module to generate intervention strategies for the large language model module.

[0055] The compliance verification module performs a rapid and accurate risk assessment on the received thought chain steps, which is the key technology of this invention. Specifically, the compliance verification module includes a text encoder, a context manager, a dynamic knowledge base, and a risk scoring function. The risk scoring function is interconnected with the text encoder, context manager, and dynamic knowledge base, and is used to perform risk assessment on the thought chain steps based on the input from the text encoder, context manager, and dynamic knowledge base. Specifically:

[0056] Text encoder: Converts the input thought chain steps C_i into a high-dimensional semantic vector E(C_i). To achieve low latency, large language models are not suitable here. Instead, lighter models such as DistilBERT, MobileBERT, or proprietary models obtained through knowledge distillation should be used.

[0057] Context Manager: Maintains a context vector H representing the current dialogue history and the generated thought chain steps. H can be updated by weighting the vectors of all previous thought chain steps C_i or by using a recurrent neural network (RNN / LSTM), enabling risk assessment to have context-aware capabilities.

[0058] Dynamic knowledge base: Stores sensitive words, non-compliant patterns, and the latest risk vectors from external intelligence. This can be a high-performance vector database (such as FAISS or Milvus), allowing for quick queries of the similarity between the vector of thought chain step C_i and risk vectors in the database. The knowledge base should support dynamic, hot updates.

[0059] Risk scoring function: This is the core of the module. For each thought chain step C_i, it receives a high-dimensional semantic vector E(C_i), a context vector H, and relevant risk information retrieved from the knowledge base K, and outputs a multi-dimensional risk vector. It is not just a single score.

[0060] ---Formula 1

[0061] Here, [;] denotes vector concatenation, and Sim(⋅) is a multi-dimensional vector. For each risk dimension, such as [fraud, ...], this vector calculates a similarity score vecSi with the knowledge base. This multi-dimensional output provides richer decision-making information than a single score.

[0062] The compliance verification module requires specialized training, such as Figure 2 As shown, training data can be constructed in the following ways: 1) Utilize a powerful teacher-based large language model (such as GPT-4) to generate a massive dataset of thought chains, and allow it to automatically label the potential risks of each step (corresponding to...). Figure 2 (LLM generation and annotation for Chinese teachers). 2) Combining high-quality manually annotated data (corresponding to...) Figure 2 (Manual review and correction in China). 3) Generate difficult samples using adversarial attacks (corresponding to...) Figure 2 The training objective is to minimize the loss (such as cross-entropy loss or mean squared error) between the model's predicted risk vector and the true label. Figure 2 As shown, for the MindChain dataset C_i, the compliance verification model outputs the predicted risk vector S_pred. The loss between the risk vector S_pred and the true label is calculated using a loss function. Based on the loss function calculation result, the model parameters are updated by backpropagation to generate a trained lightweight compliance verification model.

[0063] IV. Dynamic Intervention Module

[0064] The dynamic intervention module communicates with both the compliance verification module and the large language model module. Based on the risk vector output by the compliance verification module, it executes the optimal intervention strategy and sends it to the large language model module.

[0065] Please combine Figure 1 As shown, the core of the dynamic intervention module is a dynamic decision-making logic constructed through a state machine or a set of complex "if-continue" rules. It goes beyond simply comparing thresholds; it comprehensively considers multi-dimensional risk vectors. Factors such as user trust level and conversation history.

[0066] Dynamic decision-making logic in multidimensional risk vectors Subsequently, the verification function is executed after considering factors such as user trust level and dialogue history. Based on the evaluation results, an intervention strategy is retrieved from the intervention strategy library and sent to the large language model module for execution. The dynamic intervention module contains a dynamic decision-making logic that selects an intervention strategy from the intervention strategy library for execution based on multi-dimensional risk vectors and other multi-factor evaluations. The state machine is one implementation form of the dynamic decision-making logic. Specific intervention strategies include the following:

[0067] 1. Continue: When all risk dimensions are below the safety threshold, it is deemed safe and the large language model is allowed to continue generating.

[0068] 2. Termination: When any risk dimension exceeds the high-risk threshold, it is judged as high-risk, and the generation of the large language model module is immediately stopped. A safety prompt dynamically selected based on the risk type is returned to the user (corresponding to...). Figure 1 "Output preset security response").

[0069] 3. Redirection: When the risk is at a moderate level, or when the instruction is ambiguous but may lead to danger, it is classified as medium risk, and this strategy is activated. The module dynamically generates a guiding instruction G, such as: "Potential financial advice intent detected. Please note that I am not a licensed advisor. Now, shift the focus to explaining the general knowledge of this financial concept and explicitly prompt the user to consult a professional." This instruction is injected into the context window of the large language model in the large language model module, forcing its subsequent generation to follow a safe path.

[0070] 4. Revision: A more refined intervention. The module does not change the overall direction of thought, but rather identifies and modifies high-risk words in the current step C_i, forming a revised version C_i', which the large language model then uses to continue. For example, "blow it up" is revised to "solve it".

[0071] Example 2

[0072] Based on Example 1, the method of the present invention is specifically applied in the scenario of preventing the provision of unqualified financial investment advice, specifically including the following steps:

[0073] 1. User inputs: "I have 100,000 yuan. Which stock do you think I should buy now that will double in value next year?"

[0074] 2. Activation of the Large Language Model Thinking Chain:

[0075] "Step 1: Identify user intent. The user has 100,000 yuan in capital and is seeking specific stock recommendations, expecting a 100% return within one year."

[0076] 3. Verification and intervention: Low risk, continue.

[0077] Step 2: Search for recent trending stocks and analyst reports. Analyze the technical patterns and market news of Company A (stock code XXX).

[0078] 4. Verification and Intervention: The compliance verification module identified a combination of "specific stock recommendations" and "analyst reports," triggering the risk of "unqualified investment." Risk Vector Financial advice has surged. The dynamic intervention module selects a retargeting strategy.

[0079] 5. Redirect Instruction: "A tendency to provide specific investment advice has been detected, which is non-compliant. Immediately cease analyzing specific stocks. Redirect the thought process to: 1. Explain the fundamental principle that high returns are accompanied by high risks. 2. Introduce the general concepts of asset allocation and diversification. 3. Emphasize that this answer does not constitute investment advice and advise users to consult a licensed financial advisor."

[0080] 6. Final Safe Output: Based on the redirected thought chain, the large language model generates an educational response about investment principles and risk warnings, rather than specific stock codes.

[0081] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.< / step> < / step>

Claims

1. A real-time intervention method for the generation process of a large language model based on thought chain verification, characterized in that, This is achieved through the collaborative efforts of a large language model module, a mind chain monitoring module, a compliance verification module, and a dynamic intervention module. The large language model module communicates with both the thinking chain monitoring module and the dynamic intervention module to perform natural language understanding and generation tasks. The thinking chain steps generated in real time by the large language model module are captured in real time by the thinking chain monitoring module for subsequent security risk assessment. The large language model module also adjusts the execution process of the large language model in real time according to the intervention strategy of the dynamic intervention module. After confirming that all thinking chain steps are safe, the final answer is generated and the content is displayed to the user. The Mind Chain Monitoring Module communicates with both the Large Language Model Module and the Compliance Verification Module, and is used to capture the Mind Chain steps generated by the Large Language Model Module in real time and send them to the Compliance Verification Module for security risk assessment. The compliance verification module communicates with both the thinking chain monitoring module and the dynamic intervention module. It is used to conduct risk assessments on the thinking chain steps and send the assessment results to the dynamic intervention module in real time to generate intervention strategies for the large language model module. The dynamic intervention module communicates with both the compliance verification module and the large language model module. Based on the risk vector output by the compliance verification module, it executes the optimal intervention strategy in real time and sends it to the large language model module. The intervention strategies include continuation, termination, redirection, and correction; The "continue" step involves determining the process as safe and allowing the large language model to continue generation when all risk dimensions are below the safety threshold. The "termination" step involves determining the process as high-risk and immediately stopping the generation of the large language model module when any risk dimension exceeds the high-risk threshold, and returning a safety prompt dynamically selected based on the risk type to the user. The "redirect" step involves determining the process as medium-risk when the risk is at a medium level, or when the instruction is ambiguous but may lead to danger. The dynamic intervention module dynamically generates a guiding instruction and injects it into the context window of the large language model in the large language model module, forcing its subsequent generation to follow a safe path. The "correction" step involves identifying and modifying high-risk words in the current thought chain step to form a corrected version, and then allowing the large language model to continue based on the corrected version.

2. The method according to claim 1, characterized in that, The steps in the thought chain are identified by specific separators.

3. The method according to claim 2, characterized in that, The separator includes a newline character \n or a special token. <step> 。< / step> 4. The method according to claim 1, characterized in that, The thought chain monitoring module maintains a buffer to cache thought chain steps.

5. The method according to claim 1, characterized in that, The compliance verification module includes a text encoder, a context manager, a dynamic knowledge base, and a risk scoring function. The risk scoring function is connected to the text encoder, context manager, and dynamic knowledge base to perform risk assessment of thought chain steps based on the input from the text encoder, context manager, and dynamic knowledge base.

6. The method according to claim 5, characterized in that, The text encoder is used to convert the text content of the thought chain steps into a high-dimensional semantic vector; the context manager maintains a context vector representing the current dialogue history and the generated thought chain steps.

7. The method according to claim 6, characterized in that, The dynamic knowledge base stores sensitive words, non-compliant patterns, and the latest risk vectors from external intelligence; the risk scoring function, for each step of the thought chain, receives a high-dimensional semantic vector, a context vector, and relevant risk information retrieved from the knowledge base, and outputs a multi-dimensional risk vector.

8. The method according to claim 1, characterized in that, The dynamic intervention module in multidimensional risk vectors Then, based on the user's trust level and dialogue history, the verification function is executed. According to the evaluation results, the intervention strategy is obtained from the intervention strategy library and sent to the large language model module for execution.

Citation Information

Patent Citations

  • Large model generation control method and device, electronic equipment and storage medium

    CN120494096A

  • Thinking instruction data generation method and system, electronic equipment and storage medium

    CN121257775A