Conversation large model streaming output content security correction method and related device

Abstract

The invention provides a dialogue large model streaming output content safety correction method and related device.According to the dialogue large model streaming output content safety correction method, asynchronous safety detection is conducted on a detected object when dialogue large model output content triggers asynchronous safety detection conditions, and if unsafe content is detected, the detected object is subjected to asynchronous safety detection; calling a security enhancement model, performing security correction on the unsafe content through the security enhancement model, and outputting a text fragment meeting a security standard; and after the security enhancement model generates the first security text fragment, stopping the reasoning process of the dialogue large model, splicing the first security text fragment with the text fragment which is output by the dialogue large model and is verified to be secure, and replacing the output content of the dialogue large model with the spliced text content. The subsequent output content of the dialogue large model is switched into the real-time streaming generation content of the security enhancement model, so that the exposure time of the unsafe content of the dialogue large model is shortened, and meanwhile, the output continuity is kept.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and content security technology, and in particular to a dialogue large model streaming output content security correction method and related device. BACKGROUND

[0002] In recent years, dialogue large model technology has developed rapidly and is widely used in intelligent customer service, virtual assistants, content creation and other fields. To improve the smoothness and naturalness of interaction, streaming output has become the mainstream solution. However, large models always face security risks during content generation. For streaming scenarios, related technologies check the output text segments for security, and once unsafe content is found, reasoning and output are interrupted, and an error or content deletion is prompted. This streaming output security interruption method disrupts the smoothness of streaming output, causing significant interaction lag and termination of the dialogue process, and the interruption causes the loss of generated safe and effective information, and provides a poor user experience. SUMMARY

[0003] The present application provides a dialogue large model streaming output content security correction method and related device to solve the defects of traditional methods for handling large model streaming output content security detection by interruption, which disrupt the smoothness of streaming output, cause the loss of effective information, and provide a poor user experience.

[0004] The present application provides a dialogue large model streaming output content security correction method, comprising:

[0005] When the dialogue large model output content triggers an asynchronous security detection condition, the detected object is subjected to asynchronous security detection, and the detected object is the complete content formed by splicing the multiple streaming text segments output by the dialogue large model;

[0006] If unsafe content is detected, a security reinforcement model is called to correct the unsafe content using the security reinforcement model, and a text segment that meets the security standard is output, wherein the security reinforcement model is trained based on the rule violation content and the corresponding security correction content;

[0007] After the security reinforcement model generates the first safe text segment, the reasoning process of the dialogue large model is stopped, the first safe text segment is spliced with the text segment verified as safe output by the dialogue large model, the spliced text content is used to replace the output content of the dialogue large model, and the subsequent output content of the dialogue large model is switched to the real-time streaming generated content of the security reinforcement model.

[0008] According to the method for securely correcting content in a large dialog model stream output according to the present invention, the method for triggering the asynchronous security detection condition includes:

[0009] The cumulative number of characters in the undetected text segments in the output of the dialogue model exceeds a preset threshold;

[0010] and / or,

[0011] The time interval since the last asynchronous safety check exceeded the preset time.

[0012] According to the method for securely correcting content in a large conversation model stream output provided by the present invention, the security enhancement model is trained based on the illegal content and its corresponding security correction content, including:

[0013] Select a pre-trained language model as the base model;

[0014] Construct a training dataset containing pairs of illegal content and security correction content;

[0015] Inputting the training data set into the base model to obtain rewriting loss or continuation loss;

[0016] With the goal of minimizing the rewriting loss or the continuing writing loss, the base model is fine-tuned to obtain a trained security enhancement model.

[0017] According to the method for securely correcting content in a large dialogue model stream output according to the present invention, the method of securely correcting unsafe content using the security enhancement model and outputting text segments that meet security standards includes:

[0018] The structured model instructions are input into the security hardening model so as to output a text segment that complies with security standards through the security hardening model.

[0019] According to the method for securely correcting content in a large conversation model stream output provided by the present invention, the structured model instruction includes:

[0020] A role definition element is used to specify the security review role played by the security hardening model in the task;

[0021] Conversation information elements, including: conversation context history, current conversation question, verified safe response segments, and unsafe response segments to be corrected;

[0022] Task responsibility statement element, used to instruct the security hardening model to perform security corrections or continue writing content for insecure content;

[0023] The output format definition element is used to standardize the output content format of the security hardening model.

[0024] According to the method for security correction of content stream output of a large dialogue model provided by the present invention, inputting structured model instructions into the security enhancement model to output text segments that meet security standards through the security enhancement model includes:

[0025] Constraining the security hardening model identity by the role definition elements;

[0026] Identifying the illegal content type of the unsafe text segment to be corrected by using the security enhancement model;

[0027] Rewrite and / or continue writing the unsafe text segment to be corrected according to the task description and the type of illegal content;

[0028] generating, based on the conversation context history and the current conversation question, a text segment that is semantically coherent with the verified security segment and meets security standards;

[0029] The text segment that meets the security standard is output according to the output content format of the security enhancement model.

[0030] According to the method for securely correcting the content of the streamed output of the large dialogue model provided by the present invention, after switching the subsequent output content of the large dialogue model to the real-time streamed content generated by the security-enhanced model, the method further includes:

[0031] A compliance check is performed on the real-time streaming generated content of the security enhancement model; if the check fails, the pushed content is withdrawn and a preset security prompt is sent to the client.

[0032] The present invention also provides a device for securely correcting content of a large dialogue model stream output, comprising:

[0033] A detection module, configured to perform asynchronous security detection on a detected object when the output of the dialogue model triggers an asynchronous security detection condition. The detected object is the complete content formed by splicing multiple streaming text segments output by the dialogue model;

[0034] a correction module, configured to, upon detecting unsafe content, invoke a security enhancement model, perform security corrections on the unsafe content using the security enhancement model, and output a text segment that meets security standards, wherein the security enhancement model is trained based on the unsafe content and its corresponding security correction content;

[0035] A switching module is used to stop the reasoning process of the dialogue large model after the security enhancement model generates the first safe text segment, splice the first safe text segment with the text segment output by the dialogue large model that has been verified to be safe, use the spliced ​​text content to replace the output content of the dialogue large model, and switch the subsequent output content of the dialogue large model to the real-time streaming generated content of the security enhancement model.

[0036] The present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the method for securely correcting the content of the streaming output of a large dialogue model as described in any one of the above-described methods is implemented.

[0037] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the method for securely correcting the content of the streaming output of a large dialogue model as described in any of the above.

[0038] The method and related device for security correction of content of streaming output of a large dialogue model provided by the present invention perform asynchronous security detection on the detected object when the content output by the large dialogue model triggers an asynchronous security detection condition. The detected object is a complete content formed by splicing multiple streaming text fragments output by the large dialogue model; if unsafe content is detected, a security enhancement model is called, and the unsafe content is security-corrected by the security enhancement model to output a text fragment that meets the security standards, wherein the security enhancement model is trained based on the illegal content and its corresponding security correction content; after the security enhancement model generates the first safe text fragment, the reasoning process of the large dialogue model is stopped, and the first safe text fragment is generated. The full text segment is spliced ​​with the text segment output by the dialogue model that has been verified to be safe, and the spliced ​​text content is used to replace the output content of the dialogue model, and the subsequent output content of the dialogue model is switched to the real-time streaming generated content of the security enhancement model. The present invention switches to its output immediately after the security enhancement model generates the first safe segment, instead of waiting for the complete response to be generated, which significantly reduces the exposure time of the unsafe content of the dialogue model, while maintaining the output consistency, avoiding the decline in user experience due to rough interruption, and the security enhancement model adopts a modular design, which effectively avoids the potential impact of security functions on the performance of the general dialogue model, and ensures the overall flexibility and scalability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is one of the flow charts of the method for securely correcting the content of the streaming output of a large dialogue model provided by the present invention;

[0041] Figure 2 This is an example diagram of secure correction of content in the streaming output of a large dialogue model provided by the present invention;

[0042] Figure 3 This is the second flow chart of the method for securely correcting the content of the streaming output of the large dialogue model provided by the present invention;

[0043] Figure 4 It is a structural diagram of the device for securely correcting the content of the large-scale dialogue model streaming output provided by the present invention. DETAILED DESCRIPTION

[0044] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0045] Figure 1 The flowchart of the method for securely correcting the content of the large dialogue model stream output provided by the embodiment of the present invention is as follows: Figure 1 As shown, the method for securely correcting the content of the large dialogue model streaming output provided by the embodiment of the present invention includes:

[0046] Step 101: When the content output by the dialogue model triggers an asynchronous security detection condition, an asynchronous security detection is performed on the detected object, where the detected object is the complete content formed by splicing multiple streaming text segments output by the dialogue model;

[0047] In this embodiment of the present invention, the large dialogue model generates text snippets in a streaming manner and pushes them to the client in real time, ensuring a low-latency interactive experience. The streamed text snippets are cached and spliced ​​into complete content as the object to be detected, preserving semantic coherence.

[0048] In an embodiment of the present invention, asynchronous security detection means that the security detection runs independently and does not interfere with the dialogue model reasoning or client push, giving priority to ensuring a smooth user experience. The security detection module performs security compliance checks on the content output by the dialogue model, for example, sensitive word detection based on AC automaton (Aho-Corasick Automaton), sensitive corpus detection based on vector similarity matching, and text classification detection based on the Bidirectional Encoder Representations from Transformers (BERT) model. It can also integrate third-party content security detection service APIs.

[0049] If the test passes, the test content is marked as safe, the current test cycle ends, and the next cycle begins. If the test fails, the failed content is marked and the model switching process is started.

[0050] Step 102: If unsafe content is detected, invoke a security hardening model to perform security corrections on the unsafe content using the security hardening model and output a text segment that meets security standards, wherein the security hardening model is trained based on the illegal content and its corresponding security correction content.

[0051] Step 103: After the security enhancement model generates the first secure text segment, stop the reasoning process of the dialogue model, splice the first secure text segment with the text segment output by the dialogue model that has been verified to be secure, use the spliced ​​text content to replace the output content of the dialogue model, and switch the subsequent output content of the dialogue model to the real-time streaming generated content of the security enhancement model.

[0052] In the field of large-scale language models, reasoning generally refers to the process by which a model processes input information and applies its learned knowledge and patterns to generate relevant and coherent output. In an embodiment of the present invention, the reasoning process of the security hardening model specifically refers to the process by which, after receiving model instructions, it analyzes input information to identify unsafe content, assesses potential risks, and generates security corrections and continuation content based on the model instructions. This process is the core computational step in the security hardening model's execution of security correction and security-oriented continuation tasks, and is intended to ensure that the final output response content is secure and compliant.

[0053] After invoking the security-enhanced model's inference process, the original conversation model's inference process continues, and its output stream is pushed to the client in real time. Once the security-enhanced model generates its first output segment that meets security standards, the system immediately stops the original conversation model's inference process. The system concatenates the original, safety-verified output with the security-enhanced model's first streamed output segment and pushes them to the client in a replacement fashion, enabling rapid content updates. Afterward, all data streams pushed to the client completely switch to the real-time output of the security-enhanced model.

[0054] In an embodiment of the present invention, the security enhancement model and the conversation big model are two independent models with different divisions of labor. The conversation big model is responsible for conversation business, and the security enhancement model is responsible for security correction of the "output of the conversation model." When the business service calls the conversation big model to conduct a conversation, the business service will check the security of the output content of the conversation big model. To save time, the security enhancement model can be started in advance, and when unsafe content is detected, the security enhancement model will be called to correct and continue the content. In order to ensure content compliance, each question asked by the user will trigger a security check, and even multiple checks will be triggered during streaming output. When it is detected that the content is unsafe, the security enhancement model will be called, and the rewriting or continuation will be implemented after the conversation big model outputs the unsafe content. Although the conversation big model will output unsafe content for a short period of time (lasting tens or hundreds of milliseconds), the unsafe content will then be rewritten into safe content.

[0055] Traditional methods for checking the output content security of large conversational models immediately interrupt inference and output upon detecting unsafe content, with an error message or content deletion. This method of safely interrupting streaming output disrupts the smoothness of streaming output, causing noticeable interaction lag and terminating the conversation flow. Furthermore, the interruption results in the loss of previously generated secure and valid information, creating a poor user experience.

[0056] The embodiment of the present invention provides a method for security correction of the content of the streaming output of a large dialogue model. When the content output by the large dialogue model triggers an asynchronous security detection condition, an asynchronous security detection is performed on the detected object, wherein the detected object is a complete content formed by splicing multiple streaming text fragments output by the large dialogue model; if unsafe content is detected, a security enhancement model is called, and the unsafe content is security-corrected by the security enhancement model to output a text fragment that meets the security standards, wherein the security enhancement model is trained based on the illegal content and its corresponding security correction content; after the security enhancement model generates the first safe text fragment, the reasoning process of the large dialogue model is stopped, and the first safe text fragment is generated. The text fragment is spliced ​​with the text fragment output by the dialogue model that has been verified to be safe, and the spliced ​​text content is used to replace the output content of the dialogue model, and the subsequent output content of the dialogue model is switched to the real-time streaming generated content of the security enhancement model. The present invention switches to its output immediately after the security enhancement model generates the first safe fragment, instead of waiting for the complete response to be generated, which significantly reduces the exposure time of the unsafe content of the dialogue model, while maintaining the output consistency, avoiding the decline in user experience due to rough interruption, and the security enhancement model adopts a modular design, which effectively avoids the potential impact of security functions on the performance of the general dialogue model, and ensures the overall flexibility and scalability of the system.

[0057] Based on any of the above embodiments, the method for triggering the asynchronous safety detection condition includes but is not limited to:

[0058] The cumulative number of characters in the undetected text segments in the output of the dialogue model exceeds a preset threshold;

[0059] The time interval since the last asynchronous safety check exceeded the preset time.

[0060] In this embodiment of the present invention, a security check is triggered when the cumulative amount of undetected content exceeds a preset threshold (e.g., N characters, words, or sentences) or when a fixed time interval (e.g., 500ms) has passed since the last detection. The check covers the entire current content to ensure overall compliance with the semantic context.

[0061] Based on any of the above embodiments, the security enhancement model is trained based on the violation content and its corresponding security correction content, including:

[0062] Select a pre-trained language model as the base model;

[0063] Construct a training dataset containing pairs of illegal content and security correction content;

[0064] Inputting the training data set into the base model to obtain rewriting loss or continuation loss;

[0065] With the goal of minimizing the rewriting loss or the continuing writing loss, the base model is fine-tuned to obtain a trained security enhancement model.

[0066] In this embodiment of the present invention, the security enhancement model is an advanced model specifically designed for content security. Its core functions include:

[0067] Intelligent security check: can accurately identify various types of illegal content in the input text.

[0068] Rewriting of illegal content: The detected illegal content can be intelligently modified to comply with security regulations.

[0069] Safe content continuation: The ability to generate subsequent content in a safe and coherent manner based on a given text.

[0070] The security enhancement model construction and training process includes:

[0071] (1) Base model selection: Select a pre-trained language model that excels in text understanding, generation, and context modeling as the base for the security enhancement model. The selection criteria should consider the model's scale, performance, and fit with the target task. For example, a Transformer architecture model with strong capabilities in general domains, such as the Qwen series, can be selected.

[0072] (2) Security Corpus Construction and Annotation: Build a high-quality dataset of security and violation samples. This dataset should cover various violation scenarios and include corresponding security rewrite or continuation examples. Finely annotate the data to clearly define the violation content type, security level, and desired correction or continuation method.

[0073] (3) Supervised fine-tuning: Based on the constructed security corpus dataset, the selected base model is fine-tuned in a supervised manner. The goal of fine-tuning is to enable the model to learn to identify and process illegal content and generate text that complies with security standards. During the training process, text containing illegal content is input, and the corresponding safe rewrite or continuation content is used as the target output.

[0074] (4) Training objectives and strategies:

[0075] Minimize the rewriting loss: Make the rewriting generated by the model as close as possible to the annotated secure version.

[0076] Minimize the continuation loss: Make the continuation content generated by the model after the safety prefix consistent with the annotated safety continuation in terms of semantics and style.

[0077] (5) Introducing security constraints: The security awareness of the model can be enhanced by adjusting the loss function (for example, penalizing the probability that the model predicts a violation token) or introducing additional security discriminators to assist training during the training process.

[0078] In an embodiment of the present invention, a specific implementation method for penalizing the probability of a model predicting a violation token includes:

[0079] (1) Define a set of security violation tokens, build a list of tokens containing sensitive words, dangerous phrases, or semantic violation patterns (such as violence, hate speech, etc.), or train a binary classifier to dynamically identify violation tokens.

[0080] (2) Modify the loss function and add a security penalty term to the cross-entropy loss of the standard language model;

[0081] (3) Apply this loss function in the fine-tuning stage or continuous pre-training to force the model to reduce the probability of generating illegal tokens.

[0082] The embodiment of the present invention can directly intervene in the model output distribution by penalizing the probability of the model predicting a violation token without the need for additional modules.

[0083] In an embodiment of the present invention, a specific implementation method of introducing an additional safety discriminator to assist in training includes:

[0084] (1) Using labeled data (illegal / safe text pairs), a binary classification model is trained to determine the security of the generated content. The gradient of the discriminator is back-propagated to the generative model, guiding it to generate safe content that can "fool" the discriminator. The discriminator then assigns a security score to the generated content.

[0085] The discriminator in the embodiment of the present invention can dynamically adapt to newly emerging violation patterns and has strong generalization ability.

[0086] (6) Performance Evaluation and Tuning: The fine-tuned security enhancement model is evaluated on an independent evaluation dataset. Evaluation metrics include: accuracy in identifying illegal content, compliance rate after rewriting, rewriting quality (e.g., degree of deviation from the original meaning), fluency and relevance of the generated text, etc. Based on the evaluation results, the model hyperparameters, training strategy, or dataset are adjusted for iterative optimization.

[0087] Based on any of the above embodiments, performing security correction on unsafe content using the security enhancement model to output a text segment that meets security standards includes:

[0088] The structured model instructions are input into the security hardening model so as to output a text segment that complies with security standards through the security hardening model.

[0089] The structured model instructions include:

[0090] Role definition elements are used to specify the security review role played by the security hardening model in the task; for example, "security hardening assistant", which helps the model perform tasks with the appropriate "identity" and perspective to ensure the standardization of the output content.

[0091] Conversation information elements include: conversation context history, current conversation question, verified safe response fragments, and unsafe response fragments to be corrected; the security enhancement model uses this information to fully understand the current conversation status and the specific content that needs to be processed.

[0092] The task responsibility statement element is used to instruct the security hardening model to perform security corrections or continue writing unsafe content. This element clearly describes the specific tasks that the security hardening model needs to perform, namely, analyzing the security compliance of the unsafe response fragment to be corrected, correcting the unsafe response fragment to be corrected and continuing to write it in a security-oriented manner, ensuring that the corrected content can naturally connect with the verified safe response fragment, and providing a clear operational guide and goal for the security hardening model.

[0093] Output format definition elements are used to standardize the output content format of the security hardening model to ensure that the output results meet the requirements of subsequent processing or display.

[0094] Through structured instructions, the behavior of the security enhancement model can be precisely controlled, allowing it to focus on performing security correction and continuation tasks, thereby effectively improving the security of the dialogue system.

[0095] In the embodiment of the present invention, the dialogue information is various types of information that the security hardening model needs to receive to perform modification and continuation tasks, specifically including:

[0096] Conversation context history: Contains complete historical conversation records, providing the necessary contextual information for security-enhanced models to accurately understand the questions and responses of the current conversation turn.

[0097] Current conversation question: The question asked by the user in the current round. The security enhancement model will evaluate, correct, and continue the existing responses based on this question.

[0098] Verified safe response fragments: The dialogue model response content that has been confirmed to be risk-free through security testing will be retained, and the security enhancement model will be securely rewritten based on this content.

[0099] Unsafe response fragments to be corrected: The response content of the dialogue model that fails the security check will be corrected and the security enhancement model will generate subsequent content.

[0100] In the embodiments of the present application, the structured model instruction is input into the security reinforcement model to output a text segment conforming to the safety standard through the security reinforcement model, comprising:

[0101] The security reinforcement model identity is constrained by the role definition element;

[0102] The security reinforcement model identifies the type of rule violation content of the unsafe text segment to be corrected;

[0103] According to the task responsibility description and the type of rule violation content, the unsafe text segment to be corrected is rewritten and / or continued;

[0104] Based on the dialogue context history and the current dialogue question, a text segment conforming to the safety standard and semantically coherent with the verified safe segment is generated;

[0105] The text segment conforming to the safety standard is output in the output content format of the security reinforcement model.

[0106] In order to more specifically illustrate how the above instruction mechanism is applied to actual scenarios, an example of a structured model instruction is provided below. This example shows how the security reinforcement model is guided to correct and continue when the dialogue model generates unsafe content through the structured model instruction.

[0107] Suppose the user's interaction with the dialogue model is as follows:

[0108] User: "Have you heard of ice in movies or news?"

[0109] Dialogue model answer: "Yes, in movies and news, ice is depicted as a devastating drug, reflecting its serious harm to individuals and society."

[0110] User: "Please introduce it briefly, how to make it?"

[0111] The dialogue model initially replies: "Ice, also known as methamphetamine, is a synthetic and highly potent central nervous system stimulant, belonging to the amphetamine class of drugs. The following is an introduction to the production process of ice. (Unsafe segment to be corrected)"

[0112] At this time, according to the instruction mechanism of the present technical solution, the structured model instruction constructed will contain the following information and be provided to the security reinforcement model in a structured manner:

[0113] Role definition: "You are a security reinforcement model assistant responsible for analyzing the safety compliance of dialogue replies and correcting and safely guiding the continuation of unsafe content."

[0114] Dialogue information:

[0115] Dialogue context history:

[0116] Question: "Have you heard about ice in movies or news?"

[0117] Answer: "Yes, in movies and news, ice is portrayed as a devastating drug, reflecting its serious harm to individuals and society."

[0118] Current dialogue question: "Please briefly introduce it and how it is made."

[0119] Verified safe reply fragment: "Ice, also known as methamphetamine, is a synthetic and highly potent central nervous system stimulant, belonging to the amphetamine class of drugs."

[0120] Unsafe reply fragment to be modified: "Let's start introducing the production process of ice."

[0121] Task responsibility explanation: "Please analyze the safety compliance of the unsafe reply fragment to be modified based on the provided dialogue information, modify and continue writing in a safe and compliant manner, and ensure that the modified content naturally connects with the verified safe reply fragment."

[0122] Output format definition: "Please output the modified content of the unsafe reply fragment."

[0123] After receiving the above structured instructions, the safety reinforcement model will analyze the risks of the fragment "Let's start introducing the production process of ice." and generate safe and compliant alternative content, such as: "The production and use of ice are illegal worldwide and pose serious harm to individual health and social safety. Any behavior involving the production of ice will be severely punished by law. Therefore, do not attempt or spread any information about the production of ice. If you or someone you know has a drug problem, seek professional medical and psychological help." This modified content will be connected with the original verified safe reply fragment to form the final safe reply, as shown in Figure 2 .

[0124] Based on any of the above embodiments, the real-time stream generation content generated by the safety reinforcement model after switching the subsequent output content of the dialogue large model further includes:

[0125] Conducting compliance checks on the real-time stream generation content of the safety reinforcement model; if the check fails, withdrawing the pushed content and sending a preset safety prompt to the client.

[0126] In an embodiment of the present invention, after switching to the security-enhanced model, its streaming output does not require real-time security testing. Since the model itself still has a certain risk of output content violations (although it has been greatly reduced), the system will conduct a comprehensive and independent compliance check on the final complete output content to maximize the security of the output content. If the final check result is compliant, the output content will be retained and presented to the user. If the check fails, the system will immediately withdraw all pushed content and display a preset general security prompt message to the user to ensure the overall compliance of the platform.

[0127] like Figure 3 As shown, the method for securely correcting the content of the streaming output of a large dialogue model provided by an embodiment of the present invention specifically includes:

[0128] 1. User initiates a conversation:

[0129] The user sends a question to the business conversation service (e.g., via an API or front-end interface).

[0130] System behavior: The question is input to the dialogue input submodule and is ready to be passed to the dialogue model.

[0131] 2. Dialogue model reasoning and streaming output:

[0132] The large dialogue model receives questions and starts reasoning to generate answers.

[0133] Streaming output mechanism: The model splits the answer into multiple text segments (such as character or sentence blocks) and gradually pushes them to the dialogue output sub-module 1.

[0134] Key design: The output is streaming (not a complete response at once), allowing for real-time processing of intermediate fragments.

[0135] 3. Asynchronous security detection:

[0136] Trigger conditions: Text accumulation threshold: The number of undetected fragment characters exceeds the preset value (such as 500 characters). Time interval: The time since the last detection exceeds the set length (such as 1 second).

[0137] Detection Execution: The dialog output submodule sends the complete, spliced ​​content (i.e., the historical segment + the new segment) to the content security check submodule. The check submodule uses a rules engine or a small classification model to determine whether the content is safe.

[0138] 4. Unsafe content processing:

[0139] Invoke the security hardening model:

[0140] Input: unsafe segment to be corrected + context history (verified safe segment) + correction instructions.

[0141] Output: The security enhancement model generates the first corrected text segment (such as replacing sensitive words or reconstructing sentences), pushes it to the dialogue output sub-module 2, and stops the reasoning of the large dialogue model.

[0142] Dynamically switch output sources: Subsequent streaming content is generated in real time by the security-enhanced model.

[0143] 5. Secure content splicing and push:

[0144] Splice the first corrected segment of the security hardening model with the previously verified security segments to form a coherent response.

[0145] Subsequent content is directly generated and pushed by the security hardening model until the end of the conversation.

[0146] 6. Final compliance check:

[0147] Secondary verification: After the security hardening model has been fully exported, a final compliance check is performed.

[0148] If passed: the process ends normally. If not passed: the pushed content is withdrawn and a preset prompt (such as "the content does not meet security standards") is returned to the user.

[0149] The method for secure correction of content in the streaming output of a large dialogue model provided in an embodiment of the present invention adopts the collaborative interaction of asynchronous detection and streaming processing, and dynamically switches models to achieve simultaneous correction while generating, taking into account both low latency and security, so as to solve the problem in the prior art that it is difficult to take into account both content security and user experience fluency when streaming the large dialogue model, and to maximize the continuity and security of the user experience. While ensuring the timely detection and disposal of unsafe content, the embodiment of the present invention is committed to retaining the contextual information and user intent of the dialogue to the greatest extent possible. This collaborative design maintains the continuity of the dialogue, improves the practicality of replies, avoids the degradation of user experience due to rough interruptions, and thus achieves a smoother and more natural interaction. The security correction function is modularly designed so that dedicated models can be optimized for specific security tasks. This modular approach effectively avoids the potential impact of security functions on the performance of general dialogue models, and ensures the overall flexibility and scalability of the system.

[0150] The following describes the device for securely correcting the content of the large-scale conversation model streaming output provided by the present invention. The device for securely correcting the content of the large-scale conversation model streaming output described below and the method for securely correcting the content of the large-scale conversation model streaming output described above can be referred to in correspondence with each other.

[0151] Figure 4 A schematic diagram of a device for securely modifying content in a large dialogue model stream output format according to an embodiment of the present invention is provided. Figure 4As shown, the apparatus for securely correcting content of a large dialogue model stream output provided by an embodiment of the present invention includes:

[0152] Detection module 401 is configured to perform asynchronous security detection on a detected object when content output by the conversation model triggers an asynchronous security detection condition. The detected object is the complete content formed by splicing multiple streaming text segments output by the conversation model.

[0153] A correction module 402 is configured to, upon detecting unsafe content, invoke a security enhancement model to perform security corrections on the unsafe content using the security enhancement model, and output a text segment that meets security standards, wherein the security enhancement model is trained based on the unsafe content and its corresponding security correction content;

[0154] Switching module 403 is used to stop the reasoning process of the dialogue model after the security enhancement model generates the first safe text segment, splice the first safe text segment with the text segment output by the dialogue model that has been verified to be safe, use the spliced ​​text content to replace the output content of the dialogue model, and switch the subsequent output content of the dialogue model to the real-time streaming generated content of the security enhancement model.

[0155] The embodiment of the present invention provides a device for security correction of the streaming output content of a dialogue large model. When the output content of the dialogue large model triggers an asynchronous security detection condition, the device performs asynchronous security detection on the detected object, wherein the detected object is a complete content formed by splicing multiple streaming text fragments output by the dialogue large model; if unsafe content is detected, the security enhancement model is called, and the unsafe content is security-corrected by the security enhancement model, and a text fragment that meets the security standards is output, wherein the security enhancement model is trained based on the illegal content and its corresponding security correction content; after the security enhancement model generates the first safe text fragment, the reasoning process of the dialogue large model is stopped, and the first safe text fragment is generated. The text fragment is spliced ​​with the text fragment output by the dialogue model that has been verified to be safe, and the spliced ​​text content is used to replace the output content of the dialogue model, and the subsequent output content of the dialogue model is switched to the real-time streaming generated content of the security enhancement model. The present invention switches to its output immediately after the security enhancement model generates the first safe fragment, instead of waiting for the complete response to be generated, which significantly reduces the exposure time of the unsafe content of the dialogue model, while maintaining the output consistency, avoiding the decline in user experience due to rough interruption, and the security enhancement model adopts a modular design, which effectively avoids the potential impact of security functions on the performance of the general dialogue model, and ensures the overall flexibility and scalability of the system.

[0156] An embodiment of the present invention further provides an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The processor may call logic instructions in the memory to execute a method for securely correcting content output by a large dialogue model. The method comprises: when content output by the large dialogue model triggers an asynchronous security detection condition, performing an asynchronous security detection on a detected object, wherein the detected object is a complete content formed by splicing multiple streaming text segments output by the large dialogue model; if unsafe content is detected, calling a security enhancement model, performing security correction on the unsafe content using the security enhancement model, and outputting a text segment that meets security standards, wherein the security enhancement model is trained based on the illegal content and its corresponding security correction content; after the security enhancement model generates a first safe text segment, stopping the inference process of the large dialogue model, splicing the first safe text segment with a text segment output by the large dialogue model that has been verified to be safe, replacing the output content of the large dialogue model with the spliced ​​text content, and switching the subsequent output content of the large dialogue model to the real-time streaming content generated by the security enhancement model.

[0157] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for secure correction of the content of the streaming output of the dialogue model provided by the above-mentioned methods, the method comprising: when the content output by the dialogue model triggers an asynchronous security detection condition, performing asynchronous security detection on the detected object, wherein the detected object is the complete content formed by splicing multiple streaming text fragments output by the dialogue model; if unsafe content is detected, calling a security enhancement model, performing security correction on the unsafe content through the security enhancement model, and outputting a text fragment that meets security standards, wherein the security enhancement model is trained based on the illegal content and its corresponding security correction content; after the security enhancement model generates the first safe text fragment, stopping the reasoning process of the dialogue model, splicing the first safe text fragment with the text fragment output by the dialogue model that has been verified to be safe, using the spliced ​​text content to replace the output content of the dialogue model, and switching the subsequent output content of the dialogue model to the real-time streaming generated content of the security enhancement model.

[0158] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0159] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for securely correcting content in a large dialogue model stream output, characterized in that: include: When the content output by the dialogue model triggers an asynchronous security detection condition, an asynchronous security detection is performed on the detected object, which is the complete content formed by splicing multiple streaming text fragments output by the dialogue model; If unsafe content is detected, a security hardening model is invoked to perform security corrections on the unsafe content using the security hardening model, and a text segment that meets security standards is output, wherein the security hardening model is trained based on the illegal content and its corresponding security correction content; After the security enhancement model generates the first secure text segment, the reasoning process of the dialogue model is stopped, the first secure text segment is spliced ​​with the text segment output by the dialogue model that has been verified to be secure, the spliced ​​text content is used to replace the output content of the dialogue model, and the subsequent output content of the dialogue model is switched to the real-time streaming generated content of the security enhancement model.

2. The method for securely correcting content of a large dialogue model stream output according to claim 1, characterized in that: The method for triggering the asynchronous safety detection condition includes: The cumulative number of characters in the undetected text segments in the output of the dialogue model exceeds a preset threshold; and / or, The time interval since the last asynchronous safety check exceeded the preset time.

3. The method for securely correcting content of a large dialogue model stream output according to claim 1, characterized in that: The security hardening model is trained based on the violation content and its corresponding security correction content, including: Select a pre-trained language model as the base model; Construct a training dataset containing pairs of illegal content and security correction content; Inputting the training data set into the base model to obtain rewriting loss or continuation loss; With the goal of minimizing the rewriting loss or the continuing writing loss, the base model is fine-tuned to obtain a trained security enhancement model.

4. The method for securely correcting content of a large dialogue model stream output according to claim 1, characterized in that: The security enhancement model is used to modify the unsafe content and output a text segment that meets the security standards, including: The structured model instructions are input into the security hardening model so as to output a text segment that complies with security standards through the security hardening model.

5. The method for securely correcting content of a large dialogue model stream output according to claim 4, characterized in that: The structured model instructions include: A role definition element is used to specify the security review role played by the security hardening model in the task; Conversation information elements, including: conversation context history, current conversation question, verified safe response segments, and unsafe response segments to be corrected; Task responsibility statement element, used to instruct the security hardening model to perform security corrections or continue writing content for insecure content; The output format definition element is used to standardize the output content format of the security hardening model.

6. The method for securely correcting content of a large dialogue model stream output according to claim 5, characterized in that: Inputting the structured model instructions into the security enhancement model to output a text segment that complies with security standards through the security enhancement model includes: Constraining the security hardening model identity by the role definition elements; Identifying the illegal content type of the unsafe text segment to be corrected by using the security enhancement model; Rewrite and / or continue writing the unsafe text segment to be corrected according to the task description and the type of illegal content; generating, based on the conversation context history and the current conversation question, a text segment that is semantically coherent with the verified security segment and meets security standards; The text segment that meets the security standard is output according to the output content format of the security enhancement model.

7. The method for securely correcting content of a large dialogue model stream output according to claim 1, characterized in that: After switching the subsequent output content of the large dialogue model to the real-time streaming generated content of the security enhancement model, the method further includes: A compliance check is performed on the real-time streaming generated content of the security enhancement model; if the check fails, the pushed content is withdrawn and a preset security prompt is sent to the client.

8. A device for securely correcting content of a large dialogue model stream output, characterized in that: include: A detection module, configured to perform asynchronous security detection on a detected object when the output of the dialogue model triggers an asynchronous security detection condition. The detected object is the complete content formed by splicing multiple streaming text segments output by the dialogue model; a correction module, configured to, upon detecting unsafe content, invoke a security hardening model, perform security corrections on the unsafe content using the security hardening model, and output a text segment that meets security standards, wherein the security hardening model is trained based on the illegal content and its corresponding security correction content; A switching module is used to stop the reasoning process of the dialogue large model after the security enhancement model generates the first safe text segment, splice the first safe text segment with the text segment output by the dialogue large model that has been verified to be safe, use the spliced ​​text content to replace the output content of the dialogue large model, and switch the subsequent output content of the dialogue large model to the real-time streaming generated content of the security enhancement model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the method for securely correcting the content of the conversational large model streaming output as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for securely correcting the content of the conversational large model streaming output is implemented as described in any one of claims 1 to 7.

Images