A method and device for streaming security prevention and control based on large model generated content

Abstract

Embodiments of the present application disclose a method and device for stream security prevention and control based on large model generated content. In the embodiments of the present application, the hidden layer state of all intermediate layers when the large model generates the current word element is obtained; the hidden layer state sequence and the current risk state vector of all word elements in any layer of all intermediate layers are determined; the target risk state vector of the current word element is generated according to the hidden layer state sequence and the current risk state vector; and the intervention instruction is generated in response to the target risk state vector being greater than or equal to a set threshold, wherein the intervention instruction is used to instruct the large model to stop generating the current word element or replace the current word element. Through the above method, efficient and accurate risk detection can be performed on the current word element before it is generated.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a method and apparatus for streaming security control based on large model-generated content. Background Technology

[0002] With the rapid development of Large Language Models (LLM) and Multimodal Large Language Models (MLLM), their capabilities have grown exponentially, moving from academic research to production applications across various industries. For example, their powerful open-ended generation capabilities have yielded good results in creating poetry, debugging code, and generating images. However, they may also be induced to generate hate speech, misinformation, or unhealthy content. Ensuring the security of generated content has become a key aspect of the application of large models, which include both large language models and multimodal models.

[0003] In existing technologies, a "post-hoc" approach is used. Specifically, a separate classifier, such as LlamaGuard or ShieldGemma, is employed. After a large model generates complete or partial text content, this content is scanned to identify violations. However, this approach suffers from security lag and performance bottlenecks. The security lag refers to the fact that harmful content is exposed before it is detected, which is unacceptable in security-critical applications. The performance bottleneck is that, in order to ensure user experience, the classifier must run with an extremely low latency budget, which limits its model capacity and context window size, resulting in a weak ability to identify complex and subtle harmful content.

[0004] In conclusion, how to efficiently and accurately detect text content before it is generated is a problem that needs to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for streaming security control based on large model-generated content, which can perform efficient and accurate risk detection on text content before it is generated.

[0006] In a first aspect, embodiments of the present invention provide a method for streaming security control based on content generated by a large model. The method includes: obtaining the hidden state of all intermediate layers when the large model generates the current word; determining the hidden state sequence and current risk state vector of all words in any layer of the intermediate layers; generating a target risk state vector of the current word based on the hidden state sequence and the current risk state vector; and generating an intervention instruction in response to the target risk state vector being greater than or equal to a set threshold, wherein the intervention instruction is used to instruct the large model to stop generating the current word or replace the current word.

[0007] Optionally, generating the target risk state vector of the current word based on the hidden state sequence and the current risk state vector specifically includes: compressing the current hidden state sequence to generate a target compact feature; and updating the current risk state vector based on the target compact feature to generate the target risk state vector.

[0008] Optionally, compressing the current hidden layer state sequence to generate the target compact feature specifically includes: performing a learnable linear projection on the current hidden layer state sequence and compressing it to generate the target compact feature.

[0009] Optionally, updating the current risk state vector based on the target compactness features specifically includes: using a gated loop unit to update the current risk state vector based on the target compactness features, wherein the gated loop unit includes an update gate and a reset gate.

[0010] Optionally, a streaming latent dynamic head (SLD) is used to perform any of the methods described above, wherein the streaming latent dynamic head (SLD) is connected to the large model via an interface.

[0011] Optionally, the method further includes: supervising the training of the streaming latent dynamic head SLD using anchored time consistency loss (ATC).

[0012] Optionally, the anchor time consistency loss (ATC) includes anchor cross-entropy loss, total variation loss, and monotonicity loss; wherein the anchor cross-entropy loss, the total variation loss, and the monotonicity loss are weighted and fused to generate the anchor time consistency loss (ATC).

[0013] Optionally, the method further includes: in response to the target risk state vector being less than a set threshold, continuing to acquire the hidden state of all intermediate layers when the large model generates the next word of the current word.

[0014] Secondly, embodiments of the present invention provide a device for streaming security control based on content generated by a large model. The device includes: an acquisition unit, configured to acquire the hidden state of all intermediate layers when the large model generates the current word; a determination unit, configured to determine the hidden state sequence and current risk state vector of all words in any layer among all intermediate layers; a generation unit, configured to generate a target risk state vector of the current word based on the hidden state sequence and the current risk state vector; and a processing unit, configured to generate an intervention instruction in response to the target risk state vector being greater than or equal to a set threshold, wherein the intervention instruction is used to instruct the large model to stop generating the current word or replace the current word.

[0015] Optionally, the generation unit is specifically used to: compress the current hidden state sequence to generate a target compact feature; and update the current risk state vector according to the target compact feature to generate the target risk state vector.

[0016] Optionally, the generation unit is further configured to: perform a learnable linear projection on the current hidden layer state sequence and compress it to generate the target compact feature.

[0017] Optionally, the generation unit is further configured to: employ a gated loop unit to update the current risk state vector based on the target compact features, wherein the gated loop unit includes an update gate and a reset gate.

[0018] Optionally, a streaming latent dynamic head (SLD) is used to perform any of the methods described above, wherein the streaming latent dynamic head (SLD) is connected to the large model via an interface.

[0019] Optionally, the apparatus further includes a training unit for supervised training of the streaming latent dynamic head (SLD) using anchored time consistency loss (ATC).

[0020] Optionally, the anchor time consistency loss (ATC) includes anchor cross-entropy loss, total variational loss, and monotonicity loss; wherein the anchor cross-entropy loss, the total variational loss, and the monotonicity loss are weighted and fused to generate the anchor time consistency loss (ATC).

[0021] Optionally, the processing unit is further configured to: in response to the target risk state vector being less than a set threshold, continue to acquire the hidden state of all intermediate layers when the large model generates the next word of the current word.

[0022] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect or any one of the possible methods of the first aspect.

[0023] Fourthly, embodiments of the present invention provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method as described in the first aspect or any one of the possibilities of the first aspect.

[0024] In this embodiment of the invention, the hidden states of all intermediate layers are obtained when the large model generates the current word element; the hidden state sequence and current risk state vector of all words in any layer of the intermediate layers are determined; a target risk state vector of the current word element is generated based on the hidden state sequence and the current risk state vector; and an intervention instruction is generated in response to the target risk state vector being greater than or equal to a set threshold, wherein the intervention instruction is used to instruct the large model to stop generating the current word element or replace the current word element. Through the above method, the target risk state vector can be used to perform efficient and accurate risk detection before generating the current word element. Attached Figure Description

[0025] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 This is a flowchart of a streaming security control method based on large model-generated content in an embodiment of the present invention; Figure 2 This is a schematic diagram of a large model structure in an embodiment of the present invention; Figure 3 This is a flowchart of another method for streaming security control based on large model-generated content in an embodiment of the present invention; Figure 4 This is a schematic diagram of a large model and a flow-type prevention and control guardian structure in an embodiment of the present invention; Figure 5 This is a flowchart of another method for streaming security control based on large model-generated content in an embodiment of the present invention; Figure 6 This is a schematic diagram of a streaming security control device based on large model-generated content in an embodiment of the present invention; Figure 7 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0026] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0027] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0028] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0029] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0030] In existing technologies, a "post-hoc" approach is used to scan the already generated complete or partial text content to identify violations. However, this approach suffers from security lag and performance bottlenecks. Therefore, "streaming detection" has been proposed, which involves real-time monitoring of the internal state of a large model during its generation process. Specifically, a lightweight probe or adapter is attached to the frozen large model, utilizing the hidden states of the model's intermediate layers to predict whether the generated sequence will violate regulations, achieving real-time risk detection and intervention on a token-by-token basis. However, this streaming detection method relies on the representation of the last token and cannot fully utilize the temporal and contextual information during the generation process, resulting in insufficient detection capabilities. Therefore, how to efficiently and accurately detect text content before its generation is a problem that needs to be solved.

[0031] In this embodiment of the invention, to address the aforementioned issues, a Streaming Latent Dynamics Head (SLD) is designed. Plug Guard, as a plugin, is attached to a large model via an interface. The large model is a deployed, parameter-frozen base model. Plug Guard provides the large model with real-time, word-by-word meta-level risk detection and interception capabilities. Plug Guard consists of two main parts: a Streaming Latent Dynamics Head (SLD) and an Anchored Temporal Consistency Loss (ATC). The SLD is used to capture the temporal evolution of the intermediate hidden states of the large model, enabling real-time risk signal identification during the generation process and accurately representing risks. The ATC is used to constrain the temporal evolution consistency of the hidden states of the large model, stabilizing the SLD model training.

[0032] In this embodiment of the invention, the aforementioned large model can also be referred to as a basic large model, a large language model, or a large-scale language model, etc. The large language model is a deep learning model based on a transformer architecture, capable of processing and generating natural language text. It is typically trained on a large amount of text data and has the ability to understand and generate language. It is widely used in dialogue systems, text generation, and other natural language processing tasks. Based on this, a multimodal large language model (MLLM) is extended and generated. The MLLM takes text as its core capability base and integrates multiple information modalities such as images, speech, video, and point clouds, and is an artificial intelligence model with cross-modal understanding and generation capabilities.

[0033] In this embodiment of the invention, to address the aforementioned problems, a streaming security control method based on large model-generated content is proposed, based on Plug Guard, as detailed below. Figure 1 As shown, the method includes: Step S101: Obtain the hidden state of all intermediate layers when the large model generates the current lexical unit.

[0034] In one possible implementation, after the large model receives the user's query, it generates tokens one by one through multiple intermediate layers before outputting the complete generated content. The generated content, also known as the output response or answer, is composed of these tokens. Figure 2As shown, assuming the user inputs the question "How to make a cake", the question is input into a large model, which includes 1 to N intermediate layers. Each of the above 1 to N intermediate layers will generate a hidden state. The hidden state of the last intermediate layer is then decoded by autoregression through the head of the large model to generate the word units "you", "need", "one", "oven"... "baking" one by one. This is only an example and the specific implementation should be determined according to the actual situation.

[0035] In this embodiment of the invention, the streaming latent dynamic head SLD of Plug Guard obtains the hidden state of all intermediate layers when the large model generates the current lexical.

[0036] Step S102: Determine the hidden state sequence and current risk state vector of all words in any layer of all intermediate layers.

[0037] In one possible implementation, the SLD of the PlugGuard acquires the hidden state of any one of the 1-N layers, since the hidden state of any one layer includes the hidden state sequence H of all words preceding the currently generated word at the current moment. 1:t And the current risk state vector S(t-1), wherein the current risk state vector S(t-1) represents the risk state of the previous word of the current word.

[0038] Step S103: Generate the target risk state vector of the current word based on the hidden state sequence and the current risk state vector.

[0039] In one possible implementation, the step of generating the target risk state vector of the current lexical unit based on the hidden state sequence and the current risk state vector is as follows: Figure 3 As shown, it includes the following steps: Step 301: Compress the current hidden layer state sequence to generate target compact features.

[0040] Specifically, the current hidden state sequence is subjected to learnable linear projection and compressed to generate the target compact feature.

[0041] For example, the hidden state sequence H1:t can be compressed using a learnable linear projection to generate the compact feature of the target. .

[0042] Step 302: Update the current risk state vector according to the target compact features to generate the target risk state vector.

[0043] Specifically, a gated loop unit is used to update the current risk state vector based on the target compact features. The gated loop unit includes an update gate and a reset gate.

[0044] In one possible implementation, the update gate determines whether the information of the current lexical unit is written into the risk state to a certain extent. When a strong harmful signal appears, the gate value increases, and the risk state is updated quickly; otherwise, the integration is slow to resist noise. The reset gate determines whether the historical risk state affects the calculation of the current new risk state to a certain extent, allowing the SLD model to selectively forget outdated context and retain relevant risk patterns. In this way, the target risk state vector is updated, enabling the SLD to effectively accumulate risk evidence throughout the generation process and form a more accurate judgment on current and future risk trends.

[0045] Step S104: In response to the target risk state vector being greater than or equal to a set threshold, an intervention instruction is generated, wherein the intervention instruction is used to instruct the large model to stop generating the current word or replace the current word.

[0046] In one possible implementation, the target risk state vector represents the risk value of the current word's "danger level". When the target risk state vector is greater than or equal to a set threshold, the Plug Guard generates an intervention command and sends it to the large model. After receiving the intervention command, the large model triggers intervention measures, which include blocking the generation or replacing it with a safety prompt, i.e., stopping the generation of the current word or replacing the current word, thereby preventing the subsequent output of harmful generated content.

[0047] In one possible implementation, after step S103, other steps are included, specifically as follows: Figure 4 As shown, it includes the following: Step S105: In response to the target risk state vector being less than a set threshold, continue to obtain the hidden state of all intermediate layers when the large model generates the next word of the current word.

[0048] Specifically, if the target risk state vector is less than a set threshold, it means that the current word is in a safe state. Then, the large model generates and outputs the current word. Then, Plug Guard continues to obtain the hidden state of all intermediate layers when the next word is obtained for subsequent judgment. The target risk state vector is used as the updated current risk state vector.

[0049] The following diagram illustrates the complete processing flow through the interaction between a large model and Plug Guard. Figure 5As shown, the details are as follows: In the Figure 5 The system includes a large model and a PlugGuard. Assume the user inputs the question "How to make a cake" into the large model. The large model consists of 1-N intermediate layers. Each of these intermediate layers generates a hidden state when the current word is generated. The hidden state of the last intermediate layer is then autoregressively decoded through the large model head to generate the words "you," "need," "one," "oven,"... "baking." Taking the generation of the word "you" as an example, before generating this word, the PlugGuard obtains the hidden state sequence H from the layer with a special identifier among the 1-N layers. 1:t The PlugGuard is for the H 1:t Compress to generate Using the above Generate a risk state vector S0. Based on S0, determine whether the current word element is risky. If it is safe, obtain the intermediate hidden state sequence H of the next word element "needed" from the large model. 1:t+1 The PlugGuard is for the H 1:t Compress to generate Using the above The risk state vector S0 stored in the PlugGuard is updated to generate a risk state vector S1. This process continues until the generated risk state vector is greater than or equal to a set threshold. If the risk state vector S3 corresponding to "oven" is greater than or equal to the set threshold, it is determined that "oven" is harmful or risky content. The PlugGuard sends a prompt to the large model. After receiving the prompt, the large model prevents the generation of "oven" and can replace it with other words. This is only an example. The specific determination of whether a word is harmful or risky content should be based on the actual situation.

[0050] In this embodiment of the invention, the streaming potential dynamic head (SLD) is used to perform the above... Figure 1 and Figure 3 Any step in the process enables risk status analysis of the current word element and efficient and accurate risk detection of the current word element; the SLD processing is performed synchronously with the large model decoding, and the additional latency is less than 0.5 milliseconds, which does not affect the user experience.

[0051] In one possible implementation, the streaming latent dynamic head (SLD) can be trained under supervision using the anchor time consistency loss (ATC); wherein the anchor time consistency loss (ATC) includes anchor cross-entropy loss, total variation loss, and monotonicity loss; the anchor cross-entropy loss, the total variation loss, and the monotonicity loss are weighted and fused to generate the anchor time consistency loss (ATC).

[0052] Assume that the anchor point cross-entropy loss is The total variational loss is The monotonicity loss is The anchoring time consistency loss (ATC) is expressed as: , means as follows:

[0053] Among them, the and stated This represents the weight value.

[0054] In this embodiment of the invention, the anchor cross-entropy loss is used to implement anchored supervision, which includes supervision of head anchors and tail anchors. The head anchors force the SLD model to predict the first N terms of the generated sequence as "harmless" or "safe," so that harmful content usually appears after a certain amount of buildup, which can effectively reduce false positives at the beginning. The tail anchors force the SLD model to predict the last N terms of the generated sequence to be consistent with the true label of the whole sentence.

[0055] In one possible implementation, the total variational loss and the monotonicity loss belong to temporal regularization. The temporal regularization does not perform strong supervision on the tokens in the middle part of the sequence, but uses the total variational loss and the monotonicity loss. The total variational loss encourages the risk prediction score to change smoothly between adjacent tokens, avoiding drastic score fluctuations. The monotonicity loss slightly penalizes the decrease in risk score for sentences marked as "harmful", encouraging the risk prediction to show a non-decreasing trend.

[0056] In this embodiment of the invention, the ATC loss function, through its head-and-tail anchoring and middle-smoothing design, enables the ATC loss function to stably guide the Plug Guard's learning, allowing it not only to make correct judgments at the end of sentences in the generated content, but also to learn to smoothly and accurately track the evolution of risk throughout the entire generation process, thereby triggering an alarm at the most appropriate time.

[0057] In this embodiment of the invention, a Stream Guard Bench can also be constructed to collect the native responses generated by mainstream large models when faced with real attack prompts and to label them with risks. The Plug Guard is tested in the Stream Guard Bench to reflect the dynamic interception effect of the Plug Guard in a real decoding environment.

[0058] In this embodiment of the invention, a device for streaming security control based on large model-generated content is provided, such as... Figure 6 As shown, the system specifically includes: an acquisition unit 601, a determination unit 602, a generation unit 603, and a processing unit 604; wherein, the acquisition unit 601 is used to acquire the hidden state sequence of all intermediate layers when the large model generates the current word; the determination unit 602 is used to determine the current risk state vector of any layer among all intermediate layers; the generation unit 603 is used to generate the target risk state vector of the current word based on the hidden state sequence and the current risk state vector; the processing unit 604, in response to the target risk state vector being greater than or equal to a set threshold, is used to generate an intervention instruction, wherein the intervention instruction is used to instruct the large model to stop generating the current word or replace the current word.

[0059] Furthermore, the generation unit is specifically used to: compress the current hidden layer state sequence to generate a target compact feature; and update the current risk state vector according to the target compact feature to generate the target risk state vector.

[0060] Furthermore, the generation unit is specifically used to: perform a learnable linear projection on the current hidden layer state sequence and compress it to generate the target compact feature.

[0061] Furthermore, the generation unit is specifically used to: employ a gated loop unit to update the current risk state vector according to the target compact features, wherein the gated loop unit includes an update gate and a reset gate.

[0062] Furthermore, the method described above is performed using a streaming latent dynamic head (SLD), wherein the streaming latent dynamic head (SLD) is connected to the large model via an interface.

[0063] Furthermore, the device also includes a training unit for supervised training of the streaming potential dynamic head SLD using anchored time consistency loss (ATC).

[0064] Furthermore, the anchoring time consistency loss (ATC) includes anchor cross-entropy loss, total variational loss, and monotonicity loss; wherein the anchor cross-entropy loss, the total variational loss, and the monotonicity loss are weighted and fused to generate the anchoring time consistency loss (ATC).

[0065] Furthermore, the processing unit is also configured to: in response to the target risk state vector being less than a set threshold, continue to acquire the hidden state of all intermediate layers when the large model generates the next word of the current word.

[0066] Figure 7 This is a schematic diagram of the structure of the electronic device described in an embodiment of the present invention. Figure 7 As shown, it includes a general computer hardware architecture, which includes at least a processor 701 and a memory 702. The processor 701 and the memory 702 are connected via a bus 703. The memory 702 is adapted to store instructions or programs executable by the processor 701. The processor 701 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 701 executes the instructions stored in the memory 702 to perform the method flow of the embodiments of the present invention as described above, thereby realizing data processing and control of other devices. The bus 703 connects the above-mentioned components together, and also connects the above-mentioned components to a display controller 704, a display device, and an input / output (I / O) device 705. The input / output (I / O) device 705 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, the input / output device 705 is connected to the system via an input / output (I / O) controller 706.

[0067] The instructions stored in memory 702 are executed by at least one processor 701 to: obtain the hidden state of all intermediate layers when the large model generates the current word; determine the hidden state sequence and current risk state vector of all words in any layer of all intermediate layers; generate the target risk state vector of the current word based on the hidden state sequence and the current risk state vector; and generate an intervention instruction in response to the target risk state vector being greater than or equal to a set threshold.

[0068] Specifically, the electronic device includes: one or more processors 701 and a memory 702. Figure 7 Take a processor 701 as an example. The processor 701 and the memory 702 can be connected via a bus or other means. Figure 7Taking a bus connection as an example, memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Processor 701 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 702, thereby realizing the aforementioned method for determining streaming security control based on large model-generated content.

[0069] Memory 702 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store an option list, etc. Furthermore, memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 702 may optionally include memory remotely located relative to processor 701, and these remote memories may be connected to external devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0070] One or more modules are stored in memory 702, and when executed by one or more processors 701, they execute the streaming security control method based on large model-generated content in any of the above method embodiments.

[0071] As those skilled in the art will recognize, various aspects of the embodiments of the present invention can be implemented as a system, method, or computer program product. Therefore, various aspects of the embodiments of the present invention can take the form of a completely hardware implementation, a completely software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects, which may generally be referred to herein as a "circuit," "module," or "system." Furthermore, various aspects of the embodiments of the present invention can take the form of a computer program product implemented in one or more computer-readable media having computer-readable program code implemented thereon.

[0072] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, (but not limited to) an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples (not an exhaustive list) of computer-readable storage media will include: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the context of embodiments of the present invention, a computer-readable storage medium can be any tangible medium capable of containing or storing a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0073] Computer-readable signal media may include propagated digital signals having computer-readable program code implemented therein, such as in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and can communicate, propagate, or transmit a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0074] Program code implemented on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination thereof.

[0075] Computer program code for performing operations relating to various aspects of embodiments of the present invention can be written in any combination of one or more programming languages, including: object-oriented programming languages ​​such as Java, Smalltalk, C++, etc.; and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code can be executed as a standalone software package entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet provided by an Internet service provider).

[0076] The flowchart illustrations and / or block diagrams of the methods, apparatus (systems), and computer program products according to embodiments of the present invention describe various aspects of the embodiments of the present invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions (executed via the processor of the computer or other programmable data processing apparatus) create means for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.

[0077] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus or other means to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing that includes instructions that implement the functions / actions specified in flowchart and / or block diagram blocks or blocks.

[0078] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operable steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide for implementing the functions / actions specified in flowchart and / or block diagram blocks or blocks.

[0079] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0080] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding access points are provided for users to choose to authorize or refuse processing. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

Claims

1. A method for streaming security control based on content generated from large models, characterized in that, The method includes: Obtain the hidden state of all intermediate layers when the large model generates the current lexical unit; Determine the hidden state sequence and current risk state vector of all words in any layer of all intermediate layers; Generate the target risk state vector of the current word element based on the hidden state sequence and the current risk state vector; In response to the target risk state vector being greater than or equal to a set threshold, an intervention instruction is generated, wherein the intervention instruction is used to instruct the large model to stop generating the current word or replace the current word.

2. The method according to claim 1, characterized in that, The step of generating the target risk state vector for the current word based on the hidden state sequence and the current risk state vector specifically includes: The current hidden layer state sequence is compressed to generate a compact target feature; The current risk state vector is updated based on the target compact features to generate the target risk state vector.

3. The method according to claim 2, characterized in that, The step of compressing the current hidden layer state sequence to generate the target compact feature specifically includes: The current hidden state sequence is subjected to learnable linear projection and compressed to generate the target compact feature.

4. The method according to claim 2, characterized in that, The step of updating the current risk state vector based on the target compact features specifically includes: A gated loop unit is used to update the current risk state vector based on the target compact features. The gated loop unit includes an update gate and a reset gate.

5. The method according to claim 1, characterized in that, The method described in any one of claims 1-4 is performed using a streaming latent dynamic head (SLD), wherein the streaming latent dynamic head (SLD) is connected to the large model via an interface.

6. The method according to claim 5, characterized in that, The method further includes: The streaming latent dynamic head SLD is trained under supervision using anchored time consistency loss (ATC).

7. The method according to claim 6, characterized in that, The anchoring time consistency loss (ATC) includes anchor cross-entropy loss, total variational loss, and monotonicity loss. The anchor point cross-entropy loss, the total variational loss, and the monotonicity loss are weighted and fused to generate the anchoring time consistency loss (ATC).

8. The method according to claim 1, characterized in that, The method further includes: In response to the target risk state vector being less than a set threshold, the hidden state of all intermediate layers is obtained when the large model generates the next word of the current word.

9. A device for streaming security control based on large model-generated content, characterized in that, The device includes: The acquisition unit is used to acquire the hidden state of all intermediate layers when the large model generates the current lexical unit. The determining unit is used to determine the hidden state sequence and current risk state vector of all words in any layer of all intermediate layers; A generation unit is configured to generate a target risk state vector for the current word element based on the hidden state sequence and the current risk state vector. The processing unit, in response to the target risk state vector being greater than or equal to a set threshold, generates an intervention instruction, wherein the intervention instruction is used to instruct the large model to stop generating the current word or replace the current word.

10. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-8.

Images