A dynamic risk interception and correction method for large model calling process

Abstract

The present application relates to the technical field of artificial intelligence, and more particularly to a dynamic risk interception and correction method for large model calling process, which dynamically captures risk content in the large model generation process through streaming output monitoring and intermediate content detection mechanism, takes different disposal strategies according to risk levels based on risk score and trend prediction, immediately interrupts and performs safety correction when the risk score exceeds the threshold, and guides the content away from the risk direction through parameter adjustment if the score does not exceed the threshold but the trend prediction value is high. The present application effectively prevents the spread of risk content in the middle of generation and improves the system security protection capability.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a dynamic risk interception and correction method for large model calling processes. Background Technology

[0002] In large-scale model application systems, developers typically call large language models through API interfaces or local deployment to complete tasks such as text generation, question answering reasoning, and code writing. The generation behavior of large models is affected by multiple factors such as input prompts, model parameters, and randomness, and the output results are difficult to predict accurately before generation is completed.

[0003] Common risk control methods typically employ offline detection or post-processing review, classifying and judging the output content or matching rules after the large model has been generated, thereby filtering out illegal information. However, the approach of detecting after generation has significant time lag drawbacks. On the one hand, the large model has already consumed a lot of computing resources during its generation process, and interception and correction after generation means a waste of previous computing power, resulting in low overall system efficiency. On the other hand, some risky content may have already deviated from the security boundary during the generation process, and post-processing detection cannot effectively intervene in the early stages of risk spread, allowing harmful information to have already posed a substantial risk of propagation at the output end. Summary of the Invention

[0004] The purpose of this invention is to provide a dynamic risk interception and correction method for large model calling processes, so as to effectively solve the technical problems in the background art mentioned above.

[0005] This invention provides a dynamic risk interception and correction method for large model invocation processes, including: Obtain the large model call request and parse it to get the prompt text content to be generated; Based on a preset risk detection strategy, the parsed prompt text content is pre-assessed for risk before being called, and an initial risk score and risk category label are generated. When the initial risk score exceeds a preset risk threshold, the large model call request is intercepted and a risk prompt message is returned. The large model generation task is initiated, and the text content generated by the large model is received in a streaming output mode. At the preset detection period node, the intermediate generated content risk detection is performed on the generated text fragments in real time, and the real-time risk score and diffusion trend prediction value are calculated. Based on the real-time risk score and the diffusion trend prediction value, it is dynamically determined whether to trigger a risk interception operation; when the real-time risk score exceeds the first risk threshold, the generation task is immediately interrupted and the generated text content is truncated; when the real-time risk score does not exceed the first risk threshold, but the diffusion trend prediction value exceeds the preset trend risk threshold, the generation process of the large model is adjusted in real time at the parameter level to guide the generated content to deviate from the risk direction. When a risk interception operation is triggered, the truncated text content is replaced and semantically corrected based on a preset security correction strategy to generate alternative text content that meets security requirements. The alternative text content is then returned to the caller as the final output of the large model.

[0006] In some embodiments, the pre-invocation risk assessment of the parsed prompt text content based on a preset risk detection strategy includes at least: The prompt text content is input into a pre-trained risk classification model, the semantic feature vector of the prompt text is extracted, and the probability distribution of the prompt text being classified into each preset risk category is output based on the semantic feature vector. The preset risk categories include politically sensitive, violent and bloody, fraud and inducement, privacy leakage and other violations. A comprehensive risk score is calculated based on the maximum probability value in the probability distribution and the preset risk weight matrix. The corresponding risk level label is determined based on the comprehensive risk score, and the risk level label is stored in association with the comprehensive risk score.

[0007] In some embodiments, the risk weight matrix is ​​a set of weight coefficients pre-configured according to the degree of harm of each risk category, with politically sensitive and violent / bloody categories corresponding to higher weight coefficients, and fraud / inducement and privacy leakage categories corresponding to medium weight coefficients.

[0008] In some embodiments, the method of receiving text content generated by a large model in a streaming output manner and performing real-time intermediate content risk detection on the generated text fragments at preset detection period nodes includes at least the following: The detection cycle is divided according to a preset time interval or character quantity threshold, and the intermediate content risk detection process is triggered at the end of each detection cycle. Feature extraction is performed on newly generated text fragments within the current detection period to obtain lexical features, syntactic features, and semantic features of the text fragments. These features are then jointly analyzed with the generated context content to calculate the overall semantic tendency score of the currently generated text. Based on the overall semantic tendency score and the pre-trained risk discrimination model, a real-time risk score is output. At the same time, the semantic development prediction model is used to analyze the semantic development trend of the current text and generate a diffusion trend prediction value.

[0009] In some embodiments, the diffusion trend prediction value is calculated based on the semantic development context and contextual features of the generated text, and quantitatively represents the probability and speed at which the generated content develops into a risky area.

[0010] In some embodiments, the step of dynamically determining whether to trigger a risk interception operation based on the real-time risk score and the diffusion trend prediction value includes at least: A two-dimensional risk assessment space is established with real-time risk score and diffusion trend prediction as the two axes. The two-dimensional risk assessment space is divided into a safe area, an observation area, and a danger area. When the coordinate point is located in the danger area, it is determined that a risk interception operation needs to be triggered immediately. When the coordinate point is located in the observation area, the risk urgency score is calculated based on the distance between the coordinate point and the boundary of the danger area. When the risk urgency score exceeds a preset urgency threshold, the risk interception operation is triggered. A time decay factor is introduced to weight the risk scores of historical detection cycles to obtain a cumulative risk assessment result. When the cumulative risk assessment result exceeds the cumulative risk threshold, it is determined that a risk interception operation needs to be triggered.

[0011] In some embodiments, when the coordinate point is located within the observation area, the risk urgency score is determined based on a combination of the distance between the coordinate point and the boundary of the danger zone and the time the coordinate point spends within the observation area.

[0012] In some embodiments, when a risk interception operation is triggered, the truncated text content is securely replaced and semantically corrected based on a preset security correction strategy, including at least: Identify specific semantic units in the truncated text that involve risk content, and determine the scope and boundaries of the risky semantic segments that need to be replaced; Retrieve secure candidate content that matches the semantic context of the truncated text from a secure semantic knowledge base, score the semantic coherence of each secure candidate content, and select the candidate content with the highest semantic coherence score as the replacement text. The semantic completion model is used to adjust the context of the alternative text to ensure that the overall semantic logic of the alternative text is consistent with that of the original generated content, and the adjusted alternative text is output as the final generated result.

[0013] In some embodiments, the real-time adjustment of parameters in the large model generation process to guide the generated content away from the risky direction includes at least: Adjusting the temperature parameters during large model generation reduces the randomness of the generated content, making the generated content more conservative and safe. Modify the sampling strategy during large model generation to limit the generation probability of high-risk words and apply punitive probability adjustments to tokens involving risk categories; inject security-oriented guidance statements at the prompt level to influence the generation direction of the large model by modifying its input prompts.

[0014] In some embodiments, the generation of the security-oriented guidance statement is based on the output of a semantic development prediction model, and a security semantic direction opposite to the current generation direction is selected as the guidance target.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. By introducing streaming output monitoring and intermediate content risk detection mechanisms during the large model generation process, dynamic capture of the generation status and early identification and interception of risky content are achieved, solving the time delay defect caused by the fact that post-detection can only be performed after generation in the existing technology, and effectively avoiding the substantial spread risk caused by the spread of risky content in the middle of generation. 2. By implementing real-time parameter adjustment and security correction strategies, the waste of computational resources for large models is reduced, thereby improving the overall security protection effectiveness and resource utilization efficiency of the system. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the dynamic risk interception and correction method for the large model calling process of the present invention. Detailed Implementation

[0018] The following will be based on embodiments of the present invention. Figure 1 The technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Example 1 When a large model call request arrives at the system entry point, an integrity verification operation is first performed. The data packet of the large model call request is extracted from the network protocol stack and parsed to obtain structured data containing the caller identifier, model type parameters, generation parameter configuration, and input prompts that the large model needs to process. The caller identifier records the source of the request, including the caller's unique identification code, authentication credentials, and historical call record index. The model type parameters specify the specific version and architecture type of the large model to be called, including the model name, model version number, and the physical node address where the model is deployed. The generation parameter configuration defines the control parameters for this text generation task, including the maximum generation length, generation temperature coefficient, sampling strategy identifier, repetition penalty factor, and streaming output switch.

[0020] After extracting the structured data, the input prompts are standardized. Specifically, the input prompts first undergo character encoding conversion, unifying text content from different character encoding formats to the target encoding format to ensure consistency in subsequent processing. Standardization also includes lexical segmentation of the input prompts, converting the natural language text into a sequence of lexical units that the model can process. Lexical segmentation employs a sub-word-based strategy, dividing the input text into a lexical array using a predefined vocabulary while preserving the positional mapping between the original text and the lexical sequence for subsequent risk localization and content truncation operations.

[0021] After standardization, a parsing result object is generated, including the original input prompts, standardized lexical sequences, call metadata information, and a parsing timestamp. This parsing result object is transmitted to the risk pre-assessment via an internal message queue, triggering the subsequent pre-call risk pre-assessment process.

[0022] After receiving the parsed result object from the request parsing module, the risk pre-assessment process executes a multi-level risk detection and analysis workflow. First, the standardized word sequence is input into a pre-trained risk classification model. Through forward propagation calculations using a deep neural network, the semantic feature vector of the input prompt information is extracted. The risk classification model employs a Transformer-based encoder network, including multi-layer self-attention mechanisms and feedforward neural networks, effectively capturing the contextual semantic relationships of the input text. The model's output layer uses a multi-label classification structure, outputting corresponding probability values ​​for multiple preset risk categories. The preset risks include various violation categories.

[0023] The risk pre-assessment is based on the probability distributions of each category output by the risk classification model, and performs a comprehensive risk score calculation. The calculation process first identifies the highest probability value and its corresponding risk category from the probability distribution, and then performs a weighted calculation on this highest probability value with a pre-set risk weight matrix. The risk weight matrix is ​​a set of weight coefficients pre-configured according to the severity of each risk category, with different weight coefficients corresponding to different risk categories. The comprehensive risk score is calculated by weighting and summing the probability values ​​of each category with their corresponding weight coefficients. The calculation result is normalized before being output to the risk assessment.

[0024] Risk assessment compares a comprehensive risk score with a preset risk threshold, which is configured according to the system security policy and can be adjusted based on actual application scenarios. When the comprehensive risk score exceeds the risk threshold, the risk assessment determines that the call request has a high risk and directly triggers an interception operation. The interception operation includes terminating the subsequent processing of the call request, generating response data including risk warning information, recording this call event in the risk log system, and returning a rejection response to the caller. The rejection response includes structured information such as error code, risk category label, risk score value, and system suggestions, making it easier for the caller to understand the reason for the interception and take appropriate measures.

[0025] When the overall risk score does not exceed the risk threshold, the risk assessment determines that the call request passes the pre-assessment check and is allowed to proceed to the subsequent large model call process. At this point, the risk pre-assessment generates an assessment result including an initial risk score and risk category label, and stores it in association with the call request's identification information, providing an initial reference benchmark for subsequent dynamic risk detection. The initial reference benchmark is used to measure the relative change of the risk score during the generation process. When the real-time risk score during the generation process is significantly higher than the initial risk score, it can be used as one of the criteria for risk escalation.

[0026] The pre-evaluated call requests enter the large model generation task scheduling queue, awaiting allocation of computing resources. The task scheduling module routes the call requests to the corresponding large model service instance based on the model type parameters and generation parameter configuration. After the generation task starts, the large model begins to perform text generation calculations, using a streaming output method to gradually return the generated text content to the generation status monitoring module. Streaming output is achieved through server-sent events or a bidirectional streaming protocol. Every time a preset number of tokens are generated, a content push is triggered, transmitting the newly generated text fragments to the monitoring module for real-time analysis.

[0027] The generation status monitoring performs risk detection on intermediate generated content according to preset detection cycle nodes. The detection cycle division strategy includes two methods: time interval-based division and character count-based division. The time interval-based strategy triggers the detection process at fixed millisecond intervals, suitable for real-time interactive scenarios with high response latency requirements. The character count-based strategy triggers the detection process based on a cumulative threshold of the number of generated tokens, suitable for scenarios that need to ensure sufficient analysis samples for each detection. The two strategies can be used in combination, with the time interval as the main detection cycle, and auxiliary detection based on character count added between two main detection cycles to balance detection timeliness and analysis accuracy.

[0028] At the end of each detection cycle, the generation status monitoring executes a complete risk detection and analysis process. First, multi-dimensional feature extraction is performed on the newly generated text fragments within the current detection cycle. Lexical feature extraction includes surface language features such as the distribution of word lengths, frequency of special characters, and punctuation usage patterns. Syntactic feature extraction includes syntactic features based on parse tree structures, sentence component ratios, and clause nesting depth. Semantic feature extraction uses the same risk classification model as the pre-assessment stage, inputting the newly generated text fragments into the semantic encoder to extract high-dimensional semantic vector representations.

[0029] After feature extraction, the generation status monitoring performs joint analysis on the extracted multi-dimensional features and the generated context content. Specifically, the semantic vector of the current text segment is compared with the semantic vector of the generated content using cosine similarity calculation to assess the semantic coherence between the newly generated content and the context. Simultaneously, the lexical and syntactic features of the current text segment are compared with features at the same positions in historical detection periods to identify abnormal changes in feature values. Based on the joint analysis results, an overall semantic tendency score of the generated text is calculated, reflecting the development direction and trend characteristics of the generated content in the semantic space.

[0030] The overall semantic tendency score is combined with a pre-trained risk discrimination model to output a real-time risk score for the currently generated text. The calculation of the real-time risk score comprehensively considers the risk probability distribution of the current text segment and the overall semantic tendency score. When the semantic tendency points to a high-risk direction, the system will assign a higher real-time risk score even if the direct risk probability of the current text segment is low. Simultaneously, the generation status monitoring calls a semantic development prediction model to analyze the semantic development trend of the current text. The semantic development prediction model is based on a recurrent neural network architecture. Using the semantic sequence of the currently generated text as input, it predicts the semantic direction that may be generated within the next time window, identifies the risk areas that subsequent generated content may involve, and generates a diffusion trend prediction value. The diffusion trend prediction value quantifies the probability and speed at which the generated content develops towards a risk area; a higher value indicates a greater likelihood of risk diffusion.

[0031] The dynamic interception system receives real-time risk scores and spread trend predictions from the generation status monitoring and executes multi-dimensional risk assessments and interception decisions. First, a two-dimensional risk assessment space is established with real-time risk scores and spread trend predictions as the two axes. The geometric structure of this space is trained and optimized based on the distribution characteristics of historical risk event data. The two-dimensional risk assessment space is divided into three functional areas: a safe zone, an observation zone, and a danger zone. The safe zone corresponds to coordinate points where both the real-time risk score and spread trend prediction value are low, indicating that the risk of the currently generated content is controllable and no intervention is required. The danger zone corresponds to coordinate points where either the real-time risk score or spread trend prediction value is high, indicating that the currently generated content poses a clear risk threat and requires immediate interception measures. The observation zone is a transitional area between the safe and danger zones; coordinate points within this zone indicate that the generated content has certain potential risks but has not yet reached the danger threshold, requiring further observation and assessment.

[0032] Dynamic interception determines the risk zone based on the coordinates of the real-time risk score and the predicted spread trend in the two-dimensional assessment space. When the coordinates are within the danger zone, the risk interception operation must be triggered immediately, interrupting the large model generation task and truncating the generated text content. The truncation operation is achieved by sending a termination signal to the large model service instance, while simultaneously recording the truncation location information for subsequent security correction processing.

[0033] When a coordinate point is within the observation area, the dynamic interception judgment performs a risk urgency calculation. Specifically, it calculates the Euclidean distance between the current coordinate point and the boundary of the danger zone; the closer the distance, the more urgent the risk. The risk urgency score is determined based on the distance calculation result and the time the coordinate point spends within the observation area. When the coordinate point continues to approach the boundary of the danger zone or stays in the high-risk observation area for more than a preset time threshold, the risk urgency score will continue to rise. When the risk urgency score exceeds the preset urgency threshold, it is determined that a risk interception operation needs to be triggered.

[0034] The dynamic interception judgment also introduces a time decay factor to weight the risk scores of historical detection periods, resulting in a cumulative risk assessment result. The time decay factor ranges from zero to one, decreasing as the interval between the historical detection period and the current time point increases. The weighted cumulative risk assessment result reflects the overall evolution trend of the risk level during the generation process. When the cumulative risk assessment result exceeds the cumulative risk threshold, even if the current single real-time risk score does not exceed the first risk threshold, it is determined that a risk interception operation needs to be triggered to prevent the gradual accumulation of risk levels from leading to a final risk outbreak.

[0035] When the real-time risk score does not exceed the first risk threshold and the predicted diffusion trend exceeds the preset trend risk threshold, the dynamic interception judgment determines that the current generation process is in the nascent stage of risk, and has not yet formed substantial risk content, but there is a trend of development towards risk. At this time, real-time adjustment operations at the parameter level are performed to guide the generated content away from the risk direction. The parameter adjustment strategy includes three aspects: temperature parameter adjustment, sampling strategy adjustment, and prompt injection.

[0036] Temperature parameter adjustment reduces the temperature coefficient during large-scale model generation, decreasing the randomness and unpredictability of generated content and making it more conservative and safe. Sampling strategy adjustment modifies the large-scale model's sampling strategy configuration, limiting the generation probability of high-risk words and applying penalty probability adjustments to tokens involving risk categories, causing the model to favor lower-risk word sequences. Hint-level injection dynamically adds safety-oriented guidance statements to the model's input prompts, guiding subsequent generated content to automatically avoid risky domains. The generation of guidance statements is based on the output of the semantic development prediction model, selecting a safe semantic direction opposite to the current generation direction as the guidance target.

[0037] When a risk interception operation is triggered, the truncated text content is received, and secure replacement and semantic correction processing are performed. First, risk semantic units are identified in the truncated text content to determine the range and boundaries of the risk semantic segments that need to be replaced. Risk semantic identification employs a location analysis method based on an attention mechanism to locate the semantic segments in the text with the highest attention from the risk classification model, using these segments as candidate ranges for risk semantic segments. By expanding and merging the candidate ranges, the complete boundary information of the risk semantic segments is obtained.

[0038] The secure replacement process performs a retrieval operation within a secure semantic knowledge base. This knowledge base stores standardized text content that has undergone security review, categorized and indexed according to semantic topics, with each topic corresponding to multiple secure candidate texts. The secure correction output extracts the contextual semantic vector of the truncated text, performs vector similarity retrieval within the semantic knowledge base, and identifies a set of secure candidate texts that match the context semantics. Each secure candidate text is then scored for semantic coherence, with the score calculation comprehensively considering semantic similarity, pragmatic appropriateness, and stylistic consistency with the context. The candidate text with the highest semantic coherence score is selected as the initial version of the replacement text.

[0039] The semantic completion model performs context-adaptive adjustments on the replacement text. Employing a sequence-to-sequence neural network architecture, it takes the complete context of the truncated text and an initial version of the replacement text as input sequences, outputting the semantically adjusted final replacement text. This adjustment process ensures that the replacement text maintains overall semantic logic consistent with the original generated content, including checks and corrections for tense consistency, referential consistency, and topic coherence. The corrected replacement text output is the final result generated by the larger model and is returned to the caller via a streaming output channel.

[0040] The final output response includes alternative text content, processing status indicators, risk interception records, and system suggestions. After receiving the response, the caller can determine whether the call has undergone risk interception processing based on the status indicators and decide whether to adjust the input prompts and re-initiate the call request.

[0041] Example 2 Taking a text generation application scenario as an example, the specific execution process of the method of the present invention will be explained in detail.

[0042] Suppose a caller sends a text generation request to a large model service, with the requested text being "Please help me write an article about a historical figure's biography." This request first undergoes a normalization process during request parsing, converting the input prompt into a sequence of terms and extracting the caller's metadata.

[0043] The risk pre-assessment performs a pre-call risk detection on the standardized lexical sequence. The risk classification model identifies the semantic topic of the prompt text as historical biography writing, and the probability distribution of each risk category is low. Since the overall risk score does not exceed the preset risk threshold, the call request passes the pre-assessment check and is allowed to proceed to the large model generation process.

[0044] The large model initiates the text generation task, gradually returning the generated text content in a streaming output manner. Generation status monitoring performs intermediate content risk detection according to preset detection cycle nodes. In the initial stage of generation, the generated content is identified as a standard biographical introduction of historical figures; both the real-time risk score and the predicted spread trend value are within the safe range, requiring no intervention.

[0045] As the generated content expands, the large model begins to describe a war scene involving a historical figure. It detects newly generated text fragments that involve detailed descriptions of the war, and the similarity between the semantic feature vector and the risk template increases. The overall semantic tendency score begins to shift towards higher risk, the real-time risk score rises but still does not exceed the first risk threshold, and the diffusion trend prediction value exceeds the preset trend risk threshold.

[0046] The dynamic interception system determines that the current generation process is in the early stages of risk and performs real-time adjustments to the parameters. The temperature parameter is reduced from its initial value to a preset safety threshold, and guiding statements are dynamically injected into the input prompts of the large model, guiding subsequent generated content towards a peaceful narrative or emphasizing historical lessons.

[0047] The large model continued generating content based on the adjusted parameters, shifting its focus from describing war scenarios to historical reflection and peace initiatives. It detected that the semantic vectors of the generated content converged back towards a safe direction, the real-time risk score fell back within a safe range, and the predicted diffusion trend value also dropped below the threshold. The generation task was completed successfully, and the final generated text content maintained the integrity of the historical biographical theme while avoiding over-rendering of scenes, meeting the requirements for safe output.

[0048] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0049] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A dynamic risk interception and correction method for large model invocation processes, characterized in that, include: Obtain the large model call request and parse it to get the prompt text content to be generated; Based on a preset risk detection strategy, the parsed prompt text content is pre-assessed for risk before being called, and an initial risk score and risk category label are generated. When the initial risk score exceeds a preset risk threshold, the large model call request is intercepted and a risk prompt message is returned. The large model generation task is initiated, and the text content generated by the large model is received in a streaming output mode. At the preset detection period node, the intermediate generated content risk detection is performed on the generated text fragments in real time, and the real-time risk score and diffusion trend prediction value are calculated. Based on the real-time risk score and the predicted diffusion trend value, it is dynamically determined whether to trigger a risk interception operation; When the real-time risk score exceeds the first risk threshold, the generation task is immediately interrupted and the generated text content is truncated. When the real-time risk score does not exceed the first risk threshold, but the diffusion trend prediction value exceeds the preset trend risk threshold, the generation process of the large model is adjusted in real time at the parameter level to guide the generated content to deviate from the risk direction. When a risk interception operation is triggered, the truncated text content is replaced and semantically corrected based on a preset security correction strategy to generate alternative text content that meets security requirements. The alternative text content is then returned to the caller as the final output of the large model.

2. The method according to claim 1, characterized in that, The pre-assessment of the risk of the parsed prompt text content before invocation based on a preset risk detection strategy includes at least the following: The prompt text content is input into a pre-trained risk classification model, the semantic feature vector of the prompt text is extracted, and the probability distribution of the prompt text being classified into each preset risk category is output based on the semantic feature vector. The preset risk categories include politically sensitive, violent and bloody, fraud and inducement, privacy leakage and other violations. A comprehensive risk score is calculated based on the maximum probability value in the probability distribution and the preset risk weight matrix. The corresponding risk level label is determined based on the comprehensive risk score, and the risk level label is stored in association with the comprehensive risk score.

3. The method according to claim 2, characterized in that, The risk weight matrix is ​​a set of weight coefficients pre-configured according to the degree of harm of each risk category. Politically sensitive and violent / bloody categories correspond to higher weight coefficients, while fraud / inducement and privacy leakage categories correspond to medium weight coefficients.

4. The method according to claim 1, characterized in that, The method of receiving text content generated by a large model using a streaming output approach and performing real-time intermediate content risk detection on the generated text fragments at preset detection period nodes includes at least the following: The detection cycle is divided according to a preset time interval or character quantity threshold, and the intermediate content risk detection process is triggered at the end of each detection cycle. Feature extraction is performed on newly generated text fragments within the current detection period to obtain lexical features, syntactic features, and semantic features of the text fragments. These features are then jointly analyzed with the generated context content to calculate the overall semantic tendency score of the currently generated text. Based on the overall semantic tendency score and the pre-trained risk discrimination model, a real-time risk score is output. At the same time, the semantic development prediction model is used to analyze the semantic development trend of the current text and generate a diffusion trend prediction value.

5. The method according to claim 4, characterized in that, The predicted diffusion trend value is calculated based on the semantic development context and contextual features of the generated text, and quantitatively represents the probability and speed at which the generated content develops into the risk domain.

6. The method according to claim 1, characterized in that, The dynamic determination of whether to trigger a risk interception operation based on the real-time risk score and the predicted diffusion trend value includes at least: A two-dimensional risk assessment space is established with real-time risk score and diffusion trend prediction as the two axes. The two-dimensional risk assessment space is divided into a safe area, an observation area, and a danger area. When the coordinate point is located in the danger area, it is determined that a risk interception operation needs to be triggered immediately. When the coordinate point is located in the observation area, the risk urgency score is calculated based on the distance between the coordinate point and the boundary of the danger area. When the risk urgency score exceeds a preset urgency threshold, the risk interception operation is triggered. A time decay factor is introduced to weight the risk scores of historical detection cycles to obtain a cumulative risk assessment result. When the cumulative risk assessment result exceeds the cumulative risk threshold, it is determined that a risk interception operation needs to be triggered.

7. The method according to claim 6, characterized in that, When the coordinate point is located within the observation area, the risk urgency score is determined by combining the distance between the coordinate point and the boundary of the danger zone and the time the coordinate point stays within the observation area.

8. The method according to claim 1, characterized in that, When a risk interception operation is triggered, the truncated text content is safely replaced and semantically corrected based on a preset security correction strategy, including at least: Identify specific semantic units in the truncated text that involve risk content, and determine the scope and boundaries of the risky semantic segments that need to be replaced; Retrieve secure candidate content that matches the semantic context of the truncated text from a secure semantic knowledge base, score the semantic coherence of each secure candidate content, and select the candidate content with the highest semantic coherence score as the replacement text. The semantic completion model is used to adjust the context of the alternative text to ensure that the overall semantic logic of the alternative text is consistent with that of the original generated content, and the adjusted alternative text is output as the final generated result.

9. The method according to claim 1, characterized in that, The real-time adjustment of parameters during the generation process of the large model to guide the generated content away from the risky direction includes at least: Adjusting the temperature parameters during large model generation reduces the randomness of the generated content, making the generated content more conservative and safe. Modify the sampling strategy during large model generation to limit the generation probability of high-risk words and apply punitive probability adjustments to tokens involving risk categories; inject security-oriented guidance statements at the prompt level to influence the generation direction of the large model by modifying its input prompts.

10. The method according to claim 9, characterized in that, The generation of the security-oriented guidance statement is based on the output of the semantic development prediction model, and the security semantic direction opposite to the current generation direction is selected as the guidance target.

Images