Large model safety fence enhancement method and system based on multi-dimensional risk assessment and retrieval enhancement

Abstract

The invention discloses a large model safety guardrail enhancement method and system based on multi-dimensional risk assessment and retrieval enhancement. Comprising a safety guardrail back-end service module, and an input side safety detection sub-module, an output side safety detection sub-module, a retrieval enhancement sub-module, a safety pickup sub-module, a case backflow sub-module and a log and evaluation sub-module which are respectively connected with the safety guardrail back-end service module. Bidirectional safety control over input and output of a large model is achieved through the integrated process of multi-dimensional routing analysis, discrete risk assessment, violation judgment based on a difficulty mode, on-demand retrieval enhancement and safety pickup rewriting.

Description

Technical Field

[0001] This invention relates to a technology in the field of artificial intelligence, specifically a method and system for enhancing safety barriers based on multidimensional risk assessment and retrieval enhancement of large models. Background Technology

[0002] As large language models are increasingly used in dialogue, search, and content generation, the risk of generating inappropriate content such as illegal, self-harming, and privacy-leaking content is rising, necessitating the construction of security safeguards at both the input and output ends outside the application. Existing security protection and defense technologies for large language models suffer from a lack of auditable and reproducible security judgment criteria, and difficulty in supporting fine-grained, configurable differentiated handling and false positive control. Summary of the Invention

[0003] To address the aforementioned shortcomings of existing technologies, this invention proposes a method and system for enhancing the security guardrail of a large model based on multidimensional risk assessment and retrieval enhancement. Relying solely on a single lightweight general-purpose large language model, this invention achieves bidirectional security control over the input and output of the large model through an integrated process of "multidimensional routing analysis—discrete risk assessment—criteria determination based on difficulty patterns—on-demand retrieval enhancement—security-assisted answer rewriting".

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a method for enhancing safety barriers in large models based on multidimensional risk assessment and retrieval enhancement, comprising:

[0006] Step 1: Unify multi-dimensional security routing and risk modeling, specifically including:

[0007] 1.1 For the text to be detected, whether input by the user or output by the model Obtain structured routing context through a lightweight large model: It includes at least: language (lang), high-level intent (intent), attack intent (attack_intent), security scenario domain set (domain), sentiment label (sentiment), entity and sensitive slot set (sensitive_slots), etc.

[0008] The lightweight large model is a pre-trained language model fine-tuned from instructions with a small parameter size, preferably around 4 bytes, such as Qwen3-4B-Instruct-2507. This lightweight large model does not require parameter updates during use and can be invoked as an independent inference service to generate the structured routing context.

[0009] 1.2 Construct a unified set of risk categories: ={PHYSICAL_HARM,SELF_HARM,CRIMES_ILLEGAL,…,OTHER_RISK}, This represents any one of the risk categories. For any text to be detected, the security model outputs a discrete risk level for each category c. {NONE,LOW,MEDIUM,HIGH}, and define a mapping function from level to value. , where: , when =NONE =0; when =LOW =1; when =MEDIUM =2; when =HIGH =3. This yields the discrete risk vector. This forms a multidimensional and interpretable risk profile.

[0010] Step 2: Calculate the overall risk level based on rule fusion, specifically including:

[0011] 2.1 Let the highest level of all risk categories be... and define the overall risk level. Defined as: when , ;when , ;when , ;when , to This is a preset four-level overall risk level label.

[0012] 2.2 Define the core high-risk category set ={SELF_HARM, CRIMES_ILLEGAL, MALICIOUS_USE, PHYSICAL_HARM, PRIVACY_PROPERTY, FINANCIAL_RISK, FRAUD_DECEPTION, ATTACK_MODEL}, where: the elements of the set are risk category identifiers, and all belong to the risk category set. If it exists {SELF_HARM, CRIMES_ILLEGAL, MALICIOUS_USE, PHYSICAL_HARM}, satisfy =3, then the overall risk level is forcibly increased to If it exists {PRIVACY_PROPERTY, FINANCIAL_RISK, FRAUD_DECEPTION}, satisfy =2, then the overall level is at least 2. .

[0013] 2.3 The overall risk level is obtained after the above fusion: { }

[0014] Step 3: A violation determination mechanism based on detection difficulty mode, specifically including:

[0015] 3.1 Design three detection modes: lenient, normal, and strict. The specific modes are as follows: lenient mode is used to reduce false interception in low-risk and discussion contexts; normal mode is used to strike a balance between false interception and false negatives and can be used as the default configuration; strict mode is used for high-sensitivity scenarios or high-risk business and focuses on reducing false negatives.

[0016] 3.2 Design of a High-Risk Set ={PHYSICAL_HARM,SELF_HARM,CRIMES_ILLEGAL,MALICIOUS_USE,PRIVACY_PROPERTY,FRAUD_DECEPTION,FINANCIAL_RISK,ATTACK_MODEL,SEXUAL_CONTENT}, Soft risk set The backslash "\" represents the difference operation, meaning that the soft risk set represents the set of risk categories other than the strong risk set.

[0017] 3.3 Define a threshold for a given pattern 𝛿∈{lenient, normal, strict}. , For high-risk collection The threshold for judgment, For soft risk set The threshold for determining violations. Define the violation prediction result. The violation determination function is: when or hour, In other cases, The detection mode The threshold parameter pair used to select the mode This allows for different levels of violation detection strategies by configuring parameters without altering the same risk assessment output. The threshold parameters... These are preset or configurable parameters, and they satisfy the following conditions: the lenient mode corresponds to a higher threshold, the strict mode corresponds to a lower threshold, and the standard mode is in between.

[0018] Step 4, Difficult Sample Identification and On-Demand Retrieval Enhancement, specifically includes:

[0019] 4.1 Constructing a trouble sample indicator function based on the risk vector and routing context Typical scenarios include: 1. The existence of two or more risk categories. ,satisfy 2. Vague categories such as MISINFORMATION, POLITICAL_SENSITIVE, ETHICS_MORALITY, and OTHER_RISK exhibit LOW or MEDIUM ratings; 3. The overall rating is... or Furthermore, it lacks a clear core high-risk category. When At that time, the search enhancement process is triggered. Search query Obtained by concatenating the text to be detected and the optional original response: or This represents a concatenation operation. It is achieved through a pre-trained vector model. Calculate query and document vectors: .

[0020] 4.2 After L2 normalizing the vectors obtained in step 4.1, use inner product or cosine similarity. Retrieve the highest-scoring results from the security knowledge base One document. Derive the risk label set based on routing context and risk vector. Only retain those that meet the requirements. The document allows for the automatic selection of relevant laws, platform policies, or historical security cases in different scenarios such as criminal offenses, self-harm and suicide, and privacy breaches.

[0021] 4.3 After obtaining the retrieved documents, construct comprehensive suggestion terms that include text content, routing context, and search results. Then, call the same lightweight large model again to perform a second-stage risk assessment. The results were then conservatively merged with the results from the first stage, taking the highest level for each category.

[0022] Step 5, Input / Output Two-Way Barrier and Safe Answering Mechanism, specifically includes:

[0023] 5.1 Simultaneously detect user input and model output: Input side: If the above decision function is obtained... If the input is not in violation of regulations but the model's initial answer is risky, the system will directly enter the safe answering module, explaining the relevant laws and safety risks to the user and providing compliant alternative suggestions; Output side: when the input side is not in violation of regulations but the model's initial answer is risky ( This invention utilizes input text, initial response, risk overview, and retrieved security documents to guide a lightweight large model in generating a final response after security rewriting, avoiding the output of specific illegal operation details while maintaining information content and usability as much as possible.

[0024] 5.2 When no violation is detected on the input side but a violation is detected on the output side, the question-and-answer process is treated as a typical boundary sample, and a sample containing user input is constructed. Original answers from the model The final answer after security rewriting Risk vector Overall risk level and trigger detection mode Structured records with fields such as [field name] are written into the security knowledge base. And tag the record with a risk category set. Corresponding tag set .

[0025] 5.3 In the subsequent stage of enhancing the retrieval of difficult samples, priority should be given to retrieving samples from the general security knowledge base. Historical boundary samples similar to the current risk vector and routing context are retrieved to assist in the second-stage risk assessment and security response, thus forming a closed-loop self-reinforcing mechanism of "detection-correction-precipitation-reuse".

[0026] This invention relates to a large-scale security barrier enhancement system based on multidimensional risk assessment and retrieval enhancement, which implements the above-mentioned method. The system includes a security barrier backend service module and several interconnected sub-modules: an input-side security detection sub-module, an output-side security detection sub-module, a retrieval enhancement sub-module, a security proxy response sub-module, a case feedback sub-module, and a log and evaluation sub-module. Specifically: the security barrier backend service module, based on user input text, session context, and policy configuration parameters, sequentially calls the input-side security detection sub-module to obtain gating results; when generation is allowed, it performs multidimensional risk assessment and violation judgment processing based on user input text and structured context information obtained through information analysis, outputting an input-side risk profile, violation prediction results, and input-side handling action information; subsequently, it calls the output-side security detection sub-module to obtain post-processing actions; when review, rewriting, or proxy response is triggered, it calls the retrieval enhancement sub-module to obtain an evidence set and injects the evidence set into the generation input of the security proxy response sub-module; finally, the security barrier backend service module outputs a compliant response and synchronizes the intermediate states of the entire chain to the case feedback sub-module and the log and evaluation sub-module.

[0027] The output-side security detection submodule performs post-generation verification and correction on candidate responses, and performs multi-dimensional risk assessment and violation judgment processing based on the candidate response text, the corresponding user input text, and structured context information. It outputs an output-side risk profile, violation prediction results, and output-side handling action information. When a candidate response is determined to meet the constraints, a release signal is output. When a risk of exceeding the limit or non-compliance is determined, a rewriting or proxy answer trigger signal is output.

[0028] The retrieval enhancement submodule provides evidence fragments for risk review and compliance expression generation. Based on the retrieval query information (including user input text, candidate response text, or a combination of both) and risk tags, language markers, and document type constraints in the structured context, it performs vector retrieval and meta-information filtering processing, and outputs an evidence set and its source meta-information. The evidence set is provided to the input-side security detection submodule or the output-side security detection submodule for review and judgment, or provided to the security proxy submodule for generating compliance expressions.

[0029] The security-assisted response submodule generates compliant responses in high-risk or unanswerable scenarios and performs response generation or rewriting generation processing based on user input text, candidate response text, structured context information, risk profile information, and evidence set output by the retrieval enhancement submodule, outputting compliant response text. The response generation outputs risk warnings and compliance guidance, while the rewriting generation removes or abstracts executable risk details while retaining the amount of task information.

[0030] The case feedback submodule accumulates and reuses typical and boundary samples, and performs sample screening, field normalization and storage processing based on information such as the input text, candidate response text, final response text, structured context, risk profile, handling actions and evidence set of this round of interaction, and outputs case data records that can be used for subsequent strategy iteration and offline evaluation.

[0031] The aforementioned log and evaluation sub-modules form an auditable operational trajectory and support offline evaluation. Based on the intermediate and final results output by each sub-module, they perform log recording, statistical summarization, and evaluation indicator calculation and processing, and output audit logs, policy trigger records, and evaluation reports. The audit logs include at least the input-side actions, output-side actions, risk profiles, evidence source identifiers, and final response results to support retrospective reproduction and compliance review.

[0032] Technical effect

[0033] This invention implements a multi-task reusable security guardrail based on a single lightweight general-purpose large language model. Without introducing multiple dedicated detection models and independent rewriting models, it uses different prompt templates and structured output constraints to enable the same lightweight instruction fine-tuning pre-trained model to separately complete routing context extraction, multi-dimensional risk discrete grading, and compliance response generation, thereby forming a unified structured intermediate representation and reducing system deployment and inference resource overhead. A two-stage risk review and targeted retrieval enhancement mechanism triggered by difficult samples is implemented. Boundaries and uncertain samples are automatically identified through a difficult sample indicator function, triggering enhanced retrieval review only when necessary. Based on risk labels and document type constraints obtained from the mapping of routing context and risk vectors, relevant evidence fragments are selected from security policies, handling guidelines, and historical case knowledge bases to participate in the second-stage discrimination, achieving on-demand retrieval and evidence injection under relevance constraints, reducing single-discrimination fluctuations and improving review consistency. Input-output linkage facilitates the accumulation of difficult examples and non-parametric continuous enhancement. Question and answer samples that are not triggered on the input side but are triggered on the output side (i.e., the input and output judgments are inconsistent) are automatically structured, recorded, and fed back to form a searchable security case library. Subsequently, in the similarity retrieval stage, such difficult case evidence is prioritized for matching, so as to continuously absorb new boundary samples without modifying the model parameters, thereby achieving continuous enhancement that is auditable and traceable.

[0034] Compared with the prior art, the technical effects of the present invention include:

[0035] 1) Improved detection performance across multiple datasets with flexible adjustment of precision and recall. On four detection datasets—ChineseSafe, SafetyPrompts, Safety-en-14k, and Salad-Data (Base set)—the F1-Score and overall accuracy of this invention for detecting both illegal and safe samples outperform various open-source and commercial security detection baseline models. Furthermore, while maintaining detection accuracy, this invention allows for controllable adjustment of detection precision and recall by adjusting the detection mode.

[0036] 2) The violation rate of the end-to-end question-and-answer chain is significantly reduced. Compared with security models that only have detection functions, this invention has a secure proxy answering function. After integrating this invention as a security fence into the question-and-answer process, on the same set of benchmark datasets, when the input-side standard mode fence and the output-side standard mode fence are used simultaneously, the violation rate of the model's final answer is reduced to 8.19%, 7.98%, 3.22%, and 5.31%, respectively, which is better than the currently open-source secure proxy answering fences. Attached Figure Description

[0037] Figure 1 This is a flowchart of the present invention;

[0038] Figure 2This is a schematic diagram of the multi-dimensional security routing and risk assessment process of the present invention;

[0039] Figure 3 This is a schematic diagram of the violation determination logic based on the detection difficulty mode of the present invention;

[0040] Figure 4 This is a schematic diagram illustrating the workflow of the retrieval enhancement module of the present invention;

[0041] Figure 5 This is a schematic diagram comparing the detection F1-Score of the method of the present invention and the comparison method on different datasets;

[0042] Figure 6 This diagram illustrates the comparison of precision and recall of different modes of the method of the present invention on different datasets. Detailed Implementation

[0043] like Figure 1 As shown in the figure, this embodiment relates to a large-model security fence enhancement method based on multidimensional risk assessment and retrieval enhancement, including:

[0044] Step 1, Deployment of Large Language Model Service: Using the vLLM framework, the model service is started via command line, specifically including:

[0045] 1) Install the CUDA driver and vLLM runtime environment on the server to ensure that the GPU device can be recognized.

[0046] 2) Specify the GPU device number to use through environment variables, and set the model path, service name, maximum context length, and memory utilization.

[0047] Step 2, Large Language Model Client Initialization and Basic Parameter Settings: Initialize the large language model client compatible with the vLLM service in the security fence backend, uniformly configure the calling parameters, and ensure that all subsequent nodes use the same model in a consistent manner. Specifically, this includes:

[0048] 1) Install and import the OpenAI client library, construct an OpenAI-compatible client object, and set base_url to point to the HTTP interface exposed by vLLM in step 1, such as "http: / / 127.0.0.1:8000 / v1". api_key can be set to any placeholder value.

[0049] 2) Set a unified model name variable MODEL_NAME with the value "Qwen" to specify the model when calling it later.

[0050] 3) Encapsulate the basic dialogue function llm_chat, with input parameters including system_prompt, user_prompt, temperature, and max_tokens. Temperature is set to between 0.0 and 0.7 by default, and commonly used values ​​are 0.3 or 0.7. max_tokens is set to between 512 and 1024 by default, which is used to limit the length of a single generation.

[0051] 4) Encapsulate the JSON output function llm_chat_json. If the large language model deployed in step one is a non-thinking model, adding response_format={"type":"json_object"} when calling llm_chat can force the model to output in JSON format; otherwise, clean the model output after returning, remove code block markers, scan the curly braces in the string, and parse out the last object that can be parsed by JSON as the structured result.

[0052] Step 3: Construct a security question-and-answer flowchart: Use the stateful graph framework LangGraph to construct the overall processing flow of the security guardrail, encapsulate input-side detection, main answer, output-side detection, and security proxy answer as multiple nodes, and connect them through conditional edges to realize a visualized and programmable security question-and-answer graph.

[0053] Step 4, Implementation of the input-side routing analysis module: Implement the input-side routing analysis node route_input, which is used to perform high-level semantic analysis on the user's original question and output a unified routing context route_ctx_in for subsequent risk assessment and retrieval enhancement.

[0054] Step 5, Input-side risk assessment and violation determination: Implement the input-side risk detection node risk_input, which is used to perform multi-dimensional risk assessment on input questions based on routing context and text content, and provide a discrete determination of whether a violation has occurred under a given detection mode.

[0055] Step 6, Input-side decision-making and main answer generation: Based on the input-side violation judgment results, this step determines whether to directly proceed to the safe answer-generating process or call the main model to generate the original answer.

[0056] Step 7, Output-side routing analysis and risk assessment: This step is used to perform secondary routing analysis and risk assessment on the answers generated by the main model, preventing the model from outputting high-risk content even when the input is valid.

[0057] Step 8, Enhanced Triggering for Difficult Sample Identification and Retrieval: Implement enhanced triggering logic for difficult sample identification and retrieval in the safe_answer node to improve the accuracy of boundary sample determination.

[0058] Step Nine, Safe Proxy Answering and Output Rewriting: The safe proxy answering node `safe_answer` handles two scenarios: first, direct input violations; second, risky output responses, specifically including:

[0059] 1) The safe_answer node reads user_query, model_answer, input_risk_profile, output_risk_profile, router_ctx_in, and rag_docs from the status to construct a safe answering system prompt, explicitly requiring that specific steps such as illegal and criminal activities, self-harm and suicide, violent injury, privacy leaks, and high-risk financial operations not be provided.

[0060] 2) The user's original question, original answer (if any), input and output risk detection results, and retrieved document text docs_text are combined into user_prompt, which is then passed to llm_chat to call the large language model. The temperature is set to 0.3 and the max_tokens is set to 1024 to ensure that the output is relatively robust and conservative.

[0061] 3) For cases where the input side violates the rules and the main model is not invoked, safe_answer will provide a rejection explanation, risk warning, and feasible safety alternatives based on input_risk_profile and rag_docs, such as suggesting that the user consult a professional organization or handle the matter through formal channels.

[0062] 4) For cases where the output is illegal but the input itself is legal, safe_answer rewrites the answer using the original answer and the retrieved documents: deleting or weakening harmful details, adding safety precautions and policy explanations, and retaining as much general information as possible that is beneficial to the user, to obtain final_answer.

[0063] 5) The safe_answer node writes the final generated safe answer to the final_answer status field and serves as the final output of the graph.

[0064] Step 10: Recording of violation output samples and updating the case knowledge base. After completing the safe proxy answer generation, the system checks the input and output judgment results of this round of question and answer. If satisfied... but The sample from that round of conversation is then written into the security case vector library for subsequent retrieval enhancement and strategy optimization. The specific sample records are then constructed. Enter text for this round of users. This is the original answer from the lightweight model. This is the final answer after a secure rewrite. As a multidimensional risk vector, For the overall risk level, For detection mode, For timestamps or business scenario identifiers.

[0065] Step 11: Final confirmation of a normal answer. If both input and output are deemed acceptable, the `finalize_answer` node directly copies `model_answer` to the `final_answer` field without further rewriting, thus reducing unnecessary content drift.

[0066] Step 12, Graph Application Call and Interface Encapsulation: Encapsulate the above security question-and-answer graph into an external service interface for easy integration into business systems such as dialogue assistants, search and question answering, and intelligent customer service.

[0067] In practical application experiments, this invention ran under the following hardware and software environment: the server processor was an Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz; the GPU was an NVIDIA GeForce RTX 4090; and the total memory was approximately 24GiB. The operating system was Ubuntu 24.04.3 LTS (x86_64), with a Linux kernel version of 6.8.0-88-generic. For the software stack, the experiment used Python 3.10.19, with core dependencies including: PyTorch 2.8.0, Transformers 4.57.3, and Sentence-Transformers 5.1.2 for large model inference and vector representation; vLLM 0.11.0 for model service deployment and API calls; LangGraph 1.0.4 for safety barrier workflow orchestration; and NumPy 2.2.6, scikit-learn 1.7.2, and other components were used for evaluation and statistical analysis. The runtime environment includes CUDA 12 series runtime and operator libraries (such as nvidia-cuda-runtime-cu12, nvidia-cudnn-cu12, nvidia-cublas-cu12, etc.) to support GPU-accelerated inference and vector computation. Experimental data obtained by running this invention in the above environment are as follows:

[0068] (1) Table 1 shows the detection accuracy of the present invention and other methods on ChineseSafe, SafetyPrompts, Safety-en-14k, and Salad-Data (Base_set). The detection accuracy of the present invention is significantly better than that of various open-source and commercial security models.

[0069] (2) Figure 5 The figure shows the F1-scores of the safety and violation samples of this invention and other methods on ChineseSafe, SafetyPrompts, Safety-en-14k, and Salad-Data(Base_set). The overall F1-score of this invention is superior to other methods on both types of samples.

[0070] (3) Figure 6 The diagram shows the detection precision and recall rates under three detection modes of this invention—relaxed mode, standard mode, and strict mode—in ChineseSafe, SafetyPrompts, Safety-en-14k, and Salad-Data(Base_set). This invention achieves a controllable trade-off between precision and recall by switching detection modes and adjusting the judgment threshold and upgrade rules accordingly: the relaxed mode prioritizes improving precision to reduce false positives, the strict mode prioritizes improving recall to reduce the risk of missed detections, and the standard mode achieves a relative balance between the two. This adapts to the differentiated requirements for security strength and availability in different business scenarios, verifying the effectiveness of the multi-mode adjustable strategy of this invention.

[0071] (3) Table 2 shows the violation rates of the output content of the Qwen3-4B-Instruct-2507 model on ChineseSafe, SafetyPrompts, Safety-en-14k, and Salad-Data(Base_set). When using standard mode guardrails on both the input and output sides, the violation rate of this invention is lower than other safety guardrail methods.

[0072] Table 1

[0073] Table 2

[0074] Compared to existing technologies, this invention achieves a controllable trade-off between precision and recall in detecting violations by using a unified multi-dimensional risk assessment output and configurable detection modes, without changing the model or altering the main business process. It also achieves superior detection performance on multiple benchmark datasets. Through a two-way collaborative approach of input-side gating and output-side post-hoc correction, security control is expanded from a single detection to a complete closed-loop strategy encompassing release, rewriting, and secure proxy answers, significantly reducing the final violation output rate in the end-to-end question-and-answer chain. Furthermore, an on-demand triggered retrieval enhancement mechanism introduces locatable policy clauses, handling templates, and historical cases as external evidence for difficult or boundary samples, improving consistency and traceability in judgment and handling while avoiding unnecessary retrieval and inference overhead for low-risk samples. The invention's clear process structure allows for seamless integration with existing large-scale model applications: the security detection flowchart and security question-and-answer flowchart are encapsulated as reusable process nodes that can be embedded into application chains such as search question-and-answer, customer service robots, and content generation, demonstrating excellent engineering feasibility, scalability, and maintainability.

[0075] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A large-model safety barrier enhancement system based on multidimensional risk assessment and retrieval enhancement, characterized in that, include: The security barrier backend service module consists of several interconnected sub-modules: an input-side security detection sub-module, an output-side security detection sub-module, a retrieval enhancement sub-module, a security-assisted response sub-module, a case feedback sub-module, and a log and evaluation sub-module. Specifically: The security barrier backend service module, based on user input text, session context, and policy configuration parameters, sequentially calls the input-side security detection sub-module to obtain gating results. When generation is permitted, it performs multi-dimensional risk assessment and violation determination based on user input text and structured context information obtained through information analysis, outputting an input-side risk profile, violation prediction results, and input-side handling action information. Subsequently, it calls the output-side security detection sub-module to obtain post-processing actions. When review, rewriting, or proxy response is triggered, it calls the retrieval enhancement sub-module to obtain an evidence set and injects this evidence set into the generation input of the security-assisted response sub-module. Finally, the security barrier backend service module outputs a compliant response and synchronizes the intermediate states of the entire chain to the case feedback sub-module and the log and evaluation sub-module.

2. The large-model safety barrier enhancement system based on multidimensional risk assessment and retrieval enhancement as described in claim 1, characterized in that, The output-side security detection submodule generates and verifies candidate responses, corrects them, and performs multi-dimensional risk assessment and violation judgment based on the candidate response text, the corresponding user input text, and structured context information. It then outputs an output-side risk profile, violation prediction results, and output-side handling action information. When a candidate response is determined to meet the constraints, a release signal is output; when a risk of going out of bounds or non-compliance is determined, a rewrite or proxy response trigger signal is output.

3. The large-model safety barrier enhancement system based on multidimensional risk assessment and retrieval enhancement as described in claim 1, characterized in that, The retrieval enhancement submodule provides evidence fragments for risk review and compliance expression generation. Based on retrieval query information including user input text, candidate response text, or a combination of both, as well as risk tags, language markers, and document type constraints in the structured context, it performs vector retrieval and meta-information filtering processing, and outputs an evidence set and its source meta-information. The evidence set is provided to the input-side security detection submodule or the output-side security detection submodule for review and judgment, or provided to the security proxy submodule for generating compliance expressions.

4. The large-model safety barrier enhancement system based on multidimensional risk assessment and retrieval enhancement as described in claim 1, characterized in that, The security-assisted response submodule generates compliant responses in high-risk or unanswerable scenarios and performs response generation or rewriting generation processing based on user input text, candidate response text, structured context information, risk profile information, and evidence set output by the retrieval enhancement submodule, outputting compliant response text. The response generation outputs risk warnings and compliance guidance, while the rewriting generation removes or abstracts executable risk details while retaining the amount of task information.

5. The large-model safety barrier enhancement system based on multidimensional risk assessment and retrieval enhancement according to claim 1, characterized in that, The case feedback submodule accumulates and reuses typical and boundary samples, and performs sample screening, field normalization and storage processing based on information such as the input text, candidate response text, final response text, structured context, risk profile, handling actions and evidence set of this round of interaction, and outputs case data records that can be used for subsequent strategy iteration and offline evaluation.

6. The large-model safety barrier enhancement system based on multidimensional risk assessment and retrieval enhancement according to claim 1, characterized in that, The aforementioned log and evaluation sub-modules form an auditable operational trajectory and support offline evaluation. Based on the intermediate and final results output by each sub-module, they perform log recording, statistical summarization, and evaluation indicator calculation and processing, and output audit logs, policy trigger records, and evaluation reports. The audit logs include at least the input-side actions, output-side actions, risk profiles, evidence source identifiers, and final response results to support retrospective reproduction and compliance review.

7. A method for enhancing a large-scale safety barrier based on multidimensional risk assessment and retrieval enhancement according to any one of claims 1-6, characterized in that, include: Step 1: Unify multi-dimensional security routing and risk modeling; Step 2: Calculate the overall risk level based on rule fusion; Step 3: Violation determination based on detection difficulty mode; Step 4, Difficult Sample Identification and On-Demand Retrieval Enhancement, specifically includes: 4.1 Constructing a trouble sample indicator function based on the risk vector and routing context Typical scenarios include:

1. The existence of two or more risk categories. ,satisfy 2. Vague categories such as MISINFORMATION, POLITICAL_SENSITIVE, ETHICS_MORALITY, and OTHER_RISK exhibit LOW or MEDIUM ratings; 3. The overall rating is... or Furthermore, it lacks a clear core high-risk category. When this occurs, the search enhancement process is triggered, and the search query is retrieved. Obtained by concatenating the text to be detected and the optional original response: or This indicates a concatenation operation, performed using a pre-trained vector model. Calculate query and document vectors: ; 4.2 After L2 normalizing the vectors obtained in step 4.1, use inner product or cosine similarity. Retrieve the highest-scoring results from the security knowledge base Each document derives a set of risk labels based on routing context and risk vectors. Only retain those that meet the requirements. The document allows for the automatic selection of relevant laws, platform policies, or historical security cases in different scenarios such as criminal offenses, self-harm and suicide, and privacy breaches. 4.3 After obtaining the retrieved documents, construct comprehensive suggestion terms that include text content, routing context, and search results. Then, call the same lightweight large model again to perform a second-stage risk assessment. And conservatively merge the results with the first stage results, taking the highest level for each category; Step 5, Input / Output Two-Way Barrier and Safety Proxy, specifically includes: 5.1 Simultaneously detect user input and model output: Input side: If the above decision function is obtained... If the input is not in violation of regulations but the model's initial answer is risky, the system will directly enter the safe answering module, explaining the relevant laws and safety risks to the user and providing compliant alternative suggestions; Output side: when the input side is not in violation of regulations but the model's initial answer is risky ( This invention utilizes input text, initial response, risk overview, and retrieved security documents to guide a lightweight large model in generating a final response after security rewriting, avoiding the output of specific illegal operation details while maintaining information content and usability as much as possible. 5.2 When no violation is detected on the input side but a violation is detected on the output side, the question-and-answer process is treated as a typical boundary sample, and a sample containing user input is constructed. Original answers from the model The final answer after security rewriting Risk vector Overall risk level and trigger detection mode Structured records with fields such as [field name] are written into the security knowledge base. And tag the record with the risk category set. Corresponding tag set ; 5.3 In the subsequent stage of enhancing the retrieval of difficult samples, priority should be given to retrieving samples from the general security knowledge base. Historical boundary samples similar to the current risk vector and routing context are retrieved to assist in the second-stage risk assessment and security response, thus forming a closed-loop self-reinforcing mechanism of "detection-correction-precipitation-reuse".

8. The method for enhancing large-scale safety barriers according to claim 7, characterized in that, Step 1 specifically includes: 1.1 For the text to be detected, whether input by the user or output by the model Obtain structured routing context through a lightweight large model: It includes at least: language (lang), high-level intent (intent), attack intent (attack_intent), security scenario domain set (domain), sentiment tag (sentiment), entity and sensitive slot set (sensitive_slots), etc. 1.2 Construct a unified set of risk categories: ={PHYSICAL_HARM,SELF_HARM,CRIMES_ILLEGAL,…,OTHER_RISK}, For any given risk category, and for any text to be detected, the security model outputs a discrete risk level for each category c. {NONE,LOW,MEDIUM,HIGH}, and define a mapping function from level to value. , where: , when =NONE =0; when =LOW =1; when =MEDIUM =2; when =HIGH =3, from which the discrete risk vector is obtained. This forms a multidimensional and interpretable risk profile.

9. The method for enhancing large-scale safety barriers according to claim 7, characterized in that, Step 2 specifically includes: 2.1 Let the highest level of all risk categories be... and define the overall risk level. Defined as: when , ;when , ;when , ;when , to The pre-set four-level overall risk level label; 2.2 Define the core high-risk category set ={SELF_HARM,CRIMES_ILLEGAL,MALICIOUS_USE,PHYSICAL_HARM,PRIVACY_PROPERTY,FINANCIAL_RISK,FRAUD_DECEPTION,ATTACK_MODEL}, where: the elements of the set are risk category identifiers, and all belong to the risk category set. If it exists {SELF_HARM,CRIMES_ILLEGAL,MALICIOUS_USE,PHYSICAL_HARM}, satisfies =3, then the overall risk level is forcibly increased to If it exists {PRIVACY_PROPERTY,FINANCIAL_RISK,FRAUD_DECEPTION}, satisfy =2, then the overall level is at least 2. ; 2.3 The overall risk level is obtained after the above fusion: { } 10. The method for enhancing large-scale safety barriers according to claim 7, characterized in that, Step 3 specifically includes: 3.1 Design three detection modes: lenient, normal, and strict. The specific modes are as follows: lenient mode is used to reduce false interception in low-risk and discussion contexts; normal mode is used to strike a balance between false interception and false negatives and is the default configuration; strict mode is used for high-sensitivity scenarios or high-risk business and focuses on reducing false negatives. 3.2 Design of a High-Risk Set ={PHYSICAL_HARM,SELF_HARM,CRIMES_ILLEGAL,MALICIOUS_USE,PRIVACY_PROPERTY,FRAUD_DECEPTION,FINANCIAL_RISK,ATTACK_MODEL,SEXUAL_CONTENT}, Soft risk set , "\" represents the difference operation, that is, the soft risk set represents the set of risk categories other than the strong risk set; 3.3 Define a threshold for a given pattern 𝛿∈{lenient,normal,strict}. , For high-risk collection The threshold for judgment, For soft risk set The threshold for determining violations is defined to predict the outcome of violations. The violation determination function is: when or hour, In other cases, The detection mode The threshold parameter pair used to select the mode To achieve different levels of violation determination strategies without altering the same risk assessment output, the threshold parameter is configured to... These are preset or configurable parameters, and they satisfy the following conditions: the lenient mode corresponds to a higher threshold, the strict mode corresponds to a lower threshold, and the standard mode is in between.

Images