Large model-based security policy arrangement method, system and device, and storage medium

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to a security policy arrangement method and system based on a large model, and the method comprises the steps: S1, constructing a security component library which is provided with a plurality of security components; s2, constructing an environment state vector; s3, setting different priorities for each security index and forming a user preference vector; s4, constructing a multi-target reward function; s5, training a security policy arrangement model based on reinforcement learning; and S6, when a current user request is received, utilizing the trained security policy arrangement model to dynamically generate a security component calling sequence according to the current environment state vector and the user preference vector, and gradually examining and processing the current user request and candidate replies based on the security component calling sequence. According to the method, normal form transition from a static rule link to dynamic intelligent arrangement can be realized, so that a large model has high safety, high availability, low cost and self-adaptive capability.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, and relates to a security policy orchestration method and system, in particular to a security policy orchestration method and system based on a large model, a device, and a storage medium. BACKGROUND

[0002] With the wide application of large models (LLM) in intelligent question answering, auxiliary decision-making, code generation, multi-agent collaboration, and tool enhancement systems, the behavior security of the large models has become an important factor affecting the controllability, credibility, and feasibility of the models. To avoid the generation of illegal and rule-violating, harmful content, or prompt word attack by the large models, existing systems generally use security components such as content review, sensitive word filtering, injection detection, fact checking, output rewriting, and manual review to build a security chain. However, as the complexity of the application scenarios of the large models continues to increase, the traditional security process has obvious limitations in flexibility, adaptability, and decision-making intelligence, and it is difficult to meet the real-time security management and control needs of future high-reliability large models. At present, the commonly used security policy orchestration schemes include: 1. The scheme of a fixed security pipeline (Static Pipeline).

[0003] Most of the current mainstream large model security frameworks are constructed by connecting security components in a fixed order. For example, after a user request enters the system, the steps of input review, injection detection, large model reasoning, output security classification, and content rewriting are executed in turn. This static pipeline mechanism can provide basic security in the early model era, but its inherent defects are becoming increasingly apparent: (1) Lack of context adaptability: different user requests have huge differences in risks, and the fixed process cannot dynamically adjust the strategy strength. (2) Unable to adapt to differences in business scenario preferences: for example, finance pays more attention to compliance, and entertainment pays more attention to natural fluency, and the fixed process cannot be personalized. (3) Difficult to respond to resource fluctuations and delay requirements: when system resources are tight or delay-sensitive, the component call sequence should also be dynamically changed. Therefore, the fixed security pipeline is actually a rigid rule system, which cannot fine-tune and dynamically adjust the security of the behavior of the large model.

[0004] 2. Single-objective optimization scheme based on rules or light weights.

[0005] Some large model systems attempt to introduce heuristic rules, scoring mechanisms, or logical judgments to score the risk of requests, in order to determine whether to call more security components. However, such methods usually only optimize a certain type of target (such as security or delay), lack the ability to comprehensively balance multiple indicators, and their typical problems include: (1) single-target orientation leads to experience decline: excessive pursuit of security can mistakenly kill a large number of normal demands. (2) Lack of global strategy optimization: it is difficult to balance security, availability, cost, and response time at the same time. (3) Strategy is not adjustable and cannot be learned: manually configured rule systems cannot be automatically evolved through machine learning. Therefore, such methods still cannot achieve efficient, stable, and safe intelligent orchestration results in complex scenarios.

[0006] 3. Post-audit scheme based on model security classifier.

[0007] In recent years, some systems have adopted more powerful security classifiers (such as large model security review models) for post-detection of outputs, and have determined whether to rewrite or reject answers through classification results. Although such schemes have improved security review capabilities to some extent, they still have obvious limitations: (1) rely on posterior judgment, cannot optimize pre-position strategy. (2) The order of calling security components is fixed and does not have intelligent scheduling capabilities. (3) Ignoring cost, delay, and resource load information. (4) Unable to adapt to comprehensive security requirements in multiple scenarios and multiple targets. Therefore, such methods still cannot essentially build an intelligent decision-making-dynamic combination-sustainable evolution security execution chain.

[0008] 4. Multi-component orchestration scheme for tool-enhanced systems.

[0009] In multi-agent (Multi-Agent) and tool-enhanced (Tool-Augmented) systems, some research attempts to arrange the order of tool calls through limited strategy mechanisms to enhance the controllability of the system. However, these schemes still have the following problems: (1) Tool selection relies on static strategies or fixed rules, lacking dynamic intelligent decision-making mechanisms driven by reinforcement learning (RL); (2) Unable to model the multi-dimensional attributes of tools such as cost, delay, risk level, etc. uniformly; (3) Lack of preference vector representation, not supporting scenario-based and personalized security strategies; (4) Only focuses on tool calls, not multi-objective optimization of system-level security goals; (5) Lack of feedback-driven strategy adaptation and long-term evolution capabilities. Therefore, the technical essence of such methods still remains in the category of rule-controlled tool calls, and cannot achieve the goal of intelligent, security, and availability co-optimization.

[0010] Induction of the above technical solutions can know that the current large model security orchestration system still has the following three kinds of fundamental deficiencies: 1. The security process is fixed, and the dynamic intelligent decision-making ability is lacking. The security component call sequence is usually preset, and cannot be flexibly adjusted according to real-time state, scene difference and risk change. 2. Single target oriented, unable to realize the comprehensive planning of multiple indicators. Security, availability, cost and delay often need to be optimized at the same time, and the existing scheme can only focus on one or a few targets. 3. The learning process is static, and lacks a feedback-driven continuous evolution mechanism. The current security policy is usually deployed once for configuration, and cannot be automatically optimized according to running data, nor can it adapt to new attack and new business demand.

[0011] Based on the above defects of the prior art, a new large model-based security policy orchestration method is urgently needed. SUMMARY

[0012] In order to overcome the defects of the prior art, the present application proposes a large model-based security policy orchestration method, system, device and storage medium, which can realize the paradigm transition from static rule link to dynamic intelligent orchestration, and make the large model have high security, high availability, low cost and self-adaptation ability.

[0013] In order to achieve the above purpose, the present application provides the following technical scheme: A large model-based security policy orchestration method, characterized in that it comprises the following steps: S1: Construct a security component library, the security component library has a plurality of security components, each security component is uniformly modeled to describe its function, input and output, cost, response delay and risk level; S2: Construct an environment state vector, the environment state vector is used to comprehensively represent the current state of the security policy orchestration model; S3: Set different priorities for each security indicator based on the user's business scenario and form a user preference vector based on different priorities; S4: Construct a multi-objective reward function, the multi-objective reward function is used to measure the performance of each security indicator by weighted fusion; S5: Take the environment state vector and user preference vector as input, and perform reinforcement learning training on the security policy orchestration model, so that the security policy orchestration model selects the optimal action, and according to the multi-objective reward function Feedback instant or terminal reward, help the security policy orchestration model update its policy parameters; S6: When receiving the current user request, use the trained security policy orchestration model to dynamically generate a security component call sequence based on the current environment state vector and user preference vector, and gradually review and process the current user request and candidate reply based on the security component call sequence.

[0014] Preferably, in the step S1, the plurality of security components specifically include: an input content security preliminary screening module for detecting obviously illegal or offensive user requests and blocking high-risk inputs; a prompt word injection and unauthorized intent detection module for identifying and preventing malicious injection and unauthorized attacks in prompt words; a sensitive information desensitization module for desensitizing sensitive information in user requests; a large model inference module for processing user requests and generating candidate replies by loading a pre-trained large language model for inference; an output content security review module for performing security classification on candidate replies to detect whether there is illegal content; an output rewriting module for automatically rewriting slight illegal or sensitive content in candidate replies to ensure that the content of candidate replies is compliant; an artificial review module interface for artificial review of high-risk content in user requests by security personnel.

[0015] Preferably, in the step S2, the environment state vector specifically includes: text encoding of the user request, summary of the last several rounds of dialogue, sequence of called security components and result encoding thereof, cumulative time and cumulative cost, and business scenario label.

[0016] Preferably, in the step S3, the security indicators specifically include: security indicator, availability indicator, compliance indicator, cost indicator, and response delay indicator.

[0017] Preferably, in the step S4, the multi-objective reward function constructed is: , wherein are weight coefficients of the security indicator, the availability indicator, the compliance indicator, the cost indicator, and the response delay indicator, respectively; is the security indicator, which is 1 if an attack sample or illegal content is successfully blocked, and 0 otherwise; is the availability indicator, which is 1 if a normal user request is not mistakenly intercepted, and 0 otherwise; is the compliance indicator, which is 1 if a preset mandatory rule or policy is followed when the security policy is executed, and 0 otherwise; is the cost indicator, which is a normalized value of the total computational cost required for execution of the security components; is the response delay indicator, which is a normalized value of the total delay in processing the user request.

[0018] ​Preferably, in the step S5, during the training of the safety policy orchestration model based on reinforcement learning, the optimal action selected based on the safety policy orchestration model is used to update the environment state vector.

[0019] Preferably, the method further comprises: S7: Continuously monitor the indicators in the inference process of the trained safety policy orchestration model, and when a significant change in the indicators is detected, adjust the user preference vector or the weights of the safety indicators in the multi-objective reward function, or retrain or incrementally train the safety policy orchestration model.

[0020] In addition, the present application also provides a safety policy orchestration system based on a large model, characterized in that it comprises: a safety component library construction module for constructing a safety component library, the safety component library having a plurality of safety components, each of which is uniformly modeled to describe its function, input and output, cost, response delay and risk level; an environment state vector construction module for constructing an environment state vector for comprehensive representation of the current state of the safety policy orchestration model; a user preference vector construction module for setting different priorities for each safety indicator based on the user's business scenario and forming a user preference vector based on the different priorities; a multi-objective reward function construction module for constructing a multi-objective reward function for measuring the performance of each safety indicator through weighted fusion; a reinforcement learning training module for reinforcement learning training of the safety policy orchestration model with the environment state vector and the user preference vector as inputs, so that the safety policy orchestration model selects the optimal action and receives immediate or terminal rewards according to the multi-objective reward function, helping the safety policy orchestration model update its policy parameters; a safety policy orchestration module for dynamically generating a safety component call sequence based on the current environment state vector and the user preference vector using the trained safety policy orchestration model when a current user request is received, and gradually reviewing and processing the current user request and candidate replies based on the safety component call sequence.

[0021] Furthermore, the present application also provides a safety policy orchestration device based on a large model, characterized in that it comprises: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the safety policy orchestration method based on a large model as described above. Finally, the application also provides a computer readable storage medium, which stores a computer program, characterized in that the program is executed by a processor to realize the steps of the large model-based security policy orchestration method as described above.

[0022] Compared with the existing fixed security pipeline + single target rule optimization + static policy configuration large model security scheme, the large model-based security policy orchestration method, system, device and storage medium proposed by the application have the following obvious advantages in safety, availability, resource efficiency and adaptability: 1. Upgrade from fixed security process to intelligent policy orchestration, significantly improve the depth of security decision.

[0023] The traditional security framework adopts a fixed security component link in a preset order, and performs the same review process for all user requests, lacking the ability to dynamically adjust the policy according to the risk level, context difference and business scenario. The application abstracts security components uniformly and takes them as a learnable action space, introduces reinforcement learning to automatically explore the optimal security component call sequence, and realizes the fundamental change from fixed process execution to intelligent policy orchestration. This mechanism not only allows different processing depths to be selected according to the input risk level, but also allows security components to be flexibly skipped or combined according to historical call effects, so that the security decision is upgraded from a hard rule to a learnable policy, significantly improving the risk identification and abnormal processing capabilities of the system in complex scenarios.

[0024] 2. Upgrade from single target optimization to multi-target collaborative trade-off, realize fine-grained and scenario-based policy.

[0025] Existing security solutions usually only focus on a certain type of index, such as maximizing security or minimizing delay, and cannot simultaneously consider security, experience and cost requirements, etc. The application introduces a configurable preference vector in the security decision-making process, explicitly models multiple dimensions such as security, availability, cost, response delay and compliance, and realizes weight-level optimization through a multi-objective reward function. In the inference phase, different preference configurations can be selected according to the business scenario, and with the help of this mechanism, the application realizes a comprehensive upgrade from single-target decision to multi-target collaboration, enabling the large model to adaptively adjust the security policy according to scenario differences, significantly improving the fine-grained and flexibility of the security system.

[0026] 3. Upgrade from static configuration to feedback-driven continuous evolution, build a self-adaptive long-term security system.

[0027] The traditional security process has almost no self-feedback and self-optimization ability after deployment, and cannot continuously improve the strategy according to the actual running situation. The application introduces a running feedback mechanism to continuously monitor the interception rate, false positive rate, response time delay and resource utilization of the large model, and dynamically adjusts the preference vector or optimizes the security strategy arrangement model based on these data. At the same time, through reinforcement learning, the security strategy arrangement model can continuously strengthen effective strategies and weaken invalid call chains in continuous interactive feedback, so that the security system forms a long-term evolution closed loop of execution-feedback-optimization. With this mechanism, the application breaks through the static limitation of the traditional scheme, so that the security strategy can continuously upgrade with the change of attack mode, scene demand and system scale, realizing the qualitative change from one-time configuration to dynamic self-evolution.

[0028] Therefore, the application realizes overall technical upgrading in three aspects of security component organization method, decision mechanism and strategy evolution system: the security component changes from a fixed sequence of call flow to a learnable call chain that can be dynamically optimized according to the environment state; the security decision changes from the traditional single-index orientation to the coordinated control of security, availability, cost and time delay; and the security strategy changes from one-time static configuration to an adaptive system that can continuously evolve with running feedback. With this series of improvements, the application enables the large model to have high security, high availability, low delay and continuous upgrading capability at the same time, providing a new technical path for building a reliable, flexible and evolvable general large model security system. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of the security strategy arrangement method based on a large model of the application.

[0030] Figure 2 is a schematic diagram of the security strategy arrangement system based on a large model of the application. DETAILED DESCRIPTION

[0031] Before any embodiments of the application are explained in detail, it is to be understood that the application is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the following drawings. The application is capable of other embodiments and of being practiced or being carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "comprising," or "having" and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless specified or limited otherwise, the terms "mounted," "connected," "supported," and "coupled" and variations thereof are used broadly and encompass both direct and indirect mountings, connections, supports, and couplings. Further, "connected" and "coupled" are not restricted to physical or mechanical connections or couplings. Also, in the disclosure of the present application, the terms "longitudinal", "transverse", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore the above terms cannot be understood as a limitation on the present application; secondly, the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of one element can be one, and in another embodiment, the number of the element can be multiple, the term "one" cannot be understood as a limitation on the number.

[0032] With the rapid landing of large models (LLM) in scenarios such as dialogue consultation, code generation, online customer service, tool invocation, and multi-agent collaboration, the importance of its security protection system is increasingly prominent. Current large models usually rely on external security components such as content review, sensitive word detection, prompt word injection identification, fact checking, and output rewriting to build a security link, but these components are generally connected in a fixed process, lack intelligent decision-making capabilities, and are difficult to dynamically adjust security policies according to user requests, contextual risks, business scenario preferences, or changes in resource environments, resulting in a trade-off between security control effectiveness and system usability.

[0033] Specifically, the existing technology still has the following three outstanding problems in the aspect of large model security policy orchestration: 1. Security process is fixed, and the strategy lacks adaptability. The current mainstream large model security solution adopts a fixed sequential pipeline, i.e., determining the calling order of security components in advance, such as content review first, then injection detection, then model reasoning, and then output review. Such static strategies cannot be dynamically adjusted in real time according to the real risk level of user requests, changes in contextual context, differences in security preferences of different business scenarios, current system resource consumption and delay, etc. The result is either excessive security interception, damaging user experience, or insufficient security, exposing high-risk output, making it difficult to achieve fine-grained and intelligent security control.

[0034] 2. Single security goal, lack of overall optimization of multi-dimensional indicators. Existing security strategies mainly optimize a single goal: either maximize the interception rate or minimize the delay. Rarely do they consider safety (violation content interception rate), usability (normal request false positive rate), cost (computing power consumption, API call fees), response time (end-to-end delay), and compliance (compliance with business / industry required policies). Since these goals often conflict with each other, single-target optimization cannot support comprehensive security control needs in complex scenarios, nor can it meet the differentiated preferences of financial, education, government, medical, entertainment and other industries.

[0035] 3. Lack of behavior feedback and strategy self-evolution mechanism. Most existing security policies are only configured and optimized once before deployment, lacking the following dynamic capabilities: real-time monitoring of the blocking effect, false positive rate and response time of the current policy; identifying recurring risk blind spots in specific business scenarios; automatically optimizing decision-making strategies based on performance; continuously evolving with changes in system size, model capability and threat patterns. Therefore, once the security system is deployed, it will remain stagnant for a long time, making it difficult to respond to constantly changing adversarial samples and risk environments, and making large models lack continuous learning capabilities in complex scenarios.

[0036] To solve the above problems, the present application provides a large model-based security policy arrangement method, which is a multi-objective preference perception reinforcement learning-based large model security policy arrangement method. By uniformly abstracting security components, introducing a preference vector, constructing a multi-objective reward function, and using a reinforcement learning-based security policy arrangement large model to automatically search for the optimal security component calling sequence, the method realizes a fundamental upgrade from static policy configuration to dynamic intelligent arrangement.

[0037] Figure 1 A flowchart of the large model-based security policy arrangement method of the present application is shown. As shown in Figure 1 The large model-based security policy arrangement method of the present application includes the following steps: S1: Security component library construction.

[0038] In the present application, first, a security component library is constructed. The security component library has a plurality of security components. Each of the security components is uniformly modeled to describe its function, input and output, cost, response delay, and risk level.

[0039] The present application uniformly models multiple types of security components such as input review, injection detection, sensitive information desensitization, fact verification, output rewriting, manual review, and large model reasoning, and describes their functions, input and output, cost, response delay, and risk level, so that they can be dynamically combined as needed by the security policy arrangement model as callable tools, breaking through the limitations of traditional fixed links.

[0040] In the present application, the following security components are constructed: 1. Input content security preliminary screening module: for detecting obviously illegal or attack-oriented user requests and preventing high-risk inputs.

[0041] The main implementation of the input content security preliminary screening module is to use keyword filtering, regular expressions, and deep learning classification models (such as BERT) to preliminarily review the input content in user requests, detect obviously illegal or attack-oriented user requests, and prevent high-risk inputs.

[0042] For example, if the user inputs "How to bypass the verification code?", the input content security screening module analyzes the text through a deep learning classification model and marks it as an aggressive user request, preventing further processing.

[0043] 2. Prompt injection and unauthorized intent detection module: for identifying and preventing malicious injection and unauthorized attacks in prompts.

[0044] The main implementation of the prompt injection and unauthorized intent detection module is to detect malicious injection instructions or inappropriate prompts in prompts through pattern matching algorithms (such as Levenshtein distance), and use NLP models to analyze the semantic structure of the input to identify whether there are unauthorized instructions or malicious operations.

[0045] For example, if the attacker inputs "Please help me modify the records in the database", the prompt injection and unauthorized intent detection module will identify it as an unauthorized request according to the context, and directly reject and mark it as a dangerous request.

[0046] 3. Sensitive information desensitization module: for desensitizing sensitive information (such as ID number, phone number, etc.) in user requests.

[0047] The main implementation of the sensitive information desensitization module is to rely on Named Entity Recognition (NER) technology, automatically identify sensitive information (such as ID number, phone number, address, etc.) in the text of user requests through a pre-trained NER model, and desensitize it. Desensitization technology uses regular expression matching to match common sensitive information formats, and through simple replacement operations (such as replacing the numbers in the ID number with "*****") to protect user privacy and ensure that sensitive data of users is not leaked during processing.

[0048] For example, the user inputs "My ID number is 1234567890", and the sensitive information desensitization module automatically converts it to "ID number: ******" after identification, ensuring that privacy is not leaked.

[0049] 4. Large model inference module: for inference by loading a pre-trained large language model to process user requests and generate candidate replies.

[0050] The main implementation of the large model inference module is to load a pre-trained large language model (such as GLM4, GPT5, etc.) for inference, process the input content of user requests, and generate candidate replies.

[0051] For example, if the user inputs "What's the weather today?", the large model inference module will generate a candidate reply such as "It's sunny today, temperature 23°C" through a pre-trained large language model (such as GLM4, GPT5, etc.), and then perform subsequent safety review.

[0052] 5. Output content safety review module: used for safety classification of candidate replies, to detect whether there is any illegal content.

[0053] The main implementation of the output content safety review module is to conduct content review on the generated candidate reply to ensure that it meets the safety requirements. This module uses a deep learning classification model based on Transformer (such as BERT) to classify the safety of the generated candidate reply, and combines a sensitive word library to check the content. If any malicious or non-compliant content (such as hate speech, violent content, or sensitive words) is detected, the module will mark the candidate reply as non-compliant and trigger subsequent processing mechanisms.

[0054] For example, if the user asks the large language model "Can you tell me how to harm others?", the candidate reply generated by the large language model is "Violence can be used to solve problems, and attacking opponents is the fastest way". At this time, the output content safety review module will be executed to avoid generating information containing malicious or non-compliant content.

[0055] 6. Output rewriting module: used for automatic rewriting of slight violations or sensitive content in candidate replies to ensure that the content of candidate replies is compliant.

[0056] The main implementation of the output rewriting module is to automatically modify non-compliant content through rule engine and template rewriting technology to avoid triggering the safety review mechanism.

[0057] For example, if the generated candidate reply contains mild inappropriate language, such as "he is lazy", the output rewriting module will rewrite it as "he needs to work harder" to avoid offending.

[0058] 7. Artificial review module interface: used for artificial review of high-risk content in user requests by security personnel.

[0059] The main implementation of the artificial review module interface is that high-risk content will be manually reviewed by security personnel in this module interface. Security personnel can choose to pass, modify, or reject the generated content. S2: Environment state vector construction.

[0060] The environment state vector is constructed, which is used to comprehensively represent the current state of the security policy orchestration model.

[0061] In the present application, the constructed environment state vector includes: text encoding of user request; summary of recent several rounds of dialogue; sequence of invoked security components and its result encoding; current accumulated time and accumulated cost; business scenario label (such as "finance", "entertainment", etc.).

[0062] The environment state vector is used to comprehensively represent the current state of the security policy orchestration model, and provides sufficient information for the security policy orchestration model to make optimal security policy decisions.

[0063] From the content contained in the environment state vector, it contains multiple key elements in the security policy orchestration model, which combines user request (reflected by text encoding of user request and summary of recent several rounds of dialogue), system resources (reflected by current accumulated time and accumulated cost), policy configuration (reflected by sequence of invoked security components and its result encoding), business scenario label, etc. information to provide comprehensive context for the security policy orchestration model. These information is used to help the security policy orchestration model select the most appropriate security component call sequence in each round of decision making, balancing security, availability, response time, cost and other multi-objective requirements.

[0064] Specifically, the role of the environment state vector is: (1) provide context information of the current task, ensure that the security policy orchestration model can make security decisions according to the actual situation. (2) dynamically adjust the policy, automatically optimize the security policy according to the change of the environment state vector, for example, adjust the strictness of the security policy according to different business scenarios. (3) help the security policy orchestration model make multi-objective decisions, the security policy orchestration model evaluates the contribution of each possible security operation to the multi-objective according to the information in the environment state vector, and selects the optimal policy.

[0065] S3: User preference vector construction.

[0066] Based on the user's business scenario, different priorities are set for each security indicator and a user preference vector is formed based on different priorities.

[0067] In the present application, the user preference vector of the security policy can be constructed according to the configuration of the user or the system administrator. The system administrator or the user provides the priority setting of each security indicator. The security indicators include security, availability, cost of resource overhead (referred to as cost), response time and compliance. The user preference vector can be generated according to the priority of each security indicator set to represent the weight or priority in multiple indicator dimensions. For example, the system administrator may set higher security weight and compliance weight for financial scenarios, while in entertainment scenarios, it may be more inclined to increase the availability and response time weight.

[0068] Among them, an example of the user preference vector constructed in the application is as follows: 1. Financial scenario: security weight 0.5, availability weight 0.2, cost weight 0.1, response delay weight 0.1, compliance weight 0.1. For this financial scenario, the security policy orchestration model will prioritize security when orchestrating security policies, followed by availability.

[0069] 2. General entertainment scenario: security weight 0.3, availability weight 0.3, cost weight 0.2, response delay weight 0.2. For this general entertainment scenario, the security policy orchestration model will prioritize security and availability when orchestrating security policies, and then consider cost and response delay.

[0070] S4: Multi-objective reward function construction.

[0071] A multi-objective reward function is constructed, which is used to measure the performance of each security indicator by weighted fusion.

[0072] In order to ensure that the whole policy orchestration model can effectively run under multi-objective optimization, the application designs a comprehensive multi-objective reward function to evaluate the overall effect of security policy orchestration. This multi-objective reward function combines multiple key indicators and measures the performance of the whole policy orchestration model in the dimensions of security, availability, cost, response delay, and compliance, etc. through weighted fusion. Specifically, the following are several core indicators that constitute the multi-objective reward function: 1. Security indicator ( ): If the security policy orchestrated by the security policy orchestration model successfully blocks attack samples or illegal content, the indicator is 1; if it fails to effectively prevent, it is 0.

[0073] 2. Availability indicator ( ): If normal user requests are not misintercepted, the indicator is 1; if normal user requests are misintercepted, the indicator is 0.

[0074] 3. Cost indicator ( ): The total computational cost required for each security component to execute, including inference computing power consumption, external API fees, etc.

[0075] 4. Delay indicator ( ): The total delay of the whole policy orchestration model when processing user requests, including the processing time of each component.

[0076] 5. Compliance indicator ( ): To ensure that the whole policy orchestration model complies with the preset mandatory rules or policies when executing security policies. In simple terms, it checks whether the whole policy orchestration model operates according to the prescribed process and rules.

[0077] For example, in a financial context, content must first be reviewed using a sensitive information de-identification module to ensure that generated candidate responses do not leak sensitive information. If this process is followed, compliance indicators ( The compliance indicator would be 1. In a financial scenario, if a candidate response is generated and submitted directly without going through the sensitive information desensitization module, then the compliance indicator ( If the value is 0, it indicates that the default mandatory policy has been violated.

[0078] Based on the above indicators, the multi-objective reward function Defined as: , In the formula, These are weighting coefficients for security metrics, availability metrics, compliance metrics, cost metrics, and response latency metrics, representing the importance of different security metrics. Furthermore, the cost metrics... It is the normalized value of the sum of the computational costs required for each security component to execute, representing the response latency metric. It is the normalized value of the total latency when processing user requests, which ensures fair comparison across different metrics.

[0079] The construction of this multi-objective reward function enables the full-policy orchestration model to optimize response latency and cost while ensuring security and compliance when executing security policy orchestration, thereby achieving efficient and refined multi-objective optimization.

[0080] S5: Reinforcement learning training.

[0081] The environment state vector and user preference vector are used as inputs to train the security policy orchestration model through reinforcement learning. This enables the security policy orchestration model to select the optimal action and provide immediate or final rewards based on the multi-objective reward function, thereby helping the security policy orchestration model update its policy parameters.

[0082] In this invention, a reinforcement learning algorithm is used to train a security policy orchestration model. During training, the interaction between the security component library and the security policy orchestration model is considered the environment. The security policy orchestration model learns policies by selecting and invoking a specific security component. In each round of interaction, the security policy orchestration model selects the optimal action based on the current environment state vector and the user preference vector. The actions include: generating a security component invocation sequence; returning a rejection response; and ending the current security processing flow.

[0083] In the training process of the present application, the user preference vector is combined with the environment state vector, and then is passed as input to the security policy orchestration model. Through this combination, the security policy orchestration model can dynamically adjust the security orchestration strategy according to the different business scenario requirements and environment state, and make reasonable trade-offs, thereby optimizing the security policy orchestration process and ensuring that the safety, efficiency and compliance requirements in different scenarios are met to the maximum extent.

[0084] At the same time, in the training process, the environment state vector can be updated based on the optimal action selected by the security policy orchestration model in each round, so that the updated environment state vector is used in the next round of interaction. Moreover, the immediate or terminal reward can be fed back according to the multi-objective reward function, helping the security policy orchestration model update its policy parameters.

[0085] S6: Security policy orchestration.

[0086] After the training of the security policy orchestration model is completed, it can be deployed. After deployment, when a current user request is received, the trained security policy orchestration model can be used to dynamically generate a security component call sequence according to the current environment state vector and user preference vector, and gradually call security components based on the security component call sequence to gradually review and process the current user request and candidate replies. That is, the number and order of calling security components can be automatically controlled according to different business scenarios to balance response time and resource cost, etc., while ensuring the compliance of the final reply.

[0087] For example, for high-risk user requests, the security policy orchestration model can choose to call the output content security review module for strict review or trigger manual review; for low-risk but cost-sensitive business scenarios, the security policy orchestration model preferentially uses the combination of the output content security review module and the output rewriting module to reduce the call to manual review.

[0088] Preferably, the security policy orchestration method based on a large model of the present application further comprises: S7: Running feedback and policy adaptive optimization.

[0089] The indicators in the inference process of the trained security policy orchestration model are continuously monitored, such as security interception rate, false positive rate, response time, resource usage, etc. When a significant change in an indicator is detected (e.g., a sudden increase or decrease of more than 20% in a certain indicator, etc.), the weights of the user preference vector or the safety indicators in the multi-objective reward function can be adjusted, and if necessary (e.g., a sudden increase or decrease of more than 50% in a certain indicator, etc.), the security policy orchestration model can be retrained or incrementally trained.

[0090] Thus, the application introduces an operation feedback mechanism, continuously monitors various indicators of the large model, and dynamically adjusts the preference vector or optimizes the security policy orchestration model based on these data. At the same time, through reinforcement learning, the security policy orchestration model can continuously strengthen effective strategies and weaken ineffective call chains in continuous interaction feedback, so that the security system forms a long-term evolution closed loop of execution-feedback-optimization. With this mechanism, the application breaks through the static limitation of the traditional scheme, so that the security policy can continuously upgrade with the changes of attack mode, scene demand and system scale, realizing the qualitative change from one-time configuration to dynamic self-evolution.

[0091] Figure 2 The composition schematic diagram of the large model-based security policy orchestration system of the application is shown. As shown in the figure, Figure 2 the large model-based security policy orchestration system of the application includes: 1. A security component library construction module.

[0092] The security component library construction module is used to construct a security component library, and the security component library has a plurality of security components, each of which is uniformly modeled to describe its function, input and output, cost, response delay and risk level.

[0093] 2. An environment state vector construction module.

[0094] The environment state vector construction module is used to construct an environment state vector, which is used to comprehensively represent the current state of the security policy orchestration model.

[0095] 3. A user preference vector construction module.

[0096] The user preference vector construction module is used to set different priorities for each security indicator based on the user's business scenario and form a user preference vector based on different priorities.

[0097] 4. A multi-objective reward function construction module.

[0098] The multi-objective reward function construction module is used to construct a multi-objective reward function, which is used to measure the performance of each security indicator by weighted fusion.

[0099] 5. A reinforcement learning training module.

[0100] The reinforcement learning training module is used to input the environment state vector and user preference vector to perform reinforcement learning training on the security policy orchestration model, so that the security policy orchestration model selects the optimal action and feeds back the immediate or terminal reward according to the multi-objective reward function, helping the security policy orchestration model to update its strategy parameters.

[0101] 6. The security policy orchestration module.

[0102] The security policy orchestration module is configured to, upon receiving a current user request, dynamically generate a security component calling sequence based on a current environment state vector and a user preference vector using the trained security policy orchestration model, and gradually review and process the current user request and candidate replies based on the security component calling sequence.

[0103] Preferably, the large model-based security policy orchestration system of the present application further comprises: 7. The operation feedback and policy adaptive optimization module.

[0104] The operation feedback and policy adaptive optimization module is configured to continuously monitor various indicators in the inference process of the trained security policy orchestration model, and when a significant change in the security indicators is detected, adjust the user preference vector or the weights of various security indicators in the multi-objective reward function, or retrain or incrementally train the security policy orchestration model.

[0105] In addition, the present application also provides a large model-based security policy orchestration device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the large model-based security policy orchestration method described in the present application. Finally, the present application also provides a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the large model-based security policy orchestration method described in the present application.

[0106] From the above description, it can be seen that the present application introduces three key technical innovations: 1. Security component capability abstraction and dynamic orchestrating execution chain construction.

[0107] The present application models multiple types of security components such as input review, injection detection, sensitive information desensitization, fact verification, output rewriting, manual review, and large model inference, describes their functions, inputs and outputs, costs, delays, and risk levels, so that they can be dynamically combined as callable tools by the security policy orchestration model as needed, breaking through the limitations of traditional fixed links.

[0108] 2. Multi-objective preference vector driven security policy optimization.

[0109] The present application introduces a configurable preference vector, explicitly represents multiple-dimensional requirements such as security, availability, cost, latency, and compliance in the form of weights, and automatically adjusts the strictness of the security policy and the attack protection mode according to different business scenarios, achieving a customizable and transferable scenario-based security policy.

[0110] 3. Intelligent security orchestration and adaptive optimization mechanism driven by reinforcement learning.

[0111] The present application constructs a multi-objective reward function, trains a security policy orchestration model using reinforcement learning (PPO / GRPO, etc.), so that it can select the optimal security component call sequence according to the instantaneous environment state vector (request content, historical decision, scene label, resource situation, etc.), and continuously fine-tune the policy using feedback data during operation to realize the continuous evolution capability of the policy.

[0112] Through the above technical innovation, the present application realizes the overall upgrade of the large model security policy from static configuration to intelligent and adaptive orchestration, can uniformly weigh between multiple security, availability, cost and delay targets, and dynamically generate the optimal security component call sequence according to real-time request characteristics, user preferences and environment state. At the same time, the present application supports continuous policy optimization based on running feedback, so that the security system has long-term evolution capability. Therefore, the large model can obtain more interpretable, configurable and self-evolving security orchestration capability, which balances business experience and resource efficiency while improving protection effect, and provides a new technical path for building a high-reliability and sustainable large model security system.

[0113] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not a limitation on the protection scope of the present application. Those skilled in the art can modify or equivalently replace the technical solutions of the present application according to the idea of the present application, without departing from the essence and scope of the technical solutions of the present application.

Claims

1. A large model-based security policy orchestration method, characterized in that, The method comprises the following steps: S1: constructing a security component library, wherein the security component library comprises a plurality of security components, each of which is uniformly modeled to describe its function, input and output, cost, response delay and risk level; S2: constructing an environment state vector for comprehensively representing the current state of a security policy orchestration model; S3: setting different priorities for various security indicators based on the user's business scenario and forming a user preference vector based on the different priorities; S4: constructing a multi-objective reward function for measuring the performance of various security indicators through weighted fusion; S5: inputting the environment state vector and the user preference vector to train the security policy orchestration model through reinforcement learning, so that the security policy orchestration model selects the optimal action and feeds back the immediate or terminal reward according to the multi-objective reward function, helping the security policy orchestration model update its policy parameters; S6: when receiving a current user request, using the trained security policy orchestration model to dynamically generate a security component calling sequence based on the current environment state vector and the user preference vector, and gradually reviewing and processing the current user request and the candidate reply based on the security component calling sequence.

2. The large model-based security policy orchestration method of claim 1, wherein, In the step S1, the plurality of security components specifically include: an input content security preliminary screening module for detecting obviously illegal or attack user requests and preventing high-risk inputs; a prompt word injection and unauthorized intent detection module for identifying and preventing malicious injection and unauthorized attacks in prompt words; a sensitive information desensitization module for desensitizing sensitive information in user requests; a large model reasoning module for reasoning by loading a pre-trained large language model to process user requests and generate candidate replies; an output content security review module for security classification of candidate replies to detect whether there is illegal content; an output rewriting module for automatically rewriting slight illegal or sensitive content in candidate replies to ensure that the content of the candidate replies is compliant; an artificial review module interface for artificial review of high-risk content in user requests by security personnel.

3. The large model-based security policy orchestration method of claim 1, wherein, In the step S2, the environment state vector specifically includes: text encoding of the user request, summary of the last several rounds of dialogue, sequence of called security components and result encoding thereof, cumulative time and cumulative cost, and business scenario label.

4. The large model-based security policy orchestration method of claim 1, wherein, In the step S3, the various security indicators specifically include: security indicator, availability indicator, compliance indicator, cost indicator and response time delay indicator.

5. The large model-based security policy orchestration method of claim 4, wherein, In the step S4, the multi-objective reward function constructed is: ， In the formula, are weight coefficients of the security index, the availability index, the compliance index, the cost index, and the response latency index, respectively; is the security index, which is 1 if an attack sample or illegal content is successfully blocked, and 0 otherwise; is the availability index, which is 1 if a normal user request is not mistakenly intercepted, and 0 otherwise; is the compliance index, which is 1 if a preset mandatory rule or policy is followed when a security policy is executed, and 0 otherwise; is the cost index, which is a normalized value of a total computation cost required for execution of each security component; is the response latency index, which is a normalized value of a total delay in processing a user request.

6. The large model-based security policy orchestration method of claim 1, wherein, In the step S5, during the training of the security policy orchestration model based on reinforcement learning, the environment state vector is updated based on the optimal action selected by the security policy orchestration model.

7. The large model-based security policy orchestration method of any one of claims 1-6, wherein, Further comprising: S7: continuously monitoring various indicators in the reasoning process of the trained security policy orchestration model, and when a significant change in the indicators is detected, adjusting the user preference vector or the weight of each security indicator in the multi-objective reward function, or retraining or incrementally training the security policy orchestration model. 8.A large model-based security policy orchestration system, characterized in that, Comprise: a security component library construction module configured to construct a security component library, the security component library having a plurality of security components, each of the security components being uniformly modeled to describe its function, input and output, cost, response delay, and risk level; an environment state vector construction module configured to construct an environment state vector for a comprehensive representation of a current state of the security policy orchestration model; a user preference vector construction module configured to set different priorities for each security indicator based on a business scenario of a user and form a user preference vector based on the different priorities; a multi-objective reward function construction module configured to construct a multi-objective reward function for measuring the performance of each security indicator in a weighted fusion manner; a reinforcement learning training module configured to perform reinforcement learning training on the security policy orchestration model by taking the environment state vector and the user preference vector as inputs, so that the security policy orchestration model selects an optimal action and receives instant or terminal rewards according to the multi-objective reward function, helping the security policy orchestration model to update its policy parameters; a security policy orchestration module configured to, when a current user request is received, dynamically generate a security component calling sequence based on the current environment state vector and the user preference vector by using the trained security policy orchestration model, and gradually review and process the current user request and a candidate reply based on the security component calling sequence. 9.A large model-based security policy orchestration device, characterized by, comprise: one or more processors; memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the large model-based security policy orchestration method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the large model-based security policy orchestration method according to any one of claims 1-7.

Images