Adaptive runtime safety orchestration system and method for large language models.

Abstract

The invention relates to a system and method that enables the dynamic and adaptive management of security at runtime in software systems based on the big language model (108), instead of predefined fixed security chains. In the system, the incoming request is normalized by the request normalizer (101); evaluated by the threat analysis engine (102) and the multi-layered adaptive risk scoring engine (103); the adaptive runtime security planner (105) generates a custom security execution plan (106) in line with the strategy determined by the security policy decision engine (104), and a dynamic security adapter chain (107) is created at runtime in accordance with this plan. The output production of the big language model (108) is continuously monitored by the production monitor (109), and if the security risk changes, the runtime security replanning engine (110) reconfigures the existing plan during the process.In the system running on the processing unit (115) and memory unit (116), the output is transmitted as a secure response (112) after being verified by the response analyzer (111).

Description

1 TARIFF Adaptive runtime safety orchestration system for large language models and method Technical Area The invention relates to software systems based on Large Language Models (LLM). a system that provides dynamic and adaptive security orchestration at runtime and it relates to the method. 10 State of the Art Today, in order to provide security in applications based on large language models... The systems used are mostly predefined fixed security chains (fixed security 15 It operates with pipelines. In this approach, security controls are implemented through prompt injection (prompt). injection filters, jailbreak detection, personal data masking, content from mechanisms such as moderation, security policy controls and output validation These controls are formed and these controls are applied to each user request in a predetermined fixed order. It is implemented as follows. 20 Current security solutions generally rely on one of two main approaches: The first approach is rule-based fixed security chains. In these systems, security controls are applied. It operates within predefined rules. For example, each user request sequentially follows a 25 a prompt injection filter, a content moderation check, and an output validation. It goes through stages. These stages are fixed and depend on the content of the request, the level of risk, or the use. Regardless of the scenario, it is applied in the same way. The second approach is API-based security gateways (API Gateway Security). These systems have 30 Security checks are constants placed before or after large language model API calls. This is done through filters. Pre-request (preprocessing) and post-request (postprocessing) Security checks are run in a specific order. Some systems have limited AI-assisted threat analysis or predefined security features. 35 While there are rules in place, these solutions depend on the nature of the request, the level of risk, and the use. 2 According to the scenario, it has the ability to dynamically adapt its security mechanisms. No. No existing solution provides a different security measure at runtime for each user request. creating the architecture, reconfiguring this architecture during the process, and security. It does not offer the ability to adaptively orchestrate its components. The main shortcomings of these current techniques are explained in detail below: Static safety chain structure: Existing solutions provide the same safety control to all user requests. It employs a sequence of actions. A low-risk conversation request (e.g., "What's the weather like today?") is followed by a high-risk approach. a risky financial transaction (for example, "Query customer's account information and set credit limit to 10") The update) goes through the same security chain. This makes it unnecessary in low-risk transactions. This leads to delays, increased processing costs, and wasted resources; it is also high-risk. In these processes, the inadequacy of the fixed chain and the lack of critical security controls are the reasons. This leads to implementation problems and security vulnerabilities. Lack of runtime adaptation: Security plan, request in existing systems. It is determined at the beginning and cannot be changed throughout the process. For example, an Information The RAG (Rail-Based Manufacturing) system processes the user's request from an external document source. It can pull unexpected sensitive data or involve request injection when the model generates output. can create content. In such cases, existing security chains allow new 20 in the middle of the process. You cannot add security components, remove existing components, or change the security sequence. It cannot be changed. Ability to recreate the safety plan during operation. It is not available. Lack of task-based security differentiation: Different major language model use cases 25 They have fundamentally different security requirements. The security needs of a Chat Assistant. The security needs of an SQL Agent are entirely different from those of a Chat Assistant. While filtering and content moderation may be sufficient, SQL injection detection for SQL Agent, Additional controls such as query validation and database access control are mandatory. Similarly, While a Healthcare Assistant requires HIPAA compliance, a Banking Assistant requires financial 30 It requires compliance checks and transaction policy verification. Lack of risk-based decision-making mechanisms: Existing systems are generally single-layered and rule-based. security decisions are being made. However, a true risk assessment involves user profile risk, requests, etc. content risk, data sensitivity risk, model risk, tool call risk, compliance risk, and past 35 It requires a combined assessment of multiple dimensions, such as threat history. Current 3 a multi-layered, adaptive risk assessment that primarily combines these dimensions in solutions There is no mechanism. Limitations of personal data protection: In current solutions, personal data is often protected through simple masking. or protected by regular expression (regex) based identification methods. Token level 5 early conversion, placeholder mapping, recyclable encryption, and secure return Advanced data protection mechanisms such as transformation are included in existing solutions. It is not available. Lack of policy and regulation integration: Existing systems, corporate security 10 policies and regulatory rules (KVKK, GDPR, HIPAA, PCI-DSS, SOX, etc.) security It is unable to evaluate policy decisions in an integrated manner with the decision-making process. Policy decisions are generally It is handled outside the chain of security, in a separate process. Lack of threat memory and learning: Current security systems treat each request independently. evaluating and analyzing previously encountered attack patterns, threat signatures, or They cannot recall failed attack attempts. Threat memory and history at the semantic level. Adaptive risk scoring mechanisms based on threat data are included in existing solutions. It is not available. For these reasons, current approaches fail to address security in applications based on large language models. It fails to ensure that it is managed dynamically, adaptively, and contextually. As a result, the institution And organizations face either performance loss and high transaction costs resulting from unnecessary security controls. data leaks, policy breaches, and resulting costs or inadequate security controls It is exposed to the risks of regulatory non-compliance. 25 The summary of application number US20250209208A1, which emerged as a result of technical research, is as follows: "The topics within the request that are to be given as input to a language model are determined, 30 Whether the request is harmful depends on the degree of deviation and inconsistency between the issues. The language model of the request that was determined to be harmful and not present was found to be harmful. "a method by which processing is prevented". 4 As can be seen, the invention is based on examining the user request through semantic analysis of the request. This relates to the detection of injection attacks, as well as the issues mentioned above. It does not mention a structure that could provide solutions to the disadvantages. In conclusion, due to the negative aspects described above and the current solutions, topic 5 Due to its shortcomings, it has become necessary to make improvements in the relevant technical field. Purpose of the Invention The invention represents a new breakthrough in this field, unlike the structures used in the current technology. The aim is to create a structure with different technical specifications that bring these elements together. The primary purpose of the invention is to improve the safety mechanisms used in large language models. Instead of predefined fixed security chains, it works for each user request. 15 The goal is to develop a system and method that provides this. The invention utilizes artificial intelligence, large language models, Generative AI, cybersecurity, secure AI systems, and artificial intelligence. It is used in the field of intelligence management (AI governance). In this invention, each user request is evaluated independently, and the security context of the request is 20. The security context is created based on the nature of the user request and the type of task (Chat, Information). (Model-driven generation, agent, SQL, code generation, etc.), large language model used, data sensitivity, user trust level, user permissions, corporate security policies, regulation requirements (KVKK, GDPR, HIPAA, PCI-DSS, etc.), acceptable delay for processing and It is created by taking into account cost targets and other runtime parameters. 25 In the invention, a dynamic safety adapter chain was created in accordance with the safety implementation plan. Only the security components necessary for the specific request are activated. For example, a low-risk request... When only the request filter and output validation are run in the chat request, it poses a high risk. Detection of request injection in financial transactions, detection of circumvention of restrictions, personal 30 at the token level. data transformation, policy validation, information access validation, and audit logging, etc. All components can be included in the supply chain. One of the key features of the invention is that the safety plan is only required at the request stage. Failure to create one means the safety status remains constant at 35 while the model continues to generate output. monitoring and if changes in security conditions are detected, the existing execution plan will be modified. It can be recreated (Runtime Security Rescheduling). For example, When the model generates output and creates content containing sensitive data, an unexpected tool emerges. When a call is made or content containing request injection is detected, the system, existing The safety plan can be restructured during the operation. 5 The system's ability to differentiate security based on task type is suitable for different use cases. It automatically selects the appropriate security adapters. For example, for the chat assistant. Request filtering and output validation; information access validation for knowledge-based production systems. Source validation and citation validation; SQL injection detection for SQL agent, query 10 Authentication and access control; static code analysis for code generation agent, sandbox policy. Verification and dependency verification; HIPAA policy for health assistant, token level. Personal data protection and medical data verification; financial compliance for banking assistants. Verification, transaction policy control, and audit logging adapters are implemented by the system. It is determined automatically. 15 The system's multi-layered adaptive risk scoring capability ensures that safety decisions are made according to a single rule. No, it ensures that it is based on a weighted assessment of multiple risk dimensions. User Profile risk, request content risk, data sensitivity risk, model risk, tool call risk, compliance By evaluating components such as risk and historical threat score together, the total security is calculated as 20. The risk is being calculated. In terms of personal data protection, the system utilizes existing simple masking methods. Furthermore, it offers an early conversion mechanism at the token level. Large language model Before the call, the input text is tokenized to identify sensitive data, and token replacement takes place on 25 days. placeholder matching, hashing, encryption, or dynamic masking It is converted using one of the methods. The system also recalls previously identified security threats through its semantic threat memory. By semantically storing attack patterns and threat signatures, the risk of subsequent requests is reduced. It includes this in its assessment. The system in question can be used on any software platform that works with large language models. It can be positioned as a security layer. These platforms include the following: 6  Chat applications (Chatbots) and virtual assistant systems: These systems allow users to interact with natural language. customer service, technical support and information provision that it interacts with applications,  Enterprise AI platforms: Business processes of large-scale organizations artificial intelligence (AI) is used to automate, provide decision support, and perform data analysis. intellectual infrastructures,  Artificial intelligence agents (AI Agents): External tools, databases, and Agent-based systems that perform autonomous tasks by interacting with APIs,  Knowledge-based generation (RAG) systems: Generating large language models from external knowledge sources (data from databases, document repositories, infographics) Enriched responses by extracting data 10 The systems it produces,  Code generation systems: Software code from large language models, database queries, or Development environments that generate configuration files,  Decision support applications: Such as medical diagnosis, financial analysis, and legal evaluation. Systems where critical decisions are made with the support of artificial intelligence, 15  Multiple model orchestration platforms: Different major language model providers working together Platforms where the model is used and dynamically selected. The invention involves the dynamic evaluation of user requests and model outputs, and security. multi-layered analysis of risks, corporate security policies and regulations 20 automatic enforcement of rules, protection of personal data at the token level and each Creating adaptive safety orchestration at runtime for user request. It can be implemented in software-based security infrastructures that provide this. The system works fundamentally differently from existing fixed security chains: Each user 25 For the request, the content of the request, the risk level, the type of task, the model used, the data sensitivity, and A unique security implementation plan is being created by evaluating corporate policies; this The plan can be reconfigured during the process if necessary. Furthermore, the invention creates particular value in regulated sectors: 30  Financial sector: In banking and insurance applications that work with large language models. transaction security, financial compliance control, and fraud detection,  Healthcare sector: Protecting patient data in medical AI applications, HIPAA Compliance and safety checks for medical decision support,  Public sector and defense: Artificial intelligence used in government institutions and the defense industry 35 data protection and access control at the national security level in their systems, 7  Telecommunications: AI-based telecommunications where customer data is processed. KVKK and GDPR compliance in its services,  Law: Artificial intelligence that performs legal document analysis, contract review, and legal research. professional confidentiality and data protection in their practices,  Education: 5 in learning management systems and artificial intelligence applications in education Student data protection. The invention will work on cloud-based, on-premise, or hybrid infrastructures. It is designed to be a middleware for existing large language model infrastructures. It can be integrated as such. The system supports 10 different major languages ​​without being tied to a single model provider. It can work with various models and provides model-independent safety orchestration. To fulfill the objectives described above, the invention provides dynamic and efficient solutions for large language models. It is a system that provides adaptive safety orchestration, and its feature is;  Receives the raw request from the user or client application and formats the request 15 standardizes, cleans input data, verifies character encoding, and requests a request that converts the system into a standard structure that can be processed by other components of the system normalizing,  analyzing the normalized request, identifying potential security threats in the request content threat analysis engine that assesses its presence, 20  User profile that receives threat analysis results generated by the threat analysis engine. risk, request content risk, data sensitivity risk, model risk, tool call risk, compliance by weighting multiple risk dimensions such as risk and historical threat score historical threats derived from threat memory, which generates a composite security risk score. Multilayer adaptive 25 that performs adaptive evaluation using data as well. risk scoring engine,  Risk information obtained, organization's safety policies and regulatory rules considering together corporate security policies, regulatory requirements, by considering user permissions, task type, and system configurations together. Security policy decision 30 specifies the security strategy to be implemented for the request. engine,  located at the center of the system and using all the information from previous components creating a customized security context and providing security based on that context. Generating an execution plan, identifying which security adapters are used for each user request. that they will be operated, their order of operation, their dependencies, their working conditions and 35 8 Adaptive runtime security that dynamically determines decision points. planner,  generated by the adaptive runtime safety planner, for the relevant request defining the security architecture to be implemented, and the security adapters to be run, the order in which the adapters operate, the dependencies between the adapters, and under what conditions 5 A replanning will be carried out, including security measures such as risk thresholds and exit decisions. execution plan,  Created during the work period in accordance with the security execution plan, including Security adapters will be installed only according to the security requirements of the relevant request. The selected dynamic safety adapter chain, 10  Processing the request passed through the dynamic security adapter chain, according to the user's request a large language model that produces suitable output,  Content produced and working status throughout the output generation process of the large language model by continuously monitoring it and generating sensitive data in the model output. whether it was produced, whether it contains content that violates security policies, and 15 whether unexpected vehicle requests are being made in real time If the assessor determines that the safety risk has changed, the working time will be adjusted accordingly. Production monitor that triggers the safety rescheduling engine,  Re-enactment of the current security execution plan, triggered by the production monitor. The detection of an involuntary injection during the procedure, which allows for its evaluation, 20 Events such as the discovery of sensitive data or unexpected model output may occur. New security adapters that come into play as a result of replanning. Workarounds that can add, remove existing adapters, or stop the process. Time security rescheduling engine,  Security policies can be developed by evaluating the output produced by the large language model. checking for compliance, whether the output contains personal data, and security. It evaluates whether content that violates its policies has been produced and carries out the necessary verification. Response analyzer that performs conversion operations,  Verified by the response analyzer and necessary conversion processes completed. The output is the response sent to the user. The security of the response received by the user is 30. A secure response that complies with its policies and guarantees the protection of personal data.  Operating within a dynamic security adapter chain, ensuring the protection of personal data. going beyond the masking level, the introduction before the call for a grand language model text tokenization, sensitive areas identification, token substitution, placeholder 35 with one of the following methods: matching, hashing function, encryption or dynamic masking 9 early personal data conversion at the token level that performs the conversion mechanism,  Security threats and attack patterns previously identified by the system, and Semantic threat memory, which semantically stores and queries threat signatures.  Request normalizer, threat analysis engine, multi-layered adaptive risk scoring engine, 5 security policy decision engine, adaptive worktime security planner, dynamic safety adapter chain, production tracker, runtime safety rescheduling engine, response analyzer, token-level early personal data transformation the mechanism and processes carried out by semantic threat memory processing unit, 10  associated with the processing unit and holding the data generated throughout the processing of the request, a memory unit that can be temporary or permanent It includes. The structural and characteristic features and all the advantages of the invention are given in the figures below and in these 15 Thanks to the detailed explanation written with references to the figures, it becomes clearer. This will be understood, and therefore the evaluation should also take these figures and detailed explanations into account. It must be done by taking it. Figures to Help Understand the Invention 20 Figure 1 is a schematic representation of the system that is the subject of the invention. The drawings do not necessarily need to be scaled and are necessary for understanding the invention. Details that are not present may have been overlooked. Furthermore, at least to a large extent, 25 Elements that are identical or at least have substantially identical functions are numbered the same. It is shown. Explanation of Part References 101. Desire Normalizer 30 102. Threat Analysis Engine 103. Multilayer adaptive risk scoring engine 104. Security policy decision engine 105. Adaptive working time safety planner 106. Security execution plan 35 107. Dynamic safety adapter chain 107.1 Detection of request injection 107.2 Detection of circumvention of restrictions 107.3 Information access security 107.4 Policy verification 107.5 Task-specific adapters 5 108. Major Language Model (MLM) 109. Production viewer 110. Work time safety rescheduling engine 111. Response Analyzer 112. Safe response 10 113. Early personal data transformation mechanism at the token level. 114. Semantic threat memory 115. Processing unit 116. Memory unit Detailed Description of the Invention In this detailed explanation, the preferred configurations of the invention are not merely for better understanding the subject. in order to facilitate understanding and without imposing any limiting effects It is explained. 20 The invention relates to software systems based on Large Language Models (LLM). a system that provides dynamic and adaptive security orchestration at runtime and it is related to method. The elements and functions used in the system and method that is the subject of the invention are as follows: Request normalizer (101) takes the raw request from the user or client application and standardizes the request format, cleans the input data, and verifies character encoding. and transforms the request into a standard structure that can be processed by other components of the system. It is a unit. The normalization process ensures that requests from different client applications are consistent. It allows for the matter to be addressed. The threat analysis engine (102) analyzes the normalized request, identifying potential threats in the content of the request. security threats (probabilistic injection attempts, restriction bypass attempts, malicious content, 35 It is the unit that assesses the existence of manipulative patterns, etc. Threat analysis engine (102); analysis 11 The results include a threat profile, along with the types of threats identified and their reliability levels. It generates the report and passes it on to subsequent elements. Multilayer adaptive risk scoring engine (103), threat analysis engine (102) Receiving the generated threat analysis results, user profile risk, request content risk, data sensitivity 5 multiple risks such as model risk, vehicle call risk, compliance risk and historical threat score a composite security risk score that produces a threat by primarily evaluating its dimensions using historical threat data obtained from memory (114) an adaptive assessment It is the unit that carries out the work. The security policy decision engine (104) uses the risk information obtained to determine the organization's security considering policies and regulatory rules (KVKK, GDPR, HIPAA, etc.), corporate security policies, regulatory requirements, user permissions, task type and by considering the system configurations together, what needs to be done for the relevant request. It is the unit that determines the security strategy. 15 The adaptive runtime safety scheduler (105) is central to the system and is the previous all information from the components (normalized request, threat analysis, risk score, policy) (using decisions) to create a customized security context and acting in accordance with that context 20 generating a security execution plan (106), specifying which security adapters for each user request to be executed, their order of operation, their dependencies, working conditions and decision It is a unit that dynamically determines the points. The security execution plan (106) is prepared by the adaptive runtime security planner (105). generated, defining the security architecture to be applied for the relevant request, security to be executed 25 adapters, the order in which the adapters operate, the dependencies between the adapters, which replanning will be done under the conditions, risk thresholds and exit decisions (continue, add This is a plan that includes (apply control, seek human approval, reject the request). Dynamic safety adapter chain (107), working in accordance with safety execution plan (106) 30 created in a timely manner, the security adapters to be included are only those required by the relevant request. It is the chain selected according to the safety requirements. Dynamic safety adapter chain (107); request injection detection (107.1), restriction breach detection (107.2), early personal injection at token level data transformation mechanism (113), policy validation (107.4), information access security (107.3) and may include task-specific adapters (107.5). 35 12 The large language model (108) processes the request passed through the dynamic security adapter chain (107), It is the model that produces the output that is in accordance with the user's request. The operation of the large language model (108) During production, continuous monitoring is carried out by the production monitor (109). The production tracker (109) is the content produced throughout the output production process of the big language model (108) and 5 It continuously monitors the operating status and produces precise data in the model output. whether it was produced, whether content that violates security policies was created, and unexpected tools. assessing in real time whether calls are being made and evaluating security risks. If it is determined that the runtime has changed, the safety rescheduling engine will be activated. (110) is the triggering unit. 10 Working time safety rescheduling engine (110), production tracker (109) triggered, which led to a reassessment of the current security execution plan (106), Detection of unsolicited injection during the process, presence of sensitive data, or unexpected events When events such as model output occur, it is activated, resulting in replanning of 15 can add new security adapters, remove existing adapters, or modify the process. It is a unit that can be stopped. Updated plan, adaptive runtime safety planner (105) with a feedback loop to the component (Replanning Feedback Loop) It is being transmitted. The response analyzer (111) evaluates the output produced by the large language model (108) Checking compliance with security policies, and whether personal data is present in the output, Evaluating whether content violating security policies has been produced and conducting the necessary verification. It is the unit that performs conversion operations. The safe response (112) has been verified by the response analyzer (111) and the necessary conversion has been completed. The completed process is the output sent to the user as a response. The response received by the user... It complies with security policies and guarantees the protection of personal data. Early personal data transformation mechanism at token level (113), dynamic security 30 The level of masking of personal data protection that works within the adapter chain (107) tokenize the input text before calling on the big language model (108) and beyond. Identifying sensitive areas, token substitution, placeholder matching, hash function, encryption, or It is a unit that performs transformation using one of the dynamic masking methods. Large language The model (108) only sees the converted data. Secure post-processing recycling 35 applicable. 13 Semantic threat memory (114) contains security threats previously detected by the system, memory that semantically stores and queries attack patterns and threat signatures It is the structure. The multilayer adaptive risk scoring engine (103) uses this memory to analyze past It incorporates threat data into the risk assessment. Continuously updated with new threat detections. 5 It is updated. Processing unit (115), request normalizer (101), threat analysis engine (102), multilayer adaptive risk scoring engine (103), safety policy decision engine (104), adaptive working time safety planner (105), dynamic safety adapter chain (107), production tracker (109), 10 Runtime Security Rescheduling Engine (110), Response Analyzer (111), Token early personal data transformation mechanism (113) and semantic threat memory at the level It is the unit that carries out the operations performed by (114). The memory unit (116) is connected to the processing unit (115) and generates 15 during the processing of the request. It is a unit that stores data, which can be temporary or permanent. As a result of these elements working together, the system dynamically responds to each user request. an adaptive security system that is created and can be reconfigured during operation if necessary. to produce an execution plan; thus ensuring the security of large language models through static security chains 20 instead with an adaptive safety orchestration that adapts to working time conditions. It makes it possible to provide this. The process carried out with the system which is the subject of the invention and performed by the processing unit (115) The steps are as follows; 25  The request from the user or client application is normalized by the request normalizer (101) normalization and conversion to a standard processing format,  The normalized request is analyzed by the threat analysis engine (102) and potential Identifying security threats,  Multilayer adaptive risk scoring engine (103); user profile, content of request, data 30 sensitivity, task type, model characteristics, and other runtime parameters by evaluating and creating a multi-layered security risk score, and this evaluation During this time, past threat data obtained from semantic threat memory (114) are also risk including it in the scoring, (so that the system can identify previously encountered attack patterns) (By recognizing the risks, a more accurate assessment can be made.) 35 14  The risk information obtained should be combined with organizational policies and regulatory rules. Evaluation by the security policy decision engine (104) and security policy the security strategy that should be implemented for the relevant request by the decision engine (104) by determining and transferring it to the adaptive work time safety planner (105),  The Adaptive Runtime Security Planner (105) creates the security context 5 using a custom security execution plan (106) and this execution plan; the safety adapters to be implemented, the operating sequences, the execution conditions and the decision defining the points and the security execution plan (106) on the memory unit (116) eclipse,  Dynamic security adapter 10 in accordance with the security execution plan (106) creation of the chain (107) at working time and the security to be included in this chain adapters should be selected only according to the security requirements of the relevant request.  Early personal data at the token level within a dynamic security adapter chain the transformation mechanism (113), personal data before the big language model call By detecting and converting at the token level, it provides different security features depending on the task type. 15 Automatic activation of adapters (for example; Request for Chat Assistant) Filtering and Output Validation; Information Access Validation for Knowledge-Based Manufacturing systems. Source Validation and Citation Validation; SQL Injection Detection and Query for SQL Agent. Authentication and Access Control; Static Code Analysis for Code Generation Agent, Sandboxing Policy Verification and Dependency Verification; HIPAA Policy for Health Assistant, Token 20 Personal Data Protection and Medical Data Verification at the Level; for Banking Assistant Financial Compliance Verification, Transaction Policy Control and Audit Logging (adapters are selected.)  Request passed through the dynamic security adapter chain (107) to the large language model (108) 25 by transmitting and processing and the production process of the big language model (108) throughout the output generation process. the viewer (109) continuously monitors the produced content and working status. to accomplish,  If a change in security risk is identified during monitoring, the operating time will be adjusted. Security replanning engine (110) is activated and current security execution to ensure the reassessment of the plan (106) and the updated plan, feedback 30 adaptive runtime through a cycle (rescheduling feedback loop) The operation of the large language model (108) is communicated to the security planner (105) component. continuation in a safe manner,  Finally, the response analyzer (111) analyzes the output produced by the large language model (108) by evaluating and checking its compliance with security policies and 35 After the necessary verification and conversion processes are completed, the secure system is sent to the user. Transmission of the response (112).

Claims

16 REQUESTS 1. It is a system that provides dynamic and adaptive security orchestration for large language models, feature;  User 5 receives the threat analysis results generated by the threat analysis engine (102). Profile risk, request content risk, data sensitivity risk, model risk, tool call risk, weighted by multiple risk dimensions such as compliance risk and historical threat score by evaluating and generating a composite security risk score from threat memory (114) Performing an adaptive assessment using historical threat data. multilayer adaptive risk scoring engine (103), 10  Risk information obtained, organization's safety policies and regulatory rules considering together corporate security policies, regulatory requirements, by considering user permissions, task type, and system configurations together. Security policy decision that determines the security strategy to be implemented for the request engine (104), 15  located at the center of the system and using all the information from previous components creating a customized security context and providing security based on that context. The execution plan (106) generates which security adapters for each user request. that they will be run, their order of operation, their dependencies, their working conditions, and Adaptive runtime security 20 that dynamically determines decision points planner (105),  generated by the adaptive runtime safety planner (105) for the relevant request defining the security architecture to be implemented, and the security adapters to be run, the order in which the adapters operate, the dependencies between the adapters, and under what conditions Security measures including replanning, risk thresholds, and exit decisions will be implemented. execution plan (106),  Created during the working time in accordance with the security execution plan (106), The security adapters included will only be used for the security of the relevant request. Dynamic safety adapter chain selected according to requirements (107),  processing the request passed through the dynamic security adapter chain (107), user 30 The large language model that produces the output that suits its needs (108),  content and work produced throughout the output generation process of the big language model (108) By continuously monitoring the situation and generating sensitive data in the model output whether it was produced, whether it contains content that violates security policies, and 35. See in real-time whether unexpected vehicle requests are being made. 17 If the assessor determines that the safety risk has changed, the working time will be adjusted accordingly. Production monitor (109) which triggers the safety replanning engine (110),  Triggered by the production monitor (109), the current security execution plan (106) Detection of involuntary injection during the procedure, which allows for re-evaluation. Events such as the discovery of sensitive data or unexpected model output 5 new security measures were implemented as a result of replanning when it occurred. adapters can be added, existing adapters can be removed, or the process can be stopped. Runtime safety rescheduling engine (110),  working within the dynamic security adapter chain (107) to protect personal data 10 before the call of the big language model (108) which deals beyond the masking level tokenizes the input text and identifies sensitive fields, token substitution, placeholder with one of the following methods: matching, hashing function, encryption or dynamic masking early personal data conversion at the token level that performs the conversion mechanism (113),  Security threats, attack patterns, and 15 previously identified by the system. semantic threat memory that stores and queries threat signatures semantically (114), It includes.

2. The system compliant with Request 1 is characterized by its raw data (20) received from the user or client application. It receives the request, standardizes the request format, cleans the input data, and sorts the characters. verifying its coding and allowing the request to be processed by other components of the system. It contains a request normalizer (101) which converts it into a standard structure.

3. The system compliant with Request 1 is characterized by its ability to analyze the normalized request, and the request's 25 a threat analysis engine that assesses the presence of potential security threats (102) is included.

4. The system is compliant with claim 1 and its feature is that it produces the output generated by the large language model (108). Evaluated and checked for compliance with security policies, personal data in the output 30 whether or not it exists, and whether or not content that violates security policies is being produced. the response that evaluates and performs the necessary verification and conversion processes. It includes the analyzer (111).

5. The system is compliant with claim 1, and its feature is that it has been verified by the response analyzer (111) and 35 The response is the output sent to the user after the necessary conversion processes have been completed. 18 The response received by the user complies with security policies and the protection of personal data. It contains a secure response (112) which guarantees that it will be safe.

6. The system compliant with Request 1, its features are; request normalizer (101), threat analysis engine (102), multi-layered adaptive risk scoring engine (103), security policy decision engine 5 (104), adaptive runtime safety scheduler (105), dynamic safety adapter chain (107), production tracker (109), runtime safety rescheduling engine (110), Response Analyzer (111), early personal data transformation mechanism at token level (113) and the process that performs the operations carried out by semantic threat memory (114) It contains unit (115). 10 7. The system is compliant with Request 1, and its feature is that it is connected to the processing unit (115) and the request A memory unit that holds data generated during processing, and which can be temporary or permanent. (116) is included.

8. Dynamic and adaptive security via the processing unit (115) for large language models It is a method that provides orchestration; its characteristic feature is:  Request normalizer (101) of the request from the user or client application normalization and conversion into a standard processing format by,  The normalized request is analyzed by the threat analysis engine (102) and 20 Identifying potential security threats,  Multilayer adaptive risk scoring engine (103); user profile, content of the request, data sensitivity, task type, model characteristics, and other runtime parameters by evaluating and creating a multi-layered security risk score, and this evaluation During this time, past threat data obtained from semantic threat memory (114) were also 25 inclusion in risk scoring,  The risk information obtained should be combined with organizational policies and regulatory rules. security policy decision engine (104) evaluation and security Security policy decision engine (104) that should be applied for the relevant request by determining its strategy, adaptive worktime safety planner (105) 30 transfer,  The Adaptive Runtime Security Planner (105) creates the security context using it to generate a custom security execution plan (106) and this execution the plan; the safety adapters to be implemented, the sequences of operations, the execution conditions and defining decision points and security execution plan (106) memory unit 35 (116) to be kept on 19  Dynamic security adapter in line with the security implementation plan (106) creation of the chain (107) at working time and the security to be included in this chain adapters should be selected only according to the security requirements of the relevant request.  Early personal data at the token level within a dynamic security adapter chain the transformation mechanism (113), personal data before the big language model call 5 It detects and converts at the token level and provides different security features depending on the task type. automatic activation of adapters,  Request passed through the dynamic security adapter chain (107) to the large language model (108) production by transmission and processing and output generation process of the large language model (108) Continuous monitoring of the produced content and working status of the viewer (109) 10 to accomplish,  If it is determined during monitoring that the security risk has changed, the study time security rescheduling engine (110) is activated current security to ensure the reassessment of the implementation plan (106) and the updated plan, Adaptive runtime safety scheduler via feedback loop (105) 15 By passing it to the component, the operation of the large language model (108) can continue safely. to be made to happen,  Finally, the response analyzer (111) analyzes the output produced by the large language model (108) by evaluating and checking its compliance with security policies and After the necessary verification and conversion processes are completed, the user will be provided with a secure 20 Transmission of the response (112) It includes the steps of the process.

Images