Method, device and electronic equipment for secure routing of large language model requests

Abstract

The application provides a large language model request security routing method and device and electronic equipment, and belongs to the technical field of computer software. In the method, synchronous interception and parallel detection are used before model routing, effectively solving the problem of high delay of the asynchronous detection mechanism in the prior art, realizing millisecond-level real-time sensitive data identification, and meeting the performance requirements of large language model real-time interaction. At the same time, according to the real-time availability of the security computing resource, the request original text is dynamically routed to the security computing resource or routed to the external computing resource after the sensitive data is desensitized, overcoming the defect that the single desensitization strategy unconditionally masks data, resulting in loss of semantic information, and maximizing the preservation of the original semantics under the premise of ensuring data security, thereby improving the reasoning quality of the model.

Description

Technical Field

[0001] This invention relates to the field of computer software technology, and in particular to a secure routing method, apparatus, and electronic device for large language model requests. Background Technology

[0002] With the widespread application of large language model technology, enterprises face a dilemma in balancing the risk of sensitive data leakage and the quality assurance of model inference when managing access to model services in a unified manner through proxy gateways.

[0003] In existing technologies, sensitive data protection for requests from large language models typically employs asynchronous queue detection or a single desensitization strategy. However, asynchronous detection mechanisms introduce significant response latency, failing to meet the millisecond-level response performance requirements of real-time interactive scenarios for large language models. On the other hand, a single desensitization strategy often unconditionally masks the detected sensitive data before sending it to the external model. This masking process prevents lossless inference, leading to the loss of semantic information and a decline in inference quality.

[0004] In summary, traditional secure routing schemes for large language model requests suffer from technical problems such as high latency in sensitive data detection and a lack of diverse protection strategies that compromise model inference quality. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a secure routing method, apparatus and electronic device for large language model requests, so as to alleviate the technical problems of high latency in sensitive data detection and single protection strategy leading to impaired model inference quality in traditional secure routing schemes for large language model requests.

[0006] In a first aspect, the present invention provides a secure routing method for large language model requests, comprising: Intercept large language model requests synchronously before model routing, and extract the message content of the request initiator role as the content to be detected; At least two pattern matching algorithms are used to perform synchronous detection on the content to be detected in parallel to obtain the sensitive data detection results. In response to the sensitive data detection result indicating that sensitive data has been hit, the real-time availability of secure computing resources is assessed; If the secure computing resource is available, the large language model request is routed to the secure computing resource; if the secure computing resource is unavailable, the sensitive data in the large language model request is de-identified and then routed to an external computing resource.

[0007] Furthermore, the request initiator role includes user input role or tool invocation role, and excludes model generation role; The at least two pattern matching algorithms include state machine-based regular expression matching and multi-modal automaton-based keyword matching.

[0008] Furthermore, the state machine-based regular expression matching includes: The content to be detected is pattern matched using a pre-compiled deterministic finite automaton, and the compilation result of the deterministic finite automaton is stored in a cache protected by a read-write lock. The keyword matching based on multi-modal automata includes: The Aho-Corasick automaton is used to search for keywords in the content to be detected. The Aho-Corasick automaton is built based on enabled keyword rules.

[0009] Furthermore, the secure computing resources include a locally deployed security model pool, and the real-time availability assessment is determined based on the activation status of the vendors and the validity of their credentials in the security model pool; The desensitization process is reversible, and the method further includes: After receiving the response data returned by the external computing resource, the response data is restored based on the reversible desensitization mapping relationship.

[0010] Furthermore, the security model pool is a security routing vendor group subject to uniqueness constraints, and the system allows at most one security routing vendor group to exist at any given time. Routing the large language model request to the secure computing resource includes: Randomly select a target model from the available models within the secure routing provider group, and forward the original text of the large language model request to the target model.

[0011] Furthermore, the method also includes: In response to a runtime exception occurring during the parallel execution of the synchronization detection, the runtime exception is captured, and the large language model request is downgraded and routed to the external computing resource to keep the request link unblocked.

[0012] Furthermore, the method also includes: In response to the failure to load the detection rule cache, the detection engine is started with an empty rule set to perform the synchronous detection and an asynchronous retry is triggered to load the detection rule cache.

[0013] Furthermore, the method also includes: In response to the sensitive data detection result indicating that sensitive data has been hit, the hit event is written to the audit log in an asynchronous batch manner. The audit log records the detection process, the handling actions, and the actual model information used.

[0014] Secondly, the present invention also provides a secure routing apparatus for large language model requests, comprising: The interception and extraction unit is used to synchronously intercept large language model requests before model routing and extract the message content of the request initiator role as the content to be detected. A synchronous detection unit is used to perform synchronous detection on the content to be detected in parallel using at least two pattern matching algorithms to obtain sensitive data detection results. An evaluation unit is used to evaluate the real-time availability of secure computing resources in response to the sensitive data detection result indicating that sensitive data has been detected. The routing unit is configured to route the large language model request to the secure computing resource if the secure computing resource is available, and to route it to an external computing resource after performing desensitization processing on the sensitive data hit in the large language model request if the secure computing resource is unavailable.

[0015] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method described in the first aspect.

[0016] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the method described in the first aspect.

[0017] This invention provides a secure routing method for large language model requests, comprising: synchronously intercepting the large language model request before model routing and extracting the message content of the request initiator as the content to be detected; employing at least two pattern matching algorithms to perform synchronous detection in parallel on the content to be detected to obtain sensitive data detection results; in response to the sensitive data detection results indicating that sensitive data has been hit, assessing the real-time availability of secure computing resources; if secure computing resources are available, routing the large language model request to secure computing resources; if secure computing resources are unavailable, performing de-identification processing on the sensitive data hit in the large language model request and then routing it to external computing resources. As can be seen from the above description, the secure routing method for large language model requests of this invention effectively solves the problem of high latency in asynchronous detection mechanisms in the prior art by employing synchronous interception and parallel detection before model routing, achieving millisecond-level real-time sensitive data identification and meeting the performance requirements of real-time interaction of large language models. Meanwhile, by dynamically selecting whether to route the original request to a secure computing resource or to anonymize sensitive data and then route it to an external computing resource based on the real-time availability of secure computing resources, the defect of a single anonymization strategy that unconditionally masks data and causes loss of semantic information is overcome. Under the premise of ensuring data security, the original semantics are preserved to the maximum extent, thereby improving the inference quality of the model. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a secure routing method for large language model requests provided in an embodiment of the present invention; Figure 2 A schematic diagram of a secure routing device for large language model requests provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] To facilitate understanding of this embodiment, a secure routing method for large language model requests disclosed in this embodiment of the invention will first be described in detail.

[0022] Example 1: like Figure 1 As shown, this embodiment of the invention provides a secure routing method for large language model requests. This method is applied to the large language model proxy gateway side and aims to solve the problems of high latency in sensitive data detection and the degradation of model inference quality due to a single protection strategy. The method mainly includes the following steps: Step S100: Intercept large language model requests synchronously before model routing, and extract the message content of the request initiator role as the content to be detected.

[0023] Specifically, this step is executed after the proxy gateway (located between the client (such as an enterprise instant messaging tool or internal AI assistant) completes basic authentication (such as virtual key authentication), concurrency limits, and quota checks), but before the target model is determined and the request is forwarded. This timing ensures that only legitimate large language model requests that meet quota limits enter the security detection chain, avoiding wasting detection resources on invalid large language model requests, and also ensuring that the detection results can be effectively utilized before the request leaves the gateway boundary. "Synchronous interception" here means that the detection process acts as a blocking link in the main request processing chain. The request (i.e., the large language model request) must wait for the detection to complete and obtain a clear result before it can continue to flow. This differs from the existing technology's mode of forwarding the request first and then asynchronously calling back for detection, thus eliminating the queuing delay and context switching overhead caused by asynchronous queues, providing a timing basis for subsequent millisecond-level routing decisions. Regarding the "request initiator role," it refers to the identity of the entity that generates the original input data in the dialogue sequence, such as the user end that directly inputs commands or the tool end that performs operations on behalf of the user, excluding assistant responses generated by the model itself. By precisely defining the extraction scope, the system can eliminate a large amount of redundant text that does not contain the user's original sensitive information, significantly reducing the computational load of the detection engine, while avoiding false alarms caused by the content generated by the detection model.

[0024] Step S200: At least two pattern matching algorithms are used to perform synchronous detection on the content to be detected in parallel to obtain the sensitive data detection results.

[0025] Specifically, to ensure a high detection rate while meeting the stringent low-latency requirements of real-time interaction of large language models, this embodiment abandons the traditional approach of serial processing with a single algorithm and instead adopts a multi-dimensional parallel detection architecture. The "at least two pattern matching algorithms" here encompass recognition logic for different data features, such as structured pattern matching algorithms for recognizing ID card numbers, mobile phone numbers, and other data with fixed format patterns, and keyword matching algorithms for recognizing discrete words such as specific project codes and internal terms. At the execution level, these two or more algorithms are started and run concurrently on the same content to be detected, and the final sensitive data detection result is aggregated and output after all algorithms have completed. This parallel mechanism ensures that the overall detection time depends on the slowest algorithm rather than the sum of the times of all algorithms, thus maintaining stable millisecond-level response performance even when the number of rules increases. It should be understood that although this embodiment uses two algorithms as an example, in other implementations, it can be expanded to a parallel combination of three or more algorithms according to business needs, as long as the logic of synchronous execution and result aggregation remains unchanged, all of which fall within the scope of protection of this invention.

[0026] In addition to employing at least two pattern matching algorithms to perform synchronous detection of the content to be detected in parallel, a context-adaptive dynamic orchestration of the detection strategy can also be used. This is achieved by introducing a detection strategy orchestration engine, specifically including: Request context feature extraction: When intercepting large language model requests, multi-dimensional context features of the large language model requests are extracted simultaneously, including: the historical sensitive behavior score of the request initiator role (obtained by weighting the following dimensions: sensitive data access frequency (e.g., the percentage of times accessing high-security data segments); the number of abnormal operations (e.g., access outside of working hours, unauthorized attempts); historical violation or alarm association records; the inherent sensitivity of the role (e.g., administrator, external user, etc., which can be obtained from the role's historical access logs and operation audit records in the system), the semantic complexity of the request (e.g., information entropy value), the cumulative number of sensitive data hits in the current session, and the target model type of the request (e.g., code generation model or dialogue model), etc. Dynamic decision-making for detection strategy: Based on the above contextual features, a lightweight decision model (such as gradient boosting tree or small neural network) dynamically determines which combination of detection algorithms to use (selecting 2 to N algorithms from the algorithm pool as needed), the execution order of each algorithm (serial / parallel / hybrid), and the matching strength threshold of each algorithm (strict / relaxed), and then performs sensitive data detection according to the determined detection strategy.

[0027] This detection strategy's context-adaptive dynamic orchestration process can reduce the number of algorithms to lower latency in low-risk scenarios and increase detection depth to improve recall in high-risk scenarios, achieving a dynamic Pareto optimal balance between detection accuracy and computational cost, thus balancing detection accuracy and response latency.

[0028] Step S300: In response to the sensitive data detection result indicating that sensitive data has been detected, assess the real-time availability of secure computing resources.

[0029] Specifically, when the detection result in step S200 indicates that the request contains sensitive information, the system does not mechanically execute a single action. Instead, it first triggers a dynamic perception of the status of secure computing resources. These secure computing resources typically refer to model service clusters with sensitive data processing capabilities deployed locally or in a private cloud environment. Real-time availability assessment is a multi-dimensional process, encompassing not only network-level connectivity checks but also indicators such as the health status of service instances, the validity of access credentials, current load levels, and remaining quotas. Only when all these indicators meet preset conditions is the resource deemed "available." This step constitutes the core branching point of the two-layer adaptive routing strategy, ensuring that routing decisions are based on the current actual resource status, rather than static configuration, thereby avoiding service interruptions caused by forwarding requests to actually faulty or overloaded local nodes.

[0030] For each match, a match result record is generated, which includes the match rule identifier (rule_id) and rule name (rule_name); the sensitive category (category), such as id_card, phone, bank_card, password, email, credential, network, pii, address, corporate, custom; the original content matched (matched_content); the match start position (match_position), i.e., the character offset; and the detection type (detect_type), i.e., regex or keyword.

[0031] In step S400, if the secure computing resource is available, the large language model request is routed to the secure computing resource; if the secure computing resource is unavailable, the sensitive data in the large language model request is de-identified and then routed to the external computing resource.

[0032] Specifically, based on the evaluation results of step S300, the system automatically selects the optimal processing path. When secure computing resources are available, the request is directly routed to that resource for processing. In this case, the original sensitive data does not need to be transformed or masked, and can retain complete semantic information for inference within the local security domain, thereby maximizing the quality and accuracy of the model's response. Conversely, when secure computing resources are unavailable, the system automatically downgrades to a defense mode. It first performs desensitization processing on the sensitive segments hit in the request, replacing them with meaningless placeholders or masks, and then sends the processed request to external computing resources (such as public cloud big model services). This two-layer adaptive mechanism effectively solves the problem of inference quality loss caused by the one-size-fits-all desensitization in existing technologies. That is, under the premise of ensuring the bottom line of data security, it utilizes local security resources as much as possible to provide high-quality services, and only sacrifices some semantic integrity to maintain service continuity when local resources are unavailable. The entire routing switch and desensitization process is completely transparent to the end user. The response does not carry any metadata tags about the execution of security policies, which maintains the user experience and prevents external attackers from reverse probing the enterprise's security protection rules through response characteristics.

[0033] Desensitization of sensitive data can be achieved through semantic density-based adaptive control of desensitization granularity. This involves introducing semantic density evaluation and granularity control, specifically including: Semantic density calculation: Semantic density quantification of the hit sensitive data - calculating the proportion of sensitive data in the request text, the semantic relevance between sensitive data (based on the dependency syntactic distance between named entities), and the contribution of sensitive data to the core semantics of the request (evaluated by calculating the semantic understanding bias of LLM after masking sensitive data). Adaptive desensitization granularity: Automatically selected from the three-level desensitization strategy based on semantic density. Mild desensitization: When the semantic density is low, only explicitly sensitive entities (such as ID number → [ID]) are replaced, while the semantic skeleton is preserved; Moderate desensitization: When the semantic density is moderate, generalized replacement is performed on sensitive entities and their associated context (e.g., "Zhang San visited Peking Union Medical College Hospital in March 2024" → "[Someone] visited [a certain institution] at [a certain time]"). Heavy desensitization: When the semantic density is extremely high, the desensitized route is rejected and the preset security rejection template is returned directly to avoid the risk of information leakage; The above approach achieves a fine balance between security and LLM response availability, avoiding situations where excessive desensitization prevents the LLM from understanding the request intent.

[0034] In the above content, the request initiator role includes the user input role or the tool invocation role, and excludes the model generation role; at least two pattern matching algorithms include state machine-based regular expression matching and multi-pattern automaton-based keyword matching.

[0035] Specifically, after synchronously intercepting requests from the large language model, the request interception middleware parses the message sequence in the request body, accurately extracting message text identified by the role of user input (user) or tool invocation (tool) as the content to be detected. The user input role corresponds to instructions or dialogue content sent directly from the end user to the large language model, and is the primary source of sensitive data. The tool invocation role corresponds to the result data returned by the large language model when calling external plugins or APIs to perform complex tasks; this type of data may also contain unverified sensitive information and is therefore included in the detection scope. At the same time, message content from the model-generated role (assistant) is explicitly excluded. This is because the content of the model-generated role is generated by the large language model's own reasoning, not the user's original input. Detecting it not only fails to prevent data leakage risks but also leads to numerous false positives because the model may repeat sensitive words in its responses, while unnecessarily increasing the computational load on the detection engine. Through this precise role filtering mechanism, the system can concentrate limited detection computing power on truly high-risk data inflow directions, significantly improving detection efficiency and accuracy.

[0036] Furthermore, state machine-based regular expression matching includes: using a pre-compiled deterministic finite automaton to perform pattern matching on the content to be detected, with the compilation result of the deterministic finite automaton stored in a cache protected by read-write locks; multi-pattern automaton-based keyword matching includes: using an Aho-Corasick automaton to search for keywords in the content to be detected, with the Aho-Corasick automaton built based on enabled keyword rules. Each regular expression rule and keyword rule includes a name, detection type, a list of matching patterns / keywords, a sensitivity category, a description, and an enabled status. Regular expression rules support online testing; administrators can enter test text and view the matching results in real time.

[0037] Specifically, the sensitive data detection engine integrates two core components: a regular expression matcher and a keyword matcher. Both are specifically optimized for structured data and unstructured discrete vocabulary. For the regular expression matcher, instead of using traditional regular expression interpretation, it utilizes a deterministic finite automaton (DFA) as its runtime matching engine. DFAs have deterministic state transition paths, enabling character matching in constant time and completely avoiding performance fluctuations caused by backtracking. Crucially, these DFAs are not compiled on demand with each request, but are pre-compiled when the rule management and caching module loads or updates rules, and the compiled state machine objects are stored in an in-memory cache. To handle the hot rule updates required in high-concurrency scenarios, the cache employs a read-write lock mechanism: when a large number of requests concurrently read the cache for detection, only a read lock is acquired, or even lock-free reading, without blocking; only when an administrator changes rules triggering a cache refresh is a write lock briefly acquired to atomically replace the new DFA object. This design ensures that the detection chain maintains millisecond-level responsiveness during dynamic rule adjustments, preventing request backlogs due to lock contention.

[0038] For the keyword matcher, the Aho-Corasick (AC) automaton algorithm is used to achieve efficient multi-pattern string matching. The AC automaton constructs a Trie tree of all enabled keyword rules during the preprocessing stage and establishes failure links on this tree, enabling simultaneous keyword searches during a single linear scan of the input text. Its time complexity is only O(n+m), where n is the length of the input text and m is the number of matched results, completely decoupled from the total number of keyword rules. This means that even with hundreds or thousands of keyword rules configured in the system, the matching time for the same content to be detected will hardly increase with the number of rules, thus providing a solid algorithmic foundation for real-time detection under large-scale rule sets. Keywords support batch import (CSV / TXT / JSON format) and group management by sensitivity category; the system has built-in keyword templates for passwords / passwords, keys / credentials, tokens, database credentials, identity information, bank information, mobile phone information, address information, etc. It should be understood that although this embodiment uses the Aho-Ciel automaton as an example for illustration, other multi-pattern matching algorithms such as the Commentz-Walter algorithm and the Wu-Manber algorithm can also be used in other embodiments to achieve the same function. As long as the effect of matching multiple patterns in a single traversal can be achieved, they all fall within the protection scope of this invention.

[0039] At the execution level, the regular expression matcher and keyword matcher are launched and executed synchronously in parallel for the same content to be detected. The system schedules the two matching tasks simultaneously through an asynchronous concurrency framework (such as coroutines or thread pools) and aggregates their respective hit result lists after both are completed. This parallel architecture makes the overall detection time depend on the slower of the two algorithms, rather than the sum of their times, thus maximizing the utilization of the multi-core parallel capabilities of modern processors. In actual testing, when the system simultaneously enables 100 regular expression rules and 500 keyword rules and performs detection on an input text of 2000 characters in length, the latency of a single complete detection process is stably controlled within 10 milliseconds. This performance metric fully verifies the effectiveness of the DFA pre-compiled cache, AC automaton, and parallel execution architecture adopted in this embodiment, proving that the technical solution can fully meet the stringent requirements of millisecond-level real-time synchronous detection in the scenario of a large language model proxy gateway, providing timely and reliable data support for subsequent two-layer adaptive routing decisions.

[0040] In the above content, secure computing resources include a locally deployed security model pool, and the real-time availability assessment is determined based on the activation status of the vendors and the validity of their credentials in the security model pool; the de-identification process is reversible de-identification, and the method further includes: after receiving response data returned by external computing resources, restoring the response data based on the mapping relationship of reversible de-identification.

[0041] Specifically, when the sensitive data detection result indicates that sensitive data has been detected, the system enters a two-layer adaptive routing decision process. This process is not a simple either-or switch, but a decision chain containing multiple levels of judgment. First, the system checks whether the current instance has enabled secure routing. If not enabled, it directly executes the strategy of routing to external computing resources after de-identification processing. This is suitable for tenant instances that have not yet deployed local security models or do not require localization processing for the time being. If enabled, the system further evaluates the real-time availability of secure computing resources. Here, secure computing resources specifically refer to the locally deployed security model pool. Its availability determination depends not only on network connectivity, but more importantly, on a comprehensive confirmation based on the provider's activation status and credential validity. For example, even if a local model service process is running, if its corresponding API key has expired, the base URL is configured incorrectly, or the administrator has manually disabled the provider, the system will still determine that the resource is unavailable. This multi-dimensional availability assessment mechanism effectively avoids invalid routing attempts caused by configuration drift or service freezes, ensuring that every routing decision is based on truly available service capabilities.

[0042] Furthermore, the security model pool is subject to uniqueness constraints on the security routing vendor group, and the system allows at most one security routing vendor group to exist at any given time; routing large language model requests to secure computing resources includes: randomly selecting a target model from the available models within the security routing vendor group, and forwarding the original text of the large language model request to the target model.

[0043] Specifically, this invention introduces a strict uniqueness constraint in its architectural design: only one secure routing provider group, marked as a secure routing group, can exist in the entire system's database at any given time. This design is not merely a management restriction, but rather stems from a deep consideration of decision-making efficiency and system stability. In traditional multi-active or master-slave routing architectures, the system often needs to prioritize, calculate weights, or compare health among multiple candidate groups. This not only increases the processing latency of each request but also easily leads to routing oscillations due to configuration conflicts. By forcibly limiting the secure routing provider group to a single group, the system simplifies the complex multi-objective selection problem into a single-group availability verification problem, enabling the availability judgment logic to be completed in constant time, greatly improving the decision throughput in high-concurrency scenarios. When it is determined that a usable model exists within a secure routing provider group, the system randomly selects one from the available models within that group as the target model. This random selection strategy achieves automatic load balancing of the local secure model pool, preventing service overload or slow response caused by excessive concentration of requests on a specific model instance, while also improving the overall utilization of local resources. The selected target model will receive the original text requested by the large language model without any anonymization processing, thus preserving complete semantic information within the local security domain and maximizing the inference quality of the model.

[0044] For the de-identification processing branch, this embodiment employs a reversible de-identification mechanism to construct a complete data security closed loop. When the decision result is a de-identified route (ActionMask), the system first calls the de-identification module to replace the sensitive segments hit in the request, generating the de-identified request text and a mapping table recording the correspondence between the original content and the replaced placeholders. Subsequently, the de-identified request is forwarded to external computing resources for processing. Since the external model receives disguised harmless data, there is no risk of sensitive data leakage. After receiving the response data returned by the external computing resource, the system uses the previously generated mapping table to reverse-engineer any placeholders or de-identification traces that may be contained in the response content, restoring the complete response containing the original sensitive information, and then returning it to the client. It should be understood that although this embodiment uses a locally deployed security model pool as an example to illustrate secure computing resources, in other implementations, secure computing resources can also be dedicated model clusters in a private cloud environment, lightweight model services on edge computing nodes, or even enterprise-certified third-party trusted model services, as long as they meet data security compliance requirements and can accept original requests. Similarly, although this embodiment describes a reversible desensitization and restoration method based on a mapping table, in practical applications, other reversible desensitization techniques such as token-based replacement restoration and context-aware semantic backfilling can also be used. As long as the effect of external masking and internal restoration can be achieved, they all fall within the protection scope of this invention. By combining the above-mentioned two-layer adaptive decision-making with the reversible desensitization mechanism, this embodiment not only solves the problem of inference quality loss caused by a single desensitization strategy, but also ensures the continuity of security protection capabilities when local resources are lacking. At the same time, the entire process is completely transparent to the end user, and the response does not carry any metadata tags about the execution of security policies, thus balancing security, usability, and user experience.

[0045] In some embodiments, in response to a runtime exception occurring during the parallel execution of synchronization detection, the runtime exception is captured, and the large language model request is downgraded and routed to an external computing resource to keep the request link unblocked.

[0046] Specifically, this embodiment addresses various fault scenarios that may occur in the detection engine during runtime, which is fundamentally different from configuration loading failures during system startup. Runtime anomalies typically include, but are not limited to: thread panic caused by complex backtracking in the regular expression matcher, memory access exceeding limits during keyword matching, deadlock caused by concurrent contention, or timeout interruption triggered when a single detection takes longer than a preset threshold. When any of the above anomalies occur, the system does not directly throw an error code or interrupt the connection like a traditional security gateway. Instead, it immediately takes over control through a pre-built exception handling mechanism (such as a Try-Catch block or a Recover function). This mechanism executes a strict atomic sequence of operations: First, the exception stack, trigger rule identifier, and hash value of the input content are recorded in the error log for subsequent maintenance and troubleshooting, but the original sensitive text is never recorded to ensure compliance; second, the current sensitive data detection result is forcibly reset to a "miss" state in memory or the detection process is skipped directly; finally, the control flow seamlessly falls back to the main routing logic, forwarding the large language model request to external computing resources for processing according to the default path. This design embodies the high-availability architecture principle that security components should not become single points of failure for business operations. While security is crucial in real-world production environments, its priority should not supersede core business continuity. If a flawed new regular expression rule or a temporary resource fluctuation causes all LLM requests to be blocked or fail, it will have a disastrous impact on user experience. By downgrading abnormal requests to external computing resources, the system effectively strikes a rational engineering balance between potential data risks and guaranteed service interruptions—prioritizing continued user access to model services in the event of a security component failure, while relying on the external model's own basic security capabilities as a last line of defense. It should be understood that although this embodiment uses downgrading to external computing resources as an example, in other implementations, if the system is configured with multiple mutually backup external service nodes, the downgrading target can also be a pre-defined backup external node; or in scenarios with strict restrictions on data export, the downgrading strategy can be configured to return a friendly prompt message instead of forwarding the request. As long as this strategy can prevent the request chain from entering an infinite waiting or crashing state when anomalies are detected, it falls within the scope of this invention. Furthermore, this fault tolerance mechanism is decoupled from the two-layer adaptive routing decision in the aforementioned embodiments. Regardless of whether the current instance has enabled the secure routing function or whether the local security model pool is available, as long as the detection engine itself experiences a runtime exception, the degradation process described in this embodiment will be triggered unconditionally, thereby ensuring the independence and reliability of the fault tolerance logic.

[0047] In some embodiments, in response to a failure to load the detection rule cache, the detection engine is started with an empty rule set to perform synchronous detection and an asynchronous retry is triggered to load the detection rule cache.

[0048] Specifically, this embodiment addresses configuration retrieval anomalies that may occur during system startup initialization or runtime rule hot updates. This is fundamentally different from the runtime anomalies that occur when the detection engine executes matching logic, as described in the previous embodiments. Common causes of detection rule cache loading failure include network unavailability of the configuration center, database connection timeouts, rule file format parsing errors, or storage medium read failures. In traditional security gateway designs, such critical configuration loading failures are often considered fatal errors, and the system typically throws an exception and terminates the startup process, or refuses to process subsequent requests until the configuration is repaired. However, in high-availability scenarios for large language model proxy gateways, the security detection component, as middleware in the business chain, should have its configuration dependencies designed as weak dependencies rather than strong blocking dependencies. Therefore, this invention adopts an empty rule set fallback fault-tolerance strategy. When a rule cache loading failure is detected, the system does not interrupt the service startup process or suspend the request processing thread. Instead, it immediately constructs a valid, empty sensitive data detection engine instance in memory, but without any matching rules. This empty rule set engine can respond normally to detection calls, but returns a miss result for all input content to be detected. This design ensures that even in extreme cases where security policies are temporarily absent, the main forwarding link for large language model requests can remain unobstructed, avoiding the risk of a complete service outage due to configuration issues with security components.

[0049] Meanwhile, to restore full security capabilities as quickly as possible, the system, while enabling an empty rule set as a fallback, triggers an independent asynchronous retry task in the background to reload the detection rule cache. This asynchronous retry mechanism is completely decoupled from the main business request processing chain, and will not consume computing resources for request forwarding or increase response latency. In specific implementation, asynchronous retries typically use an exponential backoff algorithm to control the retry frequency, for example, the initial retry interval is 1 second, then it increases sequentially to 2 seconds, 4 seconds, 8 seconds, etc., until the preset maximum retry interval or retry limit is reached. This backoff strategy can effectively avoid retry storms caused by brief failures of the configuration source, reducing the pressure on downstream configuration services or databases. Once the asynchronous retry task successfully obtains the latest rule data and completes DFA pre-compilation and AC automaton construction, the system will replace the empty rule set engine in memory with a new engine instance containing complete rules through atomic operations. Since the replacement process uses concurrency safety mechanisms such as read-write locks or RCU, requests that are being processed will not be affected, and newly arriving requests will be automatically detected using the restored rules. It should be understood that although this embodiment uses the exponential backoff algorithm as an example to illustrate the retry frequency control strategy, in other embodiments, fixed-interval retries, linear backoff, or intelligent retry strategies triggered by external health check signals can also be used. As long as non-blocking configuration self-healing can be achieved, they all fall within the protection scope of this invention. Through the above-mentioned startup fault tolerance mechanism, this invention downgrades the security detection component's dependence on the configuration source from a startup precondition to a runtime enhancement capability, significantly improving the system's robustness and service recovery speed in complex operation and maintenance environments. This ensures that the lack of security protection capabilities is only temporary and controllable, and will never come at the expense of the continuity of core business.

[0050] In some embodiments, in response to a sensitive data detection result indicating that sensitive data has been hit, the hit event is written to the audit log in an asynchronous batch manner. The audit log records the detection process, the handling actions, and the actual model information used.

[0051] Specifically, although the audit log writing step has been mentioned in the two-layer adaptive routing decision process of the aforementioned embodiment, this embodiment further defines the specific execution mechanism of the writing operation to ensure that the compliance recording function will not cause any performance loss to the main processing link of large language model requests. In the traditional synchronous audit mode, each sensitive data hit requires waiting for the database I / O operation to be completed before the subsequent routing or forwarding process can continue, which can easily become a bottleneck for system throughput during peak periods. To solve this problem, the present invention adopts a fully decoupled asynchronous batch writing architecture. When the sensitive data detection engine outputs the hit result, the system does not immediately initiate a database write request, but instead quickly serializes the audit event object containing the hit details and pushes it into a circular buffer queue in memory. Since the time complexity of memory operations is O(1) and the time consumption is usually in the nanosecond range, this action is almost instantaneous for the request processing main thread, thereby completely eliminating the impact of audit records on request forwarding delay.

[0052] Regarding the triggering conditions for batch writes, this embodiment employs a dual triggering strategy combining a time window and a threshold number of records to balance data real-time performance and I / O efficiency. Specifically, a separate persistent coroutine or thread continuously monitors the status of the buffer queue in the background. When the number of audit events accumulated in the queue reaches a preset threshold number (e.g., 100 records), or when the time interval since the last write operation exceeds a preset time window (e.g., 1 second), the system triggers a batch write operation. This design avoids the database connection overhead caused by frequently initiating small transactions in high-concurrency scenarios, while also preventing the risk of audit data being lost due to prolonged stagnation in memory in low-traffic scenarios. During batch writes, the system packages multiple audit events in the buffer into a transaction or batch insert statement and submits it to persistent storage. Even if a write operation fails due to network jitter or database overload, the system will only record an error log in the background and retry a limited number of times, never blocking or rolling back the request processing flow on the main business chain, thus ensuring that the introduction of security auditing functionality does not reduce the availability of core services.

[0053] The audit logs (which record the hit time, hit rules, sensitive categories, matched content (after anonymization), instance identifier, username, detection stage, handling action, detection time, matching start position, actual model name used, and originally planned external model name, etc.) are meticulously designed to meet the multi-dimensional needs of enterprise-level compliance auditing and security operations. Specifically, the "detection stage" field identifies whether sensitive data originates from a user input role or a tool invocation role, helping to distinguish whether the source of data leakage risk is active input from end-users or passive return from third-party plugins. The "handling action" field clearly records the specific security measures taken by the system for the request, such as lossless routing to secure computing resources or routing to external computing resources after anonymization, providing direct evidence for evaluating the effectiveness of security policy implementation. The "actual model information" field not only records the target model name that ultimately processes the request but also optionally records the originally planned external model name. By comparing the two, it is possible to statistically analyze how many requests are diverted to the local secure model due to sensitive data issues, thereby quantifying the utilization rate and return on investment of local security resources. It should be understood that although this embodiment lists the above-mentioned core fields, in other embodiments, the audit log can also be extended to record auxiliary information such as hit rule identifiers, sensitive categories, matching content summaries, detection time, usernames, and instance identifiers according to specific compliance requirements. As long as the collection and writing process of this information maintains the asynchronous and non-blocking characteristics, it falls within the protection scope of this invention. Through the above-mentioned asynchronous batch auditing mechanism, this invention, while ensuring the millisecond-level response performance of the large language model proxy gateway, constructs a complete, reliable, and traceable security and compliance data foundation.

[0054] For keyword matching hit events, the system additionally records fine-grained hit details at the keyword level, including the specific keyword content hit, the rule it belongs to, and the detection stage, for subsequent keyword popularity analysis and terminology optimization. In addition, a statistical dashboard is provided: the system offers multi-dimensional statistical capabilities: overview statistics (today / weekly hit count, local routing model hit count, anonymization handling count, number of enabled rules, number of secure routing instances); hit trend analysis (displaying a line graph showing the trend of local routing models and anonymization handling by date); sensitive category distribution (displaying the hit percentage by category); rule hit ranking (displaying the Top 10 rules by hit count); instance hit ranking (displaying the Top 10 instances by hit count); keyword monitoring (keyword hit cloud, hit details, high-frequency keyword trends).

[0055] To more clearly illustrate the operational logic and technical effects of the present invention's technical solution in practical applications, the following section, in conjunction with the foregoing embodiments and accompanying drawings, comprehensively describes the secure routing method for large language model requests provided by the present invention through six typical application scenarios. These scenarios cover the entire lifecycle from normal business flow to exception fault tolerance handling, aiming to verify how the present invention can ensure data security while balancing model inference quality and service availability.

[0056] In the first application scenario, the main focus is on verifying the normal routing process and its performance baseline when sensitive data is not detected. Assume an end user sends a request to the proxy gateway containing the message "Help me write a poem about spring." After authentication and quota checks, the request is synchronously intercepted by the request interception middleware. The sensitive data detection engine then starts, with the regular expression matcher and keyword matcher performing synchronous detection in parallel on the content to be detected. Since the text does not contain any preset sensitive patterns or keywords, the detection result list is empty, and the entire detection process takes only 1 millisecond. Upon receiving the empty result, the security routing decision module determines that no sensitive data has been detected and directly makes a "normal routing" decision. The decision execution module then forwards the original request to external computing resources along the original path. In this scenario, the system completes the security check with extremely low additional latency, ensuring that non-sensitive requests can pass through the gateway unnoticed, verifying the performance advantages of the parallel detection architecture under normal business conditions.

[0057] In the second application scenario, the lossless routing process is primarily verified when sensitive data is hit and secure computing resources are available. Assume the enterprise has enabled secure routing for the current instance, and a usable target model exists in the locally deployed security model pool. When a user sends a request containing "Please help me query user information for ID number 110105199003071234", the sensitive data detection engine completes parallel detection within 3 milliseconds. The regular expression matcher successfully identifies the ID number format, and the keyword matcher hits the phrase "ID number," outputting a detection result containing two matching records. The secure routing decision module first confirms that secure routing is enabled on the instance, then evaluates the status reported by the security vendor group management module, confirming that a local model with an enabled status and valid credentials exists within the group. Based on this, the decision module makes a "lossless routing" decision, randomly selecting a target model from the available models. The decision execution module directly forwards the original request containing the raw sensitive data to the local security model for processing. In this scenario, the sensitive data never leaves the enterprise security domain and is not anonymized or transformed in any way, enabling the local model to reason based on the complete semantic context, maximizing the accuracy and usability of the response. At the same time, the audit log asynchronously records the handling actions as routing the local model and the actual target model used.

[0058] In the third application scenario, the de-identification and degradation process is primarily verified when sensitive data is detected but secure computing resources are unavailable. Continuing from the previous scenario, it is assumed that all local models within the security vendor group are temporarily disabled for maintenance and upgrades. When a user sends a request to organize customer information containing multiple phone numbers, the detection engine quickly detects the sensitive data. However, during the decision-making phase, the security routing decision module assesses that no models are available within the security vendor group and automatically triggers a degradation strategy, making a de-identification routing decision. The decision execution module calls the de-identification component to perform reversible de-identification on the detected phone numbers, generates a mapping table, and forwards the replaced request to external computing resources. After the external model returns a response, the system restores the original phone numbers to the response content based on the mapping table. In this scenario, despite the lack of local security resources, the system still ensures business continuity through dynamic degradation, preventing the plaintext leakage of sensitive data and ensuring the semantic integrity of the response content received by the end user through the reversible restoration mechanism, achieving a balance between security protection and user experience.

[0059] In the fourth application scenario, the control effect of instance-level configuration switches on routing policies is primarily verified. Assume a tenant instance has not purchased or enabled local secure routing services, i.e., the configuration item `secure_routing_enabled` is false. When a user under this instance sends a request containing sensitive information and it is detected by the detection engine, the secure routing decision module identifies the disabled secure routing function at an early node in the decision chain, directly skipping the assessment step of local resource availability and making a de-identified routing decision. This design avoids invalid resource status queries on instances without local resources, shortens the decision path, and also meets the needs of differentiated security configurations for different tenants. Regardless of whether a local physical model exists, as long as the logical switch is turned off, the system strictly enforces the de-identified outgoing policy, ensuring the effectiveness of security policy isolation in a multi-tenant environment.

[0060] In the fifth application scenario, the fault tolerance and degradation mechanism of the detection engine during runtime anomalies is primarily verified. Suppose a newly deployed regular expression rule triggers a backtracking panic or timeout exception in the underlying engine when processing a specific combination of special characters. When a user happens to send a request that triggers this exception, the anomaly capture mechanism within the sensitive data detection engine immediately takes over control, records an error log containing the rule identifier and input hash, and forcibly sets the detection result to "miss". The main business link is not blocked in any way, and the request is automatically degraded and routed to external computing resources. In this scenario, the fault of the security component itself is strictly isolated and does not propagate to the core forwarding link, reflecting the high availability design principle of security without compromising availability. Operations personnel can subsequently discover the anomaly alarm through the audit dashboard and promptly correct or remove the problematic rule, while normal user access remains completely unaffected during this period.

[0061] In the sixth application scenario, the immediate effectiveness of hot rule updates is verified. Assume a security operations personnel adds a rule containing project codes such as "Atlas" and "Phoenix" in the rule management interface and clicks save. The rule management and caching module immediately detects the change, rebuilds the Aho-Corasick automaton in the background, and atomically replaces the old cached object in memory. After the save operation is complete, the next incoming request containing "compare Atlas and Phoenix projects" will be accurately matched by the new rule. The entire process does not require restarting the gateway service or interrupting ongoing requests. In this scenario, the read-write lock-protected caching mechanism ensures concurrency security and zero downtime during rule updates, enabling enterprises to quickly respond to sudden intelligence leakage risks or compliance requirement changes, significantly improving the agility and timeliness of security operations.

[0062] It should be understood that the above six application scenarios are merely illustrative examples for explaining the technical solutions of this invention and are not intended to limit the scope of protection of this invention. In actual deployment, these scenarios may occur in combination, such as triggering fault tolerance degradation while hot rule updates are being performed, or normal non-sensitive request forwarding occurring on instances where secure routing is not enabled. Regardless of the specific business model, as long as the core technical means fall within the scope of synchronous interception, parallel detection, two-layer adaptive routing, and fault tolerance auditing described in this invention, they should be considered equivalent implementations of this invention. Through comprehensive verification in these scenarios, it can be confirmed that this invention can effectively solve the technical problems of high detection latency, single protection strategy, and complex operation and maintenance in the prior art, achieving the optimal balance of security, availability, and inference quality in the large language model proxy gateway scenario.

[0063] The method of the present invention has the following effects: Millisecond-level real-time synchronous detection: The detection engine of this invention adopts a method of pre-compiling regular expressions into DFA and caching them, as well as the Aho-Corasick multi-pattern matching algorithm. Under the condition that 100 regular expression rules + 500 keyword rules are enabled at the same time, the latency to complete the full matching of 2000 character inputs does not exceed 10 milliseconds. The detection process is executed synchronously without introducing asynchronous queues, so that subsequent routing decisions can be made based on reliable results within milliseconds, which is an order of magnitude improvement compared to existing DLP solutions.

[0064] A dual-layer adaptive security protection strategy: The security model-priority routing + de-identification and degradation strategy proposed in this invention can automatically select the optimal protection method based on the real-time availability of the local security model. When the local model is available, sensitive data can be securely processed locally without de-identification, achieving the best inference quality. When the local model is unavailable, it automatically degrades to de-identification mode to ensure that sensitive data is not sent to the external model. This strategy ensures both the reliability of security protection and maximizes the availability and inference quality of the LLM service.

[0065] Precise link integration location: This invention precisely inserts the sensitive data detection middleware into the authentication → rate limiting → quota → detection → model routing → forwarding position, so that requests that fail authentication / rate limiting / quota do not enter the detection link (saving detection resources). At the same time, the detection results can be used by the routing decision module before forwarding, avoiding the problem of data being sent out and irretrievably lost due to forwarding first and then detecting.

[0066] Unified governance and observability: This invention supports full lifecycle management of rules (CRUD, batch import, pre-built templates, online testing, and instant cache activation) through a rule management subsystem, implements a simple management model of "is_secure_group + uniqueness constraint" through a security vendor group management subsystem, and provides complete observability through multi-dimensional auditing and statistical dashboards, significantly reducing the cost of enterprise deployment, operation and maintenance and compliance auditing.

[0067] End-to-end user transparency: Routing switching, de-identification, audit logging, and other processes are completely invisible to end users, and no security routing-related tags or metadata are added to the response; external attackers also cannot probe the enterprise's DLP policy through response characteristics.

[0068] Zero-blocking anomaly detection: When the detection engine encounters an anomaly, it automatically degrades to a normal route (i.e., forwards the request to external computing resources (i.e., external models) according to the original route) + error logging; if the rule cache fails to load, it starts with an empty rule set; neither of these blocks LLM requests, ensuring that a failure in the detection system does not affect business continuity.

[0069] Example 2: like Figure 2 As shown, this embodiment provides a secure routing device for large language model requests. The device includes: an interception and extraction unit 10, a synchronization detection unit 20, an evaluation unit 30, and a routing unit 40, wherein: The interception and extraction unit 10 is used to synchronously intercept large language model requests before model routing and extract the message content of the request initiator role as the content to be detected. The synchronous detection unit 20 is used to perform synchronous detection on the content to be detected in parallel using at least two pattern matching algorithms to obtain the sensitive data detection result. Evaluation unit 30 is used to evaluate the real-time availability of secure computing resources in response to the sensitive data detection result indicating that sensitive data has been detected. The routing unit 40 is used to route the large language model request to the secure computing resource if the secure computing resource is available; if the secure computing resource is unavailable, it performs desensitization processing on the sensitive data hit in the large language model request and then routes it to the external computing resource.

[0070] Optionally, the request initiator role includes a user input role or a tool invocation role, excluding the model generation role; at least two pattern matching algorithms include state machine-based regular expression matching and multi-pattern automaton-based keyword matching.

[0071] Optionally, state machine-based regular expression matching includes: performing pattern matching on the content to be detected using a pre-compiled deterministic finite automaton, with the compilation result of the deterministic finite automaton stored in a cache protected by read-write locks; multi-pattern automaton-based keyword matching includes: performing keyword search on the content to be detected using an Aho-Corasick automaton, with the Aho-Corasick automaton constructed based on enabled keyword rules.

[0072] Optionally, the secure computing resources include a locally deployed security model pool, and the real-time availability assessment is based on the enable status of the vendors and the validity of their credentials in the security model pool; the routing unit is also used to: restore the response data based on the reversible de-identification mapping relationship after receiving the response data returned by the external computing resources.

[0073] Optionally, the security model pool is a security routing vendor group subject to uniqueness constraints, and the system allows at most one security routing vendor group to exist at any given time; the routing unit is also used to: randomly select a target model from the available models within the security routing vendor group, and forward the original text of the large language model request to the target model.

[0074] Optionally, the device is also configured to: in response to a runtime exception occurring during the parallel execution of synchronization detection, capture the runtime exception and degrade the large language model request to an external computing resource to keep the request link unblocked.

[0075] Optionally, the device is also used to: in response to a failure to load the detection rule cache, start the detection engine with an empty rule set to perform synchronous detection, and trigger an asynchronous retry to load the detection rule cache.

[0076] Optionally, the device is also used to: in response to a sensitive data detection result indicating that sensitive data has been hit, write the hit event to the audit log in an asynchronous batch manner, and record the detection process, handling actions, and actual model information used in the audit log.

[0077] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0078] like Figure 3 As shown, an electronic device 600 provided in this embodiment of the invention includes: a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 via the bus. The processor 601 executes the machine-readable instructions to perform the steps of the secure routing method requested by the large language model described above.

[0079] Specifically, the memory 602 and processor 601 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 601 runs the computer program stored in the memory 602, it can execute the secure routing method for the large language model request mentioned above.

[0080] The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 601 or by instructions in software form. The processor 601 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 602, and processor 601 reads the information from memory 602 and, in conjunction with its hardware, completes the steps of the above method.

[0081] Corresponding to the secure routing method for large language model requests described above, this application also provides a computer-readable storage medium storing machine-executable instructions. When these machine-executable instructions are invoked and executed by a processor, they cause the processor to perform the steps of the secure routing method for large language model requests described above.

[0082] The secure routing device for large language model requests provided in this application embodiment can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this application embodiment are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0083] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0084] For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0085] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0086] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0087] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the secure routing method for large language model requests described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0088] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0089] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A secure routing method for large language model requests, characterized in that, include: Intercept large language model requests synchronously before model routing, and extract the message content of the request initiator role as the content to be detected; At least two pattern matching algorithms are used to perform synchronous detection on the content to be detected in parallel to obtain the sensitive data detection results. In response to the sensitive data detection result indicating that sensitive data has been hit, the real-time availability of secure computing resources is assessed; If the secure computing resource is available, the large language model request will be routed to the secure computing resource; If the secure computing resources are unavailable, the sensitive data in the large language model request is de-identified and then routed to external computing resources.

2. The method according to claim 1, characterized in that, The request initiator role includes user input role or tool invocation role, and excludes model generation role; The at least two pattern matching algorithms include state machine-based regular expression matching and multi-modal automaton-based keyword matching.

3. The method according to claim 2, characterized in that, The state machine-based regular expression matching includes: The content to be detected is pattern matched using a pre-compiled deterministic finite automaton, and the compilation result of the deterministic finite automaton is stored in a cache protected by a read-write lock. The keyword matching based on multi-modal automata includes: The Aho-Corasick automaton is used to search for keywords in the content to be detected. The Aho-Corasick automaton is built based on enabled keyword rules.

4. The method according to claim 1, characterized in that, The secure computing resources include a locally deployed security model pool, and the real-time availability assessment is determined based on the activation status of the vendors and the validity of their credentials in the security model pool. The desensitization process is reversible, and the method further includes: After receiving the response data returned by the external computing resource, the response data is restored based on the reversible desensitization mapping relationship.

5. The method according to claim 4, characterized in that, The security model pool is a security routing vendor group subject to uniqueness constraints, and the system allows at most one security routing vendor group to exist at any given time. Routing the large language model request to the secure computing resource includes: Randomly select a target model from the available models within the secure routing provider group, and forward the original text of the large language model request to the target model.

6. The method according to claim 1, characterized in that, The method further includes: In response to a runtime exception occurring during the parallel execution of the synchronization detection, the runtime exception is captured, and the large language model request is downgraded and routed to the external computing resource to keep the request link unblocked.

7. The method according to claim 6, characterized in that, The method further includes: In response to the failure to load the detection rule cache, the detection engine is started with an empty rule set to perform the synchronous detection and an asynchronous retry is triggered to load the detection rule cache.

8. The method according to claim 1, characterized in that, The method further includes: In response to the sensitive data detection result indicating that sensitive data has been hit, the hit event is written to the audit log in an asynchronous batch manner. The audit log records the detection process, the handling actions, and the actual model information used.

9. A secure routing device for large language model requests, characterized in that, include: The interception and extraction unit is used to synchronously intercept large language model requests before model routing and extract the message content of the request initiator role as the content to be detected. A synchronous detection unit is used to perform synchronous detection on the content to be detected in parallel using at least two pattern matching algorithms to obtain sensitive data detection results. An evaluation unit is used to evaluate the real-time availability of secure computing resources in response to the sensitive data detection result indicating that sensitive data has been detected. A routing unit is configured to route the large language model request to the secure computing resource if the secure computing resource is available. If the secure computing resources are unavailable, the sensitive data in the large language model request is de-identified and then routed to external computing resources.

10. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 8.

Images