Method and system for realizing large model access authentication and flow control based on APISIX gateway

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a method and system for achieving large model access authentication and flow control based on an APISIX gateway, and the method comprises the following steps: a client generates an HTTPS request containing an APIKey, and sends the HTTPS request to an APISIX gateway cluster; the gateway extracts the APIKey and forwards the APIKey to an authentication service, and the validity of the key is verified; after the verification is passed, the authentication service returns a user permission matrix, and the gateway verifies whether the request is within a permission range; after the permission verification is passed, the flow control plug-in detects a flow index according to a preset strategy; when the flow index does not exceed the limit, the gateway forwards the request to the large model service and returns a result; when the flow index exceeds the limit, the gateway executes flow limiting / fusing processing and returns a corresponding error response; the method has the beneficial effects that the gateway executes request forwarding or flow limiting control based on the preset strategy, the problems of illegal access risk and resource allocation imbalance faced by the large model service are effectively solved, the service stability and the resource utilization efficiency are improved, and the method is suitable for security operation and maintenance of the large model service in multiple scenes.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence, in particular to a method and system for realizing large model access authentication and traffic control based on an APISIX gateway. BACKGROUND

[0002] With the rapid development of generative artificial intelligence technology, large models have been widely used in intelligent customer service, content generation, data analysis and other core business scenarios. Such services have characteristics such as high consumption of computing resources, high access frequency and strong data sensitivity, and their security protection and resource control have become key challenges for enterprise-level applications.

[0003] In terms of access authentication, the existing technology has the following limitations: the traditional authentication method based on username and password is vulnerable to brute force attacks, and it is difficult to implement fine-grained permission control; the single sign-on (SSO) solution improves user experience, but has cross-domain authentication delay problems in distributed large model clusters; some systems use hard-coded permission configuration, which cannot meet the permission adjustment requirements in dynamic business scenarios, resulting in the need to restart services for permission updates, affecting availability.

[0004] In terms of traffic control, the current solution has obvious shortcomings: the static threshold throttling strategy cannot adapt to the tidal access characteristics of large model services, and may cause service avalanches during peak periods and resource idling during trough periods; there is a lack of differentiated traffic scheduling mechanism based on user levels, resulting in resource contention between core business and ordinary requests; most gateway products only support single-dimensional throttling (such as QPS), making it difficult to implement precise control in combination with multiple dimensions such as concurrency and bandwidth occupancy.

[0005] APISIX, as a high-performance gateway in the field of cloud native, has advantages such as dynamic configuration and plug-in extension, but there is no specialized authentication and traffic control solution for large model services in the existing technology, and the native plug-in system lacks adaptation to the characteristics of large model services, which cannot meet the security protection needs of enterprise-level applications. SUMMARY

[0006] The purpose of the present application is to provide a method and system for realizing large model access authentication and traffic control based on an APISIX gateway, to build a multi-dimensional authentication system, support fine-grained permission control, and dynamically bind user identity and access permissions to solve the problems raised in the background technology.

[0007] To achieve the above purpose, the present application provides the following technical solution: a method for realizing large model access authentication and traffic control based on an APISIX gateway, comprising the following steps: S1. The client generates an HTTPS request containing APIKey and sends it to the APISIX gateway cluster; S2. The gateway extracts the APIKey and forwards it to the authentication service to verify the key's validity. S3. After verification, the authentication service returns the user's permission matrix, and the gateway checks whether the request is within the permission range. S4. After permission verification, the traffic control plug-in detects traffic indicators according to the preset strategy. S5. When the traffic indicators are not exceeded, the gateway forwards the request to the large model service and returns the result. S6. When the traffic indicators are exceeded, the gateway performs flow control / fuse processing and returns the corresponding error response.

[0008] Preferably, the key verification in step S2 includes: Check if the APIKey exists in the MySQL database; Verify if the current time is within the key's validity period; Confirm that the key has not been marked as revoked; If any of the above conditions is not met, return a 401 Unauthorized response.

[0009] Preferably, the traffic detection in step S4 includes: Collect the current QPS, number of concurrent connections, and bandwidth usage; Match the traffic threshold corresponding to the user level; Combine the CPU usage of the large model service node for comprehensive judgment; High-priority user requests automatically increase the traffic quota by 20%.

[0010] Preferably, it also includes a dynamic configuration update step: The administrator modifies the strategy parameters through the management platform; The configuration center generates a new version of the configuration and synchronizes it to the gateway cluster; The gateway node incrementally updates the local configuration without interrupting service; After the configuration takes effect, return confirmation information containing the version number.

[0011] A large model access authentication and traffic control system based on APISIX gateway, comprising: An APISIX gateway cluster composed of at least 3 nodes, the gateway integrates authentication plug-ins, traffic control plug-ins, and monitoring plug-ins, and supports dynamic configuration synchronization through etcd; An independent authentication service module, including an identity authentication unit, a permission management unit, and a Redis cache unit, the authentication service communicates with the gateway through the gRPC protocol; Traffic control center, including indicator collection unit, strategy engine and dynamic adjustment unit, the control center uses PID algorithm to realize adaptive adjustment of traffic threshold; The large model service cluster is deployed using Kubernetes orchestration and includes a model inference engine and a health check agent. At the client level, a request is initiated via the HTTPS protocol, with the APIKey field carried in the request header.

[0012] Preferably, the authentication service module performs the following operations: Receive the APIKey forwarded by the gateway and verify its validity, expiration date, and revocation status; Based on the RBAC model, the user permission matrix can be queried, and the permission granularity can be refined to the model type, interface version and data range. By broadcasting permission change events through the Kafka message queue, the consistency of permission data in the gateway cluster is ensured.

[0013] Preferably, the traffic control plugin supports multi-dimensional rate limiting: QPS-based token bucket rate limiting supports configuring differentiated thresholds for different users; Based on the connection limit for concurrency, the maximum number of concurrent connections per IP address is configurable; Bandwidth-based flow control applies rate limits to responses to large files.

[0014] Preferably, the strategy engine supports the following dynamic adjustment mechanisms: Based on the prediction of traffic trends within 15 minutes using LSTM neural networks, the traffic limiting strategy can be adjusted in advance. The traffic allocation ratio is automatically adjusted based on the service load, with an adjustment range of ±10% / minute. When the error rate exceeds 5% for 30 consecutive seconds, the circuit breaker mechanism is triggered. The circuit breaker states include three states: open, half-open, and closed.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention proposes a method and system for large-model access authentication and traffic control based on the APISIX gateway. Through an innovative APIKey authentication mechanism and dynamic traffic control strategy, it constructs a full-link, intelligent security protection system. When a client initiates a large-model access request, the HTTP / HTTPS request carrying the APIKey is first intercepted in real-time by the APISIX gateway based on a Lua plugin mechanism. Subsequently, the gateway executes a three-tier verification process: Key validity verification: By connecting to the backend key management database, the APIKey is hashed using the HMAC-SHA256 algorithm, and the key validity period is verified to identify illegally forged or expired keys; Permission scope matching: Based on the pre-configured RBAC permission model, the requester's operation permissions are verified to ensure they match the target API interface, accurately blocking unauthorized access; Dynamic traffic detection: Combining the sliding window algorithm and token bucket rate limiting strategy, the request frequency is monitored in real-time. When the QPS exceeds a preset threshold, a tiered rate limiting mechanism is automatically triggered to prioritize the processing of high-priority requests. The gateway performs request forwarding or rate limiting based on preset policies, effectively solving the risks of illegal access and resource allocation imbalance faced by large model services, significantly improving service stability and resource utilization efficiency, and is suitable for the security operation and maintenance of large model services in multiple scenarios. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the present invention clear and complete, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some, not all, embodiments of the present invention, and are merely illustrative of the embodiments of the present invention. They are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1: This invention provides a technical solution: a method for implementing large-model access authentication and traffic control based on an APISIX gateway, comprising: I. Authentication Process Implementation This invention employs a "two-stage authentication" mechanism, the specific process of which is as follows: (I) First stage: Identity verification 1. The user terminal generates an HTTPS request containing the APIKey and sends it to the APISIX gateway via the public or intranet; 2. The gateway triggers the auth-plugin to extract the APIKey field from the request header and forwards it to the authentication service via the gRPC protocol; 3. The authentication service queries the key database (MySQL) to verify the validity of the APIKey (whether it exists, whether it is within the validity period, or whether it has been revoked). 4. When verification fails, the gateway returns a 401 Unauthorized response, including error codes (such as KEY_EXPIRED, KEY_INVALID) and retry suggestions; 5. If the verification is successful, proceed to the second stage and synchronously return the user's unique identifier (UserID) and role information.

[0019] (II) Second Phase: Permission Verification 1. The authentication service queries the permission matrix based on UserID to obtain the list of models that the user can access, the interface permissions and their expiration time limits; 2. Permission information is returned to the gateway in JSON format. 3. The gateway compares the target model of the request (such as / model / gpt-4 in the request URL) with the allowedModels list. If they do not match, it returns a 403 Forbidden error. 4. Check if the current time is within the validity period of the access permission. If the validity period has expired, deny access and prompt for key renewal. 5. After the permission verification is successful, the gateway injects the X-User-ID and X-Role fields into the request header and forwards it to the large model service.

[0020] (III) Dynamic permission update mechanism: 1. When an administrator modifies user permissions through the web console, the system automatically generates a permission change event; 2. The event is broadcast to all gateway nodes via a Kafka message queue (topic: permission-changes); 3. After consuming the message, the gateway updates the local permission cache (TTL=5 minutes) and records the change log to the ELK cluster.

[0021] II. Traffic Control Strategies This invention innovatively designs a "three-dimensional traffic control model," which is implemented as follows: (I) Dimension 1: Basic Rate Limiting 1. Implement QPS control based on the token bucket algorithm, and support configuring independent thresholds for each APIKey (default 10-1000 times / second). 2. Concurrent connection limit: The maximum number of concurrent requests per IP address can be controlled via the limit-conn plugin (default 50). 3. Bandwidth control: Enable bandwidth limiting (default 2Mbps) for requests with a response body size exceeding 1MB to avoid excessive resource consumption during large file transfers.

[0022] (II) Dimension Two: Intelligent Scheduling 1. Priority queue based on user level: VIP user requests are assigned to the high-priority queue (70% weight), and ordinary users are assigned to the default queue (30% weight). 2. Service health awareness: When the CPU utilization of a model node exceeds 80%, the traffic allocation ratio for that node is automatically reduced; (III) Dimension Three: Circuit Breaker Protection 1. The circuit breaker mechanism is adopted, which triggers the circuit breaker when the model service error rate exceeds 5% for 30 consecutive seconds; 2. The circuit breaker state has three states: Open, Half-Open, and Closed. The state switching conditions are configurable. 3. During the circuit breaker period, the gateway returns a 503 Service Unavailable response and notifies the operations and maintenance personnel via SMS / email.

[0023] III. APISIX Gateway Configuration Implementation (I) Plugin Chain Configuration: 1. Using APISIX's plugin chain mechanism, plugins are loaded in the following order: 2. CORS: Handles cross-origin requests 3. Prometheus: Metrics Collection 4. auth-plugin: Identity authentication 5. traffic-control-plugin: Traffic management 6. proxy-rewrite: Request rewrite 7. proxy-pass: Request forwarding (II) Dynamic Configuration Update Process: 1. Administrators modify policy parameters on the management platform (such as adjusting a user's QPS threshold). 2. The configuration center (etcd) receives change requests and generates version numbers (based on the Raft protocol). 3. Gateway nodes detect configuration changes through the Watch mechanism and incrementally update local configurations (without requiring a restart); 4. After the configuration takes effect, the gateway returns a configuration success log, which includes the version number and the effective time.

[0024] (III) High Availability Configuration: 1. The gateway cluster should be deployed with at least 3 nodes, and load balancing should be achieved through Nginx; 2. Enable a health check mechanism, sending a probe request every 3 seconds; if it fails 3 times in a row, the node will be removed. 3. Configure the data synchronization timeout to 500ms. If the timeout occurs, a retry mechanism will be triggered (up to 3 times).

[0025] Example 2, based on Example 1, proposes a system for large-scale model access authentication and traffic control based on an APISIX gateway, including: a) An APISIX gateway cluster consisting of at least 3 nodes, wherein the gateway integrates authentication plugins, traffic control plugins and monitoring plugins, and supports dynamic configuration synchronization via etcd; b) An independently deployed authentication service module, comprising an identity authentication unit, a permission management unit, and a Redis caching unit, wherein the authentication service communicates with the gateway via the gRPC protocol; c) Traffic control center, including indicator collection unit, strategy engine and dynamic adjustment unit, wherein the control center uses PID algorithm to realize adaptive adjustment of traffic threshold; d) A large model service cluster, deployed using Kubernetes orchestration, includes a model inference engine and a health check agent; e) At the client level, a request is initiated via the HTTPS protocol, with the APIKey field carried in the request header.

[0026] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for implementing large-scale model access authentication and traffic control based on an APISIX gateway, characterized in that: Includes the following steps: S1. The client generates an HTTPS request containing the APIKey and sends it to the APISIX gateway cluster; S2. The gateway extracts the APIKey and forwards it to the authentication service to verify the validity of the key; S3. After successful verification, the authentication service returns the user permission matrix, and the gateway verifies whether the request is within the permission scope; S4. After the permission verification is passed, the traffic control plugin detects traffic indicators according to the preset policy; S5. When the traffic metric is within limits, the gateway forwards the request to the large model service and returns the result; S6. When the traffic limit is exceeded, the gateway performs rate limiting / circuit breaking and returns the corresponding error response.

2. The method for implementing large-scale model access authentication and traffic control based on APISIX gateway according to claim 1, characterized in that: Key verification in step S2 includes: Check if the APIKey exists in the MySQL database; Verify whether the current time is within the key's validity period; Confirm that the key is not marked as revoked; If any of the above conditions are not met, a 401 Unauthorized response will be returned.

3. The method for implementing large-scale model access authentication and traffic control based on APISIX gateway according to claim 2, characterized in that: The flow detection in step S4 includes: Collect current QPS, concurrent connections, and bandwidth usage; Match the traffic threshold corresponding to the user level; A comprehensive judgment is made based on the CPU utilization of the large model service nodes; High-priority users will have their traffic quota automatically increased by 20%.

4. The method for implementing large-scale model access authentication and traffic control based on APISIX gateway according to claim 3, characterized in that: It also includes a dynamic configuration update step: Administrators modify policy parameters through the management platform; The configuration center generates a new version of the configuration and synchronizes it to the gateway cluster; Gateway nodes incrementally update their local configurations without interrupting service during the update process. After the configuration takes effect, a confirmation message containing the version number will be returned.

5. A system for large-scale access authentication and traffic control based on an APISIX gateway, employing the method described in claim 4, characterized in that: include: An APISIX gateway cluster consisting of at least 3 nodes, wherein the gateway integrates authentication plugins, traffic control plugins and monitoring plugins, and supports dynamic configuration synchronization via etcd; The independently deployed authentication service module includes an identity authentication unit, a permission management unit, and a Redis caching unit. The authentication service communicates with the gateway via the gRPC protocol. The traffic control center includes an indicator collection unit, a strategy engine, and a dynamic adjustment unit. The control center uses a PID algorithm to achieve adaptive adjustment of traffic thresholds. The large model service cluster is deployed using Kubernetes orchestration and includes a model inference engine and a health check agent. At the client level, a request is initiated via the HTTPS protocol, with the APIKey field carried in the request header.

6. A system for large-scale model access authentication and traffic control based on an APISIX gateway as described in claim 5, characterized in that: The authentication service module performs the following operations: Receive the APIKey forwarded by the gateway and verify its validity, expiration date, and revocation status; Based on the RBAC model, the user permission matrix can be queried, and the permission granularity can be refined to the model type, interface version and data range. By broadcasting permission change events through the Kafka message queue, the consistency of permission data in the gateway cluster is ensured.

7. A system for large-scale model access authentication and traffic control based on an APISIX gateway as described in claim 6, characterized in that: The traffic control plugin supports multi-dimensional rate limiting: QPS-based token bucket rate limiting supports configuring differentiated thresholds for different users; Based on the connection limit for concurrency, the maximum number of concurrent connections per IP address is configurable; Bandwidth-based flow control applies rate limits to responses to large files.

8. A system for large-scale model access authentication and traffic control based on an APISIX gateway as described in claim 7, characterized in that: The strategy engine supports the following dynamic adjustment mechanisms: Based on the prediction of traffic trends within 15 minutes using LSTM neural networks, the traffic limiting strategy can be adjusted in advance. The traffic allocation ratio is automatically adjusted based on the service load, with an adjustment range of ±10% / minute. When the error rate exceeds 5% for 30 consecutive seconds, the circuit breaker mechanism is triggered. The circuit breaker states include three states: open, half-open, and closed.

Images