A high-performance API gateway system and method based on the RUST language

Abstract

The application relates to a high-performance API gateway system and method based on a RUST language, which comprises a gateway core module, is constructed based on a Pingora asynchronous framework, and is used for receiving and processing HTTP requests; a routing engine module, which comprises a multistage routing cache structure; a lock-free configuration updating module, which adopts an ArcSwap atomic pointer exchange mechanism and is used for realizing hot updating of routing configuration and plug-in configuration without restarting a service; a plug-in chain processing module, which is used for sequentially performing authentication, signature verification, flow limiting and gray release processing on a request matched through routing; and a configuration synchronization module, which receives a configuration change notification through a message queue and triggers the lock-free configuration updating module to perform configuration updating. The application can realize high-concurrency and low-latency request processing, lock-free configuration hot updating without service interruption, an efficient multistage routing cache mechanism and flexible and extensible enterprise-level plug-in functions, thereby helping to reduce server resource consumption and improve the overall performance and maintainability of the system.

Description

Technical Field

[0001] This invention relates to the field of computer network communication technology, and in particular to a high-performance API gateway system and method based on the RUST language. Background Technology

[0002] In modern microservice architectures, API gateways act as an intermediary layer between clients and backend services, undertaking functions such as request routing and forwarding, protocol conversion, traffic control, and security authentication. With the deepening of enterprise digital transformation, API gateways have become the traffic entry point for distributed systems. Currently, commonly used API gateway technologies include Spring Cloud Gateway (Java-based), Nginx (C-based), and Kong (Lua-based). These gateway systems typically employ reactive programming and filter chaining to improve throughput and use message queues to achieve configuration change notifications and system decoupling. Regarding configuration management, existing gateway systems generally use a gateway configuration center to pull and distribute service configurations. When routes change, a message component sends a change notification to the gateway instance, and the gateway updates its in-memory routes upon receiving the notification. For traffic control, some systems use Count-Min Sketch and other probabilistic data structures to achieve efficient traffic statistics and rate limiting.

[0003] However, existing API gateway solutions face several challenges in practical applications. First, Java Virtual Machine-based gateway systems suffer from performance bottlenecks in high-concurrency scenarios, with limited queries per second (QPS) and high response latency, while also consuming significant memory, necessitating the deployment of numerous servers to support the traffic. Second, existing gateway configuration change mechanisms typically require a service restart to take effect, leading to service interruptions and traffic loss, impacting business continuity. Third, open-source gateway products lack enterprise-level functionality, such as missing features like canary releases, signature verification, and encryption / decryption, or requiring complex configurations to implement. Furthermore, existing gateway configuration management relies on files or manual operations, resulting in low efficiency and a high risk of errors. Finally, existing gateway plugin systems have limited scalability, and custom development is costly, making it difficult to meet the diverse business needs of enterprises. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a high-performance API gateway system and method based on the Rust language, which can realize high-concurrency and low-latency request processing, hot configuration updates without service interruption, efficient multi-level route caching mechanism, and flexible and scalable enterprise-level plug-in functions, thereby helping to reduce server resource consumption and improve the overall performance and maintainability of the system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution.

[0006] Firstly, the present invention provides a high-performance API gateway system based on the Rust language, which adopts the following technical solution: The gateway core module, built on the Pingora asynchronous framework, is used to receive and process HTTP requests. The routing engine module includes a multi-level routing cache structure, which includes: A HashMap data structure used for static route matching; A prefix tree data structure for prefix route matching; and A radix tree data structure used for dynamic route matching; The lock-free configuration update module adopts the ArcSwap atomic pointer exchange mechanism to achieve hot updates of routing configuration and plugin configuration without restarting the service; A plug-in chain processing module, which sequentially performs authentication, signature verification, rate limiting, and canary release processing on requests that have undergone route matching; and The configuration synchronization module receives configuration change notifications through a message queue and triggers the lock-free configuration update module to perform configuration updates.

[0007] Furthermore, in the above system, the plug-in chain processing module includes a dual-track plug-in architecture, which includes: System plugins implemented using enumeration structures; and Custom plugins implemented using trait interfaces.

[0008] Furthermore, in the above system, the system plugins and the custom plugins are registered through an automatic registration mechanism, which is implemented based on the linkme library.

[0009] Furthermore, in the above system, the authentication processing of the plug-in chain processing module adopts a two-level caching structure, which includes: A local LRU cache for storing authentication tokens; and A Redis cache used to store authentication tokens.

[0010] Furthermore, in the above system, the rate limiting processing of the plug-in chain processing module uses the Count-Min Sketch algorithm for traffic statistics and rate limiting judgment.

[0011] Furthermore, in the above system, the canary release processing of the plug-in chain processing module supports percentage mode and whitelist mode, which are used to forward requests to the baseline service or the canary service.

[0012] Furthermore, the system also includes a visual management module, which provides a web console interface to manage routing configuration, plugin configuration, and canary release configuration, and writes configuration changes to the database and sends configuration change notifications through the message queue.

[0013] Secondly, the present invention provides an API gateway request processing method based on the Rust language, which adopts the following technical solution: HTTP requests are received through the gateway core module built on the Pingora asynchronous framework; The HTTP request is routed using a multi-level route caching structure, wherein the multi-level route caching structure sequentially uses a HashMap data structure for static route matching, a prefix tree data structure for prefix route matching, and a radix tree data structure for dynamic route matching. When the route is successfully matched, the HTTP request is sequentially processed through the plugin chain, including authentication, signature verification, rate limiting, and canary release. The requests processed through the plug-in chain are forwarded to the backend service; and When a configuration change notification is received, the routing configuration and plugin configuration are updated without restarting the service through the ArcSwap atomic pointer exchange mechanism.

[0014] Furthermore, in the above method, the authentication process employs a two-level caching structure, which includes: A local LRU cache for storing authentication tokens; and A Redis cache used to store authentication tokens; The authentication process first queries the local LRU cache, and if no match is found, queries the Redis cache.

[0015] Furthermore, in the above method, the signature verification process uses the HMAC-SHA256 algorithm to verify the signature of the request in order to prevent the request from being tampered with.

[0016] Furthermore, in the above method, the rate limiting process uses the Count-Min Sketch algorithm to perform traffic statistics and rate limiting judgment, and returns an error response when the judgment of the request exceeds the rate limiting threshold.

[0017] Furthermore, in the above method, the canary release process supports percentage mode and whitelist mode, and determines whether to forward the request to the canary service or the baseline service based on the user identifier.

[0018] Thirdly, the present invention provides a readable storage medium, which adopts the following technical solution: A readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any one of the second aspects above.

[0019] In summary, compared with the prior art, the present invention has at least one of the following beneficial technical effects: The high-performance API gateway system based on the Rust language described in this invention improves request processing concurrency and reduces response latency by using the Pingora asynchronous framework to build the gateway core module. Through a multi-level route caching structure, appropriate data structures can be used to match different types of routes, thereby improving route lookup efficiency. Lock-free configuration updates implemented through the ArcSwap atomic pointer exchange mechanism can complete hot configuration updates without restarting the service, helping to avoid service interruptions and traffic loss. The plug-in chain processing module sequentially performs authentication, signature verification, rate limiting, and canary release processing on requests, meeting the security and traffic control requirements of enterprise-level applications. The combination of a configuration synchronization module and a message queue enables timely synchronization of configuration changes, improving the system's configuration management efficiency and ease of maintenance. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A block diagram of an embodiment of the high-performance API gateway system of the present invention is shown.

[0022] Figure 2 An architectural block diagram of one embodiment of the plug-in chain processing module of the present invention is shown.

[0023] Figure 3 A flowchart of an embodiment of the API gateway request processing method of the present invention is shown.

[0024] Figure 4 A flowchart of an embodiment of the authentication processing and signature verification method of the present invention is shown.

[0025] Figure 5A flowchart of an embodiment of the rate limiting and canary release processing method of the present invention is shown. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, it should be understood that the specific embodiments described herein are only for illustration and explanation of this application and are not intended to limit this application.

[0027] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments of this application. Furthermore, the descriptions of each embodiment in the following embodiments have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0028] The method steps described in this embodiment of the invention can be executed in the order described in the specific implementation, or the execution order of each step can be adjusted according to actual needs, provided that the technical problem can be solved. These are not listed one by one here.

[0029] The present invention will be further described in detail below with reference to the accompanying drawings.

[0030] Reference Figure 1 The high-performance API gateway system 100 based on the Rust language disclosed in this embodiment of the invention includes a gateway core module 102, a routing engine module, a lock-free configuration update module, a plugin chain processing module, and a configuration synchronization module. The gateway core module 102 is built on the Pingora asynchronous framework 104 and is used to receive and process HTTP requests. The Pingora asynchronous framework 104 is an open-source Rust HTTP proxy framework from Cloudflare, which has been proven in production to handle over 1 trillion requests per day, providing high-performance asynchronous I / O and network processing capabilities.

[0031] The gateway core module 102 employs zero-copy network I / O technology, avoiding memory copying during data transmission and directly transmitting data from the kernel buffer to the network, reducing CPU consumption and memory usage. The high-performance API gateway system 100 utilizes the zero-cost abstraction feature of the Rust language, achieving high performance without increasing runtime overhead. This zero-cost abstraction feature allows high-level abstractions to generate machine code with the same efficiency as hand-written low-level code after compilation, thus achieving high performance while maintaining code readability and maintainability.

[0032] In terms of performance metrics, the high-performance API gateway system 100 achieves a processing capacity of over 118,000 QPS on an 8-core machine, with a P99 latency of 19.56ms. Regarding memory usage, the high-performance API gateway system 100 consumes only 997MB of memory when processing over 120,000 QPS, a 90% reduction in memory usage compared to traditional Java gateways. Figure 1 As shown, after receiving the HTTP request through the Pingora asynchronous framework 104, the gateway core module 102 passes the request to the routing engine module for route matching, and performs authentication, signature verification, rate limiting and canary release processing through the plug-in chain processing module, and forwards the processed request to the backend service cluster.

[0033] The routing engine module 106 is connected to the gateway core module 102. The routing engine module 106 includes a multi-level routing cache structure. The multi-level routing cache structure includes a HashMap data structure for static route matching, a prefix tree data structure for prefix route matching, and a radix tree data structure for dynamic route matching.

[0034] In the multi-level route caching structure, the HashMap static route 108 is used for static route matching, achieving a lookup operation with O(1) time complexity. The HashMap data structure maps route paths to corresponding backend service addresses through a hash function, making the lookup time of static routes independent of the route table size, thus achieving extremely low latency lookup with near constant time. Static routes are suitable for API requests with fixed paths, such as route paths like " / api / v1 / users" that do not contain dynamic parameters.

[0035] Prefix tree routing (110) is used for prefix route matching, enabling a lookup operation with O(n) time complexity, where n is the length of the route path. The prefix tree data structure is built into a tree structure layer by layer according to the characters of the route path, with each node representing a character or path segment in the path. Prefix routing is suitable for API requests with a common prefix, such as multiple route paths starting with " / api / v1 / ".

[0036] Radix tree dynamic routing (112) is used for dynamic route matching, enabling lookup operations with O(log n) time complexity. The radix tree data structure is a compressed prefix tree, reducing the tree depth and number of nodes by merging paths with a single child node. Dynamic routing is suitable for API requests containing path parameters, such as route paths with dynamic parameters like " / api / v1 / users / {id}".

[0037] In some implementations, the routing engine module 106 employs a hierarchical routing matching strategy of static → prefix → dynamic. When an HTTP request arrives, the routing engine module 106 performs route matching sequentially through the HashMap static route 108, the prefix tree route 110, and the radix tree dynamic route 112. The hierarchical routing matching strategy blocks most requests from the lowest-cost data structure. Since static routes account for most of the traffic in practical applications, using the HashMap data structure with O(1) time complexity to process static routes can reduce the overall P99 latency.

[0038] In some implementations, the routing engine module 106 employs CPU cache optimization technology to improve data access efficiency and overall system performance. This technology optimizes the memory layout of data structures, allowing frequently accessed routing data to reside in the CPU cache, reducing memory access latency. The routing engine module 106 stores hotspot routing data aligned to the cache line size, improving cache hit rate and further enhancing route matching performance.

[0039] The high-performance API gateway system 100 also includes a lock-free configuration update module 114. This module 114 employs the ArcSwap atomic pointer exchange mechanism 116 to enable hot updates of routing and plugin configurations without restarting the service. ArcSwap is an atomic pointer exchange library in Rust that uses atomic operations to achieve lock-free configuration updates, avoiding the performance overhead of traditional locking mechanisms.

[0040] In the lock-free configuration update module 114, the ArcSwap atomic pointer 116 reads the current configuration through an atomic load operation, with a read operation time of less than 1 nanosecond. When the configuration changes, the ArcSwap atomic pointer 116 replaces the old configuration with the new configuration through an in-place swap operation, with the configuration change taking effect in less than 10 microseconds. The atomic pointer swapping mechanism ensures that the configuration update process does not block ongoing requests, achieving zero-downtime changes.

[0041] In some implementations, the lock-free configuration update module 114 automatically releases the old configuration resources after the configuration update. When the ArcSwap atomic pointer 116 completes the configuration swap and the reference count of the old configuration drops to zero, Rust's ownership mechanism automatically reclaims the memory resources occupied by the old configuration, eliminating the need for manual memory management. This automatic release mechanism avoids memory leaks while ensuring the security of the configuration update process.

[0042] The high-performance API gateway system 100 includes a configuration synchronization module 128, which is connected to a lock-free configuration update module 114 and a message queue 130. The configuration synchronization module 128 receives configuration change notifications through the message queue 130 and triggers the lock-free configuration update module 114 to perform configuration updates.

[0043] Message queue 130 is implemented using RocketMQ, an open-source distributed message queue from Alibaba that supports message persistence, ordered messages, and transactional messages. When the management console modifies the routing configuration or plugin configuration, the configuration change is written to the database, and a configuration change notification is sent through message queue 130. The configuration synchronization module 128 listens for configuration change messages in message queue 130. Upon receiving a message, it parses the configuration content, constructs a new routing engine and plugin configuration, and triggers the lock-free configuration update module 114 to perform an atomic swap operation through the ArcSwap atomic pointer 116, making the new configuration take effect immediately.

[0044] In some implementations, the high-performance API gateway system 100 also includes a connection to the Nacos service registry for service discovery and configuration management. The Nacos service registry stores the address information and health status of backend services. The high-performance API gateway system 100 obtains the real-time service list of the backend service cluster 132 through Nacos, enabling dynamic service discovery and load balancing. When a backend service instance comes online or goes offline, the Nacos service registry notifies the high-performance API gateway system 100 to update the service list, ensuring that requests are forwarded to healthy backend service instances.

[0045] The high-performance API gateway system 100 also includes a plug-in chain processing module 118, which is connected to the routing engine module 106 and is used to sequentially perform authentication, signature verification, rate limiting, and canary release processing on requests that have been matched by the route. The plug-in chain processing module 118 includes authentication processing 120, signature verification 122, rate limiting processing 124, and canary release 126.

[0046] After the routing engine module 106 completes route matching, the HTTP request enters the plugin chain processing module 118. The plugin chain processing module 118 executes each processing step sequentially according to a predefined order. Authentication processing 120 authenticates the request, verifying the validity of the authentication token carried in the request. Signature verification 122 verifies the signature of the request, checking whether the request content has been tampered with. Rate limiting processing 124 controls the flow of the request, determining whether the request exceeds a preset rate limiting threshold. Canary release 126 determines whether the request should be forwarded to the baseline service or the canary service based on the canary release strategy.

[0047] The plug-in chain processing module 118 is connected to the backend service cluster 132. After an HTTP request passes through all processing stages—authentication processing 120, signature verification 122, rate limiting processing 124, and canary release 126—the plug-in chain processing module 118 forwards the processed request to the backend service cluster 132. The backend service cluster 132 includes multiple backend service instances used to process business logic and return response results.

[0048] The plug-in chain processing module 118 employs a fail-fast mechanism during request processing. When authentication processing 120 determines authentication failure, signature verification 122 determines signature failure, or rate limiting processing 124 determines the request triggers rate limiting, the plug-in chain processing module 118 directly returns an error response, without executing subsequent processing steps or forwarding the request to the backend service cluster 132. This fail-fast mechanism protects resources and prevents invalid requests from consuming backend service resources.

[0049] In some implementations, after forwarding the request to the backend service cluster 132, the plug-in chain processing module 118 collects performance metrics, logs, and tracing information. Performance metrics include data such as request processing time and backend service response time. Tracing information is used to trace the complete call chain of the request in the distributed system, facilitating problem localization and performance analysis. The plug-in chain processing module 118 returns the response result from the backend service cluster 132 to the client, completing the entire request processing flow.

[0050] Furthermore, refer to Figure 2 The plugin chain processing module 200 includes a dual-track plugin architecture 202, an authentication processing module 214, a rate limiting processing module 220, and a canary release module 224. The dual-track plugin architecture 202 is the core architecture design of the plugin chain processing module 200, used to support the implementation and management of different types of plugins.

[0051] The dual-track plug-in architecture 202 includes system plug-in 204 and custom plug-in 208. System plug-in 204 is implemented using an enumeration structure 206, while custom plug-in 208 is implemented using a trait interface 210. By distinguishing between system plug-in 204 and custom plug-in 208, the dual-track plug-in architecture 202 achieves a balance between performance optimization and flexible scalability.

[0052] System plugin 204 is implemented using an enumeration structure (206) to achieve performance optimization and zero overhead. The enumeration structure implementation (206) leverages Rust's enumeration type feature, defining all system plugins as different variants of the enumeration. The enumeration structure determines the type and size of all variants at compile time, allowing the compiler to perform inline optimization and static dispatch, avoiding runtime dynamic dispatch overhead. System plugin 204 includes built-in functions such as authentication processing, signature verification, rate limiting, and canary releases, which are invoked with zero runtime overhead through the enumeration structure implementation (206).

[0053] In some implementations, the enumeration structure implementation 206 ensures that the call path of the system plugin 204 is completely determined at compile time. The compiler generates the corresponding machine code branch based on the enumeration variant and jumps directly to the corresponding processing logic through pattern matching, without needing indirect calls through virtual function tables. The zero-overhead characteristic of the enumeration structure implementation 206 makes the execution efficiency of the system plugin 204 comparable to hand-written low-level code, while maintaining code readability and maintainability.

[0054] Custom plugin 208 uses trait interface implementation 210 to support flexible extension. Trait interface implementation 210 utilizes the trait feature of the Rust language to define the plugin interface specification. Custom plugin 208 extends the gateway functionality by implementing the trait interface. Trait interface implementation 210 supports dynamic dispatch, enabling custom plugin 208 to be loaded and executed at runtime, meeting customized business needs.

[0055] Trait interface implementation 210 defines unified plugin lifecycle methods, including interfaces for plugin initialization, request processing, and resource release. Custom plugins 208 can integrate into the processing flow of the plugin chain processing module 200 by implementing the methods defined in trait interface implementation 210. The flexible extensibility of trait interface implementation 210 allows developers to create custom plugins according to business needs without modifying the gateway's core code.

[0056] The dual-track plugin architecture 202 also includes an automatic registration mechanism 212. The automatic registration mechanism 212, implemented based on the linkme library, is used to register and manage system plugins 204 and custom plugins 208. The linkme library is a distributed slicing library for the Rust language, supporting the collection of plugin definitions scattered across different modules at compile time and their assembly into a complete plugin list at runtime.

[0057] In some implementations, the automatic registration mechanism 212 leverages the distributed slicing feature of the Linkme library to eliminate the need for boilerplate code in plugin development. When defining a custom plugin 208, developers use macro-annotation plugins provided by the Linkme library. The automatic registration mechanism 212 automatically collects all annotated plugins during compilation and completes plugin registration upon gateway startup. This automatic registration mechanism 212 avoids the tedious work of manually maintaining the plugin registry, reducing the complexity of plugin development.

[0058] The automatic registration mechanism 212 supports hot-swapping of plugins. Once a new custom plugin 208 is compiled and deployed, the automatic registration mechanism 212 can load the new plugin without restarting the gateway service. This hot-swapping functionality, combined with the ArcSwap atomic pointer exchange mechanism of the lock-free configuration update module, ensures that adding and removing plugins has no impact on currently processed requests, guaranteeing business continuity.

[0059] In some implementations, the plug-in chain processing module 200 also includes AES encryption / decryption functionality for encrypting and decrypting request or response data. The AES encryption / decryption function uses the AES symmetric encryption algorithm to protect sensitive data. When a request carries encrypted data, the plug-in chain processing module 200 decrypts the data using AES encryption / decryption before passing it to the backend service. When the backend service returns sensitive data, the plug-in chain processing module 200 encrypts the response data using AES encryption / decryption before returning it to the client. The AES encryption / decryption function meets the enterprise's need for secure data transmission, protecting sensitive information from being stolen during network transmission.

[0060] Furthermore, the plug-in chain processing module 200 includes an authentication processing module 214, which employs a two-level caching structure to improve the efficiency of authentication processing. The two-level caching structure includes a local LRU cache 216 for storing authentication tokens and a Redis cache 218 for storing authentication tokens.

[0061] like Figure 2 As shown, the local LRU cache 216 is the first-level cache in a two-level caching structure, used to store authentication token information locally on the gateway. The local LRU cache 216 employs an LRU (Least Recently Used) eviction policy, automatically evicting the least recently used cache entry when the cache capacity reaches its limit. The local LRU cache 216 is configured to hold 200,000 tokens, capable of storing 200,000 authentication token records. The cache hit rate of the local LRU cache 216 is 75-85%, meaning that 75-85% of authentication requests can directly obtain authentication token information from the local LRU cache 216 without accessing an external caching service.

[0062] Redis cache 218 is the second-level cache in a two-level cache architecture, serving as a backup cache when the local LRU cache 216 misses. Redis cache 218 uses the Redis distributed caching service to store authentication token information, supporting shared cache data across multiple gateway instances. The hit rate of Redis cache 218 is 10-15%, meaning that when the local LRU cache 216 misses, 10-15% of authentication requests can retrieve authentication token information from Redis cache 218.

[0063] The local LRU cache 216 and Redis cache 218 are connected by a dotted line, representing the query relationship of the two-level cache structure. When performing authentication processing, the authentication processing module 214 accesses the two-level cache structure according to a predefined query order. After retrieving the authentication token from the HTTP request, the authentication processing module 214 queries the local LRU cache 216. When the local LRU cache 216 is hit, the authentication processing module 214 directly obtains the user information corresponding to the authentication token and completes the authentication process. When the local LRU cache 216 is not hit, the authentication processing module 214 queries the Redis cache 218 to obtain the authentication token information.

[0064] In some implementations, when Redis cache 218 is hit, the authentication processing module 214 fills the obtained authentication token information back into the local LRU cache 216, so that subsequent requests with the same authentication token can directly retrieve information from the local LRU cache 216. This two-level caching structure reduces the number of accesses to Redis cache 218 through the high hit rate of the local LRU cache 216, thus mitigating the impact of network latency on authentication processing performance. The access latency of the local LRU cache 216 is in the nanosecond range, while the access latency of the Redis cache 218 is in the millisecond range. The layered design of the two-level caching structure allows most authentication requests to complete cache queries within nanoseconds, improving the overall performance of authentication processing.

[0065] Furthermore, the plug-in chain processing module 200 also includes a rate limiting processing module 220, which uses the Count-Min Sketch algorithm 222 for traffic statistics and rate limiting judgment. The Count-Min Sketch algorithm 222 is a probabilistic data structure that uses multiple hash functions and a counter array to achieve approximate counting, and is used to estimate traffic frequency in high-concurrency scenarios.

[0066] The Count-Min Sketch algorithm 222 achieves high-concurrency frequency control with O(1) time complexity. When performing traffic statistics, the Count-Min Sketch algorithm 222 maps request identifiers to different positions in a counter array using multiple independent hash functions and increments the counters at those positions. When performing rate limiting judgment, the Count-Min Sketch algorithm 222 reads the counter values ​​at the mapped positions of multiple hash functions and takes the minimum value as an estimate of the request frequency. The query and update operations of the Count-Min Sketch algorithm 222 have constant time complexity, independent of the total number of requests and the number of rate limiting rules, enabling the rate limiting module 220 to maintain stable processing performance in high-concurrency scenarios.

[0067] In some implementations, the rate limiting module 220 supports multiple rate limiting dimensions, including global rate limiting, IP-level rate limiting, and user-level rate limiting. Global rate limiting performs unified traffic control on all requests, IP-level rate limiting controls traffic based on the request's source IP address, and user-level rate limiting controls traffic based on the user identifier. The rate limiting module 220 constructs a rate limiting key based on the rate limiting rules in the routing configuration and uses the Count-Min Sketch algorithm 222 to determine whether a request exceeds a preset rate limiting threshold. When a request exceeds the rate limiting threshold, the rate limiting module 220 returns a 429 error response and a Retry-After response header, instructing the client to retry after a specified time. When a request does not exceed the rate limiting threshold, the rate limiting module 220 allows the request to continue with subsequent processing steps.

[0068] The plug-in chain processing module 200 includes a canary release module 224, which implements the canary release function, gradually releasing the new version to a subset of users to verify its stability. The canary release module 224 supports two canary release strategies: percentage mode 226 and whitelist mode 228, used to distribute requests according to different business needs.

[0069] Percentage mode 226 distributes requests according to a preset grayscale percentage. In percentage mode 226, the grayscale release module 224 calculates a hash value based on the user identifier and compares the hash value with the grayscale percentage to determine whether the request should be forwarded to the baseline service 230 or the grayscale service 232. Percentage mode 226 is suitable for scenarios where traffic needs to be allocated to grayscale services at a fixed ratio, such as forwarding 5% of traffic to grayscale service 232 for new version verification.

[0070] Whitelist mode 228 distributes requests based on a preset user whitelist. In whitelist mode 228, the canary release module 224 extracts the user identifier from the request and determines whether the user identifier exists in the canary whitelist. If the user identifier exists in the canary whitelist, the canary release module 224 forwards the request to the canary service 232. If the user identifier does not exist in the canary whitelist, the canary release module 224 forwards the request to the baseline service 230. Whitelist mode 228 is suitable for scenarios requiring canary testing with specific users, such as verifying a new version with internal test users or specific customers.

[0071] The canary release module 224 connects to the baseline service 230 and the canary service 232. Baseline service 230 is the currently stable backend service version, while canary service 232 is the new version of the backend service awaiting verification. Based on the judgment result of either the percentage mode 226 or the whitelist mode 228, the canary release module 224 forwards requests to the corresponding backend service. Through the judgment of the canary release strategy, the canary release module 224 achieves a gradual release of the new version, reducing the risk of new version releases.

[0072] In some implementations, the canary deployment module 224 supports dynamic adjustment of the canary deployment percentage. Once the canary service 232 is running stably, the canary deployment percentage is gradually increased via the management console, forwarding more traffic to the canary service 232. When the canary service 232 experiences an anomaly, the canary deployment percentage is reduced to zero or switched back to the baseline service 230 via the management console, achieving a rapid rollback. The dynamic adjustment of the canary deployment percentage is achieved through the configuration synchronization module and the lock-free configuration update module; configuration changes take effect without restarting the service.

[0073] Furthermore, in some implementations, the high-performance API gateway system also includes a visual management module. This module provides a web console interface for managing route configurations, plugin configurations, and canary release configurations. The visual management module uses the React and Ant Design technology stack to build a modern web console interface. React is an open-source JavaScript front-end framework from Facebook used to build user interface components. Ant Design is an open-source enterprise-grade UI design language and React component library from Ant Financial, providing rich interface components and interaction patterns.

[0074] The visual management module provides route management functionality through a web console interface, supporting the creation, editing, deletion, and querying of routes. Route management allows administrators to configure parameters such as route paths, backend service addresses, route types, and route priorities. The visual management module also provides plugin configuration functionality through the web console interface, supporting parameter configuration for plugins such as authentication processing, signature verification, and rate limiting. Plugin configuration allows administrators to enable or disable specific plugins and configure their runtime parameters.

[0075] The visual management module provides canary release configuration functionality through a web console interface, supporting the creation and management of canary versions. This configuration allows administrators to set the canary mode to percentage-based or whitelist-based modes, configure the canary percentage, or add canary users to the whitelist. The visual management module also provides performance monitoring capabilities, displaying gateway performance metrics such as QPS, latency, and error rate, facilitating monitoring of the gateway's operational status.

[0076] The configuration synchronization module uses a MySQL database for persistent configuration storage. When administrators modify routing configurations, plugin configurations, or canary release configurations through the visual management module, the visual management module writes the configuration changes to the MySQL database for persistent storage. The MySQL database stores complete information on all routing configurations, plugin configurations, and canary release configurations, ensuring that configuration data can be restored after the gateway restarts.

[0077] After writing configuration changes to the MySQL database, the visualization management module sends configuration change notifications via a message queue. These notifications include the type and content of the changed configuration, instructing all gateway instances to perform configuration updates. The message queue uses a publish-subscribe model, with the visualization management module acting as the publisher and all gateway instances as subscribers. This asynchronous notification mechanism decouples the visualization management module from the gateway instances, ensuring real-time synchronization of configuration changes across all gateway instances and guaranteeing configuration consistency in multi-instance deployments.

[0078] This invention also discloses an API gateway request processing method based on the Rust language.

[0079] Reference Figure 3 , Figure 3 A flowchart of API gateway request processing method 300 based on the Rust language is shown. Method 300 describes the complete process of an HTTP request from receipt to completion, as well as the mechanism for hot configuration updates.

[0080] like Figure 3As shown, method 300 begins with step 302, in which an HTTP request is received through a gateway core module built on the Pingora asynchronous framework. The Pingora asynchronous framework uses an asynchronous I / O model to handle network requests, enabling concurrent processing of multiple HTTP connections in a single thread, thus improving request throughput. In step 302, the gateway core module listens on a specified port through the Pingora asynchronous framework, receives HTTP requests from clients, and passes the requests to subsequent processing steps.

[0081] Continue to refer to Figure 3 Method 300 proceeds to step 304 after step 302, where a multi-level route caching structure is used to perform route matching for the HTTP request. The multi-level route caching structure sequentially employs a HashMap data structure for static route matching, a prefix tree data structure for prefix route matching, and a radix tree data structure for dynamic route matching. The route matching in step 304 is performed sequentially in the order of static route, prefix route, and dynamic route. When any level of route matching is successful, subsequent levels of route matching are stopped, and the matching result is returned.

[0082] Method 300 proceeds to step 306 after step 304, where it determines whether route matching was successful. Step 306 makes a branch decision based on the route matching result of step 304. If route matching is successful, method 300 follows the yes branch to step 308 to continue executing the plug-in chain processing. If route matching fails, method 300 follows the no branch to step 312 and returns an error response.

[0083] When step 306 determines that the route match is successful, method 300 proceeds to step 308. In step 308, the HTTP request is sequentially processed through a plugin chain, including authentication, signature verification, rate limiting, and canary release. The plugin chain in step 308 executes each processing step in a predefined order: authentication verifies the validity of the authentication token carried in the request; signature verification verifies whether the request content has been tampered with; rate limiting determines whether the request exceeds a preset rate limiting threshold; and canary release determines whether the request should be forwarded to the baseline service or the canary service based on the canary release strategy.

[0084] The plugin chain processing in step 308 employs a fail-fast mechanism. When authentication fails, signature verification fails, or rate limiting triggers rate limiting, the plugin chain directly returns an error response and ceases further processing. This fail-fast mechanism prevents invalid requests from consuming backend service resources, thus protecting resources.

[0085] Method 300 proceeds to step 310 after step 308, where the request processed through the plugin chain is forwarded to the backend service. Step 310 forwards the request to the corresponding backend service instance based on the routing matching result and the canary release processing judgment result. The backend service instance processes the business logic and returns a response result. Step 310 then returns the response result to the client, completing the full request processing flow.

[0086] When step 306 determines that the route match is unsuccessful, method 300 proceeds to step 312, where an error response is returned. Step 312 returns a route mismatch error response to the client, indicating that the requested route path does not exist or is invalid. The error response in step 312 includes an error code and error description information, making it easier for the client to identify the cause of the error.

[0087] Method 300 also includes step 314, in which, upon receiving a configuration change notification, the routing configuration and plugin configuration are updated without restarting the service via the ArcSwap atomic pointer exchange mechanism. The configuration update process in step 314 includes: the configuration synchronization module receiving the configuration change notification through a message queue, parsing the configuration change content, constructing a new routing engine and plugin configuration, and replacing the old configuration with the new configuration via the ArcSwap atomic pointer exchange mechanism. The configuration update operation in step 314 is an atomic operation; the configuration change takes effect in less than 10 microseconds, and the update process does not block ongoing requests, achieving zero-downtime changes.

[0088] In some implementations, after forwarding the request to the backend service in step 310, method 300 records tracing information. The tracing information includes data such as request identifier, request path, backend service address, request start time, request end time, and response status code. The tracing information is used to trace the complete call chain of the request in the distributed system, facilitating problem localization and performance analysis. Method 300 writes the tracing information to a logging system, supporting subsequent log queries and analysis.

[0089] In some implementations, method 300 collects performance metrics during request processing. These performance metrics include data such as request processing time, route matching time, plugin chain processing time, backend service response time, request success rate, and error rate. Method 300 reports the collected performance metrics to a monitoring system, supporting real-time monitoring of the gateway's operational status. The performance metric collection and monitoring functions enable administrators to promptly identify performance bottlenecks and anomalies, ensuring the stable operation of the gateway service.

[0090] Furthermore, refer to Figure 4 , Figure 4A flowchart of authentication processing and signature verification method 400 is shown. Method 400 illustrates the query process of a two-level cache structure and the processing logic of the HMAC-SHA256 signature verification algorithm.

[0091] Method 400 begins with step 402, in which an authentication token is extracted from the HTTP request. Step 402 retrieves the authentication token information from the request header. The authentication token is used to identify the request initiator. The authentication token format may include JWT (JSON Web Token) or other custom token formats. Step 402 parses the Authorization field or other specified fields in the request header to obtain the authentication token.

[0092] Method 400 proceeds to step 404 after step 402, where the local LRU cache is queried. Step 404 uses the authentication token extracted in step 402 as the cache key to retrieve the corresponding user information from the local LRU cache. The local LRU cache is the first-level cache in a two-level cache structure, employing an LRU eviction policy to manage cache entries, and its capacity is configured to 200,000 tokens. The local LRU cache query operation in step 404 is a memory access operation with a latency in the nanosecond range.

[0093] Method 400 proceeds to step 406 after step 404, where it determines whether the local LRU cache has been hit. Step 406 makes a branch decision based on the cache query result of step 404. If the local LRU cache is hit, it means that the user information corresponding to the authentication token exists in the local LRU cache, and method 400 proceeds to step 410 for signature verification processing along the yes branch. If the local LRU cache is not hit, it means that the user information corresponding to the authentication token does not exist in the local LRU cache, and method 400 proceeds to step 408 to query the Redis cache along the no branch.

[0094] When step 406 determines that the local LRU cache has been missed, method 400 proceeds to step 408, where the Redis cache is queried. Step 408 uses the authentication token as the cache key to retrieve the corresponding user information from the Redis cache. The Redis cache is the second-level cache in a two-level cache structure, serving as a backup cache when the local LRU cache is missed. The Redis cache query operation in step 408 is a network access operation with a latency in milliseconds. When the Redis cache is hit, step 408 retrieves the user information corresponding to the authentication token and populates it back into the local LRU cache so that subsequent requests with the same authentication token can directly retrieve information from the local LRU cache. After step 408 is completed, method 400 proceeds to step 410 for signature verification processing.

[0095] Method 400 proceeds to step 410 after determining a local LRU cache hit in step 406 or after completing the Redis cache query in step 408. In step 410, the HMAC-SHA256 algorithm is used for signature verification. HMAC-SHA256 is a message authentication code based on the SHA-256 hash algorithm, used to verify the request signature and prevent request tampering and replay attacks. Step 410 extracts the signature information and data to be signed from the request, calculates the signature value using the pre-shared key and the HMAC-SHA256 algorithm, and compares the calculated signature value with the signature information carried in the request. The HMAC-SHA256 algorithm uses the key in the hash calculation, making it impossible for attackers to forge a valid signature without knowing the key, thus ensuring the integrity and authenticity of the request content.

[0096] Method 400 proceeds to step 412 after step 410, where it determines whether the signature verification passed. Step 412 performs a branching decision based on the signature comparison result from step 410. If the calculated signature value matches the signature information carried in the request, the signature verification passes, and method 400 proceeds to step 414 along the yes branch to continue subsequent processing. If the calculated signature value does not match the signature information carried in the request, the signature verification fails, and method 400 proceeds to step 416 along the no branch to return a signature verification failure response.

[0097] When step 412 determines that the signature verification is successful, method 400 proceeds to step 414, where further processing continues. Step 414 injects the user information obtained during authentication into the request header for use in subsequent processing stages. This user information includes user identifier, user role, and user permissions. After being injected into the request header, subsequent rate limiting and canary release processes can perform user-level rate limiting and canary release decisions based on this user information. After step 414 completes the user information injection, the request continues to the subsequent processing stages of the plugin chain.

[0098] When step 412 determines that signature verification fails, method 400 proceeds to step 416, where a signature verification failure response is returned. Step 416 returns an error response to the client indicating that the requested signature is invalid or the request content has been tampered with. The error response in step 416 includes an error code and error description, allowing the client to identify the cause of the error and take appropriate action. After step 416 returns the error response, method 400 terminates, and the request is no longer processed further or forwarded to the backend service.

[0099] In some implementations, the signature verification process in step 410 also includes timestamp verification to prevent replay attacks. The signature information carried in the request includes a timestamp indicating that the request was initiated. During signature verification, step 410 checks whether the difference between the timestamp and the current time is within an allowed time window. If the time difference exceeds the allowed time window, step 410 determines the request to be a replay request, and the signature verification fails. The timestamp verification mechanism prevents attackers from resending intercepted valid requests after the time window has expired, thus preventing replay attacks.

[0100] Furthermore, refer to Figure 5 , Figure 5 A flowchart of method 500 for rate limiting and canary release processing is shown. Method 500 describes the process of rate limiting using the Count-Min Sketch algorithm for traffic statistics and rate limiting determination, and the process of canary release processing supporting percentage mode and whitelist mode to determine whether to forward requests to the canary service or the baseline service based on user identifier.

[0101] like Figure 5 As shown, method 500 begins at step 502, where the Count-Min Sketch algorithm is used for traffic statistics. The Count-Min Sketch algorithm is a probabilistic data structure that uses multiple independent hash functions and a counter array to achieve approximate counting. Step 502 constructs a rate-limiting key based on the rate-limiting rules in the routing configuration. The method of constructing the rate-limiting key depends on the rate-limiting level configuration. Rate-limiting processing supports multiple rate-limiting levels, including global rate-limiting, IP-level rate-limiting, and user-level rate-limiting. When configured for global rate-limiting, step 502 uses the route identifier as the rate-limiting key to perform unified traffic statistics for all requests on that route. When configured for IP-level rate-limiting, step 502 uses a combination of the route identifier and the request source IP address as the rate-limiting key to perform traffic statistics for requests from the same IP address. When configured for user-level rate-limiting, step 502 uses a combination of the route identifier and the user identifier as the rate-limiting key to perform traffic statistics for requests from the same user. Step 502 uses multiple hash functions of the Count-Min Sketch algorithm to map the rate limiting key to different positions in the counter array, and increments the counters at the mapped positions to complete the traffic statistics.

[0102] Method 500 proceeds to step 504 after step 502, where it determines whether the request exceeds the rate limiting threshold. Step 504 uses the Count-Min Sketch algorithm to read counter values ​​from multiple hash function mapping locations and takes the minimum value as the estimated request frequency within the current time window. Step 504 compares the estimated request frequency with the preset rate limiting threshold in the routing configuration to determine if the request exceeds the threshold. The rate limiting threshold configuration includes a time window and a maximum number of requests, for example, a maximum of 1000 requests per minute. When the estimated request frequency exceeds the rate limiting threshold, Method 500 proceeds along the yes branch to step 506 and returns an error response. When the estimated request frequency does not exceed the rate limiting threshold, Method 500 proceeds along the no branch to step 508 to continue the canary release process.

[0103] When step 504 determines that the request exceeds the rate limiting threshold, method 500 proceeds to step 506, where an error response is returned. Rate limiting returns a 429 error response and a Retry-After response header when the limit is exceeded. Step 506 returns an HTTP 429 status code to the client, indicating too many requests. Step 506 sets the Retry-After field in the response header, instructing the client to retry after a specified time. The value of the Retry-After field is calculated based on the time window configuration of the rate limiting rule; for example, when the rate limiting rule is a maximum of 1000 requests per minute, the value of the Retry-After field is set to the remaining time window in seconds. After step 506 returns the error response, method 500 terminates, and the request is no longer processed further or forwarded to the backend service.

[0104] When step 504 determines that the request does not exceed the rate limiting threshold, method 500 proceeds to step 508, where the user identifier is obtained for canary deployment determination. Step 508 extracts the user identifier information from the request header, which is injected into the request header by the authentication process. The user identifier is used in the canary deployment process to determine whether the request should be forwarded to the baseline service or the canary service. After obtaining the user identifier in step 508, method 500 proceeds to step 510 for canary mode determination.

[0105] Method 500 proceeds to step 510 after step 508, where the canary deployment mode is determined. Step 510 determines the canary deployment mode used by the current route based on the canary deployment configuration in the routing configuration. Canary deployment processing supports two canary deployment strategies: percentage mode and whitelist mode. When the canary deployment mode is configured as percentage mode, method 500 proceeds along the yes branch to step 512 to determine whether to forward to the canary service based on the percentage. When the canary deployment mode is configured as whitelist mode, method 500 proceeds along the no branch to step 512 to determine whether to forward to the canary service based on the whitelist.

[0106] Method 500 proceeds to step 512 after step 510. In step 512, it determines whether to forward the request to the grayscale service based on the percentage or a whitelist. When the grayscale mode is percentage mode, step 512 calculates a hash value based on the user identifier and takes the hash value modulo 100 to obtain a value between 0 and 99. Step 512 compares this value with the grayscale percentage. If the value is less than the grayscale percentage, it is determined that the request should be forwarded to the grayscale service, and Method 500 proceeds to step 514 along the yes branch. If the value is greater than or equal to the grayscale percentage, it is determined that the request should be forwarded to the baseline service, and Method 500 proceeds to step 516 along the no branch. When the grayscale mode is whitelist mode, step 512 determines whether the user identifier exists in the grayscale whitelist. If the user identifier exists in the grayscale whitelist, it is determined that the request should be forwarded to the grayscale service, and Method 500 proceeds to step 514 along the yes branch. If the user identifier does not exist in the grayscale whitelist, it is determined that the request should be forwarded to the baseline service, and Method 500 proceeds to step 516 along the no branch.

[0107] When step 512 determines that the request should be forwarded to the canary service, method 500 proceeds to step 514, where the request is forwarded to the canary service. The canary service is a new version of the backend service to be verified. Step 514 forwards the request to the canary service instance according to the canary service address in the routing configuration. The canary service instance processes the business logic and returns a response result, which is then returned to the client in step 514.

[0108] When step 512 determines that the request should be forwarded to the baseline service, method 500 proceeds to step 516, where the request is forwarded to the baseline service. The baseline service is the currently stable backend service version. Step 516 forwards the request to the baseline service instance based on the baseline service address in the routing configuration. The baseline service instance processes the business logic and returns a response result, which is then returned to the client in step 516.

[0109] In some implementations, canary releases support gradually increasing the canary percentage or switching back to the baseline version. Once the canary service is running stably, administrators use a visual management module to gradually increase the canary percentage, forwarding more traffic to the canary service for verification. The canary percentage increase is gradual, for example, from 5% to 10%, 20%, 50%, and finally 100%. When the canary percentage reaches 100%, all requests are forwarded to the canary service, completing the full release of the new version. If the canary service experiences an anomaly, administrators use the visual management module to reduce the canary percentage to zero or switch back to the baseline version for rapid rollback. Adjustments to the canary percentage are implemented through a configuration synchronization module and a lock-free configuration update module. Configuration changes are notified to all gateway instances via a message queue, and gateway instances update the canary configuration using the ArcSwap atomic pointer exchange mechanism. Configuration changes take effect without requiring a service restart.

[0110] This invention also discloses a readable storage medium.

[0111] A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any of the above embodiments. The computer-readable storage medium may include any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. The computer program includes computer program code. The computer program code may be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer-readable storage medium may include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0112] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0113] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a system including a processing module or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0114] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-performance API gateway system based on the Rust language, characterized in that, include: The gateway core module, built on the Pingora asynchronous framework, is used to receive and process HTTP requests. The routing engine module includes a multi-level routing cache structure, which includes: A HashMap data structure used for static route matching; A prefix tree data structure for prefix route matching; and A radix tree data structure used for dynamic route matching; The lock-free configuration update module adopts the ArcSwap atomic pointer exchange mechanism to achieve hot updates of routing configuration and plugin configuration without restarting the service; A plug-in chain processing module, which sequentially performs authentication, signature verification, rate limiting, and canary release processing on requests that have undergone route matching; and The configuration synchronization module receives configuration change notifications through a message queue and triggers the lock-free configuration update module to perform configuration updates.

2. The system according to claim 1, characterized in that, The plug-in chain processing module includes a dual-track plug-in architecture, which includes: System plugins implemented using enumeration structures; and Custom plugins implemented using trait interfaces.

3. The system according to claim 2, characterized in that, The system plugins and the custom plugins are registered through an automatic registration mechanism, which is implemented based on the linkme library.

4. The system according to claim 1, characterized in that, The authentication process of the plug-in chain processing module adopts a two-level caching structure, which includes: A local LRU cache for storing authentication tokens; and A Redis cache used to store authentication tokens.

5. The system according to claim 1, characterized in that, The rate limiting process of the plug-in chain processing module uses the Count-Min Sketch algorithm for traffic statistics and rate limiting judgment.

6. The system according to claim 1, characterized in that, The canary release processing module of the plug-in chain supports percentage mode and whitelist mode, which are used to forward requests to the baseline service or the canary service.

7. The system according to claim 1, characterized in that, It also includes a visual management module, which provides a web console interface to manage routing configuration, plugin configuration, and canary release configuration, and writes configuration changes to the database and sends configuration change notifications through the message queue.

8. A method for processing API gateway requests based on the Rust language, characterized in that, include: HTTP requests are received through the gateway core module built on the Pingora asynchronous framework; The HTTP request is routed using a multi-level route caching structure, wherein the multi-level route caching structure sequentially uses a HashMap data structure for static route matching, a prefix tree data structure for prefix route matching, and a radix tree data structure for dynamic route matching. When the route is successfully matched, the HTTP request is sequentially processed through the plugin chain, including authentication, signature verification, rate limiting, and canary release. The requests processed through the plug-in chain are forwarded to the backend service; as well as When a configuration change notification is received, the routing configuration and plugin configuration are updated without restarting the service through the ArcSwap atomic pointer exchange mechanism.

9. The method according to claim 8, characterized in that, The authentication process employs a two-level caching structure, which includes: A local LRU cache for storing authentication tokens; and A Redis cache used to store authentication tokens; The authentication process first queries the local LRU cache, and if no match is found, queries the Redis cache.

10. The method according to claim 9, characterized in that, The signature verification process uses the HMAC-SHA256 algorithm to verify the signature of the request in order to prevent the request from being tampered with.

11. The method according to claim 8, characterized in that, The rate limiting process uses the Count-Min Sketch algorithm for traffic statistics and rate limiting judgment. When the number of requests exceeds the rate limiting threshold, an error response is returned.

12. The method according to claim 8, characterized in that, The canary release process supports percentage mode and whitelist mode, and determines whether to forward the request to the canary service or the baseline service based on the user identifier.

13. A readable storage medium, characterized in that, The readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 8-12.

Images