Service-oriented grid redundancy computation inhibiting type routing cache cooperation method and system

Abstract

The application discloses a service grid-oriented redundant computation inhibition type routing cache cooperation method and system. The method comprises the following steps: constructing a control plane perception and preheating module in a service grid, aggregating cluster load indicators and call dependency topology, and preheating the cache based on historical high-frequency data; a double-time-scale routing decision is executed at the calling end, a target instance is located through request feature consistency hash, and routing selection is performed in combination with slow-scale virtual node weight adjustment and fast-scale safety threshold checking; a two-level semantic cache is executed at the receiving end, and the calculation results are multiplexed by using a cuckoo filter pre-check and hash mapping table matching; the service state change is listened to, and cross-service cache invalidation and consistency management are realized based on an inverted index. The application is used for relieving the cache fragmentation and transient hotspot overload problems caused by traditional stateless scheduling, reducing repeated calculation, reducing long tail delay risk, and improving the throughput efficiency and resource utilization of the micro-service cluster.

Description

Technical Field

[0001] This invention relates to the field of cloud computing and microservice architecture technology, and in particular to a redundancy-suppressing routing cache coordination method and system for service mesh. Background Technology

[0002] With the development of cloud computing technology, microservice architecture has become the mainstream paradigm for building large-scale distributed systems. To reduce the complexity of microservice governance, service meshes (such as Istio and Envoy) are widely adopted as a standard cloud-native traffic infrastructure. However, in service meshes, existing default scheduling mechanisms typically employ stateless load balancing algorithms, such as round-robin or minimum request count. While this approach ensures even distribution of requests, it randomly distributes a large number of identical and deterministic requests, such as fan-out calls or client retries, across different service instances. This severely disrupts the locality of execution, causing the same computational logic to be executed repeatedly across multiple replicas. These massive amounts of redundant computation not only waste significant computing resources and limit the throughput of the entire cluster, but also directly exacerbate system queuing latency. Addressing the issue of numerous duplicate requests in microservices, fully leveraging application-layer semantics for cache reuse and affinity scheduling is crucial for improving the operational efficiency of microservice clusters and reducing long-tail latency.

[0003] However, relying solely on static consistent hashing routing is no longer sufficient to solve the load balancing problem under high concurrency in microservices. While traditional Layer 4 consistent hashing routing technology can maintain a certain level of forwarding stability, it lacks awareness of Layer 7 application semantics and cannot identify reusable computation results within request content. This can lead to severe instance load imbalance when facing skewed hotspot traffic. Furthermore, existing Layer 7 application routing solutions often require deeply embedding and hard-coding routing control logic into the business application code, representing a highly intrusive design that fundamentally violates the core principles of transparent and non-intrusive governance in service meshes.

[0004] While caching systems based on microservice semantics have been proposed, these solutions lack global coordination with underlying traffic scheduling. Simply relying on caching leads to random request distribution and severe cache fragmentation. Furthermore, some distributed caching solutions rely on centralized dependency graphs for cache invalidation control, facing a severe risk of state explosion when the service topology dynamically scales; others require passing read / write set metadata in the data plane request header, increasing latency overhead on critical paths at the microsecond level. Moreover, these empirical heuristic designs lack rigorous mathematical models supporting queuing theory, making them highly susceptible to severe queuing backlogs and long-tail latency (P99 latency) violations when facing bursty traffic, failing to meet the stability requirements of large-scale production environments.

[0005] In high-concurrency traffic management for microservices, long-term cache affinity and short-term instantaneous load balancing are inherently contradictory. By collecting instance resource utilization and queuing status in the sidecar proxy, various characteristics reflecting the microservice's processing capacity can be extracted, such as instantaneous load and smoothed load. These characteristics can reflect the node's true throughput capacity and potential congestion risks. There is an urgent need for a dual-timescale scheduling mechanism that can maintain cache locality in a macro-level trend while also instantaneously deflecting overloaded requests at the micro-level.

[0006] The transparency, non-intrusiveness, and sidecar proxy architecture of service mesh naturally provide the foundation for collaborative optimization of routing and caching. After obtaining the key characteristics of business requests at the proxy layer, by maintaining an efficient and lightweight filtering and hash mapping structure, it can establish the uniqueness of reusable requests and quickly intercept them. Furthermore, by employing decentralized event tracing technology, it can jointly analyze inter-service call dependencies and state change operations, generating atomic cache invalidation strategies to meet the system's strong consistency requirements. This technology can also be used for time-series analysis of historical telemetry data from microservice clusters. Through an offline preheating mechanism, high-frequency cached data can be injected in advance during instance scaling, effectively eliminating the performance cliff of cold starts and thus comprehensively protecting the efficient and stable development of the microservice ecosystem. Summary of the Invention

[0007] This invention addresses the problems in existing service meshes where stateless load balancing mechanisms randomly distribute identical requests, disrupt cache locality, and lead to massive redundant computation, cache fragmentation, and uncontrollable long-tail latency under burst traffic. It proposes a redundant computation suppression-type routing and caching collaborative method and system for service meshes. Unlike caching schemes that only set filters or hierarchical caching at the microservice interface layer, this invention collaboratively executes call-end routing, receiver-end cache reuse, and cross-service cache invalidation at the service mesh proxy layer: the call-end sidecar proxy performs dual-time-scale consistent hash routing based on request characteristics, prioritizing forwarding identical or equivalent requests to the same target instance; the receiver-end sidecar proxy generates cache keys based on corresponding request characteristics and reuses cached responses through a cuckoo filter and hash mapping table; when business state changes, the sidecar proxy clears affected cross-service cache entries based on call dependency topology and local inverted index. This method alleviates cache fragmentation, redundant computation, and instantaneous hotspot overload problems, improving the throughput efficiency and resource utilization of microservice clusters.

[0008] The technical solution to achieve the purpose of this invention is:

[0009] A redundant computation suppression-type routing cache coordination method for service mesh is characterized by coordinated execution among the control plane, the calling end sidecar agent, and the receiving end sidecar agent of the service mesh, so that calling requests with the same deterministic request characteristics maintain semantic association during routing selection, cache lookup, and cache invalidation, including the following steps:

[0010] Step S1, Construct a service mesh control plane awareness and preheating module: Construct a control plane awareness and preheating module in the service mesh, periodically collect resource load indicators of each instance in the microservice cluster, construct the service call dependency topology based on distributed tracing telemetry data, and perform offline cache preheating based on historical high-frequency access data during the instance expansion phase;

[0011] Step S2: Perform consistent hash routing decision on the calling end with dual time scales: The sidecar proxy on the calling end intercepts the network request sent by the caller, extracts the request features that determine the business response result and generates a routing hash key, locates the main target instance on the consistent hash ring according to the routing hash key, and prioritizes routing call requests with the same request features to the same target instance; adjusts the weight of the virtual node of the hash ring on the slow time scale according to the load index collected by the control plane, and determines the request forwarding target on the fast time scale based on the instantaneous queuing status and security utilization threshold of the main target instance;

[0012] Step S3: Perform two-level cache interception and result reuse at the receiving end: The receiving end sidecar agent intercepts the incoming request, generates a cache key and hash fingerprint based on the request characteristics corresponding to the route hash key, performs fingerprint pre-checking first through the Cuckoo filter, and then performs exact cache key matching through the hash mapping table; when the cache is hit, the receiving end sidecar agent directly returns the corresponding cache response to reuse the business calculation results on the target instance.

[0013] Step S4: Perform decentralized cache consistency management based on inverted index: The receiving end sidecar proxy listens for application layer state change operations, cleans up the corresponding local cache, and pushes the compressed signature of the change event to the upstream proxy with call dependency based on the call dependency topology; the upstream proxy builds a local inverted index based on the downstream dependency resources recorded in the cache response generation process, and locates and clears the affected cache entries through the inverted index.

[0014] Furthermore, the resource load metrics in step S1 include CPU utilization, memory utilization, and queuing depth. The control plane perception and preheating module aggregates the resource load metrics of each service instance according to a preset period, and generates the inter-service call dependency topology based on the call path, call frequency, and upstream and downstream service identifiers in the distributed tracing telemetry data.

[0015] Furthermore, the offline cache preheating in step S1 includes: analyzing the seventh layer application layer telemetry logs in the control plane, extracting historical high-frequency requests and their corresponding business response results, assigning sampling weights to request records using a time-weighted sampling algorithm, injecting the selected response data and their corresponding cache keys and hash fingerprints into the hash mapping table and cuckoo filter of the newly launched instance, and configuring the initial survival time for the preheating entries.

[0016] Furthermore, the request features in step S2 include the caller service identifier, the request resource identifier, and deterministic parameters that determine the business response result. The request resource identifier includes the HTTP URI for HTTP requests and the gRPC method name, RPC service name, and operation name for gRPC and RPC requests. The deterministic parameters include the request method, path parameters, query parameters, business fields in the request body that participate in the response result calculation, the request body summary, and request header fields related to the business response result. After the calling end-side vehicle proxy normalizes the request features, it generates a route hash key and locates the primary target instance on the consistent hash ring based on the route hash key. The deterministic parameters are also used to generate a cache key in step S3, so that route location and cache matching are performed based on the same request semantics.

[0017] Furthermore, the process of calling the edge vehicle proxy to normalize the request features includes: selecting fields for calculation according to a preset field whitelist, and removing the tracing identifier, request timestamp, random number, temporary session identifier, and retry count identifier; sorting query parameters and path parameters by field name; performing stable serialization of JSON fields, Protobuf fields, and form fields in the request body according to the field path; filling default fields with preset default values; uniformly processing case sensitivity, encoding format, and semantically indistinguishable delimiters; and performing hash operations on the normalized field sequence to generate a routing hash key.

[0018] Furthermore, in the slow time scale of step S2, the control plane uses an exponentially weighted moving average algorithm to calculate the smoothed load of each service instance, compares the smoothed load of each service instance with the cluster average load, and adjusts the number of virtual nodes of the service instance on the consistent hash ring according to the reciprocal of the deviation ratio when the deviation ratio exceeds the preset tolerance band.

[0019] Furthermore, in the fast timescale of step S2, the control plane collects historical request arrival rate, average service time, service time variance, queue length, historical network timeout time, and target latency threshold for each service instance within a preset statistical window. Let the average service time of the service instance be S, and the service rate be... The target latency threshold is Reserved network transmission and proxy processing overhead. The allowed queuing time is:

[0020]

[0021] when At that time, the control surface solves for the condition that the predicted waiting time does not exceed the relationship between queuing time and utilization rate. The maximum utilization rate is determined and used as the safe utilization rate threshold. If the M / M / 1 approximation model is used, the safe utilization rate threshold can be calculated as follows:

[0022]

[0023] in, Indicates the safety utilization threshold, and will The limit is set between 0.3 and 0.85; Let the current queue length of the instance be q, the concurrent processing capacity of the instance be c, the queue correction coefficient be α, and the corrected security utilization threshold be:

[0024]

[0025] Before forwarding the current request, the edge vehicle proxy obtains the instantaneous queuing utilization rate of the primary target instance. When the instantaneous queuing utilization rate is greater than... When this happens, the request routing target is changed to the next healthy instance in the clockwise direction of the consistent hash ring. The healthy instance is a service instance that has passed the service mesh health check and has not been circuit-broken or removed.

[0026] Furthermore, in step S3, when the receiving end-side vehicle agent generates the cache key, it adopts the same request feature selection rules and normalization processing rules as the calling end-side vehicle agent in generating the route hash key. When the cache key is not hit and the business container returns a response, the receiving end-side vehicle agent verifies the response status code and cache control header, extracts deterministic parameters from the response that meets the preset cache rules to generate the cache key, writes the cache key, response value and time to live into the hash mapping table, and performs a hash operation on the cache key to generate a fixed-length fingerprint and inserts it into the cuckoo filter after successful writing.

[0027] Furthermore, the Cuckoo Filter uses two independent hash functions to determine the candidate bucket position, performs fingerprint shifting when a fingerprint collision occurs, and deletes the corresponding fixed-length fingerprint when a cache entry expires or is deleted; Step S4 includes: recording the Uniform Resource Identifier (URI) of the downstream dependent resource when the upstream service proxy initiates a sub-request to the downstream service; performing hash compression on the URI to generate a fixed-length feature signature when saving the final cache response of the upstream service and establishing a cache key, and constructing an inverted index with the fixed-length feature signature as the key and the upstream cache key as the value; when the downstream service performs a state change write operation, the downstream service proxy generates an invalidation event message containing the hash signature of the write operation resource identifier and pushes it to the corresponding upstream service proxy; after receiving the invalidation event message, the upstream service proxy searches for the affected cache key in the inverted index and simultaneously removes the affected cache entry in the hash mapping table and the Cuckoo Filter.

[0028] A service mesh-oriented redundant computation suppression routing cache coordination system, characterized in that it is used in the service mesh-oriented redundant computation suppression routing cache coordination method as described in any one of claims 1 to 9, wherein the service mesh-oriented redundant computation suppression routing cache coordination system includes a control plane awareness and preheating module, a caller-side dual-timescale routing module, a receiver-side two-level cache management module, and a decentralized cache consistency module; the control plane awareness and preheating module is deployed on the service mesh control node, used to aggregate service instance load metrics, maintain inter-service call dependency topology, and provide global state information required for route adjustment and cache invalidation to the caller-side dual-timescale routing module and the decentralized cache consistency module; the caller-side dual-timescale routing module is deployed on the sidecar proxy of the microservice caller. The system is used to generate route hash keys based on request characteristics, and through consistent hash routing, hash ring virtual node weight adjustment, and instantaneous load bias checking, prioritizes forwarding call requests with the same deterministic request characteristics to target instances that can reuse corresponding cached results; the receiving end two-level cache management module is deployed on the sidecar proxy of the microservice receiver, generates cache keys and hash fingerprints based on the request characteristics corresponding to the route hash keys, and returns cached responses through cuckoo filter pre-checking and hash mapping table matching; the decentralized cache consistency module is deployed on each sidecar proxy, used to record downstream dependent resources during the cache response generation process, construct an inverted index with the compressed signature of dependent resources as the key and the upstream cache key as the value, and locate and remove affected cache entries when a state change event arrives.

[0029] Compared with the prior art, the present invention, employing the above technical solution, has the following beneficial effects:

[0030] (1) The redundant computation suppression routing cache collaborative method and system for service mesh proposed in this invention can completely eliminate massive redundant computations and significantly reduce resource overhead. This invention ensures that the same requests converge to the same instance through load-aware consistent hash routing, and with the two-level semantic cache in the sidecar proxy, it avoids multiple calculations of duplicate business logic from the root. It effectively eliminates up to 75% of redundant computation tasks and reduces the memory overhead of microservice runtime by more than half.

[0031] (2) The redundant computation suppression routing cache coordination method and system proposed in this invention for service mesh can strictly guarantee the long-tail latency of microservices and the system has strong robustness. In response to the common high-concurrency instantaneous burst of request traffic in microservice environments, this invention creatively introduces a fast load deflection mechanism (fast time scale) based on theoretical safety thresholds, which can instantly divert requests accumulated on hot nodes, greatly alleviate queuing backlog, and reduce P99 long-tail latency by more than 80%.

[0032] (3) The redundant computation suppression routing cache collaborative method and system proposed in this invention for service mesh can achieve a non-intrusive and transparent governance mechanism with extremely low performance loss. The system utilizes the sidecar architecture of the service mesh, and all route deflection, cache reading and invalidation strategies are intercepted at the proxy layer without modifying any microservice business code. At the same time, the innovative cuckoo filter pre-detection mechanism reduces the overhead of judging most of the non-hit requests to O(1), completely solving the problem of traditional caching systems occupying network critical path computing resources.

[0033] (4) The redundant computation suppression routing cache coordination method and system proposed in this invention can solve the cold start performance cliff problem caused by the elastic scaling of microservices. By using time-weighted sampling and offline preheating injection in the control plane, newly launched service instances can have an initial cache hit rate of over 90% at startup, which perfectly matches the dynamic characteristics of frequent instance changes in the cloud-native environment. Attached Figure Description

[0034] Figure 1 The main flowchart of the redundancy-suppressing routing cache coordination method and system for service mesh proposed in this invention is shown below.

[0035] Figure 2 This is the overall architecture diagram of the redundant computation suppression routing cache coordination method and system for service mesh proposed in this invention;

[0036] Figure 3This is a schematic diagram of the dual-time-scale load-aware consistent hash routing (LACHR) and hash ring dynamic adjustment mechanism at the calling end in an embodiment of the service mesh-oriented redundant computation suppression routing cache coordination method and system proposed in this invention.

[0037] Figure 4 This is a lifecycle management logic diagram of the two-level semantic cache (cuckoo filter and hash mapping table) at the receiving end in the embodiment of the service mesh-oriented redundant computation suppression routing cache collaborative method and system proposed in this invention;

[0038] Figure 5 This is a bar chart comparing the end-to-end latency of the proposed service mesh-oriented redundant computation suppression routing cache coordination method and system with the default service mesh mechanism under different concurrent connection numbers in Implementation 1.

[0039] Figure 6 This is a comparison chart of the average latency and long-tail latency performance of different routing and caching collaboration strategies under multiple target request rates in Embodiment 2 of the service mesh-oriented redundant computation suppression routing and caching collaboration method and system proposed in this invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] A redundancy-suppressing routing cache coordination method for service mesh, the process of which is as follows: Figure 1 As shown, the feature is that it includes the following steps:

[0042] Step S1, Construct a service mesh control plane awareness and preheating module: Construct a control plane awareness and preheating module in the service mesh, periodically collect resource load indicators of each instance in the microservice cluster, construct the service call dependency topology based on distributed tracing telemetry data, and perform offline cache preheating based on historical high-frequency access data during the instance expansion phase;

[0043] Step S2, execute dual-time-scale routing decision on the calling end, such as... Figure 2 As shown in the call-side proxy module: the call-side proxy intercepts the network requests sent by the caller, extracts the request features and performs consistent hash calculation to locate the main target instance, adjusts the weight of the hash ring virtual node on the slow time scale according to the load indicators collected by the control plane, and determines the request forwarding target on the fast time scale based on the instantaneous queuing status and security utilization threshold of the main target instance.

[0044] Step S3: Execute two-level cache interception and result reuse at the receiving end: The receiving end sidecar proxy intercepts the incoming request, extracts the key network semantics of the request to generate cache keys and hash fingerprints, first performs fingerprint pre-checking through the cuckoo filter, and then performs exact cache key matching through the hash mapping table. When a match is found, the corresponding cache response is returned.

[0045] Step S4: Decentralized cache consistency management based on inverted index: The receiving end sidecar proxy listens for application layer state change operations, cleans up the corresponding local cache, and pushes the compressed signature of the change event to the upstream proxy with call dependency topology, so that the upstream proxy can locate and clear the affected cache entries through the local inverted index.

[0046] Furthermore, the resource load metrics in step S1 include CPU utilization, memory utilization, and queuing depth. The control plane perception and preheating module aggregates the resource load metrics of each service instance according to a preset period, and generates an inter-service call dependency topology based on the call path, call frequency, and upstream and downstream service identifiers in the distributed tracing telemetry data. Through this setting, the control plane can obtain the running status of service instances and the inter-service dependencies, providing basic data support for subsequent route adjustments, cache preheating, and cache invalidation.

[0047] Furthermore, the offline cache preheating in step S1 includes: analyzing the Layer 7 application layer telemetry logs from the control plane, extracting historical high-frequency requests and their corresponding business response results, assigning sampling weights to request records using a time-weighted sampling algorithm, injecting the selected response data into the hash mapping table and cuckoo filter of the newly launched instance, and configuring an initial lifespan time for the preheating entries. Through this setting, the newly launched instance can obtain some high-frequency cache data in the early stages of expansion, reducing cache misses and redundant calculations during the instance's cold start phase.

[0048] Further, the request features in step S2 include the caller service identifier, the request resource identifier, and deterministic parameters that determine the business response result. The calling edge vehicle agent normalizes the request features to generate a routing hash key and locates the primary target instance on a consistent hash ring based on the routing hash key. The request resource identifier may include at least one of HTTP URI, gRPC method name, RPC service name, and operation name; the deterministic parameters may include at least one of request method, path parameters, query parameters, business fields in the request body that participate in the response result calculation, request body summary, and request header fields related to the business response result. For non-deterministic fields that do not affect the business response result, including the tracing identifier, request timestamp, random number, temporary session identifier, and retry count identifier, the calling edge vehicle agent does not participate in the calculation when generating the routing hash key and cache key. After extracting the request features, the calling edge vehicle agent performs normalization processing on the request features. The normalization process includes: selecting fields for calculation according to a preset field whitelist; sorting query parameters and path parameters by field name; performing stable serialization of JSON, Protobuf, or form fields in the request body according to field paths; filling default values ​​into default fields; uniformly processing case sensitivity, encoding format, and semantically indistinguishable delimiters; and performing hash operations on the normalized field sequence to generate a routing hash key. Through the above processing, requests with the same or equivalent business logic can generate the same routing hash key, thus preferentially routing to the same target instance, and generating a corresponding cache key on the receiving end to reuse the calculation result. The caller-side vehicle proxy performs normalization processing on the request characteristics including: selecting fields for calculation according to a preset field whitelist and removing non-deterministic fields that do not affect the business response result; performing stable serialization of query parameters, path parameters, and request body fields in a preset order; and performing hash operations on the normalized field sequence to generate a routing hash key. The dual-timescale routing bias mechanism in step S2 is as follows: Figure 3 As shown;

[0049] Furthermore, in the slow timescale of step S2, the control plane uses an exponentially weighted moving average algorithm to calculate the smoothed load of each service instance. The smoothed load of each service instance is compared with the cluster average load. When the deviation exceeds a preset tolerance band, the number of virtual nodes for that service instance on the consistent hash ring is adjusted according to the reciprocal of the deviation ratio. Through this setting, the system can adjust the request proportions handled by different service instances based on long-term load trends, alleviating the long-term load imbalance problem caused by static hash allocation.

[0050] Furthermore, in the fast timescale of step S2, the edge vehicle agent is invoked to obtain a security utilization threshold generated from the control plane based on historical network timeout time and queuing model. This security utilization threshold is generated by the control plane based on at least one parameter among historical network timeout time, target latency threshold, average instance service time, arrival rate, and queue length. Before forwarding the current request, a non-blocking queuing status check is performed on the primary target instance. When the instantaneous queuing utilization of the primary target instance exceeds the security utilization threshold, the request routing target is changed to the next healthy instance in the clockwise direction of the consistent hash ring. Through this setting, the system can deflect requests when hotspot instances experience short-term queue backlogs, reducing the impact of sudden traffic surges on long-tail latency.

[0051] In this embodiment, the control plane periodically generates a security utilization threshold for each service instance. Within a preset statistical window, the control plane collects historical request arrival rate, average service time, service time variance, queue length, historical network timeout time, and target latency threshold for each service instance. The target latency threshold can be determined by a preset proportion of the service-level target SLO, interface timeout time, or historical timeout time. For example, when the interface timeout time is... At that time, the target delay threshold It can be set to 60% to 90%.

[0052] As an optional implementation, the control plane uses a single-server queuing model to estimate the security utilization threshold. Let the average service time of a service instance be S, and the service rate be... The target latency threshold is Reserved network transmission and proxy processing overhead. The allowed queuing time is .when At that time, the control surface solves for the condition that the predicted waiting time does not exceed the relationship between queuing time and utilization rate. The maximum utilization rate is determined and used as the safe utilization rate threshold. If the M / M / 1 approximation model is used, the safe utilization rate threshold can be calculated as follows:

[0053]

[0054] in, This represents the safety utilization threshold. To avoid queue backlogs caused by excessively high thresholds, the control plane can also... It is limited to a preset upper and lower bound, for example, not lower than 0.3 and not higher than 0.85.

[0055] Furthermore, the control plane can adjust the safety utilization threshold based on the instance's current queue length. Let the instance's current queue length be q, the instance's concurrent processing capacity be c, and the queue adjustment coefficient be... The corrected security utilization threshold can then be expressed as:

[0056]

[0057] in, This is the revised security utilization threshold. As the instance queue length increases, the revised threshold decreases, causing the caller's sidecar proxy to trigger load skewing earlier; when the instance queue length is small, the threshold is close to the basic security utilization threshold to prioritize maintaining cache affinity.

[0058] The control plane distributes the security utilization threshold for each service instance to the caller-side proxy at preset intervals. Before forwarding a request, the caller-side proxy obtains the instantaneous queuing utilization of the primary target instance through locally cached instance status or lightweight non-blocking probes. If the instantaneous queuing utilization is not greater than the security utilization threshold, the request is still forwarded to the primary target instance located by consistent hashing to maintain cache affinity; if the instantaneous queuing utilization is greater than the security utilization threshold, a fast timescale load skew is triggered, forwarding the request to a subsequent healthy instance on the consistent hashing ring to avoid hotspot instances continuing to back up requests.

[0059] Furthermore, the two-level cache interception and result reuse mechanism in step S3 is as follows: Figure 4 As shown, when a cache key is not found and the business container returns a response, the receiving sidecar proxy verifies the response status code and cache control header. For responses that meet the caching conditions, it extracts deterministic parameters to generate a cache key, writes the cache key, response value, and time-to-live (TTL) into a hash mapping table, and after successful writing, performs a hash operation on the cache key to generate a fixed-length fingerprint, which is then inserted into a cuckoo filter. When generating the cache key, the receiving sidecar proxy uses the same request feature selection and normalization rules as the calling end's route hash key, or a subset thereof, to ensure that route affinity can be converted into a local cache hit on the receiving end. Through this setting, the receiving end can complete cache writing and subsequent request reuse without modifying the business code, reducing the repeated execution of the same computational logic.

[0060] Furthermore, the cuckoo filter uses two independent hash functions to determine the candidate bucket position, performs fingerprint shifting when a fingerprint collision occurs, and deletes the corresponding fixed-length fingerprint when a cache entry expires or is deleted. With this configuration, the receiving-end sidecar agent can quickly pre-check whether a request is likely to hit the cache and supports synchronously updating the filter structure when a cache entry is deleted, reducing the additional overhead caused by invalid cache queries.

[0061] Further, step S4 includes: recording the Uniform Resource Identifier (URI) of the downstream dependent resources when the upstream service proxy initiates a sub-request to the downstream service; hashing and compressing the URI to generate a fixed-length feature signature when saving the final cached response of the upstream service; and constructing an inverted index with the fixed-length feature signature as the key and the upstream cache key as the value. When a state change write operation occurs in the downstream service, the downstream service proxy generates an invalidation event message containing the hash signature of the write operation resource identifier and pushes it to the corresponding upstream service proxy. After receiving the invalidation event message, the upstream service proxy searches for the affected cache key in the inverted index and synchronously removes the affected cache entries from the hash mapping table and the cuckoo filter. Through this setting, the system can achieve cross-service cache cascading invalidation when there are call dependencies between services, reducing the risk of inconsistent cache results due to changes in business data.

[0062] like Figure 2 As shown, this is a service mesh-oriented redundant computation suppression routing cache coordination system, used in the service mesh-oriented redundant computation suppression routing cache coordination method. The service mesh-oriented redundant computation suppression routing cache coordination system includes a control plane awareness and warm-up module, a caller-end dual time-scale routing module, a receiver-end two-level cache management module, and a decentralized cache consistency module.

[0063] The control plane perception and preheating module is deployed on the service mesh control node. It is used to aggregate service instance load indicators, maintain the service call dependency topology, and provide the global state information required for route adjustment and cache invalidation to the dual time scale routing module and decentralized cache consistency module of the calling end.

[0064] The dual-timescale routing module of the calling end is deployed on the sidecar proxy of the microservice caller. It is used to generate routing hash keys based on request characteristics, and through consistent hash routing, hash ring virtual node weight adjustment and instantaneous load deflection check, it prioritizes forwarding call requests with the same deterministic request characteristics to the target instance that can reuse the corresponding cached results.

[0065] The receiving end two-level cache management module is deployed on the sidecar proxy of the microservice receiver. It generates a cache key and hash fingerprint based on the request characteristics corresponding to the route hash key, and returns a cache response through pre-detection by the cuckoo filter and matching by the hash mapping table.

[0066] The decentralized cache consistency module is deployed on each sidecar agent to record downstream dependent resources during the cache response generation process, build an inverted index with the compressed signature of the dependent resources as the key and the upstream cache key as the value, and locate and remove the affected cache entries when a state change event arrives.

[0067] In this embodiment, the route caching collaboration does not involve deploying routing, caching, and invalidation mechanisms independently. Instead, it establishes a collaborative relationship around the same request semantics at the service mesh proxy layer. The calling end-side proxy generates a route hash key based on request characteristics to increase the probability of the same deterministic request reaching the same instance. The receiving end-side proxy generates a cache key based on the corresponding request characteristics, enabling route affinity to be directly converted into a local cache hit. During the cache response generation process, the proxy records the identifiers of the downstream resources it depends on and locates the affected cache key through an inverted index when a state change event occurs, thereby preventing the cached result from being reused after it expires in the cross-service call chain.

[0068] Example 1 demonstrates the comparative experimental results of end-to-end latency and resource consumption of the routing caching collaboration method proposed in this invention and the native mechanism (such as Round-Robin) in a service mesh environment.

[0069] We selected the industry-standard microservice benchmark applications Bookinfo and SocialNetwork as our research subjects and deployed them in a Kubernetes cluster with dozens of nodes.

[0070] Experimental results are as follows Figure 5 As shown, by simulating burst concurrent traffic with Markov Modulated Poisson Process (MMPP) characteristics, the P50 (average delay) and P99 (long-tail delay) of the system are recorded.

[0071] As can be seen, with the increase of concurrent connections, the P99 latency of the native mechanism shows a precipitous increase, while the present invention, due to its use of a dual-time-scale load deflection mechanism (such as...), achieves a more stable performance. Figure 3 As shown in the figure, the two-level cache direct interception mechanism effectively absorbs instantaneous bursts, resulting in a significant reduction of P99 long-tail latency by more than 80%.

[0072] Example 2 demonstrates the ablation experiment results of the impact of different routing and caching coordination strategies on overall performance.

[0073] The performance of "using only caching", "using only affinity routing" and "the complete solution of this invention" were tested under multiple target request rates (RPS).

[0074] like Figure 6 As shown, when the request rate is in a high-load range, if only caching is relied upon and routing scheduling is lacking, the cache hit rate will be severely fragmented; if only static affinity routing is relied upon, it will cause hot nodes to overload and crash.

[0075] This invention combines dynamic weight adjustment and local deflection to maintain a strict balance of cluster load while eliminating redundant calculations.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A service mesh oriented redundant computation suppression type routing cache coordination method, characterized in that, The method is executed collaboratively among the control plane of the service mesh, the calling end sidecar proxy, and the receiving end sidecar proxy, and calls with the same deterministic request characteristics maintain semantic association during routing, cache lookup, and cache invalidation processes, including the following steps: Step S1, Construct a service mesh control plane awareness and preheating module: Construct a control plane awareness and preheating module in the service mesh, periodically collect resource load indicators of each instance in the microservice cluster, construct the service call dependency topology based on distributed tracing telemetry data, and perform offline cache preheating based on historical high-frequency access data during the instance expansion phase; Step S2: Perform consistent hash routing decision on the calling end with dual time scales: The sidecar proxy on the calling end intercepts the network request sent by the caller, extracts the request features that determine the business response result and generates a routing hash key, locates the main target instance on the consistent hash ring according to the routing hash key, and prioritizes routing call requests with the same request features to the same target instance; adjusts the weight of the virtual node of the hash ring on the slow time scale according to the load index collected by the control plane, and determines the request forwarding target on the fast time scale based on the instantaneous queuing status and security utilization threshold of the main target instance; Step S3: Execute two-level cache interception and result reuse at the receiving end: The receiving end sidecar agent intercepts the incoming request, generates a cache key and hash fingerprint based on the request characteristics corresponding to the route hash key, performs fingerprint pre-checking first through the Cuckoo filter, and then performs exact cache key matching through the hash mapping table; when the cache is hit, the receiving end sidecar agent directly returns the corresponding cache response, reusing the business calculation results on the target instance. Step S4: Perform decentralized cache consistency management based on inverted index: The receiving end sidecar proxy listens for application layer state change operations, cleans up the corresponding local cache, and pushes the compressed signature of the change event to the upstream proxy with call dependency based on the call dependency topology; the upstream proxy builds a local inverted index based on the downstream dependency resources recorded in the cache response generation process, and locates and clears the affected cache entries through the inverted index.

2. The service mesh oriented redundant computation suppressing type routing cache coordination method according to claim 1, characterized in that, The resource load metrics in step S1 include CPU utilization, memory utilization, and queuing depth. The control plane perception and preheating module aggregates the resource load metrics of each service instance according to a preset period, and generates the inter-service call dependency topology based on the call path, call frequency, and upstream and downstream service identifiers in the distributed tracing telemetry data.

3. The service mesh oriented redundant computation suppressing type routing cache coordination method according to claim 2, characterized in that, The offline cache preheating in step S1 includes: analyzing the Layer 7 application layer telemetry logs in the control plane, extracting historical high-frequency requests and their corresponding business response results, assigning sampling weights to request records using a time-weighted sampling algorithm, injecting the selected response data and their corresponding cache keys and hash fingerprints into the hash mapping table and cuckoo filter of the newly launched instance, and configuring the initial survival time for the preheating entries.

4. The service mesh oriented redundant computation suppressing type routing cache cooperation method according to claim 1, characterized in that, The request features in step S2 include the caller service identifier, the request resource identifier, and deterministic parameters that determine the business response result. The request resource identifier includes the HTTP URI for HTTP requests and the gRPC method name, RPC service name, and operation name for gRPC and RPC requests. The deterministic parameters include the request method, path parameters, query parameters, business fields in the request body that participate in the response result calculation, the request body summary, and request header fields related to the business response result. After the calling end-side vehicle proxy normalizes the request features, it generates a route hash key and locates the primary target instance on the consistent hash ring based on the route hash key. The deterministic parameters are also used to generate a cache key in step S3, so that route location and cache matching are performed based on the same request semantics.

5. The service mesh oriented redundant computation suppressing type routing cache coordination method according to claim 4, characterized in that, The process of calling the edge vehicle proxy to normalize the request features includes: selecting fields for calculation according to a preset field whitelist, and removing the tracing identifier, request timestamp, random number, temporary session identifier, and retry count identifier; sorting query parameters and path parameters by field name; performing stable serialization of JSON fields, Protobuf fields, and form fields in the request body according to the field path; filling default fields with preset default values; uniformly processing case sensitivity, encoding format, and semantically indistinguishable delimiters; and performing hash operations on the normalized field sequence to generate a routing hash key.

6. The service mesh oriented redundant computation suppressing type routing cache coordination method according to claim 5, characterized in that, In the slow time scale of step S2, the control plane uses an exponentially weighted moving average algorithm to calculate the smoothed load of each service instance, compares the smoothed load of each service instance with the cluster average load, and adjusts the number of virtual nodes of the service instance on the consistent hash ring according to the reciprocal of the deviation ratio when the deviation ratio exceeds the preset tolerance band.

7. The service mesh oriented redundant computation suppressing type routing cache cooperation method according to claim 5, characterized in that, In the fast timescale of step S2, the control plane collects historical request arrival rate, average service time, service time variance, queue length, historical network timeout time, and target latency threshold for each service instance within a preset statistical window. Let the average service time of the service instance be S, and the service rate be... The target latency threshold is Reserved network transmission and proxy processing overhead. The allowed queuing time is: when At that time, the control surface solves for the condition that the predicted waiting time does not exceed the relationship between queuing time and utilization rate. The maximum utilization rate is used as the safe utilization rate threshold; if the M / M / 1 approximation model is adopted, the safe utilization rate threshold can be calculated as follows: in, Indicates the safety utilization threshold, and will The limit is set between 0.3 and 0.85; Let the current queue length of the instance be q, the concurrent processing capacity of the instance be c, the queue correction coefficient be α, and the corrected security utilization threshold be: Before forwarding the current request, the edge vehicle proxy obtains the instantaneous queuing utilization of the primary target instance. The revised safety utilization threshold is defined as the instantaneous queuing utilization rate being greater than [a certain threshold]. When this happens, the request routing target is changed to the next healthy instance in the clockwise direction of the consistent hash ring. The healthy instance is a service instance that has passed the service mesh health check and has not been circuit-broken or removed.

8. The service mesh oriented redundant computation suppressing type routing cache cooperating method according to claim 1, characterized in that, In step S3, when the receiving end sidecar agent generates a cache key, it adopts the same request feature selection rules and normalization processing rules as the calling end sidecar agent in generating the route hash key. When the cache key is not found and the business container returns a response, the receiving end sidecar agent verifies the response status code and cache control header, extracts deterministic parameters for responses that meet the preset cache rules to generate a cache key, writes the cache key, response value and time to live into the hash mapping table, and performs a hash operation on the cache key to generate a fixed-length fingerprint and inserts it into the cuckoo filter after successful writing.

9. The service mesh oriented redundant computation suppressing type routing cache coordination method according to claim 8, characterized in that, The Cuckoo Filter uses two independent hash functions to determine the candidate bucket position, performs fingerprint shifting when a fingerprint collision occurs, and deletes the corresponding fixed-length fingerprint when a cache entry expires or is deleted. Step S4 includes: recording the Uniform Resource Identifier (URI) of the downstream dependent resources when the upstream service proxy initiates a sub-request to the downstream service; performing hash compression on the URI to generate a fixed-length feature signature when saving the final cached response of the upstream service and establishing a cache key; and constructing an inverted index with the fixed-length feature signature as the key and the upstream cache key as the value; when the downstream service performs a state change write operation, the downstream service proxy generates an invalidation event message containing the hash signature of the write operation resource identifier and pushes it to the corresponding upstream service proxy; after receiving the invalidation event message, the upstream service proxy searches for the affected cache key in the inverted index and synchronously removes the affected cache entries in the hash mapping table and the cuckoo filter.

10. A service mesh oriented redundant computation suppressing type routing cache coordination system, characterized in that, The redundant computation suppression routing cache coordination method for service mesh as described in any one of claims 1 to 9, wherein the redundant computation suppression routing cache coordination system for service mesh includes a control plane awareness and warm-up module, a caller-side dual-timescale routing module, a receiver-side two-level cache management module, and a decentralized cache consistency module; the control plane awareness and warm-up module is deployed on the service mesh control node, used to aggregate service instance load metrics, maintain inter-service call dependency topology, and provide global state information required for route adjustment and cache invalidation to the caller-side dual-timescale routing module and the decentralized cache consistency module; the caller-side dual-timescale routing module is deployed on the sidecar proxy of the microservice caller, used to generate routes based on request characteristics. The system uses a consistent hash key, hash ring virtual node weight adjustment, and instantaneous load bias check to prioritize forwarding calls with the same deterministic request characteristics to target instances that can reuse the corresponding cached results. The receiving end's two-level cache management module is deployed on the sidecar proxy of the microservice receiver. It generates cache keys and hash fingerprints based on the request characteristics corresponding to the route hash key, and returns cached responses through pre-checking with a cuckoo filter and matching with a hash mapping table. The decentralized cache consistency module is deployed on each sidecar proxy to record downstream dependent resources during the cache response generation process, construct an inverted index with the compressed signature of the dependent resource as the key and the upstream cache key as the value, and locate and remove affected cache entries when a state change event arrives.

Images