A microservice mesh optimization method based on cache consistency protocol

Abstract

The application relates to the technical field of distributed caching, and discloses a micro-service grid optimization method based on a cache consistency protocol, which comprises the following steps: when a read-only request starts, a wrapper generates a call parameter identifier and a read dependency set, and a cache manager records a start event; when the read-only request is executed, a data key and a downstream call parameter identifier are recorded, a cache hit is returned, and a cache miss calls a downstream; when the read-only request ends, the call parameter identifier, the read dependency set and a request result are sent; the cache manager searches for an invalid event after the start event, sends a cache saving message when there is no intersection, and saves a request result; a called party establishes an inverted relationship; after a data key is written, a cache invalidation message is sent in an inverted relationship; and the saving and invalidation sequences are kept through an ordered message channel. The application realizes non-blocking consistency control of micro-service caching, reduces response time delay and improves service throughput.

Description

Technical Field

[0001] This invention relates to the field of distributed caching technology, and in particular to a microservice mesh optimization method based on a cache consistency protocol. Background Technology

[0002] With the development of microservice architecture, call center applications are typically broken down into multiple interconnected microservices such as call control, agent management, customer management, and report management. A single user request may involve multiple layers of calls and generate significant call fanout. Since each edge in the microservice call graph corresponds to a network request, complex call relationships can easily lead to increased response time and service resource consumption. By caching the responses of downstream services, historical results can be reused directly when the same request parameters are used and the downstream service status remains unchanged, thereby reducing cross-service calls.

[0003] Existing microservice caching solutions primarily rely on manually written consistency logic for specific applications, backend storage layer caching, or time-based cache eviction mechanisms. Manually written consistency logic is costly to develop and difficult to adapt to dynamically changing microservice call graphs; backend storage layer caching cannot prematurely terminate upstream service calls; and time-based caching may continue to return expired results after data updates. Existing service meshes typically do not provide a general cache consistency mechanism for inter-service call results. Since microservice responses are determined by request parameters, local data state, and multi-level downstream call results, changes to underlying data require synchronous identification and invalidation of upstream cache entries that directly or indirectly depend on that data. Traditional cache consistency solutions for single data objects are difficult to apply directly. Summary of the Invention

[0004] One objective of this invention is to propose a microservice mesh optimization method based on a cache consistency protocol. This invention utilizes wrappers and cache managers to track service calls and data access dependencies, control cache retention and invalidation propagation, reduce unnecessary cross-service calls, and has the advantages of fast response speed, high throughput, strong cache consistency, and low critical path overhead.

[0005] A microservice mesh optimization method based on a cache consistency protocol according to an embodiment of the present invention includes the following steps:

[0006] When a read-only request begins, the wrapper generates call parameter identifiers and reads the dependency set and sends a start message; the cache manager records the call start event.

[0007] During request execution, the wrapper writes the read data key and downstream call parameter identifier into the read dependency set, queries the cache before calling the downstream read-only endpoint, returns the cached result if a match is found, and initiates the downstream call if a match is not found.

[0008] After the request is completed, the wrapper will send the call parameter identifier, the read dependency set, and the request result association to the cache manager;

[0009] The cache manager locates the call start event based on the call parameter identifier, retrieves the invalidation event after the call start event, and sends a cache save message to the upstream cache manager when there is no intersection between the dependency corresponding to the invalidation event and the read dependency set.

[0010] The upstream cache manager stores the request results, and the callee cache manager uses the data key and downstream call parameter identifier as indexes to establish an inverted relationship between dependencies and upstream cache entries;

[0011] After the data key is written, the wrapper sends an invalidation message. The cache manager determines the affected cache entries according to the inverted index and sends a cache invalidation message to the upstream cache manager. The upstream cache manager deletes the affected cache entries and continues to propagate the invalidation message upstream according to the corresponding call parameter identifier.

[0012] The cache manager sends cache save messages and cache invalidation messages through the same ordered message channel, maintaining the order of events identified by the same call parameter.

[0013] Optionally, the query cache before invoking the downstream read-only endpoint specifically includes:

[0014] When a read-only request begins, the wrapper establishes a set of accessed services corresponding to the current request;

[0015] After the downstream call returns, the wrapper writes the services accessed during the execution of the downstream call into the set of services accessed for the current request.

[0016] When the upstream cache manager saves the request result, it forms a set of dependent services for the services accessed during the generation of the request result, and associates and saves the set of dependent services with cache entries;

[0017] Before the wrapper calls the downstream read-only endpoint, it reads the cached entries and the set of dependent services corresponding to the downstream call parameter identifier, and performs an intersection judgment between the set of accessed services and the set of dependent services corresponding to the current request.

[0018] If the visited service set and the dependent service set have no intersection and the cache entry is hit, the wrapper returns the cached result;

[0019] When the visited service set and the dependent service set intersect, the wrapper bypasses the cache entries and initiates a downstream call.

[0020] Optionally, after the downstream call returns, the wrapper writes the services accessed during the execution of the downstream call into the set of accessed services corresponding to the current request, specifically including:

[0021] Assign a unique binary bit to each microservice, and combine the binary bits corresponding to all microservices to form a service access code;

[0022] When a sub-request accesses a microservice, the binary bits in the service access code corresponding to the accessed microservice are set to the first preset value, forming a set of accessed services corresponding to the sub-request.

[0023] When a sub-request returns to a parent request, the wrapper merges the bits set to the first preset value in the visited service set corresponding to the sub-request into the visited service set corresponding to the parent request.

[0024] When a parent request contains multiple levels of sub-requests, the visited service sets are merged level by level according to the order in which the sub-requests are returned, forming the visited service set corresponding to the parent request.

[0025] When the upstream cache manager saves the request result corresponding to the parent request, it registers the set of services accessed by the parent request as the set of dependent services corresponding to the cache entry.

[0026] Optionally, after the request is completed, the wrapper will send the call parameter identifier, the read dependency set, and the request result association to the cache manager, specifically including:

[0027] During request execution, the wrapper writes the data key to the read dependency set when reading the data key, and writes the downstream call parameter identifier to the read dependency set when initiating a downstream call;

[0028] When the request ends, the wrapper associates the set of read dependencies formed during the request execution with the call parameter identifiers and the request result, and sends it to the cache manager all at once via an end message;

[0029] The cache manager locates the call start event based on the call parameter identifier and determines the time range between the call start event and the receipt of the end message as the invalidation event retrieval range.

[0030] The cache manager sequentially extracts the dependencies corresponding to the invalid events from the scope of invalid event retrieval, and performs an intersection judgment between the dependencies corresponding to the invalid events and the read dependency set.

[0031] When a dependency intersects with the read dependency set, the cache manager prohibits saving request results and stops sending cache save messages. When no dependency intersects with the read dependency set, the cache manager sends a cache save message to the upstream cache manager.

[0032] Optionally, the cache manager's location of the call start event based on the call parameter identifier specifically includes:

[0033] Read-only requests are routed according to the call parameter identifier, and read-only requests with the same call parameter identifier are sent to the same service shard.

[0034] The cache manager corresponding to the service shard receiving the read-only request records the start event and sends a cache save message based on the invalidation event retrieval result after the request ends. It also sends a cache invalidation message after the data key is written.

[0035] After the data key in the service shard is written, the wrapper sends an invalidation message to the corresponding cache manager, and the cache manager broadcasts the invalidation message to the cache managers corresponding to other service shards in the same microservice.

[0036] Each service shard's corresponding cache manager determines the affected cache entries based on the inverted index and sends a cache invalidation message to the upstream cache manager.

[0037] The upstream cache manager deletes the affected cache entries and continues to propagate the cache invalidation message upstream with the corresponding call parameter identifier. The cache invalidation message propagated along the upstream call relationship is not broadcast again between service shards.

[0038] Optionally, the upstream cache manager storing the request results and the cache manager determining the affected cache entries according to the inverted index specifically include:

[0039] When the upstream cache manager's cache usage reaches the cache capacity threshold, select and delete cache entries;

[0040] After a cache entry is deleted, the cache manager continues to process cache invalidation messages for the deleted cache entry;

[0041] When the cache manager receives a cache invalidation message corresponding to a deleted cache entry, it determines the affected upstream cache entry based on the retained inverted index and sends a cache invalidation message.

[0042] Optionally, establishing the inverted index relationship between dependencies and upstream cache entries specifically includes:

[0043] When the memory usage of the inverted index maintained by the cache manager reaches the memory threshold, the inverted index to be deleted is determined.

[0044] The cache manager determines the affected cache entries based on the inverted list to be deleted, and sends a cache invalidation message to the upstream cache manager corresponding to the affected cache entries before deleting the inverted list to be deleted;

[0045] The cache manager deletes the inverted list items to be deleted;

[0046] After the cache manager processes the end message, it deletes the most recent call start event corresponding to the call parameter identifier from the history.

[0047] Optionally, the cache manager sends cache save messages and cache invalidation messages through the same ordered message channel, maintaining the order of events corresponding to the same call parameter identifier. Specifically, this includes:

[0048] The cache manager collects cache save messages and cache invalidation messages destined for the same upstream cache manager within a preset batch processing cycle, and forms message batches according to the order in which the messages are generated;

[0049] The cache manager matches cached messages and cached invalid messages in a message batch according to the call parameter identifier;

[0050] When the same call parameter identifier corresponds to multiple cache operations, the cache manager retains the cache operation that is last in the event sequence and deletes the cache operation that is before the last cache operation.

[0051] The cache manager generates batch messages in the order of events that preserve cache operations and sends the batch messages to the upstream cache manager through the same ordered message channel.

[0052] Optionally, when the read-only request begins, the wrapper establishes the set of accessed services corresponding to the current request, specifically including:

[0053] Once the client request is completed, the wrapper associates the set of accessed services corresponding to the client request with the request result and sends it to the client.

[0054] The client saves the set of services accessed corresponding to the previous client request and sends the set of services accessed corresponding to the previous client request to the wrapper when making subsequent requests;

[0055] After receiving a subsequent request from the same client, the wrapper writes the set of services accessed corresponding to the previous client request into the set of services accessed corresponding to the subsequent request.

[0056] During the execution of subsequent requests, the wrapper will continue to add the services accessed during the execution of downstream calls to the set of services accessed for the corresponding subsequent requests;

[0057] Before calling the downstream read-only endpoint, the wrapper performs an intersection check between the set of accessed services corresponding to the subsequent request and the set of dependent services associated with the cached entry. If there is no intersection, the cached result is used; if there is an intersection, the cached entry is bypassed and the downstream call is initiated.

[0058] Optionally, the query cache before invoking the downstream read-only endpoint specifically includes:

[0059] Declare third-party service endpoints that do not have a cache manager deployed as read-only endpoints that use persistence;

[0060] Before calling a third-party service endpoint, the wrapper queries the cache based on the downstream call parameter identifier corresponding to the third-party service endpoint. If the cache is hit and the retention time has not expired, the cache result is returned.

[0061] When a cache miss occurs, the wrapper initiates a call to a third-party service. After the third-party service returns the request result, the upstream cache manager saves the request result and associates the retention time with the cache entry corresponding to the request result.

[0062] After the retention period expires, the upstream cache manager deletes the cache entry corresponding to the request result.

[0063] The beneficial effects of this invention are:

[0064] (1) The present invention collects the data key dependencies and downstream call dependencies of read-only requests through a wrapper, and the cache manager determines whether the request result can be saved by combining the failure event after the request ends, thereby avoiding the generation of expired cache during concurrent writing, improving the consistency of cached data and the reliability of request processing.

[0065] (2) The present invention identifies service access conflicts in the dynamic call graph by judging the intersection of the accessed service set and the dependent service set. When there is a conflict, it bypasses the cache and initiates downstream calls to avoid the same request returning incorrect cache results when repeatedly accessing backend services along different call paths.

[0066] (3) The present invention propagates cache invalidation messages step by step through an inverted index relationship, and adopts ordered message channels, fragmented invalidation broadcasting and cache message batch processing, so that cache saving and invalidation processing are completed outside the critical path of the request, reducing cross-service calls and synchronous communication overhead, reducing response time and improving overall throughput. Attached Figure Description

[0067] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0068] Figure 1 This is a schematic diagram of a microservice graph caching framework for a microservice mesh optimization method based on a cache consistency protocol proposed in this invention.

[0069] Figure 2 This is a schematic diagram of the microservice call relationship in a microservice mesh optimization method based on a cache consistency protocol proposed in this invention.

[0070] Figure 3 This is a schematic diagram of the cache invocation process of a microservice mesh optimization method based on a cache consistency protocol proposed in this invention;

[0071] Figure 4This is a schematic diagram illustrating cache failure propagation in a microservice mesh optimization method based on a cache consistency protocol proposed in this invention.

[0072] Figure 5 This is a schematic diagram of the diamond-shaped call relationship of a microservice mesh optimization method based on cache consistency protocol proposed in this invention;

[0073] Figure 6 This is a schematic diagram illustrating the out-of-order cached messages in a microservice mesh optimization method based on a cache consistency protocol proposed in this invention.

[0074] Figure 7 This is a schematic diagram illustrating read / write event tracking for a microservice mesh optimization method based on a cache consistency protocol proposed in this invention. Detailed Implementation

[0075] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0076] Microservice Graph: Microservice applications integrate information from multiple backend services and refine it into a single user interface with the help of an intermediate processing layer. This design naturally results in a directed graph of microservices, which collectively implement the behavior of the application. In this graph, vertices represent microservices, and edges represent calls between them.

[0077] Service mesh: A distributed application runtime that coordinates service calls and data storage access through its API. Service mesh supports diverse data storage through the same API.

[0078] Wrapper: A wrapper is an intermediate layer that intercepts all communication between services and their data storage. The framework's wrappers are implemented on top of the service mesh, inheriting the service mesh's compatibility to accommodate various data storage methods.

[0079] Cache Manager: The cache manager tracks all inter-service communication and data storage access through a wrapper, saving and deleting cached entries to maintain consistency. It is deployed as a separate executable on the same node as the service.

[0080] Non-blocking cache consistency protocol: Requests in a microservice application only block while waiting for the sub-requests they invoke to return and complete execution; there is no blocking communication between independent requests. In other words, if a trace can be observed in the application, we can choose to execute any pending request or any of its sub-requests until it produces an execution event, and the new trace will also become part of the application's trace set.

[0081] Linearizable data storage: Data storage for each service is linearized: operations on objects occur atomically, in the same order as the real-time order of operations. If a write is completed before a read begins, the read must observe the effects of the write and complete after the write.

[0082] Endpoints: Each microservice has its own data store and exposes a set of methods that can be called by clients or other services. Borrowing the terminology of REST, we call these methods endpoints.

[0083] refer to Figures 1-7 A microservice mesh optimization method based on a cache consistency protocol includes the following steps:

[0084] When a read-only request begins, the wrapper generates call parameter identifiers and reads the dependency set and sends a start message; the cache manager records the call start event.

[0085] During request execution, the wrapper writes the read data key and downstream call parameter identifier into the read dependency set, queries the cache before calling the downstream read-only endpoint, returns the cached result if a match is found, and initiates the downstream call if a match is not found.

[0086] After the request is completed, the wrapper will send the call parameter identifier, the read dependency set, and the request result association to the cache manager;

[0087] The cache manager locates the call start event based on the call parameter identifier, retrieves the invalidation event after the call start event, and sends a cache save message to the upstream cache manager when there is no intersection between the dependency corresponding to the invalidation event and the read dependency set.

[0088] The upstream cache manager stores the request results, and the callee cache manager uses the data key and downstream call parameter identifier as indexes to establish an inverted relationship between dependencies and upstream cache entries;

[0089] After the data key is written, the wrapper sends an invalidation message. The cache manager determines the affected cache entries according to the inverted index and sends a cache invalidation message to the upstream cache manager. The upstream cache manager deletes the affected cache entries and continues to propagate the invalidation message upstream according to the corresponding call parameter identifier.

[0090] The cache manager sends cache save messages and cache invalidation messages through the same ordered message channel, maintaining the order of events identified by the same call parameter.

[0091] In this embodiment, querying the cache before calling the downstream read-only endpoint specifically includes:

[0092] When a read-only request begins, the wrapper establishes a set of accessed services corresponding to the current request;

[0093] After the downstream call returns, the wrapper writes the services accessed during the execution of the downstream call into the set of services accessed for the current request.

[0094] When the upstream cache manager saves the request result, it forms a set of dependent services for the services accessed during the generation of the request result, and associates and saves the set of dependent services with cache entries;

[0095] Before the wrapper calls the downstream read-only endpoint, it reads the cached entries and the set of dependent services corresponding to the downstream call parameter identifier, and performs an intersection judgment between the set of accessed services and the set of dependent services corresponding to the current request.

[0096] If the visited service set and the dependent service set have no intersection and the cache entry is hit, the wrapper returns the cached result;

[0097] When the visited service set and the dependent service set intersect, the wrapper bypasses the cache entries and initiates a downstream call.

[0098] In this embodiment, after the downstream call returns, the wrapper writes the services accessed during the execution of the downstream call into the set of accessed services corresponding to the current request. Specifically, this includes:

[0099] Assign a unique binary bit to each microservice, and combine the binary bits corresponding to all microservices to form a service access code;

[0100] When a sub-request accesses a microservice, the binary bits in the service access code corresponding to the accessed microservice are set to the first preset value, forming a set of accessed services corresponding to the sub-request.

[0101] When a sub-request returns to a parent request, the wrapper merges the bits set to the first preset value in the visited service set corresponding to the sub-request into the visited service set corresponding to the parent request.

[0102] When a parent request contains multiple levels of sub-requests, the visited service sets are merged level by level according to the order in which the sub-requests are returned, forming the visited service set corresponding to the parent request.

[0103] When the upstream cache manager saves the request result corresponding to the parent request, it registers the set of services accessed by the parent request as the set of dependent services corresponding to the cache entry.

[0104] like Figure 5As shown, when the same request accesses the same backend service through multiple call paths, a diamond-shaped call relationship may form. Service S1 first calls service S2, service S2 further calls service S4 and writes the data to service S4's data storage; subsequently, service S1 calls service S3, and service S3 again calls service S4 to read the aforementioned write result. If service S1 directly uses the previously saved result of the service S3 call, it may return a cached result formed before service S4 wrote the data, resulting in an inconsistency between the execution result with caching enabled and the execution result without caching enabled.

[0105] To avoid the above situation, the wrapper establishes a set of visited services corresponding to the current request at the beginning of the request and records the microservices accessed by the current request and its sub-requests at all levels during request processing. After the sub-request returns, the wrapper merges the set of visited services corresponding to the sub-request into the set of visited services corresponding to the parent request. When the upstream cache manager saves the request result, it forms a set of dependent services for the microservices accessed during the generation of the request result and associates the set of dependent services with the cache entry. Before calling the downstream read-only endpoint, the wrapper performs an intersection check between the set of visited services corresponding to the current request and the set of dependent services corresponding to the cache entry; if the two sets do not intersect and the cache entry is hit, the cached result is returned; if the two sets intersect, the cache entry is bypassed and the downstream call is initiated. The set of visited services is represented using binary encoding, with each microservice corresponding to a unique binary bit, to reduce the storage footprint of service access relationships.

[0106] In this implementation, after the request is completed, the wrapper sends the call parameter identifier, the read dependency set, and the request result association to the cache manager, specifically including:

[0107] During request execution, the wrapper writes the data key to the read dependency set when reading the data key, and writes the downstream call parameter identifier to the read dependency set when initiating a downstream call;

[0108] When the request ends, the wrapper associates the set of read dependencies formed during the request execution with the call parameter identifiers and the request result, and sends it to the cache manager all at once via an end message;

[0109] The cache manager locates the call start event based on the call parameter identifier and determines the time range between the call start event and the receipt of the end message as the invalidation event retrieval range.

[0110] The cache manager sequentially extracts the dependencies corresponding to the invalid events from the scope of invalid event retrieval, and performs an intersection judgment between the dependencies corresponding to the invalid events and the read dependency set.

[0111] When a dependency intersects with the read dependency set, the cache manager prohibits saving request results and stops sending cache save messages. When no dependency intersects with the read dependency set, the cache manager sends a cache save message to the upstream cache manager.

[0112] like Figure 3 As shown, when the Page service first retrieves a work order process with the ID 'id', it generates a downstream call parameter identifier based on the work order identifier and queries the cache entry corresponding to the downstream call parameter identifier. Since the query fails to find a cached entry, the Page service initiates a downstream call to the Plot service. The Plot service reads the work order process with the ID 'id' and returns the requested result to the Page service. The Plot service's cache manager, based on the read dependency set formed by this call, determines that there are no invalid events intersecting with the read dependency set after the call start event, and then sends a cache save message to the Page service's cache manager. The Page service's cache manager receives the cache save message and associates the downstream call parameter identifier with the request result. When the Page service retrieves a work order process again using the same parameters, it queries the corresponding cached entry and directly returns the cached result without calling the Plot service again.

[0113] In this embodiment, the cache manager locates the call start event based on the call parameter identifier, specifically including:

[0114] Read-only requests are routed according to the call parameter identifier, and read-only requests with the same call parameter identifier are sent to the same service shard.

[0115] The cache manager corresponding to the service shard receiving the read-only request records the start event and sends a cache save message based on the invalidation event retrieval result after the request ends. It also sends a cache invalidation message after the data key is written.

[0116] After the data key in the service shard is written, the wrapper sends an invalidation message to the corresponding cache manager, and the cache manager broadcasts the invalidation message to the cache managers corresponding to other service shards in the same microservice.

[0117] Each service shard's corresponding cache manager determines the affected cache entries based on the inverted index and sends a cache invalidation message to the upstream cache manager.

[0118] The upstream cache manager deletes the affected cache entries and continues to propagate the cache invalidation message upstream with the corresponding call parameter identifier. The cache invalidation message propagated along the upstream call relationship is not broadcast again between service shards.

[0119] Each microservice's wrapper communicates with its corresponding cache manager through an ordered message channel, ensuring that request start messages, request end messages, cache save messages, and cache invalidation messages are transmitted in a predetermined order. Downstream cache managers send cache save messages and cache invalidation messages to upstream cache managers through the same ordered message channel. When multiple service shards are deployed for the same microservice, the cache managers corresponding to each service shard communicate with each other through invalidation broadcasts.

[0120] When deploying microservices using sharding, a corresponding cache manager is configured for each service shard, and read-only requests are routed according to the call parameter identifier, ensuring that read-only requests with the same call parameter identifier are always processed by the same service shard. The cache manager corresponding to the service shard receiving the read-only request is responsible for recording the call start event and sending cache save and cache invalidation messages for the same call parameter identifier, thereby preventing multiple service shards from managing the same cache entry simultaneously.

[0121] Once a data key in any service shard is written, the wrapper sends an invalidation message to the corresponding cache manager. The cache manager then broadcasts the invalidation message to the cache managers corresponding to other service shards within the same microservice. Each service shard's cache manager determines the affected cache entries based on the inverted index and sends a cache invalidation message to its upstream cache manager. Cache invalidation messages that propagate tier by tier along the upstream call hierarchy are no longer broadcast between service shards within the same microservice. This invalidation broadcasting is performed outside the critical path of request processing and does not increase the inter-shard synchronization overhead during read-only request processing.

[0122] The wrapper establishes a read dependency set and sends a start message at the beginning of a read-only request. It records the data key when requesting to read the data key, records the downstream call parameter identifier and queries the cache before initiating the downstream call, sends the read dependency set and request result at the end of the request, and sends an invalidation message after the data key is written. Upon receiving these messages, the cache manager records the call start event and invalidation event, determines whether there are any invalidation events intersecting with the read dependency set during request execution, decides whether to send a cache save message to the upstream cache manager, and identifies the affected cache entries and propagates the cache invalidation message based on the inverted index.

[0123] The wrapper maintains a read dependency set and a request context in each service shard. The read dependency set is used to record the data keys read by the suspended read-only request and the downstream call parameter identifiers according to the request identifier, so as to form the dependency relationship during request execution. The request context is passed along with the request processing, recording the request identifier, call parameter identifiers, caller information, accessed service set, and read-only attributes.

[0124] When a read-only request begins, the wrapper establishes a read dependency set and sends a start message to the corresponding cache manager. When requesting to read a data key, the wrapper writes the data key into the read dependency set. When a downstream call is initiated, the wrapper writes the downstream call parameter identifier into the read dependency set, queries the cache before the call, and returns the cached result directly if a cache hit occurs. After the downstream call returns, the wrapper merges the services accessed during the downstream call into the current request's visited service set. When the request ends, the wrapper associates the call parameter identifier, read dependency set, request result, and visited service set and sends them to the cache manager. After the data key is written, the wrapper sends an invalidation message to the cache manager, which then performs subsequent cache invalidation processing.

[0125] In this embodiment, the upstream cache manager saving the request result and the cache manager determining the affected cache entries according to the inverted index specifically include:

[0126] When the upstream cache manager's cache usage reaches the cache capacity threshold, select and delete cache entries;

[0127] After a cache entry is deleted, the cache manager continues to process cache invalidation messages for the deleted cache entry;

[0128] When the cache manager receives a cache invalidation message corresponding to a deleted cache entry, it determines the affected upstream cache entry based on the retained inverted index and sends a cache invalidation message.

[0129] When cache usage reaches the cache capacity threshold, the upstream cache manager selects and deletes cache entries according to preset eviction rules, freeing up storage space for new cache entries. For deleted cache entries, the cache manager can still perform invalidation processing when it subsequently receives a corresponding cache invalidation message, preventing premature deletion of cache entries from affecting the operation of the cache consistency protocol.

[0130] In this embodiment, establishing the inverted index relationship between dependencies and upstream cache entries specifically includes:

[0131] When the memory usage of the inverted index maintained by the cache manager reaches the memory threshold, the inverted index to be deleted is determined.

[0132] The cache manager determines the affected cache entries based on the inverted list to be deleted, and sends a cache invalidation message to the upstream cache manager corresponding to the affected cache entries before deleting the inverted list to be deleted;

[0133] The cache manager deletes the inverted list items to be deleted;

[0134] After the cache manager processes the end message, it deletes the most recent call start event corresponding to the call parameter identifier from the history.

[0135] When the memory occupied by the inverted index maintained by the cache manager reaches the memory threshold, the cache manager determines the data key or downstream call parameter identifier to be deleted from the saved dependencies, determines the affected upstream cache entries based on the inverted index corresponding to the dependency to be deleted, and sends a cache invalidation message to the upstream cache manager corresponding to the affected cache entry before deleting the dependency to be deleted; after receiving the cache invalidation message, the upstream cache manager deletes the corresponding cache entry, and then the cache manager releases the storage space occupied by the dependency to be deleted.

[0136] The cache manager is used to maintain cached content, inverted index, and event history. Each service shard's corresponding cache manager stores its own inverted index and historical records. The inverted index uses a data key and downstream call parameter identifier as indices, recording upstream cache entries that depend on the corresponding data key and downstream call parameter identifier. When a cached result corresponding to a data key or downstream call parameter identifier becomes invalid, the cache manager determines the affected cache entry based on the inverted index and sends a cache invalidation message to the upstream cache manager corresponding to the affected cache entry, causing the cache invalidation message to propagate cascading along the microservice call relationship.

[0137] The history record is used to save call start events and expiration events. After receiving a request end message, the cache manager locates the call start event based on the call parameter identifier and retrieves expiration events generated after the call start event in reverse chronological order. It then performs an intersection check between the dependencies corresponding to the expiration events and the read dependency set. If intersecting dependencies exist, no cache save message is sent; otherwise, a cache save message is sent to the upstream cache manager, which saves the request result. The retrieval scope of the history record is jointly determined by the request processing time and request rate to reduce the number of events that need to be processed for cache security judgment.

[0138] In this embodiment, the cache manager sends cache save messages and cache invalidation messages through the same ordered message channel, maintaining the order of events corresponding to the same call parameter identifier. Specifically, this includes:

[0139] The cache manager collects cache save messages and cache invalidation messages destined for the same upstream cache manager within a preset batch processing cycle, and forms message batches according to the order in which the messages are generated;

[0140] The cache manager matches cached messages and cached invalid messages in a message batch according to the call parameter identifier;

[0141] When the same call parameter identifier corresponds to multiple cache operations, the cache manager retains the cache operation that is last in the event sequence and deletes the cache operation that is before the last cache operation.

[0142] The cache manager generates batch messages in the order of events that preserve cache operations and sends the batch messages to the upstream cache manager through the same ordered message channel.

[0143] In this embodiment, when a read-only request begins, the wrapper establishes the set of accessed services corresponding to the current request, specifically including:

[0144] Once the client request is completed, the wrapper associates the set of accessed services corresponding to the client request with the request result and sends it to the client.

[0145] The client saves the set of services accessed corresponding to the previous client request and sends the set of services accessed corresponding to the previous client request to the wrapper when making subsequent requests;

[0146] After receiving a subsequent request from the same client, the wrapper writes the set of services accessed corresponding to the previous client request into the set of services accessed corresponding to the subsequent request.

[0147] During the execution of subsequent requests, the wrapper will continue to add the services accessed during the execution of downstream calls to the set of services accessed for the corresponding subsequent requests;

[0148] Before calling the downstream read-only endpoint, the wrapper performs an intersection check between the set of accessed services corresponding to the subsequent request and the set of dependent services associated with the cached entry. If there is no intersection, the cached result is used; if there is an intersection, the cached entry is bypassed and the downstream call is initiated.

[0149] In this embodiment, querying the cache before calling the downstream read-only endpoint specifically includes:

[0150] Declare third-party service endpoints that do not have a cache manager deployed as read-only endpoints that use persistence;

[0151] Before calling a third-party service endpoint, the wrapper queries the cache based on the downstream call parameter identifier corresponding to the third-party service endpoint. If the cache is hit and the retention time has not expired, the cache result is returned.

[0152] When a cache miss occurs, the wrapper initiates a call to a third-party service. After the third-party service returns the request result, the upstream cache manager saves the request result and associates the retention time with the cache entry corresponding to the request result.

[0153] After the retention period expires, the upstream cache manager deletes the cache entry corresponding to the request result.

[0154] like Figure 1As shown, when deploying a microservice graph caching framework, read-only attributes are declared for the endpoints exposed by each microservice. Endpoints that do not cause changes to the data storage content are designated as read-only endpoints. When using REST interfaces, GET endpoints can be identified as read-only endpoints. Each microservice sets up a wrapper, a cache manager, and a cache. The wrapper is used to intercept service calls and data access, while the cache manager is used to maintain cache entries and cache dependencies. When Service1 calls the read-only endpoint of Service2, Service1 first queries its local cache. If the cache misses, Service1 sends a request to Service2 and saves the corresponding request result after the request returns successfully. During Service2's request processing, Service2's wrapper records the data key to be read, and Service2's cache manager establishes a dependency relationship between the data key and the cache entry in Service1. When any dependent data key in Service2's data storage is modified by write, Service2's cache manager sends a cache invalidation message to Service1's cache manager according to the dependency relationship. Service1's cache manager then deletes the corresponding cache entry, thereby preventing subsequent requests from continuing to use the invalidated cache result.

[0155] like Figure 2 As shown, the client sends an access request for a specific work order page to the Page service. The Page service initiates downstream calls to the CustomerReview service and the Plot service respectively based on the work order identifier. The CustomerReview service reads the user feedback associated with the work order and generates the user feedback result. The Plot service reads the work order process associated with the work order and generates the work order process result. After receiving the user feedback result and the work order process result, the Page service summarizes the request results of the two downstream calls, generates the page result corresponding to the specific work order, and returns it to the client. Figure 2 The call relationships shown constitute a microservice call graph corresponding to a client request, with the Page service as the caller and the CustomerReview and Plot services as the callees.

[0156] like Figure 4As shown, the first client writes new user feedback to the work order with ID id. After the write operation is completed, the wrapper sends an invalidation message to the corresponding cache manager. The cache manager determines the affected cache entry based on the inverted index relationship between the data key and the upstream cache entry, and propagates the cache invalidation message upstream along the microservice call relationship, invalidating the cache entry in the Page service that depends on the work order feedback data with ID id. During the propagation of the cache invalidation message, the second and third clients can respectively hit the cache entries that have not yet completed invalidation processing and obtain the page result. Since the write request and the two read requests come from different clients, there is no intra-request dependency relationship between the three requests. By adjusting the execution order of the events corresponding to the independent requests, it is possible to form a cache structure that is not enabled in the application. Figure 4 The execution process is equivalent to the client behavior shown.

[0157] Example 1: To prove the correctness of the microservice graph caching framework, it is shown that clients cannot distinguish between an application with the microservice graph caching framework enabled and the original application without caching. Observable execution events and traces are used to illustrate microservice applications with and without caching. An event is an indivisible operation that a microservice application can perform; examples of events include reading keys from a data store and receiving responses to completed sub-requests. An application can be uniquely described by a set of traces observed within it. When all events in one set of traces exist in another set of traces but in a different order, the two sets of traces are called equivalent modulo reorderings. Reordering is necessary for proving the correctness theorem to examine the case where reads and writes occur simultaneously (e.g., ...). Figure 4 (As shown). The proof of this theorem has three crucial assumptions: the first two apply to all microservice applications, and the last assumption is a requirement of microservice graph caching frameworks. The main theorem is then stated, and the framework of the proof is given.

[0158] Always enable requests. In a microservice application, requests only block while waiting for the sub-requests they invoke to return and complete execution; there is no blocking communication between independent requests. If a trace can be observed in the application, then any pending request or any of its sub-requests can be executed until it produces an execution event, and the new trace will also become part of the application's tracing set.

[0159] Reordering independent events. Two events are related when the first event affects the execution of the second: some examples include two events belonging to the same request, or write and read events to the same key in a service data store. Two events are independent when the first event does not affect the execution of the second. Assuming that independent events can be swapped due to multithreading, the result of reordering any two consecutive independent events from a trace in the application trace results in an event that can be observed in the application.

[0160] Linearizable data storage. This assumes that the data storage for each service is linear: operations on objects occur atomically, in the same order as the real-time order of operations. For example, if a write completes before a read begins, the read must observe the effect of the write and complete after the write. This is necessary because the microservice graph caching framework cannot modify the underlying data storage and can only observe writes to the data storage before or after the write is complete. If non-linear data storage were used, writes might take effect after a return, making it impossible to track which call invalidated them.

[0161] Theorem, Protocol Correctness: For all traces in an application with caching enabled, there exists a trace in the original application without caching such that all client events in both traces are equivalent modulo reordered.

[0162] To prove the correctness of the protocol, a corresponding non-caching execution trace can be constructed for any cache-enabled execution trace. This ensures that, aside from downstream requests omitted in cache hits, the remaining request sub-traces remain consistent, and that the application state at the end of both execution traces is the same. Specifically, write events that occurred before a cache hit and have no execution dependency on the current cached result are first moved to after the cache hit. At the end of the adjusted execution trace, cache entries related to the write events and their corresponding dependencies are invalidated. Then, based on the already constructed shorter execution trace, the execution prefix corresponding to the portion of the non-caching execution process before the cache hit is obtained. Subsequently, downstream request events omitted due to cache hits are supplemented, and write events and related events are supplemented according to the real-time atomic operation order of data storage, thus forming a non-caching execution trace with consistent client-observable behavior and the same final application state.

[0163] The microservice graph caching framework implementation includes a wrapper that intercepts calls and performs state access, and a cache manager that makes invalidation and retention decisions. Communication between the wrapper and the cache manager is via RocketMQ, while cache managers communicate with each other via HTTP. The current implementation uses Redis as the cache, but any in-memory storage can be used instead. We use a 32-bit FNV-1a algorithm to calculate the hash value of the call parameters.

[0164] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A microservice mesh optimization method based on a cache consistency protocol, characterized in that, Includes the following steps: When a read-only request begins, the wrapper generates call parameter identifiers and reads the dependency set and sends a start message; the cache manager records the call start event. During request execution, the wrapper writes the read data key and downstream call parameter identifier into the read dependency set, queries the cache before calling the downstream read-only endpoint, returns the cached result if a match is found, and initiates the downstream call if a match is not found. After the request is completed, the wrapper will send the call parameter identifier, the read dependency set, and the request result association to the cache manager; The cache manager locates the call start event based on the call parameter identifier, retrieves the invalidation event after the call start event, and sends a cache save message to the upstream cache manager when there is no intersection between the dependency corresponding to the invalidation event and the read dependency set. The upstream cache manager stores the request results, and the callee cache manager uses the data key and downstream call parameter identifier as indexes to establish an inverted relationship between dependencies and upstream cache entries; After the data key is written, the wrapper sends an invalidation message. The cache manager determines the affected cache entries according to the inverted index and sends a cache invalidation message to the upstream cache manager. The upstream cache manager deletes the affected cache entries and continues to propagate the invalidation message upstream according to the corresponding call parameter identifier. The cache manager sends cache save messages and cache invalidation messages through the same ordered message channel, maintaining the order of events identified by the same call parameter.

2. The microservice mesh optimization method based on cache consistency protocol according to claim 1, characterized in that, The query cache before invoking the downstream read-only endpoint specifically includes: When a read-only request begins, the wrapper establishes a set of accessed services corresponding to the current request; After the downstream call returns, the wrapper writes the services accessed during the execution of the downstream call into the set of services accessed for the current request. When the upstream cache manager saves the request result, it forms a set of dependent services for the services accessed during the generation of the request result, and associates and saves the set of dependent services with cache entries; Before the wrapper calls the downstream read-only endpoint, it reads the cached entries and the set of dependent services corresponding to the downstream call parameter identifier, and performs an intersection judgment between the set of accessed services and the set of dependent services corresponding to the current request. If the visited service set and the dependent service set have no intersection and the cache entry is hit, the wrapper returns the cached result; When the visited service set and the dependent service set intersect, the wrapper bypasses the cache entries and initiates a downstream call.

3. The microservice mesh optimization method based on cache consistency protocol according to claim 2, characterized in that, After the downstream call returns, the wrapper writes the services accessed during the execution of the downstream call into the set of accessed services corresponding to the current request, specifically including: Assign a unique binary bit to each microservice, and combine the binary bits corresponding to all microservices to form a service access code; When a sub-request accesses a microservice, the binary bits in the service access code corresponding to the accessed microservice are set to the first preset value, forming a set of accessed services corresponding to the sub-request. When a sub-request returns to a parent request, the wrapper merges the bits set to the first preset value in the visited service set corresponding to the sub-request into the visited service set corresponding to the parent request. When a parent request contains multiple levels of sub-requests, the visited service sets are merged level by level according to the order in which the sub-requests are returned, forming the visited service set corresponding to the parent request. When the upstream cache manager saves the request result corresponding to the parent request, it registers the set of accessed services corresponding to the parent request as the set of dependent services corresponding to the cache entry.

4. The microservice mesh optimization method based on cache consistency protocol according to claim 3, characterized in that, After the request is completed, the wrapper will send the call parameter identifier, the read dependency set, and the request result association to the cache manager, specifically including: During request execution, the wrapper writes the data key to the read dependency set when reading the data key, and writes the downstream call parameter identifier to the read dependency set when initiating the downstream call; When the request ends, the wrapper associates the set of read dependencies formed during the request execution with the call parameter identifiers and the request result, and sends it to the cache manager all at once via an end message; The cache manager locates the call start event based on the call parameter identifier and determines the time range between the call start event and the receipt of the end message as the invalidation event retrieval range. The cache manager sequentially extracts the dependencies corresponding to the invalid events from the scope of invalid event retrieval, and performs an intersection judgment between the dependencies corresponding to the invalid events and the read dependency set. When a dependency intersects with the read dependency set, the cache manager prohibits saving request results and stops sending cache save messages. When no dependency intersects with the read dependency set, the cache manager sends a cache save message to the upstream cache manager.

5. A microservice mesh optimization method based on a cache consistency protocol according to claim 4, characterized in that, The cache manager locates the call start event based on the call parameter identifier, specifically including: Read-only requests are routed according to the call parameter identifier, and read-only requests with the same call parameter identifier are sent to the same service shard. The cache manager corresponding to the service shard receiving the read-only request records the start event and sends a cache save message based on the invalidation event retrieval result after the request ends. It also sends a cache invalidation message after the data key is written. After the data key in the service shard is written, the wrapper sends an invalidation message to the corresponding cache manager, and the cache manager broadcasts the invalidation message to the cache managers corresponding to other service shards in the same microservice. Each service shard's corresponding cache manager determines the affected cache entries based on the inverted index and sends a cache invalidation message to the upstream cache manager. The upstream cache manager deletes the affected cache entries and continues to propagate the cache invalidation message upstream with the corresponding call parameter identifier. The cache invalidation message propagated along the upstream call relationship is not broadcast again between service shards.

6. A microservice mesh optimization method based on a cache consistency protocol according to claim 5, characterized in that, The upstream cache manager storing request results and the cache manager determining affected cache entries according to the inverted index specifically include: When the upstream cache manager's cache usage reaches the cache capacity threshold, select and delete cache entries; After a cache entry is deleted, the cache manager continues to process cache invalidation messages for the deleted cache entry; When the cache manager receives a cache invalidation message corresponding to a deleted cache entry, it determines the affected upstream cache entry based on the retained inverted index and sends a cache invalidation message.

7. A microservice mesh optimization method based on a cache consistency protocol according to claim 6, characterized in that, The establishment of the inverted index relationship between dependencies and upstream cache entries specifically includes: When the memory usage of the inverted index maintained by the cache manager reaches the memory threshold, the inverted index to be deleted is determined. The cache manager determines the affected cache entries based on the inverted list to be deleted, and sends a cache invalidation message to the upstream cache manager corresponding to the affected cache entries before deleting the inverted list to be deleted; The cache manager deletes the inverted list items to be deleted; After the cache manager processes the end message, it deletes the most recent call start event corresponding to the call parameter identifier from the history.

8. A microservice mesh optimization method based on a cache consistency protocol according to claim 7, characterized in that, The cache manager sends cache save messages and cache invalidation messages through the same ordered message channel, maintaining the order of events identified by the same call parameter. Specifically, this includes: The cache manager collects cache save messages and cache invalidation messages destined for the same upstream cache manager within a preset batch processing cycle, and forms message batches according to the order in which the messages are generated; The cache manager matches cached messages and cached invalid messages in a message batch according to the call parameter identifier; When the same call parameter identifier corresponds to multiple cache operations, the cache manager retains the cache operation that is last in the event sequence and deletes the cache operation that is before the last cache operation. The cache manager generates batch messages in the order of events that preserve cache operations and sends the batch messages to the upstream cache manager through the same ordered message channel.

9. A microservice mesh optimization method based on a cache consistency protocol according to claim 8, characterized in that, When the read-only request begins, the wrapper establishes the set of accessed services corresponding to the current request, specifically including: Once the client request is completed, the wrapper associates the set of accessed services corresponding to the client request with the request result and sends it to the client. The client saves the set of services accessed corresponding to the previous client request and sends the set of services accessed corresponding to the previous client request to the wrapper when making subsequent requests; After receiving a subsequent request from the same client, the wrapper writes the set of services accessed corresponding to the previous client request into the set of services accessed corresponding to the subsequent request. During the execution of subsequent requests, the wrapper will continue to add the services accessed during the execution of downstream calls to the set of services accessed for the corresponding subsequent requests; Before calling the downstream read-only endpoint, the wrapper performs an intersection check between the set of accessed services corresponding to the subsequent request and the set of dependent services associated with the cached entry. If there is no intersection, the cached result is used; if there is an intersection, the cached entry is bypassed and the downstream call is initiated.

10. A microservice mesh optimization method based on a cache consistency protocol according to claim 9, characterized in that, The query cache before invoking the downstream read-only endpoint specifically includes: Declare third-party service endpoints that do not have a cache manager deployed as read-only endpoints that use persistence; Before calling a third-party service endpoint, the wrapper queries the cache based on the downstream call parameter identifier corresponding to the third-party service endpoint. If the cache is hit and the retention time has not expired, the cache result is returned. When a cache miss occurs, the wrapper initiates a call to a third-party service. After the third-party service returns the request result, the upstream cache manager saves the request result and associates the retention time with the cache entry corresponding to the request result. After the retention period expires, the upstream cache manager deletes the cache entry corresponding to the request result.

Images