A distributed cache system and method based on istio

Abstract

The embodiment of the present invention provides a distributed cache system and method based on istio, including: an istio inlet gateway, which is used to receive HTTP requests sent by users during the process of business migration container, and the HTTP requests are used to request data from the current container of the business; forwarding the received HTTP requests to the cache subsystem, or forwarding the received HTTP requests to the envoy filter; the cache subsystem, which is used to receive the HTTP requests forwarded by the istio inlet gateway, search for corresponding cache data based on the HTTP requests, and return them to the user; the envoy filter, which is used to receive the HTTP requests forwarded by the istio inlet gateway, and obtain corresponding data from the current container of the business based on the HTTP requests. By adopting isito on kubernetes, dynamic caching of HTTP service content is realized, and the construction program is simple and low-cost.

Description

Technical Field

[0001] The present invention relates to container deployment, and in particular to an istio-based distributed caching system and method. Background Art

[0002] In the process of business migration container, it is necessary to use kubernetes orchestration tools to deploy container services, istio to manage the relationship between services, and istio gateway as a seven-layer load balancer or gateway to manage the inflow and outflow of traffic. It is also necessary to cache the service data of the business migration container. Currently, a combination of nginx+varnish is usually used to cache HTTP services, or page cache software such as squid is used to implement the cache function. The above-mentioned HTTP service cache method needs to be built separately, and the construction program is complex and costly. Summary of the invention

[0003] The embodiment of the present invention provides a distributed caching system and method based on Istio. In the process of business migration container, by adopting Istio on Kubernetes, dynamic caching of HTTP service content is realized, and the construction program is simple and low-cost.

[0004] To achieve the above objectives, on the one hand, an embodiment of the present invention provides an istio-based distributed cache system, including:

[0005] The istio inlet gateway is used to receive the http request sent by the user during the service migration container, and the http request is used to request data from the current service container; forward the received http request to the cache subsystem, and when the corresponding cache data is not obtained from the cache subsystem or the cache subsystem is not enabled, forward the received http request to the envoy filter; wherein the istio inlet gateway is set in the container;

[0006] The cache subsystem is used to receive the http request forwarded by the istio ingress gateway, search for corresponding cache data based on the http request, and return the found cache data to the user;

[0007] The envoy filter is used to receive the http request forwarded by the istio ingress gateway, and obtain corresponding data from the current container of the business based on the http request.

[0008] On the other hand, an embodiment of the present invention provides a distributed caching method based on istio, including:

[0009] During the process of business migration container, the istio inlet gateway is used to receive the http request sent by the user, and the request is used to request data from the current container of the business; the received http request is forwarded to the cache subsystem, and when the corresponding cache data is not obtained from the cache subsystem or the cache subsystem is not turned on, the received http request is forwarded to the envoy filter; wherein the istio inlet gateway is set in the container;

[0010] receiving the http request forwarded by the istio ingress gateway through the cache subsystem, searching for corresponding cache data based on the http request, and returning the found cache data to the user; or

[0011] The http request forwarded by the istio ingress gateway is received through the envoy filter, and data is obtained from the current container of the business based on the http request.

[0012] The above technical solution has the following beneficial effects: because the istio gateway is between the user and the business migration container of the http request data, in the process of business migration container, the http request is forwarded by using the isito entry gateway on kuberentes. When the cache subsystem is turned on, the http request is forwarded to the cache subsystem. When the cache subsystem does not obtain the corresponding data or the cache subsystem is not turned on, the http request is forwarded to the envoy filter, and the corresponding data is obtained from the current container of the business through the envoy filter. The use of the isito entry gateway can realize the dynamic caching of the http service content, and the construction program is simple and low-cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 It is a schematic diagram of the structure of a distributed cache system based on istio according to an embodiment of the present invention;

[0015] Figure 2 It is a flowchart of a distributed caching method based on istio according to an embodiment of the present invention;

[0016] Figure 3 It is a flow chart of another distributed caching method based on istio according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] The technical terms involved in the present invention are defined as follows:

[0019] Kubernetes: A container orchestration management tool

[0020] Applications on Kubernetes: Services deployed using Kubernetes, such as nginx service, mysql service, etc.

[0021] Kubernetes resources: such as deployment, server, ingress, etc., which are used to deploy services on Kubernetes

[0022] Kubernetes ecosystem software: such as Prometheus monitoring system, Alert Manager alarm system, Helm management tool, etc.

[0023] Istio is an open source software; an open platform for connecting, securing, controlling and observing services. It is currently a popular microservice management software

[0024] Istio gateway is a proxy for the client and server sides of the HTTP protocol with a simple structure.

[0025] Envoy: a proxy software that supports tcp udp http grpc protocol

[0026] Pod: The smallest scheduling unit of Kubernetes, containing one or more containers

[0027] Istio ingress gateway is a proxy for standard HTTP protocol network traffic

[0028] Container: A container image is a lightweight, independent executable package of software that contains everything needed to run it: code, runtime, system tools, system libraries, settings, etc. A container is the running state of a program in a container.

[0029] Cache

[0030] Filter is called filter or interceptor

[0031] Single flight to prevent cache breakdown

[0032] like Figure 1 As shown, in combination with an embodiment of the present invention, a distributed cache system based on istio is provided, including:

[0033] The istio inlet gateway 11 is used to receive the http request sent by the user during the business migration container, and the http request is used to request data from the current container of the business; forward the received http request to the cache subsystem, and when the corresponding cache data is not obtained from the cache subsystem or the cache subsystem is not turned on, forward the received http request to the envoy filter 13; wherein the istio inlet gateway 11 is set in the container;

[0034] The cache subsystem 12 is used to receive the http request forwarded by the istio inlet gateway 11, search for corresponding cache data based on the http request, and return the found cache data to the user;

[0035] The envoy filter 13 is used to receive the http request forwarded by the istio inlet gateway 11, and obtain corresponding data from the current container of the business based on the http request.

[0036] Preferably, the cache subsystem 12 includes a cache policy device and at least one cache, wherein the at least one cache is deployed in a distributed manner, and the cache policy device is connected to each of the caches, wherein:

[0037] The istio inlet gateway 11 is used to forward the received http request to the cache policy device;

[0038] The cache policy device is used to receive the http request forwarded by the istio entry gateway 11, determine the target cache where the cache data corresponding to the http request is located, and send the http request to the target cache.

[0039] Preferably, the cache policy device is also used to obtain the header information of the http protocol in the istio inlet gateway 11 before the business migration container, and use the header information of the http protocol as the first key; and in the process of migrating the business container, use the url and url parameters of the http request for accessing the current container of the business as the first key; form corresponding first key-value pairs through each first key; obtain the http protocol version in the istio inlet gateway 11; form a second key according to the first key-value pair and / or the http protocol version; and send the second key to each of the caches;

[0040] The type of the second key includes at least one of the following:

[0041] Using the first key of the first key-value pair as the second key;

[0042] Using the first key-value pair as the second key;

[0043] Match the first key and the corresponding first value of the first key-value pair according to a regular expression, use the first key-value pair that satisfies POSIX regularity as a regular key-value pair, and use the regular key-value pair as a second key;

[0044] The http protocol version in the istio ingress gateway 11 or the http protocol version corresponding to the software implementing the http protocol is used as the second key;

[0045] Use the http protocol method as the second key.

[0046] Preferably, the cache is further used to obtain and save corresponding data from the current business container according to the second key before the business migrates to the container, to form cache data, and use the cache data as the second value corresponding to the second key.

[0047] Preferably, the system further comprises a key access counting module, wherein:

[0048] The target cache is used to receive the http request forwarded by the cache policy device, access the corresponding second key-value pair according to the second key in the http request, and return the second value of the accessed second key-value pair to the user;

[0049] The key access counting module is used to count the number of accesses to the second key according to the second key in the http request after the istio ingress gateway 11 forwards the http request to the target cache;

[0050] Each of the caches comprises:

[0051] A grouping module: used to group the second key according to a preset grouping rule to form multiple key groups before the service migration container, and store the cache data corresponding to the same key group into the same cache data module; maintain the link between the cache data modules through the peer network module peer, and the cache data module includes a hotspot cache data module and a regular cache data module;

[0052] The hotspot cache data module is used to store hotspot cache data, and set the hotspot cache data storage time to a first time threshold; adopt a least-used algorithm to put cache data corresponding to the second key whose access times are less than the hotspot number threshold into a regular cache data module; wherein the hotspot cache data refers to cache data corresponding to the second key whose access times are not less than the hotspot number threshold;

[0053] The conventional cache data module is used to store cache data other than hotspot cache data, and to eliminate cache data corresponding to the second key whose access times are less than the conventional times threshold by adopting the least used algorithm;

[0054] A hash module, for, when the cached data cannot be obtained from the cache according to the http request, updating the second key corresponding to the http request by using a hash algorithm, and dispatching the http request with the updated second key to the same cache to obtain the cached data;

[0055] The peer network module peer is used to complete the transmission of cache data between the hotspot cache data module and the regular cache data module.

[0056] Preferably, it also includes a plug-in wasm, the plug-in wasm is built into the istio entry gateway 11, and the cache subsystem 12 is built into the plug-in wasm;

[0057] The plug-in wasm is used to obtain the http request received by the istio ingress gateway 11 and send the http request to the cache policy device;

[0058] The plug-in wasm is also used to build the cache subsystem 12 according to preset requirements before the business migration container; and during the business migration container, load the newly built cache subsystem 12 in this plug-in wasm.

[0059] Preferably, the istio entry gateway 11 is specifically used to: when no cache data corresponding to the http request is found in all caches or when the cache subsystem 12 is not turned on, only allow the first received http request to be sent to the envoy filter 13 within a preset time period;

[0060] The envoy filter 13 is specifically used to: receive the http request received for the first time by the istio entry gateway 11 within a preset time period, forward the http request to the current container of the business, obtain the corresponding data from the current container of the business and return it to the user; at the same time, save the obtained data in the cache corresponding to the second key of the http request.

[0061] like Figure 2 As shown, in combination with an embodiment of the present invention, a distributed caching method based on istio is provided, including:

[0062] S101: During the process of business migration container, the istio inlet gateway 11 is used to receive the http request sent by the user, and the request is used to request data from the current container of the business; the received http request is forwarded to the cache subsystem 12, and when the corresponding cache data is not obtained from the cache subsystem 12 or the cache subsystem 12 is not turned on, the received http request is forwarded to the envoy filter; wherein the istio inlet gateway is set in the container;

[0063] S102: receiving the http request forwarded by the istio ingress gateway 11 through the cache subsystem 12, searching for corresponding cache data based on the http request, and returning the found cache data to the user; or

[0064] S103: Receive the http request forwarded by the istio ingress gateway 11 through the envoy filter, and obtain data from the current container of the business based on the http request.

[0065] Preferably, the cache subsystem 12 includes a cache policy device and at least one cache, wherein the at least one cache is deployed in a distributed manner, and the cache policy device is connected to each of the caches; wherein:

[0066] In step 101, the istio inlet gateway 11 is used to forward the received http request to the cache subsystem 12, which specifically includes:

[0067] S1011: using the istio ingress gateway 11 to forward the received http request to the cache policy device;

[0068] S1012: The cache strategy receives the http request forwarded by the istio entry gateway 11, determines the target cache where the cache data corresponding to the http request is located, and sends the http request to the target cache.

[0069] Preferably, it also includes:

[0070] S104: Before the business migration container, the cache policy device obtains the header information of the http protocol in the istio inlet gateway 11, and uses the header information of the http protocol as the first key; and in the process of migrating the business container, uses the url and url parameters of the http request for accessing the current container of the business as the first key; forms corresponding first key-value pairs through each first key; obtains the http protocol version in the istio inlet gateway 11; forms a second key according to the first key-value pair and / or the http protocol version; and sends the second key to each of the caches;

[0071] The type of the second key includes at least one of the following:

[0072] Using the first key of the first key-value pair as the second key;

[0073] Using the first key-value pair as the second key;

[0074] Match the first key and the corresponding first value of the first key-value pair according to a regular expression, use the first key-value pair that satisfies POSIX regularity as a regular key-value pair, and use the regular key-value pair as a second key;

[0075] The http protocol version in the istio ingress gateway 11 or the http protocol version corresponding to the software implementing the http protocol is used as the second key;

[0076] Use the http protocol method as the second key.

[0077] Preferably, it also includes:

[0078] S105: Before the service migration container, the cache obtains and stores corresponding data from the current service container according to the second key to form cache data, and uses the cache data as the second value corresponding to the second key.

[0079] Preferably, it also includes:

[0080] S106: before the service migration container, the second key is grouped by the grouping module according to a preset grouping rule to form multiple key groups, and the cache data corresponding to the same key group is stored in the same cache data module; wherein the cache data module includes a hotspot cache data module and a regular cache data module; wherein each of the caches includes: a grouping module, a hotspot cache data module, a regular cache data module, a peer network module peer, and a hash module;

[0081] The istio-based distributed caching method further includes:

[0082] S107: receiving the http request forwarded by the cache policy device through the target cache, accessing the corresponding second key-value pair according to the second key in the http request, and returning the second value of the accessed second key-value pair to the user;

[0083] S108: when the cache data cannot be obtained from the target cache according to the http request, the hash module uses a hash algorithm to update the second key corresponding to the http request, and dispatches the http request with the updated second key to the same cache to obtain the cache data;

[0084] S109: After the istio ingress gateway 11 forwards the http request to the target cache, according to the second key in the http request, the number of accesses to the second key is accumulated and counted by a key access counting module;

[0085] S110: storing cache data corresponding to the second key whose access times are less than the hotspot times threshold in the regular cache data module, and eliminating cache data corresponding to the second key whose access times are less than the regular times threshold by adopting the least used algorithm;

[0086] S112: storing the cache data corresponding to the second key whose access times are not less than the hotspot number threshold as hotspot cache data in the hotspot cache data module, and setting the hotspot cache data storage time to the first time threshold; and after calculating the cache data of the second key whose access times are less than the hotspot number threshold by using the least used algorithm, placing the cache data corresponding to the second key whose access times are less than the hotspot number threshold into the regular cache data module;

[0087] S113: maintaining the links between the cache data modules through the peer network module peer, and completing the transmission of cache data between the hotspot cache data module and the regular cache data module.

[0088] Preferably, it also includes:

[0089] S114: Obtain the http request received by the istio ingress gateway 11 through the plug-in wasm, and send the http request to the cache policy device; wherein the plug-in wasm is built into the istio ingress gateway 11, and the cache subsystem 12 is built into the plug-in wasm;

[0090] The istio-based distributed caching method further includes:

[0091] S115: Before the business migration container, the cache subsystem 12 is built according to the preset requirements through the plug-in wasm; and during the process of business migration container, the newly built cache subsystem 12 is loaded in the plug-in wasm.

[0092] Preferably, when the corresponding data is not obtained from the cache subsystem or the cache subsystem is not enabled, forwarding the received http request to the envoy filter specifically includes:

[0093] When no cache data corresponding to the http request is found in any cache or the cache subsystem is not enabled, only the first received http request is allowed to be sent to the envoy filter within a preset time period;

[0094] The HTTP request forwarded by the istio ingress gateway is received through the envoy filter, and data is obtained from the current container of the business based on the HTTP request, specifically including:

[0095] Receive the http request received by the istio entry gateway for the first time within a preset time period, forward the http request to the current container of the business, obtain the corresponding data from the current container of the business and return it to the user; at the same time, save the obtained data in the cache corresponding to the second key of the http request.

[0096] The above technical solution of the embodiment of the present invention is described in detail below in conjunction with specific application examples. For technical details not introduced during the implementation process, please refer to the relevant description in the previous text.

[0097] In the process of business migration container, you need to use kubernetes orchestration tools to deploy container services, you need to use istio to manage the relationship between services, and you need to use istio gateway as a seven-layer load balancing or gateway to manage the entry and exit of traffic.

[0098] The embodiment of the present invention is a dynamic web page distributed caching system and method based on Istio. The main goal is to implement caching of network requests for HTTP protocol based on the extension mechanism of Istio, and the structure is relatively simple. Because Istiogateway is between the user and the HTTP server, it can manage the user's entry behavior and the HTTP server's exit behavior. Due to the importance of the location of Istio Gateway, the following scenarios can be very effectively optimized in Istio Gateway: such as caching of specific active pages, caching of frequently visited pages, redirection to some pages, etc., which can reduce the pressure on the HTTP server and reduce the use of server hardware resources. At the same time, because there is no need to obtain data from the HTTP server, it can provide users with smaller delays, bring better user experience, and users can see the accessed content faster. The caching of dynamic HTTP service content during business migration containers is realized by Kubernetes+Istio. Specifically, an extension is added to the istio inlet gateway directly under the istio gateway to implement the HTTP request sent to the business migration container to obtain data to the cache subsystem. When the cache subsystem fails to obtain the corresponding data or the cache subsystem is not turned on, the envoy filter is used to collect and prevent penetration to obtain the corresponding data from the business migration container, and the cache subsystem has a group cache.

[0099] like Figure 3 As shown, the system includes multiple components, each of which is responsible for a part of the function.

[0100] a.http client: http client (ie user)

[0101] like Figure 3 As shown in step 1, any browser or command line tool that supports the HTTP protocol is used to initiate an HTTP request. The request reaches the istio ingress gateway (istio ingress gateway, which is a proxy for standard HTTP protocol network traffic). The request is proxied by the istio ingress gateway, which obtains data from the cache subsystem or the HTTP server (the HTTP server belongs to the container of the business being migrated, the current container of the business) according to certain rules, and then returns it to the httpclient.

[0102] b.istio ingress gateway: (istio ingress gateway)

[0103] 1. As Figure 3As shown in step 2, if the ingress gateway determines that the cache subsystem is enabled for the request of the http client, the data is obtained through the cache subsystem through the wasm filter. If the cache subsystem is not enabled, as shown in step 3, the data is obtained through the envoy filter in step 3. When isito is configured, the wams filer component is configured for the virtual service of istio, and the web dynamic cache filter is configured. Whether to enable the cache subsystem is specified during configuration. The http request of the http client refers to initiating an http request, the http protocol, etc.

[0104] 2. Provide http service proxy function, realize http routing function, support seven-layer http load balancing, convert istio's gateway resources, virtual service resources, target rule resources, service entry resources, workload group resources, workload entry resources, etc., and combine the above resources into legal http routes, etc.

[0105] 3. By supporting the configuration of filters, multiple filters can be freely combined into a filter chain, which is bound to the http route to manage the http requests and responses of the http route.

[0106] c.Envoy filter: It is a set of filters of Envoy software (built-in components in Istio ingressgateway).

[0107] like Figure 3 As shown in step 3, an http request is received from the upper layer, and corresponding data is obtained from the current container of the business based on the http request.

[0108] Envoy filters can be composed of multiple filters or one filter. Envoy filters include log filters, monitoring filters, tracing filters, TCP filters, and other built-in filters of Envoy software. These filters are combined with each other to form a set of filter chains that meet business needs. This set of filter chains completes the processing of HTTP services. Filterchain is a set of filters, and HTTP requests will be processed once by all filters in the filter chain.

[0109] d.wasm filter (plugin wasm):

[0110] Envoy filter is built-in in the Istio ingress gateway software. Wasm filter is a type of envoyfilter and an extension mechanism of envoy filter. Based on this extension mechanism, a policy cache (web dynamic cache filter) and a cache subsystem are developed; that is, the policy cache and the cache subsystem are built-in in the plug-in wasm. However, the wasm filter and the envoy filter pointed to in step 3 are completely parallel.

[0111] like Figure 3 As shown in step 2, the http request arrives at the wasm filer from the istio ingress gateway, and sends the http request to the cache policy:

[0112] 1. It is used to freely customize a functional component to support some functions that envoy filter does not have and process HTTP traffic in some specific application scenarios.

[0113] 2.wasm filter supports multiple development languages ​​to develop customized filters as required.

[0114] 3. Compared with envoy filter, it can use any development language to develop filters, and can dynamically load the developed cache subsystem in real time without restarting the service.

[0115] 4.wasm filter is a development framework, that is, it provides a framework for developing actual functions, allowing users to develop their own actual functions as needed. For example, before the business migration container, develop a cache subsystem and web dynamic cache filter (web dynamic cache filter is the core component of the cache function).

[0116] e.web dynamic cache filter: policy cache.

[0117] like Figure 3 As shown in step 5, the http request is received from the previous step, the cache policy receives the http request forwarded by the istio inlet gateway to the wasm filter, and then the wasm filter forwards it, and determines the target cache where the cache data corresponding to the http request is located, and sends the http request to the target cache.

[0118] 1. Customize cache rules according to the http protocol

[0119] 2. It is a filter developed on the basis of wasm filter. It must rely on wasm filter and develop web dynamic cache filter on the wasmfilter framework.

[0120] 3. Use golang language to develop this component

[0121] 4. The cache rules are as follows:

[0122] (1) Customize the first key according to the header information of the http request. For example, all http requests with the key testa in the header are cached. For example, all requests with the key testa and the value testb in the header hit the cache. testa represents the user. For example, if the header key contains testa and testb, and all requests with the values ​​of testa and testb are test1 and test2, they hit the cache. This means that these two keys can obtain data from the cache system, the cache system stores the data, and it has not expired. The corresponding first key-value pairs are formed by each first key. The header is a string sent by the server before transmitting HTML data to the browser using the HTTP protocol. There must be a blank line between the header and the HTML file.

[0123] (2) Customize the cache strategy based on the key and value in the header of step (1), that is, form a second key;

[0124] Rule 1: Use the first key of the first key-value pair as the second key of the second key-value pair.

[0125] Rule 2: Use the first key-value pair as the second key of the second key-value pair.

[0126] Rule 3: Match the first key and the first value of the first key-value pair according to the POSIX regular expression, and use the first key-value pair that satisfies the regular expression (POSIX regular expression) as the second key of the second key-value pair.

[0127] Rule 4, matches the http protocol version as the second key of the second key-value pair, and uses the http protocol version in the istio ingress gateway 11 or the http protocol version corresponding to the software that implements the http protocol as the second key; wherein, the http protocol is an open Internet text protocol, and any software that implements the http protocol follows this standard; the http protocol versions are divided into http1.0 http1.1 http2.0.

[0128] Rule 5: Use the method of the http protocol as the second key of the second key-value pair, where the http protocol method refers to the get, post, and head methods.

[0129] That is, the second key is determined based on various standard headers in the http protocol, the first key composed of the customized header native first key, the url and url parameters of the http request, and various valid http protocol contents.

[0130] (3) Through the five methods for determining the second key in (2), logical operations are used to meet the requirements of mixed use of various parts of the http protocol to form a key in multiple scenarios, and the second key in the second key value is obtained. The second key is sent to each of the caches, and the cache of the cache subsystem is constructed through the second key.

[0131] f.cache system: cache subsystem

[0132] like Figure 3 As shown in step 6, the http request is received from the previous step, the corresponding cache data is searched based on the http request, and the found cache data is returned to the user.

[0133] 1. The main function of this component is to use the second key passed by the policy cache, and before the business migration container, obtain and save the corresponding data from the current business container according to the second key to form cache data, and use the cache data as the second value corresponding to the second key.

[0134] 2. The cache subsystem includes a cache policy device and at least one cache. The at least one cache is deployed in a distributed manner. The cache policy device is connected to each cache. The http request forwarded by the cache policy device is received through the target cache. The corresponding second key-value pair is accessed according to the second key in the http request, and the second value of the accessed second key-value pair is returned to the user.

[0135] Each cache includes: a hot cache data module (hot cache part), a regular cache data module (cache part), a peer network module peer, a hash module, a grouping module, etc.; wherein, the cache subsystem also includes: a key access counting module.

[0136] hotcache part: used to store hotspot cache data, and set the hotspot cache data storage time to a first time threshold; adopt a least-used algorithm to put cache data corresponding to the second key whose access times are less than the hotspot number threshold into a regular cache data module; wherein, the hotspot cache data refers to cache data corresponding to the second key whose access times are not less than the hotspot number threshold.

[0137] Cache part: stores cache data other than hotspot cache data, and adopts the least used algorithm to eliminate cache data corresponding to the second key whose access times are less than the normal number threshold.

[0138] Key access counting module: After the istio entry gateway 11 forwards the http request to the target cache, the number of accesses to the second key in the http request is cumulatively counted.

[0139] Peer network module peer: completes the communication between caches of multiple cache systems, maintains the link relationship, and performs remote query data. Completes the transmission of cache data between hotspot cache data module and regular cache data module.

[0140] Hash module: When the cached data cannot be obtained from the cache according to the http request, the second key corresponding to the http request is updated by the hash algorithm, and the http request with the updated second key is dispatched to the same cache to obtain the cached data. It is ensured that the data obtained from the remote by each key is always on a peer node, which is actually to obtain the cached content by using the key.

[0141] Grouping module: Before the business migration container, the second key is grouped according to the preset grouping rules to form multiple key groups, and the cache data corresponding to the same key group is stored in the same cache data module; the link between the cache data modules is maintained through the peer network module peer, and the cache data module includes a hotspot cache data module and a regular cache data module. That is, the second key of the cache data is artificially grouped and classified. Put similar second keys together, or put the same type of business together, etc., artificial specific divisions meet different usage scenarios. Putting similar keys in the same cache group can improve data query efficiency, first check the group where the key is located, and then check the data from the group. If there is no group, directly checking all caches will cause a large amount of irrelevant cache data to be searched, affecting efficiency.

[0142] 3. Maintain the peer link relationship between data module groups, synchronize data, keep the hot data in the hot cache updated, and decide whether to put it in the hot cache based on the number of accesses to the cache key.

[0143] 4. When the cache data cannot be obtained from the cache according to the http request, the second key corresponding to the http request is updated by a hash algorithm, and the http request with the updated second key is dispatched to the same cache to obtain the cache data. That is, if there is no cache data locally, one is selected from the peer IP to obtain the data. The peer IP refers to the IP of the remote address relative to the local system, and the remote and local together constitute a distributed cache system.

[0144] 5. The anti-penetration module Sigle flight is used to allow only the first received http request to penetrate the envoy filter to the current container of the business within a preset time period when the cache data corresponding to the http request is not found in all caches or when the cache subsystem is not turned on, and to obtain the corresponding data from the current container of the business and return it to the user; at the same time, the obtained data is stored in the cache corresponding to the second key of the http request. That is, if there is no cache locally and the cache data is not obtained through the peer, only one http request within the preset time is allowed to penetrate the envoy filter to the back end (current container of the business) to obtain data, in order to prevent a large number of accesses to the back end http service when there is no cache data, thereby causing the service to crash. Specifically explained as follows: an http request refers to a request initiated by an http client to request the back end of the http server, and the allowed http request refers to the first one to reach the back end of the server, and if the requested data does not exist in the cache, then this http request needs to be passed to the real back end of the back end http server, that is, the module that actually provides the service to obtain data, and return the obtained data to the http client, and at the same time, a copy of the data is also stored in the cache, so that the next same request can directly obtain data from the cache.

[0145] g.Pod (sidecar):

[0146] like Figure 3 As shown in steps 4 and 7, receive the http request and return the corresponding data:

[0147] 1. Pod is the smallest unit that carries the http service and is the current container of the business. A group of pods with sidecar constitutes an http service, which is used to provide web services for Internet users to access and obtain web pages and other data.

[0148] 2. Sidecar provides the proxy function of the http service. You can choose to enable sidecar or not. When sidecar is enabled, it has the filter chain capability to process http traffic. After enabling it, it will affect the overall performance of the http service. When it is not enabled, it does not have the filter chain capability and has the best http response performance.

[0149] The beneficial effects achieved by the embodiments of the present invention are as follows:

[0150] 1. Support Kubernetes + Isito to dynamically cache the HTTP service content in the business migration container. When the business migrates the container, it can quickly and easily cache the HTTP service content provided by Kubernetes and Istio where the business is located. However, the existing technology based on Kubernetes and Istio has no dynamic content caching, and cannot support Kubernetes + Isito to achieve dynamic HTTP service content caching.

[0151] 2. Support adding extensions on the istio gateway, define the behavior of the istio ingress gateway through the istio gateway configuration, and realize the coordinated use of the istio ingress gateway and the istio gateway.

[0152] 3. Implement the group cache function based on the http service provided by kubernetes and istio.

[0153] 4. The http cache service provided by kubernetes and istio can realize the anti-penetration function of the current container of the business.

[0154] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0155] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0156] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0157] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0158] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0159] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0160] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0161] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and blue-ray disk. Disks usually copy data magnetically, while discs usually copy data optically with lasers. The above combination can also be included in computer readable media.

[0162] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A distributed cache system based on istio, characterized in that: include: The istio ingress gateway is used to receive HTTP requests sent by users during the process of business migration containers. The HTTP requests are used to request data from the current container of the business. forwarding the received http request to the cache subsystem, and forwarding the received http request to the envoy filter if the corresponding cache data is not obtained from the cache subsystem or the cache subsystem is not enabled; wherein the istio ingress gateway is located in the container; The cache subsystem is used to receive the http request forwarded by the istio ingress gateway, search for corresponding cache data based on the http request, and return the found cache data to the user; The envoy filter is used to receive the http request forwarded by the istio ingress gateway, and obtain corresponding data from the current container of the business based on the http request; The system further includes a plug-in wasm, the plug-in wasm is built into the istio ingress gateway, and the cache subsystem is built into the plug-in wasm; The plug-in wasm is used to obtain the http request received by the istio ingress gateway and send the http request to the cache subsystem; The plug-in wasm is also used to build the cache subsystem according to preset requirements before the business migration container; and during the business migration container, load the newly built cache subsystem in this plug-in wasm.

2. The istio-based distributed cache system according to claim 1, characterized in that: The cache subsystem includes a cache policy device and at least one cache, wherein the at least one cache is deployed in a distributed manner, and the cache policy device is connected to each of the caches, wherein: The istio ingress gateway is used to forward the received http request to the cache policy device; The cache policy device is used to receive the http request forwarded by the istio entry gateway, determine the target cache where the cache data corresponding to the http request is located, and send the http request to the target cache.

3. The istio-based distributed cache system according to claim 2, characterized in that: The cache policy device is also used to obtain the header information of the http protocol in the istio inlet gateway before the business migration container, and use the header information of the http protocol as the first key; and in the process of migrating the business container, use the url and url parameters of the http request for accessing the current container of the business as the first key; Form corresponding first key-value pairs respectively through each first key; obtain the http protocol version in the istio inlet gateway; form a second key according to the first key-value pair and / or the http protocol version; send the second key to each cache; The type of the second key includes at least one of the following: Using the first key of the first key-value pair as the second key; Using the first key-value pair as the second key; Match the first key and the corresponding first value of the first key-value pair according to a regular expression, use the first key-value pair that satisfies POSIX regularity as a regular key-value pair, and use the regular key-value pair as a second key; Use the http protocol version in the istio ingress gateway or the http protocol version corresponding to the software that implements the http protocol as the second key; Use the http protocol method as the second key.

4. The istio-based distributed cache system according to claim 3, characterized in that: The cache is further used to obtain and save corresponding data from the current business container according to the second key before the business migrates to the container, to form cache data, and use the cache data as the second value corresponding to the second key.

5. The istio-based distributed cache system according to claim 4, characterized in that: The system further comprises a key access counting module, wherein: The target cache is used to receive the http request forwarded by the cache policy device, access the corresponding second key-value pair according to the second key in the http request, and return the second value of the accessed second key-value pair to the user; The key access counting module is used to accumulate and count the number of accesses to the second key according to the second key in the http request after the istio ingress gateway forwards the http request to the target cache; Each of the caches comprises: A grouping module: used to group the second key according to a preset grouping rule to form multiple key groups before the service migration container, and store the cache data corresponding to the same key group into the same cache data module; maintain the link between the cache data modules through the peer network module peer, and the cache data module includes a hotspot cache data module and a regular cache data module; The hotspot cache data module is used to store hotspot cache data, and set the hotspot cache data storage time to a first time threshold; adopt a least-used algorithm to put cache data corresponding to the second key whose access times are less than the hotspot number threshold into a regular cache data module; wherein the hotspot cache data refers to cache data corresponding to the second key whose access times are not less than the hotspot number threshold; The conventional cache data module is used to store cache data other than hotspot cache data, and to eliminate cache data corresponding to the second key whose access times are less than the conventional times threshold by adopting the least used algorithm; A hash module, for, when the cached data cannot be obtained from the cache according to the http request, updating the second key corresponding to the http request by using a hash algorithm, and dispatching the http request with the updated second key to the same cache to obtain the cached data; The peer network module peer is used to complete the transmission of cache data between the hotspot cache data module and the regular cache data module.

6. The istio-based distributed cache system according to claim 5, characterized in that: The istio ingress gateway is specifically used to: when no cache data corresponding to the http request is found in all caches or when the cache subsystem is not enabled, only allow the first received http request to be sent to the envoy filter within a preset time period; The envoy filter is specifically used to: receive the http request received by the istio entry gateway for the first time within a preset time period, forward the http request to the current container of the business, obtain the corresponding data from the current container of the business and return it to the user; at the same time, save the obtained data in the cache corresponding to the second key of the http request.

7. A distributed caching method based on istio, characterized in that: include: During the process of business migration container, the istio ingress gateway is used to receive the http request sent by the user, which is used to request data from the current container of the business; forwarding the received http request to the cache subsystem, and forwarding the received http request to the envoy filter when the corresponding cache data is not obtained from the cache subsystem or the cache subsystem is not enabled; wherein the istio ingress gateway is arranged in the container, the plug-in wasm is built into the istio ingress gateway, the cache subsystem is built into the plug-in wasm, the http request received by the istio ingress gateway is obtained through the plug-in wasm, and the http request is sent to the cache subsystem; receiving the http request forwarded by the istio ingress gateway through the cache subsystem, searching for corresponding cache data based on the http request, and returning the found cache data to the user; or Receive the http request forwarded by the istio ingress gateway through the envoy filter, and obtain data from the current container of the business based on the http request; The istio-based distributed caching method further includes: Before the business migration container, the cache subsystem is built according to preset requirements through the plug-in wasm; and during the business migration container, the newly built cache subsystem is loaded in this plug-in wasm.

8. The distributed caching method based on istio according to claim 7, characterized in that: The cache subsystem includes a cache policy device and at least one cache, wherein the at least one cache is deployed in a distributed manner, and the cache policy device is connected to each of the caches; wherein: The istio ingress gateway is used to forward the received http request to the cache subsystem, specifically including: The istio ingress gateway is used to forward the received http request to the cache policy device; The cache strategy receives the http request forwarded by the istio entry gateway, determines the target cache where the cache data corresponding to the http request is located, and sends the http request to the target cache.

9. The distributed caching method based on istio according to claim 8, characterized in that: Also includes: Before the service migration container, the cache policy obtains the header information of the http protocol in the istio ingress gateway and uses the header information of the http protocol as the first key; And, in the process of business migration container, the url and url parameters of the http request to access the current container of the business are used as the first key; Form corresponding first key-value pairs respectively through each first key; obtain the http protocol version in the istio inlet gateway; form a second key according to the first key-value pair and / or the http protocol version; send the second key to each cache; The type of the second key includes at least one of the following: Using the first key of the first key-value pair as the second key; Using the first key-value pair as the second key; Match the first key and the corresponding first value of the first key-value pair according to a regular expression, use the first key-value pair that satisfies POSIX regularity as a regular key-value pair, and use the regular key-value pair as a second key; Use the http protocol version in the istio ingress gateway or the http protocol version corresponding to the software that implements the http protocol as the second key; Use the http protocol method as the second key.

10. The distributed caching method based on istio according to claim 9, characterized in that: Also includes: Before the service migration container, the cache obtains and stores corresponding data from the current service container according to the second key to form cache data, and uses the cache data as the second value corresponding to the second key.

11. The distributed caching method based on istio according to claim 10, characterized in that: Also includes: Before the service migration container, the second key is grouped according to a preset grouping rule by a grouping module to form multiple key groups, and the cache data corresponding to the same key group is stored in the same cache data module; wherein the cache data module includes a hotspot cache data module and a regular cache data module; wherein each of the caches includes: a grouping module, a hotspot cache data module, a regular cache data module, a peer network module peer, and a hash module; The istio-based distributed caching method further includes: receiving, through the target cache, the http request forwarded by the cache policy device, accessing a corresponding second key-value pair according to a second key in the http request, and returning a second value of the accessed second key-value pair to the user; When the cache data cannot be obtained from the target cache according to the http request, the hash module uses a hash algorithm to update the second key corresponding to the http request, and dispatches the http request with the updated second key to the same cache to obtain the cache data; After the istio ingress gateway forwards the http request to the target cache, according to the second key in the http request, the number of accesses to the second key is accumulated and counted by a key access counting module; The cache data corresponding to the second key whose access times are less than the hotspot times threshold value is stored in the regular cache data module, and the cache data corresponding to the second key whose access times are less than the regular times threshold value is eliminated by adopting the least used algorithm; The cache data corresponding to the second key whose access times are not less than the hotspot number threshold is stored as hotspot cache data in the hotspot cache data module, and the hotspot cache data storage time is set to the first time threshold; and after the cache data of the second key whose access times are less than the hotspot number threshold is calculated by using the least used algorithm, the cache data corresponding to the second key whose access times are less than the hotspot number threshold is placed in the regular cache data module; The links between the cache data modules are maintained through the peer network module peer, and the transmission of cache data between the hotspot cache data module and the regular cache data module is completed.

12. The distributed caching method based on istio according to claim 11, characterized in that: When the corresponding data is not obtained from the cache subsystem or the cache subsystem is not enabled, forwarding the received http request to the envoy filter specifically includes: When no cache data corresponding to the http request is found in any cache or the cache subsystem is not enabled, only the first received http request is allowed to be sent to the envoy filter within a preset time period; The HTTP request forwarded by the istio ingress gateway is received through the envoy filter, and data is obtained from the current container of the business based on the HTTP request, specifically including: Receive the http request received by the istio entry gateway for the first time within a preset time period, forward the http request to the current container of the business, obtain the corresponding data from the current container of the business and return it to the user; at the same time, save the obtained data in the cache corresponding to the second key of the http request.

Images