Dpu service mesh distributed throttling method and device, equipment and medium

Abstract

The embodiment of the disclosure relates to a DPU service grid distributed flow limiting method and device, equipment and medium; each Sidecar agent corresponding to each service on a host is unloaded, and a centralized agent corresponding to each service is installed on a DPU, and the method comprises the following steps: the centralized agent receives a service request and obtains a target service; whether flow limiting is performed is determined based on the target service and a pre-cached flow limiting rule; in the case of performing flow limiting, a control plane component on the host requests a distributed lock corresponding to the target service from Kubernetes; when the centralized agent receives the distributed lock sent by the control plane component, a current remaining flow limiting quota of the distributed lock is obtained; and when the request traffic of the service request is less than or equal to the current remaining flow limiting quota, the service request is responded. Therefore, flow limiting execution is unloaded to the DPU, the burden of the host CPU is reduced, flow limiting processing is performed through the centralized agent, and the processing efficiency and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of cloud computing, and particularly relates to a DPU (Data Processing Unit) service grid distributed rate limiting method and device, equipment and medium. BACKGROUND

[0002] Generally, in a complex micro-service system, the control and management of service traffic is crucial. Rate limiting is an important means to protect services from overload and abuse, by limiting the number of requests or concurrent connections per unit of time, to prevent services from being overwhelmed by surges in traffic, and to ensure the stability and availability of services.

[0003] Traditional rate limiting methods are usually implemented by the service application itself or its front API (Application Programming Interface) gateway. In a service grid, the Sidecar (a special transparent network proxy such as Envoy) proxy is naturally suitable as the execution point of the rate limiting policy, as it is on the service communication path and can flexibly control all traffic in and out of the service.

[0004] However, in the distributed architecture of the service grid, the Sidecar proxy of each service instance usually works independently, lacking global coordination. When the traffic of multiple service replicas converges to a downstream service, the actual traffic may exceed the expected value. SUMMARY

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a DPU service grid distributed rate limiting method, device, equipment and medium.

[0006] The present disclosure provides a DPU service grid distributed rate limiting method, which offloads each Sidecar proxy corresponding to each service on a host, and installs a centralized proxy corresponding to each service on a DPU. The method comprises:

[0007] The centralized proxy receives a service request and obtains a target service;

[0008] Based on the target service and a pre-cached rate limiting rule, it is determined whether to perform rate limiting;

[0009] In the case of performing rate limiting, a control plane component on the host requests a distributed lock corresponding to the target service from a container orchestration platform Kubernetes;

[0010] The centralized agent receives the distributed lock sent by the control plane component, and acquires a current remaining throttling quota of the distributed lock;

[0011] When the request traffic of the service request is less than or equal to the current remaining throttling quota, the service request is responded to.

[0012] Embodiments of the present disclosure also provide a DPU service mesh distributed throttling device, each Sidecar agent corresponding to each service on a host is offloaded, and a centralized agent corresponding to each service is installed on a DPU, and the device comprises:

[0013] A receiving and acquiring module is configured to receive a service request by the centralized agent, and acquire a target service;

[0014] A determining module is configured to determine whether to perform throttling based on the target service and a pre-cached throttling rule;

[0015] A requesting module is configured to request, in a case of performing throttling, a distributed lock corresponding to the target service from a container orchestration platform Kubernetes through a control plane component on the host;

[0016] An acquiring module is configured to acquire, when the centralized agent receives the distributed lock sent by the control plane component, a current remaining throttling quota of the distributed lock;

[0017] A responding module is configured to respond to the service request when the request traffic of the service request is less than or equal to the current remaining throttling quota.

[0018] Embodiments of the present disclosure also provide an electronic device, which comprises a processor, a memory for storing executable instructions of the processor, and the processor is configured to read the executable instructions from the memory and execute the instructions to implement the DPU service mesh distributed throttling method provided by the embodiments of the present disclosure.

[0019] Embodiments of the present disclosure also provide a computer readable storage medium, which stores a computer program, and the computer program is used to execute the DPU service mesh distributed throttling method provided by the embodiments of the present disclosure.

[0020] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art: the distributed current limiting solution based on the DPU service grid provided by the embodiments of the present disclosure uninstalls the Sidecar agents corresponding to the various services on the host, and installs a centralized agent corresponding to the various services on the DPU, and the method includes: the centralized agent receives the service request and obtains the target service; determines whether to perform current limiting based on the target service and the pre-cached current limiting rules; when performing current limiting, requests the distributed lock corresponding to the target service from Kubernetes through the control plane component on the host; when the centralized agent receives the distributed lock sent by the control plane component, it obtains the current remaining current limiting quota of the distributed lock; when the request traffic of the service request is less than or equal to the current remaining current limiting quota, responds to the service request. Therefore, the DPU hardware offloads the Sidecar agent. In traditional service mesh solutions, the Sidecar agent is deployed on the same host as the application, occupying a large amount of CPU and memory resources and affecting the performance of the application. By offloading the Sidecar agent to the smart network card DPU, the DPU's high-performance network processing capabilities are used to perform traffic management and flow control functions, which greatly reduces the host's resource overhead and improves the performance and scalability of the service mesh. In addition, requesting distributed locks through the control plane components on the host side ensures the consistency and accuracy of flow control, and the centralized agent's flow control rule management also simplifies the update and distribution of flow control policies, improving the scalability of the service mesh. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0022] Figure 1A An example diagram of independent processing current limiting based on a DPU provided in an embodiment of the present disclosure;

[0023] Figure 1B Another example diagram of independent processing current limiting based on DPU provided in an embodiment of the present disclosure;

[0024] Figure 2 A schematic diagram of a distributed throttling method based on a DPU service grid provided in an embodiment of the present disclosure;

[0025] Figure 3 A flow chart of another distributed current limiting method based on a DPU service grid provided in an embodiment of the present disclosure;

[0026] Figure 4 This is an example diagram of distributed throttling based on a DPU service grid provided in an embodiment of the present disclosure;

[0027] Figure 5 Another example diagram of DPU service mesh distributed flow limiting based on the embodiment of the present disclosure is provided;

[0028] Figure 6 A structural schematic diagram of a DPU service mesh distributed flow limiting device provided by the embodiment of the present disclosure is provided;

[0029] Figure 7 A structural schematic diagram of an electronic device provided by the embodiment of the present disclosure is provided. DETAILED DESCRIPTION

[0030] Embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, but rather, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the scope of protection of the present disclosure.

[0031] It should be understood that each step described in the method embodiment of the present disclosure can be executed in different order and / or in parallel. In addition, the method embodiment can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0032] The term "comprising" and variations thereof as used herein are open-ended, that is "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions will be given in the description below.

[0033] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0034] It should be noted that the modification of "one" or "multiple" mentioned in the present disclosure is illustrative and not limiting, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0035] The names of the messages or information exchanged between the plurality of devices in the embodiment of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0036] A service mesh is an architectural pattern used to manage inter-service communication in microservice applications. It provides a transparent, reliable, and flexible service communication mechanism by decoupling service communication functionality from application code and implementing it as a separate infrastructure layer. A service mesh typically consists of a series of lightweight network proxies (such as Envoy) that are deployed alongside application services to take over their network communication, implementing functions such as service discovery, load balancing, traffic management, security authentication, observability, and more.

[0037] The present disclosure unloads the Sidecar proxy in the service mesh from the host where the application is located to the intelligent network card (such as DPU) is an emerging hardware acceleration technology. DPU is a programmable intelligent network card with powerful network processing and computing capabilities. By unloading the Sidecar proxy from the host to the DPU, the CPU and memory load of the host can be reduced, and valuable computing resources can be saved for the application itself. At the same time, the hardware acceleration and dedicated processing provided by the DPU can greatly improve the network performance and throughput of the proxy, and achieve more efficient service communication.

[0038] It can be understood that under the distributed architecture of the service mesh, the Sidecar proxy of each service instance usually works independently, lacking global coordination. When the traffic of multiple service replicas converges to a downstream service, the actual traffic may exceed the expected value.

[0039] To solve the above problems, a centralized throttling mechanism is needed to coordinate the throttling behavior of all distributed proxies. Kubernetes, as a mature container orchestration platform, provides etcd distributed key-value storage, which is suitable as a centralized throttling rule configuration center. Through the distributed lock and transaction mechanism of etcd, the allocation and use of throttling quotas among all proxies can be safely coordinated, thereby achieving globally consistent throttling effect.

[0040] Among them, Kubernetes is a widely used container orchestration platform that provides a series of mechanisms to manage and coordinate large-scale containerized applications. Among them, the distributed lock etcd is a key component of Kubernetes, used as the implementation basis for core functions such as cluster configuration storage and distributed lock.

[0041] Among them, etcd is a highly available distributed key-value store system that uses the Raft consensus algorithm to ensure strong data consistency and fault tolerance. Kubernetes uses etcd to persist all cluster configuration information, including the status of various resources such as nodes, services, pods, and network policies. In addition to being a reliable storage, etcd also provides a distributed locking function, ensuring that only one of multiple competitors can obtain a lock and access a shared resource at a time until the lock is released or the session times out. This distributed locking mechanism is widely used in various distributed system scenarios that require global coordination and mutually exclusive resource access.

[0042] The distributed current limiting method based on the DPU service grid provided by the embodiments of the present disclosure utilizes the distributed lock feature of Kubernetes etcd to solve the problem of traffic exceeding the limit caused by independent current limiting of multiple centralized agents, and realizes precise global current limiting control.

[0043] Specifically, the current limiting rule configuration in this solution is issued and dynamically updated by the agent component (control plane component) of the host, maintaining a certain degree of decentralization and flexibility. When the current limiting action is actually performed, the centralized agent will request an etcd distributed lock from the Kubernetes cluster on the host side. When multiple agents request concurrently, only one agent can successfully obtain the lock, and the other agents need to wait or retry. The agent that obtains the lock will query the current global current limiting quota usage, make a decision to release or reject the request based on the size of the requested traffic, and release the lock after completing the processing, and update the global quota information. Due to the mutual exclusion of the etcd lock, at most only one agent can process the request and consume the current limiting quota at the same time, thus avoiding the problem of total traffic exceeding the limit caused by the convergence of multiple inlet traffic.

[0044] Specifically, the problem of total traffic exceeding the limit caused by the convergence of traffic from multiple ingress points. In a distributed deployment environment of a service mesh, the same service may have multiple instances running on different nodes, each with a corresponding centralized proxy. These proxies independently limit ingress traffic. When traffic from multiple ingress points converges on the same service instance, the actual total traffic may exceed the preset limit threshold, affecting service stability.

[0045] For example, Figure 1A As shown in the figure, you can see that the access of Node1's Pod-A service to Node2's Pod-B service is restricted by the Node1 DPU SOC (System On Chip, the operating system deployed on the DPU) side agent. Figure 1BAs shown, the diagram shows that the external traffic accessing the Pod-B service of Node2 is limited by the Node2 DPU SOC side proxy. Assume there is a service mesh, where the instance Pod-B of service B is deployed on Node2. The centralized proxy of service B sets a traffic limit of 100MB / s. Now there are two ingress traffics accessing service B simultaneously: Pod-A on Node1 initiates a request to service B, and the traffic of the request is 100MB / s. At the same time, an external user request also directly accesses service B through the proxy on Node2, and the traffic is also 100MB / s.

[0046] Specifically, the working steps are as follows: 1. The request of Pod-A first reaches the centralized proxy on Node1. 2. The proxy on Node1 limits the traffic of the request and finds that the traffic is 100MB / s, which is equal to the limit threshold, so the request is released. 3. The request of Pod-A is forwarded to the instance Pod-B on Node2 through the routing mechanism of the service mesh. 4. At the same time, the request of the external user reaches the ingress gateway of Node2 and is forwarded to the centralized proxy of Node2. 5. The proxy of Node2 also limits the request independently and releases the traffic of 100MB / s. 6. The external request is also forwarded to the instance Pod-B.

[0047] Therefore, in the above process, although the proxies of Node1 and Node2 correctly perform the limit of 100MB / s, since they are processed independently, the traffics of the two entrances are both released, and the total traffic converging on the instance Pod-B reaches 200MB / s, which significantly exceeds the preset limit threshold of 100MB / s, which may cause the instance Pod-B to be overloaded or unstable.

[0048] Among them, Pod is the smallest deployment and management unit in kubernetes cluster, and is cooperatively addressed and cooperatively scheduled. Pod is a collection of one or more containers, and is an abstract collection of one or a group of services (processes). Pods can share networks and storage (which can be simply understood as a logical virtual machine, but is not a virtual machine); bridge is a network bridge, agent is a control plane component, and docker is an open source application container engine.

[0049] Specifically, Figure 2A flowchart of a DPU service grid distributed flow limiting method provided by an embodiment of the present disclosure is shown in the figure. The method can be executed by a DPU service grid distributed flow limiting device. The device can be implemented by software and / or hardware, and can be integrated in an electronic device. Each Sidecar agent corresponding to each service on the host is offloaded, and a centralized agent corresponding to each service is installed on the DPU, as shown in Figure 2 The method includes the following steps.

[0050] In step 101, the centralized agent receives a service request and obtains a target service.

[0051] Specifically, one or more Pods in Kubernetes can simultaneously initiate a request for the same service or multiple services. Therefore, the centralized agent receives a service request, which can include service identification and other information. By analyzing the service request, the target service can be obtained. For example, in the foregoing Figure 1A and Figure 1B Pod-B of service B is deployed on Node2, and Pod-A on Node1 initiates a service request for service B. It can be determined that the target service is service B.

[0052] In step 102, it is determined whether to perform flow limiting based on the target service and pre-cached flow limiting rules.

[0053] In the embodiment of the present disclosure, a centralized control plane component is deployed on the host side of the Kubernetes cluster, which is responsible for monitoring the changes of services, dynamically generating and issuing flow limiting rules to the centralized agent of the DPU.

[0054] Specifically, the control plane component monitors the dynamic changes of services in the Kubernetes cluster, including the creation, destruction, scaling, and other events of services. According to the changes of services and the preset flow limiting strategy, the flow limiting rules for each service are dynamically generated and updated, and the flow limiting rules are issued to the centralized agent on the DPU SOC side corresponding to the service instance for local execution of flow limiting.

[0055] Therefore, after obtaining the target service, the target service identification is matched with the flow limiting rules, and it is determined whether to perform flow limiting based on the matching result.

[0056] Specifically, the flow control strategy of each centralized agent and the flow control rule can be different. The flow control rule usually includes service identification, using the name, namespace, and other unique identifiers of the service; flow control threshold, specifying the maximum flow control limit of the service, such as the number of requests per second (QPS, Queries-Per-Second) or the amount of data per second (BPS, Bits Per Second); flow control algorithm, specifying the implementation algorithm of flow control, such as token bucket, leaky bucket, etc.; flow control behavior, specifying the processing behavior when the threshold is exceeded, such as rejecting requests, queuing, etc.; priority, specifying the priority of the service, used to determine the order of flow control when resources are insufficient.

[0057] For example, assuming that a service named "my-service" is created in the Host-side K8s cluster, the control plane component listens to the events of the Kubernetes API Server and discovers that a new service is created, which can generate the following flow control rule: {"service":"my-service","namespace":"default","maxQPS":1000,"algorithm":"token-bucket","behavior":"reject","priority":"medium"},which specifies that the maximum QPS of "my-service" is 1000, uses the token bucket algorithm, and directly rejects requests when the limit is exceeded, with a medium priority.

[0058] Step 103, in the case of executing flow control, the control plane component on the host requests the distributed lock corresponding to the target service from the container orchestration platform Kubernetes.

[0059] Step 104, when the centralized agent receives the distributed lock sent by the control plane component, the current remaining flow control quota of the distributed lock is obtained.

[0060] Step 105, in the case where the request flow of the service request is less than or equal to the current remaining flow control quota, the service request is responded.

[0061] In the embodiments of the present disclosure, in the case of determining to execute flow control, the control plane component on the host requests the distributed lock corresponding to the target service from the container orchestration platform Kubernetes, and if the centralized agent successfully obtains the distributed lock, i.e., the etcd lock, it means that it is the globally unique agent authorized to handle the flow control of the service. It will query the current remaining flow control quota recorded in the etcd.

[0062] Further, the centralized agent compares the request flow with the current remaining flow control quota, and if the request flow is less than or equal to the current remaining flow control quota, the centralized agent will release the request, i.e., respond to the service request.

[0063] The distributed traffic limiting scheme based on the DPU service mesh provided in the embodiments of the present disclosure offloads each Sidecar agent corresponding to each service on the host and installs a centralized agent corresponding to each service on the DPU, and the method comprises: the centralized agent receives a service request and acquires a target service; whether to perform traffic limiting is determined based on the target service and a pre-cached traffic limiting rule; in the case of performing traffic limiting, a distributed lock corresponding to the target service is requested from the Kubernetes through a control plane component on the host; when the centralized agent receives the distributed lock sent by the control plane component, a current remaining traffic limiting quota of the distributed lock is acquired; and in the case of the request traffic of the service request being less than or equal to the current remaining traffic limiting quota, the service request is responded. In this way, the DPU hardware offloads the Sidecar agent. In the conventional service mesh scheme, the Sidecar agent is deployed on the same host as the application, which occupies a large amount of CPU and memory resources and affects the performance of the application. By offloading the Sidecar agent to the intelligent network card DPU, the high-performance network processing capability of the DPU is utilized to perform traffic management and traffic limiting functions, which greatly reduces the resource overhead of the host and improves the performance and scalability of the service mesh. In addition, the distributed lock is requested from the control plane component on the host side, which ensures the consistency and accuracy of traffic limiting, and the traffic limiting rule management of the centralized agent also simplifies the update and distribution of the traffic limiting strategy, thereby improving the scalability of the service mesh.

[0064] Figure 3 The flowchart of another distributed traffic limiting method based on the DPU service mesh provided in the embodiments of the present disclosure is provided, and the method comprises:

[0065] Step 201, the centralized agent receives and caches the traffic limiting rule corresponding to each service sent by the control plane component on the host; wherein the control plane component monitors the update event of the service in the container orchestration platform Kubernetes cluster and generates the traffic limiting rule corresponding to the service based on the update event and a preset traffic limiting strategy.

[0066] Step 202, the centralized agent receives a service request, acquires a target service, acquires a service identifier of the target service, performs matching based on the service identifier and the traffic limiting rule, and determines whether to perform traffic limiting based on the matching result.

[0067] Step 203, in the case of performing traffic limiting, a distributed lock corresponding to the target service is requested from the container orchestration platform Kubernetes through the control plane component on the host, and when the centralized agent receives the distributed lock sent by the control plane component, a current remaining traffic limiting quota of the distributed lock is acquired.

[0068] Step 204, in response to the service request, updating the current remaining throttling quota and releasing the distributed lock when the request traffic of the service request is less than or equal to the current remaining throttling quota.

[0069] Step 205, retrying or rejecting the service request and releasing the distributed lock when the request traffic is greater than the current remaining throttling quota.

[0070] Step 206, when the centralized agent does not receive the distributed lock sent by the control plane component, requesting the distributed lock corresponding to the target service from the container orchestration platform Kubernetes through the control plane component on the host after a preset time period or rejecting the service request.

[0071] Step 207, the centralized agent sends throttling execution feedback information to the control plane component on the host; wherein the throttling execution feedback information is used to instruct the control plane component to perform global flow monitoring and analysis.

[0072] Specifically, the Sidecar agent is offloaded from the host where the application is located to the intelligent network card DPU, and the high-performance network processing capability of the DPU is used to perform traffic management and throttling functions, reducing the burden of the host CPU; the control plane on the host side, a centralized control plane component is deployed on the host side of the Kubernetes cluster, responsible for monitoring the changes of services, dynamically generating and issuing throttling rules to the centralized agent of the DPU; global throttling scheduling based on etcd distributed lock: using the distributed lock mechanism of Kubernetes etcd, coordinating the throttling behavior of multiple centralized agents, ensuring that the global throttling quota is not exceeded.

[0073] As shown in the example, Figure 4 The functions or working steps of the Host-side agent component include: 1. monitoring the dynamic changes of services in the Kubernetes cluster, including service creation, destruction, scaling, etc. events; 2. dynamically generating and updating throttling rules for each service according to the changes of services and the preset throttling strategy; 3. issuing throttling rules to the centralized agent on the DPU SOC side corresponding to the service instance for local throttling; collecting feedback information of throttling execution from the SOC agent in time for global traffic monitoring and analysis; initializing the etcd distributed lock of the Kubernetes cluster for the SOC agent.

[0074] The feedback information may be throttling statistical information, including the total number of requests, throttling times, rejection times, queuing times, etc. for each service or interface, as well as corresponding QPS, success rate, etc. derived indicators, so as to help the control plane understand the actual traffic situation of the service and evaluate the effect of the throttling rule.

[0075] Specifically, the DPU SOC side centralized proxy component function or working steps include: 1. The centralized proxy receives a service request, and matches the locally cached throttling rules according to the target service and related metadata of the request; 2. If the request needs to be throttled, the centralized proxy will try to apply for a distributed lock for the service from the Kubernetes etcd; 3. If the centralized proxy successfully obtains the etcd lock, it means that it is the globally unique centralized proxy that is authorized to process the service throttling at present, and it will query the current remaining throttling quota recorded in the etcd; 4. The centralized proxy compares the request traffic with the remaining quota: a. If the request traffic is less than or equal to the current remaining quota, the proxy will release the request, write the consumed quota back to the etcd, and then release the lock; b. If the request traffic exceeds the remaining quota, the proxy will reject the request and directly release the lock without consuming the quota. 5. If the proxy does not obtain the etcd lock, it means that another proxy is currently processing the throttling of the service, and the current proxy will enter a retry waiting period or directly reject the request; the preset time period can be set according to the application needs.

[0076] Among them, there are many ways to match the locally cached throttling rules according to the target service and related metadata of the request, such as using a linear table (such as an array or a list) to store all the throttling rules, and then traversing and matching each service request. Some optimizations can also be done to improve matching efficiency, such as setting indexes and filters. When storing throttling rules, indexes can be established for key fields (such as service name, namespace, priority, etc.) to speed up the search speed; when matching, the centralized proxy can use the index to quickly locate the candidate throttling rules according to the properties of the service request, and then perform further comparison and filtering to find the final applicable throttling rule.

[0077] As an example, as shown in Figure 5 Step 5.1, the service request arrives at the centralized proxy; Step 5.2, whether the service request needs to be throttled; if so, Step 5.3, match the throttling rules corresponding to the service request; Step 5.4, try to obtain the etcd distributed lock; if successful, Step 5.5, query the remaining quota in the etcd; if the request traffic is less than or equal to the remaining quota, Step 5.6, release the service request and update the etcd quota; if the request traffic is greater than the remaining quota, Step 5.7, retry or reject the request. In addition, if the etcd distributed lock is not obtained, Step 5.7 is also executed to retry or reject the request; Step 5.8, release the etcd distributed lock; if throttling is not required, Step 5.9, release the service request; Step 5.10, the service request is processed.

[0078] Thus, by offloading the Sidecar agent through the DPU hardware, the Sidecar agent and the application are deployed on the same host in the traditional service mesh solution, occupying a large amount of CPU and memory resources, affecting the performance of the application. The Sidecar agent is offloaded to the intelligent network card DPU in the embodiment of the disclosure, and the high-performance network processing capability of the DPU is used to perform traffic management and flow limiting functions, greatly reducing the resource overhead of the host and improving the performance and scalability of the service mesh.

[0079] Thus, centralized flow limiting rule management, the embodiment of the disclosure introduces a control plane component deployed on the host side, which is responsible for monitoring the dynamic changes of services in the Kubernetes cluster, automatically generating and issuing flow limiting rules to the centralized agent of the DPU; such centralized flow limiting rule management simplifies the operation and maintenance work, improves the consistency and real-time performance of the flow limiting strategy. Compared with traditional manual configuration or template generation, this scheme realizes the automation and intelligentization of the flow limiting rules.

[0080] Thus, global flow limiting coordination based on etcd distributed lock, by using the distributed lock mechanism of Kubernetes etcd, the global flow over-limit problem caused by independent flow limiting of multiple centralized agents in the service mesh is ingeniously solved; by competing for the etcd lock, at most one agent can consume the global flow limiting quota at the same time, ensuring the consistency and accuracy of cross-agent flow limiting.

[0081] Thus, fusion of Kubernetes ecology, the embodiment of the disclosure makes full use of various native components and mechanisms of the Kubernetes platform, such as etcd, service discovery, configuration management, etc., so that the flow limiting scheme can be seamlessly integrated into the Kubernetes environment, reducing the implementation and operation complexity. At the same time, the embodiment of the disclosure also embodies a best practice of the combination of service mesh and Kubernetes, showing the advantages of cloud native architecture.

[0082] Thus, high performance and scalability, through DPU hardware offloading and centralized flow limiting management, the embodiment of the disclosure can support high-performance traffic management of large-scale service mesh, offloading the Sidecar agent from the host to the DPU, not only reducing the host overhead, but also using the high-speed network processing capability of the DPU to improve the throughput and reduce the delay of the agent. At the same time, centralized flow limiting rule management also simplifies the update and distribution of flow limiting strategies, improving the scalability of the service mesh.

[0083] In the above embodiments, the high-performance network processing capability of the DPU hardware is fully utilized to offload traffic limiting execution to the SOC side, reducing the burden of the host CPU; the configuration and management of the traffic limiting rules are centralized to the agent component on the host side, simplifying the operation and maintenance work and improving the consistency of the traffic limiting policy; the agent component is connected to the Kubernetes cluster to automatically generate and distribute traffic limiting rules by monitoring the dynamic changes of service instances, reducing errors and lags caused by manual configuration; finally, with the help of the distributed lock mechanism of the Kubernetes etcd, the agent component coordinates the traffic limiting behaviors of multiple SOC agents to ensure that the global traffic limiting quota is not exceeded.

[0084] Figure 6 A structure diagram of a DPU service mesh distributed traffic limiting device provided by the embodiments of the present disclosure is shown in the figure. The device can be implemented by software and / or hardware and can be integrated in an electronic device. As shown in the figure, each Sidecar agent corresponding to each service on the host is offloaded, and a centralized agent corresponding to each service is installed on the DPU. The device comprises: Figure 6

[0085] The receiving and obtaining module 601 is configured to receive a service request by the centralized agent and obtain a target service;

[0086] The determining module 602 is configured to determine whether to perform traffic limiting based on the target service and pre-cached traffic limiting rules;

[0087] The request module 603 is configured to request a distributed lock corresponding to the target service from a container orchestration platform Kubernetes through a control plane component on the host in the case of performing traffic limiting;

[0088] The obtaining module 604 is configured to obtain a current remaining traffic limiting quota of the distributed lock when the centralized agent receives the distributed lock sent by the control plane component;

[0089] The response module 605 is configured to respond to the service request when the request traffic of the service request is less than or equal to the current remaining traffic limiting quota.

[0090] Optionally, the device further comprises:

[0091] The updating and releasing module is configured to update the current remaining traffic limiting quota and release the distributed lock.

[0092] Optionally, the device further comprises:

[0093] The retry and rejection module is configured to retry or reject the service request and release the distributed lock when the request traffic is greater than the current remaining traffic limiting quota. ​

[0094] Optionally, the apparatus further comprises:

[0095] The processing module is configured to, when the centralized agent does not receive the distributed lock sent by the control plane component, request the distributed lock corresponding to the target service from the container orchestration platform Kubernetes through the control plane component on the host or reject the service request after a preset time period.

[0096] Optionally, the apparatus further comprises:

[0097] The first sending module is configured to receive and cache the flow limiting rules corresponding to each service sent by the control plane component on the host; wherein the control plane component monitors update events of services in the container orchestration platform Kubernetes cluster, and generates the flow limiting rules corresponding to the services based on the update events and a preset flow limiting strategy.

[0098] Optionally, the apparatus further comprises:

[0099] The second sending module is configured to send flow limiting execution feedback information to the control plane component on the host; wherein the flow limiting execution feedback information is used to instruct the control plane component to perform global flow monitoring and analysis.

[0100] The determination module 602 is specifically configured to:

[0101] Obtain a service identifier of the target service;

[0102] Match based on the service identifier and the flow limiting rules, and determine whether to perform flow limiting based on a matching result.

[0103] The DPU service mesh distributed flow limiting apparatus provided by the embodiments of the present disclosure can perform the DPU service mesh distributed flow limiting method provided by any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of performing the method.

[0104] The embodiments of the present disclosure further provide a computer program product, which comprises computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the DPU service mesh distributed flow limiting method provided by any of the embodiments of the present disclosure.

[0105] Figure 7 A structural schematic diagram of an electronic device provided by the embodiments of the present disclosure is provided. The following will be specifically referred to Figure 7, which shows a schematic structural diagram of an electronic device 700 suitable for implementing the embodiments of the present disclosure. The electronic device 700 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0106] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0107] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0108] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the DPU service grid distributed current limiting method of the embodiment of the present disclosure are performed.

[0109] It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer-readable program code is contained. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wire, cable, optical fiber, RF, etc., or any suitable combination of the above.

[0110] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0111] The computer-readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device, and can be accessed via the electronic device.

[0112] The computer readable medium carries one or more programs when the one or more programs are executed by the electronic device, so that the electronic device generates a high-availability data processing unit (DPU) group; wherein the high-availability DPU group includes at least two DPUs, and each DPU has a corresponding priority identifier; generates a logical router, and configures a logical port of the logical router according to the fact that each DPU has a corresponding priority identifier; when a container is created, configures network mode information corresponding to the container; wherein the network mode information corresponds to the logical port of the logical router.

[0113] Computer program code for carrying out operations of the present disclosure can be written in one or more programming languages or combinations of languages including object oriented programming languages such as Java, Smalltalk, C++ as well as conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0114] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the opposite order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0115] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0116] The functionality described above in this document can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0117] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0118] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, comprising:

[0119] a processor;

[0120] a memory for storing the processor-executable instructions;

[0121] the processor, configured to read the executable instructions from the memory and execute the instructions to implement the DPU service mesh distributed traffic limiting method according to any of the embodiments of the present disclosure.

[0122] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, which stores a computer program for executing the DPU service mesh distributed traffic limiting method according to any of the embodiments of the present disclosure.

[0123] The above description is merely preferred embodiments of the present disclosure and a description of principles of the applied technology. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or equivalent features without departing from the above disclosed concept. For example, the above technical features can be replaced with the technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

[0124] Moreover, while operations are depicted in a particular order, this should not be understood as requiring such an order unless otherwise specifically stated. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, while several specific implementation details have been discussed, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0125] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A distributed current limiting method based on DPU service grid, characterized in that: Uninstalling each sidecar agent corresponding to each service on the host and installing a centralized agent corresponding to each service on the DPU, the method comprising: The centralized agent receives the service request and obtains the target service; Determining whether to perform current limiting based on the target service and pre-cached current limiting rules; wherein the pre-cached current limiting rules are based on the control plane component on the host monitoring the update events of the services in the container orchestration platform Kubernetes cluster, and are generated based on the update events and a preset current limiting strategy; In the case of current limiting, a distributed lock corresponding to the target service is requested from the container orchestration platform Kubernetes through the control plane component on the host; When the centralized agent receives the distributed lock sent by the control plane component, it obtains the current remaining current limit quota of the distributed lock; When the request flow of the service request is less than or equal to the current remaining flow limiting quota, respond to the service request.

2. The distributed current limiting method based on the DPU service grid according to claim 1 is characterized in that: After responding to the service request, the method further includes: The current remaining current limit quota is updated and the distributed lock is released.

3. The distributed current limiting method based on the DPU service grid according to claim 1 is characterized in that: The method further comprises: When the request flow is greater than the current remaining flow limit quota, the service request is retried or rejected and the distributed lock is released.

4. The distributed current limiting method based on the DPU service grid according to claim 1 is characterized in that: The method further comprises: When the centralized agent does not receive the distributed lock sent by the control plane component, it requests the distributed lock corresponding to the target service from the container orchestration platform Kubernetes through the control plane component on the host after a preset time period or rejects the service request.

5. The distributed current limiting method based on the DPU service grid according to claim 1 is characterized in that: The method further comprises: The centralized agent receives and caches the flow limiting rules corresponding to each service sent by the control plane component on the host; wherein, the control plane component monitors the update events of the services in the container orchestration platform Kubernetes cluster, and generates the flow limiting rules corresponding to the services based on the update events and the preset flow limiting strategy.

6. The distributed current limiting method based on the DPU service grid according to claim 1 is characterized in that: The method further comprises: The centralized agent sends current limiting execution feedback information to the control plane component on the host; wherein the current limiting execution feedback information is used to instruct the control plane component to perform global process monitoring and analysis.

7. The distributed current limiting method based on the DPU service grid according to claim 1 is characterized in that: The determining whether to perform current limiting based on the target service and the pre-cached current limiting rule includes: Obtaining the service identifier of the target service; A match is performed based on the service identifier and the current limiting rule, and whether current limiting is performed is determined based on the matching result.

8. A distributed current limiting device based on a DPU service grid, characterized in that: Uninstalling each sidecar agent corresponding to each service on the host and installing a centralized agent corresponding to each service on the DPU, the device includes: A receiving and obtaining module, configured for the centralized agent to receive a service request and obtain a target service; A determination module, configured to determine whether to perform current limiting based on the target service and pre-cached current limiting rules; wherein the pre-cached current limiting rules are based on a control plane component on the host monitoring update events of services in a container orchestration platform Kubernetes cluster, and are generated based on the update events and a preset current limiting policy; A request module is used to request a distributed lock corresponding to the target service from the container orchestration platform Kubernetes through the control plane component on the host when current limiting is performed; An acquisition module, configured to acquire the current remaining current limiting quota of the distributed lock when the centralized agent receives the distributed lock sent by the control plane component; The response module is used to respond to the service request when the request flow of the service request is less than or equal to the current remaining flow limit quota.

9. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the DPU service grid distributed current limiting method described in any one of claims 1-7 above.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which is used to execute the DPU service grid distributed current limiting method described in any one of claims 1 to 7.

Images