Traffic request scheduling method, electronic equipment and storage medium

Abstract

The invention discloses a traffic request scheduling method, electronic equipment and a storage medium. The method comprises the following steps: acquiring a user request corresponding to at least one workload; the service grid agent corresponding to the at least one workload is used for intercepting and analyzing the user request to obtain request metadata, and the request metadata is used for describing request attributes of the user request; scheduling decision making is carried out according to the request metadata, a scheduling decision result corresponding to the user request is obtained, and the scheduling decision result is used for determining a request processing mode corresponding to the user request; and in response to determining to accept the user request according to the scheduling decision result, forwarding the user request to the target server. According to the method and the device, the technical problems of low scheduling efficiency and poor flexibility when high-concurrency traffic requests are processed in a service grid in the prior art are solved.

Description

Technical Field

[0001] This application relates to the field of service mesh technology, and more specifically, to a traffic request scheduling method, an electronic device, and a storage medium. Background Technology

[0002] As a modern network architecture, Service Mesh focuses on managing interactions between microservices, providing a dedicated infrastructure layer for handling inter-service communication. It enables functions such as service discovery, load balancing, traffic management, secure transmission, and observability to enhance the reliability and efficiency of microservice architectures. In a service mesh environment, efficiently and fairly managing high-concurrency traffic requests, especially in dynamically changing application scenarios such as sudden traffic spikes, cascading failures between services, and resource constraints, is crucial for service stability. The traffic control strategies provided in related technologies struggle to adapt to dynamically changing load conditions, easily leading to uneven resource allocation, failing to effectively prevent cascading failures, and lacking flexibility in handling traffic requests with different priorities and weights.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a traffic request scheduling method, an electronic device, and a storage medium to at least solve the technical problems of low scheduling efficiency and poor flexibility in the related technologies when processing high-concurrency traffic requests within a service mesh.

[0005] According to one aspect of the embodiments of this application, a traffic request scheduling method is provided, comprising: acquiring user requests corresponding to at least one workload; intercepting and analyzing the user requests using a service mesh proxy corresponding to at least one workload to obtain request metadata, wherein the request metadata is used to describe the request attributes of the user requests; making scheduling decisions based on the request metadata to obtain scheduling decision results corresponding to the user requests, wherein the scheduling decision results are used to determine the request processing method corresponding to the user requests; and responding by accepting the user requests based on the scheduling decision results and forwarding the user requests to a target server.

[0006] According to another aspect of the embodiments of this application, a traffic request scheduling system is also provided, including: a service mesh control plane and a service mesh data plane; the service mesh control plane includes: a load scheduling policy generator, which is used to generate extended configuration information based on workload context and scheduling policy description information, and send the extended configuration information to the service mesh data plane, wherein the workload context is generated based on workload metadata corresponding to at least one workload, the workload context is used to represent context model data associated with at least one workload, and the extended configuration information is used to configure the scheduling enable method and scheduling parameter information associated with the service mesh proxy; the service mesh data plane includes: at least one workload and a service mesh proxy and a request flow controller corresponding to at least one workload, the service mesh proxy is used to intercept and analyze user requests corresponding to at least one workload based on the extended configuration information to obtain request metadata, wherein the request metadata is used to describe the request attributes of the user request; the request flow controller is used to make scheduling decisions based on the request metadata, obtain the scheduling decision result corresponding to the user request, and respond to the decision to receive the user request based on the scheduling decision result, and forward the user request to the target server, wherein the scheduling decision result is used to determine the request processing method corresponding to the user request.

[0007] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the traffic request scheduling method in various embodiments of this application when it runs.

[0008] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to execute the traffic request scheduling method in various embodiments of this application.

[0009] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the traffic request scheduling method in various embodiments of this application.

[0010] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the traffic request scheduling method in various embodiments of this application.

[0011] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the traffic request scheduling method in various embodiments of this application.

[0012] In this embodiment, by acquiring user requests corresponding to at least one workload, and then intercepting and analyzing these user requests using a service mesh proxy corresponding to at least one workload, request metadata is obtained. This achieves the interception and analysis of user requests, enabling a deeper understanding of user request behavior and needs, and providing more accurate data support for subsequent scheduling decisions. Furthermore, scheduling decisions are made based on the request metadata to obtain the scheduling decision result corresponding to the user request, thereby determining the request processing method. Finally, the response, based on the scheduling decision result, accepts the user request and forwards it to the target server to ensure timely response and processing. By intelligently scheduling user requests, a stable service level can be maintained under different load scenarios, while achieving efficient resource utilization and fair allocation. This achieves the goal of efficient, fair, and flexible handling of high-concurrency traffic requests, improving the scheduling efficiency and flexibility of traffic requests in the service mesh, and solving the technical problems of low scheduling efficiency and poor flexibility in related technologies when handling high-concurrency traffic requests within the service mesh.

[0013] It is worth noting that the general description above and the detailed description that follow are merely for illustrative purposes and do not constitute a limitation on this application. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0015] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a traffic request scheduling method according to an embodiment of this application;

[0016] Figure 2 This is a structural block diagram of a computing environment according to an embodiment of this application;

[0017] Figure 3 This is a structural block diagram of a service mesh according to an embodiment of this application;

[0018] Figure 4 This is a flowchart of a traffic request scheduling method according to an embodiment of this application;

[0019] Figure 5 This is a schematic diagram of a traffic request scheduling system according to an embodiment of this application;

[0020] Figure 6 This is a schematic diagram of a traffic scheduling method according to an embodiment of this application;

[0021] Figure 7 This is a structural block diagram of a traffic request scheduling device according to an embodiment of this application;

[0022] Figure 8 This is a structural block diagram of another traffic request scheduling device according to an embodiment of this application;

[0023] Figure 9 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0027] Service mesh: Commonly used to describe the network of microservices that make up an application and the interactions between applications. As an infrastructure layer that handles inter-service communication, the service mesh is responsible for reliably delivering requests across the complex service topologies of modern cloud-native applications.

[0028] Service Mesh Proxy: A service mesh typically consists of a control plane and a data plane. Specifically, the control plane is a set of services running in a dedicated namespace. These services perform control and management functions, including aggregating telemetry data, providing user-facing application programming interfaces (APIs), and providing control data to the data plane proxy. The control plane and data plane together drive the behavior of the data plane. The data plane, in turn, consists of a series of transparent proxies running alongside each service instance.

[0029] Weighted fair queue scheduling: This is an algorithm in the field of network scheduling. It usually refers to assigning weights to each traffic category and determining the processing priority based on the weight of each queue. Traffic with higher weights will be processed before traffic with lower weights.

[0030] According to an embodiment of this application, a traffic request scheduling method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a traffic request scheduling method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0032] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0033] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method in the embodiments of this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the method in the above embodiments. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0035] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0036] Figure 1 The hardware structure block diagram shown can serve not only as an exemplary block diagram of the aforementioned computer terminal 10 (or mobile device), but also as an exemplary block diagram of the aforementioned server. In one optional embodiment, Figure 2 The use of the above is illustrated in a block diagram. Figure 1The computer terminal 10 (or mobile device) shown is an embodiment of a computing node in computing environment 201. Figure 2 A block diagram of a computing environment is shown, such as Figure 2 As shown, computing environment 201 includes multiple computing nodes (such as servers) running on a distributed network (shown as 210-1, 210-2, ... in the diagram). Each computing node contains local processing and memory resources, and end user 202 can remotely run applications or store data within computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3, and 220-4 within computing environment 201, representing services "A", "D", "E", and "H", respectively.

[0037] End user 202 can provide and access services through a web browser or other software application on a client. In some embodiments, the provisioning and / or requests of end user 202 can be provided to ingress gateway 230. Ingress gateway 230 may include a corresponding agent to handle the provisioning and / or requests for services (one or more services provided in computing environment 201).

[0038] The services are provided or deployed based on various virtualization technologies supported by the computing environment 201. In some embodiments, services may be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. VM-based virtualization can simulate a real computer by initializing a virtual machine, executing programs and applications without directly accessing any actual hardware resources. While the machine is virtualized by a virtual machine, container-based virtualization can launch containers to virtualize an entire operating system (OS), allowing multiple workloads to run on a single OS instance.

[0039] In one embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). For example, such as Figure 2 As shown, service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods). A Pod can include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers). One or more containers within a Pod handle requests related to one or more corresponding functions of the service. Proxy 245 typically controls service-related network functions such as routing and load balancing. Other services can also be equipped with similar Pods.

[0040] During operation, executing a user request from end user 202 may require invoking one or more services in computing environment 201, and executing one or more functions of one service may require invoking one or more functions of another service. For example... Figure 2 As shown, service "A" 220-1 receives user requests from terminal user 202 from ingress gateway 230. Service "A" 220-1 can call service "D" 220-2, and service "D" 220-2 can request service "E" 220-3 to perform one or more functions.

[0041] The aforementioned computing environment can be a cloud computing environment, where resource allocation is managed by cloud services, allowing functionality development without needing to consider implementation, adjustment, or server scaling. This computing environment allows developers to execute event-responsive code without building or maintaining complex infrastructure. Services can be partitioned into a set of functions that can automatically and independently scale, rather than scaling a single hardware device to handle potential loads.

[0042] In another alternative embodiment, Figure 3 The use of the above is illustrated in a block diagram. Figure 1 The computer terminal 10 (or mobile device) shown is an embodiment of a service mesh. Figure 3 A block diagram of a service mesh structure is shown, such as Figure 3 As shown, the service mesh 300 is mainly used to facilitate secure and reliable communication between multiple microservices. Microservices refer to the decomposition of an application into multiple smaller services or instances, which are distributed across different clusters / machines to run.

[0043] like Figure 3 As shown, a microservice may include application service instance A and application service instance B, which together form the functional application layer of service mesh 300. In one implementation, application service instance A runs as a container / process 308 on machine / workload container group 314 (Pod), and application service instance B runs as a container / process 310 on machine / workload container group 316 (Pod).

[0044] In one implementation, application service instance A can be a product query service, and application service instance B can be a product order placement service.

[0045] like Figure 3As shown, application service instance A and grid agent (sidecar) 303 coexist in machine / workload container group 314, and application service instance B and grid agent 305 coexist in machine / workload container group 316. Grid agents 303 and 305 form the data plane layer of service mesh 300. Grid agents 303 and 305 run as containers / processes 304 and 306 respectively, and can receive requests 312 for product query services. Grid agent 303 and application service instance A can communicate bidirectionally, and grid agent 305 and application service instance B can also communicate bidirectionally. Furthermore, grid agents 303 and 305 can also communicate bidirectionally with each other.

[0046] In one implementation, traffic from application service instance A is routed to the appropriate destination via mesh proxy 303, and network traffic from application service instance B is routed to the appropriate destination via mesh proxy 305. It should be noted that the network traffic mentioned here includes, but is not limited to, Hypertext Transfer Protocol (HTTP), Representational State Transfer (REST), high-performance, general-purpose open-source frameworks (Google Remote Procedure Call, gRPC), and open-source in-memory data structure storage systems (Redis).

[0047] In one implementation, the functionality of the extended data plane layer can be achieved by writing custom filters for the proxy (Envoy) in service mesh 300. The service mesh proxy configuration can enable the service mesh to correctly proxy service traffic, achieving service interoperability and service governance. Mesh proxy 303 and mesh proxy 305 can be configured to perform at least one of the following functions: service discovery, health checking, routing, load balancing, authentication and authorization, and observability.

[0048] like Figure 3 As shown, the service mesh 300 also includes a control plane layer. This control plane layer can consist of a set of services running in a dedicated namespace, hosted by the control plane component 301 within machine / workload container groups (machine / Pod) 302. Figure 3 As shown, control plane component 301 communicates bidirectionally with grid agents 303 and 305. Control plane component 301 is configured to perform various control and management functions. For example, control plane component 301 receives telemetry data transmitted by grid agents 303 and 305 and can further aggregate this telemetry data. In addition to these services, control plane component 301 can also provide a user-facing application programming interface (API) to facilitate easier manipulation of network behavior and to provide configuration data to grid agents 303 and 305.

[0049] Under the aforementioned operating environment, this application provides the following: Figure 4 The traffic request scheduling method shown. Figure 4 This is a flowchart of a traffic request scheduling method according to an embodiment of this application, such as... Figure 4 As shown, the method includes the following steps:

[0050] Step S41: Obtain at least one user request corresponding to a workload;

[0051] Step S42: Intercept and analyze user requests using at least one service mesh proxy corresponding to a workload to obtain request metadata, wherein the request metadata is used to describe the request attributes of the user request.

[0052] Step S43: Make a scheduling decision based on the request metadata to obtain the scheduling decision result corresponding to the user request. The scheduling decision result is used to determine the request processing method corresponding to the user request.

[0053] Step S44: The response determines whether to receive the user request based on the scheduling decision result and forwards the user request to the target server.

[0054] In the context of service mesh and service architecture, at least one workload can be an application instance or service instance running in the service mesh environment. These instances are typically containerized or virtualized and communicate and are managed through a service mesh broker. A workload can be any service in a microservices architecture, with each service instance having its own independent lifecycle and resource requirements. Service mesh improves application reliability and performance by monitoring and managing workload communication, security, traffic control, and load balancing.

[0055] The aforementioned user request is a network request initiated by a user or application to interact with at least one workload. For example, a user request can be a Hypertext Transfer Protocol (HTTP) request or any form of network communication request.

[0056] In a service mesh, when a user request is sent to a workload, it is captured by the service mesh proxy corresponding to that workload. The service mesh proxy determines how to handle the user utilization request based on configured rules and scheduling policies, including but not limited to routing, load balancing, authentication, monitoring, and weighted fair queue scheduling, to ensure high availability and responsiveness of the service while optimizing resource usage. A workload can be a specific microservice instance or application instance, and the target server is a physical or virtual server node in the service mesh selected to handle a specific user request.

[0057] User requests are intercepted and analyzed using at least one service mesh proxy corresponding to a workload, and request metadata is extracted from these requests. Specifically, request metadata may include, but is not limited to, request headers, request body, source IP and port, destination IP and port, service information, and request tags. Request headers are the most common type of metadata in HTTP requests, including the user-agent, request method, Uniform Resource Locator (URL), authentication information, and content type. Request header information can be used to identify the request source, type, and requester identity. For some requests, the service mesh proxy may also analyze the request body, especially when routing or applying specific scheduling strategies based on data in the request body, such as load balancing based on request parameters. The source IP address and port of the request can provide information about the requester's geographic location and application, while the destination IP and port can be used to determine the request's destination and how to forward the request to the correct service instance using routing rules in the service mesh. The request may also contain service name and version information, which is crucial for service discovery and routing across different service versions. Request tags are specific labels or identifiers that can be used to distinguish different types of requests, such as different user types, request priorities, or specific service scenarios. Request tags will affect the scheduling and processing of requests.

[0058] Accurate capture and analysis of request metadata by the service mesh broker is fundamental to achieving intelligent traffic scheduling, security, and observability within the service mesh. By leveraging request metadata, the service mesh broker can make more informed scheduling decisions, optimize resource allocation, and ultimately improve service responsiveness and stability.

[0059] Furthermore, the request flow controller in the traffic request scheduling system can make scheduling decisions based on request metadata to obtain the scheduling decision results corresponding to user requests. These results are used to determine the request processing method for each user request. The request flow controller typically runs as a daemon set on each node in the service mesh data plane, acting as a globally applicable node extension proxy. As the decision executor, the request flow controller can provide metric measurement services, traffic classification services, and traffic control services to obtain the scheduling decision results.

[0060] The scheduling decision results described above are used to determine the request processing method corresponding to a user request. Specifically, the scheduling processing method includes, but is not limited to, priority allocation, rate limiting, concurrency limiting, and load balancing. Based on the weight and priority label of the user request, the load scheduler can decide which requests should be processed first. For example, requests from certain important users or critical services may be given higher priority to ensure that user requests are responded to in a timely manner even under high load.

[0061] During peak traffic periods, the load balancer can implement rate limiting to prevent system overload. This involves controlling the token consumption rate for specific request types to ensure the system isn't overwhelmed by a large number of low-priority requests. To prevent cascading failures, the load balancer can limit the number of concurrent requests per service instance. Requests exceeding this limit will be queued or temporarily rejected until load resources become available. The load balancer can also intelligently distribute requests across different service instances, ensuring even load distribution and improving the overall system responsiveness and reliability.

[0062] During the scheduling decision-making process, adjustments can be made dynamically based on the current system load and resource usage. For example, if an increase in latency is detected for a service instance, the load scheduler can reduce the number of requests allocated to it or increase the queuing time for requests to avoid further cascading failures. The scheduling decision results can also be used to further optimize strategies and parameters through feedback mechanisms. For instance, if some requests are frequently rejected, the load scheduler can adjust priorities or token allocation strategies to improve the success rate of processing these requests. After making a scheduling decision, if it is determined that a user request will be accepted, the service mesh broker can forward the user request to the target server.

[0063] Based on steps S41 to S44 above, by acquiring user requests corresponding to at least one workload, and then using the service mesh proxy corresponding to at least one workload to intercept and analyze the user requests, request metadata is obtained. This achieves the interception and analysis of user requests, enabling a deeper understanding of user request behavior and needs, and providing more accurate data support for subsequent scheduling decisions. Furthermore, scheduling decisions are made based on the request metadata to obtain the scheduling decision results corresponding to the user requests, determining the request processing method for each user request. Finally, the response, based on the scheduling decision results, determines whether to accept the user request and forward it to the target server to ensure timely response and processing. By intelligently scheduling user requests, a stable service level can be maintained under different load scenarios, while achieving efficient resource utilization and fair allocation. This achieves the goal of efficient, fair, and flexible handling of high-concurrency traffic requests, improving the scheduling efficiency and flexibility of traffic requests in the service mesh, and thus solving the technical problems of low scheduling efficiency and poor flexibility in related technologies when handling high-concurrency traffic requests within the service mesh.

[0064] The method for scheduling traffic requests in the embodiments of this application will be further described below.

[0065] In an optional embodiment, in step S43, a scheduling decision is made based on the request metadata to obtain the scheduling decision result corresponding to the user request, including:

[0066] Step S431: Classify user requests based on request metadata to obtain request classification results, wherein the request classification results are used to determine the target request flow corresponding to the user request;

[0067] Step S432: Perform strategy matching using preset matching rules and request classification results to obtain strategy matching results;

[0068] Step S433: Based on the strategy matching result, make a scheduling decision on the user request and obtain the scheduling decision result.

[0069] Specifically, the traffic request scheduling system also includes a load classifier, a policy matcher, and a load scheduler. The load classifier categorizes user requests based on request metadata, obtaining request classification results, which determine the target request flow corresponding to each user request. Within the service mesh context, requests with similar characteristics or attributes can be grouped together based on request metadata and scheduling policies to obtain request flows, enabling unified scheduling and resource allocation. The target request flow represents a series of user requests received and processed under specific conditions. The policy matcher uses preset matching rules to select a specific load scheduler that matches the request classification results. These preset matching rules include at least: service instance, access control point, and traffic label set. The load scheduler performs fair scheduling among multiple user requests, returning information such as whether the request can be processed and the remaining token quantity based on whether the request can be processed.

[0070] The preset matching rules can be selectors based on workload metadata. These selectors can define information such as service instances, access control points, and traffic tag sets. Table 1 shows an example of a selector definition.

[0071] Table 1 Selectors

[0072]

[0073] The selector is used to specify a particular agent group (agent_group), where the control point is the ingress and the service is service1-demo-app.demoapp.svc.cluster.local. The settings in Table 1 allow you to specify which agent group handles which service at which control point. Further, based on the policy matching results, a scheduling decision is made for the user request, yielding the scheduling result.

[0074] Based on the above optional embodiments, by classifying request metadata, the target request flow corresponding to a user request can be determined more accurately, thereby improving the system's response speed and resource utilization. Simultaneously, by using preset matching rules and strategy matching results for scheduling decisions, appropriate scheduling methods can be selected for user requests based on different situations, further improving scheduling efficiency and flexibility.

[0075] In an optional embodiment, in step S431, the user request is classified based on the request metadata to obtain the request classification result, including:

[0076] Step S4311: Perform hash calculation using the target random number and request metadata to obtain the request stream identifier. The target random number is used to adjust the scheduling randomness of the target request stream, and the request stream identifier is used to match the user request to the target request stream.

[0077] Step S4312: Determine the request classification result based on the request flow identifier.

[0078] The aforementioned target random number, also known as the fairness seed, is used in conjunction with a hash function to compute a unique identifier for each request stream. This identifier is used to match each user request to its corresponding target request stream, thereby determining the request classification result based on the request stream identifier. The fairness seed can address the persistent unfairness that may arise in Weighted Fair Queueing (WFQ) mechanisms. For example, without a fairness seed, user requests with the same weight may consistently receive different processing times due to their arrival order, leading to some request streams being consistently prioritized while others are unfairly delayed. Introducing a fairness seed increases the randomness in the hash calculation, ensuring that user requests are scheduled more fairly, even within request streams with the same weight.

[0079] Hash calculations can map requests with similar characteristics into the same request stream, facilitating unified management and scheduling of specific request types and ensuring that requests with the same attributes are processed according to predetermined strategies. Utilizing target random numbers increases the randomness of request scheduling, preventing certain request streams from being consistently prioritized due to the characteristics of hash functions. This ensures that all request streams receive relatively fair processing opportunities during load balancing and scheduling, maintaining fairness even in high-concurrency scenarios. Precise control over the processing of each request stream effectively prevents cascading effects caused by the failure of a service or application. For example, limiting concurrent requests or implementing average latency strategies ensures that overload is avoided even when service pressure increases, protecting the stability and reliability of critical services.

[0080] Based on the above optional embodiments, by combining the target random number and request metadata to generate a request flow identifier, it is possible to achieve fine-grained control over the request flow, enhance the fairness and randomness of scheduling, dynamically adjust strategies to adapt to changes, prevent cascading failures, optimize resource allocation, and provide real-time monitoring and tuning capabilities, thereby significantly improving the efficiency of traffic request scheduling in the service mesh.

[0081] In an optional embodiment, in step S433, a scheduling decision is made on the user request based on the policy matching result, and the scheduling decision result includes:

[0082] Step S4331: Based on the policy matching result, determine the target scheduling method corresponding to the user request from multiple candidate scheduling methods, wherein the multiple candidate scheduling methods include: quota scheduling method, concurrent scheduling method and load scheduling method;

[0083] Step S4332: Use the target scheduling method to make scheduling decisions on user requests and obtain scheduling decision results.

[0084] Specifically, a policy matcher analyzes the characteristics of user requests, such as request metadata, workload metadata, and the weights and priorities carried by the requests, to determine the appropriate scheduling policy for the request, thus obtaining a policy matching result. Based on the policy matching result, a target scheduling method corresponding to the user request is determined from multiple candidate scheduling methods, including quota scheduling, concurrent scheduling, and load balancing. The target scheduling method is then used to make a scheduling decision for the user request, resulting in a scheduling decision result.

[0085] For example, in determining the target scheduling method, if a user request has high weight and priority and sufficient resource quota, quota scheduling is given priority to ensure the processing of critical requests. If the system detects that the number of concurrent requests to a service instance is close to or exceeds the limit, concurrent scheduling is given priority to avoid overloading the service instance. When a service instance is close to its processing capacity limit or the overall system load is high, load balancing is used, dynamically adjusting the request processing rate through a WFQ mechanism to protect the service from overload. Through this dynamic decision-making logic based on policy matching results and system status, the system can intelligently select the most suitable scheduling method for the current scenario, ensuring efficient resource utilization while achieving fair processing of requests and avoiding service interruptions and cascading failures.

[0086] Quota scheduling is a token bucket-based scheduling method. Each user or application receives a certain number of tokens based on their quota, which can be used to access system resources. When a user request arrives, the system checks if the user has enough tokens in their quota. If so, the request is processed; otherwise, it may be queued or rejected outright. Quota scheduling ensures fair resource allocation and prevents resource abuse.

[0087] Concurrency scheduling, on the other hand, schedules requests based on the concurrency limits of the workload. Each workload can have a concurrency limit set. When the limit is reached, new requests will be queued or rejected until the number of requests currently being processed decreases. Concurrency scheduling helps prevent service instability or crashes caused by excessive concurrent requests.

[0088] Load balancing can protect the service under high load by dynamically adjusting the request rate. It uses a local token bucket to estimate the allowed token rate and determines the processing order of requests based on a weighted fair queue algorithm. Load balancing effectively handles traffic spikes, avoids service overload, and ensures that important requests are processed first.

[0089] Based on the above optional embodiments, the target scheduling method corresponding to the user request is determined from multiple candidate scheduling methods based on the policy matching result. Then, the user request is scheduled using the target scheduling method to obtain the scheduling decision result. This enables more flexible and intelligent management of user requests in the service mesh, further improving scheduling efficiency.

[0090] In an optional embodiment, the traffic request scheduling method in this application further includes:

[0091] Obtain the virtual completion time information and request attribute information corresponding to the user request. The virtual completion time information is used to determine the order in which the user requests are to be sorted, and the request attribute information is used to determine the order in which the user requests are to be processed and the processing time.

[0092] The system adds user requests to the target queue position in the target request queue based on virtual completion time information, and determines the target processing order and target processing time corresponding to the target queue position based on request attribute information. The target request queue is used to store multiple user requests arranged according to virtual completion time information.

[0093] The aforementioned virtual completion time information can be the virtual completion time corresponding to the user request. In the WFQ mechanism, virtual completion time is used to simulate the service time of the request flow, ensuring that each request flow receives processing time corresponding to its weight. The aforementioned request attribute information is used to represent the weight and priority information corresponding to the user request. When a user request arrives, a weight can be assigned to it based on specific attributes in the user request. This weight can reflect the expected amount of system resource consumption by the user request, as well as the relative importance of the request.

[0094] In the WFQ (Weighted Free Query) mechanism, the order in which multiple user requests are queued refers to their arrangement. Virtual completion time information is used to calculate the estimated virtual completion time for each request. Based on this virtual time, user requests are inserted into the appropriate position in the queue to ensure that high-weight requests receive priority access to processing resources. The order can be dynamically adjusted using the WFQ algorithm to achieve weight-based fair scheduling.

[0095] The processing order refers to the priority order in which user requests are actually processed. This order is related to the weight and priority of the user requests and the current system load. Request attribute information, including the request's weight, priority label, and service requirements, is used to determine the processing order. In the WFQ scheduling mechanism, the request weight directly affects resource allocation; requests with higher weights are given more service time and therefore may have higher priority in actual processing.

[0096] Each user request entering the system is added to the target request queue, and the queue is sorted based on its virtual completion time. The target processing order and target processing time corresponding to the position in the target queue are determined based on the request attribute information. The target request queue is used to store multiple user requests arranged according to the virtual completion time information.

[0097] By leveraging virtual completion time and request attribute information, and maintaining a target request queue sorted by virtual completion time, weighted allocation and fair scheduling of computing resources are achieved. The cost required for each user request is calculated based on its weight and fairness seed, which determines its processing order within the system. The virtual completion times of the target request queue and request stream are updated in real time to ensure that requests with higher weights receive proportionally more processing time.

[0098] Based on the above optional embodiments, by obtaining the virtual completion time information and request attribute information corresponding to the user request, and then adding the user request to the target queue position in the target request queue based on the virtual completion time information, and determining the target processing order and target processing time corresponding to the target queue position based on the request attribute information, it is possible to intelligently schedule and manage user requests, thereby further improving scheduling efficiency.

[0099] In an optional embodiment, in step S4332, the user request is scheduled using a target scheduling method, and the scheduling decision result includes:

[0100] The number of available tokens is determined based on the target scheduling method;

[0101] The scheduling decision for user requests is made using the number of available tokens, the estimated number of tokens corresponding to the user request, and the request attribute information. The estimated number of tokens is proportional to the target processing time.

[0102] Specifically, the load balancer dynamically throttles the request rate under high load conditions, scheduling user requests based on importance and adhering to quota and concurrency limits to protect services from overload or cascading failures. The load balancer uses a local token bucket to estimate the allowed token rate. The token bucket's filling rate is adjusted according to the specified target scheduling method. Internally, the load balancer includes a token manager that tracks and allocates tokens to requests, ensuring reasonable resource usage.

[0103] A certain number of tokens is allocated to each request, with the number of tokens proportional to the target processing time of the request. To accurately calculate concurrency, user requests are distributed to different workloads. For each workload, the estimated number of tokens used by the request is estimated using an automatic token function. The number of tokens available per second is calculated based on the current concurrency and load offloading factor. Then, for each user request, the number of available tokens is determined based on the target scheduling method. A scheduling decision is made using the number of available tokens, the estimated number of tokens for the user request, and the priority information of the user request, resulting in a scheduling decision; for example, the user request may be served immediately or added to a queue to wait.

[0104] Based on the above optional embodiments, the number of available tokens is determined based on the target scheduling method, and then the scheduling decision for the user request is made using the number of available tokens, the estimated number of tokens corresponding to the user request, and the request attribute information, thereby obtaining the scheduling decision result and further improving the scheduling efficiency.

[0105] In one optional embodiment, determining the number of available tokens based on the target scheduling method includes: in response to the target scheduling method being a quota scheduling method, determining the number of available tokens from a shared token set, wherein the shared token set is used to determine the total number of available tokens shared by multiple workloads.

[0106] In quota-based scheduling, the number of available tokens is determined from a shared token set, which is used to determine the total number of available tokens shared by multiple workloads. Scheduling requests based on importance essentially provides scheduling functionality on top of the rate limiter. A token bucket represents a fixed quota allocated among service mesh agents and can serve as a shared ledger for agents within an agent group. The shared ledger allocates the total number of available tokens among service mesh agents. Tokens are consumed when a request is accepted. If the tokens in the shared ledger run out, new user requests are queued until more tokens become available or a timeout occurs.

[0107] In one optional embodiment, determining the number of available tokens based on the target scheduling method includes: in response to the target scheduling method being a concurrent scheduling method, determining the number of available tokens based on the total number of requests currently being processed concurrently.

[0108] In concurrent scheduling, the number of available tokens is determined based on the total number of concurrent requests being processed. Requests are then scheduled according to their importance, while ensuring that the application adheres to concurrency limits. Essentially, it provides scheduling functionality on top of a concurrency limiter. The maximum number of requests in transit is recorded in a global request ledger, which is shared by service mesh proxies in the proxy group. This maximum number of requests in transit is the total number of requests currently in progress recorded in the ledger. If the ledger exceeds the configured maximum concurrency value, new user requests will be queued until the number of ongoing requests falls below the limit or a timeout occurs.

[0109] In one optional embodiment, determining the number of available tokens based on the target scheduling method includes: in response to the target scheduling method being a load scheduling method, determining the number of available tokens based on the incoming request rate of user requests and the token replenishment rate of the local token set, wherein the local token set is used to represent the total number of available tokens exclusively for each workload.

[0110] In load balancing, the number of available tokens is determined based on the incoming request rate of user requests and the token replenishment rate of the local token set. The local token set is the total number of available tokens exclusively for each workload. During periods of high load, the request rate is dynamically limited to protect the service from overload and cascading failures. This is achieved by estimating the allowable token rate using a local token bucket and adjusting the token bucket's fill rate according to a specified strategy.

[0111] If the incoming request rate exceeds the token replenishment rate, user requests will be queued. Queued requests will be accepted in the order determined by a weighted fair queuing algorithm when tokens become available. Requests that fail to be scheduled within the specified time frame will be rejected. By defining workloads with different priorities and weights, load balancing can prioritize certain requests, thereby mitigating service degradation during periods of high traffic.

[0112] In an optional embodiment, the traffic request scheduling method in this application further includes: discarding the user request in response to the user request not being scheduled within the scheduling time range.

[0113] Specifically, each user request has a corresponding scheduling time range, i.e., a timeout period. In computer networks and distributed systems, the timeout period is typically a time limit set for a user request or operation. If the user request does not receive a response or the operation is not completed within this time limit, the user request or operation is considered to have timed out. In the traffic scheduling scenario of a service mesh, the timeout period refers to the maximum waiting time for each user request to be scheduled and processed by the system. If a user request is not scheduled within the set scheduling time range, it will be considered a timeout, and the system may take corresponding measures, such as rejecting the request, marking it as failed, or rerouting it to another service instance, to prevent the request from waiting indefinitely, affecting user experience and the reasonable allocation of system resources. Within the scheduling time range, the user request should be scheduled. If the user request is not scheduled within the scheduling time range, for example, due to a lack of available tokens, the user request is discarded.

[0114] Based on the above optional embodiments, in response to a user request that is not scheduled within the scheduling time range, the user request is discarded, which helps maintain the stable operation of the system, ensures the response efficiency of critical services, and achieves fair service allocation under limited resources, thereby improving the overall user experience and system performance.

[0115] In an optional embodiment, the traffic request scheduling method in this application further includes: performing circuit breaker evaluation based on the collected metric data to obtain an evaluation result, wherein the evaluation result is used to determine whether load offloading needs to be performed by blocking user requests.

[0116] Specifically, the system responds to user request processing feedback, such as acceptance, rejection, and waiting time, thereby comprehensively collecting metric data on queues, requests, and token usage to achieve real-time monitoring of system performance. To perform load balancing, some traffic needs to be dropped. The traffic request scheduling system includes a circuit breaker evaluator, which can use a gradient controller to implement flow control strategies for circuit breaking. The circuit breaker evaluator performs circuit breaking evaluation based on the collected metric data, obtaining an evaluation result that determines whether load balancing should be performed by blocking user requests.

[0117] Based on the above optional embodiments, the circuit breaker assessment processing based on metric data can significantly improve the stability and performance of the service mesh, especially in high-concurrency and variable service environments, and achieve system self-protection and resource optimization through automated traffic control strategies.

[0118] In an optional embodiment, the traffic request scheduling method in this application further includes:

[0119] Obtain workload metadata corresponding to at least one workload, wherein the workload metadata is used to describe the load attribute information of at least one workload; generate a workload context based on the workload metadata, wherein the workload context is used to represent the context model data associated with at least one workload; generate extended configuration information using the workload context and scheduling policy description information, wherein the extended configuration information is used to configure the scheduling enable method and scheduling parameter information associated with the service mesh agent; and send the extended configuration information to the service mesh agent.

[0120] Specifically, at least one workload's metadata is retrieved. This metadata includes attribute information about the workload, such as its type, size, resource requirements, runtime, and health status. The retrieval of workload metadata can be accomplished through the service mesh's control plane component.

[0121] Workload contexts are generated based on the collected workloads. A workload context is a type of context model data that integrates metadata from multiple workloads, providing a more comprehensive contextual view of each workload. Context model data includes the workload's runtime environment, associated services and network information, and any other factors that may affect its operational status.

[0122] Extended configuration information is generated using workload context and scheduling policy descriptions. This extended configuration information contains specific instructions for configuring the service mesh broker, including scheduling enabling methods such as whether WFQ scheduling, quota scheduling, and concurrency control are enabled, as well as scheduling parameters such as weights in WFQ, the number of tokens in quotas, and concurrency limit thresholds. The extended configuration information can inform the service mesh broker how to handle traffic requests for specific workloads and which scheduling policies to apply.

[0123] Finally, the generated extended configuration information is sent to the service mesh broker. The service mesh broker executes specific scheduling policies based on the received configuration information to ensure that user requests are processed fairly according to their weights and priorities. The service mesh broker can adjust its internal queue management, resource allocation, request filtering, and other operations based on the extended configuration information to ensure the implementation of traffic control and scheduling policies.

[0124] Based on the above optional embodiments, by obtaining workload metadata corresponding to at least one workload, and then generating a workload context based on the workload metadata, extended configuration information is generated using the workload context and scheduling policy description information. Finally, the extended configuration information is sent to the service mesh proxy, thereby guiding the service mesh proxy to intelligently process traffic requests, optimize resource allocation and avoid service overload, thereby improving the stability and responsiveness of the service mesh.

[0125] Specifically, Figure 5 This is a schematic diagram of a traffic request scheduling system according to an embodiment of this application, such as... Figure 5 As shown, the traffic request scheduling system mainly consists of a service mesh control plane 501 and a service mesh data plane 502. The service mesh control plane 501 is a collection of service mesh control plane components in either a managed or unmanaged mode. The managed mode means that these service mesh components run on a separate server, rather than within a data plane cluster. Cloud vendors often support service mesh capabilities in a managed mode to reduce the operational complexity and cost for users. The service mesh control plane 501 is responsible for managing and configuring the service mesh data plane cluster under the service mesh data plane 502, as well as the workloads 503a and their service mesh proxies 504a and 503b and their service mesh proxies 504b running within it.

[0126] Service Mesh Data Plane 502 is a logical partition comprising multiple service mesh data plane clusters, managed uniformly by Service Mesh Control Plane 501. For example, a service mesh data plane cluster can be a Kubernetes cluster or other types of computing environments.

[0127] The service mesh controller 506 is used to discover and collect workloads 503a and 503b in the service mesh data plane 502, and generate a workload context 506 based on the workload metadata. The workload metadata describes the attributes and characteristics of the workload, including but not limited to the workload's type, size, resource requirements, runtime, and other relevant attributes. The workload context 506 refers to the workload-related context model data calculated by fusing relevant information from multiple workloads and the workload metadata.

[0128] Users can define request load balancing policies using cloud-native declarative APIs (description 507), including several built-in policy templates: Average Delay Load Balancing Policy: Used to dynamically control the request rate during high loads, thus protecting the service from overload and cascading failures. The allowed token rate is estimated using a local token bucket, and the token bucket's filling rate is adjusted by the controller according to the specified policy. Smooth Load Increase Policy: Gradually increases the system's workload or request volume based on closed-loop feedback, rather than applying a sudden surge in load. This helps the system gradually adapt to increased load, ensuring stable operation and minimizing system impact during load increases. Quota Scheduling Policy: Provides a token bucket-based quota scheduler policy. It controls each user's or application's use of system resources based on quotas, ensuring that allocated quotas are not exceeded, thus achieving fair allocation and efficient utilization of resources. Concurrency Limiting Scheduling Policy: Used to schedule requests based on importance while ensuring applications adhere to concurrency limits. This policy controls the number of tasks or requests running concurrently by setting concurrency limits.

[0129] For example, in a scheduling strategy based on average latency, workload scheduling is dynamically adjusted by considering multiple factors such as traffic type and workload latency, thereby ensuring high availability and responsiveness of the service. This includes prioritizing based on service-specific traffic characteristics, such as prioritizing based on access control points or service names, and dynamically adjusting the service load handling capability to cope with different workload conditions.

[0130] By requesting the load scheduling controller 508, the contents of the user-defined request load scheduling policy description 507 can be analyzed to determine which workloads should enable these load scheduling policy configurations and which policies should be used.

[0131] The load balancing strategy generator 509 dynamically injects the scheduling strategy content calculated by the request load balancing controller 508 into the mesh rule configuration, thereby deriving the mesh proxy extended configuration 510 after enabling the request load balancing function. The extended proxy configuration contains the request load balancing enabling method and related parameter information for each service mesh proxy.

[0132] The extended mesh agent configuration 510 pushes the interaction API between the control plane and the data plane to the data plane component, generating the formatted content required by the data plane component. That is, the service mesh agent 504a and the service mesh agent 504b will use this content to determine whether to enable and how to enable the request flow controller 514 to handle request load scheduling.

[0133] Once a request is sent to workload 503, it will be intercepted by service mesh broker 504. Service mesh broker 504 extracts request metadata 513 from the request, which is then used by load classifier module 515 to classify user request traffic. For each user request, policy matcher module 516 will select a specific scheduler that matches the request based on matching rules.

[0134] The request flow controller 514, acting as the decision executor, includes a metric measurement service, a traffic classification service, and a traffic control service. The request flow controller 514 continuously receives and analyzes metric data from the metric acquisition and processing service 512 to determine traffic control policies. By analyzing the metric data, one or more traffic control policies to be executed on specific nodes can be determined. These traffic control policies include rate limiting and concurrency limiting to prevent abuse and protect services, as well as applying custom labels to real-time traffic.

[0135] The metric acquisition processor 512 collects and processes metrics, storing them in the monitoring metric repository 520. Processed metrics provide evaluation data for the circuit breaker evaluator 519, and the evaluated data is stored in the load scheduling management metadata repository 517 and fed back to the load scheduler 518. When a request ends, the complete duration of the request and all used metadata are known, allowing a signal to be created in the short-term metric database to control the use of cyclic circuit breaking.

[0136] In the traffic request scheduling system, by dynamically collecting and analyzing the tag information in traffic requests, and adopting a weighted fair queue mechanism, intelligent scheduling and resource optimization allocation of traffic requests are achieved, thereby improving the overall performance and fairness of the system.

[0137] In an optional embodiment, after intercepting and analyzing user requests using at least one service mesh proxy corresponding to the workload, the traffic request scheduling method in this embodiment further includes:

[0138] Obtain the request tag information corresponding to the user request, whereby the request tag information is used to identify the scheduling priority of the user request; use the weighted fair queue mechanism to adaptively schedule the user request based on the request tag information to obtain the scheduling decision result.

[0139] Specifically, request tag information can be extracted from user requests, including user type, service identifier, and scheduling priority. Request tag information can be obtained by the service mesh proxy during traffic request interception and used as part of the request metadata for subsequent scheduling decisions.

[0140] WFQ (Win-Free Request) is a weighted scheduling strategy that ensures high-priority user requests receive more service time while preventing low-priority requests from being completely ignored. During adaptive scheduling using WFQ, service time can be dynamically allocated based on request tag information, enabling adaptive scheduling of user requests. The traffic request scheduling system can intelligently adjust its scheduling strategy based on real-time load and request characteristics, ensuring fair resource allocation and efficient utilization.

[0141] The scheduling decision results obtained through the adaptive scheduling of the WFQ mechanism can be used to determine the processing order, processing time, and whether each user request is accepted, queued, or rejected. The scheduling decision results can determine the response time of user requests and the overall service quality.

[0142] Based on the above optional embodiments, by obtaining request tag information and then using the WFQ mechanism for intelligent scheduling, the scheduling decision result is finally output, thereby ensuring that traffic requests in the service mesh environment can be processed dynamically, fairly and effectively, improving the overall performance of the service system and the user experience.

[0143] Figure 6 This is a schematic diagram of a traffic scheduling method according to an embodiment of this application, as shown below. Figure 6 As shown, user requests are acquired through a service mesh proxy, and adaptive scheduling of these requests is performed using a request flow controller. The load classifier categorizes user requests based on their corresponding request metadata, obtaining a request classification result. After obtaining the classification result, a policy matcher is consulted and a policy is applied, resulting in a policy execution result. Based on the policy execution result, the load scheduler attempts to increment a counter to obtain request scheduling, returning a scheduling decision result. Request scheduling is then performed based on the scheduling decision result. If the scheduling decision determines that the user request should be accepted, it is forwarded to the target client; otherwise, it is rejected.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0145] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0147] According to an embodiment of this application, a traffic request scheduling apparatus for implementing the above-described traffic request scheduling method is also provided. Figure 7 This is a structural block diagram of a traffic request scheduling device according to an embodiment of this application, such as... Figure 7 As shown, the device includes:

[0148] The first acquisition module 701 is used to acquire at least one user request corresponding to a workload.

[0149] The interception module 702 is used to intercept and analyze user requests using at least one service mesh proxy corresponding to the workload to obtain request metadata, wherein the request metadata is used to describe the request attributes of the user request.

[0150] The decision module 703 is used to make scheduling decisions based on request metadata and obtain the scheduling decision result corresponding to the user request. The scheduling decision result is used to determine the request processing method corresponding to the user request.

[0151] Forwarding module 704 is used to respond to the user request being accepted based on the scheduling decision result and to forward the user request to the target server.

[0152] Optionally, the decision module 703 is further configured to: classify user requests based on request metadata to obtain request classification results, wherein the request classification results are used to determine the target request flow corresponding to the user request; perform policy matching using preset matching rules and request classification results to obtain policy matching results; and make scheduling decisions on user requests based on policy matching results to obtain scheduling decision results.

[0153] Optionally, the decision module 703 is further configured to: perform hash calculation using the target random number and request metadata to obtain a request flow identifier, wherein the target random number is used to adjust the scheduling randomness of the target request flow, and the request flow identifier is used to match the user request to the target request flow; and determine the request classification result based on the request flow identifier.

[0154] Optionally, the decision module 703 is further configured to: determine the target scheduling method corresponding to the user request from multiple candidate scheduling methods based on the policy matching result, wherein the multiple candidate scheduling methods include: quota scheduling method, concurrent scheduling method and load scheduling method; and make a scheduling decision on the user request using the target scheduling method to obtain the scheduling decision result.

[0155] Optionally, the decision module 703 is further configured to: obtain virtual completion time information and request attribute information corresponding to the user request, wherein the virtual completion time information is used to determine the order in which the user request is to be arranged, and the request attribute information is used to determine the order in which the user request is to be processed and the processing time; add the user request to the target queue position in the target request queue based on the virtual completion time information, and determine the target processing order and target processing time corresponding to the target queue position based on the request attribute information, wherein the target request queue is used to store multiple user requests arranged according to the virtual completion time information.

[0156] Optionally, the decision module 703 is further configured to: determine the number of available tokens based on the target scheduling method; make scheduling decisions on user requests using the number of available tokens, the estimated number of tokens corresponding to user requests, and request attribute information, and obtain scheduling decision results, wherein the estimated number of tokens is proportional to the target processing time.

[0157] Optionally, the decision module 703 is further configured to: in response to the target scheduling mode being quota scheduling mode, determine the number of available tokens from the shared token set, wherein the shared token set is used to determine the total number of available tokens shared by multiple workloads.

[0158] Optionally, the decision module 703 is further configured to: in response to the target scheduling mode being concurrent scheduling mode, determine the number of available tokens based on the total number of requests currently being processed concurrently.

[0159] Optionally, the decision module 703 is further configured to: in response to the target scheduling mode being load scheduling mode, determine the number of available tokens based on the incoming request rate of the user request and the token replenishment rate of the local token set, wherein the local token set is used to specify the total number of available tokens for each workload.

[0160] Optionally, the decision module 703 is also configured to: discard the user request in response to the user request not being scheduled within the scheduling time range.

[0161] Optionally, the decision module 703 is further configured to: perform circuit breaker assessment based on the collected metric data to obtain an assessment result, wherein the assessment result is used to determine whether load offloading needs to be performed by blocking user requests.

[0162] Optionally, the decision module 703 is further configured to: obtain request tag information corresponding to the user request, wherein the request tag information is used to identify the scheduling priority of the user request; and use a weighted fair queue mechanism to adaptively schedule the user request based on the request tag information to obtain the scheduling decision result.

[0163] Optionally, Figure 8 This is a structural block diagram of another traffic request scheduling device according to an embodiment of this application, such as... Figure 8 As shown, except Figure 7 In addition to the modules shown, the traffic request scheduling device also includes:

[0164] The second acquisition module 705 is used to acquire workload metadata corresponding to at least one workload, wherein the workload metadata is used to describe the load attribute information of at least one workload.

[0165] The first generation module 706 is used to generate a workload context based on workload metadata, wherein the workload context is used to represent context model data associated with at least one workload.

[0166] The second generation module 707 is used to generate extended configuration information using workload context and scheduling policy description information, wherein the extended configuration information is used to configure the scheduling enable method and scheduling parameter information associated with the service mesh agent;

[0167] Sending module 708 is used to send extended configuration information to the service mesh agent.

[0168] It should be noted that the first acquisition module 701, interception module 702, decision module 703, and forwarding module 704 mentioned above correspond to steps S41 to S44 in the above embodiments. The four modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be part of a device and run in the computer terminal 10 provided in the above embodiments.

[0169] It should be noted that the preferred embodiments involved in the above embodiments of this application are the same as the solutions, application scenarios and implementation processes provided in the above embodiments, but are not limited to the solutions provided in the above embodiments.

[0170] Embodiments of this application may provide an electronic device, which may be any one of a group of electronic devices. Optionally, in this embodiment, the aforementioned electronic device may also be replaced by a terminal device such as a mobile terminal.

[0171] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0172] In this embodiment, the computer terminal described above can execute the program code in the method.

[0173] Optionally, Figure 9 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 9 As shown, the electronic device may include: one or more (only one is shown in the figure) processors 902, memory 904, memory controller, and peripheral interfaces, wherein the peripheral interfaces are connected to a radio frequency module, an audio module, and a display.

[0174] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0175] The processor can invoke information and applications stored in memory via a transmission device to perform the following steps: obtaining user requests corresponding to at least one workload; intercepting and analyzing user requests using a service mesh proxy corresponding to at least one workload to obtain request metadata, wherein the request metadata is used to describe the request attributes of the user request; making scheduling decisions based on the request metadata to obtain scheduling decision results corresponding to the user request, wherein the scheduling decision results are used to determine the request processing method corresponding to the user request; and responding by accepting the user request based on the scheduling decision results and forwarding the user request to the target server.

[0176] Optionally, the processor may also execute program code that performs the following steps: classifies user requests based on request metadata to obtain request classification results, wherein the request classification results are used to determine the target request flow corresponding to the user request; performs policy matching using preset matching rules and request classification results to obtain policy matching results; and makes scheduling decisions on user requests based on policy matching results to obtain scheduling decision results.

[0177] Optionally, the processor may also execute program code that performs the following steps: performing a hash calculation using a target random number and request metadata to obtain a request stream identifier, wherein the target random number is used to adjust the scheduling randomness of the target request stream, and the request stream identifier is used to match user requests to the target request stream; and determining the request classification result based on the request stream identifier.

[0178] Optionally, the processor may also execute program code that performs the following steps: determining the target scheduling method corresponding to the user request from multiple candidate scheduling methods based on the policy matching result, wherein the multiple candidate scheduling methods include: quota scheduling method, concurrent scheduling method and load scheduling method; and making a scheduling decision on the user request using the target scheduling method to obtain the scheduling decision result.

[0179] Optionally, the processor may also execute program code that performs the following steps: obtaining virtual completion time information and request attribute information corresponding to a user request, wherein the virtual completion time information is used to determine the order in which user requests are to be arranged, and the request attribute information is used to determine the order in which user requests are to be processed and the processing time; adding the user request to the target queue position in the target request queue based on the virtual completion time information, and determining the target processing order and target processing time corresponding to the target queue position based on the request attribute information, wherein the target request queue is used to store multiple user requests arranged according to the virtual completion time information.

[0180] Optionally, the processor may also execute program code that performs the following steps: determining the number of available tokens based on the target scheduling method; making scheduling decisions for user requests using the number of available tokens, the estimated number of tokens corresponding to the user request, and request attribute information, and obtaining scheduling decision results, wherein the estimated number of tokens is proportional to the target processing time.

[0181] Optionally, the processor may also execute program code that performs the following steps: in response to the target scheduling mode being quota scheduling mode, determining the number of available tokens from a shared token set, wherein the shared token set is used to determine the total number of available tokens shared by multiple workloads.

[0182] Optionally, the processor may also execute program code that performs the following steps: in response to the target scheduling mode being concurrent scheduling mode, determines the number of available tokens based on the total number of requests currently being processed concurrently.

[0183] Optionally, the processor may also execute program code that performs the following steps: in response to the target scheduling mode being load scheduling, determining the number of available tokens based on the incoming request rate of the user request and the token replenishment rate of the local token set, wherein the local token set is the total number of available tokens exclusively for each workload.

[0184] Optionally, the processor may also execute program code that discards the user request in response to the user request not being scheduled within the scheduling timeframe.

[0185] Optionally, the processor may also execute program code that performs the following steps: performs circuit breaker assessment based on the collected metric data and obtains the assessment result, wherein the assessment result is used to determine whether load offloading needs to be performed by blocking user requests.

[0186] Optionally, the processor may also execute program code that performs the following steps: obtaining workload metadata corresponding to at least one workload, wherein the workload metadata is used to describe the load attribute information of at least one workload; generating a workload context based on the workload metadata, wherein the workload context is used to represent the context model data associated with at least one workload; generating extended configuration information using the workload context and scheduling policy description information, wherein the extended configuration information is used to configure the scheduling enable method and scheduling parameter information associated with the service mesh agent; and sending the extended configuration information to the service mesh agent.

[0187] Optionally, the processor may also execute program code that performs the following steps: obtaining request tag information corresponding to the user request, wherein the request tag information is used to identify the scheduling priority of the user request; and using a weighted fair queue mechanism to adaptively schedule the user request based on the request tag information to obtain the scheduling decision result.

[0188] By employing the embodiments of this application, user requests corresponding to at least one workload are obtained, and then the service mesh proxy corresponding to at least one workload is used to intercept and analyze the user requests to obtain request metadata. This achieves the interception and analysis of user requests, enabling a deeper understanding of user request behavior and needs, and providing more accurate data support for subsequent scheduling decisions. Furthermore, scheduling decisions are made based on the request metadata to obtain the scheduling decision result corresponding to the user request, thereby determining the request processing method. Finally, the response, based on the scheduling decision result, determines to accept the user request and forwards it to the target server to ensure that the user request receives a timely response and processing. By intelligently scheduling user requests, a stable service level can be maintained under different load scenarios, while achieving efficient resource utilization and fair allocation. This achieves the goal of efficient, fair, and flexible handling of high-concurrency traffic requests, improving the technical effect of scheduling efficiency and flexibility of traffic requests in the service mesh, and thus solving the technical problems of low scheduling efficiency and poor flexibility in related technologies when handling high-concurrency traffic requests within the service mesh.

[0189] It will be understood by those skilled in the art that the structure shown in the figure is merely illustrative, and the electronic device may also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal device. This figure does not limit the structure of the aforementioned electronic device. For example, the electronic device may include more or fewer components (such as a network interface, a display device, etc.) than shown in the figure, or may have a different configuration than that shown in the figure.

[0190] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0191] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store program code executed by the method provided in the above embodiments.

[0192] Optionally, in this embodiment, the storage medium may be located in any one of the electronic devices in the group of electronic devices in the computer network, or in any one of the mobile terminals in the group of mobile terminals.

[0193] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: obtaining user requests corresponding to at least one workload; intercepting and analyzing the user requests using a service mesh proxy corresponding to at least one workload to obtain request metadata, wherein the request metadata is used to describe the request attributes of the user requests; making scheduling decisions based on the request metadata to obtain scheduling decision results corresponding to the user requests, wherein the scheduling decision results are used to determine the request processing method corresponding to the user requests; and responding by accepting the user requests based on the scheduling decision results and forwarding the user requests to the target server.

[0194] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: classifying user requests based on request metadata to obtain request classification results, wherein the request classification results are used to determine the target request flow corresponding to the user request; performing policy matching using preset matching rules and request classification results to obtain policy matching results; and making scheduling decisions on user requests based on policy matching results to obtain scheduling decision results.

[0195] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: performing a hash calculation using a target random number and request metadata to obtain a request stream identifier, wherein the target random number is used to adjust the scheduling randomness of the target request stream, and the request stream identifier is used to match user requests to the target request stream; and determining the request classification result based on the request stream identifier.

[0196] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: determining the target scheduling method corresponding to the user request from multiple candidate scheduling methods based on the policy matching result, wherein the multiple candidate scheduling methods include: quota scheduling method, concurrent scheduling method and load scheduling method; making a scheduling decision on the user request using the target scheduling method to obtain the scheduling decision result.

[0197] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: obtaining virtual completion time information and request attribute information corresponding to a user request, wherein the virtual completion time information is used to determine the order in which the user request is to be arranged, and the request attribute information is used to determine the order in which the user request is to be processed and the processing duration; adding the user request to a target queue position in a target request queue based on the virtual completion time information, and determining the target processing order and target processing duration corresponding to the target queue position based on the request attribute information, wherein the target request queue is used to store multiple user requests arranged according to the virtual completion time information.

[0198] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: determining the number of available tokens based on the target scheduling method; making a scheduling decision on the user request using the number of available tokens, the estimated number of tokens corresponding to the user request, and request attribute information, and obtaining a scheduling decision result, wherein the estimated number of tokens is proportional to the target processing time.

[0199] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: in response to a target scheduling mode being quota scheduling mode, determining the number of available tokens from a shared token set, wherein the shared token set is used to determine the total number of available tokens shared by multiple workloads.

[0200] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: in response to the target scheduling mode being concurrent scheduling mode, determining the number of available tokens based on the total number of requests currently being processed concurrently.

[0201] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: in response to the target scheduling mode being load scheduling, determining the number of available tokens based on the incoming request rate of user requests and the token replenishment rate of the local token set, wherein the local token set is the total number of available tokens exclusively for each workload.

[0202] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: discarding the user request in response to the user request not being scheduled within the scheduling timeframe.

[0203] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: performing a circuit breaker assessment based on the collected metric data to obtain an assessment result, wherein the assessment result is used to determine whether load offloading needs to be performed by blocking user requests.

[0204] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: obtaining workload metadata corresponding to at least one workload, wherein the workload metadata is used to describe the load attribute information of at least one workload; generating a workload context based on the workload metadata, wherein the workload context is used to represent context model data associated with at least one workload; generating extended configuration information using the workload context and scheduling policy description information, wherein the extended configuration information is used to configure the scheduling enable method and scheduling parameter information associated with the service mesh agent; and sending the extended configuration information to the service mesh agent.

[0205] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: obtaining request tag information corresponding to a user request, wherein the request tag information is used to identify the scheduling priority of the user request; and adaptively scheduling the user request based on the request tag information using a weighted fair queue mechanism to obtain a scheduling decision result.

[0206] By employing the embodiments of this application, user requests corresponding to at least one workload are obtained, and then the service mesh proxy corresponding to at least one workload is used to intercept and analyze the user requests to obtain request metadata. This achieves the interception and analysis of user requests, enabling a deeper understanding of user request behavior and needs, and providing more accurate data support for subsequent scheduling decisions. Furthermore, scheduling decisions are made based on the request metadata to obtain the scheduling decision result corresponding to the user request, thereby determining the request processing method. Finally, the response, based on the scheduling decision result, determines to accept the user request and forwards it to the target server to ensure that the user request receives a timely response and processing. By intelligently scheduling user requests, a stable service level can be maintained under different load scenarios, while achieving efficient resource utilization and fair allocation. This achieves the goal of efficient, fair, and flexible handling of high-concurrency traffic requests, improving the technical effect of scheduling efficiency and flexibility of traffic requests in the service mesh, and thus solving the technical problems of low scheduling efficiency and poor flexibility in related technologies when handling high-concurrency traffic requests within the service mesh.

[0207] Embodiments of this application also provide a traffic request scheduling system, including: a service mesh control plane and a service mesh data plane, wherein:

[0208] The service mesh control plane includes a load scheduling policy generator, which generates extended configuration information based on the workload context and scheduling policy description information, and sends the extended configuration information to the service mesh data plane. The workload context is generated based on the workload metadata corresponding to at least one workload. The workload context is used to represent the context model data associated with at least one workload. The extended configuration information is used to configure the scheduling enable method and scheduling parameter information associated with the service mesh agent.

[0209] The service mesh data plane includes: at least one workload and at least one service mesh proxy and request flow controller corresponding to the workload. The service mesh proxy is used to intercept and analyze user requests corresponding to at least one workload based on extended configuration information to obtain request metadata, wherein the request metadata is used to describe the request attributes of the user request.

[0210] The request flow controller is used to make scheduling decisions based on request metadata, obtain the scheduling decision result corresponding to the user request, and respond by determining whether to accept the user request based on the scheduling decision result and forwarding the user request to the target server. The scheduling decision result is used to determine the request processing method corresponding to the user request.

[0211] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the methods provided in the embodiments described above.

[0212] Embodiments of this application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which can be used to store a computer program that, when executed by a processor, implements the method provided in the above embodiments.

[0213] Embodiments of this application also provide a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it implements the method provided in the above embodiments.

[0214] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0215] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0216] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0217] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0218] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0219] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A traffic request scheduling method, characterized in that, include: Obtain at least one user request corresponding to a workload; The user request is intercepted and analyzed using the service mesh proxy corresponding to the at least one workload to obtain request metadata, wherein the request metadata is used to describe the request attributes of the user request; A scheduling decision is made based on the request metadata to obtain the scheduling decision result corresponding to the user request, wherein the scheduling decision result is used to determine the request processing method corresponding to the user request; The response determines to accept the user request based on the scheduling decision result and forwards the user request to the target server.

2. The method according to claim 1, characterized in that, Based on the request metadata, a scheduling decision is made to obtain the scheduling decision result corresponding to the user request, including: The user request is classified based on the request metadata to obtain a request classification result, wherein the request classification result is used to determine the target request flow corresponding to the user request; The strategy matching result is obtained by using the preset matching rules and the request classification results; Based on the strategy matching result, a scheduling decision is made on the user request to obtain the scheduling decision result.

3. The method according to claim 2, characterized in that, The user request is categorized based on the request metadata to obtain the following categorization results: A request stream identifier is obtained by performing a hash calculation using a target random number and the request metadata, wherein the target random number is used to adjust the scheduling randomness of the target request stream, and the request stream identifier is used to match the user request to the target request stream; The request classification result is determined based on the request flow identifier.

4. The method according to claim 2, characterized in that, Based on the strategy matching result, a scheduling decision is made on the user request, and the scheduling decision result includes: Based on the strategy matching result, the target scheduling method corresponding to the user request is determined from multiple candidate scheduling methods, wherein the multiple candidate scheduling methods include: quota scheduling method, concurrent scheduling method and load scheduling method; The user request is scheduled using the target scheduling method to obtain the scheduling decision result.

5. The method according to claim 4, characterized in that, The method further includes: Obtain the virtual completion time information and request attribute information corresponding to the user request, wherein the virtual completion time information is used to determine the order in which the user request is to be arranged, and the request attribute information is used to determine the order in which the user request is to be processed and the processing time. The user request is added to the target queue position in the target request queue based on the virtual completion time information, and the target processing order and target processing time corresponding to the target queue position are determined based on the request attribute information. The target request queue is used to store multiple user requests arranged according to the virtual completion time information.

6. The method according to claim 5, characterized in that, The scheduling decision for the user request is made using the target scheduling method, and the scheduling decision result includes: The number of available tokens is determined based on the target scheduling method; The scheduling decision is made on the user request using the number of available tokens, the estimated number of tokens corresponding to the user request, and the request attribute information, and the scheduling decision result is obtained, wherein the estimated number of tokens is proportional to the target processing time.

7. The method according to claim 6, characterized in that, Determining the number of available tokens based on the target scheduling method includes: In response to the target scheduling mode being the quota scheduling mode, the number of available tokens is determined from the shared token set, wherein the shared token set is used to determine the total number of available tokens shared by multiple workloads.

8. The method according to claim 6, characterized in that, Determining the number of available tokens based on the target scheduling method includes: In response to the target scheduling mode being the concurrent scheduling mode, the number of available tokens is determined based on the total number of requests currently being processed concurrently.

9. The method according to claim 6, characterized in that, Determining the number of available tokens based on the target scheduling method includes: In response to the target scheduling mode being the load scheduling mode, the number of available tokens is determined based on the incoming request rate of the user request and the token replenishment rate of the local token set, wherein the local token set is the total number of available tokens exclusively for each workload.

10. The method according to claim 4, characterized in that, The method further includes: If the user request is not scheduled within the scheduling time frame, the user request is discarded.

11. The method according to claim 4, characterized in that, The method further includes: Based on the collected metric data, a circuit breaker assessment is performed to obtain an assessment result, which is used to determine whether load offloading needs to be performed by blocking the user request.

12. The method according to claim 4, characterized in that, The method further includes: Obtain the workload metadata corresponding to the at least one workload, wherein the workload metadata is used to describe the load attribute information of the at least one workload; A workload context is generated based on the workload metadata, wherein the workload context is used to represent context model data associated with the at least one workload; Extended configuration information is generated using the workload context and scheduling policy description information, wherein the extended configuration information is used to configure the scheduling enable method and scheduling parameter information associated with the service mesh agent; The extended configuration information is sent to the service mesh agent.

13. The method according to claim 1, characterized in that, After intercepting and analyzing the user requests using the service mesh proxy corresponding to the at least one workload, the method further includes: Obtain the request tag information corresponding to the user request, wherein the request tag information is used to identify the scheduling priority of the user request; The user requests are adaptively scheduled based on the request tag information using a weighted fair queue mechanism to obtain the scheduling decision result.

14. A traffic request scheduling system, characterized in that, include: Service mesh control plane and service mesh data plane; The service mesh control plane includes a load scheduling policy generator, which generates extended configuration information based on workload context and scheduling policy description information, and sends the extended configuration information to the service mesh data plane. The workload context is generated based on workload metadata corresponding to at least one workload, and the workload context is used to represent the context model data associated with the at least one workload. The extended configuration information is used to configure the scheduling enable method and scheduling parameter information associated with the service mesh agent. The service mesh data plane includes: the at least one workload and the service mesh proxy and request flow controller corresponding to the at least one workload. The service mesh proxy is used to intercept and analyze user requests corresponding to the at least one workload based on the extended configuration information to obtain request metadata, wherein the request metadata is used to describe the request attributes of the user request. The request flow controller is used to make scheduling decisions based on the request metadata, obtain the scheduling decision result corresponding to the user request, and respond by determining to receive the user request based on the scheduling decision result and forwarding the user request to the target server. The scheduling decision result is used to determine the request processing method corresponding to the user request.

15. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program executes the traffic request scheduling method according to any one of claims 1 to 13 when it runs.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the traffic request scheduling method according to any one of claims 1 to 13.

17. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the traffic request scheduling method according to any one of claims 1 to 13.

Images