Service mesh scheduling method, apparatus, device, and medium

Abstract

The present application relates to the technical field of cloud computing, and discloses a service grid scheduling method and device, equipment and medium, the service grid comprises a plurality of service nodes, each service node comprises a plurality of service instances, each service instance is co-located with a proxy node, the method is applied to a target proxy node, the target proxy node is co-located with a current service instance in a plurality of service instances, comprising: obtaining service chain information of a request to be processed;The service chain information is sent to the current service instance;After receiving the information indicating that the current service instance is executed, the next service node of the target proxy node in the service chain information is determined;According to the running state of the service instance, the plurality of service instances of the next service node are screened to obtain a target service instance;The request to be processed is sent to the proxy node co-located with the target service instance. It can be applied to the service grid scheduling scene of financial technology and medical health, and the success rate of scheduling decision is improved.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and in particular to a service grid scheduling method, apparatus, device, and medium. Background Technology

[0002] With the deep integration of cloud computing and edge computing, the Cloud-Edge Continuum (CEC) has become a crucial infrastructure supporting low-latency, high-concurrency business applications. In scenarios such as real-time transaction risk control in fintech and remote image diagnosis in healthcare, a single user request often requires sequentially calling multiple microservices deployed in different geographical regions to form a service chain, which then coordinates between edge nodes and cloud data centers. This places stringent requirements on end-to-end response latency and service reliability, making efficient scheduling of the service chain a key factor in ensuring a smooth business experience.

[0003] However, existing microservice link scheduling mainly relies on a centralized controller, which needs to aggregate global state in real time for overall path optimization. But in a highly dynamic cloud-edge environment, due to the frequent changes in service instance state, the global state information on which the centralized scheduler relies has an unavoidable synchronization delay. This causes scheduling decisions to be based on outdated information, which can easily lead to requests being incorrectly distributed to overloaded or failed instances, resulting in decision failure. Summary of the Invention

[0004] This invention provides an artificial intelligence service grid scheduling method, apparatus, computer equipment, and medium to solve the technical problem of decision failure caused by state synchronization delay in existing centralized scheduling.

[0005] Firstly, a service mesh scheduling method is provided, wherein the service mesh includes multiple service nodes, each service node includes multiple service instances, and each service instance shares a proxy node. The method is applied to a target proxy node, wherein the target proxy node shares a proxy node with the current service instance among the multiple service instances, including: Obtain service chain information for the pending request. The service chain information indicates the M service nodes that the pending request needs to pass through sequentially. The M service nodes include the target proxy node. The pending request is used to request the execution of a target scheduling task. Send the service chain information to the current service instance; After receiving the information indicating that the current service instance has completed execution, determine the next service node of the target proxy node in the service chain information; Multiple service instances of the next service node are filtered according to the running status of the service instances to obtain target service instances. The running status is used to characterize the real-time acceptance capacity and processing efficiency of the service instance when processing the pending request. The pending request is sent to the proxy node co-located with the target service instance.

[0006] Secondly, a service mesh scheduling device is provided, wherein the service mesh includes multiple service nodes, each service node includes multiple service instances, each service instance shares a proxy node, the device is applied to a target proxy node, and the target proxy node shares a proxy node with the current service instance among the multiple service instances, including: The acquisition module is used to acquire service chain information of the request to be processed. The service chain information indicates the M service nodes that the request to be processed needs to pass through in sequence. The M service nodes include the target proxy node. The request to be processed is used to request the execution of the target scheduling task. The first sending module is used to send the service chain information to the current service instance; The determination module is used to determine the next service node of the target proxy node in the service chain information after receiving the information indicating that the current service instance has completed execution; The filtering module is used to filter multiple service instances of the next service node according to the running status of the service instances to obtain target service instances. The running status is used to characterize the real-time acceptance capacity and processing efficiency of the service instance when processing the pending request. The second sending module is used to send the request to be processed to the proxy node co-located with the target service instance.

[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the service mesh scheduling method described above.

[0008] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described service grid scheduling method.

[0009] In the above-mentioned service mesh scheduling method, device, computer equipment and storage medium, the service chain information of the request to be processed can be obtained through the proxy node co-located with the current service instance. After the current service instance is completed, the target service instance is determined by filtering according to the real-time acceptance capacity and processing efficiency of each service instance in the next service node. Then, the request to be processed is forwarded to the proxy node co-located with the target service instance, so that the request is executed hop by hop along the service chain. In this invention, for business scenarios with stringent requirements for end-to-end latency and system reliability, such as financial risk control and medical diagnosis, such as real-time financial transaction risk control or remote medical image diagnosis, a decentralized hop-by-hop scheduling scheme can be used. The proxy nodes deployed next to each service instance decide the next hop target locally based on the running status of the local cache. The selection and forwarding are directly based on the immediate handling capacity (such as remaining concurrent capacity and queue backlog) and processing efficiency (such as network transmission time and service execution time) of each candidate instance, without the need for a centralized controller. This effectively avoids scheduling decision failures caused by global state synchronization lag, eliminates centralized bottlenecks and single point of failure risks, significantly reduces scheduling-related latency, prevents traffic oscillations and request accumulation in high-concurrency scenarios, and ensures the stable flow of critical business requests in cross-domain service links, thereby improving the response speed and overall availability of core business links in the cloud-edge continuum. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an application environment for a service grid scheduling method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a service grid scheduling method according to an embodiment of the present invention; Figure 3 yes Figure 2 A schematic diagram of a specific implementation of step S40; Figure 4 This is a flowchart of a decentralized hop-by-hop scheduling method based on a service mesh sidecar proxy in one embodiment of the present invention; Figure 5 This is a schematic diagram of a service grid scheduling device in one embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 7This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] The service mesh scheduling method provided in this embodiment of the invention can be applied to, for example, Figure 1 In this application environment, the client communicates with the server via a network. The server is a service mesh deployed in a cloud-edge system. The service mesh includes an ingress gateway, multiple service nodes, and eventually consistent distributed metadata storage. Each service node contains at least one service instance and is co-located with a sidecar proxy (i.e., a proxy node). When a client initiates a pending request, the request is received by the ingress gateway and forwarded to the sidecar proxy corresponding to the first service instance in the service chain. This sidecar proxy becomes the target proxy node. Based on the service chain information carried in the request, the target proxy node uses locally cached eventually consistent metadata (such as the resource availability, queue length, network latency, and other running status of each instance) to perform feasibility domain filtering and scoring with random perturbations on multiple service instances of the next service node. After selecting the optimal target service instance, the request is forwarded hop-by-hop to the proxy node co-located with the target service instance, and so on, until the end of the service chain, finally returning the response to the client. In this invention, by decentralizing scheduling decision-making capabilities to the sidecar agent and utilizing an eventual consistency local cache and a random perturbation scoring mechanism for hop-by-hop intelligent scheduling, decentralized, low-latency, and highly scalable service chain request routing can be achieved without modifying the business application code. This effectively eliminates the performance bottlenecks and single points of failure of centralized schedulers, ensuring the stability of cloud-edge service scheduling. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0014] Please see Figure 2 As shown, Figure 2This is a flowchart illustrating a service mesh scheduling method provided in an embodiment of the present invention. The service mesh includes multiple service nodes, each service node includes multiple service instances, and each service instance shares a proxy node. The service mesh scheduling method provided in this embodiment of the present invention is applied to a target proxy node, which shares a proxy node with the current service instance among the multiple service instances. The method includes the following steps: S10: Obtain the service chain information of the pending request. The service chain information indicates the M service nodes that the pending request needs to pass through in sequence. The M service nodes include the target proxy node. The pending request is used to request the execution of the target scheduling task.

[0015] The service mesh scheduling method provided by this invention can be applied to microservice systems in a cloud computing and edge computing continuum environment. It is executed by proxy nodes deployed next to each service instance to achieve decentralized, hop-by-hop scheduling of service chains, making it particularly suitable for scenarios requiring hop-by-hop scheduling of service chain requests across multiple microservices. For example, in the fintech field, a cross-border payment request typically needs to pass through multiple microservice nodes sequentially, such as anti-fraud verification, currency conversion, account deduction, and transaction recording. The service chain information indicates the order and identifier of these nodes. In the healthcare field, the service chain of a remote consultation request may include nodes such as patient authentication, electronic health record retrieval, AI-assisted diagnosis, and medication recommendation. When a client initiates a pending request, the target proxy node co-located with the current service instance first obtains the service chain information corresponding to the request. This service chain information clearly indicates the multiple service nodes the request needs to pass through sequentially and the execution order of each service node, enabling the proxy node to identify the position of the current request in the entire business chain and the target service node that needs to be scheduled subsequently.

[0016] Specifically, the service chain information can be an ordered list of service identifiers, such as SC=[S1, S2, …,Sm], indicating which service nodes the request needs to pass through sequentially to complete the entire target scheduling task. Upon receiving the request, the target proxy node parses and obtains the service chain information, determining its position in the service chain and the execution path of the entire chain. The service chain information can be pre-configured by the system administrator, specified by the client when initiating the request, or dynamically generated through the control plane of the service mesh; this invention does not impose specific limitations on this.

[0017] S20: Send the service chain information to the current service instance.

[0018] After acquiring the service chain information, the target proxy node forwards the service chain information to the co-located current service instance, enabling the current service instance to execute the corresponding service based on the service chain information. Upon receiving the service chain information, the current service instance executes the corresponding business logic processing according to the current node task indicated in the service chain information, such as performing user identity verification in a financial transaction scenario or performing image format standardization conversion in a medical imaging scenario. During this process, the proxy node continuously monitors the execution status of the current service instance, waiting for it to complete the current node's service processing.

[0019] S30: After receiving the information indicating that the current service instance has completed execution, determine the next service node of the target proxy node in the service chain information.

[0020] After the current service instance completes its business logic, it returns the processing result to the shared target proxy node, indicating that execution is complete. Upon receiving this indication, the target proxy node, based on the service chain information, resolves the next service node to be called after the current service node. For example, if the service chain is SC=[S1, S2, S3], and the current service node is S1, then the target proxy node determines the next service node to be S2. This determination process is completed entirely locally, without needing to query any centralized scheduler, thus avoiding network round-trip latency and processing bottlenecks at central nodes.

[0021] S40: Filter the multiple service instances of the next service node according to the running status of the service instances to obtain the target service instance. The running status is used to characterize the real-time acceptance capability and processing efficiency of the service instance when processing the pending request.

[0022] After determining the next service node, the target proxy node needs to select the most suitable target service instance from the multiple service instances deployed on that next service node to handle the current request. The operational status comprehensively characterizes the availability level of each candidate service instance from two dimensions: immediate handling capacity and processing efficiency. Immediate handling capacity includes the remaining concurrent capacity and queue length of the candidate service instance. Remaining concurrent capacity represents the number of additional requests the instance can currently accept, and queue length represents the backlog of requests currently awaiting processing. Processing efficiency includes the estimated network transmission time from the current proxy node to the candidate service instance and the estimated average execution time of the corresponding service executed by the candidate service instance. The estimated network transmission time is the predicted network transmission time from the current proxy node to the next service instance, and the estimated execution time is the predicted average time for the service instance to process requests, characterizing the processing efficiency of the service instance.

[0023] Based on the aforementioned operational status, the target proxy node performs feasibility domain screening on each candidate service instance. Specifically, candidate service instances with remaining concurrent capacity less than or equal to a preset buffer threshold are eliminated to avoid scheduling requests to already saturated instances; candidate service instances with estimated total latency greater than the remaining response time of pending requests are also eliminated to ensure that end-to-end latency constraints are not breached. After feasibility domain screening, the remaining candidate service instances undergo comprehensive priority evaluation. Their scheduling priority is determined based on resource consumption costs, network transmission time, queue congestion levels, and status timeliness, and the candidate service instance with the highest priority is selected as the target service instance. Status timeliness is determined based on the degree of invalidation of operational status information; the longer the information is invalidated, the higher the timeliness penalty, reducing the risk of making incorrect decisions based on outdated information. This screening process is entirely completed on the target proxy node's local cache, without requiring requests to a centralized scheduler, resulting in extremely low scheduling latency.

[0024] In some embodiments of the invention, such as Figure 3 As shown, S40 specifically includes the following steps S41-S43: S41: Filter multiple service instances of the next service node to obtain multiple first candidate service instances, and the running status of each first candidate service instance.

[0025] In this embodiment of the invention, after determining the next service node, the target proxy node first queries its locally maintained metadata cache to retrieve all currently registered service instances of that next service node. The purpose of this initial screening is to eliminate instances that are clearly unavailable, such as those in a circuit breaker-broken state, those marked as unhealthy, or those temporarily unavailable due to ongoing rolling updates. The instances retained after this round of screening are the first candidate service instances.

[0026] Simultaneously, the target proxy node reads the runtime status information corresponding to each first candidate service instance from its local cache. This runtime status information characterizes the service instance's immediate capacity and processing efficiency when handling pending requests, and may include metrics such as resource availability, queue length or remaining concurrent capacity, and network latency estimates. This information is asynchronously synchronized to the local cache of each proxy node through an eventually consistent distributed metadata store, ensuring that the target proxy node can efficiently read it locally without initiating real-time queries to the centralized controller.

[0027] For example, in a real-time transaction risk control scenario in the fintech field, the next service node is a risk control rule verification service node. This node has three service instances deployed in the cloud region and two service instances deployed in the edge region. The target proxy node first eliminates service instances that clearly do not meet the basic deployment conditions (such as instances that are unreachable due to regional isolation policies) based on the node constraints in the service chain information, and determines all remaining available service instances as the first candidate service instances.

[0028] S42: Based on the running status of each first candidate service instance and the remaining processing time of the service chain information, select multiple second candidate service instances from the multiple first candidate service instances.

[0029] In this embodiment of the invention, after obtaining the first candidate service instance and its running status, the target proxy node needs to further filter the feasible domains based on the end-to-end constraints of the request to be processed. The service chain information may carry the end-to-end latency budget of the request. During each hop scheduling, this budget is subtracted from the consumed time to obtain the remaining processing time. The remaining processing time represents how much time subsequent service nodes need to complete the processing in order to meet the overall service quality requirements.

[0030] The target proxy node estimates the estimated time required to schedule a request to and process it for each first candidate service instance based on its running status. This estimated time may include network transmission latency, service instance execution latency, and queuing latency due to waiting in the queue. If the estimated time exceeds the remaining processing time of the service chain information, or if the remaining concurrent capacity of the candidate instance is insufficient to handle new requests, the candidate instance is determined not to meet the timeliness or capacity constraints of the current request and is eliminated. The first candidate service instances that meet the above constraints are retained as second candidate service instances.

[0031] In some embodiments of this invention, the operating state includes the remaining concurrent capacity of the service instance, queue length, estimated network transmission time, estimated service execution time, and available resources. S42 specifically includes the following steps S421-S423: S421: Remove service instances whose remaining concurrent capacity is less than or equal to a preset capacity threshold from a plurality of first candidate service instances; S422: Remove service instances from the plurality of first candidate service instances whose estimated total time is greater than the remaining processing time. The estimated total time includes the estimated queue blocking time from the target proxy node to the service instance, the estimated network transmission time, and the estimated service execution time of the service instance. The estimated queue blocking time is calculated by the queue length of the service instance and the remaining concurrent capacity of the service instance. S423: The remaining service instances are identified as multiple second candidate service instances.

[0032] In this embodiment of the invention, when the target proxy node performs feasible domain screening, it prioritizes the immediate processing capacity of candidate service instances. The remaining concurrent capacity is a key indicator of whether a service instance has the ability to handle new requests. It can be directly expressed as how many concurrent requests the instance can still process simultaneously, or it can be calculated by subtracting the current number of active connections from the instance's maximum concurrent connections. The preset capacity threshold is a configurable empirical parameter, typically greater than zero, for example, it can be set to 1 or a low buffer value. When the remaining concurrent capacity of a first candidate service instance is less than or equal to the preset capacity threshold, it indicates that the instance is currently operating at or near full capacity. Forcibly scheduling new pending requests to it would not only cause a sharp increase in the queuing time of requests, but might even trigger the instance's overload protection mechanism and directly reject the requests. Therefore, removing such instances from the candidate set in advance can effectively avoid distributing traffic to already saturated instances, ensuring the timeliness and success rate of request responses.

[0033] After filtering by capacity, the target proxy node further filters the remaining candidate instances by latency constraints. The core of this step is to estimate the total estimated time required to schedule requests to be processed to that instance and complete the processing for each candidate service instance, and compare this estimate with the remaining processing time of the service chain information. The estimated total time is a comprehensive latency metric, composed of queue blocking time estimates, network transmission time estimates, and service execution time estimates. The queue blocking time estimate is calculated from the queue length of the service instance and the remaining concurrent capacity of the service instance. The network transmission time estimate represents the estimated round-trip or one-way network transmission latency required to send a request from the current target proxy node to the location of the candidate service instance via the network, and to receive the processing result. This can be obtained by the sidecar proxy through smoothed statistics based on historical observation data, for example, dynamically updated using an exponentially weighted moving average algorithm. The service execution time estimate represents the estimated processing time required for the candidate service instance to actually execute the business logic. This can also be obtained based on the historical execution time data of the instance and can be dynamically adjusted according to the available resources such as CPU and memory of the node where the instance resides.

[0034] The target proxy node compares the estimated total processing time with the remaining processing time carried in the service chain information of each pending request. If the estimated total processing time of a candidate instance is greater than the remaining processing time, it indicates that even if the request is scheduled to that instance, it is highly likely that it will not be processed within the end-to-end latency budget, thus violating the quality of service requirements. Such instances will be removed from the candidate pool. The target proxy node then identifies the remaining service instances after the removal as multiple second-candidate service instances.

[0035] Through the above mechanism, the embodiments of the present invention refine the operating status into specific and quantifiable key performance indicators, and effectively narrow the candidate range by screening through hard constraints of capacity and latency, so that the subsequent intelligent scoring steps can perform fine selection on a set of truly feasible instances; this not only improves the overall quality of scheduling decisions, but also avoids service quality degradation caused by scheduling requests to overloaded or untimely instances, further enhancing the reliability and efficiency of end-to-end execution of the service chain in the cloud-edge system.

[0036] In some embodiments of the invention, S42 further includes the following step S424: S424: If a second candidate service instance cannot be selected from multiple first candidate service instances, or the running status is missing, a controlled rollback is executed. The controlled rollback is used to instruct the current service instance to execute the backup strategy in sequence according to a preset priority order.

[0037] Among them, the controlled rollback includes: Select a service instance located within the region to which the current proxy node belongs as the target service instance; If there are no available service instances within the region to which the current proxy node belongs, then the service instance selected when executing the previous service node in the current service chain is selected as the target service instance.

[0038] In the actual operation of cloud-edge systems, there are various abnormal scenarios that may lead to failure of feasible domain filtering or unavailable metadata. For example, in the event of network partitioning or partial failure, the local cache of the target proxy node may exceed its Time-to-Live (TTL) due to not receiving metadata updates for an extended period, resulting in the invalidation or loss of all runtime status information. Alternatively, in some extremely high-load scenarios, the remaining concurrent capacity of all first-candidate service instances may be lower than the preset capacity threshold, or the estimated total latency may exceed the remaining processing time of the service chain information, resulting in no candidate instance meeting the conditions after dual filtering by capacity and latency. In these cases, if the target proxy node simply discards the request and returns a failure, it will significantly reduce the end-to-end success rate of the service chain and the overall availability of the system.

[0039] To address the aforementioned issues, this invention provides a controlled rollback mechanism with a preset priority order. This mechanism enables the target proxy node to attempt to execute backup strategies sequentially according to a predetermined priority order when it is unable to select the optimal instance based on conventional scoring logic, thereby maximizing the protection of the continued processing of requests.

[0040] The first priority of controlled rollback is to select service instances within the region where the current proxy node belongs. Specifically, service instances within the region of the current proxy node are better suited to the current environment, have lower network latency, and more timely state synchronization, reducing the risk of cross-regional scheduling. For example, if the current proxy node is located on the edge, service instances on the edge are selected first; if no available instances are available on the edge, the cloud is then considered. In one implementation, region selection can be determined based on geographical distance, network topology, or resource ownership (such as instances managed by the edge node), ensuring that the selected instances are within the jurisdiction of the current proxy node, facilitating state monitoring and fault recovery.

[0041] If no instance is available in the current region, the second priority of controlled rollback is to select an instance already selected by the previous service node. Instances selected by the previous service node have been verified as available in the service chain, and their running status was valid when the previous node processed the request. Although their status may change over time, their reliability is higher than random selection. For example, if the service chain is "Client → Ingress Gateway → S1 → S2 → S3", and the current filtering at node S2 fails, then rollback to the instance selected by node S1 (e.g., the instance from S1 to S2) ensures that the request traces back along the original path, avoiding uncertainty caused by introducing new unknown instances. In one implementation, if the instance selected by the previous service node is still unavailable, rollback can continue to earlier nodes until an available instance is found or the request is terminated (e.g., all nodes are unavailable).

[0042] It should be noted that the above two-level fallback strategy is executed sequentially according to a preset priority order: the target proxy node first tries the local domain priority strategy. If the strategy successfully finds an available instance, it is immediately identified as the target service instance and the request is forwarded. If the local domain priority strategy fails, it continues to try the last successful path strategy. If all backup strategies fail to find an available target service instance, the request dropping or rate limiting mechanism is finally triggered, and a failure response is returned to the client. At the same time, the corresponding failure reasons and context indicators are recorded so that operation and maintenance personnel can troubleshoot problems and optimize the system.

[0043] Through the aforementioned multi-level, controlled rollback and fault-tolerance mechanisms, this invention effectively solves the problem of scheduling decision failure caused by abnormal scenarios such as missing metadata, network partitions, or extreme loads under a decentralized architecture. The system does not simply return failure when an anomaly occurs, but rather rolls back step-by-step to a more stable and predictable scheduling path according to a preset priority order. This allows requests to continue to the maximum extent possible even in the event of partial metadata anomalies or localized failures, significantly enhancing the overall fault tolerance and service availability of the cloud-edge system in complex operating environments.

[0044] S43: Scoring multiple second candidate service instances, and filtering multiple second candidate service instances based on the score of each second candidate service instance to obtain a target service instance, wherein the score is used to quantify the degree of fit between the second candidate service instance and the target scheduling task.

[0045] In this embodiment of the invention, after the feasible domain screening, there may still be multiple second candidate service instances that meet the conditions. At this time, the target proxy node needs to perform a comprehensive evaluation on them to select the instance that is most suitable for the current target scheduling task. The evaluation comprehensively considers multiple dimensions such as resource consumption cost, latency penalty, queue blocking penalty, and state timeliness penalty. The lower the score, the higher the degree of suitability.

[0046] In one alternative implementation, the scoring function can be expressed as: .

[0047] in, This indicates that the service will be executed in the region where the candidate instance is located. Resource costs; This represents the sum of the estimated network latency from the current service to the candidate instance and the estimated service execution latency. This represents the penalty calculated from the queue length or remaining concurrent capacity. This indicates a penalty based on metadata freshness. A configurable small random number is used to introduce random perturbations among candidate instances with similar scores to avoid multiple sidecars making completely identical decisions at the same time. The weighting parameter is used to balance the importance of cost, latency, queue, and freshness penalty. The specific value can be determined through offline simulation (e.g., simulating different load scenarios on simulation platforms such as SimGrid) or online A / B testing to match specific business optimization goals (such as cost-sensitive or latency-sensitive).

[0048] In some embodiments of the invention, S43 specifically includes the following steps S431-S435: S431: For each of the multiple second candidate service instances, a cost penalty value is determined based on the resource consumption cost of the region where the service instance is located, wherein the larger the resource consumption cost, the larger the cost penalty value.

[0049] In this embodiment of the invention, in a multi-region cloud-edge environment, the costs of computing resources, storage resources, and network bandwidth may vary significantly across different regions. For example, due to economies of scale, the cost per unit of computing resource in a central cloud data center may be relatively low; while the operating cost per unit of resource in edge computing nodes is typically higher due to limitations in physical space, power supply, and heat dissipation. To optimize the cost of global resources while meeting service quality requirements, this step introduces a quantitative assessment of the cost dimension.

[0050] Specifically, for each second candidate service instance, the target proxy node determines a cost penalty value based on the resource consumption cost of the region where the instance is located. This penalty value is positively correlated with the resource consumption cost; that is, the higher the resource consumption cost, the higher the cost penalty value. The specific value of the resource consumption cost can be pre-configured in the metadata cache of each proxy node and set by operations personnel according to the pricing strategy of different regions, or it can be dynamically provided by the cloud platform via API. By incorporating cost factors into the scoring system, scheduling decisions can prioritize instances with lower execution costs among candidate instances with similar performance, thereby reducing the overall resource consumption overhead of the service chain.

[0051] S432: Determine a latency penalty value based on the sum of the estimated network transmission time from the target proxy node to the service instance and the estimated service execution time, wherein the larger the sum of the estimated network transmission time and the estimated service execution time, the larger the latency penalty value.

[0052] In this embodiment of the invention, the end-to-end response time of a service chain request is one of the core indicators for measuring service quality, and the latency performance of a single hop directly affects the latency budget allocation of the overall service chain. This step evaluates candidate service instances from a latency perspective.

[0053] Specifically, the target proxy node obtains the estimated network transmission time from the current node to the candidate service instance, and the estimated service execution time of the candidate service instance, and calculates their sum. The estimated network transmission time reflects the transmission latency of cross-region or intra-region network links, while the estimated service execution time reflects the instance's own processing capacity. The larger the sum of the two, the slower the expected response when scheduling the request to that instance, and therefore the larger the latency penalty value. This positive penalty design allows the scoring function to naturally favor instances with shorter network distances and faster processing speeds, thereby effectively compressing the response time of each hop and leaving more latency budget for subsequent nodes in the service chain.

[0054] S433: Determine a queue blocking penalty value based on the ratio of the queue length of the service instance to the remaining concurrent capacity of the service instance, wherein the larger the ratio of the queue length to the remaining concurrent capacity, the larger the queue blocking penalty value.

[0055] In a multi-threaded or asynchronous microservice architecture, service instances typically maintain a request queue. When the request arrival rate exceeds the instance's processing rate, the queue length gradually increases, and newly arriving requests need to endure additional queuing time. This step incorporates the current load pressure of the candidate instance into the scoring consideration by introducing a queue blocking penalty value.

[0056] Specifically, the target proxy node obtains the queue length and remaining concurrent capacity of the service instance and calculates their ratio. This ratio reflects the current load saturation of the instance: the longer the queue and the smaller the remaining concurrent capacity, the larger the ratio, indicating greater queuing pressure on the instance and a longer expected waiting time for new requests. Therefore, the larger the ratio of queue length to remaining concurrent capacity, the greater the queue blocking penalty. Through this penalty, the scoring function can effectively perceive the real-time load differences among instances, guiding requests to lighter-loaded instances, achieving implicit load balancing, and avoiding sending new requests to instances that already have a large backlog of requests.

[0057] S434: Determine a timeliness penalty value based on the failure time of the running state of the service instance, wherein the longer the failure time of the running state, the greater the timeliness penalty value.

[0058] In the decentralized architecture of this invention, each proxy node relies on locally cached metadata to make scheduling decisions, and this metadata is asynchronously synchronized through eventually consistent distributed storage. This means that when the state of a service instance changes, the local caches of other proxy nodes may not have been updated in time, resulting in a certain information lag. To quantify the risk caused by this information lag, this step introduces a timeliness penalty value.

[0059] Specifically, the expiration time of the running state can be determined as follows: Each metadata record carries a version number or timestamp upon publication and is assigned a Time To Live (TTL) value. When the target proxy node reads its local cache, it checks the time elapsed since the metadata was published. If the time exceeds the preset TTL, the running state is considered expired. The longer the expiration time of the running state, i.e., the more outdated the information, the greater the timeliness penalty value. This mechanism enables scheduling decisions to automatically reduce the preference for an instance when the information is not fresh enough, thereby effectively avoiding the risk of scheduling requests to overloaded or faulty instances due to incorrect decisions based on outdated information, and improving the system's decision robustness in dynamic environments.

[0060] S435: Determine the score of the service instance based on the cost penalty value, the latency penalty value, the queue blocking penalty value, and the timeliness penalty value.

[0061] After calculating the penalty values ​​for the four dimensions mentioned above, the target proxy node summarizes them to determine the final score for each second candidate service instance. In an optional implementation, the score can be obtained by weighted summation of the penalty values, and a random perturbation term can be added to enhance oscillation resistance.

[0062] It should be noted that the calculation of each penalty value and the generation of the overall score are completed entirely locally on the target proxy node, without relying on the real-time participation of any external centralized components. This makes the scoring decision-making process extremely efficient, completing it in sub-millisecond time, thereby minimizing the impact on end-to-end request latency.

[0063] Through the aforementioned multi-dimensional and refined scoring mechanism, this invention transforms key factors affecting scheduling quality, such as resource cost, network latency, instance load, and information timeliness, into comparable and quantifiable penalty values, which are then efficiently aggregated locally into a comprehensive score. This design enables scheduling decisions to comprehensively balance cost and performance, real-time load and information reliability. It achieves high-quality optimal scheduling under normal conditions and adaptively adjusts based on timeliness penalty values ​​in abnormal scenarios such as metadata synchronization delays or drastic state fluctuations, ensuring the overall stability and service quality of the service chain.

[0064] In some embodiments of the invention, S43 further includes the following steps S436-S437: S436: Add a random perturbation value to the score of each second candidate service instance. The random perturbation value is used to make the target proxy node randomly select a second candidate service instance whose score difference is less than the preset threshold with equal probability when the difference between the scores of multiple second candidate service instances is less than the preset threshold.

[0065] In the actual operation of cloud-edge systems, the metadata cached locally by multiple agent nodes often exhibits high similarity because they are all synchronized from the same eventually consistent distributed metadata store. When a system state update event occurs (e.g., a new instance comes online and its metrics are significantly better than other instances), the metadata update message will propagate to all agent nodes within a relatively concentrated time window. If all agent nodes make decisions based on the exact same cached information and the exact same scoring function, then at the same moment, a large number of concurrent service chain requests are highly likely to be scheduled to the same optimal instance by all agent nodes simultaneously. This causes the instance to instantly handle a traffic surge exceeding its processing capacity, leading to a sharp decline in service quality or even instance crashes, and thus generating a new round of scheduling oscillations.

[0066] Therefore, in this embodiment of the invention, after calculating the penalty values ​​for each of the aforementioned dimensions and obtaining the initial comprehensive score for each second candidate service instance, the target proxy node does not directly make the final selection based on the initial score. Instead, it first adds a random perturbation value to each score. The mechanism of the random perturbation value is as follows: when the difference between the scores of multiple second candidate service instances is less than a preset threshold, the target proxy node randomly selects a second candidate service instance whose score difference is less than the preset threshold with equal probability.

[0067] Specifically, the preset threshold is an empirical parameter that can be flexibly configured according to actual business scenarios. When the initial scores of two or more candidate instances are very close (the difference between their scores is less than the preset threshold), these instances can be considered to have almost no difference in their overall performance in terms of cost, latency, load, and information timeliness. In this case, adding a random perturbation value of similar magnitude to the preset threshold will cause these instances with initially similar scores to exhibit a random order in the overall score ranking after the perturbation. Since each proxy node independently generates its own random perturbation value, the final ranking results of different proxy nodes for the same set of candidate instances will naturally discretize. In this way, the request traffic scheduled to these instances is effectively distributed, avoiding all requests concentrating on a single instance.

[0068] The random perturbation value can be a random number within a preset range that follows a uniform or normal distribution. Its generation range can be configured by operations personnel based on the system's tolerance for randomness. The perturbation value should not be too large, so as to overturn the bias of the scoring function itself (i.e., directing requests to obviously inferior instances), nor too small, so as not to break the tie in scoring. A reasonable perturbation value should ensure that when the scores of candidate instances differ significantly, the perturbation value is insufficient to change the order of priority; while when the scores of candidate instances are extremely close and interchangeable, the perturbation value can change their relative ranking with a certain probability, thereby achieving probabilistic traffic dispersion.

[0069] S437: Determine the target service instance based on the score after adding random perturbation values.

[0070] After adding a random perturbation value to the initial score of each second candidate service instance, the target proxy node obtains a new set of scores. Then, based on the scores with the added random perturbation value, the target proxy node determines the final target service instance according to a preset selection strategy.

[0071] Through the two-stage processing described above, this embodiment of the invention cleverly utilizes a randomization mechanism while ensuring the overall quality of scheduling decisions. This effectively suppresses traffic oscillations and instance overload risks caused by convergent distributed decisions at the cost of extremely low computational overhead and zero communication overhead. This design enables the decentralized scheduling system to maintain smooth and stable global traffic distribution even in an architecture where a large number of agent nodes make independent parallel decisions. It is a crucial technical means for this invention to achieve high scalability while ensuring system robustness.

[0072] S50: Send the pending request to the proxy node co-located with the target service instance.

[0073] After identifying the target service instance, the target proxy node forwards the request to be processed to the proxy node co-located with the target service instance. Since each service instance shares a proxy node, the proxy node adjacent to the target service instance becomes the new target proxy node upon receiving the request and repeats the above steps, continuing to forward the request hop-by-hop to subsequent nodes in the service chain until the request reaches the end of the service chain and is processed by all service nodes. This hop-by-hop scheduling mechanism ensures that each hop's forwarding decision is executed independently by the local proxy node, without the need for a centralized controller, thus avoiding decision failures caused by global state synchronization lag.

[0074] In some embodiments of the invention, prior to step S43, the following steps are specifically included: Obtain the elapsed time of the pending request on the service chain; If the elapsed time exceeds the end-to-end time budget of the pending request, the pending request is terminated at the current proxy node, and a timeout failure message is output. When the current proxy node performs a controlled rollback, if the number of consecutive rollbacks reaches a preset limit, the pending request is terminated at the current proxy node, and a rollback failure message is output.

[0075] Before sending the request to be processed to the proxy node co-located with the target service instance, to ensure that the request is processed within a reasonable time, the target proxy node must first obtain the accumulated time consumed by the request on the service chain before forwarding the request to the next service node. The accumulated time refers to the total time spent by the request in queuing, network transmission, and service execution throughout the entire chain, from the moment the request enters the first service node in the service chain until the current target proxy node is ready to execute the next-hop scheduling decision. The accumulated time is read by the target proxy node from the link context carried by the request and is continuously accumulated and updated as the request flows hop by hop in the service chain.

[0076] The target proxy node compares the elapsed time with the end-to-end time budget. If the elapsed time exceeds the end-to-end time budget, it indicates that even if subsequent service nodes can complete the processing with zero latency, the request cannot complete the entire service chain within the scheduled time, and continuing to forward the request is meaningless. In this case, the target proxy node proactively terminates the pending request at the current proxy node, stops further forwarding, and outputs a timeout failure message to the client or upstream caller. This mechanism effectively avoids expired requests from unnecessarily occupying the resources of subsequent service nodes, while allowing the client to receive failure feedback as early as possible, thus enabling timely retry or degradation measures.

[0077] Furthermore, when the target proxy node triggers the controlled fallback mechanism due to the inability to select a second candidate service instance or the lack of operational status, the system will try backup strategies sequentially according to a preset priority order. However, in extreme failure scenarios (such as large-scale network partitions causing service instances in multiple regions to become unreachable), even with multi-level fallback strategies, it may still be impossible to find an available target service instance. Allowing fallback strategies to recursively or repeatedly attempt indefinitely without restriction will not only consume a large amount of system resources but also cause requests to be suspended for a long time, ultimately ending in failure, severely impacting the overall throughput and response performance of the system.

[0078] To address this, this invention introduces a rollback limit mechanism. Specifically, when executing a controlled rollback, the target proxy node records the number of consecutive rollbacks. The consecutive rollback count refers to the cumulative number of times the same pending request triggers rollback strategies at various levels sequentially during a single scheduling decision process because the target service instance cannot be determined through conventional scoring and filtering. If the consecutive rollback count reaches the preset limit, the target proxy node terminates further rollback attempts, directly terminates the pending request locally, and outputs rollback failure information.

[0079] The preset upper limit is a configurable empirical threshold that operations and maintenance personnel can flexibly set based on the system's fault tolerance and business scenario requirements. When the target proxy node detects that the number of consecutive rollbacks has reached the preset upper limit during controlled rollback execution, it determines that the current service chain environment can no longer support the continued execution of the request, and trying more rollback strategies will be futile. At this point, the target proxy node actively terminates the pending request at the current proxy node and outputs rollback failure information. The rollback failure information may include the reason for failure, the sequence of rollback strategies attempted, the number of rollbacks, and relevant contextual indicators, providing detailed observation data for subsequent fault localization and system optimization.

[0080] Through the aforementioned two aspects of pre-verification and processing mechanisms, this embodiment of the invention establishes an effective protective barrier at the final stage of request forwarding. On the one hand, by checking for timeouts in latency budgets, it prevents requests destined to time out from continuing to consume system resources, thus achieving pre-emptive protection of end-to-end service quality in the service chain. On the other hand, by controlling the upper limit of the number of rollbacks, it prevents the resource exhaustion problem caused by the controlled rollback mechanism being triggered infinitely in extreme failure scenarios, ensuring the overall stability and resilience of the system. The combination of these two aspects enables the service mesh scheduling method of this invention to not only possess efficient normal scheduling capabilities in a decentralized architecture, but also robust self-healing and graceful degradation capabilities under abnormal boundary conditions.

[0081] The flowchart of the decentralized hop-by-hop scheduling method based on service mesh sidecar proxy is as follows: Figure 4As shown, the process starts from the "Start" node, first performs "Request Entry and Link Context Initialization", and then enters the "Next Hop Loop". In the loop, it first "Calculates the Remaining Budget and Constraints of the Next Hop", then judges whether "Check Local Cache Metadata" is available. If it is missing or expired, "Asynchronous Refresh / Use StalenessPenalty" is triggered; if it is available, "Feasible Region Screening (Eliminating Instances with Insufficient Capacity or Excessive Latency)" is performed. After screening, it is judged "Are there still feasible candidates?". If not, the "Controlled Fallback Mechanism (Same Region Priority / Last Successful Path)" is executed, and then it is judged "Are there feasible candidates after fallback?". If not, "Discard the request and output the failure reason"; if yes, "Scoring function calculation and optimal selection (including random perturbation ε)" is performed. After that, it is judged "i<n?" (whether there is a next hop). If yes, return to the "Next Hop Loop"; if no, "Asynchronous indicator reporting and cache update" is performed, and the process finally ends. The entire process implements decentralized hop-by-hop scheduling through mechanisms such as local cache, feasible region screening, controlled fallback and random perturbation, to ensure efficient and reliable transmission of requests on the service chain.

[0082] It can be seen that in the above solution, for complex service scenarios that require collaborative processing across multiple nodes, such as real-time fintech transactions and telehealth remote diagnosis, decentralized hop-by-hop scheduling can be performed using proxy nodes deployed adjacent to each service instance. Each proxy node makes a local decision on the next hop target based on the locally cached operating state information, and directly performs screening and forwarding according to the immediate receiving capacity and processing efficiency of candidate instances, without the participation of a centralized scheduler. This solution can effectively avoid scheduling decision failures caused by lag in global state synchronization, eliminate the risk of centralized bottlenecks and single points of failure, significantly reduce additional scheduling latency, prevent traffic oscillation and request accumulation in high-concurrency scenarios, and ensure the stable flow of critical service requests in cross-domain service links, so as to improve the response speed and overall availability of core service links under the cloud-edge continuum.

[0083] It should be understood that the sequence numbers of the steps in the above embodiments do not imply the order of execution, and the execution order of each process shall be determined by its function and inherent logic, and shall not constitute any limitation on the implementation process of the embodiments of the present invention.

[0084] In one embodiment, a service mesh scheduling apparatus is provided, and the service mesh scheduling apparatus is in one-to-one correspondence with the service mesh scheduling method in the above embodiments. The service mesh includes a plurality of service nodes, each service node includes a plurality of service instances, each service instance is co-located with a proxy node, and the service mesh scheduling apparatus provided in the embodiment of the present invention is applied to a target proxy node, and the target proxy node is co-located with the current service instance among the plurality of service instances. As Figure 5As shown, the service grid scheduling device includes an acquisition module 101, a first sending module 102, a determination module 103, a filtering module 104, and a second sending module 105. Detailed descriptions of each functional module are as follows: The acquisition module 101 is used to acquire service chain information of the request to be processed. The service chain information indicates the M service nodes that the request to be processed needs to pass through in sequence. The M service nodes include the target proxy node. The request to be processed is used to request the execution of the target scheduling task. The first sending module 102 is used to send the service chain information to the current service instance; The determining module 103 is used to determine the next service node of the target proxy node in the service chain information after receiving the information indicating that the current service instance has completed execution; The filtering module 104 is used to filter multiple service instances of the next service node according to the running status of the service instances to obtain target service instances. The running status is used to characterize the real-time acceptance capacity and processing efficiency of the service instance when processing the pending request. The second sending module 105 is used to send the request to be processed to the proxy node co-located with the target service instance.

[0085] In one embodiment, the filtering module 104 is specifically used for: Multiple service instances of the next service node are filtered to obtain multiple first candidate service instances, and the running status of each first candidate service instance; Based on the running status of each first candidate service instance and the remaining processing time of the service chain information, multiple second candidate service instances are selected from multiple first candidate service instances; Multiple second candidate service instances are scored, and the multiple second candidate service instances are filtered according to the score of each second candidate service instance to obtain the target service instance. The score is used to quantify the degree of fit between the second candidate service instance and the target scheduling task.

[0086] In one embodiment, the operating status includes the remaining concurrent capacity of the service instance, queue length, estimated network transmission time, estimated service execution time, and available resources; the filtering module 104 is specifically used for: Remove service instances from the plurality of first candidate service instances whose remaining concurrent capacity is less than or equal to a preset capacity threshold; Service instances whose estimated total processing time is greater than the remaining processing time are removed from a plurality of first candidate service instances. The estimated total processing time includes the estimated queue blocking time from the target proxy node to the service instance, the estimated network transmission time, and the estimated service execution time of the service instance. The estimated queue blocking time is calculated from the queue length of the service instance and the remaining concurrent capacity of the service instance. The remaining service instances are identified as multiple second candidate service instances.

[0087] In one embodiment, the filtering module 104 is specifically used for: For each of the multiple second candidate service instances, perform the following steps: A cost penalty value is determined based on the resource consumption cost of the region where the service instance is located, wherein the higher the resource consumption cost, the higher the cost penalty value. A latency penalty value is determined based on the sum of the estimated network transmission time from the target proxy node to the service instance and the estimated service execution time, wherein the larger the sum of the estimated network transmission time and the estimated service execution time, the larger the latency penalty value. The queue blocking penalty value is determined based on the ratio of the queue length of the service instance to the remaining concurrent capacity of the service instance, wherein the larger the ratio of the queue length to the remaining concurrent capacity, the larger the queue blocking penalty value. A timeliness penalty value is determined based on the failure time of the running state of the service instance, wherein the longer the failure time of the running state, the larger the timeliness penalty value. The score of the service instance is determined based on the cost penalty value, the latency penalty value, the queue blocking penalty value, and the timeliness penalty value.

[0088] In one embodiment, the filtering module 104 is specifically used for: A random perturbation value is added to the score of each second candidate service instance. The random perturbation value is used to make the target proxy node randomly select a second candidate service instance whose score difference is less than the preset threshold with equal probability when the difference between the scores of multiple second candidate service instances is less than the preset threshold. The target service instance is determined based on the score after adding random perturbation values.

[0089] In one embodiment, the filtering module 104 is specifically used for: If a second candidate service instance cannot be selected from multiple first candidate service instances, or if the running status is missing, a controlled rollback is executed. The controlled rollback is used to instruct the current service instance to execute the backup strategy in sequence according to a preset priority order. The controlled rollback includes: Select a service instance located within the region to which the current proxy node belongs as the target service instance; If there are no available service instances within the region to which the current proxy node belongs, then the service instance selected when executing the previous service node in the current service chain is selected as the target service instance.

[0090] In one embodiment, the device is further configured to: Obtain the elapsed time of the pending request on the service chain; If the elapsed time exceeds the end-to-end time budget of the pending request, the pending request is terminated at the current proxy node, and a timeout failure message is output. When the current proxy node performs a controlled rollback, if the number of consecutive rollbacks reaches a preset limit, the pending request is terminated at the current proxy node, and a rollback failure message is output.

[0091] This invention provides a service mesh scheduling device. First, it obtains service chain information of pending requests through proxy nodes co-located with the current service instance. After the current service instance completes execution, it filters based on the immediate processing capacity and efficiency of each service instance in the next service node to determine the target service instance. Then, it forwards the pending requests to the proxy nodes co-located with the target service instance, allowing the requests to be executed hop-by-hop along the service chain. This decentralized hop-by-hop scheduling scheme allows proxy nodes deployed next to each service instance to decide the next hop target locally based on their local cached running status. It directly filters and forwards requests based on the immediate processing capacity (e.g., remaining concurrent capacity, queue backlog) and processing efficiency (e.g., network transmission time, service execution time) of each candidate instance, without the need for a centralized controller. This effectively avoids scheduling decision failures caused by global state synchronization lag, eliminates centralized bottlenecks and single-point-of-failure risks, significantly reduces scheduling latency, prevents traffic oscillations and request accumulation in high-concurrency scenarios, and ensures stable flow of critical business requests in cross-domain service links, thereby improving the response speed and overall availability of core business links in the cloud-edge continuum.

[0092] Specific limitations regarding the service mesh scheduling device can be found in the limitations of the service mesh scheduling method described above, and will not be repeated here. Each module in the aforementioned service mesh scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.

[0093] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a service mesh scheduling method on the server side.

[0094] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a service mesh scheduling method on the client side.

[0095] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain service chain information for the pending request. The service chain information indicates the M service nodes that the pending request needs to pass through sequentially. The M service nodes include the target proxy node. The pending request is used to request the execution of a target scheduling task. Send the service chain information to the current service instance; After receiving the information indicating that the current service instance has completed execution, determine the next service node of the target proxy node in the service chain information; Multiple service instances of the next service node are filtered according to the running status of the service instances to obtain target service instances. The running status is used to characterize the real-time acceptance capacity and processing efficiency of the service instance when processing the pending request. The pending request is sent to the proxy node co-located with the target service instance.

[0096] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain service chain information for the pending request. The service chain information indicates the M service nodes that the pending request needs to pass through sequentially. The M service nodes include the target proxy node. The pending request is used to request the execution of a target scheduling task. Send the service chain information to the current service instance; After receiving the information indicating that the current service instance has completed execution, determine the next service node of the target proxy node in the service chain information; Multiple service instances of the next service node are filtered according to the running status of the service instances to obtain target service instances. The running status is used to characterize the real-time acceptance capacity and processing efficiency of the service instance when processing the pending request. The pending request is sent to the proxy node co-located with the target service instance.

[0097] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0098] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0100] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0101] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A service grid scheduling method, characterized in that, The service mesh includes multiple service nodes, each service node includes multiple service instances, and each service instance shares a proxy node. The method is applied to a target proxy node, which shares a proxy node with the current service instance among the multiple service instances, including: Obtain service chain information for the pending request. The service chain information indicates the M service nodes that the pending request needs to pass through sequentially. The M service nodes include the target proxy node. The pending request is used to request the execution of a target scheduling task. Send the service chain information to the current service instance; After receiving the information indicating that the current service instance has completed execution, determine the next service node of the target proxy node in the service chain information; Multiple service instances of the next service node are filtered according to the running status of the service instances to obtain target service instances. The running status is used to characterize the real-time acceptance capacity and processing efficiency of the service instance when processing the pending request. The pending request is sent to the proxy node co-located with the target service instance.

2. The service grid scheduling method as described in claim 1, characterized in that, The step of filtering multiple service instances of the next service node according to the running status of the service instances to obtain the target service instance includes: Multiple service instances of the next service node are filtered to obtain multiple first candidate service instances, and the running status of each first candidate service instance; Based on the running status of each first candidate service instance and the remaining processing time of the service chain information, multiple second candidate service instances are selected from multiple first candidate service instances; Multiple second candidate service instances are scored, and the multiple second candidate service instances are filtered according to the score of each second candidate service instance to obtain the target service instance. The score is used to quantify the degree of fit between the second candidate service instance and the target scheduling task.

3. The service grid scheduling method as described in claim 2, characterized in that, The running status includes the remaining concurrent capacity of the service instance, queue length, estimated network transmission time, estimated service execution time, and available resources; The step of selecting multiple second candidate service instances from multiple first candidate service instances based on the running status of each first candidate service instance and the remaining processing time of the service chain information includes: Remove service instances from the plurality of first candidate service instances whose remaining concurrent capacity is less than or equal to a preset capacity threshold; Service instances whose estimated total processing time is greater than the remaining processing time are removed from a plurality of first candidate service instances. The estimated total processing time includes the estimated queue blocking time from the target proxy node to the service instance, the estimated network transmission time, and the estimated service execution time of the service instance. The estimated queue blocking time is calculated from the queue length of the service instance and the remaining concurrent capacity of the service instance. The remaining service instances are identified as multiple second candidate service instances.

4. The service grid scheduling method as described in claim 3, characterized in that, The scoring of multiple second candidate service instances includes: For each of the multiple second candidate service instances, perform the following steps: A cost penalty value is determined based on the resource consumption cost of the region where the service instance is located, wherein the higher the resource consumption cost, the higher the cost penalty value. A latency penalty value is determined based on the sum of the estimated network transmission time from the target proxy node to the service instance and the estimated service execution time, wherein the larger the sum of the estimated network transmission time and the estimated service execution time, the larger the latency penalty value. The queue blocking penalty value is determined based on the ratio of the queue length of the service instance to the remaining concurrent capacity of the service instance, wherein the larger the ratio of the queue length to the remaining concurrent capacity, the larger the queue blocking penalty value. A timeliness penalty value is determined based on the failure time of the running state of the service instance, wherein the longer the failure time of the running state, the larger the timeliness penalty value. The score of the service instance is determined based on the cost penalty value, the latency penalty value, the queue blocking penalty value, and the timeliness penalty value.

5. The service grid scheduling method as described in claim 2, characterized in that, The step of filtering multiple second candidate service instances based on the score of each second candidate service instance to obtain the target service instance includes: A random perturbation value is added to the score of each second candidate service instance. The random perturbation value is used to make the target proxy node randomly select a second candidate service instance whose score difference is less than the preset threshold with equal probability when the difference between the scores of multiple second candidate service instances is less than the preset threshold. The target service instance is determined based on the score after adding random perturbation values.

6. The service grid scheduling method as described in claim 2, characterized in that, The step of selecting multiple second candidate service instances from multiple first candidate service instances based on the running status of each first candidate service instance and the remaining processing time of the service chain information includes: If a second candidate service instance cannot be selected from multiple first candidate service instances, or if the running status is missing, a controlled rollback is executed. The controlled rollback is used to instruct the current service instance to execute the backup strategy in sequence according to a preset priority order. The controlled rollback includes: Select a service instance located within the region to which the current proxy node belongs as the target service instance; If there are no available service instances within the region to which the current proxy node belongs, then the service instance selected when executing the previous service node in the current service chain is selected as the target service instance.

7. The service mesh scheduling method as described in claim 6, characterized in that, Before sending the request to be processed to the proxy node co-located with the target service instance, the method further includes: Obtain the elapsed time of the pending request on the service chain; If the elapsed time exceeds the end-to-end time budget of the pending request, the pending request is terminated at the current proxy node, and a timeout failure message is output. When the current proxy node performs a controlled rollback, if the number of consecutive rollbacks reaches a preset limit, the pending request is terminated at the current proxy node, and a rollback failure message is output.

8. A service mesh scheduling apparatus, wherein the service mesh includes multiple service nodes, each service node includes multiple service instances, each service instance shares a proxy node, the apparatus is applied to a target proxy node, the target proxy node is shared with the current service instance among the multiple service instances, characterized in that, include: The acquisition module is used to acquire service chain information of the request to be processed. The service chain information indicates the M service nodes that the request to be processed needs to pass through in sequence. The M service nodes include the target proxy node. The request to be processed is used to request the execution of the target scheduling task. The first sending module is used to send the service chain information to the current service instance; The determination module is used to determine the next service node of the target proxy node in the service chain information after receiving the information indicating that the current service instance has completed execution; The filtering module is used to filter multiple service instances of the next service node according to the running status of the service instances to obtain target service instances. The running status is used to characterize the real-time acceptance capacity and processing efficiency of the service instance when processing the pending request. The second sending module is used to send the request to be processed to the proxy node co-located with the target service instance.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the service grid scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the service grid scheduling method as described in any one of claims 1 to 7.

Images