A service overload handling method, apparatus, electronic device, and storage medium

By dynamically acquiring the task time and resource consumption of service nodes, the types of services to be discarded can be determined, thus resolving service overload issues in the CDN system, ensuring the stable execution of core services, optimizing resource utilization, and improving system performance.

CN123123341APending Publication Date: 2026-11-17TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510638235.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-16
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In CDN systems, service nodes may experience overload due to the number of tasks exceeding capacity, leading to performance degradation and increased response time. Existing fixed thresholds may not be able to adapt to changes in hardware performance, resulting in wasted or insufficient resources.

Method used

By obtaining the execution time of tasks already executed by service nodes, the waiting time is dynamically determined. Based on the total resource consumption and priority of the business type, the business types to be discarded are determined, and unexecuted tasks are discarded. The overload strategy is dynamically adjusted to restore the normal state.

Benefits of technology

Rapidly respond to sudden traffic surges, identify and lock down the types of business processes to be discarded during these surges, ensure the execution of core business processes, balance resource competition, avoid meaningless data acquisition, and improve system stability and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123341A_ABST
    Figure CN123123341A_ABST
Patent Text Reader

Abstract

This application relates to the field of computer technology, and more particularly to a service overload handling method, apparatus, electronic device, and storage medium for reducing the probability of service overload problems. Specifically, when a preset triggering condition is met, the execution time of each executed task on the service node is obtained; for at least one node task carried by the service node, the following steps are performed: according to the task execution order of the service node, based on the execution time of at least one previously executed task, the waiting time of a node task is obtained; if the service node is determined to be in an overload state based on the obtained at least one waiting time, a set of service types to be discarded is determined based on the overload policy information of each service type on the service node; and at least one unexecuted task associated with the set of service types to be discarded is discarded.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a service overload processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, in Content Delivery Network (CDN) systems and cloud computing environments, the scale of various business types is constantly expanding, and the tasks under these business types are becoming increasingly diversified. A CDN system contains multiple service nodes, each of which can be a server, a functional module, or an application. To address this, and in order for each service node in the CDN system to smoothly execute various tasks for different business types, the CDN system needs to be designed with appropriate service capacity for each service node.

[0003] However, taking a service node in a CDN system as an example, when the total number of tasks of various business types received by the service node becomes too large, the total number of tasks that the service node needs to execute can easily exceed its service capacity. This makes it impossible for the service node to process the various tasks it receives in a timely manner, resulting in service overload. When a large number of service nodes experience service overload, it will lead to a decrease in the performance of the CDN system, an increase in response time, and even service interruption. Therefore, how to handle service overload has become crucial to ensuring the stability of the CDN system and the user experience.

[0004] In related technologies, service overload can be prevented by setting a threshold for the number of tasks per second (QPS) or the length threshold of the task queue for each service node. Setting a quantity threshold can limit the number of tasks that a service node needs to execute within a fixed period of time, while setting a length threshold can limit the total number of tasks that a service node needs to execute by limiting the number of tasks in the task queue.

[0005] However, the aforementioned quantity and length thresholds are often fixed thresholds determined by combining the historical service capacity of the corresponding service nodes. The service capacity of a service node may change as the hardware performance of the CDN system improves or decreases, which may cause the number of tasks that lead to service overload to also change. Therefore, when using fixed thresholds to prevent service overload, it is difficult to adapt to changes in the hardware performance of the CDN system. When the fixed thresholds do not match the hardware performance of the CDN system, the service nodes are prone to resource waste or shortage.

[0006] For example, when hardware performance improves, if the fixed threshold of a service node is lower than its service capacity, the tasks that the service node can execute will be rejected. This will cause the object's tasks to not be responded to in a timely manner, resulting in a response delay problem. This not only reduces the user experience of the object, but also causes business losses.

[0007] For example, when hardware performance degrades, if the fixed threshold of a service node exceeds its service capacity, the service node will need to perform an excessive number of tasks, which will severely exacerbate the service overload problem.

[0008] In summary, reducing the probability of service overload issues is an urgent problem to be solved. Summary of the Invention

[0009] This application provides a service overload handling method, apparatus, electronic device, and storage medium to reduce the probability of service overload problems.

[0010] This application provides a service overload handling method, the method comprising:

[0011] When the preset triggering conditions are met, obtain the execution time of each executed task on the service node;

[0012] For at least one node task carried by the service node, the following steps are performed: according to the task execution order of the service node, based on the execution time of at least one previously executed task, the waiting time of the node task is obtained; wherein, the node task is either an executed task or an unexecuted task.

[0013] When a service node is determined to be in an overload state based on at least one obtained waiting time, a set of service types to be discarded is determined based on the overload policy information of each service type on the service node; wherein, the overload policy information includes at least one of the following: total resource consumption and service priority; the total resource consumption is the sum of the resource consumption of at least one executed task under the corresponding service type;

[0014] At least one unexecuted task associated with the set of service types to be discarded is discarded.

[0015] This application provides a service overload handling device, the device comprising:

[0016] The acquisition unit is used to acquire the execution time of each executed task on the service node when the preset triggering conditions are met.

[0017] An execution unit is configured to perform the following for at least one node task carried by the service node: according to the task execution order of the service node, and based on the execution time of at least one previously executed task, obtain the waiting time of the node task; wherein, the node task is either an executed task or an unexecuted task.

[0018] The determining unit is configured to determine a set of service types to be discarded based on the overload policy information of each service type on the service node when the service node is determined to be in an overload state based on at least one obtained waiting time; wherein the overload policy information includes at least one of the following: total resource consumption and service priority; the total resource consumption is the sum of the resource consumption of at least one executed task under the corresponding service type;

[0019] The discarding unit is used to discard at least one unexecuted task associated with the set of service types to be discarded.

[0020] Optionally, the overload strategy information includes: total resource consumption and service priority; the determining unit is specifically used for:

[0021] For each business type, if the total resource consumption of a business type exceeds the corresponding resource consumption limit for that business type, that business type will be designated as a business type to be discarded and added to the set of business types to be discarded.

[0022] When the resource consumption of each service type on the service node is lower than the corresponding resource consumption limit, the service type whose service priority does not exceed the preset overload level is designated as the service type to be discarded and added to the set of service types to be discarded.

[0023] Optionally, the executed tasks carried by the service node are located in the executed queue, and the resource consumption limit is the percentage of available resources; the determining unit is specifically used for:

[0024] Determine the ratio of the total resource consumption of a certain business type to the sum of the total resource consumption of each of the business types; wherein, the total resource consumption is the sum of the resource consumption of each reference task under the corresponding business type; each reference task is an executed task whose execution order is within a preset range.

[0025] When the ratio exceeds the proportion of available resources corresponding to the business type, the business type will be designated as a business type to be discarded.

[0026] Optionally, the total resource consumption for each service type is dynamically updated as the number of executed tasks carried by the service node changes; the discard unit is specifically used for:

[0027] When there are discardable service types whose total resource consumption exceeds the corresponding resource consumption limit, for each discardable service type, based on a preset overload ratio, unexecuted tasks under a discardable service type are discarded until the service node returns to normal, or until the latest total resource consumption corresponding to a discardable service type is lower than the corresponding resource consumption limit.

[0028] When the total resource consumption of each type of service to be discarded is lower than the corresponding resource consumption limit, and the service node is still overloaded, based on the order of service priority of each type of service to be discarded from low to high and the preset overload ratio, the unexecuted tasks under the corresponding type of service to be discarded are discarded in sequence until the service node returns to normal.

[0029] Optionally, during the process of discarding unexecuted tasks under a certain type of service to be discarded based on a preset overload ratio, the discarding unit is specifically used for:

[0030] The overload ratio is increased every first preset time interval based on a first preset step size.

[0031] Optionally, when the set of unexecuted tasks associated with the service type to be discarded is empty, and the service node is still in an overloaded state, the discarding unit is further configured to:

[0032] The overload level is gradually increased until the service node returns to normal; wherein, after each increase in the overload level, the following process is performed:

[0033] Based on the improved overload level, the set of service types to be discarded is redefined from those service types that were never added to the set of service types to be discarded.

[0034] Discard at least one unexecuted task associated with the latest set of pending discard business types.

[0035] Optionally, the device further includes an adjustment unit, the adjustment unit being used for:

[0036] After the service node returns to normal, every second preset time interval, the newly determined overload ratio is reduced based on the second preset step size, and at least one unexecuted task under the corresponding service type is discarded according to the reduced overload ratio.

[0037] Optionally, after the overload ratio is reduced to a preset ratio, the adjustment unit is further configured to:

[0038] The overload level is gradually reduced until it reaches the lowest service priority; after each reduction in the overload level, the following process is performed:

[0039] Based on the reduced overload level, the set of service types to be discarded is re-determined from each service type;

[0040] Discard at least one unexecuted task associated with the latest set of pending discard business types.

[0041] Optionally, each business type contains multiple request types, and each request type corresponds to a request priority; then the discarding unit discards unexecuted tasks under a business type to be discarded in the following manner:

[0042] For a service type to be discarded, based on the corresponding request priority from low to high and the preset overload ratio, the unexecuted tasks under the corresponding request type are discarded in sequence until the service node returns to normal, or until the unexecuted tasks under the service type to be discarded are empty.

[0043] Optionally, during the process of discarding unexecuted tasks under a request type based on a preset overload ratio, the discarding unit is further configured to:

[0044] Every third preset time interval, the overload ratio is increased based on the third preset step size.

[0045] Optionally, based on a preset overload ratio, unexecuted tasks under a request type are discarded, and the discarding unit is specifically used for:

[0046] For each unexecuted task under the aforementioned corresponding request type, perform the following operations:

[0047] Generate a random number for an unexecuted task; the random number belongs to a preset random number range;

[0048] When the proportion of the random number to the upper limit of the random number range is lower than the latest determined overload proportion corresponding to the request type, the unexecuted task is discarded.

[0049] Optionally, the determining unit is specifically used for:

[0050] The current processing delay of the service node is determined based on the at least one of the waiting times.

[0051] When the current processing delay exceeds a preset overload threshold, the service node is determined to be in an overload state.

[0052] Optionally, the unexecuted tasks carried by the service node are located in an unexecuted queue; the determining unit determines the current processing delay of the service node based on the at least one waiting time, including at least one of the following methods:

[0053] The average waiting time of multiple executed tasks is used as the current processing delay;

[0054] The waiting time of the unexecuted tasks used for detection is taken as the current processing delay; wherein the unexecuted tasks used for detection are unexecuted tasks added to the tail of the unexecuted queue.

[0055] Optionally, before determining that the service node is in an overloaded state, the determining unit is further configured to:

[0056] Based on the current processing latency and historical latency data, the updated current processing latency is predicted.

[0057] The step of determining that the service node is in an overload state when the current processing latency exceeds a preset overload threshold includes:

[0058] When the updated current processing latency exceeds a preset overload threshold, the service node is determined to be in an overload state.

[0059] Optionally, the executed tasks carried by the service node are located in the executed queue; the determining unit is specifically used to determine and update the total resource consumption of each business type in the following manner:

[0060] Using a sliding window with a fourth preset step size, perform the following operations dynamically:

[0061] Using the sliding window, a corresponding number of executed tasks are selected from the executed queue as reference tasks;

[0062] For each business type, the sum of the resource consumption of all reference tasks belonging to a business type in the reference tasks is taken as the current total resource consumption of the business type.

[0063] An electronic device provided in this application includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of any of the above-described service overload processing methods.

[0064] This application provides a computer-readable storage medium including a computer program. When the computer program is run on an electronic device, the computer program is used to cause the electronic device to perform the steps of any of the above-described service overload processing methods.

[0065] This application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. When the processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of any of the above-described service overload processing methods.

[0066] The beneficial effects of this application are as follows:

[0067] This application provides a service overload handling method, apparatus, electronic device, and storage medium. Specifically, when a preset trigger condition is met, this application first obtains the execution time of each executed task on the service node. By setting trigger conditions, the service overload handling method of this application is dynamically executed each time the preset trigger condition is triggered, which can avoid meaningless data noise and reduce the additional performance loss caused by high-frequency data acquisition.

[0068] Furthermore, based on the execution time of at least one previously executed task before a node task, the waiting time of a node task is obtained. Then, based on the obtained at least one waiting time, it is analyzed whether the service node is in an overload state. This enables the application to quickly respond to sudden traffic and latency fluctuations, and to promptly grasp the operating status of the service node, so as to perform the following service overload processing method on the service node in an overload state.

[0069] Finally, this application determines the set of business types to be discarded based on the business priority and total resource consumption of each business type on the service node, and discards at least one unexecuted task associated with the set of business types to be discarded. By introducing resource consumption, it can identify service node overload problems caused by a single or a few business bursts and pinpoint the business types to be discarded due to burst traffic. At the same time, since business priority is also introduced, this application can also ensure that tasks under higher priority, i.e., more core business types, can be executed smoothly, balancing resource competition among different business needs while maintaining the efficient operation of core businesses.

[0070] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0071] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0072] Figure 1 This is a schematic diagram of an application scenario in an embodiment of this application;

[0073] Figure 2 This is a schematic diagram of a service overload handling method provided in an embodiment of this application;

[0074] Figure 3 A flowchart illustrating the implementation of a service overload handling method provided in this application embodiment;

[0075] Figure 4 A schematic diagram illustrating the waiting time provided in an embodiment of this application;

[0076] Figure 5A A schematic diagram illustrating the determination of the current processing delay, provided as an embodiment of this application;

[0077] Figure 5B Another schematic diagram for determining the current processing delay provided in an embodiment of this application;

[0078] Figure 6A A schematic diagram illustrating the total resource consumption provided in an embodiment of this application;

[0079] Figure 6B A schematic diagram illustrating another total resource consumption as provided in an embodiment of this application;

[0080] Figure 7 A schematic diagram illustrating the total resource consumption provided in an embodiment of this application;

[0081] Figure 8 This is a schematic diagram illustrating the discarding of unexecuted tasks, provided as an embodiment of this application.

[0082] Figure 9 A schematic diagram illustrating another method for discarding unexecuted tasks, provided as an embodiment of this application;

[0083] Figure 10 This is a schematic diagram of an overload strategy information provided in an embodiment of this application;

[0084] Figure 11 A schematic diagram illustrating an implementation of a service overload handling method provided in this application embodiment;

[0085] Figure 12 This is a schematic diagram of the structure of a service overload processing device according to an embodiment of this application;

[0086] Figure 13 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application;

[0087] Figure 14This is a schematic diagram of the hardware structure of another electronic device using an embodiment of this application. Detailed Implementation

[0088] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some embodiments of the technical solutions of this application, and not all embodiments. Based on the embodiments recorded in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the technical solutions of this application.

[0089] The following describes some of the concepts involved in the embodiments of this application.

[0090] 1. Service Node: A service node refers to a device that provides services to an object and undertakes key functions such as data storage, computation, transmission, and security. In this application, it often refers to a device that provides network services or resources, receiving requests from terminal nodes via the network and providing corresponding services or resources. Upon receiving a request from a terminal node, the service node places the unexecuted task generated based on that request into its unexecuted queue. When an unexecuted task in the unexecuted queue is executed and becomes an executed task, it leaves the unexecuted queue and enters the executed queue of the service node. In this application, "node task" refers to both the unexecuted and executed tasks carried by the service node.

[0091] 2. Preset Triggering Conditions: These refer to the judgment rules predefined in this application for preventing service node overload. Each time a service node meets the preset triggering conditions, the subsequent service overload handling method will be automatically executed, transforming the service node overload risk into a controllable technical action, thus avoiding service node overload without manual intervention.

[0092] 3. Waiting Time: Also known as waiting latency or processing latency, this refers to the time interval between when an unexecuted task enters the unexecuted queue and when it actually begins to be executed by the service node. This metric is crucial for evaluating service node performance, response speed, and resource allocation efficiency. The current processing latency of a service node, determined based on at least one waiting time, reflects the current load on the service node, as well as its scheduling capabilities and optimization potential when handling high-concurrency requests.

[0093] 4. Waiting Threshold: This refers to the pre-set upper limit of the current waiting time for a service node. When the actual waiting time exceeds this threshold, the overload control step in the service overload handling method provided in this application embodiment will be automatically triggered. The waiting threshold can be set and updated according to the promised waiting delay for the object, or different waiting thresholds can be dynamically set for different time periods based on historical traffic patterns, or the waiting threshold of each service node can be dynamically set according to the hardware performance of each service node, and so on. By setting the waiting threshold to determine the operating status of the service node and executing the corresponding overload control steps, the service node can be protected from the impact of excessive load, ensuring its stability and reliability, and maintaining service quality when necessary by adjusting resource allocation or limiting requests.

[0094] 5. Business Type and Request Type: Business type, also known as namespace, includes, but is not limited to, the business type to which a task belongs: finance, live streaming, and video-on-demand. Business priorities are assigned to each business type based on its attributes (such as importance, value share, and urgency). Request type refers to the type of operation performed on the task, including but not limited to: uploading, downloading, calculation, payment, and answering questions. Similarly, for multiple request types under the same business type, request priorities can be assigned based on the attributes (such as importance, value share, and urgency) of each request type. In this embodiment, by cross-statistically calculating the waiting time based on the task's business type and request type, and selectively discarding unexecuted tasks based on both, the stability of tasks under core business types and core request types is ensured.

[0095] 6. Total Resource Consumption: This refers to the resources consumed by some or all of the executed tasks under a given business type. Total resource consumption comprehensively reflects the amount of resources required to complete the business, providing a key basis for assessing service node load, optimizing resource allocation, and controlling costs. Resource usage for different business types can be quantified through various dimensions. For example, in the time dimension, the execution time of each task reflects its resource consumption; the sum of the execution times of some or all executed tasks under a given business type is taken as the total resource consumption for that business type. Similarly, in the computing resource dimension, the resource consumption of each task can be reflected by at least one of the Central Processing Unit (CPU) consumption and memory usage; the sum of the computing resources of some or all executed tasks under a given business type is taken as the total resource consumption for that business type, and so on. By monitoring total resource consumption, high-load business types can be identified, expansion strategies and service overload strategies can be rationally planned, and efficient utilization of service nodes can be ensured, thereby improving overall service efficiency and economic benefits.

[0096] 7. Overload State: This refers to an abnormal operating state in which a service node is unable to respond normally to object requests due to a surge in instantaneous request volume exceeding its capacity or the exhaustion of critical resources (such as CPU, memory, bandwidth, and connection count). Specifically, this manifests as a sharp increase in request response latency, a spike in error rate, or a continuous backlog of tasks in the task queue (e.g., the number of unprocessed or unexecuted tasks exceeding a threshold). In this application, this overload state triggers a preset service overload handling method to prevent service node crashes and ensure global system stability, while also preventing potential cascading effects (such as a sudden increase in pressure on adjacent service nodes or a precipitous drop in user experience).

[0097] 8. Overload Level: When a service node is overloaded, it determines which low-priority business types' tasks will be subject to resource restrictions or dropped, ensuring that high-priority business types receive sufficient resources to guarantee their normal operation. This application can also dynamically adjust the types of business requiring resource restrictions by raising or lowering the overload level based on the service node's operating status. By setting overload levels, the performance and stability of core businesses are maintained even under service overload conditions, while optimizing overall resource allocation and improving the ability and efficiency to handle sudden traffic surges.

[0098] 9. Overload Ratio: When a service node is overloaded, this ratio determines the proportion of tasks of the same request type that are subject to resource constraints or dropped, restoring the service node to a normal state while minimizing task dropouts. This application can also dynamically adjust the number of tasks requiring resource constraints by increasing or decreasing the overload ratio based on the service node's operating status. By setting the overload ratio, the operating status of the service node and the execution of tasks are balanced under service overload conditions.

[0099] The design concept of the embodiments of this application is briefly introduced below:

[0100] Currently, in CDN systems and cloud computing environments, the scale of various business types is constantly expanding, and the tasks under these business types are also becoming increasingly diversified. When the total number of tasks received by service nodes from various business types becomes too large, the number of tasks that service nodes need to execute can easily exceed the design capacity of each service node in the CDN system. This makes the CDN system unable to effectively handle the received tasks or resource demands, resulting in service overload. Consequently, this leads to decreased CDN system performance, increased response time, and even service interruptions. Therefore, how to handle service overload has become crucial to ensuring the stability of the CDN system and the user experience.

[0101] In related technologies, service overload can be prevented by setting a fixed threshold for the number of tasks per second (QPS) or the task queue length for each service node. However, as the hardware performance of the CDN system improves or decreases, the designed capacity of each service node may change accordingly. When the fixed thresholds, such as the quantity threshold or length threshold, do not match the hardware performance of the CDN system, service nodes are prone to resource waste or shortage. Furthermore, since static thresholds are often determined based on historical design capacity, when sudden traffic spikes or latency fluctuations occur, the CDN system cannot quickly adjust the aforementioned fixed thresholds according to the current number of tasks or latency duration, resulting in response delays.

[0102] Furthermore, related technologies can adjust the number of tasks in the task queue by monitoring the overall waiting time of the task queue, and prevent service overload by dropping some tasks in the queue. However, this approach may drop core business tasks, leading to increased latency for core business operations.

[0103] In view of this, embodiments of this application provide a service overload processing method, apparatus, electronic device, and storage medium. Specifically, when a preset trigger condition is met, this application first obtains the execution time of each executed task on the service node. By setting trigger conditions and dynamically executing the service overload processing method of this application, meaningless data noise can be avoided, and additional performance loss caused by high-frequency data acquisition can be reduced.

[0104] Furthermore, based on the execution time of at least one previously executed task before a node task, the waiting time of a node task is obtained. Then, based on the obtained at least one waiting time, it is analyzed whether the service node is in an overload state. This enables the application to quickly respond to sudden traffic and latency fluctuations, and to promptly grasp the operating status of the service node, so as to perform the following service overload processing method on the service node in an overload state.

[0105] Finally, this application determines the set of business types to be discarded based on the business priority and total resource consumption of each business type on the service node, and discards at least one unexecuted task associated with the set of business types to be discarded. By introducing resource consumption, it can identify service node overload problems caused by a single or a few business bursts and pinpoint the business types to be discarded due to burst traffic. At the same time, since business priority is also introduced, this application can also ensure that tasks under higher priority, i.e., more core business types, can be executed smoothly, balancing resource competition among different business needs while maintaining the efficient operation of core businesses.

[0106] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0107] like Figure 1 The diagram shown is an application scenario illustration of an embodiment of this application. The application scenario diagram includes a terminal device 110 and a server 120.

[0108] In this embodiment, the terminal device 110 includes, but is not limited to, mobile phones, tablets, laptops, desktop computers, e-book readers, smart voice interaction devices, smart home appliances, and in-vehicle terminals. The terminal device may have a client installed related to business execution. This client can be software (e.g., a browser, service control software), or a webpage, mini-program, etc. The server 120 is the backend server corresponding to the software, webpage, mini-program, etc., or a server specifically used for service control; this application does not impose specific limitations. The server 120 can be an independent physical server, a server cluster composed of multiple physical servers, or a CDN system. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0109] It should be noted that the resource recommendation method in each embodiment of this application can be executed by an electronic device, which can be a terminal device 110 or a server 120. That is, the method can be executed by the terminal device 110 or the server 120 alone, or by the terminal device 110 and the server 120 together.

[0110] For example, when executed by terminal device 110 and server 120, the service node in this application can be deployed on server 120. The object submits a large number of unexecuted tasks through terminal device 110. After receiving the node tasks, terminal device 110 sends them to server 120. When the preset triggering conditions are met, server 120 obtains the execution time of each executed task on the service node; and based on the execution time of at least one executed task before a node task, obtains the waiting time of a node task; then, when server 120 determines that the service node is in an overload state based on the obtained at least one waiting time, it determines the set of business types to be discarded based on the overload policy information of each business type on the service node; finally, server 120 discards at least one unexecuted task associated with the set of business types to be discarded, executes the unexecuted tasks under the core business type, and feeds back the execution results to terminal device 110, so that the object can view the execution results on terminal device 110.

[0111] In one alternative implementation, the terminal device 110 and the server 120 can communicate via a communication network.

[0112] In one alternative implementation, the communication network is a wired network or a wireless network.

[0113] It should be noted that, Figure 1 The examples shown are merely illustrative; in reality, the number of terminal devices and servers is unlimited and is not specifically limited in the embodiments of this application.

[0114] In this embodiment of the application, when there are multiple servers, the multiple servers can form a blockchain, and the servers are nodes on the blockchain; as disclosed in the service overload processing method of this embodiment, the relevant data of service overload processing can be stored on the blockchain, such as service nodes, execution time, overload strategy information, etc.

[0115] Furthermore, the embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. Specifically, they can be applied to fields related to service overload handling within these scenarios. Below are a few specific application extensions:

[0116] For example, in a CDN system, when handling a large number of concurrent requests or when the hardware performance of a service node degrades, the resources of the service node (such as computing power and memory capacity) may be quickly exhausted, causing the service node to be unable to process newly arriving tasks in a timely manner, resulting in service overload. In this case, for each service node in the CDN system, the service overload handling method of this application can be used to promptly determine whether each service node is in an overloaded state, and discard unexecuted tasks of service nodes in an overloaded state to ensure the stable execution of tasks under core business types. In a cloud computing environment, resource sharing is achieved through virtualization technology, but this virtualization layer itself also introduces certain performance losses. For example, CPU scheduling, memory management, and input / output (I / O) operations may all be affected by the virtualization layer, thereby reducing the actual available resources. In a public cloud environment, multiple tenants share the same physical resources. A high load on one tenant may cause a "neighbor noise" effect on other tenants, affecting their service quality. When faced with a sudden increase in traffic, the automatic scaling mechanism of the cloud computing environment may have a certain delay. During this delay period, the cloud computing environment may have already experienced pressure exceeding its carrying capacity. At this time, for each service node in the cloud computing environment, the service overload handling method of this application can be used to determine in a timely manner whether each service node is in an overload state, and discard the unexecuted tasks of the service nodes in the overload state, so as to ensure the stable execution of tasks under the core business type.

[0117] For example, in an Application Programming Interface (API) gateway, the service overload handling method of this application can be used to promptly determine whether each service node is overloaded when faced with a surge of traffic (such as e-commerce promotions or flash sales) and the API gateway may crash due to insufficient processing capacity of backend service nodes. Unexecuted tasks of overloaded service nodes can be discarded to ensure the stable execution of tasks under core business types. In the Internet of Things (IoT), large-scale intelligent devices (such as industrial sensors and smart meters) simultaneously report data within fixed time windows (such as hourly meter readings). With tens of thousands of devices sending data to the central server simultaneously, exceeding the server's throughput limit, the central server may experience data loss and reporting delays, affecting real-time monitoring (such as missed factory equipment status reports). In this case, for data of different sources and types on the central server, the service overload handling method of this application can be used to discard data according to its source or data type after determining that the central server is overloaded, ensuring the accurate transmission and storage of core data.

[0118] For example, in real-time communication systems, video conferencing platforms may experience sudden access issues during unexpected events (such as global online summits). Because audio and video encoding / decoding and streaming media forwarding consume significant computing and bandwidth resources, this can lead to problems like video stuttering and dropped connections. In such cases, the service overload handling method described in this application can be used to handle the overload of some service nodes. Once it's determined that some service nodes are overloaded, data can be discarded according to its importance. For example, keyframes in the video frame can be retained while other frames are discarded, ensuring the smooth operation of the video conference. In financial trading systems, during peak opening / closing periods, stock exchanges face a surge in order requests. High-frequency trading algorithms submit massive amounts of orders to the financial trading system within the same millisecond (millions of trading instructions per second), causing overload on the backend service nodes and delaying order execution (e.g., preventing timely loss mitigation during stock market crashes). At this point, for the service nodes of the backend of the financial transaction system, the service overload handling method of this application can be used to discard some requests according to the importance of the request type after determining that some service nodes are in an overloaded state. For example, non-core requests such as uploading, downloading, and answering questions can be discarded, while core requests for financial transactions such as payment, selling, and buying can be retained to ensure the smooth execution of orders at the opening / closing of the market.

[0119] It should be noted that the above applications are merely simple examples. Other application scenarios are also applicable to the embodiments of this application, and will not be elaborated upon here. By introducing the service overload handling method of this application, it can not only be widely applied in multiple fields such as cloud technology, artificial intelligence, intelligent transportation, and assisted driving, but also achieve rapid task response in each specific scenario. This method dynamically determines whether a service node is in an overloaded state, and then discards unexecuted tasks carried by overloaded service nodes, ensuring that tasks under core business types can be executed smoothly. While solving the service overload problem, it also ensures the stability of the object's core business, bringing significant progress to the business request response, revenue improvement, and optimization of other key business indicators in various industries and fields.

[0120] It should be emphasized that the specific embodiments of this application involve data related to the handling of service overload by the object, such as the service nodes, execution time, and overload strategy information listed above. When the above embodiments of this application are applied to specific products or technologies, permission or consent from the object is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0121] The following describes the service overload handling method provided by the exemplary embodiments of this application in conjunction with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way in this respect.

[0122] It should be noted that the service overload handling method provided in this application embodiment can be applied to architectures such as Internet of Things (IoT), CDN, distributed database (Redis Cluster), and hybrid cloud, and this application does not specifically limit it.

[0123] The service nodes in this application embodiment can be different physical devices in different application scenarios, including but not limited to: servers, low-orbit satellites, routers, vehicles, etc., which will not be described in detail here.

[0124] Optional, such as Figure 2 As shown, the service overload handling method in this application can be executed by three modules:

[0125] Data acquisition module: responsible for collecting information such as entry time, execution time, business type and request type of unexecuted tasks.

[0126] Specifically, the various types of information collected by the data acquisition module are used in subsequent steps to determine whether the service node is overloaded, and which unexecuted tasks should be discarded if the service node is overloaded.

[0127] Delay detection module: Dynamically calculates the current processing delay of the service node and compares it with a preset overload threshold.

[0128] Specifically, the latency detection module determines the current processing latency of the service node based on the execution time collected by the data acquisition module. After comparing the current processing latency with a preset overload threshold, if the current processing latency exceeds the preset overload threshold, it can be determined that the service node is in an overload state and sent as the latency detection result to the decision execution module.

[0129] Decision execution module: Dynamically adjusts the service overload handling method based on latency detection results and priority strategies.

[0130] Specifically, after receiving the latency detection results and determining that the service node is in an overloaded state, the decision execution module determines the total resource consumption of each business type based on the execution time collected by the data acquisition module, and determines the priority strategy based on the total resource consumption and business priority. Then, the service overload handling method is dynamically adjusted according to the priority strategy.

[0131] It should be noted that, Figure 2The architecture and modules listed are merely examples. Any architecture and modules that can be used to execute the service overload handling method can be used in the service overload handling method of this application, and this application does not make any specific limitations on them.

[0132] See Figure 3 The diagram shown is an implementation flowchart of a service overload handling method provided in this application embodiment. Taking the server carrying the service node as the execution subject as an example, the specific implementation flow of this method is as follows: S31 to S34:

[0133] S31: When the preset triggering conditions are met, obtain the execution time of each executed task on the service node.

[0134] When a large number of service nodes are simultaneously overloaded, the platform or system containing these service nodes may experience performance degradation, increased response time, or even service interruption. Therefore, to avoid simultaneous overload of a large number of service nodes or severe overload of service nodes, this application sets preset trigger conditions. Each time these preset trigger conditions are met, subsequent service overload handling methods are automatically executed, proactively preventing service overload issues.

[0135] In order to accurately locate and handle service overload issues, the preset trigger conditions have been diversified to meet the needs of different scenarios, taking into account factors such as hardware performance, object requirements, and execution costs.

[0136] In one optional implementation, the preset triggering condition set in S31 includes at least one of the following:

[0137] Method 1: The number of node tasks executed by the service node reaches the preset number.

[0138] Specifically, this application takes into account that as the number of node tasks executed by the service node increases, the risk of the service node being overloaded also increases. Therefore, in method 1, a preset number is set for the service node. When the number of node tasks executed by the service node reaches the preset number, the subsequent steps are executed.

[0139] The preset quantity can be set based on historical data, historical design capacity, current design capacity, and the business needs of the target. For example, if the historical and current design capacities of a service node are both small, the preset quantity can be set to a smaller value such as 9,000 or 10,000. When the business needs of the target are more urgent, the preset quantity can be set to a larger value such as 15,000 or 16,000. For a specific service node, assuming the preset quantity is set to 10,000, when the number of node tasks executed by that service node reaches 10,000, subsequent steps can be performed on that service node.

[0140] Method 2: Reach the preset time interval.

[0141] Specifically, this application takes into account that high-frequency monitoring may introduce additional performance loss. Therefore, it periodically performs subsequent steps on the service node by setting a preset time interval.

[0142] The preset time interval can be set based on historical data, the business needs of the object, load volatility, and the execution time of tasks on each node. For example, if an object will have a sudden surge in tasks to execute within a certain time period, the service node may not be able to handle the sudden traffic. Therefore, the preset time interval for that time period can be adjusted to a smaller value, such as 2 seconds or 3 seconds. If the load volatility of the service node is low and the service node has had few overloads in the historical time period, the preset time interval for the service node can be adjusted to a larger value, such as 10 seconds or 11 seconds. If the execution time of most node tasks under the service node is short, the preset time interval can be adjusted to a smaller value, such as 2 seconds or 3 seconds, which helps to detect the overload status of the service node in a timely manner.

[0143] Method 3: The resource consumption of the service node reaches the preset resource threshold.

[0144] Specifically, this application considers that when there are too many service nodes, performing service overload processing on each service node may increase the pressure on the system or platform where the service node is located. Therefore, this application sets a preset resource threshold to accurately locate service nodes with service overload risk and reduce monitoring costs.

[0145] Preset resource thresholds can be set based on historical data, the business needs of the object, and hardware performance. For example, if the object will have a sudden surge in a large number of tasks to execute within a certain period of time, the resource consumption of the service node may increase sharply. Therefore, the preset resource threshold can be adjusted to a lower value such as 60% or 65%. If the load fluctuation of the service node is low and the number of times the service node has been overloaded in a historical period is low, the preset resource threshold can be adjusted to a higher value such as 80% or 85%.

[0146] It should be noted that the above-mentioned preset trigger condition setting method is just a simple example. Other setting methods are also applicable to the embodiments of this application, and will not be repeated here.

[0147] After each service node meets the preset triggering conditions, the subsequent overload judgment steps can be performed. In order to determine whether the service node is overloaded, this application first needs to obtain the execution time of each executed task on the service node in real time, so as to determine the waiting time based on the execution time and determine whether the service node is in an overloaded state based on the waiting time.

[0148] Specifically, the unexecuted tasks carried by the service node are located in the unexecuted queue. For each unexecuted task carried by the service node, its entry time, business type, etc. are recorded in real time when it enters the unexecuted queue. When the unexecuted task is completed and becomes an executed task, the execution time of the executed task is recorded. The execution time refers to the time period from when the service node starts processing an unexecuted task to when it completes the unexecuted task and outputs the result. It is usually quantified in milliseconds (ms) or seconds (s).

[0149] For example, when the data acquisition module is executing, an unexecuted task in the unexecuted queue enters the unexecuted queue at 10:23:12. The service node starts executing the unexecuted task at 10:23:16 and completes the execution at 10:23:21. The execution time of the unexecuted task is from 10:23:16 to 10:23:21, a total of 5 seconds.

[0150] S32: For at least one node task carried by a service node, execute the following respectively: according to the task execution order of the service node, based on the execution time of at least one previously executed task, obtain the waiting time of a node task.

[0151] One of the node tasks is either an executed task or an unexecuted task.

[0152] For node tasks, the execution order is the same as the order in which they entered the unexecuted queue. After an unexecuted task is completed, it still enters the executed queue in the same order as the original task.

[0153] For a node task, its waiting time is the sum of the execution times of all unexecuted tasks before it enters the unexecuted queue.

[0154] like Figure 4 As shown, it is a schematic diagram of the waiting time provided in an embodiment of this application. Figure 4 Task N is a node task whose waiting time is yet to be determined. When it enters the unexecuted queue, it is placed at the tail of the queue, preceded by three unexecuted tasks: Task 1, Task 2, and Task 3. Therefore, the waiting time of Task N can only be determined after Task 1, Task 2, and Task 3 have all been executed. Figure 4 In the process, after tasks 1, 2, and 3 have all been completed, they are all in the executed queue. Task 1 took 3ms, Task 2 took 5ms, and Task 3 took 2ms. Therefore, the waiting time for Task N is 10ms.

[0155] The above-described method for confirming waiting time can be implemented in real time. For each unexecuted task, the preceding unexecuted tasks are identified when it enters the unexecuted queue. Once all preceding unexecuted tasks have been executed, the waiting time for the current task can be determined in real time.

[0156] For executed tasks, their waiting time can be determined and recorded using the methods described above when they are still pending tasks. Alternatively, when a task is still pending, the time it enters the pending queue and the time it begins execution can be recorded, with the difference between these two times serving as its waiting time. Alternatively, only the other pending tasks preceding it in the pending queue can be recorded. Once the pending task is executed and becomes an executed task, the execution times of the other pending tasks preceding it in the executed queue can be found directly, and their sum can be used as the waiting time for the executed task.

[0157] Optionally, the steps in S32 above can be performed by the delay detection module.

[0158] S33: When it is determined that a service node is in an overload state based on at least one obtained waiting time, determine the set of service types to be discarded based on the overload policy information of each service type on the service node.

[0159] The overload strategy information includes at least one of the following: total resource consumption and business priority; the total resource consumption is the sum of the resource consumption of at least one executed task under the corresponding business type.

[0160] Specifically, total resource consumption refers to the resources consumed by some or all of the tasks executed under the corresponding business type. Total resource consumption comprehensively reflects the amount of resources required to complete the business, providing a key basis for assessing service node load, optimizing resource allocation, and controlling costs.

[0161] Before implementing subsequent service overload handling methods on the service node, it is necessary to first determine whether the service node is overloaded. After determining that the service node is overloaded, this application considers that the service node needs to serve multiple different services, and there may be a single service or a small number of services suddenly causing the service node to overload. Therefore, this application can determine the type of service that needs to be overloaded by the total amount of resources consumed.

[0162] This application also considers that, due to factors such as the needs of the objects, service nodes often adopt different support strategies for different business types. For example, for a certain middleware service, the importance of financial business is higher than that of other businesses. When the current processing latency of the service increases, other businesses should be overloaded first to ensure that the financial business is not affected. Therefore, there are different business priorities among different business types, and this application can also determine the business types that need to be overloaded based on business priority.

[0163] The following section will first explain how to determine if a service node is in an overloaded state:

[0164] When converting the operational status of a service node (overloaded or normal) into a quantifiable objective standard, this application considers that when a service node is overloaded, the waiting time for newly entered unexecuted tasks in the unexecuted queue will significantly increase due to insufficient resources (such as CPU, thread pool, network bandwidth, etc.) of the service node. Therefore, this application uses the waiting time to determine whether a service node is in an overloaded state.

[0165] In an optional implementation, when S33 determines that the service node is in an overloaded state, the specific implementation method is as follows:

[0166] The current processing latency of a service node is determined based on at least one waiting time; when the current processing latency exceeds a preset overload threshold, the service node is determined to be in an overload state.

[0167] The overload threshold is a critical point used to quantify the operating status of a service node. When the current processing latency of a service node exceeds the overload threshold, it can be determined that the service node is in an overloaded state.

[0168] To balance the business demands of the target and the responsiveness of the service nodes, this application sets the overload threshold using at least one of the following methods:

[0169] Method 1: Derive the overload threshold by working backward from the promised maximum acceptable response time.

[0170] Since response time is the sum of the current processing latency of the service node, the average execution time, and the network transmission time, the overload threshold can be determined by combining the average execution time and the network transmission time after the maximum acceptable response time is determined.

[0171] For example, in an online shopping platform, the requirement is that 99% of requests must have a response time of ≤800ms; of which network transmission takes 100ms and the average time to execute payment is 300ms, the waiting threshold is 800ms-100ms-300ms=400ms.

[0172] Method 2: Based on historical data, calculate the average processing latency under normal operating conditions and add a safety buffer.

[0173] This application considers that the average processing latency of a service node under normal operating conditions can represent the steady-state capability of that service node and can serve as a benchmark for evaluating the operational status of the service node. Furthermore, considering the natural fluctuations in the operational status of service nodes, a buffer period needs to be reserved to avoid frequent false triggers.

[0174] For example, if a service node has an average processing latency of 50ms under normal operating conditions, with a peak latency fluctuating to 80ms, then the overload threshold can be set to 100ms (80ms + 20ms safety buffer).

[0175] Method 3: Determine the overload threshold by combining queue length and processing rate.

[0176] This application takes into account that, when the service node is in a normal state, the expected processing latency is related to the unexecuted queue and the execution rate of the service node. Therefore, this application determines the overload threshold based on the current length of the unexecuted queue and the execution rate of the service node in a normal state.

[0177] For example, in an online shopping platform, if a service node has 1000 unexecuted tasks and its normal execution rate is 50 tasks / second, then the overload threshold for that service node can be set to 1000 / 50 = 20 seconds.

[0178] Method 4: Determine the overload threshold based on the hardware performance of the service node.

[0179] In this approach, this application can dynamically determine the corresponding overload threshold in real time based on changes in the hardware performance (such as CPU and memory) of the service node. When the real-time load of the service node...

[0180] For example, when the hardware performance of a service node improves, the current overload threshold is also increased, allowing the service node to execute more unexecuted tasks without triggering subsequent service overload handling methods. Conversely, when hardware performance decreases, the current overload threshold is also decreased, allowing the service node to execute only the unexecuted tasks it can handle.

[0181] It should be noted that the above-mentioned overload threshold setting method is just a simple example. Other setting methods are also applicable to the embodiments of this application, and will not be repeated here.

[0182] In the above implementation, the operational status of service nodes is transformed from subjective perception to a quantifiable objective standard by setting an overload threshold. A critical tolerance boundary is defined for the processing latency of service nodes: when the processing latency exceeds the threshold, it indicates that the service node can no longer respond to requests within an acceptable time, and an overload handling method must be executed. This design not only avoids the risk of cascading failures caused by continuous overload operation, but also finds a balance between elastic resource scheduling and user experience. Furthermore, this application adapts to different hardware performance and heterogeneous resources by dynamically adjusting the current processing latency of service nodes, without relying on a fixed QPS threshold.

[0183] The following section describes how to determine the current processing latency of a service node based on the waiting time of a node task.

[0184] This application considers that recording the waiting time of each node's tasks in real time would incur significant storage and computational overhead for a single service node. Furthermore, while the distribution of waiting times for node tasks tends to stabilize when the service node is in a normal state, a significant change in the waiting times of a small number of node tasks occurs when the service node transitions from a normal state to an overloaded state. Therefore, this application determines the current processing latency of a service node based on the average waiting time of the most recently executed tasks or the waiting time of the most recently unexecuted tasks.

[0185] In one optional implementation, when determining the current processing delay of the service node in S33, the specific implementation method is as follows:

[0186] The average waiting time of multiple executed tasks is used as the current processing delay.

[0187] Specifically, this application considers the waiting time of the most recently completed executed tasks in the executed queue, which better reflects the current processing latency of the service node. Therefore, when selecting executed tasks to determine the current processing latency, embodiments of this application can select executed tasks by limiting their start execution time. For example, this application can limit the start execution time of executed tasks within the last 10 or 15 seconds to determine the current processing latency. Embodiments of this application can also select executed tasks by limiting their execution order. For example, this application can limit the execution order of executed tasks within the last 1000 tasks to determine the current processing latency. This application can also set a sliding window with a preset step size, and determine the current processing latency based on the executed tasks within the sliding window, and so on.

[0188] The number of executed tasks can be a pre-set fixed value, or it can be determined based on the length of the executed or unexecuted queue, the execution rate of the service node, or the resources of the service node. If computing resources are sufficient, the number of executed tasks can be set to a larger value, and so on.

[0189] Furthermore, after selecting multiple executed tasks to determine the current processing delay, the current processing delay can be determined based on the waiting time of the executed tasks. This application can directly average the waiting times of multiple executed tasks and use the average as the current processing delay. This application can also assign different weights to the waiting times of different executed tasks according to their execution order. For example, this application can assign higher weights to tasks executed later in the execution order and lower weights to tasks executed earlier in the execution order, and use the weighted average of the waiting times of multiple executed tasks as the current processing delay.

[0190] The method for determining the waiting time for each executed task can be found in the above embodiments, and will not be repeated here.

[0191] For example, in an online shopping platform, if a service node processes 1,000 orders per second (node ​​task), 100 orders can be randomly selected to calculate the average waiting time (e.g., 50ms) and used as the current processing delay of that service node.

[0192] like Figure 5A As shown, this is a schematic diagram illustrating how to determine the current processing delay according to an embodiment of this application. Figure 5A In this scenario, assuming the preset triggering conditions are met at time T0, tasks 4, 5, and 6 are used to determine the preprocessing delay. Since tasks 4, 5, and 6 enter the unexecuted queue almost simultaneously, the waiting time for task 4 is determined to be 10ms, for task 5 13ms, and for task 6 15ms. The average of these three waiting times, 12.67ms, is then taken as the current processing delay of the service node.

[0193] It should be noted that the above-mentioned selection method for the executed task is just a simple example. Other selection methods are also applicable to the embodiments of this application, and will not be repeated here.

[0194] In another optional implementation, when determining the current processing delay of the service node in S33, the specific implementation method is as follows:

[0195] The waiting time for unexecuted tasks used for probing is used as the current processing delay.

[0196] Among them, the unexecuted tasks used for detection are: unexecuted tasks added to the tail of the unexecuted queue.

[0197] Specifically, in scenarios requiring rapid determination of service node overload, an empty, unexecuted task (also known as a probe task) can be placed at the tail of the unexecuted queue to probe the current processing latency of the service node. Alternatively, multiple empty, unexecuted tasks (also known as probe tasks) can be placed at the tail of the unexecuted queue, and the average waiting time of these multiple unexecuted tasks can be used as the current processing latency. Furthermore, this application can assign different weights to the waiting times of different unexecuted tasks based on their execution order. For example, this application can assign higher weights to unexecuted tasks with later execution orders and lower weights to unexecuted tasks with earlier execution orders, and use the weighted average waiting time of multiple unexecuted tasks as the current processing latency; and so on.

[0198] The method for determining the waiting time for each unexecuted task can be found in the above embodiments, and will not be repeated here.

[0199] In addition, the probe task in this application can also be a non-empty unexecuted task. In this case, the probe task with a smaller workload can be placed at the tail of the unexecuted task queue. The waiting time of the probe task can also be used to determine the current processing delay of the service node, and it will not affect the subsequent unexecuted tasks and service nodes in the unexecuted queue.

[0200] like Figure 5B As shown, this is another schematic diagram for determining the current processing delay provided in an embodiment of this application. Figure 5B In the context of the above, assuming that a preset triggering condition is met at time T0, an unexecuted task used for detection will be... Figure 5B The probe task (in the queue) is placed at the tail of the unexecuted queue. Since this unexecuted task is empty, it will not add any additional load to the service node. Figure 5B As can be seen, when the probe task enters the unexecuted queue, there are Task 1, Task 2 and Task 3 in the unexecuted queue. Therefore, the waiting time of the probe task is the sum of the execution times of Task 1, Task 2 and Task 3, which is 10ms. This is taken as the current processing delay of the service node.

[0201] In the above embodiments, this application dynamically monitors the current processing latency of the service node each time a preset trigger condition is met, and uses the average waiting time of some executed tasks or the waiting time of the latest unexecuted task as the current processing latency of the service node, seeking a balance between accuracy, real-time performance, and resource cost. This avoids large deviations in the current processing latency and covers different task types. This allows for dynamic adjustment of subsequent overload strategies after determining that the service node is in an overload state based on the current processing latency.

[0202] Based on this, this application considers that determining the current processing delay of a service node by the waiting time of a single node task or a small number of node tasks may have some deviation. In order to reduce the deviation and not consume too many resources, this application also uses the following method to combine historical delay data to further improve the current processing delay of the service node.

[0203] In one optional implementation, when S33 updates the current processing delay of the service node, the specific implementation method is as follows:

[0204] Based on current processing latency and historical latency data, the updated current processing latency is predicted; when the updated current processing latency exceeds a preset overload threshold, the service node is determined to be in an overload state.

[0205] Specifically, historical latency data can be the processing latency of service nodes obtained when preset trigger conditions were previously met. This application can directly combine the current processing latency with the historical latency data. The combination method is described below:

[0206] Method 1: Calculate the average of the current latency data and the historical latency data, and use the average as the updated current latency data.

[0207] For example, if the current latency is 10ms and there are three historical latency data points: 6ms, 13ms, and 8ms, then the updated current processing latency is (6+13+8+10) / 4 = 9.25ms.

[0208] Method 2: Calculate the weighted average of the current latency data and historical latency data, and use this average as the updated current latency data. During the weighted average calculation, historical latency data can be weighted according to the current time interval, with higher weights assigned to more recent historical latency data and lower weights assigned to more distant historical latency data.

[0209] For example, if the current latency is 60ms and there are three historical latency data points: 65ms, 70ms, and 75ms, and these four data points are assigned weights of 0.1, 0.2, 0.3, and 0.4 from left to right, then the updated current processing latency is 60×0.1 + 65×0.2 + 70×0.3 + 75×0.4 = 71ms.

[0210] Method 3: Use the exponential smoothing algorithm to update the current processing latency, and use the smoothing coefficient α (0<α<1) to balance the impact of historical latency data and current processing latency.

[0211] For example, if the historical latency is 60ms, α = 0.2, and the current processing latency is 80ms, then the updated processing latency is: 0.2 × 80 + 0.8 × 60 = 64ms.

[0212] In addition, this application can also employ a double exponential smoothing algorithm, introducing a trend term on top of exponential smoothing to capture delayed linear trends. This application can also employ seasonal exponential smoothing, adding a seasonal factor to the double exponential smoothing, which is suitable for cyclical fluctuation scenarios.

[0213] It should be noted that the above-described update method for processing delay is just a simple example. Other update methods are also applicable to the embodiments of this application, and will not be repeated here.

[0214] In addition, historical latency data can also be the waiting time of previously unexecuted tasks used for probing. For example, when obtaining the current processing latency through unexecuted tasks used for probing, the waiting time of the previous unexecuted task used for probing can be obtained if the current processing latency does not exceed the overload threshold. If the waiting time does not exceed the overload threshold, the service node is in a normal state. If the waiting time exceeds the overload threshold, the service node is determined to be in an overloaded state.

[0215] For example, if the overload threshold is 10ms, and the waiting time for the unexecuted task used for this probe is 9.5ms, then if the waiting time for the previous unexecuted task used for probe is obtained, say 13ms, then it is determined that the service node is in an overloaded state.

[0216] If the current processing delay exceeds a preset overload threshold after updating the current processing delay using one of the above methods, then the service node is determined to be in an overload state.

[0217] In the above embodiments, this application flexibly selects the method for determining whether a service node is overloaded based on the urgency of executing the service overload handling method, making it widely applicable to various application scenarios and providing a better user experience. Furthermore, this application achieves adaptive adjustment by dynamically detecting the current processing latency of the service node and combining it with historical data to predict the latency trend of the service node.

[0218] After determining that a service node is overloaded, this application considers that the service node needs to serve multiple different services, and that overload may be caused by a single service or a small number of services suddenly surging. Therefore, this application can determine the types of services that need overload handling by measuring the total resource consumption. This application also considers that different service types have different priorities due to object requirements and other reasons. Therefore, this application can also determine the types of services that need overload handling based on service priority.

[0219] Specifically, in the embodiments of this application, the resource usage of different business types can be quantified through different dimensions.

[0220] For example, in the time dimension, the resource consumption of each task can be reflected by its execution time. The sum of the execution times of some or all executed tasks under a corresponding business type can be used as the total resource consumption for that business type. For instance, for a certain business type containing 30 executed tasks, if the execution time of each of these 30 executed tasks is 5ms, then the resource consumption of each task is 5ms, and the total resource consumption for this business type is 150ms.

[0221] For example, in the dimension of computing resources, the resource consumption of each task can be reflected by at least one of the Central Processing Unit (CPU) consumption and memory usage. The sum of the computing resources of some or all executed tasks under a certain business type can be used as the total resource consumption for that business type. By monitoring the total resource consumption, high-load business types can be identified, expansion strategies and service overload strategies can be rationally planned, and efficient utilization of service nodes can be ensured, thereby improving overall service efficiency and economic benefits.

[0222] It should be noted that the above method of quantifying the total amount of resources consumed is just a simple example. Other quantification methods are also applicable to the embodiments of this application, and will not be repeated here.

[0223] Specifically, when prioritizing business types, different dimensions can be used for classification.

[0224] For example, business priorities can be categorized based on the importance of different business types, assigning higher priorities to more core business types. For instance, assuming there are three priority levels: level 3 (highest), level 2 (middle), and level 1 (lowest), then for an online shopping platform, the core business is purchasing, so it would be assigned the highest priority level 3. Browsing is less core than purchasing, so it would be assigned the second-highest priority level 2. Q&A is a peripheral business, so it would be assigned the lowest priority level 1.

[0225] For example, business priorities can be categorized based on the value contribution of each business type. Business types that generate higher revenue, contribute more profit, and attract more customers can be assigned a higher priority. For instance, assuming there are three priority levels: level 3 is the highest, level 2 is the middle, and level 1 is the lowest, then for an online shopping platform, the purchase service generates the highest revenue, thus it would be assigned the highest priority level 3. While browsing doesn't directly generate revenue for the platform, its long-term value would give it the second-highest priority level 2. And since Q&A is an after-sales service, it would be assigned the lowest priority level 1.

[0226] For example, business priorities can be categorized based on the urgency of the business type. For instance, assuming there are three priority levels: level 3 (highest), level 2 (middle), and level 1 (lowest), then for a financial platform, irregular remittance transactions, if not completed on time, could cause significant financial losses to customers and subject the platform to fines and other penalties. Therefore, this business type would be assigned the highest priority level, 3. Browsing transactions require timely responses to customer requests, so they would be assigned the second-highest priority level, 2. Application interface optimization, obviously a business that can be postponed, would be assigned the lowest priority level, 1.

[0227] It should be noted that the above-mentioned method of setting business priority is just a simple example. Other methods of setting priority are also applicable to the embodiments of this application, and will not be repeated here.

[0228] The following section describes how to determine the types of services to be discarded, taking into account both total resource consumption and business priority.

[0229] Method 1: Overload strategy information includes: total resource consumption.

[0230] Specifically, the business types are sorted according to their total resource consumption. The business types with the highest resource consumption are designated as the discardable business types and added to the discardable business type set.

[0231] In this approach, the application sorts the service types according to the total amount of resources consumed by each service type. Since the service types with the highest or highest total resource consumption occupy the most resources of the service nodes, when the service nodes are overloaded, the load on the service nodes can be relieved by discarding these service types first, so that the service nodes can be restored to normal.

[0232] For example, if a service node has three business types: Business Type 1, Business Type 2, and Business Type 3, and Business Type 2 has the highest total resource consumption, followed by Business Type 1, with Business Type 3 having the lowest, then the three business types are sorted from highest to lowest resource consumption as follows: Business Type 2, Business Type 1, and Business Type 3. Assuming that the top two business types are designated as those to be discarded, then Business Type 2 and Business Type 1 are also designated as business types to be discarded and added to the set of business types to be discarded.

[0233] Method 2: Overload strategy information includes: business priority.

[0234] Specifically, business types whose business priority does not exceed the preset overload level are designated as business types to be discarded and added to the set of business types to be discarded.

[0235] As can be seen from the above, this application can prioritize business types based on their importance, value proportion, and urgency. Then, by setting overload levels, it determines the business types to be discarded from among these.

[0236] The overload level can be determined based on the urgency of service overload handling. For example, when rapid service overload handling is required, the overload level can be set higher, allowing tasks from more business types to be dropped, enabling service nodes to recover to normal status more quickly. For instance, during a major e-commerce promotion...

[0237] Alternatively, it can be determined according to the number of business priority levels. For example, when there are many business priorities, a higher overload level can be set so that the load status of service nodes can be significantly changed, and so on.

[0238] For example, if a service node has three business types: Business Type 1, Business Type 2, and Business Type 3, after sorting these three business types according to their importance, their business priorities from highest to lowest are: Business Type 2, Business Type 1, and Business Type 3. Assuming the overload level is set to level 2, the business types ranked last two will be designated as those to be discarded; that is, Business Type 1 and Business Type 3 will be added to the set of business types to be discarded.

[0239] In addition, this application may also omit the overload level and directly select a preset number of business types with lower priority as the business types to be discarded, etc.

[0240] Method 3: Overload strategy information includes: total resource consumption and business priority.

[0241] Specifically, for each business type, when the total resource consumption of a business type exceeds the corresponding resource consumption limit of that business type, the business type is designated as a business type to be discarded and added to the set of business types to be discarded; when the resource consumption of each business type on the service node is lower than the corresponding resource consumption limit, the business type whose business priority does not exceed the preset overload level is designated as a business type to be discarded and added to the set of business types to be discarded.

[0242] In this approach, this application first considers that service nodes need to serve multiple different services, and there may be instances of single or a small number of services suddenly causing service node overload. Therefore, this application can first determine the types of services that need overload handling by setting resource consumption limits. Simultaneously, even when all service types exceed their resource consumption limits, the service node may still be overloaded. In this case, this application also considers that different service types have different priorities due to factors such as importance. Therefore, this application can further determine the types of services that need overload handling based on their priorities.

[0243] For example, if a service node has three business types: Business Type 1, Business Type 2, and Business Type 3, and assuming that the total resource consumption of these three business types exceeds their respective resource consumption limits, then based on the business priority of these three business types: Business Type 2, Business Type 1, and Business Type 3, the business types that have not exceeded the overload level (Level 2) are designated as the business types to be discarded. Specifically, Business Type 1 and Business Type 3 are designated as the business types to be discarded and added to the set of business types to be discarded.

[0244] In the above embodiments, this application combines at least one of total resource consumption and business priority to determine the business types to be discarded from each business type, successfully addressing the service overload problem caused by a single business surge, and ensuring that tasks under the core business type can be executed smoothly.

[0245] When determining the types of services to be discarded based on the total resource consumption, this application considers setting a fixed upper limit for resource consumption. Although it can determine which service types consume more total resources, it is not convenient to dynamically adjust the resource consumption ratio between each service type. Therefore, this application can set the upper limit for resource consumption in the form of the percentage of available resources.

[0246] In addition to the total resource consumption and business priority mentioned above, overload strategy information can also include path dependencies of business types. For example, some business types are core dependencies of other services. Dropping tasks in a particular business type might trigger a cascading crash. Therefore, the path dependencies between different business types can be statistically analyzed first, and the business types to be dropped can be determined based on the statistical results. Historical failure data of business types can also be used; for example, business types that have frequently experienced failures need to be given special attention.

[0247] It should be noted that the above overload strategy information is just a simple example. Other information that can be used to determine the types of services to be discarded is also applicable to the embodiments of this application, and will not be repeated here.

[0248] In an optional implementation, when determining the set of service types to be discarded based on the total resource consumption and the upper limit of resource consumption in S33, the specific implementation method is as follows:

[0249] Determine the ratio of the total resource consumption of a business type to the sum of the total resource consumption of all business types; if the ratio exceeds the proportion of available resources corresponding to a business type, then designate that business type as a business type to be discarded.

[0250] Specifically, this application considers that when the total resource consumption is set as a percentage of available resources, if the task used to determine the total resource consumption is not restricted, it is necessary to calculate the resource consumption of all tasks of each business type under the service node, which will occupy too much computing resources and may affect the execution of node tasks by the service node.

[0251] Therefore, in this application, the total resource consumption is the sum of the resource consumption of each reference task under the corresponding business type; each reference task is an executed task whose execution order is within a preset range.

[0252] Specifically, for a service node, all executed tasks within a preset order are selected. These executed tasks are those under each business type. When it is necessary to determine the available resource ratio of a certain business type, if the ratio of the total resource consumption of the executed tasks belonging to that business type to the total resource consumption of all executed tasks within the preset order exceeds the available resource ratio of that business type, then that business type can be discarded.

[0253] For example, if a service node has 1000 executed tasks within a preset order, belonging to three business types: Business Type 1, Business Type 2, and Business Type 3, and 300 of the 1000 executed tasks belong to Business Type 1, then the ratio of the total resource consumption of these 300 tasks to the total resource consumption of all 1000 tasks is used to determine whether Business Type 1 can be considered for discarding. Assuming this ratio is 56% and the available resource ratio for Business Type 1 is 45%, then Business Type 1 should be considered for discarding.

[0254] In the above embodiments, this application determines whether the total resource consumption of a certain business type exceeds the proportion of available resources by selecting executed tasks within a preset order range. This avoids statistical analysis of the resource consumption of all executed tasks under the service node, reduces the consumption of computing resources, and dynamically adjusts the proportion of resource consumption among different business types.

[0255] The following describes the method used in this application to determine the reference tasks for each business type:

[0256] Method 1: Using a sliding window with a fourth preset step size, dynamically perform the following operations:

[0257] Using a sliding window, select a corresponding number of executed tasks from the executed queue as reference tasks; for each business type, the sum of the resource consumption of all reference tasks belonging to a business type is taken as the current total resource consumption of a business type.

[0258] Specifically, since the sliding window needs to determine the reference tasks within a preset order range, the preset order range is dynamically updated as the sliding window slides. The fourth preset step size of the sliding window is the size of the preset order range. For example, if the step size of the sliding window is 20, then as the sliding window slides, the preset order range can be updated from 1 to 20, to 21 to 40, then to 41 to 60, and so on.

[0259] Because a sliding window is used to determine the reference task, the total resource consumption of the reference task for each business type is dynamically updated as the sliding window moves.

[0260] For example, when the available resources for a certain business type account for 30%, if the total execution time of all executed tasks of the service nodes in the current sliding window is 1000ms and the total execution time of the executed tasks under the current business type is 350ms, then the resource share of this business type has reached 35%, and it should be determined that the total resource consumption of this business type has exceeded the upper limit of available resources.

[0261] like Figure 6A As shown, it is a schematic diagram of the total resource consumption provided in an embodiment of this application. Figure 6A The total resource consumption is reflected by the execution time. The sliding window step size is 3, meaning the preset order range includes 3 executed tasks. At time T0, the sliding window slides to tasks 2, 3, and 4. At this time, the total resource consumption of the reference tasks for each business type is 10ms. For business type 2, its reference task is task 2, and the ratio of the total resource consumption of business type 2 to the total resource consumption of all business types is 50%. Assuming that the available resource ratio of business type 2 is 30%, then business type 2 can be considered as a business type to be discarded at time T0. At time T1, the reference tasks for each business type are: tasks 3, 4, and 5, with a total resource consumption of 7ms. The reference task for business type 2 is task 5, and the ratio of the total resource consumption of business type 2 to the total resource consumption of all business types is 28.57%. Therefore, business type 2 cannot be considered as a business type to be discarded at time T1.

[0262] Method 2: Use the latest preset order of executed tasks in the executed queue as reference tasks.

[0263] This application takes into account that the later executed tasks in the executed queue are more likely to reflect the current resource consumption of each business type. Therefore, this application may also select only a fixed number of the latest executed tasks as reference tasks.

[0264] like Figure 6B As shown, this is a schematic diagram of another total resource consumption provided in an embodiment of this application. Figure 6B The total resource consumption is reflected by the execution time. A fixed number of 3 tasks are selected as reference tasks each time. At time T0, the reference tasks for each business type are tasks 4, 5, and 6, with a total resource consumption of 9ms. For business type 2, its reference task is task 5. The ratio of the total resource consumption of business type 2 to the total resource consumption of all business types is 22.22%. Assuming that the available resource ratio for business type 2 is 30%, business type 2 cannot be considered for discarding at time T0. At time T1, the reference tasks for each business type are tasks 5, 6, and 7, with a total resource consumption of 10ms. For business type 2, its reference task is task 5. The ratio of the total resource consumption of business type 2 to the total resource consumption of all business types is 20%. Therefore, business type 2 cannot be considered for discarding at time T1.

[0265] It should be noted that the above-mentioned selection method for the reference task is just a simple example. Other selection methods are also applicable to the embodiments of this application, and will not be repeated here.

[0266] In the above embodiments, this application uses multiple methods to determine the reference task for calculating the total resource consumption, which can reflect the current total resource consumption of each business type and avoid resource occupation problems caused by statistically analyzing the execution time of too many executed tasks.

[0267] Optionally, the steps in S33 above can be executed by the decision execution module.

[0268] S34: Discard at least one unexecuted task associated with the set of business types to be discarded.

[0269] Once it is determined that the service node is in an overloaded state and the set of discardable service types is identified, this application can discard the unexecuted tasks under the discardable service types.

[0270] This application takes into account that directly discarding all tasks associated with the set of business types to be discarded may greatly reduce the user experience of the object. The object may tolerate occasional response failures, but cannot accept the complete unusability of individual business types. Therefore, this application adopts the following method to gradually discard the unexecuted tasks associated with the set of business types to be discarded.

[0271] In an optional implementation, when at least one unexecuted task associated with the set of service types to be discarded is discarded in S34, the specific implementation method is as follows:

[0272] When there are discardable service types whose total resource consumption exceeds the corresponding resource consumption limit, for each discardable service type, based on a preset overload ratio, one unexecuted task under the discardable service type is discarded until the service node returns to normal.

[0273] Specifically, as described above, the total resource consumption for each business type is dynamically updated based on the changes in the executed tasks carried by the service nodes. Therefore, when a preset trigger condition is met, and it is determined that there are business types in the set of business types to be discarded whose total resource consumption exceeds the corresponding resource consumption limit, the tasks under these business types are discarded first. This application considers that directly discarding all tasks under these business types would render the object completely unable to use these services, thereby reducing the user experience. Therefore, this application presets an overload ratio, and discards tasks under the business types to be discarded proportionally based on this overload ratio.

[0274] The following describes how the overload ratio is set in this application:

[0275] Method 1: Directly set a uniform, low overload ratio for each type of service to be discarded.

[0276] In this method, to minimize task dropping and alleviate service node overload, a low overload ratio is set directly. For example, it can be set to 1%, 2%, etc. Initially, unexecuted tasks under the service type to be discarded are dropped according to this preset overload ratio. If the service node remains overloaded after a period of time, it means that the overload ratio is still relatively low relative to the service node's overload status. In this case, the overload ratio can be increased, dropping more tasks to restore the service node to a normal state.

[0277] In one optional implementation, when the overload ratio is increased in S34, the specific implementation method is as follows:

[0278] Every first preset time interval, the overload ratio is increased based on the first preset step size.

[0279] Specifically, when discarding unexecuted tasks based on a preset overload ratio, this application can determine whether the service node is still in an overload state every first preset time interval. If the service node is still in an overload state, it means that the overload ratio is still low relative to the overload state of the service node. Therefore, the overload ratio can be increased based on the first preset step size, and more tasks can be discarded to enable the service node to recover to a normal state more quickly.

[0280] In this application, a uniform first preset duration and first preset step size can be set for each type of service to be discarded, or different first preset duration and first preset step size can be set for different types of services to be discarded, taking into account the load status of the service node and the attributes of each type of service to be discarded.

[0281] The following example illustrates how to set different first preset durations and first preset step sizes for different types of services to be discarded:

[0282] The first preset duration can be determined based on the current processing latency of the service node. For example, when the current processing latency of the service node is long, in order to alleviate the severe overload of the service node more quickly, the first preset duration can be set to a shorter time, such as 0.5s, 1s, etc. When the current processing latency of the service node is short, the first preset duration can be set to a longer time, such as 3s, 5s, etc.

[0283] The initial preset duration can also be determined based on the business priority of the business types to be discarded. For example, for business types with low priority, since they are non-core business types, a higher proportion of discarded tasks will not significantly impact the user experience. Therefore, the initial preset duration can be set shorter to quickly and repeatedly increase the overload ratio and accelerate the discarding of tasks under that business type. For business types with high priority, since they are core business types, a higher proportion of discarded tasks can significantly impact the user experience. Therefore, the initial preset duration can be set longer to ensure task execution as much as possible.

[0284] It should be noted that the above-mentioned setting method for the first preset duration is just a simple example. Other setting methods are also applicable to the embodiments of this application, and will not be repeated here.

[0285] Similarly, the first preset step size can be determined using the above method. When the current processing latency of a service node is long, to alleviate the severe overload state of the service node more quickly, the first preset step size can be set to a longer step size, such as 5%, 10%, etc. When the current processing latency of a service node is short, the first preset step size can be set to a shorter time, such as 1%, 2%, etc. It can also be determined based on the business priority of the business type to be discarded. For example, for business types with low business priority to be discarded, since they are non-core business types, a higher proportion of discarded tasks will not have a significant impact on the user experience. Therefore, the first preset step size can be set to a longer one to quickly increase the overload ratio and accelerate the discarding of tasks under that business type. For business types with high business priority to be discarded, since they are core business types, a higher proportion of discarded tasks can easily have a significant impact on the user experience. Therefore, the first preset step size can be set to a shorter one to ensure task execution as much as possible.

[0286] It should be noted that the above-mentioned setting method for the first preset step size is just a simple example. Other setting methods are also applicable to the embodiments of this application, and will not be repeated here.

[0287] In the above embodiments, this application dynamically adjusts the overload ratio, enabling service nodes in a severely overloaded state to recover to normal status more quickly, thereby improving the user experience.

[0288] Method 2: Dynamically set the overload ratio based on the real-time load indicators of the service nodes.

[0289] This approach dynamically determines the overload ratio by monitoring key load metrics of service nodes, achieving the goal of "the more severe the overload, the higher the dropout ratio." These key load metrics include, but are not limited to: CPU utilization, memory utilization, and unexecuted queue length. Specifically, CPU utilization, memory utilization, or unexecuted queue length can be positively correlated with the overload ratio.

[0290] For example, when CPU utilization is above 80% but below 90%, set the overload ratio to 10%; when it exceeds 90%, set the overload ratio to 30%. Or, when memory utilization is above 80% but below 90%, set the overload ratio to 10%; when it exceeds 90%, set the overload ratio to 30%. Or, when the unexecuted queue length is above 1000 but below 2000, set the overload ratio to 20%; when it exceeds 2000, set the overload ratio to 50%, and so on.

[0291] Method 3: Set the overload ratio based on time window or traffic pattern.

[0292] In this method, the overload ratio is set based on time period (such as peak / off-peak period) or traffic characteristics (such as sudden traffic surges). During peak periods or sudden traffic surges, the overload status of service nodes is severe, so the overload ratio can be set higher. During off-peak periods, even if service nodes are overloaded, the overload is not very severe, so the overload ratio can be set lower.

[0293] For example, if it is known that the service node's peak period is between 8 PM and 10 PM daily, then the default overload ratio during 8 PM and 10 PM (peak traffic) is 15%, and the overload ratio at other times is 5%. As another example, when traffic suddenly increases by 200%, the overload ratio increases from 10% to 50% in stages.

[0294] It should be noted that the above-mentioned overload ratio setting method is just a simple example. Other setting methods are also applicable to the embodiments of this application, and will not be repeated here.

[0295] In this embodiment, the application can restore the service node to normal status by discarding unexecuted tasks under the discardable service type whose total resource consumption exceeds the resource consumption limit.

[0296] However, in some cases, even if unexecuted tasks under the discardable business types whose total resource consumption exceeds the resource consumption limit are discarded, so that the total resource consumption of each business type is below the resource consumption limit, the service node is still in an overloaded state. In this case, this application can also use the following method to handle the overload of the service node:

[0297] In another optional implementation, when at least one unexecuted task associated with the set of service types to be discarded is discarded in S34, the specific implementation is as follows:

[0298] When there are pending discarded service types whose total resource consumption exceeds the corresponding resource consumption limit, for each pending discarded service type, unexecuted tasks under that pending discarded service type are discarded based on a preset overload ratio until the latest total resource consumption of a pending discarded service type is lower than the corresponding resource consumption limit. When the total resource consumption of each pending discarded service type is lower than the corresponding resource consumption limit, and the service node is still in an overloaded state, unexecuted tasks under the corresponding pending discarded service type are discarded in sequence based on the business priority of each pending discarded service type in the pending discarded service type set from low to high, and the preset overload ratio, until the service node returns to normal.

[0299] In this embodiment, after discarding unexecuted tasks under the discardable service types whose total resource consumption exceeds the resource consumption limit using the above-described implementation method, it was found that even after the total resource consumption of each service type is lower than the resource consumption limit, the service node is still in an overloaded state. Therefore, this application continues to discard unexecuted tasks according to the service priority of each service type.

[0300] Specifically, this application can directly use the current set of business types to be discarded, and according to the business priority of the current business types to be discarded, from low to high, first discard the unexecuted tasks under the lowest business priority business types according to the overload ratio, and then gradually increase the overload ratio until the overload ratio of the lowest business priority business types to be discarded reaches 100%. Then, according to the preset overload ratio (the initial overload ratio before it was increased), discard the unexecuted tasks under the next lowest business priority business types to be discarded, and so on, until the service node returns to normal.

[0301] For example, there are 5 business types: Business Type 1, Business Type 2, Business Type 3, Business Type 4, and Business Type 5. The preset overload ratio (the initial overload ratio before increase) is 1%. Among them, Business Type 2, Business Type 3, and Business Type 4 are in the set of business types to be discarded, and are sorted in ascending order of business priority as follows: Business Type 4, Business Type 3, Business Type 2. First, unexecuted tasks in Business Type 4 are discarded according to the 1% overload ratio, and the overload ratio is gradually increased. If the service node is still overloaded when the overload ratio is increased to 100%, or when it reaches the overload ratio threshold, then while discarding unexecuted tasks under Business Type 2, unexecuted tasks in Business Type 4 are also discarded according to the 1% overload ratio.

[0302] However, this application also considers that in some cases, even after all the unexecuted entities associated with the set of business types to be discarded have been discarded, the service node may still be in an overloaded state. In this case, this application adopts the following method to handle the overload of the service node:

[0303] Optionally, when the set of unexecuted tasks associated with the business types to be discarded is empty, and the service nodes are still in an overloaded state, the specific implementation method is as follows:

[0304] Gradually increase the overload level until the service node returns to normal.

[0305] Each time the overload level is increased, the following process is performed: based on the increased overload level, the set of business types to be discarded is redefined from those that have never been added to the set of business types to be discarded; at least one unexecuted task associated with the latest set of business types to be discarded is discarded.

[0306] Specifically, since this application determines the types of services to be discarded based on overload levels when the resource consumption of each service type on the service node is below the corresponding resource consumption limit, the overload level can be increased to prioritize service types with higher business priority. In other words, when the resource consumption of each service type on the service node is below the corresponding resource consumption limit, unexecuted tasks under lower-priority service types are discarded first, followed by unexecuted tasks under higher-priority service types.

[0307] When increasing the overload level, it can be done incrementally or by increasing it according to a preset step size. When increasing according to a preset step size, the preset step size can be determined based on the overload status of the service node. For example, a longer preset step size can be selected when the overload status of the service node is more severe, and a shorter preset step size can be selected when the overload status of the service node is less severe, and so on.

[0308] For example, there are 5 business types: Business Type 1, Business Type 2, Business Type 3, Business Type 4, and Business Type 5. The preset overload ratio (the initial overload ratio before increase) is 1%. The business types are sorted from lowest to highest priority as follows: Business Type 1 (Level 1), Business Type 5 (Level 2), Business Type 4 (Level 3), Business Type 3 (Level 4), and Business Type 2 (Level 5). Since the initial overload level is Level 1, Business Type 1 is designated as the business type to be discarded and assigned to the discardable business type set. Initially, unexecuted tasks in Business Type 1 are discarded at an overload ratio of 1%, and the overload ratio is gradually increased. If the service node remains overloaded when the overload ratio reaches 100% or the overload ratio threshold, then the unexecuted tasks associated with the discardable business type set are empty. At this point, if the overload level is increased to level 2, then business type 5 will be designated as a business type to be discarded and assigned to the set of business types to be discarded. In addition to discarding the unexecuted tasks under business type 2, 1% of the unexecuted tasks in business type 5 will be discarded according to the overload ratio, and so on, until the service node returns to normal.

[0309] In the above embodiments, this application prioritizes discarding business types with lower business priority from each business type by gradually increasing the overload level or increasing the overload level by a preset step size. While ensuring that the unexecuted tasks of the core business types can be executed stably, the overload status of the service nodes is alleviated by increasing the overload level, thereby improving the service experience of the object.

[0310] In another optional implementation, when at least one unexecuted task associated with the set of service types to be discarded is discarded in S34, the specific implementation is as follows:

[0311] When the total resource consumption of each type of business to be discarded is lower than the corresponding resource consumption limit, and the service node is still overloaded, the unexecuted tasks under the corresponding business type to be discarded are discarded in sequence according to the business priority of each type of business to be discarded in the set of business types to be discarded from low to high, and the preset overload ratio, until the service node returns to normal.

[0312] In this implementation, the set of service types to be discarded is determined directly based on the overload level, and the service priorities of all service types to be discarded exceed the overload level. This application discards unexecuted tasks according to the service priority of the service types to be discarded, from low to high.

[0313] Similar to the previous implementation, in this implementation, if the unexecuted tasks associated with the set of service types to be discarded are empty, the overload level can be increased, and based on the increased overload level, the service types to be discarded can be re-determined from the service types outside the current set of service types to be discarded, and the unexecuted tasks can be discarded according to the overload ratio. This will not be elaborated further here.

[0314] In the above implementation, this application combines the resource consumption of business types and the priority of business types to discard unexecuted tasks, which not only avoids the problem of service node overload caused by overload of a single business type, but also ensures the smooth execution of unexecuted tasks under core business types.

[0315] For each business type, this application considers that a business type can contain multiple request types, and different request types often have different request priorities within the same business type. For example, for a certain middleware service, the importance of financial business is higher than that of other businesses. When service latency increases, other businesses should be overloaded first to ensure that the financial business is not affected. If the financial business still cannot recover after other businesses are overloaded, non-critical request types in the financial business (such as historical bill inquiries) need to be overloaded first to ensure core transaction requests (such as payments and refunds). Therefore, when discarding unexecuted tasks, this application can also combine the request type of each unexecuted task to determine which unexecuted tasks should be discarded.

[0316] In an optional implementation, when discarding unexecuted tasks under a certain type of service to be discarded in S34, the specific implementation method is as follows:

[0317] For a business type to be discarded, based on the corresponding request priority from low to high and the preset overload ratio, the unexecuted tasks under the corresponding request type are discarded in sequence until the service node returns to normal, or until the unexecuted tasks under a business type to be discarded are empty.

[0318] Each request type corresponds to a request priority.

[0319] When a business type includes multiple request types, the total resource consumption of the business type can be calculated by request type.

[0320] like Figure 7 As shown, it is a schematic diagram of the total resource consumption provided in an embodiment of this application. Figure 7 The total resource consumption is quantified by time. Specifically, the execution time of each executed task is considered the resource consumption of that task; the execution time of all executed tasks under each request type is considered the resource consumption of that request type; and the execution time of all request types under each business type is considered the resource consumption of that business type. Business type 1 includes two request types: GET and POST. The total execution time for GET tasks is 55ms, and for POST tasks it is 93ms. Business type 2 includes two request types: QUERY and PAY. The total execution time for QUERY tasks is 370ms, and for PAY tasks it is 1102ms. Business type 3 includes two request types: GET and LOG. The total execution time for GET tasks is 25ms, and for LOG tasks it is 105ms. Business type 4 includes two request types: test (foo) and example (bar). The total execution time of the tasks under foo is 395ms, and the total execution time of the tasks under bar is 1107ms.

[0321] exist Figure 7 In determining whether the total resource consumption of business type 1 exceeds the resource consumption limit, the total execution time of business type 1 (148ms = 55ms + 93ms) can be compared with the overall resource consumption limit. Figure 7 The ratio of the total execution time of 3252ms in the process is 0.0455, which is compared with the available resource ratio of business type 1.

[0322] In the above implementation, this application avoids the problem that traditional single indicators cannot reflect the actual load of service nodes by cross-statistically analyzing the executed tasks of service nodes in terms of business and request dimensions.

[0323] Specifically, when discarding unexecuted tasks under a business type to be discarded, if the business type to be discarded contains multiple request types, the unexecuted tasks under the request type with the lowest request priority will be discarded first.

[0324] When discarding unexecuted tasks under a request type, similar to discarding unexecuted tasks under a business type according to the overload ratio, this application considers that directly discarding all tasks under these request types would render the object completely unable to use these requests, thereby degrading the user experience. Therefore, this application presets an overload ratio and discards tasks under the request type proportionally based on the overload ratio.

[0325] The following describes how the overload ratio is set in this application:

[0326] Method 1: Directly set a uniform, low overload ratio for each type of service to be discarded.

[0327] In this approach, to minimize task dropping and alleviate service node overload, a low overload ratio is set directly. For example, it can be set to 1%, 2%, etc. Initially, unexecuted tasks under the request type are dropped according to this preset overload ratio. If the service node remains overloaded after a period of time, it means that the overload ratio is still relatively low relative to the service node's overload status. In this case, the overload ratio can be increased, dropping more tasks to restore the service node to a normal state.

[0328] In one optional implementation, when the overload ratio is increased in S34, the specific implementation method is as follows:

[0329] Every third preset time interval, the overload ratio is increased based on the third preset step size.

[0330] The third preset duration can be the same as or different from the first preset duration; the third preset step size can be the same as or different from the first preset step size.

[0331] Specifically, when discarding unexecuted tasks based on a preset overload ratio, this application can determine whether the service node is still in an overload state every third preset time interval. If the service node is still in an overload state, it means that the overload ratio is still low relative to the overload state of the service node. Therefore, the overload ratio can be increased based on the third preset step size, and more tasks can be discarded to enable the service node to recover to a normal state more quickly.

[0332] For example, consider a business type to be discarded, which includes three request types: Request Type 1, Request Type 2, and Request Type 3. These three request types are ordered from lowest to highest priority as: Request Type 2, Request Type 1, and Request Type 3. The preset overload ratio is 1%, the third preset duration is 1 second, and the third preset step size is 1%. When discarding unexecuted tasks under this business type, first, based on the 1% overload ratio, unexecuted tasks under Request Type 2 are discarded. Every 1 second, the current overload ratio is increased by 1%, and unexecuted tasks under Request Type 2 are discarded according to the increased overload ratio. If the overload ratio increases to 100%, or to the overload ratio threshold, and the service node is still overloaded, then while discarding unexecuted tasks under Request Type 2, unexecuted tasks under Request Type 1 are also discarded based on the 1% overload ratio.

[0333] In this application, a uniform third preset duration and third preset step size can be set for each request type, or different third preset duration and third preset step size can be set for different request types based on the load status of the service node and the attributes of each request type.

[0334] The following examples illustrate how to set different third preset durations and third preset steps for different request types:

[0335] The third preset duration can be determined based on the current processing latency of the service node. For example, when the current processing latency of the service node is long, the third preset duration can be set to a shorter time, such as 0.5s, 1s, etc., to alleviate the severe overload of the service node more quickly. When the current processing latency of the service node is short, the third preset duration can be set to a longer time, such as 3s, 5s, etc.

[0336] The third preset duration can also be determined based on the request priority of the request type. For example, for request types with low priority, since they are non-core request types, a higher proportion of dropped tasks will not significantly impact the user experience. Therefore, the third preset duration can be set shorter to quickly and repeatedly increase the overload ratio and accelerate the dropping of tasks for that request type. For request types with high priority, since they are core request types, a higher proportion of dropped tasks can significantly impact the user experience. Therefore, the third preset duration can be set longer to ensure task execution as much as possible.

[0337] It should be noted that the above-mentioned third preset duration setting method is just a simple example, and other setting methods are also applicable to the embodiments of this application, and will not be repeated here.

[0338] Similarly, the third preset step size can be determined using the above method. When the current processing latency of a service node is long, to alleviate the severe overload state of the service node more quickly, the third preset step size can be set to a longer step size, such as 5%, 10%, etc. When the current processing latency of a service node is short, the third preset step size can be set to a shorter time, such as 1%, 2%, etc. It can also be determined based on the request priority of the request type. For example, for request types with low priority, since they are non-core request types, a higher proportion of dropped tasks will not significantly affect the user experience. Therefore, the third preset step size can be set to a longer one to quickly increase the overload ratio and accelerate the dropping of tasks for that request type. For request types with high priority, since they are request business types, a higher proportion of dropped tasks can significantly impact the user experience. Therefore, the third preset step size can be set to a shorter one to ensure task execution as much as possible.

[0339] It should be noted that the above-mentioned setting method for the third preset step size is just a simple example. Other setting methods are also applicable to the embodiments of this application, and will not be repeated here.

[0340] In the above embodiments, this application dynamically adjusts the overload ratio, enabling service nodes in a severely overloaded state to recover to normal status more quickly, thereby improving the user experience.

[0341] Method 2: Dynamically set the overload ratio based on the real-time load indicators of the service nodes.

[0342] This approach dynamically determines the overload ratio by monitoring key load metrics of service nodes, achieving the goal of "the more severe the overload, the higher the dropout ratio." These key load metrics include, but are not limited to: CPU utilization, memory utilization, and unexecuted queue length. Specifically, CPU utilization, memory utilization, or unexecuted queue length can be positively correlated with the overload ratio.

[0343] For example, when CPU utilization is above 80% but below 90%, set the overload ratio to 10%; when it exceeds 90%, set the overload ratio to 30%. Or, when memory utilization is above 80% but below 90%, set the overload ratio to 10%; when it exceeds 90%, set the overload ratio to 30%. Or, when the unexecuted queue length is above 1000 but below 2000, set the overload ratio to 20%; when it exceeds 2000, set the overload ratio to 50%, and so on.

[0344] Method 3: Set the overload ratio based on time window or traffic pattern.

[0345] In this method, the overload ratio is set based on time period (such as peak / off-peak period) or traffic characteristics (such as sudden traffic surges). During peak periods or sudden traffic surges, the overload status of service nodes is severe, so the overload ratio can be set higher. During off-peak periods, even if service nodes are overloaded, the overload is not very severe, so the overload ratio can be set lower.

[0346] For example, if it is known that the service node's peak period is between 8 PM and 10 PM daily, then the default overload ratio during 8 PM and 10 PM (peak traffic) is 15%, and the overload ratio at other times is 5%. As another example, when traffic suddenly increases by 200%, the overload ratio increases from 10% to 50% in stages.

[0347] It should be noted that the above-mentioned overload ratio setting method is just a simple example. Other setting methods are also applicable to the embodiments of this application, and will not be repeated here.

[0348] In this embodiment, this application discards unexecuted tasks based on the request priority and overload ratio of the request type at the request type dimension. This not only ensures that unexecuted tasks under the core request type can be executed stably, but also alleviates the overload state of service nodes and improves the user experience of the object.

[0349] This application takes into account that when discarding unexecuted tasks under a certain request type, if the number of unexecuted tasks to be discarded meets the overload ratio, unexecuted tasks in the unexecuted queue are continuously entering the unexecuted queue. Therefore, a certain method needs to be adopted for each unexecuted task so that each unexecuted task that has just entered the unexecuted queue can be discarded according to the overload ratio.

[0350] In an optional implementation, when discarding an unexecuted task under a corresponding request type in S34, the following operations are performed for each unexecuted task:

[0351] Generate a random number for an unexecuted task; if the proportion of the random number to the upper limit of the random number range is lower than the latest determined overload proportion corresponding to a request type, discard the unexecuted task.

[0352] The random number is within a preset range. To facilitate determining the current discard ratio, the random number can be set to a uniformly spaced positive integer, such as 1 to 100, 1 to 10000, etc.

[0353] Specifically, in this discarding method, a random number generator generates a random number for each unexecuted task that has just entered the unexecuted queue. If the ratio of this random number to the upper limit of the random number range is lower than the newly determined overload ratio, the unexecuted task is discarded.

[0354] like Figure 8 As shown, it is a schematic diagram of discarding unexecuted tasks provided in an embodiment of this application. Figure 8 The random number range is 1 to 100, with an upper limit of 100. Each random number is a positive integer. The overload ratio is 8%. Tasks 1, 2, 3, 4, 5, and 6 are sequentially added to the unexecuted queue. Assume that all tasks are unexecuted tasks of a request type that needs to be discarded, and that each unexecuted task generates a random number. Task 1's random number is 36, so since 36 / 100 > 0.08, Task 1 does not need to be discarded. Task 2's random number is 65, so since 65 / 100 > 0.08, Task 2 does not need to be discarded. Task 3's random number is 7, so since 7 / 100 < 0.08, Task 3 is discarded. Task 4's random number is 6, so since 6 / 100 < 0.08, Task 4 is discarded. Task 5's random number is 9, so since 9 / 100 > 0.08, Task 5 does not need to be discarded. Task 6's random number is 78, so since 78 / 100 > 0.08, Task 6 does not need to be discarded. The executed tasks can be seen in the queue as: Task 1, Task 2, Task 5, and Task 6.

[0355] like Figure 9 As shown, this is another schematic diagram of discarding unexecuted tasks provided in an embodiment of this application. Figure 9 The random number range is 1 to 100, with an upper limit of 100, and each random number is a positive integer. The overload ratio increases from 8% to 10% when task 3 enters the unexecuted queue. Tasks 1, 2, 3, 4, 5, and 6 are sequentially added to the unexecuted queue. Assume that all tasks are unexecuted tasks of a request type that needs to be discarded, and that each unexecuted task generates a random number. Task 1's random number is 36, so since 36 / 100 > 0.08, Task 1 does not need to be discarded. Task 2's random number is 65, so since 65 / 100 > 0.08, Task 2 does not need to be discarded. Task 3's random number is 7, so since 7 / 100 < 0.1, Task 3 is discarded. Task 4's random number is 6, so since 6 / 100 < 0.1, Task 4 is discarded. Task 5's random number is 9, so since 9 / 100 < 0.1, Task 5 is discarded. Task 6's random number is 78, so since 78 / 100 > 0.1, Task 6 does not need to be discarded. The executed tasks can be seen in the queue as: Task 1, Task 2, and Task 6.

[0356] In addition to generating random numbers to discard unexecuted tasks, this application can also discard them in the following ways:

[0357] Method 1: Use a sliding window to discard unexecuted tasks.

[0358] In the unexecuted queue, within a sliding window, unexecuted tasks are randomly discarded according to the overload ratio.

[0359] For example, if the overload ratio is 5% and the step size of the sliding window is 100, then for the unexecuted tasks within the sliding window, 100 * 5% = 5 unexecuted tasks will be randomly selected and discarded.

[0360] Method 2: Use hash bucket dropping to discard unexecuted tasks.

[0361] Unexecuted tasks are assigned to a fixed number of buckets based on their hash values, and all tasks in some buckets are directly disabled.

[0362] Hash 1000 tasks into 100 buckets. When the overload ratio is 20%, disable buckets 1-20, and all unexecuted tasks belonging to these buckets will be discarded.

[0363] In addition, for each request type, unexecuted tasks with an overload ratio can be randomly discarded.

[0364] It should be noted that the above-mentioned method of discarding unexecuted tasks is just a simple example. Other discarding methods are also applicable to the embodiments of this application, and will not be repeated here.

[0365] In the above embodiments, by generating random numbers for unexecuted tasks, this application can determine whether an unexecuted task needs to be discarded when an unexecuted person under each corresponding request type enters the unexecuted queue. This method can quickly discard unexecuted tasks without having to determine which unexecuted tasks need to be discarded after a certain number of unexecuted tasks have entered the unexecuted queue, thus avoiding further accumulation of unexecuted tasks carried by the service node.

[0366] In addition, this application defines a priority strategy that combines business and request dimensions based on attributes such as the importance of business type and request type. When a service node is overloaded, unexecuted tasks with high business priority or high request priority are retained, while unexecuted tasks with lower rate limiting priority are selectively discarded to ensure the stable execution of unexecuted tasks under core business type or core request type.

[0367] When a service node recovers to a normal state after being overloaded and unexecuted tasks are discarded according to the overload ratio, this application considers that if all subsequent unexecuted tasks are executed normally (i.e., the overload ratio is reduced to 0%), a large number of new unexecuted tasks may flood the unexecuted queue instantly, triggering the service node overload again, thereby overwhelming downstream services (such as databases and caches) and causing cascading failures. Therefore, this application adopts the following method to adjust the overload ratio.

[0368] In one optional implementation, when the service node returns to normal and S34 adjusts the overload ratio, the specific implementation method is as follows:

[0369] Every second preset time interval, the newly determined overload ratio is reduced based on the second preset step size, and at least one unexecuted task under the corresponding business type is discarded according to the reduced overload ratio.

[0370] The second preset duration can be the same as or different from the first preset duration. Similarly, the second preset step size can be the same as or different from the first preset step size.

[0371] The method for determining the second preset duration is the same as the method for determining the first preset duration, and the method for determining the second preset step size is the same as the method for determining the first preset duration, so they will not be repeated here.

[0372] Specifically, when a service node returns to a normal state, there may be unexecuted tasks under multiple discardable service types being discarded. Among these, the discardable service type with the latest determined overload ratio may be the one with the highest service priority among all discardable service types that are discarding unexecuted tasks. Therefore, this application prioritizes gradually reducing the latest determined overload ratio.

[0373] For example, there are 5 business types: Business Type 1, Business Type 2, Business Type 3, Business Type 4, and Business Type 5. Business Types 2, 3, and 4 are located in the set of business types to be discarded, and are ordered from lowest to highest priority as follows: Business Type 4, Business Type 3, Business Type 2. The preset overload ratio (the initial overload ratio before increase) is 1%. When the service node recovers to normal, the overload ratio of Business Type 4 has increased to 100%, or has increased to the preset overload ratio threshold, and the overload ratio of Business Type 3 has increased to 50%. That is, the latest determined overload ratio is 50% of Business Type 3. Assuming the second preset duration is 1 second and the second preset step size is 1%, this application reduces the overload ratio of Business Type 3 by 1% every 1 second, and after each reduction in overload ratio, discards unexecuted tasks under Business Type 3 based on the reduced overload ratio.

[0374] Similarly, if a business type contains multiple request types and there are unexecuted tasks under multiple request types that are being discarded, since the request type with the latest determined overload ratio may be the request type with the highest priority among all request types that are discarding unexecuted tasks, this application prioritizes gradually reducing the latest determined overload ratio.

[0375] For example, under a certain type of service to be discarded, there are 5 request types: Request Type 1, Request Type 2, Request Type 3, Request Type 4, and Request Type 5. The requests are ordered from lowest to highest priority as follows: Request Type 4, Request Type 3, Request Type 5, Request Type 2, and Request Type 1. The preset overload ratio (the initial overload ratio before increase) is 1%. When the service node recovers to normal, the overload ratio of Request Type 4 has increased to 100%, or has increased to the preset overload ratio threshold, and the overload ratio of Request Type 3 has increased to 50%. That is, the latest determined overload ratio is 50% of Request Type 3. Assuming the second preset duration is 1 second and the second preset step size is 1%, this application reduces the overload ratio of Request Type 3 by 1% every 1 second, and after each reduction in the overload ratio, discards the unexecuted tasks under Request Type 3 according to the reduced overload ratio.

[0376] In the above embodiments, after the service node returns to normal, the application gradually reduces the newly determined overload ratio, achieving a smooth transition between the stability of the service node and the availability of the task.

[0377] This application also considers that, after reducing the overload ratio of all request types under a certain type of business to be discarded to 0% according to the above implementation method, if the service node is still in a normal state, and all subsequent unexecuted tasks are executed normally (i.e., the overload level is reduced to the lowest overload level), it may also cause a large number of new unexecuted tasks to flood into the unexecuted queue instantly, triggering the service node overload again, thereby overwhelming downstream services (such as databases and caches) and causing cascading failures. Therefore, this application adopts the following method to adjust the overload ratio.

[0378] In one optional implementation, after the service node returns to normal and the overload ratio decreases to a preset ratio, when S34 adjusts the overload level, the specific implementation method is as follows:

[0379] Gradually reduce the overload level until it is reduced to the lowest service priority;

[0380] Each time the overload level is reduced, the following process is performed: based on the reduced overload level, a new set of business types to be discarded is determined from each business type; at least one unexecuted task associated with the latest set of business types to be discarded is discarded.

[0381] The preset ratio can be set to 0%, meaning that the corresponding business type to be discarded will no longer discard unexecuted tasks, or it can be set to a lower ratio, such as 1% or 2%.

[0382] Specifically, if the overload ratio of a business type to be discarded with the same business priority as the current overload level has been reduced to a preset ratio, and the service node is still in a normal state, then this application can reduce the overload ratio so that business types with lower business priority can also start to increase their overload ratio.

[0383] For example, there are 5 business types: Business Type 1, Business Type 2, Business Type 3, Business Type 4, and Business Type 5. The preset overload ratio (the initial overload ratio before any increase) is 1%. The business types are sorted from lowest to highest priority as follows: Business Type 1 (Level 1), Business Type 5 (Level 2), Business Type 4 (Level 3), Business Type 3 (Level 4), and Business Type 2 (Level 5). Since the initial overload level is Level 3, Business Type 1, Business Type 5, and Business Type 4 are designated as business types to be discarded and assigned to the discard set. At this point, the overload ratio of Business Type 4 is gradually reduced. When the overload ratio of Business Type 4 drops to 0% or the preset ratio, the overload ratio is reduced to Level 2, and the overload ratio of Business Type 5 is gradually reduced.

[0384] Similarly, if a business type contains multiple request types and there are multiple request types whose unexecuted tasks are being discarded, the overload ratio of the request type with the highest priority among the multiple request types whose unexecuted tasks are being discarded can be reduced first. When the overload ratio of this request type is reduced to 0% or a preset ratio, the overload ratio of the request type with lower priority can be reduced.

[0385] For example, a certain type of service to be discarded contains 5 request types: Request Type 1, Request Type 2, Request Type 3, Request Type 4, and Request Type 5. The preset overload ratio (the initial overload ratio before increase) is 1%. The requests are ordered from lowest to highest priority as follows: Request Type 1, Request Type 5, Request Type 4, Request Type 3, Request Type 2. Request Types 1, 5, and 4 are currently discarding unexecuted requests. In this case, the overload ratio of Request Type 4 is gradually reduced first. When the overload ratio of Request Type 4 drops to 0% or the preset ratio, the overload ratio of Request Type 5 is then gradually reduced.

[0386] In the above implementation, after the service node returns to normal, the application gradually reduces the newly determined overload ratio from high to low according to the business priority or request priority, so as to achieve a smooth transition between the stability of the service node and the availability of the task.

[0387] Optionally, the steps in S34 above can be executed by the decision execution module.

[0388] The following is combined Figure 10 and Figure 11 This application provides an overall overview of a service overload handling method based on an embodiment.

[0389] like Figure 10 As shown, it is a schematic diagram of an overload strategy information provided in an embodiment of this application. Figure 10 It includes three business types: finance, video-on-demand, and live streaming. Each business type contains multiple request types, and Figure 10 The request types are arranged from left to right according to their priority. For example, within the financial category, payment requests have the highest priority, followed by refund requests, and query requests have the lowest priority. The available resources for financial requests account for less than 50%, for on-demand requests less than 25%, and for live streaming requests less than 25%.

[0390] Assuming that when the service node is overloaded, the total resource consumption of each business type does not exceed its respective proportion of available resources, the business types to be discarded are determined according to the overload level. Assuming the preset overload level is 10, the live streaming type is selected as the business type to be discarded. Since there are two request types under the live streaming type, the download request with the lowest priority is prioritized. Unexecuted tasks are discarded according to a preset overload ratio of 1%, and the overload ratio is increased by 1% every second until the download request overload ratio reaches 100%. If the service node is still overloaded, then unexecuted tasks of upload requests are discarded according to the preset overload ratio of 1%, and the overload ratio is increased by 1% every second until the upload request overload ratio reaches 100%. If the service node is still overloaded, then the overload level is increased to 30, and upload requests under the video-on-demand service are discarded in the same way.

[0391] Suppose that when the overload ratio of upload requests under the video-on-demand service increases to 50%, the service node returns to normal. At this point, the overload ratio is reduced by 1% every 1 second until the overload ratio of upload requests under the video-on-demand service decreases to 0%, at which point the service node is still in normal operation. Then, the overload level is reduced to 10. For upload requests under the live streaming service, the overload ratio is also reduced by 1% every 1 second until the overload ratio of upload requests decreases to 0%, at which point the service node is still in normal operation. Then, the overload ratio of download requests under the live streaming service is reduced in the same way.

[0392] If the service node is in an overloaded state when the overload ratio of download requests under the live streaming service is reduced from 100% to 15%, then the above process of increasing the overload ratio and increasing the overload level will be repeated until the service node is in a normal state.

[0393] like Figure 11 As shown, it is a schematic diagram of an implementation of a service overload handling method provided in an embodiment of this application. Figure 11 Continue Figure 10 The initial assumption is that the service node contains three business types: finance, video-on-demand, and live streaming. Each business type contains multiple request types. When the service node is overloaded, the total resource consumption of each business type does not exceed its respective proportion of available resources. Therefore, the business types to be discarded are determined according to the overload level. Assuming the preset overload level is 10, unexecuted download requests under the live streaming business are discarded first. When the overload ratio of download requests under the live streaming business increases to 100%, unexecuted upload requests under the live streaming business begin to be discarded. When the overload of upload requests under the live streaming business increases to 100%, the overload level is increased to 30, and upload requests under the video-on-demand business are discarded in the same way. Suppose that when the overload ratio of upload requests under the video-on-demand service increases to 50%, the service node returns to normal. When the overload ratio of upload requests under the video-on-demand service decreases to 0%, the overload ratio of download requests under the live streaming service begins to decrease. When the overload of download requests under the live streaming service decreases to 0%, the overload level is reduced to 10, and the overload ratio of download requests under the live streaming service is reduced in the same way.

[0394] Based on the same inventive concept, embodiments of this application also provide a service overload processing device. For example... Figure 12 As shown, this is a schematic diagram of the service overload processing device, which may include:

[0395] The acquisition unit 1201 is used to acquire the execution time of each executed task on the service node when the preset triggering conditions are met.

[0396] The execution unit 1202 is configured to perform the following for at least one node task carried by the service node: according to the task execution order of the service node, based on the execution time of at least one previously executed task, obtain the waiting time of a node task; wherein a node task is either an executed task or an unexecuted task.

[0397] The determining unit 1203 is used to determine a set of service types to be discarded based on the overload policy information of each service type on the service node when the service node is determined to be in an overload state based on at least one obtained waiting time; wherein the overload policy information includes at least one of the following: total resource consumption and service priority; the total resource consumption is the sum of the resource consumption of at least one executed task under the corresponding service type;

[0398] The discarding unit 1204 is used to discard at least one unexecuted task associated with the set of business types to be discarded.

[0399] Optionally, the overload strategy information includes: total resource consumption and service priority; the determination unit 1203 is specifically used for:

[0400] For each business type, if the total resource consumption of a business type exceeds the corresponding resource consumption limit for that business type, the business type will be designated as a business type to be discarded and added to the set of business types to be discarded.

[0401] When the resource consumption of each business type on the service node is lower than the corresponding resource consumption limit, the business types whose business priority does not exceed the preset overload level are designated as business types to be discarded and added to the set of business types to be discarded.

[0402] Optionally, the executed tasks carried by the service node are located in the executed queue, and the upper limit of resource consumption is the proportion of available resources; the determining unit 1203 is specifically used for:

[0403] Determine the ratio of the total resource consumption of a business type to the sum of the total resource consumption of each business type; where the total resource consumption is the sum of the resource consumption of each reference task under the corresponding business type; each reference task is an executed task whose execution order is within a preset range.

[0404] When the ratio exceeds the available resource ratio corresponding to a business type, that business type will be designated as a business type to be discarded.

[0405] Optionally, the total resource consumption for each business type is dynamically updated as the number of executed tasks carried by the service node changes; the discard unit 1204 is specifically used for:

[0406] When there are discardable service types whose total resource consumption exceeds the corresponding resource consumption limit, for each discardable service type, based on the preset overload ratio, discard the unexecuted tasks under the discardable service type until the service node returns to normal, or until the latest total resource consumption of a discardable service type is lower than the corresponding resource consumption limit.

[0407] When the total resource consumption of each type of business to be discarded is lower than the corresponding resource consumption limit, and the service node is still overloaded, the unexecuted tasks under the corresponding business type to be discarded are discarded in sequence according to the business priority of each type of business to be discarded in the set of business types to be discarded from low to high, and the preset overload ratio, until the service node returns to normal.

[0408] Optionally, based on a preset overload ratio, during the process of discarding unexecuted tasks under a certain type of service to be discarded, the discarding unit 1204 is specifically used for:

[0409] Every first preset time interval, the overload ratio is increased based on the first preset step size.

[0410] Optionally, when the set of unexecuted tasks associated with the set of services to be discarded is empty, and the service node is still in an overloaded state, the discard unit 1204 is also used to:

[0411] The overload level is gradually increased until the service node returns to normal. After each increase in the overload level, the following process is executed:

[0412] Based on the improved overload level, the set of business types to be discarded is redefined from those business types that were never added to the set of business types to be discarded.

[0413] Discard at least one unexecuted task associated with the latest set of pending discard business types.

[0414] Optionally, the device further includes an adjustment unit 1205, which is used for:

[0415] After the service node returns to normal, every second preset time interval, the newly determined overload ratio is reduced based on the second preset step size, and at least one unexecuted task under the corresponding business type is discarded according to the reduced overload ratio.

[0416] Optionally, after the overload ratio is reduced to a preset ratio, the adjustment unit 1205 is also used for:

[0417] The overload level is gradually reduced until it reaches the lowest service priority; after each reduction in overload level, the following process is executed:

[0418] Based on the reduced overload level, the set of service types to be discarded is re-determined from each service type.

[0419] Discard at least one unexecuted task associated with the latest set of pending discard business types.

[0420] Optionally, each business type contains multiple request types, and each request type corresponds to a request priority; then, the discarding unit 1204 discards unexecuted tasks under a business type to be discarded in the following manner:

[0421] For a business type to be discarded, based on the corresponding request priority from low to high and the preset overload ratio, the unexecuted tasks under the corresponding request type are discarded in sequence until the service node returns to normal, or until the unexecuted tasks under a business type to be discarded are empty.

[0422] Optionally, during the process of discarding unexecuted tasks under a request type based on a preset overload ratio, the discarding unit 1204 is also used to:

[0423] Every third preset time interval, the overload ratio is increased based on the third preset step size.

[0424] Optionally, based on a preset overload ratio, unexecuted tasks under a request type are discarded. The discarding unit 1204 is specifically used for:

[0425] For each unexecuted task under a given request type, perform the following operations:

[0426] Generate a random number for an unexecuted task; the random number must fall within a preset range.

[0427] When the proportion of random numbers to the upper limit of the random number range is lower than the latest determined overload proportion corresponding to a request type, an unexecuted task is discarded.

[0428] Optionally, the determining unit 1203 is specifically used for:

[0429] Determine the current processing latency of the service node based on at least one waiting time;

[0430] When the current processing latency exceeds a preset overload threshold, the service node is determined to be in an overload state.

[0431] Optionally, the unexecuted tasks carried by the service node are located in the unexecuted queue; the determining unit 1203 determines the current processing delay of the service node based on at least one waiting time, including at least one of the following methods:

[0432] The average waiting time of multiple executed tasks is used as the current processing delay;

[0433] The waiting time of the unexecuted tasks used for probing is taken as the current processing delay; where the unexecuted tasks used for probing are: unexecuted tasks added to the tail of the unexecuted queue.

[0434] Optionally, before determining that the service node is in an overloaded state, the determining unit 1203 is further configured to:

[0435] Based on current processing latency and historical latency data, the updated current processing latency is predicted.

[0436] When the current processing latency exceeds a preset overload threshold, the service node is determined to be in an overload state, including:

[0437] If the updated current processing latency exceeds a preset overload threshold, the service node is determined to be in an overload state.

[0438] Optionally, the executed tasks carried by the service node are located in the executed queue; the determining unit 1203 is specifically used to determine and update the total resource consumption of each business type in the following manner:

[0439] Using a sliding window with a fourth preset step size, perform the following operations dynamically:

[0440] By using a sliding window, select a corresponding number of executed tasks from the executed queue as reference tasks;

[0441] For each business type, the total resource consumption of all reference tasks belonging to a business type is taken as the current total resource consumption of a business type.

[0442] This application provides a service overload handling method, apparatus, electronic device, and storage medium. Adaptive overload control is achieved through multi-dimensional execution time statistics, dynamic latency monitoring, and priority strategies. It is suitable for scenarios such as edge computing where services are latency-sensitive, multi-tenant, and resource specifications are inconsistent, maximizing the utilization of computing resources while ensuring stable service latency. This solves the problem of difficulty in large-scale stable operation through manual threshold setting and adjustment, as well as issues such as poor adaptability, lack of priority, and delayed response in related technologies.

[0443] Specifically, when a preset trigger condition is met, this application first obtains the execution time of each executed task on the service node. By setting trigger conditions and dynamically executing the service overload handling method of this application, meaningless data noise can be avoided, and additional performance loss caused by high-frequency data acquisition can be reduced.

[0444] Furthermore, based on the execution time of at least one previously executed task preceding a node task, the waiting time of a node task is obtained. Then, based on the obtained at least one waiting time, it is predicted that the service node is in an overloaded state. By dynamically detecting the current processing latency of the service node and combining the sliding window algorithm and exponential smoothing prediction, this application can quickly respond to sudden traffic and latency fluctuations, adjust the overload strategy in a timely manner, dynamically adjust the overload threshold, and adapt to different hardware performance and load changes.

[0445] Finally, this application determines the set of business types to be discarded based on the business priority and total resource consumption of each business type on the service node, and discards at least one unexecuted task associated with the set of business types to be discarded. By introducing resource consumption, it is possible to identify service node overload problems caused by single or a few business bursts and to identify the business types to be discarded due to burst traffic. At the same time, this application introduces a priority strategy, which selectively discards or downgrades low-priority requests when the service node is overloaded, based on the priority of request type and business type, to ensure the stable processing of unexecuted tasks under core business types or core request types.

[0446] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

[0447] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0448] Having introduced the service overload processing method and apparatus according to exemplary embodiments of this application, we will now introduce an electronic device according to another exemplary embodiment of this application.

[0449] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0450] Based on the same inventive concept as the above-described method embodiments, this application also provides an electronic device. In one embodiment, the electronic device may be a server, such as... Figure 1 The server 120 is shown. In this embodiment, the electronic device can be structured as follows: Figure 13 As shown, it includes a memory 1301, a communication module 1303, and one or more processors 1302.

[0451] The memory 1301 is used to store computer programs executed by the processor 1302. The memory 1301 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0452] Memory 1301 may be volatile memory, such as random-access memory (RAM); memory 1301 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1301 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1301 may be a combination of the above-described memories.

[0453] Processor 1302 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 1302 is used to implement the above-mentioned service overload processing method when calling computer programs stored in memory 1301.

[0454] The communication module 1303 is used to communicate with terminal devices and other servers.

[0455] This application embodiment does not limit the specific connection medium between the memory 1301, communication module 1303, and processor 1302. This application embodiment... Figure 13 The memory 1301 and the processor 1302 are connected via a bus 1304, and the bus 1304 is in Figure 13 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 1304 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 13It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0456] The memory 1301 stores a computer storage medium, which stores computer-executable instructions for implementing the service overload processing method of this application embodiment. The processor 1302 is used to execute the above-described service overload processing method, such as... Figure 3 As shown.

[0457] In another embodiment, the electronic device may also be other electronic devices, such as... Figure 1 The terminal device 110 is shown. In this embodiment, the electronic device can be structured as follows: Figure 14 As shown, it includes components such as: communication component 1410, memory 1420, display unit 1430, camera 1440, sensor 1450, audio circuit 1460, Bluetooth module 1470, processor 1480, etc.

[0458] The communication component 1410 is used to communicate with the server. In some embodiments, it may include a Circuit-Wireless Fidelity (WiFi) module, which is a short-range wireless transmission technology, and the electronic device can help the user send and receive information through the WiFi module.

[0459] The memory 1420 can be used to store software programs and data. The processor 1480 executes various functions of the terminal device 110 and performs data processing by running the software programs or data stored in the memory 1420. The memory 1420 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 1420 stores an operating system that enables the terminal device 110 to run. In this application, the memory 1420 may store the operating system and various application programs, and may also store computer programs that execute the service overload processing method of the embodiments of this application.

[0460] The display unit 1430 can also be used to display information input by the user or information provided to the user, as well as various menus of the terminal device 110, forming a graphical user interface (GUI). Specifically, the display unit 1430 may include a display screen 1432 disposed on the front of the terminal device 110. The display screen 1432 may be configured as a liquid crystal display, a light-emitting diode, or the like. The display unit 1430 can be used to display the online shopping platform interface, etc., as described in this application embodiment.

[0461] The display unit 1430 can also be used to receive input digital or character information and generate signal inputs related to user settings and function control of the terminal device 110. Specifically, the display unit 1430 may include a touch screen 1431 disposed on the front of the terminal device 110, which can collect touch operations of the user on or near it, such as clicking buttons, dragging scroll boxes, etc.

[0462] The touchscreen 1431 can be placed on top of the display screen 1432, or the touchscreen 1431 and the display screen 1432 can be integrated to realize the input and output functions of the terminal device 110. After integration, it can be referred to as a touch display screen. In this application, the display unit 1430 can display the application and the corresponding operation steps.

[0463] Camera 1440 can be used to capture still images, which users can then share via an application. There can be one or multiple cameras 1440. An object is projected onto a photosensitive element through a lens, generating an optical image. This photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the processor 1480 for conversion into a digital image signal.

[0464] The terminal device may also include at least one sensor 1450, such as an accelerometer 1451, a proximity sensor 1452, a fingerprint sensor 1453, and a temperature sensor 1454. The terminal device may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, and motion sensor.

[0465] Audio circuitry 1460, speaker 1461, and microphone 1462 provide an audio interface between the user and terminal device 110. Audio circuitry 1460 converts received audio data into electrical signals, which are then transmitted to speaker 1461, where they are converted into sound signals for output. Terminal device 110 may also be equipped with volume buttons for adjusting the volume of the sound signal. Conversely, microphone 1462 converts collected sound signals into electrical signals, which are then received by audio circuitry 1460, converted back into audio data, and output to communication component 1410 for transmission to, for example, another terminal device 110, or to memory 1420 for further processing.

[0466] The Bluetooth module 1470 is used to interact with other Bluetooth devices that also have a Bluetooth module via the Bluetooth protocol. For example, a terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smartwatch) that also has a Bluetooth module through the Bluetooth module 1470, thereby exchanging data.

[0467] The processor 1480 is the control center of the terminal device, connecting various parts of the terminal through various interfaces and lines. It executes various functions and processes data by running or executing software programs stored in the memory 1420 and calling data stored in the memory 1420. In some embodiments, the processor 1480 may include one or more processing units; the processor 1480 may also integrate an application processor and a baseband processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the baseband processor mainly handles wireless communication. It is understood that the baseband processor may not be integrated into the processor 1480. In this application, the processor 1480 can run the operating system, applications, user interface display and touch response, as well as the service overload processing method of this embodiment. Furthermore, the processor 1480 is coupled to the display unit 1430.

[0468] In some possible implementations, various aspects of the service overload handling method provided in this application can also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program causes the electronic device to perform the steps of the service overload handling method according to the various exemplary embodiments of this application described above. For example, the electronic device can perform actions such as... Figure 3 The steps are shown in the figure.

[0469] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0470] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0471] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0472] Computer programs contained on readable media may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0473] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The computer program can execute entirely on the user's electronic device, partially on the user's electronic device, as a standalone software package, partially on the user's electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user's electronic device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0474] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0475] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0476] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing a computer-usable computer program.

[0477] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0478] These computer program commands may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the commands stored in the computer-readable storage medium produce an article of manufacture including command means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0479] These computer program commands can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing the commands executed on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0480] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0481] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A service overload handling method, characterized by, The method includes: When the preset triggering conditions are met, obtain the execution time of each executed task on the service node; For at least one node task carried by the service node, the following steps are performed: according to the task execution order of the service node, based on the execution time of at least one previously executed task, the waiting time of the node task is obtained; wherein, the node task is either an executed task or an unexecuted task. When a service node is determined to be in an overload state based on at least one obtained waiting time, a set of service types to be discarded is determined based on the overload policy information of each service type on the service node; wherein, the overload policy information includes at least one of the following: total resource consumption and service priority; the total resource consumption is the sum of the resource consumption of at least one executed task under the corresponding service type; At least one unexecuted task associated with the set of service types to be discarded is discarded.

2. The method of claim 1, wherein, The overload policy information includes: total resource consumption and service priority; determining the set of service types to be discarded based on the overload policy information of each service type on the service node includes: For each business type, if the total resource consumption of a business type exceeds the corresponding resource consumption limit for that business type, that business type will be designated as a business type to be discarded and added to the set of business types to be discarded. When the resource consumption of each service type on the service node is lower than the corresponding resource consumption limit, the service type whose service priority does not exceed the preset overload level is designated as the service type to be discarded and added to the set of service types to be discarded.

3. The method of claim 2, wherein, The executed tasks carried by the service node are located in the executed queue, and the resource consumption limit is the percentage of available resources; when the total resource consumption of a certain business type exceeds the resource consumption limit corresponding to that business type, that business type will be designated as a business type to be discarded, including: Determine the ratio of the total resource consumption of a certain business type to the sum of the total resource consumption of each of the business types; wherein, the total resource consumption is the sum of the resource consumption of each reference task under the corresponding business type; each reference task is an executed task whose execution order is within a preset range. When the ratio exceeds the proportion of available resources corresponding to the business type, the business type will be designated as a business type to be discarded.

4. The method of claim 2, wherein, The total resource consumption for each service type is dynamically updated as the executed tasks carried by the service node change; discarding at least one unexecuted task associated with the set of service types to be discarded includes: When there are discardable service types whose total resource consumption exceeds the corresponding resource consumption limit, for each discardable service type, based on a preset overload ratio, unexecuted tasks under a discardable service type are discarded until the service node returns to normal, or until the latest total resource consumption corresponding to a discardable service type is lower than the corresponding resource consumption limit. When the total resource consumption of each type of service to be discarded is lower than the corresponding resource consumption limit, and the service node is still overloaded, based on the order of service priority of each type of service to be discarded from low to high and the preset overload ratio, the unexecuted tasks under the corresponding type of service to be discarded are discarded in sequence until the service node returns to normal.

5. The method of claim 4, wherein, Based on a preset overload ratio, during the process of discarding unexecuted tasks under a certain type of service to be discarded, the method further includes: The overload ratio is increased every first preset time interval based on a first preset step size.

6. The method of claim 4, wherein, When the set of unexecuted tasks associated with the set of services to be discarded is empty, and the service node is still in an overloaded state, the method further includes: The overload level is gradually increased until the service node returns to normal; wherein, after each increase in the overload level, the following process is performed: Based on the improved overload level, the set of service types to be discarded is redefined from those service types that were never added to the set of service types to be discarded. Discard at least one unexecuted task associated with the latest set of pending discard business types.

7. The method of claim 4, wherein, After the service node returns to normal, the method further includes: Every second preset time interval, the newly determined overload ratio is reduced based on the second preset step size, and at least one unexecuted task under the corresponding business type is discarded according to the reduced overload ratio.

8. The method of claim 7, wherein, After the overload ratio is reduced to a preset ratio, the method further includes: The overload level is gradually reduced until it reaches the lowest service priority; after each reduction in the overload level, the following process is performed: Based on the reduced overload level, the set of service types to be discarded is re-determined from each service type; Discard at least one unexecuted task associated with the latest set of pending discard business types.

9. The method according to any one of claims 1 to 8, characterized in that, Each business type contains multiple request types, and each request type corresponds to a request priority; therefore, unexecuted tasks under a business type to be discarded are discarded in the following way: For a service type to be discarded, based on the corresponding request priority from low to high and the preset overload ratio, the unexecuted tasks under the corresponding request type are discarded in sequence until the service node returns to normal, or until the unexecuted tasks under the service type to be discarded are empty.

10. The method of claim 9, wherein, Based on a preset overload ratio, during the process of discarding unexecuted tasks under a request type, the method further includes: Every third preset time interval, the overload ratio is increased based on the third preset step size.

11. The method of claim 10, wherein, Based on a preset overload ratio, unexecuted tasks under a certain request type are discarded, including: For each unexecuted task under the aforementioned corresponding request type, perform the following operations: Generate a random number for an unexecuted task; the random number belongs to a preset random number range; When the proportion of the random number to the upper limit of the random number range is lower than the latest determined overload proportion corresponding to the request type, the unexecuted task is discarded.

12. The method according to any one of claims 1 to 8, wherein The step of determining that the service node is in an overload state based on at least one obtained waiting time includes: The current processing delay of the service node is determined based on the at least one of the waiting times. When the current processing delay exceeds a preset overload threshold, the service node is determined to be in an overload state.

13. The method of claim 12, wherein, The unexecuted tasks carried by the service node are located in the unexecuted queue; the current processing delay of the service node is determined based on the at least one waiting time, including at least one of the following methods: The average waiting time of multiple executed tasks is used as the current processing delay; The waiting time of the unexecuted tasks used for detection is taken as the current processing delay; wherein the unexecuted tasks used for detection are unexecuted tasks added to the tail of the unexecuted queue.

14. The method of claim 12, wherein, Before determining that the service node is in an overloaded state, the method further includes: Based on the current processing latency and historical latency data, the updated current processing latency is predicted. The step of determining that the service node is in an overload state when the current processing latency exceeds a preset overload threshold includes: When the updated current processing latency exceeds a preset overload threshold, the service node is determined to be in an overload state.

15. The method according to any one of claims 1 to 8, wherein The executed tasks carried by the service node are located in the executed queue; the total resource consumption for each business type is determined and updated in the following way: Using a sliding window with a fourth preset step size, perform the following operations dynamically: Using the sliding window, a corresponding number of executed tasks are selected from the executed queue as reference tasks; For each business type, the sum of the resource consumption of all reference tasks belonging to a business type in the reference tasks is taken as the current total resource consumption of the business type.

16. A service overload handling apparatus, characterized by The device includes: The acquisition unit is used to acquire the execution time of each executed task on the service node when the preset triggering conditions are met. An execution unit is configured to perform the following for at least one node task carried by the service node: according to the task execution order of the service node, and based on the execution time of at least one previously executed task, obtain the waiting time of the node task; wherein, the node task is either an executed task or an unexecuted task. The determining unit is configured to determine a set of service types to be discarded based on the overload policy information of each service type on the service node when the service node is determined to be in an overload state based on at least one obtained waiting time; wherein the overload policy information includes at least one of the following: total resource consumption and service priority; the total resource consumption is the sum of the resource consumption of at least one executed task under the corresponding service type; The discarding unit is used to discard at least one unexecuted task associated with the set of service types to be discarded.

17. An electronic device, comprising: It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of any of the methods described in claims 1 to 15.

18. A computer-readable storage medium, characterized in that, It includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of any of the methods described in claims 1 to 15.

19. A computer program product, characterised in that, The method includes a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of any one of claims 1 to 15.