Task processing method and electronic device

Abstract

This application provides a task processing method and an electronic device that can be applied to the field of server technology. The task processing method includes: in response to receiving a task request, determining load assessment information characterizing the load distribution among multiple server nodes based on multiple indicator information of multiple server nodes and resource consumption information required to execute the task to be processed in the task request; the multiple indicator information being used to indicate the configuration and status information of each server node; and selecting a target strategy from multiple candidate strategies based on the load assessment information. The multiple candidate strategies include a first strategy using the number of connections between each server node and the task as the node selection criterion, a second strategy using the response time of each server node as the node selection criterion, and a third strategy using the expected processing latency of the task to be processed allocated to each server node as the node selection criterion.

Description

Technical Field

[0001] This application relates to the field of server technology, and more specifically, to a task processing method and an electronic device. Background Technology

[0002] Server system load capacity is a crucial indicator of system performance. To address issues such as uneven server load, slow response times, and low resource utilization in complex data processing scenarios involving high concurrency and large datasets, relevant technologies primarily rely on fixed rules or hardware-based task distribution. However, these technologies lack adaptability and flexibility for complex scenarios, making them insufficient to meet actual business needs. Summary of the Invention

[0003] In view of this, this application provides a task processing method and an electronic device.

[0004] One aspect of this application provides a task processing method, comprising: in response to receiving a task request, determining load assessment information characterizing the load distribution among multiple server nodes based on multiple indicator information of multiple server nodes and resource consumption information required to execute the task to be processed in the task request, wherein the multiple indicator information is used to indicate the configuration information and status information of each server node; selecting a target strategy from multiple candidate strategies based on the load assessment information, wherein the multiple candidate strategies include a first strategy based on the number of connections between each server node and the task as the node selection criterion, a second strategy based on the response time of each server node as the node selection criterion, and a third strategy based on the expected processing latency of the task to be processed being allocated to each server node as the node selection criterion; determining a target node from the multiple server nodes based on the target strategy, and allocating the task to be processed to the target node for task processing by the target node.

[0005] One aspect of this application provides a task processing method, comprising: in response to receiving a task request, determining load assessment information characterizing the load distribution among multiple server nodes based on multiple indicator information of multiple server nodes and resource consumption information required to execute the task to be processed in the task request, wherein the multiple indicator information is used to indicate the configuration information and status information of each server node; selecting a target strategy from multiple candidate strategies based on the load assessment information, wherein the multiple candidate strategies include a first strategy based on the number of connections between each server node and the task as the node selection criterion, a second strategy based on the response time of each server node as the node selection criterion, and a third strategy based on the expected processing latency of the task to be processed being allocated to each server node as the node selection criterion; determining a target node from the multiple server nodes based on the target strategy, and allocating the task to be processed to the target node for task processing by the target node.

[0006] Another aspect of this application provides a task processing apparatus, comprising: a determining module, configured to, in response to receiving a task request, determine load assessment information of each server node based on resource consumption information required to execute the task to be processed in the task request, and multiple indicator information indicating configuration information and status information of multiple server nodes; a selecting module, configured to select a target strategy from multiple candidate strategies based on the load assessment information, the multiple candidate strategies including a first strategy using the number of connections between each server node and the task as the node selection criterion, a second strategy using the response time of each server node as the node selection criterion, and a third strategy using the expected processing latency of the task to be processed being allocated to each server node as the node selection criterion; and an allocating module, configured to determine a target node from the multiple server nodes based on the target strategy, and allocate the task to be processed to the target node for task processing by the target node.

[0007] Another aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described above.

[0008] Another aspect of this application provides a computer-readable storage medium storing computer-executable instructions that, when executed, are used to implement the method described above.

[0009] Another aspect of this application provides a computer program product comprising computer-executable instructions which, when executed, are used to implement the method described above.

[0010] According to embodiments of this application, by combining the configuration information (e.g., static resources such as CPU, memory, and storage) and status information (e.g., dynamic indicators such as current load, number of connections, and response time) of multiple server nodes with the resource consumption information of the tasks to be processed, the adaptability between each node and the task is comprehensively evaluated from multiple dimensions. This achieves a precise quantitative evaluation of the processing capacity of the server nodes, thereby solving the problems of low resource utilization and inaccurate load evaluation, and avoiding load imbalance between nodes. Furthermore, based on the load distribution among multiple server nodes, a target strategy matching the current load distribution is dynamically selected from multiple candidate strategies, including the number of connections, response time, and predicted latency. This allows the load balancing process to flexibly adapt to real-time conditions and task requests with different characteristics, solving the problem of lack of intelligence and adaptability in traditional solutions. The above adaptive load balancing process is implemented through software logic, replacing expensive dedicated hardware devices with a software-defined approach. This achieves dynamic and accurate task scheduling while reducing costs and improving the flexibility of system configuration. Attached Figure Description

[0011] The above and other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0012] Figure 1 This illustration schematically depicts an application scenario of a task processing method and an electronic device according to embodiments of this application.

[0013] Figure 2 A flowchart illustrating a task processing method according to an embodiment of this application is shown schematically.

[0014] Figure 3 A flowchart illustrating a task processing method according to another embodiment of this application is shown schematically;

[0015] Figure 4 This schematic diagram illustrates a structural block diagram of a task processing apparatus according to an embodiment of the present application;

[0016] Figure 5 A block diagram of an electronic device suitable for implementing the methods described above, according to an embodiment of this application, is illustrated schematically. Detailed Implementation

[0017] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0018] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0019] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0020] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0021] In some examples, request distribution is based on fixed rules or strategies, such as round-robin or least connections. However, fixed rules or strategies lack intelligence and are difficult to dynamically adjust according to the server's real-time load, network conditions, and user request characteristics. Because server load cannot be accurately assessed, some servers may be overloaded while others are idle, resulting in low resource utilization and difficulty in adapting to complex and ever-changing network environments and business needs.

[0022] In some examples, task distribution and forwarding are achieved through dedicated hardware devices. However, hardware load balancers are expensive and unsuitable for small-scale applications; they are also often difficult to customize and upgrade, and pose a risk of becoming a single point of failure in the system.

[0023] In view of this, embodiments of this application provide a task processing method and an electronic device. The method includes: in response to receiving a task request, determining load assessment information characterizing the load distribution among multiple server nodes based on multiple indicator information of multiple server nodes and resource consumption information required to execute the task to be processed in the task request; selecting a target strategy from multiple candidate strategies based on the load assessment information; the multiple candidate strategies include a first strategy using the number of connections between each server node and the task as the node selection criterion, a second strategy using the response time of each server node as the node selection criterion, and a third strategy using the expected processing latency of the task to be processed allocated to each server node as the node selection criterion; determining a target node from the multiple server nodes based on the target strategy, and allocating the task to be processed to the target node for task processing by the target node.

[0024] According to embodiments of this application, by combining the configuration information (e.g., static resources such as CPU, memory, and storage) and status information (e.g., dynamic indicators such as current load, number of connections, and response time) of multiple server nodes with the resource consumption information of the tasks to be processed, the adaptability between each node and the task is comprehensively evaluated from multiple dimensions. This achieves a precise quantitative evaluation of the processing capacity of the server nodes, thereby solving the problems of low resource utilization and inaccurate load evaluation, and avoiding load imbalance between nodes. Furthermore, based on the load distribution among multiple server nodes, a target strategy matching the current load distribution is dynamically selected from multiple candidate strategies, including the number of connections, response time, and predicted latency. This allows the load balancing process to flexibly adapt to real-time conditions and task requests with different characteristics, solving the problem of lack of intelligence and adaptability in traditional solutions. The above adaptive load balancing process is implemented through software logic, replacing expensive dedicated hardware devices with a software-defined approach. This achieves dynamic and accurate task scheduling while reducing costs and improving the flexibility of system configuration.

[0025] Figure 1 The illustration shows an application scenario diagram of a task processing method and an electronic device according to embodiments of this application.

[0026] like Figure 1 As shown, application scenario 100 according to this embodiment may include terminal device 110, network 120, server 130, and backend server node pool 140. Network 120 is used as a medium to provide a communication link between terminal device 110, server 130, and backend server node pool 140. Network 120 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0027] Users can use terminal device 110 to interact with server 130 via network 120 to receive or send messages, etc.

[0028] There can be multiple terminal devices 110, which are various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers. Terminal devices 110 can act as initiators of task requests. Applications running on each terminal device (such as web browsers or mobile applications) generate and issue task requests when backend services are needed.

[0029] For example, there can be three terminal devices 110, such as a first terminal device, a second terminal device, and a third terminal device. The first terminal device initiates an online payment request, the second terminal device initiates a video transcoding request, and the third terminal device initiates a database query request. The time when multiple terminal devices initiate task requests can be the same or different.

[0030] Server 130 can be a processor providing various services and a load balancer for task processing. Server 130 acts as the executor of intelligent adaptive candidate strategies. Its core function is to receive, analyze, schedule, and forward requests.

[0031] For example, server 130 receives task requests from terminal device 110 via network 120, maintains a backend server node pool 140, and continuously collects multiple metrics (such as CPU, memory, number of connections, response time, etc.) of each node in the backend server node pool 140. When a new task request is received, the load assessment information of each server node can be determined based on the resource consumption information of the task to be processed in the task request (such as estimated computing and memory requirements) and the real-time metrics of each node.

[0032] The backend server node pool 140 can be a resource pool or cluster composed of multiple server nodes (e.g., server node 1, server node 2, and server node 3), serving as the resource entity for executing "tasks to be processed," such as business computation and data processing. Each server node runs services independently and sends its own status indicators to server 130 via network 120. For example, it reports its configuration and status information to server 130 via network 120; it receives task requests allocated by server 130 via network 120 and executes the tasks to be processed within them. After the task is completed, the result can be returned to server 130 via network 120.

[0033] It should be noted that the task processing method provided in this application embodiment can generally be executed by server 130. Correspondingly, the task processing device provided in this application embodiment can generally be located in server 130. The task processing method provided in this application embodiment can also be executed by a server or server cluster that is different from server 130 and capable of communicating with terminal device 110 and / or server 130. Correspondingly, the task processing device provided in this application embodiment can also be located in a server or server cluster that is different from server 130 and capable of communicating with terminal device 110 and / or server 130.

[0034] It should be understood that Figure 1 The number of terminal devices, networks, servers, and server nodes shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, networks, servers, and server nodes.

[0035] Figure 2 A flowchart illustrating a task processing method according to an embodiment of this application is shown schematically.

[0036] like Figure 2 As shown, the method includes operations S210~S230.

[0037] In operation S210, in response to receiving a task request, based on multiple indicator information of multiple server nodes and resource consumption information required to process the task in the task request, the load assessment information of each server node is determined. The multiple indicator information is used to indicate the configuration information and status information of each server node.

[0038] In the embodiments of this application, the task request can be a real-time request initiated by a user through a terminal device, requiring processing by a backend server cluster. Resource consumption information refers to the types of computing resources required to successfully execute the task in the task request and their estimated consumption; it represents a pre-analysis or prediction of task requirements. Multiple indicator information can refer to a multi-dimensional data set describing the configuration information (static) and status information (dynamic) of each server node, collected or queried in real time from each server node. For example, configuration information includes static resource information such as CPU, memory, and storage; status information includes dynamic information such as current node load, the number of connections between the node and the task, and response time. The indicator information can be stored in a specific system file for real-time querying.

[0039] Load assessment information can be determined by combining node capabilities and task requirements, and is used to characterize the load distribution among multiple server nodes. It can include resource types, resource availability, and the degree of fit between nodes and tasks to be processed. For example, the degree of fit between each server node and a specific task request. For instance, matching the resource requirements of the task to be processed for each resource type with the remaining resources of the nodes, calculating a comprehensive score, and obtaining the suitability between the node and the task. Alternatively, using preset weights, the suitability of each dimension (resource type and resource availability information) can be weighted and summed to obtain a weighted value, which serves as load assessment information. The weights of each dimension can be dynamically adjusted according to task characteristics (e.g., increasing CPU weight for compute-intensive tasks and increasing storage weight for I / O-intensive tasks).

[0040] For example, clients (such as web browsers, mobile applications, etc.) send HTTP requests, database query requests, etc. Based on historical data similar to the task to be processed or the task type of the task to be processed, the resource consumption type, intensity, and amount in the resource consumption information are determined. Then, combined with the configuration and status information of the processors, memory, databases, and disks of multiple server nodes, the adaptability between each server node and the task to be processed at the current time is calculated as load assessment information.

[0041] In operation S220, a target strategy is selected from multiple candidate strategies based on load assessment information. The multiple candidate strategies include a first strategy that uses the number of connections between each server node and the task as the node selection criterion, a second strategy that uses the response time of each server node as the node selection criterion, and a third strategy that uses the expected processing latency of the task to be processed as the node selection criterion when it is assigned to each server node.

[0042] In embodiments of this application, multiple candidate strategies can be predefined sets of different algorithmic rules used to select target nodes from a pool of server nodes to achieve load balancing. The first strategy can be to assign new task requests to the server node with the fewest currently active client connections. The second strategy can be to assign new task requests to the server node with the shortest historical average response time or the fastest recent response speed. The third strategy can be a predictive strategy, for example, by using a trained predictive model to comprehensively analyze the task characteristics (such as type and data volume) of the task to be processed and the real-time status of the server nodes, predicting the expected processing time (latency) required to assign the current task to each node, and selecting the node with the shortest predicted time. The target strategy can be a specific strategy dynamically selected from multiple candidate strategies based on current load assessment information and about to be executed.

[0043] For example, if load assessment information indicates the presence of hot and idle nodes among multiple server nodes, the current goal is to quickly reduce the processing pressure on the nodes. Therefore, the strategy that can most directly and quickly distribute requests is selected, namely the first strategy (least number of connections). This strategy is simple to calculate and can immediately direct new requests to the target node with the fewest current connections, thus it is selected as the target strategy.

[0044] For example, if the load assessment information indicates that the load is highly balanced among multiple server nodes and the overall load is at a medium level, and the current goal is to optimize the user experience (reduce latency), then either the second strategy (shortest response time) or the third strategy (shortest predicted latency) can be selected. However, if more emphasis is placed on accurate prediction based on real-time status, the third strategy is selected as the target strategy.

[0045] In operation S230, a target node is determined from multiple server nodes based on the target policy, and the tasks to be processed are assigned to the target node for execution.

[0046] In embodiments of this application, the multiple server nodes can be a set of all server instances that constitute a backend service cluster and are available for task allocation. The target node can be a specific server node ultimately selected from the multiple server nodes according to the rules defined by the target strategy, used to receive and process the currently pending task. The pending task can be a specific unit of work contained in a task request that requires computing resources to complete.

[0047] For example, based on any one of the strategies of minimum number of connections, shortest response time, and shortest predicted latency, Node-2 is selected as the target node from multiple server nodes Node-1, Node-2, and Node-3, and the tasks to be processed are then assigned to Node-2 for processing. While traditional load balancing schemes have various algorithms, strategy selection is often statically configured. Even with dynamic evaluation, the evaluation result (e.g., "Node 1 is the least idle") may lack a strong correlation with the final execution action (e.g., "round-robin to Node 2"), leading to the failure to execute the optimal decision and rendering the evaluation meaningless. The embodiments of this application enforce and unambiguously execute the target selection strategy, making the target strategy directly serve as the input and sole criterion for target node confirmation. This strongly coupled closed loop of "evaluation-decision-execution" ensures that the intent of intelligent scheduling is accurately implemented, eliminating the deviation between decision and execution.

[0048] For example, a distributed server architecture can be used for real-time task processing, connecting multiple server nodes through a high-speed network to form a unified backend server cluster. Clustered management enables unified scheduling and allocation of server resources. An adaptive candidate strategy is employed, dynamically adjusting the candidate strategy based on real-time monitoring of server node load to ensure load balancing across nodes. Furthermore, the most suitable candidate strategy can be flexibly selected based on business needs and data characteristics, achieving more accurate and efficient load balancing.

[0049] In one feasible embodiment, the task processing method supports an elastic scaling mechanism, which can automatically adjust the number of server nodes according to business needs. When business needs increase, new server nodes can be automatically added to improve the system's processing capacity; when business needs decrease, some server nodes can be automatically released to reduce resource waste and operating costs.

[0050] According to embodiments of this application, by combining the configuration information (e.g., static resources such as CPU, memory, and storage) and status information (e.g., dynamic indicators such as current load, number of connections, and response time) of multiple server nodes with the resource consumption information of the tasks to be processed, the adaptability between each node and the task is comprehensively evaluated from multiple dimensions. This achieves a precise quantitative evaluation of the processing capacity of the server nodes, thereby solving the problems of low resource utilization and inaccurate load evaluation, and avoiding load imbalance between nodes. Furthermore, based on the load distribution among multiple server nodes, a target strategy matching the current load distribution is dynamically selected from multiple candidate strategies, including the number of connections, response time, and predicted latency. This allows the load balancing process to flexibly adapt to real-time conditions and task requests with different characteristics, solving the problem of lack of intelligence and adaptability in traditional solutions. The above adaptive load balancing process is implemented through software logic, replacing expensive dedicated hardware devices with a software-defined approach. This achieves dynamic and accurate task scheduling while reducing costs and improving the flexibility of system configuration.

[0051] As we understand, the above has already explained how to process tasks from the overall technical solution perspective. The following will explain how to determine the load assessment information.

[0052] According to an embodiment of this application, load assessment information for each server node is determined based on multiple indicator information of multiple server nodes and resource consumption information required for the task to be processed in the execution task request. This includes: determining target indicator information associated with the task to be processed from multiple indicator information according to the resource type in the resource consumption information; quantifying the target indicator information based on the resource consumption amount in the resource consumption information to obtain the fit between each server node and the task to be processed, and using the fit as load assessment information.

[0053] In the embodiments of this application, the resource type can be the processor type (e.g., CPU or GPU), memory type, or read / write type required to execute the task to be processed. The target indicator information can be a specific subset of indicators directly related to the requirements of the current task to be processed, selected from multiple indicator information based on the resource type indicated by the resource consumption information. The suitability can be a numerical value obtained through quantization, used to characterize the suitability of a specific server node for handling the current specific task to be processed; the higher the value, the more suitable the node is for handling this task. Quantization methods include, but are not limited to, normalization, standardization, weighted scoring, mapping transformation, etc., and are not specifically limited here.

[0054] Taking an AI inference server cluster receiving a task to "perform image recognition" as an example, the target indicator information is filtered from multiple indicator information based on the resource type (GPU computing power, memory) of the image recognition task, to obtain the "available GPU computing power" and "idle video memory" of each node.

[0055] Based on the task resource consumption (15 TFLOPs, 4GB), the target indicator information of each server node is normalized, and the weighted harmonic or minimum value of "supply / demand" is calculated (barrel principle). For example, the saturation of each resource dimension is calculated: saturation = min(available quantity / demand quantity, 1), and then the minimum value of the two is taken as the fitness (the closer the value is to 1, the better the fit), to obtain the load assessment information of each server node.

[0056] Related technologies typically use general load metrics (such as overall CPU utilization and system load) that are poorly matched to the task at hand to evaluate all nodes, leading to misjudgments. For example, assigning an AI task requiring large amounts of GPU memory to a node with an idle CPU but exhausted GPU memory will not effectively handle the task, even if its "general load" is low. The embodiments of this application introduce resource consumption information and use resource type as the basis for filtering target metric information, shifting load assessment from "what a node can provide" to "what the task needs." The evaluation process is closely focused on the specific resource requirements of the current task, fundamentally avoiding resource mismatch and ensuring that tasks are assigned to nodes that truly have the corresponding processing capabilities, thereby improving the success rate and efficiency of task execution.

[0057] According to an embodiment of this application, selecting a target strategy from multiple candidate strategies based on load assessment information includes: when the dispersion of the load assessment information indicates that the load difference between each server node is greater than a difference threshold, determining a first strategy based on the valley value of the number of connections as the target strategy; when the dispersion indicates that the load difference between each server node is less than or equal to the difference threshold, and the average value of the load assessment information is greater than a preset threshold, determining a second strategy based on the valley value of the response time as the target strategy; and when the dispersion indicates that the load difference between each server node is less than or equal to the difference threshold, and the average value of the load assessment information is less than or equal to a preset threshold, determining a third strategy based on the valley value of the predicted processing latency as the target strategy.

[0058] In the embodiments of this application, dispersion can be a statistical measure of the difference in load assessment information among all server nodes. The greater the dispersion, the greater the difference in load assessment values ​​among nodes, i.e., the more unbalanced the load. Dispersion can be represented in various forms such as variance, standard deviation, range, and coefficient of variation, and is not specifically limited here. The load difference can be comprehensively indicated by the statistical measure of dispersion to indicate the degree of load difference among all nodes. The average value can reflect the average load level of the entire server cluster.

[0059] Taking a server node cluster for processing video streams as an example, the difference threshold is defined as 20, and the preset threshold is 70 (load score range 0-100). The server node cluster contains four server nodes: Node-1, Node-2, Node-3, and Node-4. At a certain moment, the load assessment information of the four server nodes is calculated as [85, 30, 88, 82]. After calculation, the variance is large, about 650, which is much greater than the difference threshold of 20, indicating that the load difference between nodes is extremely large; the average value is calculated as (85+30+88+82) / 4 = 71.25. Therefore, a strategy is selected. The variance (650) > the difference threshold (20), which satisfies the first conditional branch. There is no need to perform a second-level judgment. The first strategy is determined as the target strategy, which is to adopt the "least number of connections" strategy, which directs new video stream processing requests to the node with the fewest current connections (possibly the node with a load score of 30) to quickly balance the load.

[0060] With the load assessment information for the four server nodes becoming [65, 68, 72, 71], the calculated variance is approximately 8, which is less than the difference threshold of 20, indicating load balancing. The calculated average is 69. A first-level judgment is made: variance (8) ≤ difference threshold (20), proceeding to the second-level judgment; the second-level judgment: average (69) ≤ preset threshold (70), indicating low load. The third conditional branch is satisfied, therefore the third strategy is determined as the target strategy.

[0061] In traditional technologies, policy switching is mostly manual configuration or based on simple event triggers (such as a node failure), making it difficult to accurately switch based on the real-time and overall load quantification characteristics of the cluster. For example, using a fixed "minimum number of connections" policy will still dispatch requests to nodes with fewer connections but already bottlenecked performance, even when the cluster is already overloaded, exacerbating congestion.

[0062] The embodiments of this application introduce two key statistical quantities, variance and mean, and combine them with difference threshold and preset threshold to enable the system to perceive the cluster load status, clearly distinguish between three situations: uneven load, balanced load but high load, and balanced load but low load, and match the most suitable strategy for each situation, thereby improving the overall robustness and efficiency under various stress scenarios.

[0063] According to an embodiment of this application, determining a target node from multiple server nodes based on a target strategy includes: when the target strategy is a first strategy, determining a server node whose current active connection count is at a low value as a target node; when the target strategy is a second strategy, determining a server node whose historical average response time is at a low value as a target node; and when the target strategy is a third strategy, determining a server node whose predicted processing latency is at a low value as a target node.

[0064] In embodiments of this application, the number of currently active connections can refer to the number of Transmission Control Protocol (TCP) connections or application layer sessions that have been established and are processing requests between the server node and all clients at the evaluation time. The historical average response time can refer to the arithmetic mean of the time required for the server node to process all requests within a preset time window in the past. The predicted processing latency can refer to the estimated completion time for assigning the task to the node for execution, calculated using a prediction model (such as a supervised learning model or a context-based multi-armed gambling machine model) based on the characteristics (such as type and data volume) of the current task to be processed and the real-time state of the server node. The valley value is the minimum value in a set of values.

[0065] Taking the three service nodes (Node-1, Node-2, Node-3) managed by the online API gateway cluster as an example, we can query the historical average response time of each node over the past 5 minutes: Node-1 is 85ms, Node-2 is 120ms, and Node-3 is 65ms. We can determine that the valley of the historical average response time is 65ms, which corresponds to Node-3. Therefore, we can identify Node-3 as the target node.

[0066] In related technologies, strategy decision-making and node selection logic may be scattered across different modules or associated through complex configurations, posing a risk of inconsistency in understanding or execution deviation. For example, the decision-making module may believe that "the fastest node should be selected," but the execution module may incorrectly query the metric for "the least busy node." The embodiments of this application avoid ambiguity and errors in the execution phase by associating and mapping the target strategy with specific node evaluation metrics (number of connections, historical response, predicted latency), ensuring that the intelligent decision-making results can be accurately and reliably transformed into actual routing actions, forming a closed-loop control.

[0067] According to an embodiment of this application, the above method further includes: using the node utilization rate, memory utilization rate, number of active connections and queue length of each server node obtained from multiple indicator information as node features, processing the combined features obtained by combining the node features and the task features of the task to be processed to obtain the expected response time, wherein the task features are extracted from the context information of the task to be processed.

[0068] In the embodiments of this application, node features can be core performance indicators collected in real time from each server node to quantify its current running status and resource usage. Task features can be features extracted from the context information of the current task to be processed to describe the task's own attributes and requirements. Combined features can be a new, unified high-dimensional feature vector formed by fusing node features and task features according to predetermined rules (such as concatenation, concatenation followed by embedding, etc.). Expected response time can be a value output by processing the combined features (e.g., through a pre-trained machine learning model), which predicts the estimated completion time (latency) required to assign the current task to the corresponding server node for processing.

[0069] For example, node features are collected in real time, task features are extracted in parallel, the two are merged into a combined feature, the trained model is used to process the combined feature, and finally the expected response time is output.

[0070] Taking a 3D animation rendering task submitted by a user and received by the cloud computing rendering service platform as an example, the task is analyzed to extract its features from its context information: {api_endpoint: " / render", method: "POST", payload_size: 150GB, priority: high}. Simultaneously, the three server nodes (Node-R1, Node-R2, Node-R3) used for rendering in the cluster are polled to obtain their real-time features. For each node, its node features are combined with the current task's features to form three combined feature vectors. These three combined feature vectors are then sequentially input into a pre-trained gradient boosting decision tree model. The decision tree model infers from the combined features of each node and outputs a predicted latency value to obtain the expected response time. It is understood that the model's training and inference methods are the same or similar, and will not be elaborated upon here.

[0071] In related technologies, when prediction is required, limited and static node metrics (such as CPU utilization alone) are often used, or the characteristics of the task itself are ignored, resulting in coarse estimates. The embodiments of this application introduce task features and combine them with node features, enabling the prediction model to possess context-aware capabilities. This allows the model to jointly model the actual task and nodes during decision-making, capturing the true performance of different types of tasks under different node states, thereby making more realistic and accurate latency predictions than traditional methods.

[0072] According to an embodiment of this application, the combined features obtained by combining node features and task features of the task to be processed are processed to obtain the expected response time, including: searching in the mapping relationship library for historical features with a similarity greater than a similarity threshold with the combined features, and using the historical response time corresponding to the historical features as the expected response time.

[0073] In embodiments of this application, the mapping relationship library can be constructed based on the node features, request features, and actual response times of historical task requests. It stores a large number of historical records, each containing a historical feature and its corresponding, actually observed historical response time. Historical features can be combined feature vectors corresponding to a past task processing event stored in the mapping relationship library. Similarity can be used to measure the closeness or matching degree between the combined features of the current input and a historical feature in the mapping relationship library. Historical response times can be task processing delay times that actually occurred historically and appear in pairs with historical features, stored in the mapping relationship library.

[0074] For example, the cosine similarity between the combined feature and each historical feature in the mapping database is calculated, and the similarity threshold is set to 0.85. Through matching, it is found that the similarity between the combined feature and historical feature H1 is 0.92, and the historical response time is 450ms; the similarity between the combined feature and historical feature H2 is 0.89, and the historical response time is 480ms; the similarity between the combined feature and historical feature H3 is 0.87, and the historical response time is 510ms. Therefore, based on the historical feature H1 with the highest matching degree, the expected response time is determined to be 450ms.

[0075] In related technologies, when sufficient data is lacking to train complex models, only very coarse rules (such as linear estimation based on task size) can be used, resulting in poor accuracy. The embodiments of this application, even without identical tasks, can find the "most similar" historical scene for approximation through similarity thresholds and similarity calculations. This method is more flexible than simple linear extrapolation and can capture more complex nonlinear relationships between features. For unseen new task types or node state combinations, as long as there are records with a certain degree of similarity in the historical database, a relatively reasonable prediction can be given, improving the coverage and generalization ability of the prediction system and alleviating the cold start problem.

[0076] According to an embodiment of this application, the combined features obtained by combining node features and task features of the task to be processed are processed to obtain the expected response time, including: processing the combined features based on a decision tree to obtain a predicted cost value that indicates the expected processing cost required to allocate the task to be processed to the corresponding server node for processing, wherein the predicted cost value includes at least one of expected time delay and expected resource consumption; and using the predicted cost value as the expected response time for each server node to process the task to be processed.

[0077] In the embodiments of this application, the decision tree can be a tree-based machine learning model. When processing combined features, the model starts from the root node, determines the branch direction based on the feature values, and continues until it reaches a leaf node, using the value of that leaf node as the output. The expected processing cost can be used to measure the estimated overhead required to assign the task to a specific server node for processing. The predicted cost value can be the direct output of the decision tree model after processing the combined features, and is a quantitative representation of the expected processing cost.

[0078] For example, the node features of the target node (e.g., CPU utilization 70%, memory utilization 60%, active connections 10) can be obtained in real time, and the task features of the current task to be processed (API: / infer, load size: 2MB) can be extracted and fused into a combined feature vector. This combined feature vector is then input into a pre-trained gradient boosting decision tree model, which consists of a sequence of multiple decision trees, each learning the residuals of the previous tree. After the model performs branch judgments and accumulates the leaf node values ​​of multiple decision trees, it outputs a predicted cost value, such as 350. This value is defined as the expected time delay in milliseconds during model training, and this value of 350 is directly used as the expected response time for the node to process this task.

[0079] For example, the combined features are input into a pre-trained random forest model, which contains a large number of decision trees trained in parallel. The combined features are processed independently by each decision tree, and each decision tree outputs a prediction value. The random forest model averages the outputs of all decision trees (for regression tasks) to obtain the final prediction cost value. This value is defined as the comprehensive cost during training, which includes a weighted sum of expected time delay (weight 0.7) and expected CPU resource consumption (weight 0.3), for example, a value of 485. Then, from this comprehensive cost prediction cost value, the expected time delay component is extracted or mapped (e.g., through a preset transformation coefficient), resulting in an expected response time of 400ms.

[0080] Related techniques use linear regression models or threshold-based heuristics to predict latency. These methods struggle to effectively model the complex, non-linear interrelationships between node resources (such as CPU, memory, and I / O) and the complex effects of their interactions with different task characteristics. For example, they cannot accurately depict the phenomenon that "when memory utilization exceeds 80%, even if the CPU is idle, processing latency will increase exponentially due to frequent paging."

[0081] The embodiments of this application utilize a decision tree model to automatically learn the aforementioned complex, nonlinear decision boundaries from historical data through feature segmentation. During prediction, it comprehensively considers information from all dimensions of the combined features and their interactions, thereby making far more accurate predictions of expected processing costs (especially expected time latency) than traditional linear methods. This provides a more reliable basis for load balancing in task processing, enabling better node selection and directly translating into lower average task latency and higher cluster throughput.

[0082] According to an embodiment of this application, the combined features are processed based on a decision tree to obtain a predicted cost value that indicates the expected processing cost required to assign the task to be processed to the corresponding server node for processing. This includes: sequentially selecting paths for the combined features according to the feature judgment conditions of the tree nodes in each decision tree until multiple leaf nodes are reached; and combining the values ​​representing the expected processing cost stored in each leaf node to obtain the predicted cost value.

[0083] In the embodiments of this application, the feature judgment conditions can be judgment rules stored on the internal nodes of the decision tree, which serve as the basis for the decision tree to select a path. Path selection can involve inputting the combined features into the decision tree, starting from the root node, judging the values ​​of the corresponding dimensions in the combined features according to the feature judgment conditions at the current node, and selecting to enter the left child node or the right child node based on the judgment result, and repeating this process until it is impossible to continue descending.

[0084] A leaf node can be a terminal node in a decision tree that no longer contains child nodes. Each leaf node stores a value representing the expected processing cost, which is learned during the model training phase and represents a summary of the expected processing cost (such as average latency) for all training samples falling into that leaf node. Combining the values ​​stored in the leaf nodes can refer to the fact that when multiple decision trees are used, the input combined features are processed independently by each decision tree and each reaches a leaf node. In the embodiments of this application, it refers to performing aggregation operations (such as summation or averaging) on ​​the values ​​stored in the leaf nodes reached by all trees to obtain the final single predicted value.

[0085] For example, combined features are input into a gradient boosting-based decision tree. For the first decision tree, a path is selected based on the feature judgment conditions of its root node and subsequent internal nodes (e.g., "Is the task load size > 1MB?", "Is the node memory utilization ≤ 70%?"), eventually reaching a leaf node of the decision tree. This node stores a value, for example, +0.5. This value is the residual correction amount for the initial prediction (usually the average of all sample labels).

[0086] The prediction result of the first decision tree (initial prediction +0.5) is used as the starting point for the second decision tree. The path selection process is repeated until a leaf node of the second decision tree is reached, obtaining a new correction, such as -0.2. This process is performed sequentially on all trees. The corrections stored in the leaf nodes reached by all 100 decision trees are summed. For example, the sum is +8.3. The initial prediction value (assumed to be 200ms) is added to the sum of the corrections to obtain the final prediction cost value of 208.3 ms, which is the expected processing cost (time delay) of the prediction.

[0087] While related technologies can achieve high-precision predictions, the decision-making process is uninterpretable, resembling a "black box." When predictions deviate, it's difficult to pinpoint the cause, hindering system debugging and trust building. In this application's embodiment, the decision tree prediction process executes a series of feature judgment conditions. For any prediction result, its decision path can be clearly traced back (e.g., "Because the task load > 1MB and node memory > 70% and CPU utilization < 50%, the prediction latency falls into the high range"). This transparent, rule-chain-based reasoning method provides excellent interpretability for the prediction results. It not only helps users understand the model's decision logic but also allows for quick identification of which feature judgment condition is inconsistent with the actual situation when prediction errors occur, thus guiding feature optimization or model iteration.

[0088] According to an embodiment of this application, the combined features obtained by combining node features and task features of the task to be processed are processed to obtain the expected response time, including: using the combined features as context information of the decision function to determine the numerical value and confidence value of the expected processing benefit that can be obtained by selecting the corresponding server node; using preset weights to perform a weighted summation of the numerical value and confidence value to determine the comprehensive evaluation value of each server node; and converting the comprehensive evaluation value into the expected response time of each server node for processing the task to be processed.

[0089] In embodiments of this application, the decision function can be an algorithm capable of receiving contextual information (i.e., combined features) and outputting an estimate of the expected return and its uncertainty regarding the selected node. The contextual information of the decision function can refer to all information about the current task and node state carried by the combined features. The numerical value of the expected processing return can be used to quantitatively characterize the expected positive effect obtained by selecting the corresponding server node to process the current task. The confidence value can be used to quantitatively characterize the degree of certainty or uncertainty in predicting the numerical value of the expected processing return. The comprehensive evaluation value can be a single value obtained by weighting and summing the numerical value of the expected processing return and the confidence value according to preset weights.

[0090] For example, a prediction model is maintained for each server node based on the linear confidence upper bound algorithm, and the combined features are used as context vectors to input the prediction model of each node.

[0091] For server node A, the expected processing revenue is equal to the dot product of the context vector and the model weight vector. The confidence score is... , where α is the exploration coefficient. The confidence level represents the standard deviation (uncertainty) of the return estimate.

[0092] The output of the decision function is itself in the form of payoff value + confidence value (i.e., upper confidence bound), which is equivalent to presetting the weights as payoff weight = 1 and confidence weight = α. Therefore, the overall evaluation value = payoff value + α × confidence value. The transformation rule is defined as expected response time = baseline time / (overall evaluation value + ε). The node with the largest overall evaluation value is selected, and its corresponding transformation value is taken as the expected response time of that node.

[0093] For example, a probabilistic model can be maintained for each server node based on the Thompson sampling algorithm. By inputting combined features, the probabilistic model will output a probability distribution of the payoff, rather than a single value.

[0094] When a decision is needed, a value is sampled from this distribution as the expected benefit for this treatment. The variance (or standard deviation) of this distribution is used as the confidence level (a larger variance indicates a lower confidence level).

[0095] Set preset weights (e.g., return weight w_r=0.7, confidence weight w_c=0.3). The weighted summation of confidence levels (variance) is intended to prioritize nodes with greater cognitive uncertainty (larger variance) to facilitate exploration when sampled values ​​are similar. Similarly, a negative correlation function is used to convert the comprehensive evaluation value into the expected response time, and the node with the largest comprehensive evaluation value is selected.

[0096] In related technologies, always selecting the node with the highest predicted return (or lowest latency) ignores attempts to explore nodes that may be better but whose current predictions are inaccurate. If the initial prediction is flawed, it becomes difficult to discover the truly optimal node. The embodiments of this application output both return and confidence values ​​simultaneously through a decision function, incorporating the confidence value into the decision through preset weights, thus enabling the system to proactively explore. For a newly launched node or a rare task type that has been processed infrequently, its confidence value (uncertainty) will be relatively high. Even if its short-term return is mediocre, a higher overall evaluation value may prompt its selection, thereby collecting data and updating understanding. This method ensures the continuous discovery of new potential high-quality nodes, avoids premature strategy convergence, and ultimately converges to a globally better load distribution.

[0097] Figure 3 A flowchart illustrating a task processing method according to another embodiment of this application is shown.

[0098] like Figure 3 As shown, the terminal device is the initiator of the task request, the server is the central hub and intelligent decision-making core for executing the task processing method, and the server node pool is the cluster that constitutes the backend server nodes. The task processing method includes operations S310 to S370.

[0099] When operating S310, the terminal device sends a task request to the load balancer.

[0100] When operating the S320, the processor initiates health probes (such as heartbeat checks) to all nodes in the server node pool in real time to analyze the metric information reported by the nodes.

[0101] When operating the S330, the system integrates real-time monitoring data and metrics to select the target server from multiple load balancing strategies. These strategies include a first strategy that selects the node with the fewest current connections, a second strategy that selects the node with the fastest historical or real-time response, and a third strategy that predicts the optimal server node based on machine learning or historical data.

[0102] In operation S340, a target node is determined from multiple server nodes based on a target policy. For example, if the target policy is the second policy, the server node with the fastest response time is determined as the target node.

[0103] When operating S350, the selected server node receives the task request and executes the business logic of the backend server to process the request.

[0104] When operating S360, the server node returns the processing result to the processor.

[0105] During operation of S370, the processor ultimately returns the processing result to the terminal device.

[0106] Based on the above task processing method, this application also provides a task processing apparatus. The following will be combined with... Figure 4 The device is described in detail.

[0107] Figure 4 A schematic block diagram of a task processing apparatus according to an embodiment of this application is shown.

[0108] like Figure 4 As shown, the task processing device 400 of this embodiment includes a determination module 410, a selection module 420, and an allocation module 430.

[0109] The determining module 410 is configured to, in response to receiving a task request, determine load assessment information characterizing the load distribution among multiple server nodes based on multiple indicator information of multiple server nodes and resource consumption information required to execute the task to be processed in the task request. The multiple indicator information is used to indicate the configuration and status information of each server node. In one embodiment, the determining module 410 may be used to perform the operation S210 described above, which will not be repeated here.

[0110] The selection module 420 is used to select a target strategy from multiple candidate strategies based on the load assessment information. The multiple candidate strategies include a first strategy using the number of connections between each server node and the task as the node selection criterion, a second strategy using the response time of each server node as the node selection criterion, and a third strategy using the expected processing latency of the task to be processed being allocated to each server node as the node selection criterion. In one embodiment, the selection module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0111] The allocation module 430 is used to determine a target node from multiple server nodes based on a target strategy, and to allocate the tasks to be processed to the target node for execution. In one embodiment, the allocation module 430 can be used to perform the operation S230 described above, which will not be repeated here.

[0112] According to embodiments of this application, based on the determination module 410, selection module 420, and allocation module 430 in the task processing device 400, load assessment information is determined based on multi-dimensional indicators, thereby achieving accurate quantitative assessment of the processing capacity of server nodes. This solves the problems of low resource utilization and inaccurate load assessment, and avoids load imbalance between nodes. By dynamically selecting a target strategy from multiple strategies, including connection count, response time, and predicted latency, based on real-time load conditions, the load balancing process can flexibly adapt to real-time conditions and task requests with different characteristics, solving the problem of lack of intelligence and adaptability in traditional solutions. By implementing the above adaptive load balancing process based on software logic, expensive dedicated hardware devices are replaced with software-defined methods, achieving dynamic and accurate task scheduling while reducing costs and improving the flexibility of system configuration.

[0113] According to an embodiment of this application, the determining module 410 includes a determining submodule and a processing submodule. The determining submodule is used to determine target indicator information associated with the task to be processed from multiple indicator information based on the resource type in the resource consumption information; the processing submodule is used to quantify the target indicator information based on the resource consumption amount in the resource consumption information to obtain the fit between each server node and the task to be processed, and use the fit as load assessment information.

[0114] According to an embodiment of this application, the selection module 420 includes: a first determining submodule, a second determining submodule, and a third determining submodule. The first determining submodule is used to determine a first strategy for selecting nodes based on the valley value of the number of connections as the target strategy when the load difference between server nodes, as indicated by the dispersion of the load assessment information, is greater than a difference threshold. The second determining submodule is used to determine a second strategy for selecting nodes based on the valley value of the response time as the target strategy when the load difference between server nodes, as indicated by the dispersion, is less than or equal to the difference threshold, and the average value of the load assessment information is greater than a preset threshold. The third determining submodule is used to determine a third strategy based on the valley value of the predicted processing latency as the target strategy when the load difference between server nodes, as indicated by the dispersion, is less than or equal to the difference threshold, and the average value of the load assessment information is less than or equal to a preset threshold.

[0115] According to an embodiment of this application, the allocation module 430 includes: a first node determination submodule, a second node determination submodule, and a third node determination submodule. The first node determination submodule is used to determine server nodes whose current active connection count is at a low value as target nodes when the target policy is a first policy. The second node determination submodule is used to determine server nodes whose historical average response time is at a low value as target nodes when the target policy is a second policy. The third node determination submodule is used to determine server nodes whose predicted processing latency is at a low value as target nodes when the target policy is a third policy.

[0116] According to an embodiment of this application, the above-mentioned apparatus further includes: a feature processing module, used to process the node utilization rate, memory utilization rate, number of active connections and queue length of each server node obtained from multiple indicator information as node features, and to process the combined features obtained by combining the node features and the task features of the task to be processed to obtain the expected response time, wherein the task features are extracted from the context information of the task to be processed.

[0117] According to an embodiment of this application, the feature processing module includes: a search submodule, used to search for historical features in the mapping relation library whose similarity to the combined features is greater than a similarity threshold, and to use the historical response duration corresponding to the historical features as the expected response duration.

[0118] According to an embodiment of this application, the feature processing module includes an allocation submodule and an application submodule. The allocation submodule is used to process the combined features based on a decision tree to obtain a predicted cost value that indicates the expected processing cost required to allocate the task to be processed to the corresponding server node for processing. The predicted cost value includes at least one of expected time delay and expected resource consumption. The application submodule is used to use the predicted cost value as the expected response time for each server node to process the task.

[0119] According to an embodiment of this application, the allocation submodule includes a selection unit and a combination unit. The selection unit is used to sequentially select paths for the combined features based on the feature judgment conditions of each tree node in the decision tree until multiple leaf nodes are reached; the combination unit is used to combine the values ​​representing the expected processing cost stored in each leaf node to obtain the predicted cost value.

[0120] According to an embodiment of this application, the feature processing module includes: a confidence value determination submodule, a weighting submodule, and a transformation submodule. The confidence value determination submodule is used to determine the numerical value and confidence value of the expected processing benefit that can be obtained by selecting the corresponding server node, using the combined features as context information for the decision function; the weighting submodule is used to perform a weighted summation of the numerical value and confidence value using preset weights to determine the comprehensive evaluation value of each server node; the transformation submodule is used to convert the comprehensive evaluation value into the expected response time for each server node to process the task to be processed.

[0121] According to embodiments of this application, any plurality of modules among the determining module 410, selecting module 420, and allocating module 430 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules can be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the determining module 410, selecting module 420, and allocating module 430 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the determining module 410, selecting module 420, and allocating module 430 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0122] Figure 5 A block diagram of an electronic device suitable for implementing the methods described above, according to an embodiment of this application, is illustrated schematically.

[0123] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this application. Figure 5As shown, an electronic device 500 according to an embodiment of this application includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0124] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0125] According to embodiments of this application, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to a bus 504. The system 500 may also include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0126] According to embodiments of this application, the method flow according to embodiments of this application can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0127] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0128] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0129] For example, according to embodiments of this application, a computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0130] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this application. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the methods provided in the embodiments of this application.

[0131] When the computer program is executed by the processor 501, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0132] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0133] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not expressly stated in this application. In particular, the various embodiments and / or features described in the claims of this application may be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0135] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this application is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this application, and all such substitutions and modifications should fall within the scope of this application.

Claims

1. A task processing method, characterized in that, The method includes: In response to receiving a task request, based on multiple indicator information of multiple server nodes and resource consumption information required to execute the task to be processed in the task request, load assessment information characterizing the load distribution among multiple server nodes is determined. The multiple indicator information is used to indicate the configuration information and status information of each server node. Based on the load assessment information, a target strategy is selected from multiple candidate strategies. The multiple candidate strategies include a first strategy that uses the number of connections between each server node and the task as the node selection criterion, a second strategy that uses the response time of each server node as the node selection criterion, and a third strategy that uses the expected processing delay time of the task to be processed to each server node as the node selection criterion. Based on the target strategy, a target node is determined from the multiple server nodes, and the task to be processed is assigned to the target node for task processing.

2. The method according to claim 1, characterized in that, The load assessment information for each server node is determined based on multiple metrics information from multiple server nodes and resource consumption information required to execute the tasks to be processed in the task request, including: Based on the resource type in the resource consumption information, determine the target indicator information associated with the task to be processed from the plurality of indicator information; Based on the resource consumption amount in the resource consumption information, the target indicator information is quantified to obtain the fit degree between each server node and the task to be processed, and the fit degree is used as the load assessment information.

3. The method according to claim 1, characterized in that, The step of selecting a target strategy from multiple candidate strategies based on the load assessment information includes: When the dispersion of the load assessment information indicates that the load difference between server nodes is greater than the difference threshold, the first strategy of selecting nodes based on the valley of the number of connections is determined as the target strategy. If the load difference between server nodes indicated by the degree of dispersion is less than or equal to the difference threshold, and the average value of the load assessment information is greater than the preset threshold, the second strategy of selecting nodes based on the valley value of response time will be determined as the target strategy. If the load difference between server nodes indicated by the degree of dispersion is less than or equal to the difference threshold, and the average value of the load assessment information is less than or equal to the preset threshold, the third strategy, which uses the valley value of the predicted processing latency as the selection criterion, is determined as the target strategy.

4. The method according to claim 1, characterized in that, The step of determining the target node from the multiple server nodes based on the target strategy includes: When the target strategy is the first strategy, the server node with the current number of active connections at a low value is identified as the target node; When the target strategy is the second strategy, the server node whose historical average response time is at a valley value is determined as the target node; When the target strategy is the third strategy, the server node whose predicted processing latency is at a valley value is determined as the target node.

5. The method according to claim 4, characterized in that, The method further includes: The node utilization rate, memory utilization rate, number of active connections and queue length of each server node obtained from the multiple indicator information are used as node features. The combined features obtained by combining the node features with the task features of the task to be processed are processed to obtain the expected response time. The task features are extracted from the context information of the task to be processed.

6. The method according to claim 5, characterized in that, The process of processing the combined features obtained by combining the node features and the task features of the task to be processed to obtain the expected response time includes: Search the mapping relationship library for historical features that have a similarity greater than the similarity threshold with the combined features, and use the historical response duration corresponding to the historical features as the expected response duration.

7. The method according to claim 5, characterized in that, The process of processing the combined features obtained by combining the node features and the task features of the task to be processed to obtain the expected response time includes: The combined features are processed based on a decision tree to obtain a predicted cost value that indicates the expected processing cost required to assign the task to the corresponding server node for processing. The predicted cost value includes at least one of expected time delay and expected resource consumption. The predicted cost value is used as the expected response time for each server node to process the task to be processed.

8. The method according to claim 7, characterized in that, The process of processing the combined features based on the decision tree to obtain a predicted cost value that indicates the expected processing cost required to allocate the task to the corresponding server node for processing includes: The combined features are sequentially selected based on the feature judgment conditions of each tree node in the decision tree until multiple leaf nodes are reached. The predicted cost value is obtained by combining the values ​​representing the expected processing cost stored in each leaf node.

9. The method according to claim 5, characterized in that, The process of processing the combined features obtained by combining the node features and the task features of the task to be processed to obtain the expected response time includes: Using the combined features as contextual information for the decision function, the expected processing benefit and confidence level of selecting the corresponding server node are determined. The numerical values ​​and the confidence scores are weighted and summed using preset weights to determine the comprehensive evaluation value of each server node; The comprehensive evaluation value is converted into the expected response time for each server node to process the task to be processed.

10. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 9.

Images