Task processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a task processing method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of computers.The method comprises the steps that a to-be-processed task is obtained; obtaining a service node list based on resource demand information corresponding to the to-be-processed task; determining a target service node according to the load values of the service nodes in the service node list; and running the to-be-processed task based on the target service node. Therefore, the target service node is determined according to the resource demand of the to-be-processed task and the load condition of the service node, so that the task can be distributed more reasonably. When the target service node is determined, the load of the service node is considered, so that the task execution failure or slow execution caused by overhigh load can be avoided.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a task processing method, device, electronic device, and computer-readable storage medium. Background Art

[0002] In the related art, a fixed task allocation method is usually adopted, that is, tasks are allocated to specific server nodes according to pre-set rules, for example, tasks are allocated in order of server numbers, or simply allocated to specific types of servers according to the type of task.

[0003] This allocation method is prone to improper allocation and affect the results of task execution. For example, if a task is assigned to a server node that is already heavily loaded, it will cause the task processing speed to slow down or even cause the task execution to fail. Summary of the invention

[0004] The embodiments of the present disclosure provide a task processing method, device, electronic device, and computer-readable storage medium, which aim to solve one of the technical problems in the related art at least to a certain extent.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for processing a task, the method comprising:

[0006] Get pending tasks;

[0007] Based on the resource demand information corresponding to the task to be processed, obtain a service node list;

[0008] Determine a target service node according to the load values ​​of the service nodes in the service node list;

[0009] Based on the target service node, the task to be processed is executed.

[0010] In a second aspect, an embodiment of the present disclosure further provides a task processing device, the device comprising:

[0011] A first acquisition module, used to acquire tasks to be processed;

[0012] A second acquisition module is used to acquire a service node list based on the resource demand information corresponding to the task to be processed;

[0013] A determination module, configured to determine a target service node according to a load value of a service node in the service node list;

[0014] A running module is used to run the task to be processed based on the target service node.

[0015] In a third aspect, an embodiment of the present disclosure further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and which implements the steps in the processing method for the above-mentioned tasks when the computer program is executed by the processor.

[0016] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the processing method of the above-mentioned task are implemented.

[0017] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations of the embodiments of the present disclosure.

[0018] In the disclosed embodiment, the pending task is first obtained, then based on the resource demand information corresponding to the pending task, a service node list is obtained, then the target service node is determined according to the load value of the service node in the service node list, and finally the pending task is run based on the target service node. Thus, by determining the target service node according to the resource demand of the pending task and the load of the service node, it can be ensured that the task is more reasonably allocated. Since the load of the service node is taken into account when determining the target service node, the failure or slow execution of the task due to excessive load can be avoided.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 is a flowchart of a method for processing a task provided by the first embodiment of the present disclosure;

[0022] Figure 2 is a schematic diagram of exemplary product test values ​​provided by the second embodiment of the present disclosure;

[0023] Figure 3 is a schematic diagram of the structure of a task processing device provided by an embodiment of the present disclosure;

[0024] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] Some embodiments of the present disclosure will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, for clarity and brevity, descriptions of features known in the art may be omitted.

[0026] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0027] It should be noted that the executor of the task processing method of this embodiment can be a task processing device, which can be configured in any type of electronic device, such as a tablet computer, television, mobile phone, computer, etc., without limitation here.

[0028] In the embodiments of the present disclosure, the “task processing device” will be used as the execution subject to execute the “task processing method” for illustration, and no limitation is made herein.

[0029] It should be noted that the description order of the following embodiments is not intended to limit the priority order of the embodiments.

[0030] Figure 1 It is a flowchart of a method for processing a task provided according to the first embodiment of the present disclosure.

[0031] like Figure 1 As shown, the method includes:

[0032] Step 101, obtaining tasks to be processed.

[0033] The tasks to be processed may be any type of tasks, such as management tasks, development tasks, design tasks, compilation tasks, etc., which are not limited here.

[0034] As an implementation manner, in the embodiment of the present disclosure, the task to be processed may be a compilation task, which is not limited here.

[0035] As a possible implementation method, the task with the highest priority can be obtained from the first task queue as the task to be processed, or tasks can be obtained from other task queues as the task to be processed, which is not limited here.

[0036] Among them, the first task queue can be a priority task queue (Priority Queue). In the first task queue, each task has a priority associated with it. The processing order of each task is determined based on their priority. The task with the highest priority can be taken out first and thus get service. Tasks with the same priority can be taken out in the order in which they enter the first task queue, or in any specified order.

[0037] For example, if task task1 is the task with the highest priority in the first task queue, the device may obtain task1 from the first task queue as the task to be processed, which is not limited here.

[0038] Optionally, upon receiving the first task processing request, the device may add the task to a first task queue.

[0039] The first task processing request includes the task, priority configuration information corresponding to the task, and resource requirement information.

[0040] The first task processing request is used to request processing of a task with a priority, and thus the first task processing request may include priority configuration information corresponding to the task.

[0041] The priority configuration information is used to indicate the priority set for the task.

[0042] As an implementation method, the priority can be represented by characters, such as A represents the highest priority, B represents a moderate priority, and C represents the lowest priority. Alternatively, the priority can be represented by numbers, such as the numbers 1, 2, 3, and 4 correspond to priorities from low to high, 1 represents the lowest priority, and 4 represents the highest priority, which is not limited here.

[0043] The resource requirement information is used to describe the resources required to process any task.

[0044] For example, for task task2, when processing task2, at least 10 central processing unit (CPU) cores, 20G memory, and 30G hard disk are needed. Then "10 CPU cores, 20G memory, 30G hard disk" is the resource requirement information corresponding to task2, which is not limited here.

[0045] As a possible implementation, in the embodiments of the present disclosure, the source of the task may come from a CMS (Content Management System), which is not limited here. CMS is a software system designed for managing and publishing content. It provides software developers with tools to create, edit and publish content, allowing operation and maintenance personnel to efficiently monitor, manage and maintain the entire system.

[0046] In the disclosed embodiment, the device can obtain the task submitted by the CMS and then process the task.

[0047] Alternatively, the device may also obtain the tasks to be processed through any other source, such as project management software, workflow platform, task management tool, customer service system, email system, etc., or it may be a manually added task, which is not limited here.

[0048] As an exemplary scenario, the device receives the first task processing request sent by the CMS management background, which includes the task information corresponding to task 3, as well as the priority configuration information A (indicating the highest priority), and resource requirement information (8 CPU cores, 16G memory, 30G hard disk). Then, the device can add task 3 to the first task queue. Since the priority configuration information corresponding to task 3 is "A", task 3 can be inserted into the front position of the queue when the priority of task 3 is the highest priority among all tasks.

[0049] Furthermore, when the device obtains the task to be processed, it can obtain task3 from the first task queue as the task to be processed.

[0050] It should be noted that the above example is only an illustrative description and is not intended to limit the present disclosure.

[0051] Step 102: Obtain a service node list based on resource demand information corresponding to the task to be processed.

[0052] Among them, the service node can be an entity or component that provides a certain service in a network or distributed system, such as a server, a router, a virtual machine, a container, etc., which is not limited here.

[0053] It is understandable that each service node can be an independent entity with certain computing resources and storage resources, such as CPU, memory, hard disk, etc., which are not limited here.

[0054] The number of service nodes in the service node list may be one or more, which is not limited here.

[0055] Among them, resource requirement information is used to describe the resources needed to process any task, such as the number of CPU cores, main frequency (the clock frequency of the CPU core), memory capacity, hard disk size and hard disk type (such as mechanical hard disk and solid state drive), etc., which are not limited here.

[0056] It should be noted that the service nodes in the service node list may be service nodes whose remaining resources can meet the resource requirements of the task to be processed, so that the task to be processed can be run.

[0057] It is understandable that if the service node currently has more remaining resources, it means that the task processing capability of the service node is stronger and can adapt to the pending tasks with greater resource requirements. If the service node currently has fewer remaining resources, it means that the task processing capability of the service node is weaker and it is difficult to adapt to the pending tasks with greater resource requirements.

[0058] For example, for task 4, at least 3 CPU cores, 9G memory, and 15G hard disk are needed to process task 4, so "3 CPU cores, 9G memory, and 15G hard disk" are the resource requirement information corresponding to task 4. For example, if the remaining resources corresponding to any service node Node1 are "6 CPU cores, 15G memory, and 20G hard disk", then Node1 is the service node that can meet the resource requirements of task 4. The device can then add Node1 to the service node list.

[0059] For example, if Node2 and Node3 can also meet the resource requirements of task4, the device can also add Node2 and Node3 to the service node list, which is not limited here.

[0060] As a possible implementation method, if no service node is found that can meet the resource requirements of the pending task, the pending task can be put back into the task queue, and after being put back into the task queue, the pending task can wait to be taken out next time.

[0061] Step 103: Determine the target service node according to the load value of the service node in the service node list.

[0062] The load value can represent the current workload of the node. The smaller the load value, the more idle the node is. The larger the load value, the busier the node is.

[0063] As a possible implementation method, when determining the load value, an indicator value corresponding to any hardware resource indicator or software service indicator may be used as the load value.

[0064] The hardware resource indicator may be CPU usage, memory usage, network traffic, disk I / O rate, etc., which are not limited here.

[0065] It is understandable that the disk I / O rate can measure the busyness of the service node's disk read and write operations. A high disk I / O rate may mean that the service node is processing a large number of data read and write operations and is under heavy load. Network traffic can indicate the data transmission between the service node and the outside world. Excessive network traffic may lead to insufficient node bandwidth. High CPU usage may mean that the node is processing a large number of computing tasks. Excessive memory usage may cause the service node to perform poorly or even crash.

[0066] Among them, the software service indicator can be request processing time, number of concurrent connections, etc., which are not limited here.

[0067] Understandably, a higher number of concurrent connections may mean that the service node needs to handle multiple requests at the same time and is under heavy load. A longer request processing time may mean that the service node is overloaded and cannot process requests in a timely manner.

[0068] As another possible implementation method, multiple load indicators may be combined to determine the load value. For example, different weights may be assigned to different load indicators in advance, and then the weighted average may be calculated to comprehensively evaluate the load value of the service node.

[0069] The load index can be any of the above-mentioned hardware resource indexes or software service indexes, which are not limited here. For example, the load index corresponding to any load can be first determined as CPU utilization, memory utilization and disk I / O rate, and the corresponding weights are 45%, 30% and 25% respectively. If the values ​​corresponding to CPU utilization, memory utilization and disk I / O rate are G1, G2 and G3 respectively, then the load value can be determined to be 45%G1+30%G2+25%G3.

[0070] It should be noted that the above examples are only for illustrative purposes and are not intended to limit the present disclosure.

[0071] The target service node may be a service node selected to run the task to be processed.

[0072] As a possible implementation method, each service node may be sorted according to the load value, and then the service node with the smallest load value may be selected as the target service node.

[0073] For example, if the load value of service node 1 = 30%, the load value of service node 2 = 70%, the load value of service node 3 = 50%, and the load value of service node 4 = 20%, then service node 4 may be used as the target service node.

[0074] As another possible implementation method, in order to achieve load balancing, some service nodes with moderate loads may be selected. Therefore, in the embodiment of the present disclosure, a load value range may be predetermined, and then the service node with the smallest load value may be selected from the load value range as the target service node.

[0075] For example, if the load value range is 65%-80%, there are 3 service nodes within the load value range, and the load value of service node 1 is the smallest at 66%, service node 1 can be used as the target service node.

[0076] In this way, it is possible to avoid selecting service nodes that are too idle or overloaded, and to maintain the overall stability of the system by selecting service nodes with moderate loads.

[0077] Step 104: Run the pending task based on the target service node.

[0078] Optionally, the target service node can allocate corresponding resources to the task to be processed according to the resource requirement information of the task to be processed. Then, the task execution engine in the target service node starts to execute the task and generates a task execution result.

[0079] Optionally, the device may update the remaining resources of the target service node when the pending task is completed.

[0080] The remaining resources refer to the resources that are not fully used by the service node during its current operation, and the remaining resources have not yet been allocated to a specific task.

[0081] For example, if the remaining resources of the target service node before executing the pending task are "6 CPU cores, 15G memory, 20G hard disk", after the pending task is completed, the resources occupied by the pending task are released, and the remaining resources corresponding to the target service node are still "6 CPU cores, 15G memory, 20G hard disk", and no limitation is made here.

[0082] Specifically, after the pending task is completed, the target service node can release the resources previously allocated to the pending task. The released resources will become available resources for other tasks again. The update of remaining resources helps the device better understand the current resource usage and provide a reference for subsequent task scheduling and resource allocation.

[0083] In the disclosed embodiment, the pending task is first obtained, then based on the resource demand information corresponding to the pending task, a service node list is obtained, then the target service node is determined according to the load value of the service node in the service node list, and finally the pending task is run based on the target service node. Thus, by determining the target service node according to the resource demand of the pending task and the load of the service node, it can be ensured that the task is more reasonably allocated. Since the load of the service node is taken into account when determining the target service node, the failure or slow execution of the task due to excessive load can be avoided.

[0084] Figure 2 It is a flowchart of a method for processing a task provided according to the second embodiment of the present disclosure.

[0085] like Figure 2 As shown, the method includes:

[0086] Step 201 : in response to receiving a second task processing request, adding a task to a second task queue, wherein the second task processing request includes the task and corresponding resource requirement information.

[0087] The second task processing request is used to request processing of any task without a configured priority. That is, if the second task processing request does not contain the priority configuration information of the task, the task can be placed in the second task queue, indicating that the urgency of the task is low and the task processing time requirement is not high.

[0088] The resource requirement information is used to describe the resources required to process any task.

[0089] The task information may include task identification, task title, task purpose, task target, task content, etc., which are not limited here.

[0090] For example, for task 5, when processing task 5, at least 10 central processing unit (CPU) cores, 20G memory, and 30G hard disk are needed. Then "10 CPU cores, 20G memory, 30G hard disk" is the resource requirement information corresponding to task 5, which is not limited here.

[0091] Step 202, when the first task queue is an empty queue, obtain the to-be-processed tasks from the second task queue, wherein the priority of the tasks in the first task queue is higher than the priority of the tasks in the second task queue.

[0092] Among them, the relevant description of the first task queue can refer to the above embodiment and will not be repeated here.

[0093] It should be noted that the priority of the tasks in the first task queue is higher than the priority of the tasks in the second task queue. Usually, the first task queue is used to store tasks that need to be processed urgently, and the second task queue is used to store tasks that can be processed later and have a lower time urgency. Therefore, it is necessary to give priority to the tasks in the first task queue, and then process the tasks in the second task queue. When there are no tasks in the first task queue, it can be considered that the first task queue is an empty queue, and the device can obtain the tasks to be processed from the second task queue.

[0094] Optionally, when obtaining the to-be-processed task from the second task queue, any task may be selected as the to-be-processed task.

[0095] Alternatively, the task that is first placed in the second task queue may be prioritized as the task to be processed according to the first-in-first-out (FIFO) rule. According to the time when each task is placed in the second task queue, the task that enters the queue earliest will be processed first.

[0096] Step 203: Based on the resource requirement information of the task to be processed and the remaining resources corresponding to each service node in the service node cluster, it is determined whether there is a first service node in the service node cluster.

[0097] The first service node is a service node that can run the task to be processed.

[0098] For example, if the current remaining resources of the service node can meet the running needs of the pending task, that is, they can match the resource requirement information of the pending task, then the service node can be used as the first service node, which can avoid performance degradation or execution failure due to insufficient resources.

[0099] The service node cluster may be a resource pool of service nodes, wherein the resource pool includes multiple service nodes, and resource configuration information corresponding to different service nodes may be the same or different.

[0100] As a possible implementation method, the configuration information of each service node can be loaded, such as the number of CPU cores, CPU main frequency, memory capacity, hard disk size and hard disk type, etc., and a resource pool of the service node, that is, a service node cluster, can be generated according to the loaded configuration information.

[0101] Optionally, the maximum load value of each service node may be initialized. The maximum load value indicates the maximum load that the service node can bear. By setting a reasonable maximum load value, appropriate measures (such as limiting the allocation of new tasks) can be taken when the value is reached, thereby reducing the risk of failure.

[0102] For example, for service node x, the corresponding maximum number of CPU cores may be 2, that is, when service node x performs task processing, the maximum number of CPU cores used does not exceed 2, and is not limited here.

[0103] The maximum load values ​​corresponding to different service nodes may be the same or different.

[0104] Among them, the remaining resources refer to the resources that are not fully used by the service node at present, and the remaining resources have not yet been allocated to specific tasks. The remaining resources reflect the resources that the service node can currently allocate. By monitoring the remaining resources of the service node, you can understand which service nodes are capable of running pending tasks.

[0105] Optionally, in the disclosed embodiment, Prometheus may be used to monitor the remaining resources.

[0106] Among them, Prometheus is a monitoring and alarm system designed for modern cloud-native environments. By deploying monitoring agents on each node, it provides real-time monitoring data and analysis results to avoid system failures and performance bottlenecks.

[0107] For example, if the remaining resources of service node x are "6 CPU cores, 15G memory, 20G hard disk", the resource requirement information of the pending task P is "4 CPU cores, 11G memory, 10G hard disk". Since the remaining resources of service node x are relatively abundant relative to the resource requirements of the pending task P, service node x can allocate sufficient resources to run the pending task P. Therefore, service node x can be used as the first service node corresponding to the pending task P, which is not limited here.

[0108] Optionally, the device may traverse the service node cluster, select service nodes from the service node cluster one by one for analysis, and check whether the remaining resources thereof can meet the resource requirements of the task to be processed.

[0109] For example, first select the first service node to check whether its remaining CPU core number, memory capacity, hard disk space and other resources are greater than or equal to the resources required for the task to be processed. For each service node, compare its remaining resources with the resource requirements of the task to be processed. If the remaining CPU core number of the service node is not less than the number of CPU cores required for the task, the remaining memory capacity is greater than or equal to the memory capacity required for the task, the remaining hard disk space is greater than or equal to the hard disk space required for the task, etc., then it is considered that the remaining resources of the service node can meet the running needs of the task to be processed, and it can be used as the first service node, without limitation here.

[0110] Step 204: if there is a first service node in the service node cluster, construct a service node list according to the first service node.

[0111] The number of first service nodes in the service node list may be one or more, which is not limited here.

[0112] Specifically, all first service nodes can be put into the same list to form a service node list. The first service nodes in the service node list are all service nodes with sufficient remaining resources and capable of supporting the running of the tasks to be processed.

[0113] Step 205 : When the first service node does not exist in the service node cluster and the task to be processed is obtained from the second task queue, determine the current scheduling number corresponding to the task to be processed.

[0114] It is understandable that if the first service node does not exist in the service node cluster, it means that no service node currently meets the running conditions of the task to be processed and the running resource requirements of the task to be processed cannot be met. Therefore, the device cannot process the task to be processed temporarily.

[0115] In this case, the source of the pending task can be determined first. For example, if the pending task is obtained from the second task queue, the current scheduling number of the pending task can be determined.

[0116] The scheduling number indicates the frequency at which the task to be processed is scheduled. For example, when a task is taken out of the task queue, the scheduling number increases by one.

[0117] For example, if the pending task M is a task in the second task queue, and is taken out from the second task queue a total of three times, but is not matched to a suitable first service node to run, the scheduling number corresponding to the pending task M is three times, which is not limited here.

[0118] As another possible implementation method, when the first service node does not exist in the service node cluster and the pending task is obtained from the first task queue, the pending task is put back into the first task queue and the priority of the pending task in the first task queue is adjusted.

[0119] It should be noted that if the pending task belongs to the first task queue and the first service node capable of processing the pending task is not found, the pending task can be put back into the first task queue to wait for scheduling again.

[0120] Optionally, when putting the pending task back into the first task queue, the priority of the pending task may be adjusted, for example, the priority of the pending task may be lowered.

[0121] For example, the priority of the pending task may be lowered by one level, or the priority of the pending task may be lowered to the lowest level, which is not limited here.

[0122] It should be noted that if the first service node does not exist in the service node cluster, it means that there is currently no service node that can execute the pending task. In this case, by lowering the priority of the pending task, other tasks can be scheduled and executed, thereby avoiding a backlog of tasks in the queue and affecting system performance.

[0123] Step 206, when the scheduling number is greater than a preset threshold, add the pending task to the first task queue and configure the priority of the pending task in the first task queue, wherein the priority of the task in the first task queue is higher than the priority of the task in the second task queue.

[0124] Among them, the preset threshold value can be a threshold value of the number of scheduling times, and the specific size can be set according to actual experience and scenarios, and is not limited here.

[0125] For example, the preset threshold is 2. If the scheduling number corresponding to the pending task K is 3, which is greater than the preset threshold 2, the pending task can be added to the first task queue, which is not limited here.

[0126] Specifically, when configuring the priority of the pending task in the first task queue, a higher priority or a moderate priority may be assigned to the pending task, which is not limited here.

[0127] It is understandable that if a task has not been executed after multiple scheduling, then the task is still in the second task queue, which may cause the task to be shelved, resulting in more and more serious task squeezing. Therefore, it is necessary to put the task into the first task queue to prompt the task to be scheduled for execution as soon as possible to prevent task accumulation. By increasing its priority, the device can pay more attention to the task and allocate the required resources to it first, thereby improving the success rate of task execution.

[0128] Step 207: Score each service node according to the load value corresponding to each service node in the service node list and the type of each service node.

[0129] The types of service nodes can be pre-classified according to the functions, uses, performance characteristics, resource configuration characteristics, etc. corresponding to the service nodes, such as database server nodes, Web server nodes, and application server nodes, which are not limited here. It should be noted that there are many ways to classify the types of service nodes, which are not limited here.

[0130] It is understandable that different types of service nodes may have very different tasks and workload characteristics. For example, a database server may mainly face a large number of data read and write operations and complex query processing, while a Web server mainly handles HTTP (Hypertext Transfer Protocol) requests and page generation. Different task characteristics will cause different types of service nodes to show different performance and importance under the same load value. In addition, different types of service nodes may have different resource configurations. Different resource configurations affect the performance and applicable scenarios of service nodes. High-configuration service nodes can handle more complex and heavy tasks, while low-configuration service nodes are suitable for some simple tasks.

[0131] It is understandable that if a unified mapping relationship is used to score all types of service nodes, in some cases, the particularities of different types of service nodes may be ignored, resulting in inaccurate scoring. Type-based scoring can more specifically consider the characteristics of each type of service node, thereby improving the accuracy and rationality of the scoring.

[0132] Optionally, you can first obtain the mapping relationship corresponding to each service node based on the type of each service node, and the mapping relationship is used to record the mapping relationship between the load value and the score of the service nodes of the same type. Then, based on the mapping relationship corresponding to each service node, determine the first score corresponding to the load value of each service node. Finally, normalize the first scores of each service node to obtain the second scores corresponding to each service node.

[0133] It is understandable that for each type of service node, a mapping relationship between load value and score can be pre-established. This mapping relationship can be pre-determined through empirical data, performance testing or historical data analysis.

[0134] The first score may be a score determined directly by the load value of the service node according to the mapping relationship corresponding to the type of each service node. This score takes into account the corresponding relationship between the load value and the score in the same type of service node, without cross-type comparison and adjustment.

[0135] For example, for a service node of a database server type, according to its specific mapping relationship, if the load value of a service node corresponds to a score of 80 points, then 80 points is the first score of the service node.

[0136] The second score is the score obtained after normalization based on the first score. The purpose of normalization is to make the scores of different types of service nodes comparable and to unify the scores of all service nodes within a standard range. For example, after normalization, the first scores of different types of service nodes may be distributed in different intervals, but after processing, they are all adjusted to the range of 0-100 points or 0-1, so that the relative performance of different types of service nodes can be compared more intuitively.

[0137] For example, if the service node list contains 6 service nodes, they are recorded as Q1, Q2, Q3, Q4, Q5 and Q6. Among them, Q1, Q2, Q3 belong to type A, Q4, Q5 and Q6 belong to type B. Among them, the first score corresponding to type A ranges from 0 to 20. The first score corresponding to type B ranges from 0 to 50. The load values ​​corresponding to Q1, Q2, Q3, Q4, Q5 and Q6 are 20, 30, 60, 44, 54 and 87 respectively.

[0138] For example, based on the mapping relationship corresponding to type A, the first scores of Q1, Q2, and Q3 are 16 points, 14 points, and 8 points respectively. Based on the mapping relationship corresponding to type B, the first scores of Q4, Q5, and Q6 are 28 points, 23 points, and 6.5 points respectively. Then, the first scores of each service node are normalized, for example, mapped to a score range of 0-100, and the second scores of Q1, Q2, and Q3 are 80, 70, and 40 points respectively. The second scores of Q4, Q5, and Q6 are 56 points, 46 points, and 13 points respectively. By comparison, it can be determined that Q1 has the highest second score.

[0139] It should be noted that the above examples are only exemplary descriptions and are not intended to limit the present disclosure.

[0140] Step 208: Use the service node with the highest score as the target service node.

[0141] Specifically, when the score of any service node is the highest among all service nodes, the service node can be used as a target service node for executing the task to be processed.

[0142] For example, if the service nodes are Q1, Q2, Q3 and Q4, the corresponding scores are 35, 45, 60 and 90. Q4 has the highest score, so Q4 can be used as the target service node without limitation.

[0143] Step 209: Run the pending task based on the target service node.

[0144] It should be noted that the specific implementation of step 209 can refer to the above embodiment and will not be described in detail here.

[0145] In an embodiment of the present disclosure, in response to receiving a second task processing request, the device adds the task to the second task queue, and when the first task queue is an empty queue, obtains the task to be processed from the second task queue, wherein the priority of the task in the first task queue is higher than the priority of the task in the second task queue, and based on the resource demand information of the task to be processed and the remaining resources corresponding to each service node in the service node cluster, determines whether there is a first service node in the service node cluster, and the first service node is a service node that can run the task to be processed. When the first service node exists in the service node cluster, a service node list is constructed according to the first service node. When the first service node does not exist in the service node cluster and the task to be processed is obtained from the second task queue, the scheduling number currently corresponding to the task to be processed is determined. When the scheduling number is greater than a preset threshold, the task to be processed is added to the first task queue, and the priority of the task to be processed in the first task queue is configured. According to the load value corresponding to each service node in the service node list and the type of each service node, each service node is scored, and the service node with the highest score is used as the target service node. Finally, the task to be processed is run based on the target service node. Therefore, by distinguishing between the first task queue and the second task queue, high-priority tasks are prioritized, and tasks are dynamically allocated according to the remaining resources of the service node, optimizing resource utilization efficiency. Low-priority tasks are temporarily promoted to the high-priority queue through intelligent scheduling strategies, avoiding long waiting times for tasks and improving task processing flexibility. The service node load and type are combined for scoring, and the optimal service node is selected to execute tasks, achieving load balancing and improving the overall system performance and task processing speed.

[0146] As a possible application scenario, the disclosed embodiments can be applied to the scenario of terminal operating system OS compilation, involving the development and deployment of the operating system on the terminal device. The operating system usually needs to run on a variety of different terminal devices, including smart phones, tablet computers, smart TVs, etc. For these devices, the performance and stability of the operating system are crucial, so an efficient compilation method is needed to ensure the good performance of the software on specific hardware. When developers need to test or deploy an operating system, they hope to be able to compile the code onto the target device quickly and accurately. This can promptly discover and solve problems and improve development efficiency.

[0147] In the disclosed embodiments, the task processing method can be applied to the processing of compilation tasks in the terminal OS compilation scenario, thereby achieving smooth scheduling of compilation tasks, while ensuring the task compilation efficiency, maximizing the utilization of the resources of the entire compilation server cluster, avoiding resource waste and node overload problems, and ensuring the stability and efficiency of the compilation process.

[0148] In order to better implement the task processing method of the present disclosure, the present disclosure also provides a task processing device based on the task processing method. The meanings of the terms are the same as those in the task processing method, and the specific implementation details can refer to the description in the method embodiment.

[0149] See also Figure 3 , Figure 3 3 is a schematic diagram of the structure of a task processing device provided in an embodiment of the present disclosure. The task processing device 300 includes:

[0150] A first acquisition module 310 is used to acquire tasks to be processed;

[0151] A second acquisition module 320 is used to acquire a service node list based on the resource demand information corresponding to the task to be processed;

[0152] A determination module 330, configured to determine a target service node according to a load value of a service node in the service node list;

[0153] The running module 340 is used to run the task to be processed based on the target service node.

[0154] Optionally, the second acquisition module includes:

[0155] A first judgment unit is used to judge whether there is a first service node in the service node cluster based on the resource requirement information of the task to be processed and the remaining resources corresponding to each service node in the service node cluster, where the first service node is a service node that can run the task to be processed;

[0156] A constructing unit is configured to construct the service node list according to the first service node when the first service node exists in the service node cluster.

[0157] Optionally, the first judgment unit is further configured to:

[0158] If the first service node does not exist in the service node cluster and the to-be-processed task is obtained from a second task queue, determining a current scheduling number corresponding to the to-be-processed task;

[0159] When the scheduling number is greater than a preset threshold, the task to be processed is added to the first task queue, and the priority of the task to be processed in the first task queue is configured.

[0160] The priority of the tasks in the first task queue is higher than the priority of the tasks in the second task queue.

[0161] Optionally, the first judgment unit is further configured to:

[0162] In the case where the first service node does not exist in the service node cluster and the to-be-processed task is obtained from a first task queue,

[0163] The to-be-processed task is put back into the first task queue, and the priority of the to-be-processed task in the first task queue is adjusted.

[0164] Optionally, the determining module includes:

[0165] A scoring unit, configured to score each service node according to a load value corresponding to each service node in the service node list and a type to which each service node belongs;

[0166] A determining unit is used to select the service node with the highest score as the target service node.

[0167] Optionally, the scoring unit is specifically used to:

[0168] According to the type of each service node, a mapping relationship corresponding to each service node is obtained, where the mapping relationship is used to record the mapping relationship between the load value and the score of the service node of the same type;

[0169] Determine a first score corresponding to the load value of each service node based on the mapping relationship corresponding to each service node;

[0170] The first scores of the service nodes are normalized to obtain second scores corresponding to the service nodes.

[0171] Optionally, the first acquisition module includes:

[0172] A first acquisition unit, used to acquire the task with the highest priority from the first task queue as a task to be processed;

[0173] The second acquisition unit is used to acquire the to-be-processed tasks from the second task queue when the first task queue is an empty queue, wherein the priority of the tasks in the first task queue is higher than the priority of the tasks in the second task queue.

[0174] Optionally, the first acquisition module is further used to:

[0175] In response to receiving a first task processing request, adding a task to the first task queue, wherein the first task processing request includes the task, priority configuration information corresponding to the task, and resource requirement information;

[0176] or,

[0177] In response to receiving a second task processing request, adding the task to the second task queue, wherein the second task processing request includes the task and corresponding resource requirement information.

[0178] Optionally, the run module is also used to:

[0179] In response to the completion of the execution of the pending task, the remaining resources of the target service node are updated.

[0180] In the disclosed embodiment, the pending task is first obtained, then based on the resource demand information corresponding to the pending task, a service node list is obtained, then the target service node is determined according to the load value of the service node in the service node list, and finally the pending task is run based on the target service node. Thus, by determining the target service node according to the resource demand of the pending task and the load of the service node, it can be ensured that the task is more reasonably allocated. Since the load of the service node is taken into account when determining the target service node, the failure or slow execution of the task due to excessive load can be avoided.

[0181] In addition, the present disclosure also provides an electronic device, such as Figure 4 As shown, it shows a schematic diagram of the structure of the electronic device involved in the present disclosure, specifically:

[0182] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0183] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0184] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0185] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0186] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0187] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby implementing the steps in the processing method of any task provided in the embodiments of the present disclosure.

[0188] In the disclosed embodiment, the pending task is first obtained, then based on the resource demand information corresponding to the pending task, a service node list is obtained, then the target service node is determined according to the load value of the service node in the service node list, and finally the pending task is run based on the target service node. Thus, by determining the target service node according to the resource demand of the pending task and the load of the service node, it can be ensured that the task is more reasonably allocated. Since the load of the service node is taken into account when determining the target service node, the failure or slow execution of the task due to excessive load can be avoided.

[0189] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0190] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0191] To this end, the present disclosure provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in the processing method of any task provided by the present disclosure.

[0192] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0193] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0194] Since the instructions stored in the computer-readable storage medium can execute the steps in the processing method of any task provided in the present disclosure, the beneficial effects that can be achieved by the processing method of any task provided in the present disclosure can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0195] The above is a detailed introduction to a task processing method, device, electronic device and computer-readable storage medium provided by the present disclosure. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A task processing method, characterized in that: include: Get pending tasks; Based on the resource demand information corresponding to the task to be processed, obtain a service node list; Determine a target service node according to the load values ​​of the service nodes in the service node list; Based on the target service node, the task to be processed is executed.

2. The method according to claim 1, characterized in that The obtaining of a service node list based on the resource requirement information of the task to be processed includes: Based on the resource requirement information of the task to be processed and the remaining resources corresponding to each service node in the service node cluster, determining whether there is a first service node in the service node cluster, where the first service node is a service node that can run the task to be processed; In a case where the first service node exists in the service node cluster, the service node list is constructed according to the first service node.

3. The method according to claim 2, characterized in that After determining whether there is a first service node in the service node cluster that can run the task to be processed, the method further includes: If the first service node does not exist in the service node cluster and the to-be-processed task is obtained from a second task queue, determining a current scheduling number corresponding to the to-be-processed task; When the scheduling number is greater than a preset threshold, the task to be processed is added to the first task queue, and the priority of the task to be processed in the first task queue is configured. The priority of the tasks in the first task queue is higher than the priority of the tasks in the second task queue.

4. The method according to claim 2, characterized in that: After determining whether there is a first service node in the service node cluster that can run the task to be processed, the method further includes: In the case where the first service node does not exist in the service node cluster and the to-be-processed task is obtained from a first task queue, The to-be-processed task is put back into the first task queue, and the priority of the to-be-processed task in the first task queue is adjusted.

5. The method according to claim 1, characterized in that: The determining the target service node according to the load value of the service node in the service node list includes: Scoring each service node according to the load value corresponding to each service node in the service node list and the type of each service node; The service node with the highest score is used as the target service node.

6. The method according to claim 5, characterized in that Scoring each service node according to the load value corresponding to each service node in the service node list and the type of each service node includes: According to the type of each service node, a mapping relationship corresponding to each service node is obtained, where the mapping relationship is used to record the mapping relationship between the load value and the score of the service node of the same type; Determine a first score corresponding to the load value of each service node based on the mapping relationship corresponding to each service node; The first scores of the service nodes are normalized to obtain second scores corresponding to the service nodes.

7. The method according to claim 1, characterized in that The obtaining of tasks to be processed includes: Get the task with the highest priority from the first task queue as the task to be processed; or, When the first task queue is an empty queue, a to-be-processed task is obtained from a second task queue, wherein the priority of the tasks in the first task queue is higher than the priority of the tasks in the second task queue.

8. The method according to claim 7, characterized in that Also includes: In response to receiving a first task processing request, adding a task to the first task queue, wherein the first task processing request includes the task, priority configuration information corresponding to the task, and resource requirement information; or, In response to receiving a second task processing request, adding the task to the second task queue, wherein the second task processing request includes the task and corresponding resource requirement information.

9. The method according to claim 1, characterized in that: After the task to be processed is run based on the target service node, the method further includes: In response to the completion of the execution of the pending task, the remaining resources of the target service node are updated.

10. A task processing device, characterized in that: include: A first acquisition module, used to acquire tasks to be processed; A second acquisition module is used to acquire a service node list based on the resource demand information corresponding to the task to be processed; A determination module, configured to determine a target service node according to a load value of a service node in the service node list; A running module is used to run the task to be processed based on the target service node.

11. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the method according to any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 9 are implemented.

Images