Task scheduling method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, in particular to a task scheduling method and device, electronic equipment and a storage medium, and the method comprises the following steps: determining a plurality of candidate resources from preset resources; performing performance analysis on the plurality of candidate resources to obtain resource performance information corresponding to the plurality of candidate resources; performing data fusion processing on the resource performance information of each candidate resource and the label matching degree of the target task and each candidate resource to obtain the association degree of the target task and each candidate resource; based on the association degree between the target task and each candidate resource, determining a target resource from the plurality of candidate resources; and allocating the target task to the target resource. According to the method, the task allocation efficiency can be improved, meanwhile, load balancing of all the resources is achieved, and then the resource utilization rate and the task processing efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task scheduling method, apparatus, electronic device, and storage medium. Background Technology

[0002] In existing technologies, task scheduling typically uses message queues to smooth out peak and valley loads and decouple tasks, with resources passively acquiring tasks. Since tasks often have multiple dimensions of characteristics—for example, N dimensions—at least N message queues are needed to handle the allocation, resulting in low task allocation efficiency. Furthermore, because resources passively receive tasks, processing too few tasks leads to underutilization, while processing too many tasks leads to resource overload and slow task processing speed. This can result in a situation where some resources are busy while others are idle or there are insufficient tasks. Summary of the Invention

[0003] The technical problem to be solved by this application is to provide a task scheduling method, apparatus, electronic device and storage medium that can improve task allocation efficiency while achieving load balancing of various resources, thereby improving resource utilization and task processing efficiency.

[0004] To address the aforementioned technical problems, this application provides a task scheduling method, comprising:

[0005] Based on the task type tag of the target task and the resource tags of the preset resources, a resource search is performed on the preset resources to determine multiple candidate resources; the resource tag of each preset resource represents the task type supported by each preset resource, and the resource tag of each candidate resource includes the task type tag of the target task;

[0006] Based on the operational indicator information of the multiple candidate resources, a performance analysis is performed on the multiple candidate resources to obtain the resource performance information corresponding to each of the multiple candidate resources;

[0007] Tag matching is performed based on the task type tag of the target task and the resource tags of the multiple candidate resources to obtain the tag matching degree between the target task and each candidate resource;

[0008] The resource performance information of each candidate resource and the tag matching degree between the target task and each candidate resource are fused to obtain the correlation degree between the target task and each candidate resource.

[0009] Based on the correlation between the target task and each candidate resource, the target resource is determined from the plurality of candidate resources;

[0010] Assign the target task to the target resource.

[0011] On the other hand, this application provides a task scheduling apparatus, including:

[0012] The resource search module is used to search for the preset resources based on the task type tag of the target task and the resource tags of the preset resources, and to determine multiple candidate resources from the preset resources; the resource tag of each preset resource represents the task type supported by each preset resource, and the resource tag of each candidate resource includes the task type tag of the target task;

[0013] The performance analysis module is used to perform performance analysis on the multiple candidate resources based on the operation index information of the multiple candidate resources, and obtain the resource performance information corresponding to each of the multiple candidate resources.

[0014] The tag matching module is used to perform tag matching based on the task type tag of the target task and the resource tags of the multiple candidate resources to obtain the tag matching degree between the target task and each candidate resource;

[0015] The correlation determination module is used to perform data fusion processing on the resource performance information of each candidate resource and the label matching degree between the target task and each candidate resource to obtain the correlation degree between the target task and each candidate resource.

[0016] The target resource determination module is used to determine the target resource from the plurality of candidate resources based on the correlation between the target task and each candidate resource;

[0017] The task allocation module is used to allocate the target task to the target resource.

[0018] On the other hand, this application provides an electronic device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the task scheduling method described above.

[0019] On the other hand, this application provides a computer storage medium storing at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded by a processor and executed as described above in the task scheduling method.

[0020] On the other hand, this application provides a computer program product including computer instructions stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the task scheduling method described above.

[0021] Implementing the embodiments of this application has the following beneficial effects:

[0022] In this application, resource search is performed based on the task type tag of the target task and the resource tags of preset resources to identify multiple candidate resources. This tag-based pre-screening narrows down the range of resources to be processed. When the target task has multiple dimensions of task characteristics, suitable candidate resources can be determined from the preset resources based on the task type tag of the target task. Furthermore, resource performance information and tag matching degree are combined to determine the target resource from the candidate resources for processing the target task. There is no need to set up a corresponding message queue for each dimension to solve the task allocation problem. The task type tag of the target task and the resource tags of the preset resources can be matched directly. On this basis, the resource performance information of the candidate resources is combined to determine the final target resource. This can improve the efficiency of task allocation and achieve load balancing of each resource, thereby improving resource utilization and task processing efficiency. Attached Figure Description

[0023] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the implementation environment provided in the embodiments of this application;

[0025] Figure 2 This is a flowchart of a task scheduling method provided in an embodiment of this application;

[0026] Figure 3 This is a flowchart of a method for processing resource performance information of candidate resources provided in an embodiment of this application;

[0027] Figure 4 This is a flowchart of a tag matching degree determination method provided in an embodiment of this application;

[0028] Figure 5 This is a flowchart of a candidate resource determination method based on resource specification information provided in an embodiment of this application;

[0029] Figure 6 This is a flowchart of the rendering task scheduling method provided in the embodiments of this application;

[0030] Figure 7 This is a schematic diagram illustrating the matching of target tasks and resources provided in an embodiment of this application;

[0031] Figure 8 This is a schematic diagram of a task scheduling device provided in an embodiment of this application;

[0032] Figure 9 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0035] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0036] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0037] Please see Figure 1 The illustration shows an implementation environment provided in this application embodiment. The implementation environment may include: at least one electronic terminal 110, a task scheduling server 120, and a task processing cluster 130. The task processing cluster 130 is provided with multiple preset resources for task processing. The electronic terminal 110 and the task scheduling server 120 can communicate with each other via a network, and the task scheduling server 120 and the task processing cluster 130 can communicate with each other via a network. In one example, the task scheduling server 120 may also be integrated into the task processing cluster 130. This embodiment does not make specific limitations.

[0038] Specifically, the electronic terminal 110 can send a task processing request to the task scheduling server 120, which carries a task type tag for the target task. The task scheduling server 120 can determine suitable candidate resources from the preset resources of the task processing cluster 130 based on the task type tag of the target task. Furthermore, it combines resource performance information and tag matching degree to determine the target resource from the candidate resources for processing the target task. The task scheduling server 120 can then assign the target task to the target resource in the task processing cluster 130. After processing the target task, the target resource in the task processing cluster 130 can return the task processing result to the task scheduling server 120, so that the task scheduling server 120 can forward the task processing result to the electronic terminal 110. Alternatively, the target resource in the task processing cluster 130 can directly send the task processing result to the electronic terminal 110; this is not specifically limited here.

[0039] The task scheduling server 120 and the electronic terminal 110 can establish a communication connection via wired or wireless means. The task scheduling server 120 may include a stand-alone server, a distributed server, or a server cluster consisting of multiple servers, wherein the server may be a cloud server.

[0040] To address the issues of low task allocation efficiency and uneven resource load in existing task scheduling technologies, this application provides a task scheduling method. The execution entity of this method can be the aforementioned task scheduling server. For details, please refer to [link / reference needed]. Figure 2 The method may include:

[0041] S210. Based on the task type tag of the target task and the resource tag of the preset resources, perform resource search on the preset resources and determine multiple candidate resources from the preset resources; the resource tag of each preset resource represents the task type supported by each preset resource, and the resource tag of each candidate resource includes the task type tag of the target task.

[0042] In this embodiment, the target task may have a corresponding task type label. Each target task may have at least one task type label. Different task type labels may correspond to the task characteristics of the target task in different task dimensions. The task type label may indicate the task processing capabilities that should be available when processing a task carrying that task type label. For example, the task type label may be a label representing the processing engine required to process the target task, a label representing the business category of the target task, or a label representing a general category. The resource label of the preset resource may represent the task types supported by the preset resource. That is, the resource label of the preset resource may reflect the task processing capabilities of the preset resource. The resource label set formed by the resource labels of each preset resource may be the same as the task type label set formed by each task type label. For example, if the resource label set is {label1, label2, label3}, the task type label set may also be {label1, label2, label3}. The task processing cluster includes multiple computing nodes. Specifically, the preset resource may be multiple computing nodes in the task processing cluster, and each resource may be a computing node.

[0043] Given the task type tags of the target task, a resource search can be performed among the preset resources to find preset resources whose resource tags include the task type tags. These found preset resources are then designated as candidate resources. The number of resource tags for candidate resources can be greater than or equal to the number of task type tags of the target task, and the resource tags of the candidate resources include the task type tags of the target task. For example, if the target task's task type tags include tag 1 and tag 2, then the resource tags for candidate resource a will include tag 1 and tag 2, and the resource tags for candidate resource b will include tag 1, tag 2, and tag 3. Candidate resources are those capable of processing the target task. For instance, if the target task's task type tags include tag 1 and tag 2, it means that candidate resources need to be found that possess both the task processing capabilities corresponding to tag 1 and tag 2. Similarly, if candidate resource a's resource tags include tag 1 and tag 2, it means that candidate resource a possesses both the task processing capabilities corresponding to tag 1 and tag 2.

[0044] S220. Based on the operational indicator information of the multiple candidate resources, perform performance analysis on the multiple candidate resources to obtain the resource performance information corresponding to each of the multiple candidate resources.

[0045] In this embodiment, the operational metrics information for each resource may include multiple dimensions of operational metrics information, such as processor operational information, memory usage information, network transmission information, and disk usage information. Once the operational metrics information for each resource is obtained, the resource performance information corresponding to each candidate resource can be derived based on this information. Specifically, since the operational metrics information for each resource may change dynamically, in one example, the operational metrics information for multiple candidate resources can be obtained at preset periods to obtain the operational metrics information for multiple candidate resources within multiple preset periods. Then, based on the average value of the operational metrics information for each candidate resource within multiple preset periods, the resource performance information corresponding to each candidate resource can be obtained. In another example, the operational metrics information for each candidate resource can be obtained in real time, and then the resource performance information corresponding to each of the multiple candidate resources can be obtained based on the real-time obtained operational metrics information.

[0046] S230. Perform tag matching based on the task type tag of the target task and the resource tags of the multiple candidate resources to obtain the tag matching degree between the target task and each candidate resource.

[0047] As described above, each candidate resource's resource tags include the target task's task type tags. Some candidate resources have the same number of resource tags as the target task's task type tags, while others have more. When the weights of each resource tag in a candidate resource are equal (e.g., each resource tag has a weight of 1), the closer the number of resource tags is to the number of task type tags in the target task, the higher the matching degree between the candidate resource tags and the target task's task type tags. Conversely, the lower the matching degree, the less the matching degree. When the weights of each resource tag in a candidate resource are different, the matching degree between the candidate resource's resource tags and the target task's task type tags needs to be determined based on the weights corresponding to each candidate resource's resource tags and the number of resource tags in each candidate resource.

[0048] S240. Perform data fusion processing on the resource performance information of each candidate resource and the label matching degree between the target task and each candidate resource to obtain the correlation degree between the target task and each candidate resource.

[0049] Specifically, given the resource performance information of each candidate resource and the label matching degree between the target task and each candidate resource, a weighted sum of these two information can be performed to obtain the correlation degree between the target task and each candidate resource. Specifically, the weights corresponding to the resource performance information of each candidate resource and the label matching degree between the target task and each candidate resource can be determined based on the actual implementation situation; the weights corresponding to these two pieces of information can be the same or different.

[0050] S250. Based on the correlation between the target task and each candidate resource, determine the target resource from the plurality of candidate resources.

[0051] Based on the correlation between the target task and each candidate resource, the candidate resource with the highest correlation can be identified as the target resource. When there are multiple candidate resources with the highest correlation, in one example, the candidate resource with the largest resource performance information can be selected as the target resource from among the multiple candidate resources with the highest correlation; in another example, the candidate resource with a relatively high correlation can be selected as the target resource from among the multiple candidate resources with the highest correlation; in yet another example, the electronic terminal that sent the target task can be identified, and the historical processing information of the historical tasks of that electronic terminal can be obtained. The historical processing information includes a list of resources that have processed historical tasks. If one of the multiple candidate resources with the highest correlation exists in the list of resources that have processed historical tasks, that resource can be identified as the target resource.

[0052] S260. Assign the target task to the target resource.

[0053] In this embodiment, a corresponding task cache area can be created for each preset resource. Thus, when it is determined that a target task is assigned to a target resource, the target task can be placed in the task cache area corresponding to the target resource. The target resource can obtain the target task and process it by accessing its corresponding task cache area.

[0054] In this application, resource search is performed based on the task type tag of the target task and the resource tags of preset resources to identify multiple candidate resources. This tag-based pre-screening narrows down the range of resources to be processed. When the target task has multiple dimensions of task characteristics, suitable candidate resources can be determined from the preset resources based on the task type tag of the target task. Furthermore, resource performance information and tag matching degree are combined to determine the target resource from the candidate resources for processing the target task. There is no need to set up a corresponding message queue for each dimension to solve the task allocation problem. The task type tag of the target task and the resource tags of the preset resources can be matched directly. On this basis, the resource performance information of the candidate resources is combined to determine the final target resource. This can improve the efficiency of task allocation and achieve load balancing of each resource, thereby improving resource utilization and task processing efficiency.

[0055] In this embodiment, when performing resource searching, abnormal resources can be identified first, and then resource searching can be performed based on the abnormal resource identification results to obtain candidate resources; specifically,

[0056] The method involves searching for preset resources based on the task type tag of the target task and the resource tags of preset resources, and identifying multiple candidate resources from the preset resources, including:

[0057] Abnormal resources are identified from the preset resources; at least one operational indicator information corresponding to the abnormal resource satisfies the preset abnormal conditions;

[0058] Based on the task type tag of the target task and the resource tag of the preset resources, a resource search is performed in the preset resources other than the abnormal resources to obtain the multiple candidate resources.

[0059] In this embodiment, the operational metrics information for each resource may include multiple dimensions of operational metrics information, such as processor operation information, memory usage information, network transmission information, disk usage information, port information, and whether the number of tasks in the resource matches the number of resource cores. For any resource, if any one of the following operational metrics information—processor operation information, memory usage information, network transmission information, disk usage information, port information, or whether the number of tasks in the resource matches the number of resource cores—meets a preset abnormal condition, the resource can be identified as an abnormal resource. During this resource search process, abnormal resources are excluded from the search scope. Furthermore, when searching for resources based on the task type of the target task and the preset resource tags, resource searches can be performed among the preset resources other than abnormal resources, resulting in multiple candidate resources.

[0060] For example, the target task's task type tags include tag 1 and tag 2. Correspondingly, the resource tags for candidate resource 'a' include tag 1 and tag 2, and the resource tags for candidate resource 'b' include tag 1, tag 2, and tag 3. Candidate resources are those capable of processing the target task. For instance, the target task's task type tags including tag 1 and tag 2 indicate that we need to find candidate resources that possess both the task processing capabilities corresponding to tag 1 and tag 2. Similarly, candidate resource 'a' having tag 1 and tag 2 means that candidate resource 'a' possesses both the task processing capabilities corresponding to tag 1 and tag 2.

[0061] Therefore, in this embodiment, before performing resource search based on tags, abnormal resources in the preset resources are first excluded, and resource search is performed in the other resources in the preset resources excluding abnormal resources to obtain candidate resources. On the one hand, by excluding abnormal resources, the resource range of resource search can be narrowed, candidate resources can be found quickly, and resource search efficiency can be improved. On the other hand, by excluding abnormal resources, the problem of abnormal resources being used to process the target task and causing the task processing to fail can be avoided.

[0062] In this embodiment, the operational metrics information for each resource includes processor operational information, memory usage information, network transmission information, and disk usage information for each resource. Correspondingly, the performance analysis of the multiple candidate resources based on their operational metrics information to obtain resource performance information for each candidate resource includes:

[0063] Based on the processor running information, memory usage information, network transmission information, and disk usage information of each candidate resource, determine the processor information, memory performance information, network performance information, and disk performance information of each candidate resource;

[0064] The processor information, memory performance information, network performance information, and disk performance information of each candidate resource are fused to obtain the resource performance information corresponding to each candidate resource.

[0065] The processor running information, memory usage information, network transmission information, and disk usage information of each candidate resource are specific indicator data relative to themselves. The indicator data can be further processed to obtain indicator measurement data that can be used to compare different candidate resources, that is, to obtain the processor information, memory performance information, network performance information, and disk performance information of each candidate resource.

[0066] The processor information, memory performance information, network performance information, and disk performance information of each candidate resource can be weighted to obtain the resource performance information corresponding to each candidate resource. The weights of the processor information, memory performance information, network performance information, and disk performance information can be the same or different.

[0067] Specifically, the formula for calculating processor performance information is shown below:

[0068]

[0069] Among them, CPUCapacity represents the total CPU capacity, CPUUsed represents the CPU utilization rate, and CPUScore represents processor performance information, specifically the CPU score.

[0070] The formula for calculating memory performance information is shown below:

[0071]

[0072] Among them, MemoryCapacity represents the total memory capacity, MemoryUsed represents the memory usage rate, and MemoryScore represents memory performance information, which can be specifically a memory score.

[0073] The calculation formula for network performance information is shown below:

[0074]

[0075] Among them, RxKb represents the number of bytes received per second, TxKb represents the number of bytes sent per second, Speed ​​represents the maximum transmission size, and NetScore represents network performance information, which can be specifically a network score.

[0076] The formula for calculating disk performance information is shown below:

[0077]

[0078] Among them, DiskCapacity represents the total disk capacity, DiskUsed represents the disk usage rate, and DiskScore represents disk performance information, which can be specifically a disk score.

[0079] Assuming that processor information, memory performance information, network performance information, and disk performance information each have the same weight, the calculation formula for the resource performance information of each candidate resource is as follows:

[0080]

[0081] NodeScore represents the resource performance information of candidate resources, specifically a resource score.

[0082] In this embodiment, for each candidate resource, the performance information corresponding to the multiple dimensions of the operational indicator information is determined. Then, the performance information corresponding to the multiple dimensions of the operational indicator information is fused to obtain the resource performance information corresponding to each candidate resource. The resource performance information corresponding to each candidate resource can measure the operational status of the candidate resource, which facilitates subsequent task allocation based on the operational status of each candidate resource, so as to achieve the adaptation of task allocation to resource operational status.

[0083] When performing data fusion processing on the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource, a weighted average can be calculated by combining the weights corresponding to the performance information of each dimension to obtain the resource performance information corresponding to the respective candidate resource; for details, please refer to [link to relevant documentation]. Figure 3 It illustrates a method for processing resource performance information of candidate resources, which may include:

[0084] S310. Based on the task type label of the target task, determine the importance of the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource.

[0085] Each task type tag may have varying degrees of dependence on processor, memory, network, and disk. Accordingly, based on the degree of dependence of each task type tag on these factors, the importance of the corresponding processor performance, memory performance, network performance, and disk performance information can be determined. Specifically, the higher the dependence of a task type tag on any one of these factors, the higher the importance of the performance information corresponding to that factor. A higher dependence of a task type tag on a particular factor indicates that this factor is given more attention when processing the target task carrying that task type tag, thus making the performance information corresponding to that factor more important to the target task carrying that task type tag. Based on the task feature information corresponding to each task type tag, the dependence of each task type tag on processor, memory, network, and disk can be analyzed, thereby determining the individual importance of the processor performance, memory performance, network performance, and disk performance information for each task type tag, i.e., establishing a correspondence between each task type tag and the respective importance of these factors. In real-world applications, some task type tags require high-performance processors, some require high-performance memory, some require high-performance networks, and some require high-performance disks.

[0086] Furthermore, once the target task is obtained, the importance of the processor performance information, memory performance information, network performance information, and disk performance information corresponding to the task type label in the target task can be determined based on the correspondence between the task type label and the processor performance information, memory performance information, network performance information, and disk performance information.

[0087] For example, the dependence of task type label 1 on processor, memory, network, and disk is x1, x2, x3, and x4, respectively, where x1, x2, x3, and x4 can all be decimals greater than 0 and less than or equal to 1. Correspondingly, the importance of processor performance information, memory performance information, network performance information, and disk performance information is x1, x2, x3, and x4, respectively. Specifically, the dependence of task type label 1 on processor, memory, network, and disk is 1, 0.8, 0.8, and 0.5, respectively, indicating that task type label 1 has the highest dependence on the processor and the lowest dependence on the disk. Correspondingly, the importance of the processor performance information, memory performance information, network performance information, and disk performance information corresponding to task type label 1 is 1, 0.8, 0.8, and 0.5, respectively, meaning that processor performance information is the most important and disk performance information is the least important.

[0088] S320. Based on the importance of the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource, determine the weight of the processor information, memory performance information, network performance information, and disk performance information of each candidate resource.

[0089] Then, based on their importance, corresponding weights can be determined. Specifically, the importance of the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource can be directly used to determine the weights corresponding to the processor information, memory performance information, network performance information, and disk performance information of each candidate resource. Alternatively, the importance of the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource can be normalized to obtain the weights corresponding to the processor information, memory performance information, network performance information, and disk performance information of each candidate resource.

[0090] S330. Based on the weights corresponding to the processor information, memory performance information, network performance information, and disk performance information of each candidate resource, perform weighted processing on the processor information, memory performance information, network performance information, and disk performance information of each candidate resource to obtain the resource performance information corresponding to each candidate resource.

[0091] In this embodiment, considering real-world application scenarios, some task type tags require high-performance processors, others require high-performance memory, some require high-performance network resources, and still others require high-performance disk resources. Based on the task type tags carried by the target task, the dependence of the target task on various operational metrics can be determined. Higher dependence indicates higher importance, and consequently, higher weight for the corresponding performance information. The calculated resource performance information for each candidate resource is relative to the target task's performance, reflecting the target task's dependence on operational metrics. This personalizes the resource performance information corresponding to the target task, improving the matching degree between the target task and each candidate resource's resource performance information. Furthermore, subsequent target resource selection based on matching resource performance information improves the accuracy of target resource determination.

[0092] In this embodiment, before performing tag matching based on the task type tag of the target task and the resource tags of the plurality of candidate resources to obtain the tag matching degree between the target task and each candidate resource, the method further includes:

[0093] Based on the resource feature information of each preset resource, the tag weight of the resource tag of each preset resource is determined;

[0094] In this embodiment, the resource feature information can specifically be the task processing capabilities of the preset resource, and / or the proportion of each task processing capability when the preset resource has multiple task processing capabilities. For any preset resource with multiple resource tags, it means that the preset resource has processing capabilities corresponding to multiple resource tags. The preset resource tag may have different degrees of bias towards multiple resource tags. The higher the degree of bias, the more inclined the preset resource is to process target tasks with resource tags with high bias (i.e., task type tags). It can also mean that the preset resource is better at using the processing capabilities corresponding to resource tags with high bias to process tasks. Correspondingly, the higher the degree of bias of the preset resource towards a certain resource tag, the greater the tag weight of that resource tag. Therefore, based on the resource feature information of each preset resource, the tag weight of the resource tag of each preset resource can be determined in advance. The tag weight of the same resource tag may be different or the same in different preset resources.

[0095] For example, the resource tags of preset resource b include tag 1, tag 2 and tag 3. The weights of tag 1, tag 2 and tag 3 are 1, 0.5 and 0.5 respectively. This means that preset resource b has the task processing capabilities corresponding to tag 1, tag 2 and tag 3. Preset resource b is more inclined to process the target task carrying tag 1.

[0096] Accordingly, please refer to Figure 4 It illustrates a method for determining label matching degree, which may include:

[0097] S410. Determine the matching tag that matches the task type tag from the resource tags of each candidate resource.

[0098] Since the resource tags of the candidate resources include the task type tags of the target task, the corresponding matching tags can be determined. Specifically, the matching tags are consistent with the task type tags.

[0099] S420. Based on the tag weights corresponding to the matching tags, determine the tag matching information between the target task and each candidate resource.

[0100] When there is only one matching tag, the tag matching information between the target task and each candidate resource can be determined based on the tag weight of the matching tag; when there are multiple matching tags, the tag matching information between the target task and each candidate resource can be determined based on the tag weight of the multiple matching tags.

[0101] S430. Based on the ratio of the tag matching information to the number of resource tags of each candidate resource, determine the tag matching degree between the target task and each candidate resource.

[0102] In this embodiment, each preset resource has a corresponding resource tag, and each resource tag may have a corresponding tag weight to reflect the degree of bias of each preset resource towards different resource tags. This enables the determination of the tag matching degree between the target task and each candidate resource based on the resource feature information of the candidate resources. The calculated tag matching degree is determined based on the tag weight of the resource tags of the candidate resources, thereby realizing personalized calculation of the tag matching degree and improving the accuracy and flexibility of the tag matching degree calculation.

[0103] In this embodiment, determining the label matching information between the target task and each candidate resource based on the label weight corresponding to the matching label includes:

[0104] When there are multiple matching tags, the tag weights corresponding to each of the multiple matching tags are summed to obtain the tag matching information between the target task and each candidate resource.

[0105] The formula for calculating tag matching degree is as follows:

[0106]

[0107] in, This represents the label matching information between the target task and each candidate resource, where n represents the number of matching labels, and weight... i is the weight of any matching tag. When there are multiple matching tags, n is an integer greater than 1; CountWeight is the number of resource tags for each candidate resource; and NodeAffinityScore is the tag matching degree.

[0108] For example, the task type labels of the target task include label 1 and label 2, and the task type labels of candidate resource b include label 1, label 2, and label 3. Correspondingly, the matching labels include label 1 and label 2. The label weight corresponding to label 1 is y1, the weight corresponding to label 2 is y2, and the weight corresponding to label 3 is y3. Therefore, the label matching degree between the target task and candidate resource b is:

[0109]

[0110] In the above formula, "0" indicates that there is no tag matching tag 3 in the task type tags of the target task.

[0111] In this embodiment, when there are multiple matching tags, the tag weights corresponding to each of the multiple matching tags can be summed to obtain the tag matching information between the target task and each candidate resource, thus realizing the determination of tag matching information in a multi-matching tag scenario.

[0112] In this embodiment, the target task can also specify resources, and correspondingly, candidate resources can be determined based on the specified resources; please refer to the following for details. Figure 5 It illustrates a method for determining candidate resources based on resource specification information, which may include:

[0113] S510. Obtain the task information of the target task; the task information includes the task type label and resource specification information.

[0114] In this embodiment, the target task may have a corresponding task type label. Each target task may have at least one task type label. Different task type labels may correspond to the task characteristics of the target task in different task dimensions. The task type label may indicate the task processing capability that should be available when processing a task carrying the task type label. For example, the task type label may be a label that represents the processing engine required to process the target task, or it may be a label that represents the business category of the target task, or it may be a label that represents a general category, etc.

[0115] The resource specification information can correspond to one or more specified resources. Specifically, the resource specification information can include the resource identifier information of the specified resource, which can be the resource's IP address information or the resource's ID information, etc.

[0116] S520. Determine the specified resource corresponding to the specified resource information from the preset resources.

[0117] The specified resource can be determined from the preset resources based on the resource identifier information in the resource specification information.

[0118] S530. If at least one of the operation indicators of the specified resource meets the preset abnormal conditions, based on the task type tag of the target task and the resource tag of the remaining resources, perform resource search on the remaining resources and determine the plurality of candidate resources from the remaining resources; the remaining resources are resources other than the specified resources in the preset resources.

[0119] Once a specific resource is identified, anomaly detection can be performed based on the operational metrics information of that resource to obtain the anomaly detection result. The preset anomaly conditions can include the corresponding anomaly detection conditions for processors, memory, networks, disks, etc. For example, the anomaly detection condition for processors can be that the CPU utilization rate is greater than or equal to the preset CPU utilization rate; the anomaly detection condition for memory can be that the memory utilization rate is greater than or equal to the preset memory utilization rate; the anomaly detection condition for networks can be that the network transmission rate is less than or equal to the preset transmission rate; and the anomaly detection condition for disks can be that the disk utilization rate is greater than or equal to the preset disk utilization rate, etc.

[0120] If a specified resource is determined to be an abnormal resource, it is excluded. Based on the task type tag of the target task and the resource tags of the preset resources, a new resource search is performed among the preset resources other than the specified resource to identify multiple candidate resources. This facilitates the identification of the target resource based on the candidate resources for processing the target task, avoiding the problem of task processing failure caused by assigning the target task to the specified resource even though it is an abnormal resource. At the same time, when the specified resource is determined to be an abnormal resource, the candidate resources are re-identified, ensuring that the target task can be assigned to a normal resource for processing, thus improving the success rate of task processing.

[0121] S540. If the operation indicator information of the specified resource does not meet the preset abnormal conditions, the candidate resource is determined based on the specified resource.

[0122] When the specified resource is a normal resource, if there is only one specified resource, it can be directly identified as a candidate resource, and then the candidate resource can be identified as the target resource. If there are multiple specified resources, all of them can be identified as candidate resources, and then... Figure 2 The task scheduling method shown determines the target resource from multiple candidate resources.

[0123] Therefore, when the specified resource is a normal resource, candidate resources can be directly determined based on the specified resource. The resource range of the specified resource is relatively small compared to the preset resource. Thus, the amount of computation required to determine candidate resources is smaller than that required to determine candidate resources from the preset resource, thereby improving the efficiency of candidate resource determination.

[0124] The implementation process of this application is illustrated below with a specific example, taking a rendering task scheduling scenario as an example. The task type tags or resource tags in the rendering scenario are shown in Table 1:

[0125] Table 1

[0126]

[0127] FFmpeg is a powerful multimedia processing tool that can perform operations such as transcoding, format conversion, editing, and merging of video and audio; for example, it can convert a video file from MP4 format to WebM format.

[0128] PAG: Portable Animated Graphics is a complete motion graphics workflow solution designed to reduce or eliminate motion graphics-related R&D costs. It can export motion graphics created by designers in AE (Adobe After Effects) into material files with one click and quickly deploy them to mainstream platforms.

[0129] Please see Figure 6 The rendering task scheduling methods include:

[0130] S0: Pre-selection stage, filtering out a list of resources that meet the criteria.

[0131] S1: Optimization phase, calculating resource scores and resource affinity scores.

[0132] S2: In the selection phase, a final score is obtained by weighted summation, which determines the final resource allocation.

[0133] The process consists of three phases: pre-selection, optimization, and final selection. The pre-selection phase involves querying all resource nodes and selecting suitable resources based on a pre-selection algorithm to complete the initial screening. The optimization phase involves scoring the resources screened in the pre-selection phase using an optimization algorithm and selecting the resource with the highest score as the target resource for scheduling.

[0134] Please see Figure 7 Both cluster A and cluster B include multiple preset resources. The task type tags of the target task include 180s and PAG. The resource tags of resource 192.168.0.1 include 180s, FFMPEG and PAG. That is, the resource tags of resource 192.168.0.1 include 180s and PAG. Therefore, resource 192.168.0.1 can be identified as a candidate resource. Similarly, resource 192.168.0.3 can also be identified as a candidate resource.

[0135] The task scheduling method in this embodiment can be implemented by calling external APIs. This application can be applied to the video generation process of various products, specifically including creative element delivery (dynamic creative upgrade), automatic creative generation of micro-movies, creative workbench, template videos, independent micro-movie tools, OTT (Over The Top, providing various application services to users via the Internet) video production, intelligent editing and other creative video production tools; and through different dimensions of task matching methods, it greatly improves resource utilization and also increases productivity.

[0136] Please see Figure 8 This embodiment also provides a task scheduling device, including:

[0137] The resource search module 810 is used to search for the preset resources based on the task type tag of the target task and the resource tag of the preset resources, and to determine multiple candidate resources from the preset resources; the resource tag of each preset resource represents the task type supported by each preset resource, and the resource tag of each candidate resource includes the task type tag of the target task;

[0138] The performance analysis module 820 is used to perform performance analysis on the multiple candidate resources based on the operation index information of the multiple candidate resources, and obtain the resource performance information corresponding to each of the multiple candidate resources.

[0139] The tag matching module 830 is used to perform tag matching based on the task type tag of the target task and the resource tags of the multiple candidate resources to obtain the tag matching degree between the target task and each candidate resource;

[0140] The correlation determination module 840 is used to perform data fusion processing on the resource performance information of each candidate resource and the label matching degree between the target task and each candidate resource to obtain the correlation degree between the target task and each candidate resource.

[0141] The target resource determination module 850 is used to determine the target resource from the plurality of candidate resources based on the correlation between the target task and each candidate resource;

[0142] The task allocation module 860 is used to allocate the target task to the target resource.

[0143] Furthermore, the resource search module 810 includes:

[0144] An abnormal resource determination module is used to determine abnormal resources from the preset resources; at least one operational indicator information corresponding to the abnormal resource satisfies the preset abnormal conditions;

[0145] The candidate resource determination module is used to search for resources in the preset resources other than the abnormal resources based on the task type tag of the target task and the resource tag of the preset resources, so as to obtain the multiple candidate resources.

[0146] Furthermore, the operational metrics information for each resource includes processor operational information, memory usage information, network transmission information, and disk usage information for each resource;

[0147] The performance analysis module 820 includes:

[0148] The performance information determination module is used to determine the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource based on the processor running information, memory usage information, network transmission information, and disk usage information of each candidate resource.

[0149] The performance information fusion module is used to perform data fusion processing on the processor performance information, memory performance information, network performance information and disk performance information of each candidate resource to obtain the resource performance information corresponding to each candidate resource.

[0150] Furthermore, the performance information determination module includes:

[0151] The importance determination module is used to determine the importance of each candidate resource's processor performance information, memory performance information, network performance information, and disk performance information based on the task type label of the target task.

[0152] The first determining module is used to determine the weights of the processor information, memory performance information, network performance information, and disk performance information of each candidate resource based on the importance of each candidate resource's processor performance information, memory performance information, network performance information, and disk performance information.

[0153] The weighting module is used to perform weighted processing on the processor information, memory performance information, network performance information, and disk performance information of each candidate resource based on the weights corresponding to the processor information, memory performance information, network performance information, and disk performance information of each candidate resource, so as to obtain the resource performance information corresponding to each candidate resource.

[0154] Furthermore, the device also includes:

[0155] The tag weight determination module is used to determine the tag weight of the resource tag of each preset resource based on the resource feature information of each preset resource;

[0156] The tag matching module 830 includes:

[0157] The matching tag determination module is used to determine the matching tag that matches the task type tag from the resource tags of each candidate resource;

[0158] The tag matching information determination module is used to determine the tag matching information between the target task and each candidate resource based on the tag weight corresponding to the matching tag;

[0159] The tag matching degree determination module is used to determine the tag matching degree between the target task and each candidate resource based on the ratio of the tag matching information to the number of resource tags of each candidate resource.

[0160] Furthermore, the tag matching information determination module is also used for:

[0161] When there are multiple matching tags, the tag weights corresponding to each of the multiple matching tags are summed to obtain the tag matching information between the target task and each candidate resource.

[0162] Furthermore, the device also includes:

[0163] The task information acquisition module is used to acquire the task information of the target task; the task information includes the task type tag and resource specification information.

[0164] The designated resource determination module is used to determine the designated resource corresponding to the resource designation information from the preset resources;

[0165] If at least one of the operational metrics of the specified resource meets a preset abnormal condition, the resource lookup module 810 is further configured to:

[0166] Based on the task type tag of the target task and the resource tag of the remaining resources, a resource search is performed on the remaining resources to determine the plurality of candidate resources from the remaining resources; the remaining resources are the resources other than the specified resources in the preset resources.

[0167] The second determining module is used to determine the candidate resource based on the specified resource when the operation indicator information of the specified resource does not meet the preset abnormal conditions.

[0168] The apparatus provided in the above embodiments can execute the methods provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the above embodiments can be found in the methods provided in any embodiment of this application.

[0169] This embodiment also provides a computer-readable storage medium storing at least one instruction or at least one program, which is loaded by a processor and executed as any of the methods described above in this embodiment.

[0170] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the methods described above.

[0171] This embodiment also provides a device, the structural diagram of which can be found in the following figure. Figure 9The device 900 can vary significantly in configuration or performance, and may include one or more central processing units (CPUs) 922 (e.g., one or more processors) and memory 932, and one or more storage media 930 (e.g., one or more mass storage devices) storing application programs 942 or data 944. The memory 932 and storage media 930 may be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown), each module including a series of instruction operations on the device. Furthermore, the CPU 922 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the device 900. The device 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input / output interfaces 958, and / or one or more operating systems 941, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM Etc. Any of the methods described above in this embodiment can be based on... Figure 9 The equipment shown is used for implementation.

[0172] This specification provides the operational steps of the methods described in the embodiments or flowcharts, but more or fewer operational steps may be included based on conventional or non-inventive labor. The steps and order listed in the embodiments are merely one possible execution order among many steps and do not represent the only execution order. In actual system or interrupt product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0173] The structure shown in this embodiment is only a partial structure related to the solution of this application and does not constitute a limitation on the device to which the solution of this application is applied. Specific devices may include more or fewer components than shown, or combinations of certain components, or arrangements of different components. It should be understood that the methods, apparatuses, etc., disclosed in this embodiment can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or unit modules through some interfaces.

[0174] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0175] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0176] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A task scheduling method, characterized in that, include: Based on the task type tag of the target task and the resource tag of the preset resources, a resource search is performed on the preset resources to determine multiple candidate resources. The resource tag of each preset resource represents the task type supported by each preset resource, and the resource tag of each candidate resource includes the task type tag of the target task; Based on the operational indicator information of the multiple candidate resources, a performance analysis is performed on the multiple candidate resources to obtain the resource performance information corresponding to each of the multiple candidate resources; Tag matching is performed based on the task type tag of the target task and the resource tags of the multiple candidate resources to obtain the tag matching degree between the target task and each candidate resource; The resource performance information of each candidate resource and the tag matching degree between the target task and each candidate resource are fused to obtain the correlation degree between the target task and each candidate resource. Based on the correlation between the target task and each candidate resource, the target resource is determined from the plurality of candidate resources; Assign the target task to the target resource.

2. The method according to claim 1, characterized in that, The method involves searching for preset resources based on the task type tag of the target task and the resource tags of preset resources, and identifying multiple candidate resources from the preset resources, including: Abnormal resources are identified from the preset resources; at least one operational indicator information corresponding to the abnormal resource satisfies the preset abnormal conditions; Based on the task type tag of the target task and the resource tag of the preset resources, a resource search is performed in the preset resources other than the abnormal resources to obtain the multiple candidate resources.

3. The method according to claim 1, characterized in that, The operational metrics information for each resource includes processor operational information, memory usage information, network transmission information, and disk usage information for each resource. The process of performing performance analysis on the multiple candidate resources based on their operational metric information to obtain resource performance information corresponding to each of the multiple candidate resources includes: Based on the processor running information, memory usage information, network transmission information, and disk usage information of each candidate resource, determine the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource; The processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource are fused to obtain the resource performance information corresponding to each candidate resource.

4. The method according to claim 3, characterized in that, The process of fusing the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource to obtain the resource performance information corresponding to each candidate resource includes: Based on the task type label of the target task, determine the importance of the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource. Based on the importance of the processor performance information, memory performance information, network performance information, and disk performance information of each candidate resource, the weights of the processor information, memory performance information, network performance information, and disk performance information of each candidate resource are determined. Based on the weights corresponding to the processor information, memory performance information, network performance information, and disk performance information of each candidate resource, the processor information, memory performance information, network performance information, and disk performance information of each candidate resource are weighted to obtain the resource performance information corresponding to each candidate resource.

5. The method according to claim 1, characterized in that, Before performing tag matching based on the task type tag of the target task and the resource tags of the multiple candidate resources to obtain the tag matching degree between the target task and each candidate resource, the method further includes: Based on the resource feature information of each preset resource, the tag weight of the resource tag of each preset resource is determined; The step of performing tag matching based on the task type tag of the target task and the resource tags of the multiple candidate resources to obtain the tag matching degree between the target task and each candidate resource includes: From the resource tags of each candidate resource, determine the matching tag that matches the task type tag; Based on the tag weights corresponding to the matching tags, the tag matching information between the target task and each candidate resource is determined; The tag matching degree between the target task and each candidate resource is determined based on the ratio of the tag matching information to the number of resource tags for each candidate resource.

6. The method according to claim 5, characterized in that, The step of determining the label matching information between the target task and each candidate resource based on the label weight corresponding to the matching label includes: When there are multiple matching tags, the tag weights corresponding to each of the multiple matching tags are summed to obtain the tag matching information between the target task and each candidate resource.

7. The method according to claim 1, characterized in that, Before determining multiple candidate resources from the preset resources by performing resource search based on the task type tag of the target task and the resource tag of the preset resources, the method further includes: Obtain the task information of the target task; the task information includes the task type tag and resource specification information; Determine the specified resource corresponding to the specified resource information from the preset resources; If at least one operational metric of the specified resource meets a preset anomaly condition, the preset resource is searched based on the task type tag of the target task and the resource tag of the preset resource to determine multiple candidate resources, including: Based on the task type tag of the target task and the resource tag of the remaining resources, a resource search is performed on the remaining resources to determine the plurality of candidate resources from the remaining resources; the remaining resources are the resources other than the specified resources in the preset resources. If none of the operational metrics of the specified resource meet the preset abnormal conditions, the candidate resource is determined based on the specified resource.

8. A task scheduling device, characterized in that, include: The resource search module is used to search for the preset resources based on the task type tag of the target task and the resource tags of the preset resources, and to determine multiple candidate resources from the preset resources; the resource tag of each preset resource represents the task type supported by each preset resource, and the resource tag of each candidate resource includes the task type tag of the target task; The performance analysis module is used to perform performance analysis on the multiple candidate resources based on the operation index information of the multiple candidate resources, and obtain the resource performance information corresponding to each of the multiple candidate resources. The tag matching module is used to perform tag matching based on the task type tag of the target task and the resource tags of the multiple candidate resources to obtain the tag matching degree between the target task and each candidate resource; The correlation determination module is used to perform data fusion processing on the resource performance information of each candidate resource and the label matching degree between the target task and each candidate resource to obtain the correlation degree between the target task and each candidate resource. The target resource determination module is used to determine the target resource from the plurality of candidate resources based on the correlation between the target task and each candidate resource; The task allocation module is used to allocate the target task to the target resource.

9. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the task scheduling method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor according to any one of claims 1 to 7.

Images