A task allocation method, device and storage medium

Abstract

The present application provides a task allocation method, device and storage medium, which relates to the field of computer technology and is used to solve the problem that task allocation cannot be performed reasonably. The method includes: obtaining a task to be allocated; determining a task node that meets preset conditions among multiple task nodes as a target task node, and allocating the task to be allocated to the target task node; the preset conditions include that the node load weight of the task node is less than or equal to a preset threshold; the node load weight of the task node is the sum of the task weights of each task carried by the task node; the task weight of the task is used to represent the resource overhead corresponding to the task.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and particularly relates to a task allocation method, apparatus, and storage medium. Background Art

[0002] Currently, big data platforms usually process a large number of various types of computing tasks simultaneously. For example, report generation tasks, data collection tasks, and data processing tasks, etc. Therefore, the scheduling ability of big data platforms for these tasks has become an important supporting part for ensuring the efficient and stable operation of the system.

[0003] In related technologies, generally, tasks are fixedly allocated to task nodes corresponding to their types according to the type of the tasks. However, when there are more tasks of a certain type and fewer tasks of other types, this scheduling method may cause the problem of uneven loads on each task node, easily leading to waste of computing resources. Summary of the Invention

[0004] This application provides a task allocation method, apparatus, and storage medium for solving the problem of unreasonable task allocation.

[0005] To achieve the above object, this application adopts the following technical solutions:

[0006] In a first aspect, a task allocation method is provided, including: obtaining a task to be allocated; determining task nodes that meet a preset condition among multiple task nodes as target task nodes, and allocating the task to be allocated on the target task nodes; the preset condition includes that the node load weight of the task node is less than or equal to a preset threshold; the node load weight of the task node is the sum of the task weights of each task carried by the task node; the task weight of the task is used to represent the resource overhead corresponding to the task.

[0007] Optionally, the task to be allocated includes the expected time required to complete the task to be allocated. This task allocation method further includes: determining the initial task weight of the task to be allocated according to the expected time, preset memory parameter, and preset number of cores; updating the initial task weight of the task to be allocated in the node load weight of the target task node. The expected time, preset memory parameter, preset number of cores, and the initial task weight of the task to be allocated satisfy a first formula; the first formula includes:

[0008]

[0009] where W is the initial task weight of the task to be allocated; T is the expected time; M is the preset memory parameter; C is the preset number of cores.

[0010] Optionally, the task allocation method further includes: obtaining the running information of the task to be allocated; the running information includes the correspondence between the number of times the task to be allocated is executed on the target task node and the actual time consumption; determining the task weight corresponding to the task to be allocated after the current number of executions according to the running information; updating the task weight corresponding to the task to be allocated after the current number of executions in the node load weight of the target task node. The running information and the task weight corresponding to the task to be allocated after the current number of executions satisfy the second formula; the second formula is:

[0011]

[0012] where n is the current number of times the task to be allocated is executed on the target task node, and is a positive integer; W n is the task weight corresponding to the task to be allocated after the current number of executions; W n-1 is the task weight corresponding to the task to be allocated after n - 1 executions; T n is the actual time consumption corresponding to the current number of executions.

[0013] Optionally, the target task node is used to determine the running information of the task to be allocated and store it in the shared cache device; the specific method for obtaining the running information of the task to be allocated includes: reading the running information of the task to be allocated stored in the shared cache device.

[0014] In a second aspect, a task allocation device is provided, including: an obtaining unit, a determining unit, and an allocating unit; the obtaining unit is used to obtain the task to be allocated; the determining unit is used to determine the task node that meets the preset conditions among multiple task nodes as the target task node, and the allocating unit is used to allocate the task to be allocated on the target task node; the preset conditions include that the node load weight of the task node is less than or equal to a preset threshold; the node load weight of the task node is the sum of the task weights of each task carried by the task node; the task weight of the task is used to represent the resource overhead corresponding to the task.

[0015] Optionally, the task to be allocated includes the expected time consumption for completing the task to be allocated; the task allocation device further includes: an updating unit; the determining unit is further used to determine the initial task weight of the task to be allocated according to the expected time consumption, the preset memory parameter, and the preset number of cores; the expected time consumption, the preset memory parameter, the preset number of cores, and the initial task weight of the task to be allocated satisfy the first formula; the first formula includes:

[0016]

[0017] where W is the initial task weight of the task to be allocated; T is the expected time consumption; M is the preset memory parameter; C is the preset number of cores. The updating unit is used to update the initial task weight of the task to be allocated in the node load weight of the target task node.

[0018] Optionally, the obtaining unit is further configured to obtain the running information of the task to be assigned; the running information includes the corresponding relationship between the number of times the task to be assigned is executed on the target task node and the actual time consumed; the determining unit is further configured to determine the task weight corresponding to the current number of executions of the task to be assigned according to the running information; the running information and the task weight corresponding to the current number of executions of the task to be assigned satisfy the second formula; the second formula is:

[0019]

[0020] where n is the current number of times the task to be assigned is executed on the target task node and is a positive integer; W n is the task weight corresponding to the current number of executions of the task to be assigned; W n-1 is the task weight corresponding to the task to be assigned after n-1 executions; T n is the actual time consumed corresponding to the current number of times. The updating unit is further configured to update the task weight corresponding to the current number of executions of the task to be assigned in the node load weight of the target task node.

[0021] Optionally, the target task node is configured to determine the running information of the task to be assigned and store it in the shared cache device; the obtaining unit is further configured to read the running information of the task to be assigned stored in the shared cache device.

[0022] In a third aspect, a task allocation device is provided, including a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory through a bus; when the task allocation device runs, the processor executes the computer execution instructions stored in the memory, so that the task allocation device executes the task allocation method as in the first aspect.

[0023] The task allocation device may be a network device or a part of the network device, such as a chip system in the network device. The chip system is used to support the network device to implement the functions involved in the first aspect and any possible implementation manner thereof. For example, it receives, determines, and shunts the data and / or information involved in the above task allocation method. The chip system includes a chip and may also include other discrete devices or circuit structures.

[0024] In a fourth aspect, a computer-readable storage medium is provided, including computer execution instructions, which, when running on a computer, cause the computer to execute the task allocation method as in the first aspect.

[0025] It should be noted that the above computer instructions may be stored in whole or in part on the first computer-readable storage medium. Among them, the first computer-readable storage medium may be packaged together with the processor of the task allocation device or separately packaged with the processor of the task allocation device. This application does not make any limitations in this regard.

[0026] In this application, the name of the above task allocation device does not constitute a limitation on the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those of this application and fall within the scope of the claims of this application and their equivalent technologies.

[0027] These aspects or other aspects of this application will be made more concise and understandable in the following description.

[0028] The technical solutions provided by this application bring at least the following beneficial effects:

[0029] Based on any of the above aspects, in this application, after obtaining the task to be allocated, task nodes that meet the preset conditions among multiple task nodes can be determined as target task nodes, and the task to be allocated can be allocated to the target task nodes. Since the node load weight of the target task node is the sum of the task weights of each task carried, and the task weight of a task is used to represent the resource overhead corresponding to the task. Therefore, when the target task node meets the preset conditions, that is, when the node load weight of the target task node is less than or equal to the preset threshold, it can be determined that the target task node is a task node with less occupied resources among the multiple task nodes. Based on this, this application can allocate the task to be allocated to a relatively idle task node (i.e., the target task node). Compared with the solution of allocating tasks according to task types in the related art, this application can overcome the problem of uneven loads on each task node and avoid waste of computing resources. Therefore, this application can perform task allocation reasonably. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic structural diagram of a task allocation system provided by an embodiment of this application;

[0031] Figure 2 It is a schematic functional architecture diagram of a task allocation system provided by an embodiment of this application;

[0032] Figure 3 It is a schematic hardware structure diagram of a task node provided by an embodiment of this application;

[0033] Figure 4 It is a schematic flowchart of a task allocation method provided by an embodiment of this application;

[0034] Figure 5 It is a schematic flowchart of another task allocation method provided by an embodiment of this application;

[0035] Figure 6 It is a schematic flowchart of another task allocation method provided by an embodiment of this application;

[0036] Figure 7 A flowchart of another task allocation method provided by an embodiment of the present application;

[0037] Figure 8 A flowchart of another task allocation method provided by an embodiment of the present application;

[0038] Figure 9 A structural diagram of a task allocation device provided by an embodiment of the present application. Detailed implementation manners

[0039] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0040] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0041] To facilitate a clear description of the technical solutions in the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the words such as "first" and "second" do not limit the quantity and execution order.

[0042] In addition, the terms "including" and "having" in the embodiments of the present application and the claims and the accompanying drawings are not exclusive. For example, a process, method, system, product or device including a series of steps or modules is not limited to the listed steps or modules, and may further include steps or modules not listed.

[0043] In the related art, generally, a method of fixedly allocating tasks to task nodes corresponding to the type according to the type of the task is adopted. However, when there are more tasks of a certain type and fewer tasks of other types, this scheduling method may cause the problem that the loads of each task node are uneven, which is likely to lead to waste of computing resources.

[0044] In view of the above problems, an embodiment of the present application provides a task allocation method. After obtaining the task to be allocated, task nodes that meet the preset conditions among multiple task nodes can be determined as target task nodes, and the task to be allocated is allocated to the target task nodes. Since the node load weight of a target task node is the sum of the task weights of each task it bears, and the task weight of a task is used to represent the resource overhead corresponding to the task. Therefore, when the target task node meets the preset conditions, that is, when the node load weight of the target task node is less than or equal to the preset threshold, it can be determined that the target task node is a task node with a relatively small amount of occupied resources among the multiple task nodes. Based on this, the present application can allocate the task to be allocated to a relatively idle task node (i.e., the target task node). Compared with the related art of allocating tasks according to task types, the present application can overcome the problem of uneven loads on each task node and avoid waste of computing resources. Therefore, the present application can reasonably allocate tasks.

[0045] This task allocation method is applicable to a task allocation system. Figure 1 A structure of the task allocation system 100 is shown. As Figure 1 shown, the task allocation system 100 includes: a server 101, multiple task nodes 102, a shared cache device 103, and a terminal 104. The server 101 can be communicatively connected to the multiple task nodes 102 and the terminal 104 respectively. The shared cache device 103 can be communicatively connected to the multiple task nodes 102 respectively.

[0046] In practical applications, Figure 1 the server 101 in

[0047] can be communicatively connected to the multiple terminals 104.

[0048] Figure 1 The server 101 in Figure 1 can be a service end of various big data applications, and is used to provide various data services for institutions or individuals regarding massive, heterogeneous, and rapidly changing data.

[0049] Figure 1Multiple task nodes 102 therein can be configured with various resource files for processing data, so as to implement various data services covering activities related to the data life cycle such as collection, transmission, storage, processing (including computing, analysis, visualization, etc.), exchange, and destruction.

[0050] It should be noted that, in the embodiments of the present application, Figure 1 multiple task nodes 102 therein are all configured with lightweight scheduling services. The scheduling service can have functions such as adding tasks, starting tasks, stopping tasks, automatic task startup, task weight management, and task information query. And, at the same moment, only the scheduling service of one task node 102 is in a normal working state.

[0051] Among them, the task adding function can be used to add a specified task to the shared cache device 103.

[0052] The task starting function can start a task on the task node where the task is deployed according to the task configuration. For example, after the task to be assigned in the present application is assigned to the target task node, the task to be assigned can be started on the target task node.

[0053] The task stopping function can be used to automatically find the node where the task is located and end the execution of the task.

[0054] The task automatic startup function can automatically restart a task after the task stops due to reasons such as network.

[0055] The task weight management function can pre-set the initial task weight of a task in a non-transitory storage medium (i.e., the server 101) and synchronize it to the transitory storage medium (i.e., the shared cache device 103). Subsequently, the task weight saved in the transitory storage medium can also be dynamically updated during the task running.

[0056] The task information query function can query the current running status, running node, running information, etc. of a task.

[0057] In one implementable way, the scheduling service can also generate a task identifier for each task and store it corresponding to each resource file involved in the task, so as to implement the above various functions.

[0058] In one possible way, multiple task nodes 102 can implement that only the scheduling service of one task node 102 is in a normal working state based on the heartbeat mechanism. Regarding the specific implementation of the heartbeat mechanism, those skilled in the art can understand it with reference to related technologies and will not be elaborated here.

[0059] For ease of understanding, in the embodiments of the present application, the task node 102 in the normal working state of the scheduling service is referred to as the scheduling task node. Moreover, the task allocation method provided by the embodiments of the present application is applied to the scheduling task node. Based on this, the task allocation system 100 provided by the present application no longer needs to separately configure a scheduling node to implement the task allocation method provided by the embodiments of the present application, thereby being able to make more full use of computing resources and reduce system complexity. At the same time, system maintenance personnel no longer need to adopt different maintenance schemes for two different types of nodes (i.e., task nodes and scheduling nodes), which is convenient for maintenance personnel to carry out relevant maintenance.

[0060] Optionally, both the server 101 and the multiple task nodes 102 can be a single server, or can also be a server cluster composed of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. The embodiments of the present application do not impose any restrictions on this.

[0061] Figure 1 The shared cache device 103 therein can be a kind of temporary storage medium for quickly storing data that needs to be continuously updated, so as to facilitate access by each task node 102. For example, the node load weight of the task node, the task weight of the task, and the running information involved in the embodiments of the present application can all be stored in the shared cache device 103.

[0062] Figure 1 The terminal 102 therein can be the client of various big data applications. The terminal 102 can be configured with an input module, a communication module, etc., so as to facilitate the user to edit and initiate various types of data processing tasks.

[0063] Optionally, Figure 1 The terminal 102 therein can be a device that provides voice and / or data connectivity to the user, a handheld device with a wireless connection function, or other processing devices connected to a wireless modem. The wireless terminal can communicate with one or more core networks via a radio access network (RAN). The wireless terminal can be a mobile terminal, such as a computer with a mobile terminal, or can also be a portable, pocket-sized, handheld, or mobile device built into a computer, which exchanges language and / or data with the wireless access network. For example, mobile phones, tablet computers, laptop computers, netbooks, personal digital assistants (PDAs). The embodiments of the present application do not impose any restrictions on this.

[0064] Combined with Figure 1 As Figure 2 shown, it is a schematic diagram of a functional architecture of the task allocation system 102 provided by the embodiments of the present application. Figure 2 The fast storage medium therein isFigure 1 The shared cache device 103 therein. The task information may include task weight information, running information, etc. of the task. The node information may include information such as node load weight information of the task node. The metadata may include the correspondence between the task identifier of the task and the task information, and may also include the correspondence between the node identifier of the task node and the node load weight information, etc. For example, the correspondence between the task identifier of task 1 and the task information, the correspondence between the task identifier of task 2 and the task information, and the correspondence between the task identifier of task 3 and the task information. Or, the correspondence between the node identifier of task node 1 and the node load weight information, the correspondence between the node identifier of task node 2 and the node load weight information, and the correspondence between the node identifier of task node 3 and the node load weight information.

[0065] Optionally, when the scheduling service on the task node is working, the task node can implement the function of adding a task by adding a task, or implement the function of starting a task by starting a task, or implement the function of stopping a task by stopping a task, or implement the function of automatically starting a task by restarting a task, or implement the function of task weight management by managing a task, or implement the function of querying task information by querying a task.

[0066] As Figure 3 shown, it is a schematic hardware structure diagram of a task node 102 provided by an embodiment of the present application. The task node 102 includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, the memory 22, and the communication interface 23 can be connected through the bus 24.

[0067] The processor 21 is the control center of the task node 102, and can be a single processor or a collective term for multiple processing elements. For example, the processor 21 can be a general central processing unit (CPU), or other general processors, etc. Among them, the general processor can be a microprocessor or any conventional processor, etc.

[0068] As an embodiment, the processor 21 may include one or more CPUs, such as Figure 3 the CPU0 and CPU1 shown therein.

[0069] The memory 22 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or can also be an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0070] In a possible implementation, the memory 22 can exist independently of the processor 21. The memory 22 can be connected to the processor 21 through the bus 24 and is used to store instructions or program code. When the processor 21 calls and executes the instructions or program code stored in the memory 22, the task allocation method provided in the following embodiments of the present application can be implemented.

[0071] In another possible implementation, the memory 22 can also be integrated with the processor 21.

[0072] The communication interface 23 is used for the task node 102 to connect to other devices through a communication network. The communication network can be an Ethernet, a wireless access network, a wireless local area network (WLAN), etc. The communication interface 23 can include a receiving unit for receiving data and a sending unit for sending data.

[0073] The bus 24 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0074] It should be noted that, Figure 3 the structure shown in the figure does not constitute a limitation on the task node 102, except Figure 3In addition to the components shown, the task node 102 may include more or fewer components than those shown, or combine certain components, or have a different component arrangement.

[0075] As Figure 4 shown, it is a schematic flowchart of a task allocation method provided by an embodiment of the present application. This task allocation method is applied to Figure 1 a task node 102 in the task allocation system 100 shown. Further, for the sake of distinction, this task node 102 may be referred to as a scheduling task node. This task allocation method includes: S401 - S402.

[0076] S401. The scheduling task node obtains the task to be allocated.

[0077] In a possible way, the task to be allocated may be a big data processing task for a specific service. The specific service may be services such as short - video recommendation service and advertisement push service. The big data processing task may be tasks such as data report generation task, data collection task, data analysis task, and data visualization task.

[0078] In an implementable way, combined with Figure 1 , when the user needs to understand data information about a specific service (such as data composition and data development trend, etc.), the user can perform a task editing operation through the input module configured on the terminal, and edit the relevant parameters of the task to be allocated (such as parameters such as data range and execution period). In response to the task editing operation performed by the user, the terminal can obtain the relevant parameters of the task to be allocated edited by the user and configure the task to be allocated. Then, after the user finishes editing, the user can perform a task publishing operation (such as clicking the publish button). In response to the task publishing operation performed by the user, the terminal can send a request message for requesting the execution of the task to be allocated to the server, and carry the relevant parameters of the task to be allocated in the request message. Correspondingly, the server can receive the request message from the terminal and parse the request message to determine the relevant parameters of the task to be allocated. Then, the server can store the relevant parameters of the task to be allocated in the configured storage module.

[0079] Next, when the scheduling task node scans all the tasks stored in the server in real - time, it can identify the task to be allocated and read the relevant parameters of the task to be allocated. Subsequently, the scheduling task node can select a suitable task node to execute the task to be allocated and send the execution result of the task to be allocated to the server, so that the server can send the execution result to the terminal.

[0080] In a possible way, the relevant parameters of the task to be assigned may include the expected time consumption for completing the task to be assigned. The expected time consumption can be configured for the task to be assigned by the user when performing a task editing operation after predicting the execution completion time of the task to be assigned based on experience.

[0081] S402. The scheduling task node determines the task nodes that meet the preset conditions among the multiple task nodes as the target task nodes, and assigns the task to be assigned to the target task nodes.

[0082] It should be noted that the scheduling task node can maintain the node load weights of each task node and store the node load weights of each task node in the shared cache device. The node load weight of a task node can be the sum of the task weights of each task carried by the task node. The task weight of a task can be used to represent the resource overhead corresponding to the task.

[0083] In a possible way, the task weight of a task can be the product of the actual execution time of the task and a preset coefficient. The preset coefficient can be pre-set by the staff in the shared cache device. For example, the preset coefficient can be 0.8 or 1.5, etc.

[0084] In a possible way, the preset conditions may include that the node load weight of the task node is less than or equal to a preset threshold. The preset threshold can be pre-set by the staff in the shared cache device. Or, the preset threshold can also be the minimum value among the node load weights of multiple task nodes. In this case, the target task node can be the task node with the smallest node load weight among the multiple task nodes.

[0085] In a way that can be implemented, combined with Figure 1 , after the scheduling task node obtains the task to be assigned from the server, it can read the node load weights of each task node stored in the shared cache device, and determine the task nodes with node load weights less than or equal to the preset threshold as the target task nodes. Then, the scheduling task node can assign the task to be assigned to the target task nodes. Specifically, the scheduling task node can send an indication message for instructing the execution of the task to be assigned to the target task node, and carry the relevant parameters of the task to be assigned in the indication message. Or, the scheduling task node can also carry the storage location of the relevant parameters of the task to be assigned in the server in the indication message.

[0086] In a possible way, the task to be assigned can be executed as a separate process in the target task node.

[0087] In an embodiment, as Figure 5 shown, the task assignment method provided by the embodiment of the present application further includes: S501 - S502.

[0088] S501. The scheduling task node determines the initial task weight of the task to be assigned according to the expected duration, the preset memory parameter, and the preset number of cores.

[0089] In an implementable way, after the task to be assigned is assigned to the target task node, in order to more accurately reflect the resource overhead of the target task node for subsequent task assignment, the scheduling task node can determine the initial task weight of the task to be assigned and update it in the node load weight of the target task node. In this case, the scheduling task node can determine the initial task weight of the task to be assigned according to the expected duration, the preset memory parameter, and the preset number of cores.

[0090] In a possible way, the expected duration, the preset memory parameter, the preset number of cores, and the initial task weight of the task to be assigned satisfy the first formula. The first formula includes:

[0091]

[0092] Where, W is the initial task weight of the task to be assigned. T is the expected duration. M is the preset memory parameter. C is the preset number of cores.

[0093] In a possible way, the preset memory parameter can be pre-set by the staff in the shared cache device settings to represent the memory when the task node is based on the regular configuration. For example, the preset memory parameter can be 64 gigabytes (GB).

[0094] In a possible way, the preset number of cores can be pre-set by the staff in the shared cache device settings to represent the number of CPU cores when the task node is based on the regular configuration. For example, the preset number of cores can be 16.

[0095] It should be noted that, as can be seen from the first formula, when the preset memory parameter is 64G and the preset number of cores is 16, the initial task weight of the task to be assigned is equivalent to the expected duration of the task to be assigned. Based on this, the configuration of 64G memory and 16 cores can also be called the standard configuration. If the configuration of the task node is lower than this standard configuration, the task weight of the task to be assigned calculated according to the first formula will be higher than the expected duration of the task to be assigned. On the contrary, the task weight of the task to be assigned calculated according to the first formula will be lower than the expected duration of the task to be assigned.

[0096] S502. The scheduling task node adds the initial task weight of the task to be assigned to the node load weight of the target task node.

[0097] In one possible implementation, after determining the initial task weight of the task to be assigned, the scheduling task node may add the initial task weight of the task to be assigned to the node load weight of the target task node, and replace the node load weight of the target task node previously recorded in the shared cache device, so as to update the node load weight of the target task node.

[0098] In one embodiment, as Figure 6 shown, the task allocation method provided by the embodiment of the present application further includes: S601-S603.

[0099] S601. The scheduling task node obtains the running information of the task to be assigned.

[0100] The running information includes the corresponding relationship between the number of times the task to be assigned is executed on the target task node and the actual time consumed.

[0101] It should be noted that the task to be assigned may be a task that only needs to be executed once, or a task that needs to be executed multiple times. To facilitate determining the running state of the task to be assigned, after the scheduling task node assigns the task to be assigned to the target task node, it may store the number of times the task to be assigned is executed, the execution node, and the relevant parameters of the task to be assigned in the shared cache device, and initialize the number of times the task to be assigned is executed to 0. At the same time, each time the target task node executes the task to be assigned, it may also determine the actual time required to execute the task to be assigned, and store the corresponding relationship between the number of times the task to be assigned is executed and the actual time consumed in the shared cache device.

[0102] Based on this, in order to more accurately reflect the resource overhead of the target task node and facilitate subsequent task allocation, the scheduling task node may obtain the running information of the task to be assigned from the shared cache device in real time to facilitate determining the running state of the task to be assigned. Then, the scheduling task node may update the node load weight of the target task node according to the running state of the task to be assigned.

[0103] In one possible way, when the task to be assigned is a task that only needs to be executed once, if the running information of the task to be assigned indicates that the task to be assigned has been executed once, the scheduling task node may delete the initial task weight of the task to be assigned from the node load weight of the target task node.

[0104] In a possible way, when the task to be assigned is a task that needs to be executed multiple times, if the running information of the task to be assigned indicates that the task to be assigned has been executed at least once and is less than the number of times required to be executed, in order to more accurately reflect the resource overhead required for the target task node to execute the task to be assigned, the scheduling task node can re-determine the task weight of the task to be assigned. In this case, the scheduling task node can re-determine the task weight of the task to be assigned according to the actual time consumed by the task to be assigned.

[0105] S602. The scheduling task node determines the task weight corresponding to the current number of executions of the task to be assigned according to the running information.

[0106] In an implementable way, after obtaining the running information of the task to be assigned, the scheduling task node can determine the task weight corresponding to the current number of executions of the task to be assigned according to the running information of the task to be assigned. It should be understood that the task weight corresponding to the current number of executions of the task to be assigned is the task weight corresponding to the last execution of the task to be assigned on the target task node.

[0107] In a possible way, the running information of the task to be assigned and the task weight corresponding to the current number of executions of the task to be assigned satisfy the second formula. The second formula is:

[0108]

[0109] where n is the current number of executions of the task to be assigned on the target task node and is a positive integer. W n is the task weight corresponding to the current number of executions of the task to be assigned. W n-1 is the task weight corresponding to the task to be assigned after n - 1 executions. T n is the actual time consumed corresponding to the current number of executions.

[0110] It should be noted that when n is 1, the task weight corresponding to the first execution of the task to be assigned is the actual time consumed when the task to be assigned is executed for the first time. When n is greater than 1, that is, when the task to be assigned is executed multiple times, the task weight corresponding to the current number of executions of the task to be assigned is actually the dynamic average of the multiple actual time consumptions. Compared with the expected time consumption predicted by the user according to experience, the value calculated according to the second formula can more truly reflect the resource overhead of the task to be assigned on the target task node.

[0111] S603. The scheduling task node updates the task weight corresponding to the current number of executions of the task to be assigned in the node load weight of the target task node.

[0112] In one possible implementation, after determining the task weight corresponding to the current execution times of the task to be assigned, the scheduling task node may replace the task weight of the task to be assigned in the node load weight of the target task node with the task weight corresponding to the current execution times of the task to be assigned, and update the node load weight of the target task node stored in the shared cache device, so as to more accurately reflect the resource overhead of the target task node.

[0113] In one embodiment, in combination with Figure 6 , in S601, that is, when the scheduling task node obtains the running information of the task to be assigned, as Figure 7 shown, an optional implementation provided by an embodiment of the present application includes: S701.

[0114] S701. The scheduling task node reads the running information of the task to be assigned stored in the shared cache device.

[0115] In one possible way, the shared cache device may store the task identifier of the task to be assigned, and the corresponding relationship between the task identifier of the task to be assigned and the running information of the task to be assigned.

[0116] In one possible implementation, since the target task node can determine the running information of the task to be assigned in real time each time the task to be assigned is executed, and can store the running information of the task to be assigned in the shared cache device. Based on this, when it is necessary to determine the running status of the task to be assigned, the scheduling task node may read the running information of the task to be assigned stored in the shared cache device according to the task identifier of the task to be assigned.

[0117] In one possible way, the target task node may also determine the resource occupancy when executing the task to be assigned each time. The resource occupancy may be the increase in the memory used by the target task node and the increase in the CPU occupancy during the execution of the task to be assigned compared to before the execution of the task to be assigned. Based on this, the scheduling task node may combine the actual execution time corresponding to each execution of the task to be assigned with the resource occupancy corresponding to each execution of the task to be assigned according to a preset rule for correction, to obtain the corrected actual execution time, so as to more accurately determine the task weight of the task to be assigned.

[0118] In one possible way, the preset method may be set in the shared cache device by the staff in advance according to experience. For example, the preset method may be to multiply the resource occupancy when the task to be assigned is executed by a constant and then add the actual execution time when the task to be assigned is executed to obtain the corrected actual execution time. For example, the constant may be 5.

[0119] In one embodiment, as Figure 8As shown in the figure, it is a schematic flowchart of a task allocation method provided by an embodiment of the present application. After the user edits and publishes a task to be allocated through a terminal, the server can receive the task to be allocated from the terminal and store it in a configured storage module. Then, the scheduling task node can scan the task to be allocated in the server in real time, and after determining the target task node, allocate the task to be allocated on the target node. Then, the scheduling task node can parse the configuration file of the task to be allocated, determine the expected time of the task to be allocated, and determine the initial task weight of the task to be allocated according to the first formula. Then, the scheduling task node can configure the number of times the task to be allocated is executed to 0, and store it in the shared cache device corresponding to information such as the initial task weight of the task to be allocated. At the same time, the scheduling task node can update the node load weight of the target task node according to the initial task weight of the task to be allocated.

[0120] Subsequently, after the target task node finishes executing the task to be allocated each time, the target task node can update the number of times the task to be allocated is executed in the shared cache device in real time. Correspondingly, the scheduling task node can scan the number of times the task to be allocated is executed stored in the shared cache device in real time, and after the number of times the task to be allocated is executed is updated, re-determine the task weight of the task to be allocated according to the second formula. Then, the scheduling task node can store the re-determined task weight of the task to be allocated in the shared cache device. At the same time, the scheduling task node can update the node load weight of the target task node according to the re-determined task weight of the task to be allocated.

[0121] In the embodiment of the present application, after obtaining the task to be allocated, the scheduling task node can determine the task nodes that meet the preset conditions among multiple task nodes as the target task nodes, and allocate the task to be allocated on the target task nodes. Since the node load weight of the target task node is the sum of the task weights of each task carried, and the task weight of a task is used to represent the resource overhead corresponding to the task. Therefore, when the target task node meets the preset conditions, that is, when the node load weight of the target task node is less than or equal to the preset threshold, it can be determined that the target task node is a task node with less occupied resources among multiple task nodes. Based on this, the present application can allocate the task to be allocated on a relatively idle task node (i.e., the target task node). Compared with the related art that allocates tasks according to task types, the present application can overcome the problem of uneven loads on each task node and avoid waste of computing resources. Therefore, the present application can reasonably allocate tasks.

[0122] The above mainly introduces the solution provided by the embodiments of the present application from the perspective of methods. To implement the above functions, it includes the corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should easily realize that, combining the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0123] The embodiments of the present application can divide the functional modules of the scheduling task nodes according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. Optionally, the division of modules in the embodiments of the present application is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0124] As Figure 9 shown, it is a schematic structural diagram of a task allocation device provided by the embodiments of the present application. This task allocation device can be used to execute the task allocation method as Figures 4 to 8 shown. This task allocation device includes: an acquisition unit 801, a determination unit 802, and an allocation unit 803;

[0125] The acquisition unit 801 is used to acquire the task to be allocated; for example, in combination with Figure 4 , the acquisition unit 801 can be used to execute S401.

[0126] The determination unit 802 is used to determine the task nodes that meet the preset conditions among multiple task nodes as the target task nodes, and the allocation unit 803 is used to allocate the task to be allocated to the target task nodes; the preset conditions include that the node load weight of the task node is less than or equal to the preset threshold; the node load weight of the task node is the sum of the task weights of each task carried by the task node; the task weight of the task is used to represent the resource overhead corresponding to the task.

[0127] Optionally, the task to be allocated includes the expected time required to complete the task to be allocated; this task allocation device further includes: an update unit 804;

[0128] The determination unit 802 is further used to determine the initial task weight of the task to be allocated according to the expected time, the preset memory parameter, and the preset number of cores; the expected time, the preset memory parameter, the preset number of cores, and the initial task weight of the task to be allocated satisfy the first formula; the first formula includes:

[0129]

[0130] Among them, W is the initial task weight of the task to be assigned; T is the expected time consumption; M is the preset memory parameter; C is the preset number of cores. For example, in combination with Figure 5 , the determination unit 802 can be used to execute S501.

[0131] The update unit 804 is used to update the initial task weight of the task to be assigned in the node load weight of the target task node. For example, in combination with Figure 5 , the update unit 804 can be used to execute S502.

[0132] Optionally, the acquisition unit 801 is further used to acquire the running information of the task to be assigned; the running information includes the corresponding relationship between the number of times the task to be assigned is executed on the target task node and the actual time consumption; for example, in combination with Figure 6 , the acquisition unit 801 can be used to execute S601.

[0133] The determination unit 802 is further used to determine the task weight corresponding to the current number of executions of the task to be assigned according to the running information; the running information and the task weight corresponding to the current number of executions of the task to be assigned satisfy the second formula; the second formula is:

[0134]

[0135] Among them, n is the current number of executions of the task to be assigned on the target task node and is a positive integer; W n is the task weight corresponding to the current number of executions of the task to be assigned; W n-1 is the task weight corresponding to the n - 1 times of execution of the task to be assigned; T n is the actual time consumption corresponding to the current number of times. For example, in combination with Figure 6 , the determination unit 802 can be used to execute S602.

[0136] The update unit 804 is further used to update the task weight corresponding to the current number of executions of the task to be assigned in the node load weight of the target task node. For example, in combination with Figure 6 , the update unit 804 can be used to execute S603.

[0137] Optionally, the target task node is used to determine the running information of the task to be assigned and store it in the shared cache device; the acquisition unit 801 is further used to read the running information of the task to be assigned stored in the shared cache device. For example, in combination with Figure 7 , the acquisition unit 801 can be used to execute S701.

[0138] Those skilled in the art should be able to realize that in one or more of the above examples, the functions described in this application can be implemented by hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. The computer-readable medium includes computer-readable storage media and communication media, where the communication media includes any medium that facilitates the transfer of a computer program from one place to another. The storage media can be any available medium accessible by a general-purpose or special-purpose computer.

[0139] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above function modules is used as an example. In actual applications, the above functions can be assigned to different function modules as needed, that is, the internal structure of the device is divided into different function modules to complete all or part of the functions described above.

[0140] In several embodiments provided in this application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form. The units described as separate components may or may not be physically separated, and the components displayed as units can be one physical unit or multiple physical units, that is, they can be located in one place, or they can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] As described above, only the specific implementation manners of this application are provided, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in this application should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A task allocation method, characterized in that: include: Obtaining a task to be assigned; the task to be assigned includes an expected time required to complete the task to be assigned; Determine the task node that meets the preset conditions among the multiple task nodes as the target task node, and assign the task to be assigned to the target task node; The preset condition includes that the node load weight of the task node is less than or equal to a preset threshold; The node load weight of the task node is the sum of the task weights of the tasks carried by the task node; the task weight of the task is used to represent the resource overhead corresponding to the task; Determining the initial task weight of the task to be assigned according to the expected time consumption, the preset memory parameters and the preset number of cores; The expected time consumption, the preset memory parameter, the preset number of cores and the initial task weight of the task to be assigned satisfy a first formula; The first formula includes: ； in, is the initial task weight of the task to be assigned; The expected time taken for the above; is the preset memory parameter; is the preset number of cores; Updating the initial task weight of the task to be assigned in the node load weight of the target task node; Acquire the running information of the task to be assigned; the running information includes the corresponding relationship between the number of times the task to be assigned is executed on the target task node and the actual time consumed; According to the operation information, the task weight corresponding to the current number of executions of the task to be assigned is determined; the operation information and the task weight corresponding to the current number of executions of the task to be assigned satisfy a second formula; the second formula is: ； Wherein, n is the current number of times the task to be assigned is executed on the target task node, and is a positive integer; The task weight corresponding to the current number of times the task to be assigned is executed; The task weight corresponding to the task to be assigned after executing n-1 times; is the actual time consumption corresponding to the current number of times; The task weight corresponding to the current number of executions of the task to be assigned is updated in the node load weight of the target task node.

2. The task allocation method according to claim 1, characterized in that: The target task node is used to determine the running information of the task to be assigned and store it in a shared cache device; The obtaining the running information of the task to be assigned includes: Read the running information of the task to be assigned stored in the shared cache device.

3. A task allocation device, characterized in that: include: Get unit, determine unit, assign unit and update unit; The acquisition unit is used to acquire the tasks to be assigned; The task to be assigned includes the expected time required to complete the task to be assigned; The determining unit is used to determine the task node that meets the preset conditions among the multiple task nodes as the target task node, and the allocating unit is used to allocate the task to be allocated to the target task node; The preset condition includes that the node load weight of the task node is less than or equal to a preset threshold; The node load weight of the task node is the sum of the task weights of the tasks carried by the task node; the task weight of the task is used to represent the resource overhead corresponding to the task; The determining unit is further used to determine the initial task weight of the task to be assigned according to the expected time consumption, the preset memory parameter and the preset number of cores; The expected time consumption, the preset memory parameter, the preset number of cores and the initial task weight of the task to be assigned satisfy a first formula; The first formula includes: ； in, is the initial task weight of the task to be assigned; The expected time taken for the above; is the preset memory parameter; is the preset number of cores; The updating unit is used to update the initial task weight of the task to be assigned into the node load weight of the target task node; The acquisition unit is further used to acquire the running information of the task to be assigned; the running information includes the corresponding relationship between the number of times the task to be assigned is executed on the target task node and the actual time consumed; The determination unit is further used to determine, according to the operation information, a task weight corresponding to the current number of executions of the task to be assigned; the operation information and the task weight corresponding to the current number of executions of the task to be assigned satisfy a second formula; the second formula is: ； Wherein, n is the current number of times the task to be assigned is executed on the target task node, and is a positive integer; The task weight corresponding to the current number of times the task to be assigned is executed; The task weight corresponding to the task to be assigned after executing n-1 times; is the actual time consumption corresponding to the current number of times; The updating unit is further used to update the task weight corresponding to the current number of executions of the task to be assigned in the node load weight of the target task node.

4. The task allocation device according to claim 3, characterized in that: The target task node is used to determine the running information of the task to be assigned and store it in a shared cache device; The acquisition unit is further configured to read the running information of the task to be assigned stored in the shared cache device.

5. A task allocation device, characterized in that: It comprises a memory and a processor; the memory is used to store computer-executable instructions, and the processor is connected to the memory via a bus; when the task allocation device is running, the processor executes the computer-executable instructions stored in the memory, so that the task allocation device executes the task allocation method as described in claim 1 or 2.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer-executable instructions. When the computer-executable instructions are executed on a computer, the computer is caused to execute the task allocation method according to claim 1 or 2.

Images