Task scheduling method, storage medium and electronic device

Abstract

The application discloses a task scheduling method, a storage medium and an electronic device. The method comprises the following steps: acquiring a success rate of executing a task by any slave node in a resource scheduling system; determining a number of tasks distributed to any slave node in a second time period after a current time based on the success rate; and performing task scheduling on any slave node based on the number of tasks. The application solves the technical problem that a large number of task executions fail due to an abnormal slave node in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, in particular to a task scheduling method, a storage medium and an electronic device. BACKGROUND

[0002] In a big data computing platform, a distributed resource scheduling system is a core base. A common resource scheduling system generally adopts a master-slave mode (master+agent, wherein master is used to represent a master node in the resource scheduling system, and agent is used to represent a slave node in the resource scheduling system), the master node is used for resource scheduling and task distribution, and the slave node is used for resource reporting and task execution. When executing a task, if the heartbeat of the slave node is normal, the task execution fails due to other reasons, the slave node normally reports the resources, and the master node will always allocate tasks to the slave node. However, due to the rapid task failure of the slave node, the resources are released quickly, and the slave node can obtain more tasks with a high probability, resulting in a large number of task execution failures.

[0003] At present, no effective solution has been proposed for the above problems. SUMMARY

[0004] Embodiments of the present application provide a task scheduling method, a storage medium and an electronic device to at least solve the technical problem of a large number of task execution failures caused by an abnormal slave node in the related art.

[0005] According to an aspect of an embodiment of the present application, a task scheduling method is provided, including: obtaining a success rate of task execution of any one slave node in a resource scheduling system, wherein the success rate is used to represent a ratio of a number of tasks successfully executed by the slave node in a first time period before a current time to a total number of completed tasks; determining a number of tasks distributed to any one slave node in a second time period after the current time based on the success rate; and performing task scheduling on any one slave node based on the number of tasks.

[0006] According to another aspect of an embodiment of the present application, a resource scheduling system is also provided, including: a plurality of slave nodes for executing tasks; a master node connected with the plurality of slave nodes for distributing tasks to the slave nodes; and a resource scheduling device connected with the master node for obtaining a success rate of task execution of any one slave node, determining a number of tasks distributed to any one slave node in a second time period after a current time based on the success rate, and performing task scheduling on any one slave node based on the number of tasks, wherein the success rate is used to represent a ratio of a number of tasks successfully executed by the slave node in a first time period before the current time to a total number of completed tasks.

[0007] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which includes a stored program, wherein the program controls the device where the storage medium is located to perform the task scheduling method in the above embodiments when the program is running.

[0008] According to another aspect of the embodiments of the present application, an electronic device is also provided, which includes a memory storing an executable program, and a processor configured to run the program, wherein the program controls the device to perform the task scheduling method in the above embodiments when the program is running.

[0009] In the above embodiments of the present application, the success rate of any slave node in the resource scheduling system to perform a task is obtained, the number of tasks distributed to the any slave node in a second time period after the current time is determined based on the success rate, and the task scheduling for the any slave node is performed based on the number of tasks. It is easy to note that, in the case that the success rate of the node is lower than the preset success rate, i.e., the slave node is in an abnormal state, the number of tasks distributed to the node in the second time period is determined based on the success rate, and then the number of tasks to be processed by the node in a certain time period can be dynamically controlled until the success rate of the node is greater than the preset success rate, i.e., the node returns to normal, the flow control is ended, and the technical problem that a large number of tasks are failed to be executed due to the abnormal slave node in the related art is solved, and the technical effect of improving the task processing efficiency is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings, which are included to provide a further understanding of the present application, form a part of the present application and illustrate the illustrative embodiments of the present application and the explanation of the present application, and do not constitute improper limitations on the present application. In the drawings:

[0011] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) used to implement the task scheduling method in the embodiments of the present application;

[0012] Figure 2 is a structure block diagram of a computer terminal used by a user to implement the task scheduling method according to the embodiments of the present application;

[0013] Figure 3 is a structure schematic diagram of the computer terminal 10 (or mobile device) as a service grid computer terminal according to the above Figure 1 embodiments of the present application;

[0014] Figure 4 is a flowchart of the task scheduling method according to Embodiment 1 of the present application;

[0015] Figure 5 is a schematic diagram of the first time period and the second time period in an optional embodiment of the present application;

[0016] Figure 6 is a schematic diagram of a resource scheduling system according to an embodiment of the present application;

[0017] Figure 7 is a flow chart of a task scheduling method according to an embodiment of the present application;

[0018] Figure 8 is a schematic diagram of a task scheduling system 800 according to an embodiment of the present application;

[0019] Figure 9 is a schematic diagram of a task scheduling apparatus 900 according to an embodiment of the present application;

[0020] Figure 10 is a schematic diagram of a task scheduling apparatus 1000 according to an embodiment of the present application;

[0021] Figure 11 is a schematic diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should belong to the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] First, some of the nouns or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

[0025] TCP / IP (Transmission Control Protocol / Internet Protocol).

[0026] Ping (Packet Internet Groper) is a testing program used to verify network connectivity. For example, the Ping command sends an ICMP (Internet Control Message Protocol) response request to a target host or IP address to verify whether the host can connect to the TCP / IP network. Then, it can receive a message corresponding to the ICMP response request. Based on this message, it can determine whether the TCP / IP parameters are correct, whether the operation is normal, and whether the network is working properly.

[0027] Pong message: Used to represent the response received based on the above Ping message.

[0028] PingPong messages: including the Ping and Pong messages mentioned above.

[0029] Mesos: An open-source cluster management system that virtualizes and manages computing resources.

[0030] Master: The master node in the resource scheduling system, used for resource collection and scheduling.

[0031] Agent: A slave node in the resource scheduling system, used for resource reporting and task execution.

[0032] Heartbeat: Used to characterize the frequency at which the current node reports its current status to other nodes in a distributed system.

[0033] Resource offer: Idle resources on a node.

[0034] It should be noted that in the related art, the master node and the slave node in the distributed system monitor the PingPong message through the TCP network to perform health monitoring, the master node receives the resource offer reported by the slave node, and performs resource allocation, consumes the offer that meets the task resource requirement, and sends the task to the slave node for execution. If the slave node heartbeat is normal, but due to other reasons, such as: network card exception, system load is too high, environment configuration exception, third-party dependence does not meet the system reason, etc. The task execution fails. At this time, the resource scheduling system generally does not perceive the task execution success rate of the slave node in this period of time. Generally, as long as the slave node heartbeat is normal, the resource is normally reported, and the master node will always allocate tasks to the slave node. On the other hand, due to the system exception of the slave node, the task fails quickly, and the resource is released quickly, so this node will have a greater probability of obtaining a new task than other normal nodes, thereby causing large-scale task running failure.

[0035] Secondly, in the related art, the health status of the agent can also be calculated by collecting some system indicators of the agent, such as CPU usage, system load, disk capacity and network state, etc. If the agent health status is abnormal, the agent needs to be removed from the cluster. However, the scheme has the following disadvantages: first, the scheme needs to rely on manual removal of the agent state abnormal node, and then manually repair the removed node. After successful repair, the repaired node is added to the cluster, and the operation and maintenance cost is high. Second, the agent system indicators cannot completely reflect the task execution environment problems, for example, the task depends on the non-existent image, the environment variable setting error, etc. Large-scale task running failure can still be caused.

[0036] The present application provides a method for self-adaptive adjustment of task flow limiting of slave nodes under agent abnormal condition. The task execution success rate of the slave node in a certain time window is calculated by interval timing, the task concurrency limiting threshold of the slave node in the next time interval is calculated, and the task concurrency of the slave node in the system abnormal condition is controlled until the slave node system recovers to normal, and the flow limiting control is ended. Specifically as follows:

[0037] Embodiment 1

[0038] According to the embodiments of the present application, a task scheduling method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in different order from here.

[0039] The method provided in the embodiment one of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 is a hardware structure block diagram of a computer terminal (or a mobile device) for implementing a task scheduling method in the embodiment of the present application. As shown in Figure 1 , the computer terminal 10 (or the mobile device 10) can include one or more processors 102 (the processor 102 can include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication function. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or less components than those shown in Figure 1 , or have a different configuration than that shown in Figure 1 .

[0040] It should be noted that the one or more processors 102 and / or other data processing circuits described above can be referred to as "data processing circuits" herein. The data processing circuit can be embodied in whole or in part as software, hardware, firmware or any combination thereof. In addition, the data processing circuit can be a single independent processing module, or any one of the other elements combined into the computer terminal 10 (or mobile device) in whole or in part. As referred to in the embodiments of the present application, the data processing circuit serves as a processor control (for example, selection of a variable resistance terminal path connected to an interface).

[0041] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the task scheduling method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned task scheduling method. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory such as one or more magnetic storage devices, flash memory or other non-volatile solid state memory. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0042] The transmission device 106 is configured to receive or send data via a network. Examples of the network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network interface controller (NIC) that can connect to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module that is configured to communicate with the Internet through a wireless manner.

[0043] The display can be a liquid crystal display (LCD) that is touch screen-enabled, for example, to enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0044] Figure 1 The illustrated hardware structure diagram can be used as an exemplary block diagram of the computer terminal 10 (or mobile device) described above, and can also be used as an exemplary block diagram of the server described above, in an alternative embodiment, Figure 2 The above-described computer terminal 10 (or mobile device) is illustrated in a block diagram as an embodiment of a computing node in a computing environment 201. Figure 1 The computer terminal 10 (or mobile device) is illustrated in a block diagram as an embodiment of a computing node in a computing environment 201. Figure 2 The computer terminal 10 (or mobile device) is illustrated in a block diagram as an embodiment of a computing node in a computing environment 201. Figure 2 As illustrated, the computing environment 201 includes a plurality of computing nodes (e.g., servers) 210-1, 210-2,..., running on a distributed network. Each computing node includes local processing and memory resources, and end users 202 can remotely run applications or store data in the computing environment 201. Applications can be provided as a plurality of services 220-1, 220-2, 220-3, and 220-4 in the computing environment 201, representing services "A", "D", "E", and "H", respectively.

[0045] The end users 202 can provide and access the services through a web browser or other software applications on a client, and in some embodiments, the provisioning and / or requests of the end users 202 can be provided to an entry gateway 230. The entry gateway 230 can include a corresponding proxy to handle the provisioning and / or requests for the services 220 (one or more services provided in the computing environment 201).

[0046] Services 220 are provided or deployed in accordance with various virtualization technologies supported by computing environment 201. In some embodiments, services 220 can be provided in accordance with virtual machine (VM)-based virtualization, container-based virtualization, and / or the like. VM-based virtualization can be emulating a real computer by initializing a virtual machine to execute programs and applications without directly accessing any actual hardware resources. While a virtual machine is virtualized, in accordance with container-based virtualization, a container can be launched to virtualize an entire operating system (OS) so that multiple workloads can run on a single OS instance.

[0047] In one embodiment of container-based virtualization, several containers of a service 220 can be assembled into a Pod (e.g., a Kubernetes Pod). For example, as shown in Figure 2 Each Pod 240 can include a proxy 245 and one or more containers 242-1, 242-2,..., 242-M (collectively, containers 242). The one or more containers 242 in a Pod 240 handle requests related to one or more respective functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, and the like. Other services 220 can also have Pods similar to Pods 240.

[0048] In operation, executing a user request from end user 202 can require invoking one or more services 220 in computing environment 201, executing one or more functions of a service 220, and invoking one or more functions of another service 220. As shown in Figure 2 Service "A" 220-1 receives a user request from end user 202 from ingress gateway 230, service "A" 220-1 can invoke service "D" 220-2, and service "D" 220-2 can request service "E" 220-3 to execute one or more functions.

[0049] The computing environment described above can be a cloud computing environment, where allocation of resources is managed by a cloud service provider, allowing development of functionality without considering implementation, tuning, or scaling servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be split into a set of functions that can automatically scale independently, rather than scaling a single hardware device to handle potential loads.

[0050] In another alternative embodiment, Figure 3 The use of the above-describedFigure 1 The computer terminal 10 (or mobile device) shown is an embodiment of a service mesh. Figure 3 The above-described embodiments are used in this application. Figure 1 The diagram shown illustrates the structure of computer terminal 10 (or mobile device) as a service mesh computer terminal. Figure 3 As shown, the service mesh 300 is mainly used to facilitate secure and reliable communication between multiple microservices. Microservices refer to the decomposition of an application into multiple smaller services or instances, which are distributed across different clusters / machines.

[0051] like Figure 3 As shown, a microservice may include application service instance A and application service instance B, which together form the functional application layer of service mesh 300. In one implementation, application service instance A runs as a container / process 308 on machine / workload container group 314 (Pod), and application service instance B runs as a container / process 310 on machine / workload container group 316 (Pod).

[0052] In one implementation, application service instance A can be a product query service, and application service instance B can be a product order placement service.

[0053] like Figure 3 As shown, application service instance A and grid proxy (sidecar) 303 coexist in machine workload container group 614, and application service instance B and grid proxy 305 coexist in machine workload container 314. Grid proxy 303 and grid proxy 305 form the data plane layer of service mesh 300. Grid proxy 303 and grid proxy 305 run as container / process 304, which can receive requests 312 for product query services, and grid proxy 306, respectively. Grid proxy 303 and application service instance A can communicate bidirectionally, and grid proxy 305 and application service instance B can also communicate bidirectionally. Furthermore, grid proxy 303 and grid proxy 305 can also communicate bidirectionally with each other.

[0054] In an embodiment, all traffic of application service instance A is routed through mesh proxy 303 to the appropriate destination, and all network traffic of application service instance B is routed through mesh proxy 305 to the appropriate destination. It is noted that the network traffic mentioned herein includes, but is not limited to, Hyper Text Transfer Protocol (HTTP), Representational State Transfer (REST), high-performance, general-purpose, open-source framework (e.g., gRPC), open-source in-memory data structure storage system (Redis), and the like.

[0055] In an embodiment, the function of extending the data plane layer can be implemented by writing a custom filter for the proxy (Envoy) in the service mesh 300. The service mesh proxy configuration can be to make the service mesh correctly proxy service traffic, implement service intercommunication, and service governance. The mesh proxy 303 and the mesh proxy 305 can be configured to perform at least one of the following functions: service discovery, health checking, routing, load balancing, authentication and authorization, and observability.

[0056] As shown in Figure 3 The service mesh 300 further includes a control plane layer. The control plane layer can be a group of services running in a dedicated namespace, which are hosted by the hosting control plane component 301 in the machine / Pod 302. As shown in Figure 3 The hosting control plane component 301 communicates with the mesh proxy 303 and the mesh proxy 305 in a bidirectional manner. The hosting control plane component 301 is configured to perform some control management functions. For example, the hosting control plane component 301 receives telemetry data transmitted by the mesh proxy 303 and the mesh proxy 305, and can further aggregate the telemetry data. The services, the hosting control plane component 301 can also provide a user-oriented application programming interface (API) to facilitate manipulation of network behavior, and provide configuration data to the mesh proxy 303 and the mesh proxy 305, and the like.

[0057] Under the above operating environment, the present application provides a task scheduling method as shown in Figure 4 Figure 4 ​is a flow chart of a task scheduling method according to Embodiment 1 of the present application.

[0058] In step S402, the success rate of the task executed by any slave node in the resource scheduling system is obtained, wherein the success rate represents the ratio of the number of tasks successfully executed by the slave node to the total number of tasks completed in a first time period before the current time.

[0059] The resource scheduling system can be a distributed resource scheduling system, such as Mesos. The slave node can be any slave node in the distributed scheduling system. The resource scheduling system includes a plurality of master nodes and slave nodes, and one master node can correspond to a plurality of slave nodes. For example, the resource scheduling system includes the following master nodes: master node A, master node B, and master node C. Master node A can correspond to slave node a1, slave node a2,..., and slave node a m where m is a positive integer, a m where m is a positive integer, a n where n is a positive integer, b n where n is a positive integer, b k where k is a positive integer, a k where k is a positive integer, a The first time period can be a preset statistical time window, for example, if the success rate at time t0 is to be obtained, the length of the first time period is 30 minutes, the starting time of the first time period is t0-30min, and the ending time of the first time period is time t0.

[0060] The success rate can be the ratio of the number of tasks successfully executed by the slave node to the total number of tasks completed in a statistical time window, for example, the first time period is 30 minutes, the total number of tasks completed by slave node A in the first time period is 100, and the number of tasks successfully executed is 50, so the success rate is 50%.

[0061] In an optional embodiment, the success rate of the task executed by any slave node in the resource scheduling system can be obtained at a preset time interval. The preset time interval can be set by the staff according to actual needs, for example, if the task adjustment of the resource scheduling system is desired to be faster, the preset time interval can be set to a smaller time period, or if power saving is desired, the preset time interval can be set to a larger time period.

[0062] It should be noted that the above preset time interval is used to limit the frequency of obtaining the success rate of the slave node. For example, the preset time interval is 5 minutes, and after obtaining the slave node at 9:00, the success rate of the slave node is obtained again at 9:06 after 5 minutes. Figure 5 is a schematic diagram of the first time period and the second time period in an optional embodiment of the present application, Figure 5 The upper part is the first time period 1, and the end time of the first time period 1 is t0, Figure 5 The lower part is the first time period 2, and the end time of the second time period is t1. The interval between t0 and t1 is a preset time interval. For example, the first time period 1 is 9:00-9:30, the preset time interval is 5 minutes, and the first time period 2 can be 9:05-9:35.

[0063] As an optional embodiment, the staff needs to modify the preset time interval, can send instructions to the resource scheduling system, obtain the preset time interval input by the resource scheduling system, and then return the modified feedback time interval to the resource scheduling system. The resource scheduling system modifies the preset time interval to the feedback time interval. Then, before a new feedback time interval is received, the success rate of any one slave node of the resource scheduling system can be obtained every interval of the feedback time interval.

[0064] Step S404, determining the number of tasks distributed to any one slave node in the second time period after the current time based on the success rate.

[0065] As an optional implementation, in the case where the success rate is less than the preset success rate, it indicates that the slave node has an exception in the first time period, and the number of tasks allocated to the slave node is reduced in the second time period. Therefore, in this case. Or, in the case where the success rate is greater than or equal to the preset success rate, it indicates that the slave node is in a good state and can handle more tasks, and the number of tasks that the slave node needs to handle in the second time period can be appropriately increased. The second time period is essentially a flow adjustment period, and the start time of the second time period can be the end time of the first time period. Secondly, the length of the second time period can be the same as that of the first time period. For example, the length of the first time period and the second time period is 30 min, and at t0, the first time period is [t0-30min, t0], and the second time period is [t0, t0+30min].

[0066] Secondly, the above preset success rate can be adjusted by the staff according to the needs. For example, in the case of requiring high task processing efficiency, the preset success rate can be set to a larger value; in the case of not requiring high efficiency and requiring energy saving, the preset success rate can be appropriately reduced.

[0067] As an optional embodiment, in the case that the worker needs to modify the preset success rate, an instruction can be sent to the resource scheduling system to obtain the preset success rate input by the resource scheduling system, and then the modified feedback success rate is returned to the resource scheduling system, and the resource scheduling system modifies the preset success rate to the feedback success rate. Then, in the case that the success rate is the feedback success rate, the number of tasks that the slave node needs to process in the second time period can be determined according to the success rate.

[0068] In an optional embodiment, after obtaining the success rate, the number of tasks that the slave node needs to process in the second time period can be determined according to the obtained success rate in the following manner: the product of the duration of the second time period and the success rate is calculated, and the size of the product and the preset value is compared, if the product is greater than the preset value, the value of the product is taken as the number of tasks that the slave node needs to process in the second time period, if the product is less than the preset value, the preset value is taken as the number of tasks that the slave node needs to process in the second time period.

[0069] As an optional embodiment, if the success rate of the slave node A obtained this time is 50%, the preset success rate is 90%, and the duration of the second time period is 30 minutes, the product of the success rate and the duration of the second time period can be obtained as 15, and in the case that the preset value is 1, 15 is taken as the number of tasks that need to be processed in the second stage.

[0070] As an optional embodiment, if the success rate of the slave node A obtained this time is 3%, the preset success rate is 90%, and the duration of the second time period is 30 minutes, the product of the success rate and the duration of the second time period can be obtained as 0.9, and in the case that the preset value is 1, 1 is taken as the number of tasks that need to be processed in the second stage.

[0071] If the success rate of the slave node A obtained this time is 98%, the preset success rate is 90%, and the duration of the second time period is 30 minutes, the product of the success rate and the duration of the second time period can be obtained as 29.4, and after rounding off the product, 29 is obtained, and in the case that the preset value is 1, 29 is taken as the number of tasks that need to be processed in the second stage.

[0072] Step S406, task scheduling is performed on any slave node based on the number of tasks.

[0073] Among them, after obtaining the number of tasks of the slave node it manages, the number of tasks that need to be distributed to any slave node is determined, and the received tasks are distributed.

[0074] In an optional embodiment, after obtaining the number of tasks to be distributed in the second time period, the priority and the execution times of the tasks to be distributed are determined first, wherein the execution times represent the number of times each task to be distributed needs to be executed. In the process of executing the tasks by the slave node, the tasks to be executed with higher priority are executed preferentially.

[0075] It should be noted that the priority of the task can be pre-set or related to the execution times of the task, for example, the more the execution times of the task, the higher the priority.

[0076] As an optional embodiment, the tasks to be executed by the slave node A include task 1, task 2 and task 3, the priority of task 1 is higher than that of task 2, the priority of task 2 is higher than that of task 3, and task 1 needs to be executed twice. In the execution process, task 1 is executed twice, and then task 2 is executed, and after the execution of task 2 is completed, task 3 is executed.

[0077] As an optional embodiment, the tasks to be executed by the slave node A include task 1, task 2 and task 3, task 1 needs to be executed once, task 2 needs to be executed four times, and task 3 needs to be executed twice. In the case that the more the execution times, the higher the priority, the priority of task 2 is higher than that of task 3, and the priority of task 3 is higher than that of task 1. In the process of executing the tasks by the slave node A, task 2 is executed preferentially, task 2 is executed four times, and then task 3 is executed, and after the execution of task 3 is completed, task 1 is executed.

[0078] In an optional embodiment, Figure 6 is the architecture diagram of the resource scheduling system in an optional embodiment of the present application, as shown in Figure 6As shown, the resource scheduling system includes a resource scheduling framework and a resource scheduling cluster, the resource scheduling framework includes an agent manager and a scheduler, and the resource scheduling cluster can be a cluster composed of a master node and a slave node. When the resource scheduling system performs task scheduling, the slave node collects its own computing resource conditions, including CPU, memory, disk and other information, and then reports the above information to the master node corresponding to the slave node. The master node allocates the slave node resources in the form of an offer to the resource scheduling framework according to a preset resource scheduling strategy, and the resource scheduling framework determines whether to accept or reject the offer. It should be noted that accepting the offer includes the following two cases: accepting all offer resources, or accepting part of the offer resources. If the resource scheduling framework accepts the offer, it will submit a task to the resource scheduling cluster, receive a task state update message sent by the master of the scheduling cluster, and statistically determine the success rate of the slave node task execution at a preset time interval. After obtaining the success rate, the success rate is compared with a preset success rate. If the success rate is less than the preset success rate, the number of tasks that the slave node needs to process in the second time period is determined according to the length of the second time period, the success rate and a preset value, and the number of tasks is sent to the master node corresponding to the slave node, and the master node performs task allocation to the slave node.

[0079] In the above embodiments of the present application, the success rate of any slave node executing a task in the resource scheduling system can be obtained; the number of tasks distributed to any slave node in a second time period after the current time is determined based on the success rate; and any slave node is scheduled based on the number of tasks. It is easy to note that in the case where the node success rate is lower than the preset success rate, i.e. the slave node is in an abnormal state, the number of tasks distributed to the slave node in the second time period is determined based on the success rate, and the number of tasks that the node needs to process in a certain time can be dynamically controlled until the success rate of the node is greater than the preset success rate, i.e. the node returns to normal, the flow control ends, and the technical problem of a large number of task execution failures caused by an abnormal slave node in the related art is solved, and the technical effect of improving task processing efficiency is achieved.

[0080] In the above embodiments of the present application, the number of tasks distributed to any slave node in the second time period is determined based on the success rate, including: in the case where the success rate is less than the preset success rate, the number of tasks is determined based on the product of the length of the second time period and the success rate.

[0081] The second time period can have the same length as the first time period, and the start time of the second time period can be the end time of the first time period. In the case where the success rate is less than the preset success rate, it indicates that there is an anomaly in the first time period from the node, and the task allocation to the node is reduced in the second time period. In this case, the second time period can be essentially a flow limiting period. After obtaining the product of the length of the second time period and the success rate, the product can be compared with the preset value, and the larger value is selected as the target number. Then, the target number is set as the task number of the node in the second time period, so as to adjust the task number in the second time period.

[0082] In the above embodiments of the present application, the task number is determined based on the product of the length of the second time period and the success rate, including: in the case where the product of the length of the second time period and the success rate is greater than or equal to a preset value, the task number is determined based on the product of the length of the second time period and the success rate; in the case where the product of the length of the second time period and the success rate is less than the preset value, the task number is determined based on the preset value.

[0083] The task number = the length of the second time period * the success rate.

[0084] As an optional implementation, if the product of the length of the second time period and the success rate is less than the preset value, the preset value is selected as the task number. If the product of the length of the second time period and the success rate is greater than the preset value, the product of the length of the second time period and the success rate is selected as the task number.

[0085] As an optional implementation, the preset value can be set to 1, so that the node only needs to execute one task in the second time period. After the task fails, the success rate of the node in the second time period is 0, which can trigger the failure retry mechanism of the task scheduling system. Then, the failure retry mechanism can automatically restore the scheduling state of the node, so that manual intervention is not required, and the technical effect of automatically restoring the scheduling state of the node is achieved.

[0086] In the above embodiments of the present application, the task number is determined based on the product of the length of the second time period and the success rate, including: outputting the length of the second time period; receiving a feedback length obtained by modifying the length of the second time period; and determining a target number based on the product of the feedback length and the success rate.

[0087] The length of the second time period can be output by the resource scheduling, and then the length of the second time period can be forwarded to a terminal that can be operated only by the staff through the Internet or a local area network. The staff modifies the length of the second time period in the terminal to obtain a feedback length, and then the feedback length is encapsulated and returned to the resource scheduling system. After receiving the feedback length, the resource scheduling system can determine the target number in the following manner: target number = feedback length * success rate.

[0088] For example, the length of the second time period is output as 25 minutes, the feedback length is obtained as 35 minutes, and the success rate is 20%. The target number can be calculated as 7.

[0089] In the above embodiments of the present application, the success rate of the execution of the task by any one slave node in the resource scheduling system is obtained, including: obtaining the success rate of the execution of the task by any one slave node once every preset time interval.

[0090] In an optional embodiment, the success rate of the execution of the task by any one slave node in the above resource scheduling system can be obtained every preset time interval. The preset time interval can be set by the staff according to actual needs. For example, if it is desired that the task adjustment of the resource scheduling system be faster, the preset time interval can be set as a smaller time period. If it is desired to save power, the preset time interval can be set as a larger time period. It should be noted that the preset time interval is smaller than the above first time period. There can be multiple time intervals in a first time period. For example, the first time period is 30 minutes, and the preset time interval is 5 minutes. There are 6 time intervals in the first time period. The success rate of the slave node can be obtained once every 5 minutes in the first time period. It should be noted that if the success rate of the time interval from the 6th minute to the 10th minute in the above 30 minutes is calculated, the success rate of the time interval can be determined according to the total number of tasks executed and the number of tasks executed successfully in the time interval from the 6th minute to the 10th minute.

[0091] In the above embodiments of the present application, the success rate of the execution of the task by any one slave node is obtained once every preset time interval, including: outputting a preset time interval; receiving a feedback time interval obtained by modifying the preset time interval; and obtaining the success rate of the execution of the task by any one slave node once every feedback time interval.

[0092] As an optional embodiment, when the staff needs to modify the preset time interval, the staff can send an instruction to the resource scheduling system, and the resource scheduling system outputs the preset time interval after receiving the instruction. Then, the terminal operated by the staff acquires the preset time interval, and returns the modified feedback time interval to the resource scheduling system. The resource scheduling system modifies the preset time interval to the feedback time interval after receiving the feedback time interval. Then, the success rate of any one slave node of the resource scheduling system can be acquired every interval of the feedback time interval before a new feedback time interval is received.

[0093] In the above embodiments of the present application, when the success rate is less than the preset success rate, the task quantity distributed to any one slave node in the second time period is determined based on the success rate, including: outputting the preset success rate; receiving a feedback success rate obtained by modifying the preset success rate; and determining the task quantity based on the success rate when the success rate is less than the feedback success rate.

[0094] As an optional embodiment, when the staff needs to modify the preset success rate, the staff can send an instruction to the resource scheduling system, and the resource scheduling system outputs the preset success rate value after receiving the instruction. The staff modifies the preset success rate on the terminal, and the terminal obtains a feedback success rate and returns the feedback success rate to the resource scheduling system. The resource scheduling system modifies the preset success rate to the feedback success rate. Then, when the success rate is less than the feedback success rate, the task quantity can be determined according to the success rate.

[0095] In the above embodiments of the present application, the task quantity distributed to any one slave node in the second time period is determined based on the success rate, including: acquiring resource information reported by any one slave node when the success rate is greater than or equal to the preset success rate; and determining the task quantity based on the resource information.

[0096] In the above embodiments of the present application, when the success rate is greater than or equal to the preset success rate, it means that the slave node can operate normally and can normally receive the resource information reported by the slave node. The resource information reported by the slave node includes the computing resource information of the slave node, including CPU, memory, disk and other information. Then, the task quantity of the slave node is determined according to the resource information.

[0097] In the above embodiments of the present application, the task quantity is based on the task scheduling of any one slave node, including: determining the priority of a plurality of to-be-distributed tasks; determining the execution parameter of the to-be-distributed task distributed to any one slave node based on the priority of the plurality of to-be-distributed tasks and the task quantity, wherein the execution parameter includes at least one of the following: execution order and execution times; and distributing the to-be-distributed task to any one slave node based on the execution parameter of the to-be-distributed task.

[0098] After obtaining the number of tasks to be distributed in the second time period, the priority and the execution times of the plurality of tasks to be distributed need to be determined first, wherein the execution times represent the number of times each task to be distributed needs to be executed. In the process of executing the above tasks by the slave node, the task to be executed with a higher priority level can be executed preferentially.

[0099] It should be noted that the priority of the above task can be pre-set, or can be related to the execution times of the task, for example, the more the execution times of the task, the higher the priority.

[0100] As an optional embodiment, the tasks to be executed by the slave node A include task 1, task 2 and task 3, the priority of task 1 is higher than that of task 2, the priority of task 2 is higher than that of task 3, and task 1 needs to be executed 2 times. Therefore, in the execution process, task 1 is executed 2 times, and then task 2 is executed, and after the execution of task 2 is completed, task 3 is executed.

[0101] As an optional embodiment, the tasks to be executed by the slave node A include task 1, task 2 and task 3, task 1 needs to be executed 1 time, task 2 needs to be executed 4 times, and task 3 needs to be executed 2 times. In the case that the more the execution times, the higher the priority, the priority of task 2 is higher than that of task 3, and the priority of task 3 is higher than that of task 1. In the execution of the above tasks by the slave node A, task 2 can be executed preferentially, task 2 is executed 4 times, and after the execution of task 2 is completed, task 3 is executed, and after task 3 is executed 2 times, the execution of task 1 is started.

[0102] Embodiment 2

[0103] According to the embodiments of the present application, a task scheduling method as shown in Figure 7 Figure 7 is a flow chart of the task scheduling method according to the embodiment 2 of the present application, comprising the following steps:

[0104] In step S702, the success rate of the task executed by any slave node in the distributed cluster is obtained, wherein the success rate represents the ratio of the number of tasks successfully executed by the slave node in the first time period before the current time to the total number of tasks executed.

[0105] In step S704, the number of tasks distributed to any slave node in the second time period after the current time is determined based on the success rate.

[0106] In step S706, the task scheduling is performed on any slave node based on the number of tasks.

[0107] ​In the foregoing embodiments of the present application, determining the number of tasks distributed to any one of the slave nodes in the second time period based on the success rate comprises: in a case where the success rate is less than a preset success rate, determining the number of tasks based on a product of the length of the second time period and the success rate.

[0108] In the foregoing embodiments of the present application, determining the number of tasks based on the product of the length of the second time period and the success rate comprises: in a case where the product of the length of the second time period and the success rate is greater than or equal to a preset value, determining the number of tasks based on the product of the length of the second time period and the success rate; and in a case where the product of the length of the second time period and the success rate is less than the preset value, determining the number of tasks based on the preset value.

[0109] In the foregoing embodiments of the present application, determining the number of tasks based on the product of the length of the second time period and the success rate comprises: outputting the length of the second time period; receiving a feedback length obtained by modifying the length of the second time period; and determining the number of tasks based on a product of the feedback length and the success rate.

[0110] In the foregoing embodiments of the present application, obtaining the success rate of the task executed by any one of the slave nodes in the distributed cluster comprises: obtaining the success rate of the task executed by any one of the slave nodes once every preset time interval.

[0111] In the foregoing embodiments of the present application, obtaining the success rate of the task executed by any one of the slave nodes once every preset time interval comprises: outputting the preset time interval; receiving a feedback time interval obtained by modifying the preset time interval; and obtaining the success rate of the task executed by any one of the slave nodes once every feedback time interval.

[0112] In the foregoing embodiments of the present application, in a case where the success rate is less than a preset success rate, determining the number of tasks distributed to any one of the slave nodes in the second time period based on the success rate comprises: outputting the preset success rate; receiving a feedback success rate obtained by modifying the preset success rate; and in a case where the success rate is less than the feedback success rate, determining the number of tasks based on the success rate.

[0113] In the foregoing embodiments of the present application, determining the number of tasks distributed to any one of the slave nodes in the second time period based on the success rate comprises: in a case where the success rate is greater than or equal to a preset success rate, obtaining resource information reported by any one of the slave nodes; and determining the number of tasks based on the resource information.

[0114] In the above embodiments of the present application, the task scheduling on the arbitrary slave node based on the number of tasks comprises: determining priorities of a plurality of to-be-distributed tasks; determining execution parameters of the to-be-distributed tasks distributed to the arbitrary slave node based on the priorities of the plurality of to-be-distributed tasks and the number of tasks, wherein the execution parameters comprise at least one of the following: execution sequence and execution times; and distributing the to-be-distributed tasks to the arbitrary slave node based on the execution parameters of the to-be-distributed tasks.

[0115] Embodiment 3

[0116] The present application provides a task scheduling system as shown in Figure 8 Figure 8 is a structural schematic diagram of the task scheduling system 800 according to Embodiment 3 of the present application, comprising:

[0117] a plurality of slave nodes 82 for executing tasks.

[0118] a master node 84 connected with the plurality of slave nodes for distributing tasks to the slave nodes;

[0119] a resource scheduling device 86 connected with the master node for acquiring a success rate of the execution of tasks by the arbitrary slave node, determining a number of tasks distributed to the arbitrary slave node in a second time period after a current time based on the success rate, and performing task scheduling on the arbitrary slave node based on the number of tasks, wherein the success rate represents a ratio of the number of tasks successfully executed by the slave node in a first time period before the current time to the total number of tasks completed.

[0120] In the above embodiments of the present application, the resource scheduling device further comprises: a task number determining unit for determining the number of tasks based on a product of the success rate in the case that the success rate is less than a preset success rate.

[0121] In the above embodiments of the present application, the task number determining unit is further configured to determine the number of tasks based on a product of the length of the second time period and the success rate in the case that the success rate is less than the preset success rate.

[0122] In the above embodiments of the present application, the task number determining unit is further configured to determine the number of tasks based on a product of the length of the second time period and the success rate, comprising: determining the number of tasks based on the product of the length of the second time period and the success rate in the case that the product is greater than or equal to a preset value; and determining the number of tasks based on the preset value in the case that the product is less than the preset value.

[0123] In the above embodiments of the present application, the task number determining unit is further configured to output the length of the second time period; receive a feedback length obtained by modifying the length of the second time period; and determine a target number based on a product of the feedback length and the success rate.​

[0124] In the foregoing embodiments of the present application, the resource scheduling apparatus further comprises a success rate obtaining unit configured to obtain the success rate of the execution of the task by any slave node once every preset time interval.

[0125] In the foregoing embodiments of the present application, the success rate obtaining unit is further configured to output the preset time interval, receive a feedback time interval obtained by modifying the preset time interval, and obtain the success rate of the execution of the task by any slave node once every feedback time interval.

[0126] In the foregoing embodiments of the present application, the task quantity determining unit is further configured to output a preset success rate when the success rate is less than the preset success rate, receive a feedback success rate obtained by modifying the preset success rate, and determine the task quantity based on the success rate when the success rate is less than the feedback success rate.

[0127] In the foregoing embodiments of the present application, the resource scheduling apparatus further comprises an information obtaining unit configured to obtain resource information reported by any slave node when the success rate is greater than or equal to the preset success rate, and a task scheduling unit configured to determine the task quantity based on the resource information.

[0128] In the foregoing embodiments of the present application, the resource scheduling apparatus further comprises a priority determining unit configured to determine the priority of the plurality of to-be-distributed tasks, an execution parameter determining unit configured to determine the execution parameter of the to-be-distributed task distributed to any slave node based on the priority of the plurality of to-be-distributed tasks and the task quantity, wherein the execution parameter comprises at least one of the following: execution sequence and execution times, and a to-be-distributed task distributing unit configured to distribute the to-be-distributed task to any slave node based on the execution parameter of the to-be-distributed task.

[0129] It should be noted that, for each method embodiment described above, in order to simply describe, each method embodiment is described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0130] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software on a general hardware platform as necessary, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk), and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the method described in each embodiment of the present application.

[0131] Embodiment 4

[0132] According to the embodiments of the present application, a task scheduling device 900 for implementing the task scheduling method of Embodiment 1 is further provided, Figure 9 which is a structural schematic diagram of the task scheduling device 900 according to Embodiment 4 of the present application, as Figure 9 shown, the device includes:

[0133] The acquisition module 92 is configured to acquire a success rate of executing a task by any one of the slave nodes in the resource scheduling system, wherein the success rate is used to represent a ratio of a number of tasks successfully executed by the slave node in a first time period before the current time to a total number of tasks completed.

[0134] The distribution module 94 is configured to determine a number of tasks distributed to any one of the slave nodes in a second time period after the current time based on the success rate in a case where the success rate is less than a preset success rate.

[0135] The scheduling module 96 is configured to perform task scheduling on any one of the slave nodes based on the number of tasks.

[0136] It should be noted that the acquisition module 92, the distribution module 94, and the scheduling module 96 correspond to steps S402 to S406 in Embodiment 1, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules as part of the device can run in the computer terminal 10 provided in Embodiment 1.

[0137] In the above embodiments of the present application, the distribution module includes a target quantity determination unit configured to determine the number of tasks as a target quantity based on a product of the success rate in a case where the success rate is less than a preset success rate.

[0138] In the foregoing embodiments of the present application, the target quantity determination unit is further configured to: in a case where the product of the length of the second time period and the success rate is greater than or equal to a preset value, determine the task quantity based on the product of the length of the second time period and the success rate; and in a case where the product of the length of the second time period and the success rate is less than the preset value, determine the task quantity based on the preset value.

[0139] In the foregoing embodiments of the present application, the target quantity determination unit is further configured to: output the length of the second time period; receive a feedback length obtained by modifying the length of the second time period; and determine the target quantity based on the product of the feedback length and the success rate.

[0140] In the foregoing embodiments of the present application, the obtaining module comprises: an interval obtaining unit configured to obtain the success rate of the task executed by any one slave node once every preset time interval.

[0141] In the foregoing embodiments of the present application, the interval obtaining unit is further configured to: output the preset time interval; receive a feedback time interval obtained by modifying the preset time interval; and obtain the success rate of the task executed by any one slave node once every feedback time interval.

[0142] In the foregoing embodiments of the present application, the distribution module is further configured to: in a case where the success rate is less than a preset success rate, output the preset success rate; receive a feedback success rate obtained by modifying the preset success rate; and in a case where the success rate is less than the feedback success rate, determine the task quantity based on the success rate.

[0143] In the foregoing embodiments of the present application, the distribution module comprises: a resource information obtaining unit configured to, in a case where the success rate is greater than or equal to a preset success rate, obtain resource information reported by any one slave node; and a quantity determination unit configured to determine the task quantity based on the resource information.

[0144] In the foregoing embodiments of the present application, the distribution module comprises: a priority determination unit configured to determine priorities of the plurality of to-be-distributed tasks; an execution parameter determination unit configured to determine, based on the priorities of the plurality of to-be-distributed tasks and the task quantity, an execution parameter of a to-be-distributed task distributed to any one slave node, wherein the execution parameter comprises at least one of: an execution sequence and an execution times; and a to-be-distributed task distribution unit configured to distribute, based on the execution parameter of the to-be-distributed task, the to-be-distributed task to any one slave node.

[0145] It should be noted that the preferred embodiments involved in the foregoing embodiments of the present application have the same application scenarios and implementation processes as the scheme provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0146] Embodiment 5

[0147] According to the embodiment of the present application, a task scheduling device 1000 for implementing the task scheduling method of the above-mentioned embodiment 2 is further provided, Figure 10 is a structural schematic diagram of the task scheduling device 1000 according to the embodiment 5 of the present application, as shown in the figure, the device comprises: Figure 10

[0148] The obtaining module 1002 is configured to obtain a success rate of task execution by any slave node in the distributed cluster, wherein the success rate represents a ratio of a number of tasks successfully executed by the slave node within a first time period before the current time to a total number of completed tasks.

[0149] The distribution module 1004 is configured to determine a number of tasks distributed to any slave node within a second time period after the current time based on the success rate, in a case where the success rate is less than a preset success rate.

[0150] The scheduling module 1006 is configured to perform task scheduling on any slave node based on the number of tasks.

[0151] It should be noted that the above-mentioned obtaining module 1002, distribution module 1004 and scheduling module 1006 correspond to steps S702 to S706 in the embodiment 2, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above-mentioned embodiment 1. It should be noted that the above-mentioned modules as part of the device can run in the computer terminal 10 provided in the embodiment 1.

[0152] In the above-mentioned embodiments of the present application, the distribution module comprises a target quantity determination unit configured to determine the number of tasks as a target quantity based on a product of the success rate, in a case where the success rate is less than the preset success rate.

[0153] In the above-mentioned embodiments of the present application, the target quantity determination unit is further configured to determine the number of tasks based on a product of the length of the second time period and the success rate, in a case where the product of the length of the second time period and the success rate is greater than or equal to a preset value, and determine the number of tasks based on the preset value, in a case where the product of the length of the second time period and the success rate is less than the preset value.

[0154] In the above-mentioned embodiments of the present application, the target quantity determination unit is further configured to output the length of the second time period, receive a feedback length obtained by modifying the length of the second time period, and determine the target quantity based on a product of the feedback length and the success rate.

[0155] In the above-mentioned embodiments of the present application, the obtaining module comprises an interval obtaining unit configured to obtain the success rate of task execution by any slave node once every preset time interval.

[0156] ​In the foregoing embodiments of the present application, the interval obtaining unit is further configured to output a preset time interval; receive a feedback time interval obtained by modifying the preset time interval; and obtain the success rate of the task executed by the arbitrary slave node once per feedback time interval.

[0157] In the foregoing embodiments of the present application, the distribution module is further configured to output a preset success rate in the case that the success rate is less than the preset success rate; receive a feedback success rate obtained by modifying the preset success rate; and determine the number of tasks based on the success rate in the case that the success rate is less than the feedback success rate.

[0158] In the foregoing embodiments of the present application, the distribution module comprises a resource information obtaining unit configured to obtain the resource information reported by the arbitrary slave node in the case that the success rate is greater than or equal to the preset success rate; and a number determining unit configured to determine the number of tasks based on the resource information.

[0159] In the foregoing embodiments of the present application, the distribution module comprises a priority determining unit configured to determine the priority of the plurality of tasks to be distributed; an execution parameter determining unit configured to determine the execution parameter of the task to be distributed to the arbitrary slave node based on the priority of the plurality of tasks to be distributed and the number of tasks, wherein the execution parameter comprises at least one of the following: execution sequence and execution times; and a task to be distributed distribution unit configured to distribute the task to be distributed to the arbitrary slave node based on the execution parameter of the task to be distributed.

[0160] Embodiment 6

[0161] The embodiments of the present application can provide a computer terminal, which can be an arbitrary computer terminal device in a computer terminal group. Alternatively, in the embodiments, the computer terminal can be replaced by a mobile terminal or other terminal device.

[0162] Alternatively, in the embodiments, the computer terminal can be located in at least one network device of a plurality of network devices of a computer network.

[0163] In the embodiments, the computer terminal can execute the program codes of the following steps in the task scheduling method: obtaining the success rate of the task executed by an arbitrary slave node in a resource scheduling system, wherein the success rate represents the ratio of the number of tasks successfully executed by the slave node in a first time period before the current time to the total number of tasks completed; determining the number of tasks distributed to the arbitrary slave node in a second time period after the current time based on the success rate; and scheduling tasks for the arbitrary slave node based on the number of tasks.

[0164] Alternatively, Figure 11 is a structural block diagram of a computer terminal according to Embodiment 6 of the present application. As shown in Figure 11As shown, the computer terminal A can include one or more (only one is shown in the figure) processors 1102 and a memory 1104.

[0165] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the task scheduling method and device in the embodiments of the present application. The processor executes various functions and data processing by running the software programs and modules stored in the memory, that is, implements the task scheduling method described above. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the terminal A through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0166] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps:

[0167] S1, obtaining a success rate of executing a task by any one of the slave nodes in the resource scheduling system, wherein the success rate represents a ratio of a number of tasks successfully executed by the slave node in a first time period before the current time to a total number of tasks executed;

[0168] S2, determining a number of tasks distributed to any one of the slave nodes in a second time period after the current time based on the success rate;

[0169] S3, performing task scheduling on any one of the slave nodes based on the number of tasks.

[0170] Optionally, the processor can further execute program codes of the following steps: in a case where the success rate is less than a preset success rate, determining the number of tasks based on a product of a length of the second time period and the success rate.

[0171] Optionally, the processor can further execute program codes of the following steps: determining the number of tasks based on the product of the length of the second time period and the success rate, including: in a case where the product of the length of the second time period and the success rate is greater than or equal to a preset value, determining the number of tasks based on the product of the length of the second time period and the success rate; and in a case where the product of the length of the second time period and the success rate is less than the preset value, determining the number of tasks based on the preset value.

[0172] Optionally, the processor can further execute program codes of the following steps: outputting the length of the second time period; receiving a feedback length obtained by modifying the length of the second time period; and determining the target number based on a product of the feedback length and the success rate.

[0173] Optionally, the processor can further execute program codes of the following steps: obtaining the success rate of the task executed by any slave node once every preset time interval.

[0174] Optionally, the processor can further execute program codes of the following steps: obtaining the success rate of the task executed by any slave node once every preset time interval.

[0175] Optionally, the processor can further execute program codes of the following steps: outputting the preset time interval in the case that the success rate is less than a preset success rate; receiving a feedback time interval obtained by modifying the preset time interval; and obtaining the success rate of the task executed by any slave node once every feedback time interval.

[0176] Optionally, the processor can further execute program codes of the following steps: outputting the preset success rate in the case that the success rate is greater than or equal to the preset success rate; receiving a feedback success rate obtained by modifying the preset success rate; and determining the number of tasks based on the success rate in the case that the success rate is less than the feedback success rate.

[0177] Optionally, the processor can further execute program codes of the following steps: obtaining the resource information reported by any slave node in the case that the success rate is greater than or equal to the preset success rate; and determining the number of tasks based on the resource information.

[0178] Optionally, the processor can further execute program codes of the following steps: determining the priority of the plurality of to-be-distributed tasks; determining the execution parameter of the to-be-distributed task distributed to any slave node based on the priority of the plurality of to-be-distributed tasks and the number of tasks, wherein the execution parameter comprises at least one of the following: execution order and execution times; and distributing the to-be-distributed task to any slave node based on the execution parameter of the to-be-distributed task.

[0179] In the above embodiments of the present application, the success rate of the task executed by any slave node in the resource scheduling system can be obtained; the number of tasks distributed to any slave node in a second time period after the current time is determined based on the success rate; and the task scheduling for any slave node is performed based on the number of tasks. It is easy to note that, in the case that the success rate of the node is lower than the preset success rate, i.e., the slave node is in an abnormal state, the number of tasks distributed to the slave node in the second time period is determined based on the success rate, and then the number of tasks to be processed by the node in a certain time period can be dynamically controlled until the success rate of the node is greater than the preset success rate, i.e., the node returns to normal, the flow control is ended, and the technical problem that a large number of tasks are executed unsuccessfully due to an abnormal slave node in the related art is solved, and the technical effect of improving the task processing efficiency is achieved.

[0180] Those skilled in the art can understand that, Figure 11The structure shown is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, or the like. Figure 11 This does not limit the structure of the electronic device described above. For example, the computer terminal 11 can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in FIG. 1, or have a different configuration from that shown in FIG. 1. Figure 11 Figure 11 The structure shown is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, or the like.

[0181] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device by a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, etc.

[0182] Embodiment 7

[0183] The embodiments of the present application also provide a computer readable storage medium. Optionally, in the embodiment, the storage medium can be used to store the program code executed by the task scheduling method provided in Embodiment 1.

[0184] Optionally, in the embodiment, the storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0185] Optionally, in the embodiment, the storage medium is configured to store program code for executing the following steps:

[0186] S1, obtaining a success rate of executing a task by any one of the slave nodes in the resource scheduling system, wherein the success rate is used to represent a ratio of the number of tasks successfully executed by the slave node in a first time period before the current time to the total number of tasks executed;

[0187] S2, determining the number of tasks distributed to any one of the slave nodes in a second time period after the current time based on the success rate;

[0188] S3, performing task scheduling on any one of the slave nodes based on the number of tasks.

[0189] Optionally, the storage medium is configured to store program code for executing the following step: in the case that the success rate is less than a preset success rate, determining the number of tasks based on a product of the length of the second time period and the success rate. ​

[0190] Optionally, the storage medium is configured to store program code for performing the following steps: determining the number of tasks based on the product of the length of the second time period and the success rate, including: in a case where the product of the length of the second time period and the success rate is greater than or equal to a preset value, determining the number of tasks based on the product of the length of the second time period and the success rate; in a case where the product of the length of the second time period and the success rate is less than the preset value, determining the number of tasks based on the preset value.

[0191] Optionally, the storage medium is configured to store program code for performing the following steps: outputting the length of the second time period; receiving a feedback length obtained by modifying the length of the second time period; and determining the target number based on the product of the feedback length and the success rate.

[0192] Optionally, the storage medium is configured to store program code for performing the following steps: obtaining the success rate of the task executed by any one of the slave nodes once every preset time interval.

[0193] Optionally, the storage medium is configured to store program code for performing the following steps: obtaining the success rate of the task executed by any one of the slave nodes once every preset time interval.

[0194] Optionally, the storage medium is configured to store program code for performing the following steps: in a case where the success rate is less than a preset success rate, outputting the preset time interval; receiving a feedback time interval obtained by modifying the preset time interval; and obtaining the success rate of the task executed by any one of the slave nodes once every feedback time interval.

[0195] Optionally, the storage medium is configured to store program code for performing the following steps: in a case where the success rate is greater than or equal to a preset success rate, outputting the preset success rate; receiving a feedback success rate obtained by modifying the preset success rate; and in a case where the success rate is less than the feedback success rate, determining the number of tasks based on the success rate.

[0196] Optionally, the storage medium is configured to store program code for performing the following steps: in a case where the success rate is greater than or equal to a preset success rate, obtaining resource information reported by any one of the slave nodes; and determining the number of tasks based on the resource information.

[0197] Optionally, the storage medium is configured to store program code for performing the following steps: determining priorities of a plurality of to-be-distributed tasks; determining execution parameters of a to-be-distributed task distributed to any one of the slave nodes based on the priorities of the plurality of to-be-distributed tasks and the number of tasks, wherein the execution parameters include at least one of the following: execution order and execution times; and distributing the to-be-distributed task to any one of the slave nodes based on the execution parameters of the to-be-distributed task.

[0198] The above embodiment numbers of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0199] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0200] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the embodiment described above is only a schematic, for example, the division of units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, and can be electrical or other forms.

[0201] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0202] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0203] When the integrated unit is realized in the form of software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that makes a contribution to the prior art or the whole or part of the technical solutions can be embodied in the form of software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic or optical disk and various program code storage media.

[0204] The above merely describes the preferred embodiments of the present application, and it should be pointed out that, for those skilled in the art, some improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A task scheduling method, characterized by, The method comprises the following steps: obtaining a success rate of an arbitrary slave node in a resource scheduling system performing a task, wherein the success rate represents a ratio of a number of tasks successfully performed by the slave node within a first time period before a current time to a total number of tasks completed; determining a number of tasks distributed to the arbitrary slave node within a second time period after the current time based on the success rate; scheduling tasks for the arbitrary slave node based on the number of tasks; wherein, in a case where the success rate is less than a preset success rate, the number of tasks is determined based on a product of a length of the second time period and the success rate, wherein the number of tasks is a maximum value between the product and a preset value.

2. The method of claim 1, wherein, determining the number of tasks based on a product of the length of the second time period and the success rate, comprising: outputting the length of the second time period; receiving a feedback length obtained by modifying the length of the second time period; determining the number of tasks based on a product of the feedback length and the success rate.

3. The method of claim 1, wherein, obtaining a success rate of an arbitrary slave node in a resource scheduling system performing a task, comprising: obtaining the success rate of the arbitrary slave node performing the task once every preset time interval.

4. The method of claim 3, wherein, obtaining the success rate of the arbitrary slave node performing the task once every preset time interval, comprising: outputting the preset time interval; receiving a feedback time interval obtained by modifying the preset time interval; obtaining the success rate of the arbitrary slave node performing the task once every feedback time interval.

5. The method of claim 1, wherein, in a case where the success rate is less than a preset success rate, determining a number of tasks distributed to the arbitrary slave node within a second time period based on the success rate, comprising: outputting the preset success rate; receiving a feedback success rate obtained by modifying the preset success rate; in a case where the success rate is less than the feedback success rate, determining the number of tasks based on the success rate.

6. The method of claim 1, wherein, determining a number of tasks distributed to the arbitrary slave node within a second time period based on the success rate, comprising: in a case where the success rate is greater than or equal to the preset success rate, obtaining resource information reported by the arbitrary slave node; determining the number of tasks based on the resource information.

7. The method of claim 1, wherein, scheduling tasks for the arbitrary slave node based on the number of tasks, comprising: determining priorities of a plurality of to-be-distributed tasks; determining execution parameters of a to-be-distributed task distributed to the arbitrary slave node based on the priorities of the plurality of to-be-distributed tasks and the number of tasks, wherein the execution parameters comprise at least one of the following: an execution order and an execution times; distributing the to-be-distributed task to the arbitrary slave node based on the execution parameters of the to-be-distributed task.

8. A task scheduling method characterized by comprising: The method comprises the following steps: obtaining a success rate of an arbitrary slave node in a distributed cluster performing a task, wherein the success rate represents a ratio of a number of tasks successfully performed by the slave node within a first time period before a current time to a total number of tasks completed; determine a number of tasks to be distributed to the arbitrary slave node in a second time period after the current time based on the success rate; schedule tasks for the arbitrary slave node based on the number of tasks; wherein the determining the number of tasks to be distributed to the arbitrary slave node in the second time period after the current time based on the success rate comprises: in a case that the success rate is less than a preset success rate, determining the number of tasks based on a product of a length of the second time period and the success rate, wherein the number of tasks is a maximum value between the product and a preset value.

9. A task scheduling system, characterized by comprise: a plurality of slave nodes configured to execute tasks; a master node connected with the plurality of slave nodes and configured to distribute the tasks to the slave nodes; a resource scheduling apparatus connected with the master node and configured to acquire a success rate of an arbitrary slave node in executing the tasks, determine a number of tasks to be distributed to the arbitrary slave node in a second time period after a current time based on the success rate, and schedule tasks for the arbitrary slave node based on the number of tasks, wherein the success rate represents a ratio of a number of tasks successfully executed by the slave node in a first time period before the current time to a total number of tasks completed; wherein the determining the number of tasks to be distributed to the arbitrary slave node in the second time period after the current time based on the success rate comprises: in a case that the success rate is less than a preset success rate, determining the number of tasks based on a product of a length of the second time period and the success rate, wherein the number of tasks is a maximum value between the product and a preset value.

10. The system of claim 9, wherein, The resource scheduling apparatus comprises: a node management unit configured to determine a number of tasks to be distributed to the arbitrary slave node in a second time period based on the success rate; a scheduling unit configured to schedule tasks for the arbitrary slave node based on the number of tasks.

11. A computer readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the program, when executed, controls a device where the storage medium is located to perform the method of any one of claims 1 to 8.

12. An electronic device, comprising: comprise: a memory storing an executable program; a processor configured to execute the program, wherein the program, when executed, performs the method of any one of claims 1 to 8.

Images