Task scheduling method, system and equipment and storage medium

Abstract

The invention provides a task scheduling method, system and device and a storage medium, and the method comprises the steps that a first server locks a received target task under the condition that a first Yarn state corresponding to a database is inconsistent with a second Yarn state corresponding to a Yarn platform; the first server sends the locked target task to a scheduling component; the scheduling component verifies the locked target task; the scheduling component allocates the target task to the target server according to the resource configuration of each second server under the condition that the verification is passed; and the target server calls the Yarn platform to schedule the target task and updates the first Yarn state. In the embodiment of the invention, the target task is allocated to the target server through the scheduling component, and then the target server calls the Yarn platform to schedule the target task, so that the scheduling time is saved, and the task scheduling efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a task scheduling method, system, device and storage medium. Background Art

[0002] Task scheduling is the core link in the task management system. Through task scheduling, the resources in the cluster are coordinated to process different tasks, so as to achieve timely response and management of tasks. The task scheduling system usually includes a server, a database, and a Yarn platform. During the task scheduling process, task scheduling is achieved while ensuring that the Yarn status recorded in the database is consistent with the Yarn status corresponding to the Yarn platform.

[0003] In the prior art, if the Yarn status recorded in the database is inconsistent with the Yarn status corresponding to the Yarn platform, it is necessary to start scheduled task scheduling, that is, to perform task scheduling after a preset interval, so as to ensure the consistency between the Yarn status recorded in the database and the Yarn status corresponding to the Yarn platform.

[0004] However, the above task scheduling method adopts a timed scheduling method, which consumes extra scheduling time and thus reduces the efficiency of task scheduling. Summary of the invention

[0005] The main purpose of this application is to provide a task scheduling method, system, computer device and storage medium, aiming to solve the technical problem that the prior art adopts a timed scheduling method, which consumes extra scheduling time and thus reduces the efficiency of task scheduling.

[0006] To achieve the above object, the present application provides a task scheduling method, which is applied to a task scheduling system, wherein the task scheduling system includes a first server, a scheduling component, N second servers, a database, and a Yarn platform, where N is a positive integer greater than 1, the first server is communicatively connected to the scheduling component, the scheduling component is communicatively connected to the N second servers, the N second servers are communicatively connected to the database and the Yarn platform, and the task scheduling method includes:

[0007] When the first Yarn state corresponding to the database is inconsistent with the second Yarn state corresponding to the Yarn platform, the first server locks the received target task;

[0008] The first server sends the locked target task to the scheduling component;

[0009] The scheduling component verifies the locked target task;

[0010] When the verification is passed, the scheduling component allocates the target task to a target server according to the resource configuration of each second server; the target server belongs to the N second servers;

[0011] The target server calls the Yarn platform to schedule the target task and updates the first Yarn state.

[0012] Optionally, when the first Yarn state corresponding to the database is inconsistent with the second Yarn state corresponding to the Yarn platform, before the first server locks the received target task, the method further includes:

[0013] When the first server receives a user operation, and the user operation is used to update the first Yarn state, it determines that the first Yarn state is inconsistent with the second Yarn state.

[0014] Optionally, when the first Yarn state corresponding to the database is inconsistent with the second Yarn state corresponding to the Yarn platform, before the first server locks the received target task, the method further includes:

[0015] The first server periodically scans the database and the Yarn platform, and obtains the first Yarn status and the second Yarn status;

[0016] When the first Yarn state is different from the second Yarn state, the first server determines that the first Yarn state is inconsistent with the second Yarn state.

[0017] Optionally, the scheduling component verifies the locked target task, including:

[0018] The scheduling component obtains lock identification information corresponding to the locked target task;

[0019] If the lock identification information is stored in the preset storage table, the scheduling component locks the locked target task again and determines that the verification is passed;

[0020] Wherein, the storage table stores identification information.

[0021] Optionally, the scheduling component locks the locked target task again, including:

[0022] The scheduling component inputs the lock identification information into a storage table for query; the storage table also stores a mapping relationship between the identification information and the process information;

[0023] If the storage table does not store the process information corresponding to the lock identification information, the scheduling component creates a locking process, and calls the locking process to lock the locked target task again;

[0024] If the storage table stores the process information corresponding to the lock identification information, the locked target task is locked again.

[0025] Optionally, allocating the target task to the target server according to the resource configuration of each second server includes:

[0026] The scheduling component obtains the occupied resources corresponding to each second server;

[0027] The scheduling component determines the second server with the least occupied resources among the N second servers as the target server;

[0028] The scheduling component distributes the target task to the target server.

[0029] Optionally, the method further comprises:

[0030] When receiving the target task, the scheduling component periodically monitors the target task;

[0031] If the target task is not scheduled by the Yarn platform within a preset time, the scheduling component issues an alarm message.

[0032] In addition, to achieve the above purpose, the present application also provides a task scheduling system, including a first server, a scheduling component, N second servers, a database and a Yarn platform, N is a positive integer greater than 1, the first server is communicatively connected to the scheduling component, the scheduling component is communicatively connected to the N second servers, and the N second servers are communicatively connected to the database and the Yarn platform;

[0033] The first server is configured to lock the received target task when a first Yarn state corresponding to the database is inconsistent with a second Yarn state corresponding to the Yarn platform;

[0034] Sending the locked target task to the scheduling component;

[0035] The scheduling component is used to verify the locked target task;

[0036] If the verification is passed, the target task is allocated to the target server according to the resource configuration of each second server; the target server belongs to the N second servers;

[0037] The target server is used to call the Yarn platform to schedule the target task and update the first Yarn state.

[0038] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0039] The computer device includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of any one of the task scheduling methods proposed in the embodiments of the present application are implemented.

[0040] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0041] The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of any one of the task scheduling methods proposed in the embodiments of the present application are implemented.

[0042] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0043] The present application provides a task scheduling method, system, device and storage medium, the method comprising: when the first Yarn state corresponding to the database is inconsistent with the second Yarn state corresponding to the Yarn platform, the first server locks the received target task; the first server sends the locked target task to the scheduling component; the scheduling component verifies the locked target task; when the verification is passed, the scheduling component allocates the target task to the target server according to the resource configuration of each second server; the target server belongs to N second servers; the target server calls the Yarn platform to schedule the target task and updates the first Yarn state. In an embodiment of the present application, when the Yarn state recorded in the database is inconsistent with the Yarn state corresponding to the Yarn platform, the target task is allocated to the target server through the scheduling component, and then the target server calls the Yarn platform to schedule the target task, instead of scheduling the target task regularly, thereby saving scheduling time and improving the efficiency of task scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1is an exemplary system architecture diagram to which the present application may be applied;

[0046] Figure 2 is a flowchart of a task scheduling method provided in an embodiment of the present application;

[0047] Figure 3 is an application scenario diagram of the task scheduling method provided in an embodiment of the present application;

[0048] Figure 4 It is a structural diagram of an embodiment of a task scheduling system provided in an embodiment of the present application;

[0049] Figure 5 It is a basic structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The task scheduling method provided in the embodiment of the present application is applied to the task scheduling system. Unless otherwise defined, all technical and scientific terms used in this article have the same meaning as those generally understood by technicians in the technical field of this application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned figures are used to distinguish different objects, not to describe a specific order.

[0051] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0053] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0054] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social online platform software, etc.

[0055] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.

[0056] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0057] It should be noted that the task scheduling method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the task scheduling system is generally set in the server / terminal device.

[0058] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0059] Please refer to Figure 2 , shows a flow chart of an embodiment of the task scheduling method proposed in the present application. The embodiment of the present application can acquire and process relevant data based on artificial intelligence technology.

[0060] It should be noted that the task scheduling method provided in the embodiment of the present application can be applied to a task scheduling system, which includes a first server, a scheduling component, N second servers, a database and a Yarn platform, where N is a positive integer greater than 1.

[0061] The first server is communicatively connected to the scheduling component, the scheduling component is communicatively connected to N second servers, and the N second servers are communicatively connected to the database and the Yarn platform.

[0062] The task scheduling method provided in the embodiment of the present application comprises the following steps:

[0063] S210: When a first Yarn state corresponding to a database is inconsistent with a second Yarn state corresponding to the Yarn platform, the first server locks the received target task.

[0064] It should be understood that the resource coordinator (Yet Another Resource Negotiator, Yarn) state recorded in the database is called the first Yarn state, and the Yarn state corresponding to the Yarn platform is called the second Yarn state.

[0065] In this step, when the first Yarn state is inconsistent with the second Yarn state, the first server locks the received target task.

[0066] It should be understood that locking is to prevent multiple threads from executing the same scheduled task at the same time, which may lead to data competition and inconsistency. By locking the target task, the execution order of the task and the consistency of the data are guaranteed, avoiding problems caused by concurrency.

[0067] S220: The first server sends the locked target task to the scheduling component.

[0068] In this step, after performing a locking operation on the target task, the first server sends the locked target task to the scheduling component.

[0069] S230, the scheduling component verifies the locked target task.

[0070] Optionally, the scheduling component is a quartz component.

[0071] In this step, after receiving the locked target task, the scheduling component verifies the locked target task to ensure that the target task has not been tampered with. For specific implementation methods, please refer to the subsequent embodiments.

[0072] Among them, the locking method can use a mutex lock, a read-write lock, etc.

[0073] S240: When the verification is passed, the scheduling component allocates the target task to the target server according to the resource configuration of each second server.

[0074] In this step, if the verification is passed, the target task is allocated to the target server according to the resource configuration of each second server, wherein the target server belongs to the N second servers. For specific implementation methods, please refer to the subsequent embodiments.

[0075] S250, the target server calls the Yarn platform to schedule the target task and updates the first Yarn state.

[0076] In this step, after receiving the target task, the target server calls the Yarn platform, schedules the target task through the Yarn platform, and updates the Yarn status recorded in the database.

[0077] In an embodiment of the present application, when the Yarn status recorded in the database is inconsistent with the Yarn status corresponding to the Yarn platform, the target task is assigned to the target server through the scheduling component, and then the target server calls the Yarn platform to schedule the target task, rather than scheduling the target task on a regular basis, thereby saving scheduling time and improving the efficiency of task scheduling.

[0078] In addition, by allocating target tasks to target servers, target tasks can be executed in a distributed manner on different servers instead of scheduling all tasks on one server, thereby improving the availability of the task scheduling method.

[0079] In addition, the task scheduling system includes N second servers, and the scheduling of a single task is allocated to different second servers, thereby improving the performance of task scheduling.

[0080] Optionally, when the first Yarn state corresponding to the database is inconsistent with the second Yarn state corresponding to the Yarn platform, before the first server locks the received target task, the method further includes:

[0081] When the first server receives a user operation, and the user operation is used to update the first Yarn state, it determines that the first Yarn state is inconsistent with the second Yarn state.

[0082] One possible situation is that the user's active operation causes the Yarn status recorded in the database to be inconsistent with the Yarn status corresponding to the Yarn platform.

[0083] Specifically, the first server receives a user operation for updating the first Yarn state, and updates the first Yarn state in response to the user operation. In this case, it is determined that the first Yarn state is inconsistent with the second Yarn state.

[0084] Optionally, when the first Yarn state corresponding to the database is inconsistent with the second Yarn state corresponding to the Yarn platform, before the first server locks the received target task, the method further includes:

[0085] The first server periodically scans the database and the Yarn platform, and obtains the first Yarn status and the second Yarn status;

[0086] When the first Yarn state is different from the second Yarn state, the first server determines that the first Yarn state is inconsistent with the second Yarn state.

[0087] Another possible situation is that the running status of the Yarn task changes, that is, the Yarn status corresponding to the Yarn platform changes.

[0088] Specifically, a first server is set to periodically scan a database and a Yarn platform, and a first Yarn state corresponding to the database is periodically obtained by periodically scanning the database. A second Yarn state corresponding to the Yarn platform is periodically obtained by periodically scanning the Yarn platform. When the first Yarn state is different from the second Yarn state, it is determined that the first Yarn state is inconsistent with the second Yarn state.

[0089] In the above embodiment, whether the first Yarn state is consistent with the second Yarn state is determined in two ways.

[0090] It should be noted that, when the first Yarn state is consistent with the second Yarn state, the first server can directly assign the target task to the second server, and the second server calls the Yarn platform to schedule the target task.

[0091] The following describes in detail how to verify the locked target task:

[0092] Optionally, the scheduling component verifies the locked target task, including:

[0093] The scheduling component obtains lock identification information corresponding to the locked target task;

[0094] If the lock identification information is stored in the preset storage table, the scheduling component locks the locked target task again and determines that the verification is passed;

[0095] Wherein, the storage table stores identification information.

[0096] In this embodiment, a storage table is pre-set, and the storage table stores identification information. Optionally, the mapdata attribute in the quartz component can be used to store identification information, that is, in an optional implementation, the scheduling component includes a storage table.

[0097] In this embodiment, the scheduling component obtains the lock identification information corresponding to the locked target task, and queries the lock identification information in the storage table. If the lock identification information is obtained in the storage table, it means that the target task has been locked by the registered lock, and the verification is determined to be passed; if the lock identification information is not obtained in the storage table, it is determined that the verification has not passed. The lock identification information can be understood as a lock id.

[0098] It should be understood that, when the verification fails, the scheduling component sends an alarm message to the first server, and the first server sends the above alarm message to the client to prompt that a deadlock exists in the target task.

[0099] In this embodiment, the locked target task is verified by the scheduling component to ensure that the target task has not been tampered with during the transmission process, and the locked target task can be executed in a single line, thereby improving the efficiency of task scheduling.

[0100] Optionally, the scheduling component locks the locked target task again, including:

[0101] The scheduling component inputs the lock identification information into a storage table for query; the storage table also stores a mapping relationship between the identification information and the process information;

[0102] If the storage table does not store the process information corresponding to the lock identification information, the scheduling component creates a locking process, and calls the locking process to lock the locked target task again;

[0103] If the storage table stores the process information corresponding to the lock identification information, the locked target task is locked again.

[0104] It should be noted that the storage table also stores the mapping relationship between the identification information and the process information. Optionally, the mapdata attribute in the quartz component can be used for the mapping relationship between the identification information and the process information, that is, in an optional implementation, the scheduling component includes a storage table.

[0105] In this embodiment, the scheduling component inputs the lock identification information into the storage table. If the process information corresponding to the lock identification information is queried in the above storage table, the scheduling component locks the locked target task again; if the process information corresponding to the lock identification information is not queried in the above storage table, the scheduling component creates a locking process and calls the locking process to lock the locked target task again.

[0106] It should be noted that the above process of locking the locked target task again can be understood as reentrant locking of the target task. Reentrant locking is a lock mechanism that allows the same thread to acquire the same lock multiple times without deadlock. If a thread already holds a lock, it can acquire the lock again without being blocked.

[0107] In this embodiment, when the storage table stores process information corresponding to the lock identification information, the locked target task is locked again to avoid concurrent scheduling of tasks, thereby improving task scheduling efficiency. When the storage table does not store process information corresponding to the lock identification information, it indicates that a deadlock has occurred. In this way, a locked process is created to solve the deadlock and ensure smooth task scheduling.

[0108] Optionally, allocating the target task to the target server according to the resource configuration of each second server includes:

[0109] The scheduling component obtains the occupied resources corresponding to each second server;

[0110] The scheduling component determines the second server with the least occupied resources among the N second servers as the target server;

[0111] The scheduling component distributes the target task to the target server.

[0112] In this embodiment, after receiving the target task, the scheduling component obtains the occupied resources corresponding to each second server, determines the second server with the least occupied resources among the N second servers as the target server, and assigns the target task to the target server, thereby ensuring that the server processing the target task has the corresponding resource configuration and improving the processing efficiency of task scheduling.

[0113] To understand the overall technical solution, please refer to Figure 3 , Figure 3 The task scheduling system shown includes a first server, two second servers, a scheduling component, a database, and a Yarn platform.

[0114] exist Figure 3 In the application scenario shown, the user sends a target task to the first server, the first server locks the target task and sends the locked target task to the scheduling component. After verifying the target task, the scheduling component locks the target task again and assigns the target task to the second server. The second server schedules the target task through the Yarn platform and updates the Yarn status recorded in the database.

[0115] Optionally, the method further comprises:

[0116] When receiving the target task, the scheduling component periodically monitors the target task;

[0117] If the target task is not scheduled by the Yarn platform within a preset time, the scheduling component issues an alarm message.

[0118] In this embodiment, the scheduling component monitors the target task regularly while receiving the target task sent by the first server. If the target task is not scheduled by the Yarn platform within a preset time, that is, a scheduling timeout occurs, the scheduling component issues an alarm message.

[0119] In other embodiments, the scheduling component periodically monitors the target task while locking it. If the target task is not locked again by the scheduling component within a preset time, that is, a lock timeout occurs, the scheduling component issues an alarm message.

[0120] The embodiment of the present application monitors the target task in the above manner to ensure the smooth scheduling of the target task.

[0121] See also Figure 4 , an embodiment of the present application provides a task scheduling system 400, the task scheduling system 400 includes a first server 410, a scheduling component 420, N second servers 430, a database 440 and a Yarn platform 450, N is a positive integer greater than 1, the first server 410 is connected in communication with the scheduling component 420, the scheduling component 420 is connected in communication with the N second servers 430, and the N second servers 430 are connected in communication with the database 440 and the Yarn platform 450;

[0122] The first server 410 is used to lock the received target task when the first Yarn state corresponding to the database 440 is inconsistent with the second Yarn state corresponding to the Yarn platform 450;

[0123] Sending the locked target task to the scheduling component 420;

[0124] The scheduling component 420 is used to verify the locked target task;

[0125] In the case of passing the verification, the target task is allocated to a target server according to the resource configuration of each second server 430; the target server belongs to the N second servers 430;

[0126] The target server is used to call the Yarn platform 450 to schedule the target task and update the first Yarn state.

[0127] Optionally, the first server 410 is further configured to, upon receiving a user operation, where the user operation is used to update the first Yarn state, determine that the first Yarn state is inconsistent with the second Yarn state.

[0128] Optionally, the first server 410 is further configured to periodically scan the database and the Yarn platform, and obtain the first Yarn status and the second Yarn status;

[0129] When the first Yarn state is different from the second Yarn state, it is determined that the first Yarn state is inconsistent with the second Yarn state.

[0130] Optionally, the scheduling component 420 is further used to obtain lock identification information corresponding to the locked target task;

[0131] If the lock identification information is stored in the preset storage table, the locked target task is locked again, and it is determined that the verification is passed;

[0132] Wherein, the storage table stores identification information.

[0133] Optionally, the scheduling component 420 is further used to input the lock identification information into a storage table for query; the storage table also stores a mapping relationship between the identification information and the process information;

[0134] If the storage table does not store the process information corresponding to the lock identification information, the scheduling component creates a locking process, and calls the locking process to lock the locked target task again;

[0135] If the storage table stores the process information corresponding to the lock identification information, the locked target task is locked again.

[0136] Optionally, the scheduling component 420 is further used to obtain occupied resources corresponding to each second server 430;

[0137] Determine the second server 430 with the least occupied resources among the N second servers 430 as the target server;

[0138] Allocate the target task to the target server.

[0139] Optionally, the scheduling component 420 is further configured to perform regular monitoring on the target task when the target task is received;

[0140] If the target task is not scheduled by the Yarn platform 450 within a preset time, an alarm message is issued.

[0141] To solve the above technical problems, the present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0142] The computer device 5 includes a memory 51, a processor 52, and a network interface 53 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 5 with components 51-53, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0143] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0144] The memory 51 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 51 can be an internal storage unit of the computer device 5, such as a hard disk or memory of the computer device 5. In other embodiments, the memory 51 can also be an external storage device of the computer device 5, such as a plug-in hard disk equipped on the computer device 5, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 51 can also include both the internal storage unit of the computer device 5 and its external storage device. In this embodiment, the memory 51 is generally used to store the operating system and various application software installed on the computer device 5, such as the program code of the task scheduling method, etc. In addition, the memory 51 can also be used to temporarily store various types of data that have been output or are to be output.

[0145] The processor 52 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 52 is generally used to control the overall operation of the computer device 5. In this embodiment, the processor 52 is used to run the program code stored in the memory 51 or process data, such as running the program code of the task scheduling method.

[0146] The network interface 53 may include a wireless network interface or a wired network interface. The network interface 53 is generally used to establish a communication connection between the computer device 5 and other electronic devices.

[0147] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores the task scheduling program, and the task scheduling program can be executed by at least one processor so that the at least one processor performs the steps of the task scheduling method as described above.

[0148] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware online platform, and of course, by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0149] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0150] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A task scheduling method, characterized in that: The method is applied to a task scheduling system, which includes a first server, a scheduling component, N second servers, a database and a Yarn platform, where N is a positive integer greater than 1, the first server is communicatively connected to the scheduling component, the scheduling component is communicatively connected to the N second servers, the N second servers are communicatively connected to the database and the Yarn platform, and the task scheduling method includes: When the first Yarn state corresponding to the database is inconsistent with the second Yarn state corresponding to the Yarn platform, the first server locks the received target task; The first server sends the locked target task to the scheduling component; The scheduling component verifies the locked target task; When the verification is passed, the scheduling component allocates the target task to a target server according to the resource configuration of each second server; the target server belongs to the N second servers; The target server calls the Yarn platform to schedule the target task and updates the first Yarn state.

2. The method according to claim 1, characterized in that When the first server locks the received target task when the first Yarn state corresponding to the database is inconsistent with the second Yarn state corresponding to the Yarn platform, the method further includes: When the first server receives a user operation, and the user operation is used to update the first Yarn state, it determines that the first Yarn state is inconsistent with the second Yarn state.

3. The method according to claim 1, characterized in that When the first server locks the received target task when the first Yarn state corresponding to the database is inconsistent with the second Yarn state corresponding to the Yarn platform, the method further includes: The first server periodically scans the database and the Yarn platform, and obtains the first Yarn status and the second Yarn status; When the first Yarn state is different from the second Yarn state, the first server determines that the first Yarn state is inconsistent with the second Yarn state.

4. The method according to claim 1, characterized in that: The scheduling component verifies the locked target task, including: The scheduling component obtains lock identification information corresponding to the locked target task; If the lock identification information is stored in the preset storage table, the scheduling component locks the locked target task again and determines that the verification is passed; Wherein, the storage table stores identification information.

5. The method according to claim 4, characterized in that The scheduling component locks the locked target task again, including: The scheduling component inputs the lock identification information into a storage table for query; the storage table also stores a mapping relationship between the identification information and the process information; If the storage table does not store the process information corresponding to the lock identification information, the scheduling component creates a locking process, and calls the locking process to lock the locked target task again; If the storage table stores the process information corresponding to the lock identification information, the locked target task is locked again.

6. The method according to claim 1, characterized in that The step of allocating the target task to the target server according to the resource configuration of each second server includes: The scheduling component obtains the occupied resources corresponding to each second server; The scheduling component determines the second server with the least occupied resources among the N second servers as the target server; The scheduling component distributes the target task to the target server.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: When receiving the target task, the scheduling component periodically monitors the target task; If the target task is not scheduled by the Yarn platform within a preset time, the scheduling component issues an alarm message.

8. A task scheduling system, characterized in that: It includes a first server, a scheduling component, N second servers, a database and a Yarn platform, where N is a positive integer greater than 1, the first server is communicatively connected with the scheduling component, the scheduling component is communicatively connected with the N second servers, and the N second servers are communicatively connected with the database and the Yarn platform; The first server is configured to lock the received target task when a first Yarn state corresponding to the database is inconsistent with a second Yarn state corresponding to the Yarn platform; Sending the locked target task to the scheduling component; The scheduling component is used to verify the locked target task; If the verification is successful, the target task is allocated to the target server according to the resource configuration of each second server; The target server belongs to the N second servers; The target server is used to call the Yarn platform to schedule the target task and update the first Yarn state.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the task scheduling method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 7 are implemented.

Images