Pluggable scheduling method, system, device and storage medium based on Elastic job

Abstract

The present invention discloses a pluggable scheduling method, system, device and storage medium based on Elastic job. The method comprises: using nacos as a service registration center, registering job and web to nacos; uploading or updating the Jar of the task through the interface of the web, and reading or updating the key of the execution parameter corresponding to the task through the database; obtaining the key of the execution parameter of the job, and filling in the value corresponding to the task in the web page; calling the interface of the job through the interface of rpc through the web, dynamically loading the Jar of different tasks, and realizing pluggable task start and stop through Elastic job. The heavy dependence of Elastic Job tasks on the platform is solved, and the flexibility is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a pluggable scheduling method, system, device and storage medium based on Elastic job. Background Art

[0002] Elastic Job is a distributed scheduling solution based on Quartz provided by Dangdang.com. It consists of two sub-projects: Elastic-Job-Lite and Elastic-Job-Cloud. Job-Lite is a lightweight and decentralized solution that provides distributed task scheduling services in the form of Jar packages. However, in our actual use, if we need to add a new task, we need to develop the task and then re-publish and deploy the entire schedule. There is a strong dependency between tasks, and tasks cannot be started and stopped with one click. Summary of the invention

[0003] The present invention aims at the technical problem that the existing Elastic Job can solve the scheduling problem of tasks, but cannot achieve flexible starting and stopping of tasks, and flexible deletion of tasks, and proposes a pluggable scheduling method, system, device and storage medium based on Elastic job.

[0004] In a first aspect, an embodiment of the present application provides a pluggable scheduling method based on Elastic job, including:

[0005] Registration steps: Use nacos as the service registration center and register job and web to the nacos;

[0006] Task update step: upload or update the task Jar through the web interface, and read or update the key of the execution parameter corresponding to the task through the database;

[0007] Parameter acquisition step: obtain the key of the execution parameter of the job, and fill in the value corresponding to the task in the web page;

[0008] Task execution steps: call the job interface through the web using the RPC interface, dynamically load different task Jars, and implement pluggable task start and stop through Elastic job.

[0009] In the above-mentioned pluggable scheduling method, the task update step includes: determining whether the Jar of the task exists, if so, overwriting the original Jar and updating the key of the execution parameter; if not, storing the Jar of the task, reading the key of the execution parameter and inserting it into the database.

[0010] In the above-mentioned pluggable scheduling method, the task execution step includes:

[0011] Task start step: determine whether the Jar is in the cache. If so, read the Jar and initialize the Elastic job task, and the Elastic job starts to execute the task according to the configuration; if not, first read the path where the Jar is located, load the Jar into the cache, and then read it;

[0012] Task suspension step: deleting the node in ZK and the corresponding Jar in the cache, and calling the task stop interface in the Elasticjob to suspend the task.

[0013] In the above-mentioned pluggable scheduling method, the web and the job share a network storage, and the network storage is used to store the Jar of the task.

[0014] In the above-mentioned pluggable scheduling method, the changes in the status of the task in the job are synchronized to the database table by means of listen and displayed on the web page.

[0015] In a second aspect, an embodiment of the present application provides a pluggable scheduling system based on Elastic job, including:

[0016] Registration unit: using nacos as the service registration center, registering jobs and webs to the nacos;

[0017] Task update unit: upload or update the task Jar through the web interface, and read or update the key of the execution parameter corresponding to the task through the database;

[0018] Parameter acquisition unit: obtains the key of the execution parameter of the job, and fills in the value corresponding to the task in the web page;

[0019] Task execution unit: calls the job interface through the web using the RPC interface, dynamically loads different task Jars, and implements pluggable task start and stop through Elastic job.

[0020] The above-mentioned pluggable scheduling system, wherein the task update unit includes: determining whether the Jar of the task exists, if so, overwriting the original Jar and updating the key of the execution parameter; if not, storing the Jar of the task, reading the key of the execution parameter and inserting it into the database.

[0021] The above-mentioned pluggable scheduling system, wherein the task execution unit includes:

[0022] Task startup module: determines whether the Jar is in the cache. If so, reads the Jar and initializes the Elastic job task, which starts to execute the task according to the configuration; if not, first reads the path where the Jar is located, loads the Jar into the cache, and then reads it;

[0023] Task suspension module: deletes the node in ZK and the corresponding Jar in the cache, and calls the task stop interface in the Elasticjob to suspend the task.

[0024] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the pluggable scheduling method based on Elastic job as described in the first aspect above is implemented.

[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pluggable scheduling method based on Elastic job as described in the first aspect above.

[0026] Compared with the prior art, the advantages and positive effects of the present invention are:

[0027] The present invention adds a web page to the Elastic Job, provides a task start and stop interface and status change display, and also provides a task Jar upload interface, which can flexibly add, delete and modify tasks without redeploying the entire job platform, thus solving the heavy dependence of the Elastic Job task on the platform and greatly improving the flexibility of data capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of the steps of a pluggable scheduling method based on Elastic job provided by the present invention;

[0029] Figure 2 A pluggable scheduling solution architecture design diagram based on Elastic job provided by the present invention;

[0030] Figure 3 The flowchart of the web task Jar upload and update provided by the present invention;

[0031] Figure 4 A schematic diagram of the task start and stop process provided by the present invention;

[0032] Figure 5A framework diagram of a pluggable scheduling system based on Elastic job provided by the present invention;

[0033] Figure 6 A framework diagram of a computer device according to an embodiment of the present application.

[0034] Wherein, the accompanying drawings are marked as follows:

[0035] 1. Registration unit; 2. Task update unit; 3. Parameter acquisition unit; 4. Task execution unit; 41. Task start module; 42. Task pause module; 81. Processor; 82. Memory; 83. Communication interface; 80. Bus. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0037] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0038] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0039] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0040] The present invention is described in detail below in conjunction with the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and any equivalent transformations or substitutions in functions, methods, or structures made by ordinary technicians in the field based on these embodiments are all within the scope of protection of the present invention.

[0041] Before elaborating on each embodiment of the present invention in detail, the core inventive concept of the present invention is summarized and elaborated in detail through the following several embodiments.

[0042] The present invention proposes to register the job and the web to nacos, upload or update the task Jar through the web interface, flexibly add, delete and modify the tasks, call the job interface through the rpc interface, dynamically load the Jars of different tasks, and realize pluggable task start and stop through Elastic job.

[0043] Embodiment 1:

[0044] Figure 1 A schematic diagram of the steps of a pluggable scheduling method based on Elastic job provided by the present invention. Figure 1 As shown, this embodiment discloses a specific implementation method of a pluggable scheduling method based on Elastic job (hereinafter referred to as "method").

[0045] Specifically, the method disclosed in this embodiment mainly includes the following steps:

[0046] Step S1: Using nacos as the service registration center, register job and web to the nacos;

[0047] Specifically, the web and the job share a network storage, and the network storage is used to store the task Jar. The changes in the task status in the job are synchronized to the database table through the listen method and displayed on the web page.

[0048] Step S2: Upload or update the task Jar through the web interface, and read or update the key of the execution parameter corresponding to the task through the database;

[0049] Specifically, the step S2 includes: determining whether the Jar of the task exists, if so, overwriting the original Jar and updating the key of the execution parameter; if not, storing the Jar of the task, reading the key of the execution parameter and inserting it into the database.

[0050] Step S3: Get the key of the execution parameter of the job, and fill in the value corresponding to the task in the web page;

[0051] Step S4: Call the job interface through the web using the RPC interface, dynamically load different task Jars, and implement pluggable task start and stop through Elastic job.

[0052] Specifically, step S4 includes:

[0053] Task start step: determine whether the Jar is in the cache. If so, read the Jar and initialize the Elastic job task, and the Elastic job starts to execute the task according to the configuration; if not, first read the path where the Jar is located, load the Jar into the cache, and then read it;

[0054] Task suspension step: deleting the node in ZK and the corresponding Jar in the cache, and calling the task stop interface in the Elasticjob to suspend the task.

[0055] The existing technology only implements the scheduling of tasks, and the tasks need to be written into elastic jobs. The present invention solves the heavy dependency of elastic job tasks on the platform, which can greatly improve flexibility.

[0056] Please refer to the following Figures 2 to 4 . Figure 2 The present invention provides a pluggable scheduling solution architecture design diagram based on Elastic job, combined with Figure 2 The specific application process of this method is as follows:

[0057] 1. Take nacos as the service registration center, and register the two spring boot projects, job and web, to nacos so that they can call each other through rpc.

[0058] 2. Web and job share a network storage to store the task Jar, so that after the task Jar is updated through the interface in web, Job can obtain the latest Jar.

[0059] 3. The changes in the task status in the job will be synchronized to the database table through the listen method and displayed on the web page.

[0060] 4. Execution of jar update interface in web:

[0061] like Figure 3 As shown, when the task Jar is uploaded or updated through the interface, it is determined whether the Jar exists. If so, the original Jar is overwritten and the parameter key is updated; if not, the Jar is stored, the parameters are read and inserted into the database.

[0062] 5. Get the key of the job's execution parameter in 4, and fill in the corresponding value of the task in the web page. Enable the task in the web page, and the parameters will be sent to the job through RPC.

[0063] like Figure 4 As shown, the task startup process includes: judging whether the Jar is in the cache, if so, reading the Jar in the cache, and initializing the Elastic job task, and the Elastic job starts to execute the task according to the configuration; otherwise, first read the path where the Jar is located, load it into the cache, and then read the Jar in the cache, and initialize the Elastic job task, and the Elastic job starts to execute the task according to the configuration;

[0064] The task suspension process includes: 1. Deleting the node in ZK; 2. Deleting the corresponding jar in the cache; 3. Calling the task stop interface in elasticjob.

[0065] When the RPC interface calls the job interface, it starts to dynamically load the Jars of different tasks to implement pluggable task start and stop. Different tasks are bound to different classloaders to implement the isolation mechanism of spring beans of different tasks to isolate different tasks.

[0066] In summary, the present invention deploys job and web platforms, can build the source code of the task by uploading Jar on the web, and has different task start and stop switches on the web, through which the addition, deletion and modification of tasks can be dynamically controlled. There is no need to redeploy the entire job platform.

[0067] Embodiment 2:

[0068] In combination with the pluggable scheduling method based on Elastic job disclosed in Example 1, this embodiment discloses a specific implementation example of a pluggable scheduling system based on Elastic job (hereinafter referred to as the "system").

[0069] The existing Elastic Job can solve the problem of task scheduling, but it cannot achieve flexible task start and stop, and flexible task deletion. This system has made two extensions to Elastic Job. 1. Add a web page to provide a task start and stop interface and status change display; 2. Provide a task Jar upload interface to flexibly add, delete, and modify tasks.

[0070] Reference Figure 5 As shown, the system comprises:

[0071] Registration unit 1: using nacos as the service registration center, registering jobs and webs to the nacos;

[0072] Task updating unit 2: upload or update the task Jar through the web interface, and read or update the key of the execution parameter corresponding to the task through the database;

[0073] Specifically, the task updating unit 2 includes: determining whether the task Jar exists, if so, overwriting the original Jar and updating the key of the execution parameter; if not, storing the task Jar, reading the key of the execution parameter and inserting it into the database.

[0074] Parameter acquisition unit 3: obtains the key of the execution parameter of the job, and fills in the value corresponding to the task in the web page;

[0075] Task execution unit 4: calls the job interface through the web using the RPC interface, dynamically loads different task Jars, and implements pluggable task start and stop through Elastic job.

[0076] Specifically, the task execution unit 4 includes:

[0077] Task startup module 41: determines whether the Jar is in the cache. If so, reads the Jar and initializes the Elastic job task, and the Elastic job starts to execute the task according to the configuration; if not, first reads the path where the Jar is located, loads the Jar into the cache, and then reads it;

[0078] Task suspension module 42: Delete the node in ZK and the corresponding Jar in the cache, and call the task stop interface in the Elasticjob to suspend the task.

[0079] For the technical solutions of the pluggable scheduling system based on Elastic job disclosed in this embodiment and the pluggable scheduling method based on Elastic job disclosed in Embodiment 1, please refer to Embodiment 1 and will not be repeated here.

[0080] Embodiment three:

[0081] Combination Figure 6 As shown, this embodiment discloses a specific implementation of a computer device. The computer device may include a processor 81 and a memory 82 storing computer program instructions.

[0082] Specifically, the processor 81 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0083] Among them, the memory 82 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 82 may be inside or outside the data processing device. In a specific embodiment, the memory 82 is a non-volatile memory. In a specific embodiment, the memory 82 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable ProgrammableRead-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable ProgrammableRead-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0084] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .

[0085] The processor 81 implements any one of the pluggable scheduling methods based on Elastic job in the above embodiments by reading and executing computer program instructions stored in the memory 82.

[0086] In some of these embodiments, the computer device may further include a communication interface 83 and a bus 80. Figure 6 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.

[0087] The communication interface 83 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication port 83 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0088] The bus 80 includes hardware, software or both, and couples the components of the computer device to each other. The bus 80 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0089] In addition, in combination with the pluggable scheduling method based on Elastic job in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any pluggable scheduling method based on Elastic job in the above embodiment is implemented.

[0090] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] In summary, the beneficial effect of the present invention lies in that the present invention adds a web page to the Elastic Job, provides a task start and stop interface and a status change display, and also provides a task Jar upload interface, which can flexibly add, delete and modify tasks without redeploying the entire job platform, thereby solving the heavy dependence of the Elastic Job task on the platform and greatly improving the flexibility.

[0092] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A pluggable scheduling method based on Elastic job, characterized in that: include: Registration steps: Use nacos as the service registration center and register job and web to the nacos; Task update step: upload or update the task Jar through the web interface, and read or update the key of the execution parameter corresponding to the task through the database; Parameter acquisition step: obtain the key of the execution parameter of the job, and fill in the value corresponding to the task in the web page; Task execution step: calling the job interface through the web using the RPC interface, dynamically loading different task Jars, and realizing pluggable task start and stop through Elastic job; The task update step includes: determining whether the task Jar exists, if so, overwriting the original Jar and updating the key of the execution parameter; if not, storing the task Jar, reading the key of the execution parameter and inserting it into the database; The task execution steps include: Task start step: determine whether the Jar is in the cache. If so, read the Jar and initialize the Elasticjob task. The Elasticjob starts to execute the task according to the configuration. If not, first read the path where the Jar is located, load the Jar into the cache, and then read it. Task suspension step: deleting the node in ZK and the corresponding Jar in the cache, and calling the task stop interface in the Elastic job to suspend the task.

2. The pluggable scheduling method according to claim 1, characterized in that: The web and the job share a network storage, and the network storage is used to store the Jar of the task.

3. The pluggable scheduling method according to claim 2, characterized in that: The changes in the status of the task in the job are synchronized to the database table by means of listen and displayed on the web page.

4. A pluggable scheduling system based on Elastic job, characterized in that: include: Registration unit: using nacos as the service registration center, registering jobs and webs to the nacos; Task update unit: upload or update the task Jar through the web interface, and read or update the key of the execution parameter corresponding to the task through the database; Parameter acquisition unit: obtains the key of the execution parameter of the job, and fills in the value corresponding to the task in the web page; Task execution unit: calls the job interface through the web using the RPC interface, dynamically loads different task Jars, and implements pluggable task start and stop through Elastic job; The task updating unit includes: determining whether the task Jar exists, if so, overwriting the original Jar and updating the key of the execution parameter; if not, storing the task Jar, reading the key of the execution parameter and inserting it into the database; Wherein, the task execution unit includes: Task startup module: determines whether the Jar is in the cache. If so, reads the Jar and initializes the Elasticjob task. The Elasticjob starts to execute the task according to the configuration. If not, first reads the path where the Jar is located, loads the Jar into the cache, and then reads it. Task suspension module: deletes the node in ZK and the corresponding Jar in the cache, and calls the task stop interface in the Elastic job to suspend the task.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the pluggable scheduling method according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the pluggable scheduling method according to any one of claims 1 to 3 is implemented.

Images