Method, device and electronic device for implementing distributed processing engine tasks

Abstract

The present invention discloses a method, device and electronic device for implementing distributed processing engine tasks. The method comprises: obtaining a first engine task and a general data packet; generating a target data packet corresponding to the first engine task according to the general data packet; configuring target parameters for the target data packet through a visual interface; generating a target startup script according to the target parameters; when the start / stop button on the visual interface is clicked, starting the target startup script and running the first engine task. The present invention solves the technical problems that engine tasks cannot be developed with unified rules, cannot control the start / stop of tasks with one key, and cannot dynamically provide parameter configuration.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a method, device and electronic device for implementing distributed processing engine tasks. Background Art

[0002] Apache Flink is a framework and distributed processing engine for stateful computation on unbounded and bounded data streams. Flink can already run in all common cluster environments and perform computations at in-memory (storage engine) speeds and at any scale. However, in actual use, there are still some engineering structural problems. For example, Flink tasks deployed with Yarn (resource manager) can only be started and stopped by the Yarn API (interface), and there is no good visual interface to facilitate maintenance by operation and maintenance personnel or non-technical personnel; different Flink tasks require initialization of Flink's operating environment, resulting in duplication of different business codes, and different businesses need to maintain some common rules, and parameter configuration cannot be provided dynamically. Summary of the invention

[0003] The embodiments of the present invention provide a method, device and electronic device for implementing distributed processing engine tasks, so as to at least solve the technical problems that engine tasks cannot be developed with unified rules, cannot control the start and stop of tasks with one click, and cannot dynamically provide parameter configuration.

[0004] According to one aspect of an embodiment of the present invention, a method for implementing a distributed processing engine task is provided, including: obtaining a first engine task and a general data packet; generating a target data packet corresponding to the first engine task according to the general data packet; configuring target parameters for the target data packet through a visual interface; generating a target startup script according to the target parameters; and when the start / stop button on the visual interface is clicked, starting the target startup script and running the first engine task.

[0005] According to another aspect of an embodiment of the present invention, a device for implementing a distributed processing engine task is provided, including: a first acquisition module, used to acquire a first engine task and a general data packet; a first generation module, used to generate a target data packet corresponding to the first engine task according to the general data packet; a configuration module, used to configure target parameters for the target data packet through a visual interface; a second generation module, used to generate a target startup script according to the target parameters; and a startup module, used to start the target startup script and run the first engine task when the start / stop button on the visual interface is clicked.

[0006] As an optional example, the first acquisition module includes: a first acquisition unit, used to acquire a template script; and a first generation unit, used to generate the general data packet according to the template script.

[0007] As an optional example, the above-mentioned first generation module includes: a second acquisition unit, used to obtain the rule data packet and parameter data packet of the above-mentioned first engine task; a second generation unit, used to generate the above-mentioned target data packet according to the above-mentioned general data packet, the above-mentioned rule data packet and the above-mentioned parameter data packet.

[0008] As an optional example, the above-mentioned startup module includes: a calling unit, used to call the heartbeat interface corresponding to the above-mentioned first engine task; a third acquisition unit, used to obtain the task status of the above-mentioned first engine task through the above-mentioned heartbeat interface; and an update unit, used to update the above-mentioned task status to the above-mentioned visualization interface.

[0009] As an optional example, the above-mentioned device also includes: a second acquisition module, used to obtain the second engine task; a third acquisition module, used to obtain the first data packet and the original data packet of the above-mentioned second engine task when the above-mentioned second engine task exists in the database; a replacement module, used to replace the above-mentioned original data packet with the above-mentioned first data packet.

[0010] As an optional example, the above-mentioned device also includes: a fourth acquisition module, used to obtain the above-mentioned first data packet of the above-mentioned second engine task when the above-mentioned second engine task does not exist in the above-mentioned database; a saving module, used to save the above-mentioned second engine task and the above-mentioned first data packet to the above-mentioned database.

[0011] As an optional example, the above-mentioned device also includes: a processing module, which is used to suspend the operation of the above-mentioned first engine task when the start / stop button on the above-mentioned visual interface is clicked.

[0012] According to another aspect of an embodiment of the present invention, a storage medium is provided, in which a computer program is stored, wherein the computer program executes the implementation method of the above-mentioned distributed processing engine task when executed by a processor.

[0013] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the implementation method of the distributed processing engine task through the computer program.

[0014] The implementation method of the above-mentioned distributed processing engine task in the present application can be used in the process of data cleaning of data capabilities. In an embodiment of the present invention, the first engine task and the general data packet are obtained; the target data packet corresponding to the above-mentioned first engine task is generated according to the above-mentioned general data packet; the target parameters are configured for the above-mentioned target data packet through a visual interface; a target startup script is generated according to the above-mentioned target parameters; when the start / stop button on the above-mentioned visual interface is clicked, the above-mentioned target startup script is started, and the method of running the above-mentioned first engine task is implemented. Since in the above-mentioned method, the target data packet is generated through a general data packet, the target parameters are configured through a visual interface, and a target startup script is generated, the purpose of unified rule development, one-click start / stop of visual interface tasks, and dynamic configuration of parameters is achieved, thereby solving the technical problems that engine tasks cannot be developed in a unified rule, cannot control the start / stop of tasks with one click, and cannot dynamically provide parameter configuration. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0016] Figure 1 is a flowchart of an optional method for implementing a distributed processing engine task according to an embodiment of the present invention;

[0017] Figure 2 is an updated flow chart of an optional method for implementing a distributed processing engine task according to an embodiment of the present invention;

[0018] Figure 3 is a structural diagram of an optional device for implementing distributed processing engine tasks according to an embodiment of the present invention;

[0019] Figure 4 is a schematic diagram of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] According to a first aspect of an embodiment of the present invention, a method for implementing a distributed processing engine task is provided. Optionally, as follows: Figure 1 As shown, the above method includes:

[0023] S102, obtaining a first engine task and a general data packet;

[0024] S104, generating a target data packet corresponding to the first engine task according to the general data packet;

[0025] S106, configuring target parameters for the target data packet through a visual interface;

[0026] S108, generating a target startup script according to the target parameters;

[0027] S110, when the start / stop button on the visualization interface is clicked, the target startup script is started to run the first engine task.

[0028] Optionally, in this embodiment, the engine task is a Flink task, which is a framework and distributed processing engine for state calculation on unbounded and bounded data streams. The general data packet is a Jar file with its own logic and rules. Visual user interface is referred to as GUI, which also refers to graphical user interface. It is an interface display format for communication between people and computers, allowing users to use input devices such as mice to manipulate icons or menu options on the screen to select commands, call files, start programs or perform other daily tasks. The start and stop buttons corresponding to multiple engine tasks can be displayed on the visual interface, which are used for one-click start and stop, and are also used to display the task status of the corresponding engine task. Startup script, script, is an executable file written in a certain format using a specific descriptive language, also known as a macro or batch file. Scripts can usually be temporarily called and executed by an application, and the startup script is the executable file to run the software.

[0029] Optionally, in this embodiment, the first engine task and the general data packet are obtained, and the business developer develops corresponding logic and rules based on the first engine task and the general data packet, packages and generates a corresponding target data packet, uploads the target data packet to the visualization interface, generates a start / stop button corresponding to the first engine task through the visualization interface, configures target parameters for the target data packet, generates a target startup script corresponding to the first engine task according to the target parameters, and when the user clicks the start / stop button on the visualization interface, the corresponding first engine task starts the startup script and starts running the first engine task.

[0030] Optionally, in this embodiment, a target data packet is generated through a general data packet, target parameters are configured through a visual interface, and a target startup script is generated, thereby achieving the purpose of unified rule development, one-click start and stop of visual interface tasks, and dynamic configuration of parameters, thereby solving the technical problems that engine tasks cannot be developed with unified rules, cannot control the start and stop of tasks with one click, and cannot dynamically provide parameter configuration.

[0031] As an optional example, obtaining the first engine task and the general data packet includes:

[0032] Get the template script;

[0033] Generates a generic data package based on a template script.

[0034] Optionally, in this embodiment, the template script includes the key to the resources (memory, CPU) required by the first engine task, configured by the visual interface, the rule operator address, the parameter interface address of the business operator, and the heartbeat address. When the engine task is executed for the first time, a general data packet is generated according to the template script, and the engine tasks executed later directly use this general data packet.

[0035] As an optional example, generating a target data packet corresponding to the first engine task according to the general data packet includes:

[0036] Obtaining a rule data packet and a parameter data packet of a first engine task;

[0037] Generate a target data packet according to the common data packet, the rule data packet and the parameter data packet.

[0038] Optionally, in this embodiment, a rule data packet and a parameter data packet of the first engine task are obtained, and the rule data packet, the parameter data packet and the general data packet are packaged into a target data packet.

[0039] As an optional example, start the target startup script and run the engine task including:

[0040] Call the heartbeat interface corresponding to the first engine task;

[0041] Obtain the task status of the first engine task through the heartbeat interface;

[0042] Update the task status to the visualization interface.

[0043] Optionally, in this embodiment, after the startup script is started, the corresponding heartbeat interface is called through the visual interface, and the task status of the first engine task is obtained through the heartbeat interface. The task status can be running, running, etc., and the task status is updated to the visual interface in real time, so that the user can see the running status of the first engine task at a glance.

[0044] As an optional example, the method further includes:

[0045] Get the second engine task;

[0046] If the second engine task exists in the database, obtaining the first data packet and the original data packet of the second engine task;

[0047] The first data packet replaces the original data packet.

[0048] Optionally, in this embodiment, the visualization interface can run multiple engine tasks simultaneously. When a second engine task is received, the second engine task is obtained, and the database is searched for the existence of the second engine task. If it exists, the first data packet and the original data packet of the second engine task are obtained. The original data packet is the data packet of the second engine task in the database. The first data packet replaces the original data packet to update the data of the second engine task in the database.

[0049] As an optional example, the method further includes:

[0050] If the second engine task does not exist in the database, obtaining a first data packet of the second engine task;

[0051] The second engine task and the first data packet are saved to a database.

[0052] Optionally, in this embodiment, when the second engine task does not exist in the database, the first data packet of the second engine task is obtained and saved in the database to update the data in the database.

[0053] As an optional example, the method further includes:

[0054] When the start / stop button on the visualization interface is clicked, the first engine task is paused.

[0055] Optionally, in this embodiment, when the first engine task is in the running state, when the user clicks the corresponding start / stop button on the visualization interface, the first engine task is suspended; when the first engine task is not in the running state, when the user clicks the corresponding start / stop button on the visualization interface, the first engine task starts to run; when the first engine task is in the paused state, when the user clicks the start / stop button on the visualization interface, the first engine task continues to run.

[0056] Optionally, in conjunction with an example for explanation, the present application relates to a method for implementing a distributed processing engine task, which generates a target data packet through a general data packet, configures target parameters through a visual interface, and generates a target startup script, thereby achieving unified rule development, one-click start and stop of visual interface tasks, and dynamic configuration of parameters. The overall implementation process is as follows:

[0057] 1. The business developer develops his own business logic and rules for the first engine task based on the interface exposed by the general data package, packages it into a target data package, uploads it to the backend through the visual interface, configures the parameters in the visual interface, and clicks to start in the visual interface;

[0058] 2. The backend generates the startup script of the operator based on the parameters in the first step and submits the task to the resource manager. The resource manager runs the task, calls the heartbeat interface, and updates the task status on the visual interface;

[0059] 3. The web page is used as the visualization interface, which mainly provides the following functions: visualization of the start and stop of the first engine task, uploading of the first engine task target data packet, the first engine task parameter data packet, and the rule data packet;

[0060] 4. When the engine task is executed for the first time, a general data packet corresponding to the operator task will be generated according to the template script. The general data packet generated before will be reused in the future. The general data packet provides the data of the rule data packet and the engine task running environment that the operator depends on. There is a parameter template, which contains the key to the resources (memory, CPU) required by the engine task, configured by the visual interface, the address of the rule operator, the parameter interface address of the business operator, and the heartbeat address;

[0061] 5. Click the start / stop button corresponding to the first engine task in the visualization interface;

[0062] 6. The general data packet calls the interface of the business operator to obtain the target data packet of the first engine task;

[0063] 7. The application master will broadcast the target data packet;

[0064] 8. Kafka source receives the target data packet and initializes it;

[0065] 9. Output the target data packet to different business data according to the parameters of the first engine task.

[0066] The visualization interface can run multiple engine tasks at the same time. When the second engine task is received, the second engine task is obtained, and the existence of the second engine task is searched in the database. If it exists, the first data packet and the original data packet of the second engine task are obtained. The original data packet is the data packet of the second engine task in the database, and the first data packet replaces the original data packet to update the data of the second engine task in the database. If it does not exist, the first data packet is saved in the database. Figure 2 shown.

[0067] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0068] According to another aspect of the embodiment of the present application, a device for implementing distributed processing engine tasks is also provided, such as Figure 3 As shown, including:

[0069] A first acquisition module 302, used to acquire a first engine task and a general data packet;

[0070] A first generating module 304, configured to generate a target data packet corresponding to the first engine task according to the general data packet;

[0071] Configuration module 306, configured to configure target parameters for the target data packet through a visual interface;

[0072] The second generating module 308 is used to generate a target startup script according to the target parameters;

[0073] The start module 310 is used to start the target start script and run the first engine task when the start / stop button on the visual interface is clicked.

[0074] Optionally, in this embodiment, the engine task is a Flink task, which is a framework and distributed processing engine for state calculation on unbounded and bounded data streams. The general data packet is a Jar file with its own logic and rules. Visual user interface, referred to as GUI, also refers to graphical user interface, which is an interface display format for communication between people and computers, allowing users to use input devices such as mice to manipulate icons or menu options on the screen to select commands, call files, start programs or perform other daily tasks. The start and stop buttons corresponding to multiple engine tasks can be displayed on the visual interface, which are used for one-click start and stop, and are also used to display the task status of the corresponding engine task. Startup script, script, is an executable file written in a certain format using a specific descriptive language, also known as a macro or batch file. Scripts can usually be temporarily called and executed by applications, and startup scripts are executable files to run software.

[0075] Optionally, in this embodiment, the first engine task and the general data packet are obtained, and the business developer develops corresponding logic and rules based on the first engine task and the general data packet, packages and generates a corresponding target data packet, uploads the target data packet to the visualization interface, generates a start / stop button corresponding to the first engine task through the visualization interface, configures target parameters for the target data packet, generates a target startup script corresponding to the first engine task according to the target parameters, and when the user clicks the start / stop button on the visualization interface, the corresponding first engine task starts the startup script and starts running the first engine task.

[0076] Optionally, in this embodiment, a target data packet is generated through a general data packet, target parameters are configured through a visual interface, and a target startup script is generated, thereby achieving the purpose of unified rule development, one-click start and stop of visual interface tasks, and dynamic configuration of parameters, thereby solving the technical problems that engine tasks cannot be developed with unified rules, cannot control the start and stop of tasks with one click, and cannot dynamically provide parameter configuration.

[0077] As an optional example, the first acquisition module includes:

[0078] A first acquisition unit, used for acquiring a template script;

[0079] The first generating unit is used to generate a general data packet according to the template script.

[0080] Optionally, in this embodiment, the template script includes the key to the resources (memory, CPU) required by the first engine task, configured by the visual interface, the rule operator address, the parameter interface address of the business operator, and the heartbeat address. When the engine task is executed for the first time, a general data packet is generated according to the template script, and the engine tasks executed later directly use this general data packet.

[0081] As an optional example, the first generating module includes:

[0082] A second acquisition unit, used to acquire a rule data packet and a parameter data packet of the first engine task;

[0083] The second generating unit is used to generate a target data packet according to the general data packet, the rule data packet and the parameter data packet.

[0084] Optionally, in this embodiment, a rule data packet and a parameter data packet of the first engine task are obtained, and the rule data packet, the parameter data packet and the general data packet are packaged into a target data packet.

[0085] As an optional example, the startup module includes:

[0086] A calling unit, used for calling the heartbeat interface corresponding to the first engine task;

[0087] A third acquisition unit, used to acquire the task status of the first engine task through a heartbeat interface;

[0088] The updating unit is used to update the task status to the visual interface.

[0089] Optionally, in this embodiment, after the startup script is started, the corresponding heartbeat interface is called through the visual interface, and the task status of the first engine task is obtained through the heartbeat interface. The task status can be running, running, etc., and the task status is updated to the visual interface in real time, so that the user can see the running status of the first engine task at a glance.

[0090] As an optional example, the above device further includes:

[0091] A second acquisition module, used to acquire a second engine task;

[0092] A third acquisition module, used for acquiring the first data packet and the original data packet of the second engine task if the second engine task exists in the database;

[0093] The replacement module is used to replace the original data packet with the first data packet.

[0094] Optionally, in this embodiment, the visualization interface can run multiple engine tasks simultaneously. When a second engine task is received, the second engine task is obtained, and the database is searched for the existence of the second engine task. If it exists, the first data packet and the original data packet of the second engine task are obtained. The original data packet is the data packet of the second engine task in the database. The first data packet replaces the original data packet to update the data of the second engine task in the database.

[0095] As an optional example, the above device further includes:

[0096] A fourth acquisition module, configured to acquire a first data packet of the second engine task if the second engine task does not exist in the database;

[0097] The saving module is used to save the second engine task and the first data packet to the database.

[0098] Optionally, in this embodiment, when the second engine task does not exist in the database, the first data packet of the second engine task is obtained and saved in the database to update the data in the database.

[0099] As an optional example, the above device further includes:

[0100] The processing module is used to suspend the running of the first engine task when the start / stop button on the visual interface is clicked.

[0101] Optionally, in this embodiment, when the first engine task is in the running state, when the user clicks the corresponding start / stop button on the visualization interface, the first engine task is suspended; when the first engine task is not in the running state, when the user clicks the corresponding start / stop button on the visualization interface, the first engine task starts to run; when the first engine task is in the paused state, when the user clicks the start / stop button on the visualization interface, the first engine task continues to run.

[0102] For other examples of this embodiment, please refer to the above examples and will not be repeated here.

[0103] Figure 4 is a structural block diagram of an optional electronic device according to an embodiment of the present application, such as Figure 4 As shown, it includes a processor 402, a communication interface 404, a memory 406 and a communication bus 408, wherein the processor 402, the communication interface 404 and the memory 406 communicate with each other through the communication bus 408, wherein,

[0104] Memory 406, used to store computer programs;

[0105] The processor 402 is used to implement the following steps when executing the computer program stored in the memory 406:

[0106] Get the first engine task and the general data package;

[0107] Generate a target data packet corresponding to the first engine task according to the general data packet;

[0108] Configure target parameters for target data packets through a visual interface;

[0109] Generate target startup script according to target parameters;

[0110] When the start / stop button on the visualization interface is clicked, the target startup script is started and the first engine task is run.

[0111] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The communication interface is used for communication between the above electronic device and other devices.

[0112] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0113] As an example, the memory 406 may include, but is not limited to, the first acquisition module 302, the first generation module 304, the configuration module 306, the second generation module 308, and the startup module 310 in the distributed processing engine task implementation device. In addition, it may also include, but is not limited to, other module units in the request processing device, which will not be repeated in this example.

[0114] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0115] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0116] It can be understood by those skilled in the art that Figure 4The structure shown is for illustration only. The device for implementing the above-mentioned distributed processing engine task implementation method may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 4 The structure of the electronic device is not limited. Figure 4 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 4 Different configurations shown.

[0117] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0118] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps in the implementation method of the above-mentioned distributed processing engine task when executed by a processor.

[0119] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0120] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0121] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0122] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0123] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0126] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for implementing distributed processing engine tasks, It is characterized in that include: Get the first engine task and the general data package; Generate a target data packet corresponding to the first engine task according to the general data packet; Configuring target parameters for the target data packet through a visual interface; Generate a target startup script according to the target parameters; When the start / stop button on the visualization interface is clicked, the target startup script is started to run the first engine task; Get the second engine task; In the case where the second engine task exists in the database, obtaining a first data packet and an original data packet of the second engine task; replacing the original data packet with the first data packet; wherein the original data packet is a data packet of the second engine task in the database, and the first data packet is a target data packet corresponding to the second engine task generated according to the general data packet; When the second engine task does not exist in the database, the first data packet of the second engine task is obtained; and the second engine task and the first data packet are saved in the database.

2. The method according to claim 1, It is characterized in that The obtaining of the first engine task and the general data packet comprises: Get the template script; The general data packet is generated according to the template script.

3. The method according to claim 1, It is characterized in that Generating a target data packet corresponding to the first engine task according to the general data packet comprises: Obtaining a rule data packet and a parameter data packet of the first engine task; The target data packet is generated according to the general data packet, the rule data packet and the parameter data packet.

4. The method according to claim 1, It is characterized in that The starting the target startup script and running the engine task includes: Calling the heartbeat interface corresponding to the first engine task; Obtaining the task status of the first engine task through the heartbeat interface; The task status is updated to the visualization interface.

5. The method according to claim 1, It is characterized in that The method further comprises: When the start / stop button on the visualization interface is clicked, the first engine task is paused.

6. A distributed processing engine task implementation device, It is characterized in that include: A first acquisition module, used to acquire a first engine task and a general data packet; A first generating module, configured to generate a target data packet corresponding to the first engine task according to the general data packet; A configuration module, used to configure target parameters for the target data packet through a visual interface; A second generating module, used to generate a target startup script according to the target parameters; A startup module, configured to start the target startup script and run the first engine task when the start / stop button on the visualization interface is clicked; Get the second engine task; When the second engine task exists in the database, obtain the first data packet and the original data packet of the second engine task; replace the original data packet with the first data packet; wherein the original data packet is the data packet of the second engine task in the database, and the first data packet is the target data packet corresponding to the second engine task generated according to the general data packet; when the second engine task does not exist in the database, obtain the first data packet of the second engine task; save the second engine task and the first data packet to the database.

7. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is executed.

8. An electronic device comprising a memory and a processor, It is characterized in that A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 5 through the computer program.

Images