Computing task processing method, device and system
By combining task configuration modules and processing classes, the system enables visualized configuration and multi-engine adaptation for big data computing tasks, solving the problems of high cost and complexity in existing technologies and improving development and operation efficiency.
Patent Information
- Application Number
- CN202410865467.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-28
- Publication Date
- 2024-11-08
AI Technical Summary
Existing big data computing tasks are costly to develop and maintain, and their logic is hidden in the code, making it difficult to explore the logic later and redeploy when changes are made, which can easily lead to errors.
A computing task processing method is provided, which provides a visual interface for editing task configuration information through a task configuration module, and uses the development toolkit of processing class and computing engine to convert it into a driver program, constructs a directed acyclic dependency relationship, and supports the adaptation of multiple computing engines.
It lowers the development threshold, reduces operation and maintenance costs, facilitates task logic exploration and changes, and avoids redeployment errors.
Smart Images

Figure CN118916087A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a computing task processing method, device, and system. Background Art
[0002] Big data development is aimed at large-volume, multi-source, and multi-type data. It combines data from various relevant systems inside and outside the enterprise and big data marts to perform data processing and calculations, and plays an important role in the digital transformation of enterprises.
[0003] Existing big data computing tasks are mainly based on big data computing engines (hive, spark, flink, etc.). For simple tasks or tasks with only data marts as data sources, computing scripts are developed using SQL. For complex tasks or heterogeneous data sources, only the development toolkit of the corresponding computing engine can be used to develop corresponding computing tasks in a targeted manner. This places high demands on data developers and has high development and operation costs. The task logic is hidden in the code, which is not convenient for subsequent logic exploration and task operation and maintenance. In addition, when the program logic is changed, it needs to be repackaged and deployed, which has high operation and maintenance costs and is prone to errors. Summary of the invention
[0004] The embodiments of the present disclosure provide a computing task processing method and device.
[0005] In a first aspect, an embodiment of the present disclosure provides a computing task processing method, comprising: in response to detecting the start of a computing task, obtaining task configuration information; parsing a task configuration object and a processing class from the task configuration information; converting the task configuration object into a driver program of the computing engine through a development tool kit of the computing engine and the processing class, so that the computing engine can call the driver program to perform the computing task.
[0006] In some embodiments, the task configuration information includes: basic information of the task and node information; the basic information of the task includes at least one of the following: task name, task description, task parameters, task type; the node information includes at least one of the following: basic information of the node, parameter information and processing class.
[0007] In some embodiments, converting the task configuration object into a driver program of the computing engine through the computing engine's development toolkit and the processing class includes: parsing parameter information corresponding to the task configuration object from the task configuration information through the processing class; calling the computing engine's development toolkit according to the parameter information to generate a driver program for the computing engine.
[0008] In some embodiments, the method further includes: outputting the calculation result of the calculation task to a database of a corresponding type according to the type of the output node in the task configuration information.
[0009] In a second aspect, an embodiment of the present disclosure provides a computing task processing system, comprising: a task configuration module, configured to provide a visual interface for users to edit task configuration information; and a task execution engine, configured to execute any of the methods described in the first aspect.
[0010] In some embodiments, the system further includes: a computing engine configured to construct a directed acyclic dependency relationship according to a reference relationship between nodes in the task configuration information, and to call the driver program to execute the computing task based on the dependency relationship.
[0011] In some embodiments, the task configuration module is further configured to: output a common node plug-in for user selection; and output a node editing option for the user to customize the node plug-in.
[0012] In a third aspect, an embodiment of the present disclosure provides a computing task processing device, comprising: an acquisition unit, configured to acquire task configuration information in response to detecting the start of a computing task; a parsing unit, configured to parse a task configuration object and a processing class from the task configuration information; and a conversion unit, configured to convert the task configuration object into a driver program of the computing engine through a development toolkit of the computing engine and the processing class, so that the computing engine can call the driver program to perform the computing task.
[0013] In some embodiments, the task configuration information includes: basic information of the task and node information; the basic information of the task includes at least one of the following: task name, task description, task parameters, task type; the node information includes at least one of the following: basic information of the node, parameter information and processing class.
[0014] In some embodiments, the conversion unit is further configured to: parse parameter information corresponding to the task configuration object from the task configuration information through the processing class; call a development toolkit of the computing engine according to the parameter information to generate a driver program of the computing engine.
[0015] In some embodiments, the apparatus further includes an output unit configured to: output the calculation result of the calculation task to a database of a corresponding type according to the type of the output node in the task configuration information.
[0016] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement a method as described in any one of the first aspects.
[0017] In a fifth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a method as described in any one of the first aspects.
[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0020] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;
[0021] Figure 2 is a flow chart of an embodiment of a computing task processing method according to the present disclosure;
[0022] Figure 3a-3g is a schematic diagram of an application scenario of the computing task processing method according to the present disclosure;
[0023] Figure 4 is a structural schematic diagram of an embodiment of a computing task processing device according to the present disclosure;
[0024] Figure 5 It is a structural diagram of a computer system of an electronic device suitable for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION
[0025] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0026] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0027] Figure 1An exemplary system architecture is shown to which an embodiment of the computing task processing method or computing task processing apparatus of the present disclosure can be applied.
[0028] In this embodiment, the computing task processing system includes: a task configuration module configured to provide a visual interface for users to edit task configuration information; and a task execution engine configured to execute the method described in process 200. The computing task processing system may also include: a computing engine and external components (including a scheduling platform and a file server) for starting tasks.
[0029] In some optional implementations of this embodiment, the computing engine constructs a directed acyclic dependency relationship according to the reference relationship between nodes in the task configuration information, and calls the driver to execute the computing task based on the dependency relationship.
[0030] The solution mainly involves two parts: task configuration module and task execution engine. The task configuration module is responsible for visual configuration of tasks, task configuration persistence, task management, processor management, monitoring alarm, engine management, operation records, data source configuration and other functions. The task execution engine is responsible for reading and parsing task configurations, and converting tasks and configurations of various processors into corresponding computing engine tasks, and finally handing them over to the big data cluster for processing.
[0031] The task configuration module provides visual task configuration capabilities. By configuring the basic information of the task, selecting the connection parameters of each input node, the association, transformation, and mapping relationship of the data processing nodes, and the connection information of the output node, the task configuration is completed and the configuration information is persisted to the database or file. Various processing nodes are plug-in-based. The required parameters and processing classes of the processing nodes can be configured under the processor management function. General and business-common processing nodes can be encapsulated here for data developers to use. You can also select the computing engine version and the properties of batch or stream processing. Data developers configure data computing tasks through the visual task configuration module, and persist the task configuration to the database or JSON file after saving the task.
[0032] In some optional implementations of this embodiment, the task configuration module is further configured to: output a common node plug-in for the user to select; output node editing options for the user to customize the node plug-in. The node can be configured as a plug-in, specifying the basic information, parameter information, and processing class of the node. Figure 3f As shown, you can select common node plug-ins through the visual interface, or you can customize components.
[0033] The plug-in configuration method is used to develop big data tasks, and nodes can be expanded according to needs. Dependencies are automatically constructed according to the reference relationship of the node code, and there is no need to manually construct a directed acyclic graph when developing tasks. Task configuration is decoupled from the computing engine, and the task execution engine can adapt to a variety of big data computing engines according to the scenario.
[0034] The main function of the task execution engine is to pull JSON files through remote method calls, parse JSON files, and deserialize them into corresponding task configuration objects after the developer manually or the scheduling platform triggers the data computing task regularly. Pull the jar package from the file server according to the version information of the configured computing engine. The execution engine translates the configuration object into the driver program of the corresponding computing engine through the computing engine development toolkit and the processing class defined on the processor, and finally submits it to the big data computing engine for execution. Currently, the supported computing engines are spark and flink, two mainstream big data computing engines. The configuration information configures the connection parameters of the data reading node (Reader-plugins) in the form of plug-ins, such as the data source library. The configuration information also configures the association, transformation, and mapping relationship of the data processing node, such as the conversion configuration information of the field mapping of the sql script and java program, and the dynamic script processing information. The configuration information configures the connection information of the data output node (Writer-Plugins) in the form of plug-ins, such as the data target library.
[0035] The task execution engine can also monitor the computing task process and report the execution results to the task configuration module. The task configuration module saves the task operation record and issues a monitoring alarm when receiving a task failure message.
[0036] It should be noted that the computing task processing method provided in the embodiments of the present disclosure is generally executed by a task execution engine, and accordingly, the computing task processing device is generally arranged in the task execution engine.
[0037] Continue to refer Figure 2 , shows a process 200 of an embodiment of a computing task processing method according to the present disclosure. The computing task processing method comprises the following steps:
[0038] Step 201, in response to detecting the start of a computing task, obtaining task configuration information.
[0039] In this embodiment, the execution subject of the computing task processing method (for example Figure 1The task execution engine shown in the figure can receive a request to start a computing task from the scheduling platform through a wired connection or a wireless connection, and then obtain the task configuration information from the task configuration module through a remote procedure (RPC). The task configuration information may include: basic information and node information of the task; the basic information of the task includes at least one of the following: task name, task description, task parameters, and task type; the node information includes at least one of the following: basic information, parameter information, and processing class of the node.
[0040] A node refers to a processor (software function module) when a computing engine executes computing tasks, also known as a processing node. The task execution engine has built-in support for a variety of common input nodes, processing nodes, output nodes, algorithm nodes, script nodes, etc., and supports the expansion of custom nodes. It can achieve end-to-end data processing and transmission, big data model processing, algorithm processing, and streaming data processing capabilities.
[0041] like Figure 3e Several examples of input nodes are shown. Developers can select common nodes or customize nodes through the visual interface of the task configuration module. After selecting a node, the node icon will appear in Figure 3b The node list box on the left.
[0042] Figure 3a This is a task configuration diagram for a spam SMS identification task. The basic information mainly includes the task name, task description, task parameters, task type, etc. The task type can specify the computing engine and the properties of batch processing or stream processing. In this example, spark batch processing and version number are selected, and the task execution engine can download the corresponding version of the spark engine software package from the file cloud server.
[0043] After filling in the basic information of the task, you can fill in the information of each node involved in the task in the Task Configuration tab. The required nodes have been selected in advance and are displayed in Figure 3b In the node list box on the left, each time you select a node, you can fill in the node information, including at least one of the following: basic node information, parameter information, and processing type.
[0044] The task configuration adopts a linear configuration mode. The processing nodes have forward and backward dependencies. Each processing node has a unique node code as a representation. During the processing process, it will be registered into a corresponding temporary table according to the node code. Subsequent processing nodes can reference the corresponding node data according to the node code. During calculation, the node dependency is based on the reference relationship between nodes, and the computing engine automatically constructs a directed acyclic dependency.
[0045] like Figure 3bThe limit node shown reads the SMS data to be identified from the data mart and is coded as limit. You can write SQL scripts in the SQL logic option box.
[0046] like Figure 3c As shown in the figure, the developer selected the word segmentation node and configured the node to process the data set, the specific column name of the SMS content, and other adjustment parameters. The same applies to other nodes. By filling in the node code limit in the word segmentation node, the limit node is referenced, that is, the word segmentation node depends on the limit node.
[0047] The configuration information entered through the visual interface of the task configuration module can generate a JSON file, which is convenient for the task execution engine to parse the task configuration object. It can also generate files in other formats, such as XML.
[0048] Step 202: parse the task configuration object and processing class from the task configuration information.
[0049] In this embodiment, the task configuration object and the processing class are parsed by deserializing the JSON file. The processing class is the interface and implementation of the node, and can include a general processing class and a custom processing class.
[0050] Step 203: convert the task configuration object into a driver program of the computing engine through the development toolkit and processing class of the computing engine, so that the computing engine can call the driver program to execute the computing task.
[0051] In this embodiment, the parameter information corresponding to the task configuration object is parsed from the task configuration information through the processing class; the development toolkit of the computing engine is called according to the parameter information to generate a driver program of the computing engine. The function of the processing class itself includes parsing the parameter information of the configuration object and translating it into a target driver program. Some specific functions are also defined in the processing class. These functions comply with the specifications of the computing engine toolkit and define the specific logic of the corresponding node data processing. These functions will be called by the computing engine when the computing engine executes to complete the data processing. In this way, developers do not need to learn the classes in the development toolkit of each computing engine, and can perform computing task processing by only using the processing class in the task configuration module.
[0052] The method provided by the above-mentioned embodiment of the present disclosure is that after the computing task is started, the program obtains the task configuration through the RPC service and deserializes it into a configuration object. The execution engine has interfaces and implementations corresponding to various processing nodes, including preset general processors, custom functions, etc., and custom processors and custom functions can also be developed based on interface extensions. The execution engine parses the configuration object and finds the corresponding processor in the engine for processing through the processing class configuration. The content of the processor is to obtain the processor configuration parameters, call the computing engine toolkit, and convert it into the process of corresponding computing engine processing. For the interaction between processing nodes, please refer to Figure 3g The technical principle diagram shown in the figure. Mysql and Oracle on the left side of the figure are input nodes for reading data, and collector and processor are data processing nodes for collecting data. Hivetable on the right side of the figure is also a data processing node, which can read and process data. Writers are output nodes that can output calculated data to the data target library set in the configuration information, such as Mysql, network disk, or anywhere else.
[0053] It can lower the development threshold. Data developers do not need to master the program development capabilities of multiple computing engines such as hive, spark, and flink, and the development cost is relatively low. The task logic is displayed in a visual interface, which is convenient for subsequent logic exploration and task operation and maintenance. When changing the program logic, there is no need to repackage and deploy, and the operation and maintenance cost is low and error-prone. This solution can support the configuration development of multiple big data tasks such as big data transmission tasks, big data model processing tasks, algorithm tasks, and real-time big data tasks.
[0054] In some optional implementations of this embodiment, the calculation result of the calculation task is output to a database of the corresponding type according to the type of the output node in the task configuration information. For example, the final result after processing and calculation can be output to a data mart (non-relational database) through a sql script node, or output to a relational database through a jdbc node. The node encoding process will be registered as a temporary table, so it can be used directly in the sql script. Figure 3d As shown, the data is output to the data mart through the SQL script node.
[0055] Further references Figure 4 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a computing task processing device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0056] like Figure 4As shown, the computing task processing device 400 of this embodiment includes: an acquisition unit 401, a parsing unit 402 and a conversion unit 403. The acquisition unit 401 is configured to acquire task configuration information in response to detecting the start of a computing task; the parsing unit 402 is configured to parse the task configuration information to obtain a task configuration object and a processing class; the conversion unit 403 is configured to convert the task configuration object into a driver of the computing engine through the computing engine development toolkit and the processing class, so that the computing engine calls the driver to perform the computing task.
[0057] In this embodiment, the specific processing of the acquisition unit 401, the parsing unit 402 and the conversion unit 403 of the computing task processing device 400 can refer to Figure 2 Corresponding to step 201, step 202, and step 203 in the embodiment.
[0058] In some optional implementations of this embodiment, the task configuration information includes: basic information of the task and node information; the basic information of the task includes at least one of the following: task name, task description, task parameters, task type; the node information includes at least one of the following: basic information of the node, parameter information and processing class.
[0059] In some optional implementations of this embodiment, the conversion unit 303 is further configured to: parse parameter information corresponding to the task configuration object from the task configuration information through the processing class; and call the development toolkit of the computing engine to generate a driver program for the computing engine according to the parameter information.
[0060] In some optional implementations of this embodiment, the device also includes an output unit (not shown in the drawings), which is configured to: output the calculation result of the computing task to a database of the corresponding type according to the type of the output node in the task configuration information.
[0061] In some optional implementations of this embodiment, the task configuration information is stored in JSON format.
[0062] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.
[0063] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.
[0064] An electronic device comprises: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method described in process 200.
[0065] A computer-readable medium stores a computer program, wherein the computer program implements the method described in process 200 when executed by a processor.
[0066] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0067] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0068] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0069] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the road area planning method. For example, in some embodiments, the road area planning method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the road area planning method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the road area planning method in any other appropriate manner (e.g., by means of firmware).
[0070] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0071] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0072] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0073] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0074] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0075] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a server of a distributed system, or a server combined with a blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server may be a server of a distributed system, or a server combined with a blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.
[0076] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0077] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A computing task processing method, comprising: In response to detecting that a computing task is started, obtaining task configuration information; Parsing the task configuration object and the processing class from the task configuration information; The task configuration object is converted into a driver program of the computing engine through the development toolkit of the computing engine and the processing class, so that the computing engine can call the driver program to execute the computing task.
2. The method according to claim 1, wherein: The task configuration information includes: basic information and node information of the task; The basic information of the task includes at least one of the following: task name, task description, task parameters, and task type; The node information includes at least one of the following: basic information, parameter information and processing type of the node.
3. The method according to claim 1, wherein: The step of converting the task configuration object into a driver program of the computing engine through the computing engine development toolkit and the processing class includes: Parsing parameter information corresponding to the task configuration object from the task configuration information through the processing class; The development toolkit of the computing engine is called according to the parameter information to generate a driver program of the computing engine.
4. The method according to claim 1, wherein: The method further comprises: The calculation result of the calculation task is output to a database of a corresponding type according to the type of the output node in the task configuration information.
5. A computing task processing system, comprising: The task configuration module is configured to provide a visual interface for users to edit task configuration information; A task execution engine, configured to execute the method according to any one of claims 1 to 4.
6. The system according to claim 5, wherein: The system further comprises: The computing engine is configured to construct a directed acyclic dependency relationship according to the reference relationship between nodes in the task configuration information, and to call the driver program to execute the computing task based on the dependency relationship.
7. The system according to claim 5, wherein the task configuration module is further configured to: Output common node plug-ins for users to choose; Output node editing options for users to customize the editing node plug-in.
8. A computing task processing device, comprising: an acquisition unit, configured to acquire task configuration information in response to detecting that a computing task is started; A parsing unit, configured to parse the task configuration object and the processing class from the task configuration information; The conversion unit is configured to convert the task configuration object into a driver program of the computing engine through the development toolkit of the computing engine and the processing class, so that the computing engine calls the driver program to execute the computing task.
9. An electronic device, comprising: one or more processors; a storage device having one or more computer programs stored thereon, When the one or more computer programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.
10. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.
