Computing task acquisition method, device, storage medium and electronic device

Abstract

The embodiments of the present application provide a computing task acquisition method, device, storage medium and electronic device. The method receives a JSON definition file input by a user, where the JSON definition file is a program file that describes the computing process of the computing task through component information. The method obtains a first component corresponding to the component information in the JSON definition file from a predefined component library, encapsulates the first component, and obtains the computing task. This realizes the development of computing tasks under unified specifications and within an operating system, which not only reduces the programming workload of developers, improves the development efficiency of computing tasks, but also reduces the maintenance cost of the operating system.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method, device, storage medium and electronic device for acquiring a computing task. Background Art

[0002] Apache Spark (hereinafter referred to as Spark) is a powerful open source data processing engine and one of the largest open source communities in the field of big data. Many data products or business scenarios require cleaning and processing of massive amounts of data, and then further calculation to obtain the final result data. Therefore, developers are required to develop corresponding Spark computing tasks according to usage requirements.

[0003] In current technology, developers usually write Spark programs directly in the task editor to obtain Spark computing tasks, and upload the Spark computing tasks to the Spark computing engine, which runs the Spark computing engine and outputs the result data.

[0004] However, in the prior art, developers need to spend a lot of time and energy to write Spark programs, and different developers work independently without a unified development framework. Therefore, in the prior art, there are problems of low development efficiency and high maintenance cost when developing Spark computing tasks. Summary of the invention

[0005] The embodiments of the present application provide a computing task acquisition method, device, storage medium and electronic device, which are used to solve the problems of low development efficiency and high maintenance cost in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for acquiring a computing task, comprising:

[0007] Receive a JS object notation JSON definition file input by a user, wherein the JSON definition file includes component information;

[0008] Acquire a first component corresponding to the component information in the JSON definition file from a predefined component library;

[0009] The first component is encapsulated to obtain a computing task.

[0010] Optionally, acquiring the first component corresponding to the component information in the JSON definition file from a predefined component library includes:

[0011] Parsing the JSON definition file and obtaining a parsing result, wherein the parsing result includes a component category and a component name of the first component;

[0012] The first component is extracted from the component library according to the component category and the component name of the first component.

[0013] Optionally, when the number of the first components is at least two, the parsing result further includes a structural relationship between the at least two first components, where the structural relationship is used to represent a hierarchical relationship and / or a dependency relationship between the at least two first components;

[0014] Correspondingly, the encapsulation of the first component to obtain a computing task includes:

[0015] The first component and the structural relationship are packaged to obtain the computing task.

[0016] Optionally, packaging the first component and the structural relationship to obtain the computing task includes:

[0017] The first component and the structural relationship are packaged with a component operation toolkit to obtain the computing task.

[0018] Optionally, before obtaining the first component corresponding to the component information in the JSON definition file from the predefined component library, the method further includes:

[0019] Scan all registered components in the component library to obtain a component list; the component list includes component categories and component names of all registered components;

[0020] Accordingly, obtaining the first component corresponding to the component information in the JSON definition file from the predefined component library includes:

[0021] According to the component list, a first component corresponding to the component information in the JSON definition file is obtained from a predefined component library.

[0022] Optionally, the registered components include custom components and system default components; before scanning all registered components in the component library and obtaining a component list, the method further includes:

[0023] A custom component that receives user input;

[0024] Register the custom component.

[0025] Optionally, registering the custom component includes:

[0026] The custom component is registered through an application programming interface (API).

[0027] Optionally, the method further comprises:

[0028] Submit the computing task to the computing engine for computing.

[0029] Optionally, the method further comprises:

[0030] The calculation result of the calculation task is obtained by running the toolkit through the component.

[0031] In a second aspect, an embodiment of the present application provides a computing task acquisition device, including:

[0032] A receiving module, used for receiving a JS object notation JSON definition file input by a user, wherein the JSON definition file includes component information;

[0033] The processing module is used to obtain the first component corresponding to the component information in the JSON definition file from a pre-defined component library; and encapsulate the first component to obtain a computing task.

[0034] Optionally, the processing module is specifically used for:

[0035] Parsing the JSON definition file and obtaining a parsing result, wherein the parsing result includes a component category and a component name of the first component;

[0036] The first component is extracted from the component library according to the component category and the component name of the first component.

[0037] Optionally, when the number of the first components is at least two, the parsing result further includes a structural relationship between the at least two first components, where the structural relationship is used to represent a hierarchical relationship and / or a dependency relationship between the at least two first components;

[0038] Accordingly, the processing module is specifically used for:

[0039] The first component and the structural relationship are packaged to obtain the computing task.

[0040] Optionally, the processing module is specifically used for:

[0041] The first component and the structural relationship are packaged with a component operation toolkit to obtain the computing task.

[0042] Optionally, the processing module is further used for:

[0043] Scan all registered components in the component library to obtain a component list; the component list includes component categories and component names of all registered components;

[0044] Accordingly, the processing module is specifically used for:

[0045] According to the component list, a first component corresponding to the component information in the JSON definition file is obtained from a predefined component library.

[0046] Optionally, the registered components include custom components and system default components;

[0047] The receiving module is further used to receive a custom component input by a user;

[0048] The processing module is also used to register the custom component.

[0049] Optionally, the processing module is specifically used for:

[0050] The custom component is registered through an application programming interface (API).

[0051] Optionally, the processing module is further used for:

[0052] Submit the computing task to the computing engine for computing.

[0053] Optionally, the processing module is further used for:

[0054] The calculation result of the calculation task is obtained by running the toolkit through the component.

[0055] In a third aspect, an embodiment of the present application provides a storage medium, wherein the storage medium is used to store a computer program, and the computer program is used to implement the computing task acquisition method as described above.

[0056] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program, and the processor executes the computer program to implement the computing task acquisition method as described above.

[0057] The computing task acquisition method, device, storage medium and electronic device provided in the embodiments of the present application receive a JSON definition file input by a user, where the JSON definition file is a program file that describes the computing process of the computing task through component information, obtains the first component corresponding to the component information in the JSON definition file from a pre-defined component library, encapsulates the first component, and obtains the computing task. Since the developer only needs to configure the JSON definition file within the agreed specifications, and the remaining supporting related operations such as computing task encapsulation are automatically completed by the system, on the one hand, the programming workload of the developer is reduced, thereby improving the development efficiency of the computing task. On the other hand, the development of computing tasks is carried out under unified specifications and within the system, which is convenient for later management and maintenance, thereby reducing maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0059] Figure 1a A flowchart of a first embodiment of a method for obtaining a computing task provided in an embodiment of the present application;

[0060] Figure 1b A flowchart of a possible implementation method of the first embodiment of the computing task acquisition method provided in the embodiment of the present application;

[0061] Figure 2 A flowchart of a second embodiment of a method for obtaining a computing task provided in an embodiment of the present application;

[0062] Figure 3 A flowchart of a third embodiment of a method for obtaining a computing task provided in an embodiment of the present application;

[0063] Figure 4 A schematic diagram of the structure of an embodiment of a computing task acquisition device provided in an embodiment of the present application;

[0064] Figure 5 A schematic diagram of the structure of an electronic device embodiment provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0066] The embodiment of the present application takes obtaining a Spark computing task as an example to describe the technical solution of the present application in detail. It should be noted that when the computing engine used is different (the required computing tasks are different), the technical solution provided in the embodiment of the present application is also applicable.

[0067] First, the terms involved in the embodiments of the present application are explained:

[0068] A component is a software entity in a software system that has relatively independent functions, interfaces specified by contracts, obvious dependencies with the context, and can be independently deployed and assembled. In Spark computing tasks, components include three types: flow, segment, and step. These three components have their own basic and common composition information, and can also be expanded on the original basis. Step and segment support registering custom steps and segments in the form of components, as follows:

[0069] (1) Step is the most basic unit of Spark computing tasks. It can be referenced in the definition of segments or other steps. The basic information contained in a step is:

[0070] type: used to define the step type, which must be unique and used to register the step of this type as a component;

[0071] stepName: the name of the step;

[0072] dependencies: The dependent step, multiple ones can be configured.

[0073] (2) Segment: defines and describes a complete computing task chain. It can include one or more steps and the dependencies between multiple steps. The basic information contained in a segment includes:

[0074] type: used to define the segment type, which must be unique and used to register the segment of this type as a component;

[0075] name: the name of the segment;

[0076] steps: the step set referenced by the segment, multiple steps can be configured;

[0077] (3) Flow is used to organize and describe one or more segments. The basic information contained in flow is:

[0078] description: description of the calculation process;

[0079] version: the version information of the calculation process;

[0080] segments: the segment set of the computing process. Multiple segments can be configured.

[0081] JSON definition file, short for JS object notation definition file, is a lightweight data exchange format. It is based on a subset of ECMAScript (js specification developed by the European Computer Association), uses a text format that is completely independent of the programming language to store and represent data, has a concise and clear hierarchical structure, is easy for people to read and write, and is also easy for machines to parse and generate, and effectively improves network transmission efficiency. In this embodiment, the JSON definition file includes components that form the computing tasks defined by the JSON definition file and the structural relationships between different components. The following is an example JSON definition file:

[0082]

[0083] It can be seen that the JSON definition file of the above example contains two step definitions of type "demo1" and "demo2". Demo2 is a step that depends on demo1. These two steps belong to the segment of type "spark". There is only one segment definition in this example, and its version is v1.0.

[0084] The main idea of ​​the technical solution of this application: In the prior art, developers directly write Spark programs to obtain Spark computing tasks. On the one hand, developers need to spend a lot of time and energy in writing Spark programs, which leads to low development efficiency. On the other hand, due to the lack of a unified development architecture in the prior art, different developers work independently. Especially in a case where the data scale is large, the business scenarios are complex and changeable, and there are many computing tasks and developers involved, a lot of processing and maintenance work is required to ensure the uniformity of data caliber, which leads to high maintenance costs. Based on the above two considerations, the embodiment of the present application proposes a computing task acquisition method, device, storage medium and electronic device. Developers use custom components or system default components in JS object notation (JSON) definition files to configure Spark task computing processes. The Spark computing task development framework obtains the JSON definition file, and then parses, loads components and encapsulates them to obtain a Spark computing task. Since developers only need to develop JSON definition files and custom components within the agreed specifications, and other supporting related operations such as parsing of JSON definition files, loading of components, and packaging of Spark computing tasks are automatically completed by the development framework, as the components accumulated by the system become more and more abundant, developers only need to complete the definition of Spark computing tasks through configuration without coding. Therefore, the technical solution of the present application does not require a large amount of repetitive coding work, which greatly improves the development efficiency of Spark computing tasks. Furthermore, the standardized and streamlined development process reduces maintenance costs. In addition, the technical solution of the embodiment of the present application defines and describes the calculation process of a Spark computing task by configuration, which not only has the advantages of convenient operation and use, but also has the advantage of being easy for other developers or system maintenance personnel to understand.

[0085] Figure 1a The flowchart of the first embodiment of the computing task acquisition method provided in the embodiment of the present application is shown in FIG. The execution subject of the embodiment of the present application is an electronic device with computing capabilities, such as a computer, a server, etc. Figure 1a As shown, the computing task acquisition method of this embodiment includes:

[0086] S101. Receive a JSON definition file input by a user, where the JSON definition file includes component information.

[0087] In this step, the user configures the JSON definition file using custom components or system default components according to the agreed development specifications. When the configuration is completed, the system receives the JSON definition file input by the user based on the user's operation. The JSON definition file obtains a program file that describes the computing process to be experienced by a computing task by referencing related components (reflected as component information in the JSON definition file), such as the JSON definition file in the aforementioned example.

[0088] Exemplarily, the user configures the JSON definition file in an editor of the JSON definition file. When the configuration is completed, the user clicks the "Upload" function button, and the system can receive the JSON definition file.

[0089] Component information, that is, information describing a component, including component name, type, etc. As can be seen from the above analysis, a component can usually be described by component information. For example, in the JSON definition file of the above example, "step1":{

[0090] "type":"demo1",

[0091] "stepName":"demo_step1"

[0092] }”

[0093] That is the component information of step 1.

[0094] It can be understood that the user in the embodiment of the present application refers to the computing task developer.

[0095] S102: Obtain a first component corresponding to the component information in the JSON definition file from a predefined component library.

[0096] From the above analysis, it can be seen that the JSON definition file is a file that defines the computing process experienced by a computing task by referencing a series of components. That is, the JSON definition file includes information describing the components, such as component name, type, etc., rather than the actual components. Therefore, in order to finally obtain the computing task, this step needs to obtain the first component corresponding to the component information in the JSON definition file from the pre-defined component library.

[0097] Among them, the first component is the component corresponding to the component information in the JSON definition file. Therefore, the first component can be one or more, can be a custom combination, can be a system default component, can be a step, can be a segment, and there is no restriction here.

[0098] The component library is a database that stores all registered components.

[0099] Optionally, depending on the source of the components, the component library can be divided into a custom component library and a system default component library, which are used to store user-defined components and system default components respectively.

[0100] In one possible implementation, Figure 1b A flowchart of a possible implementation method of the first embodiment of the computing task acquisition method provided in the embodiment of the present application is shown as follows: Figure 1b As shown, obtaining the first component corresponding to the component information in the JSON definition file from the predefined component library includes:

[0101] S1021. Parse the JSON definition file and obtain a parsing result, where the parsing result includes a component category and a component name of the first component.

[0102] In order to obtain the first component, in this step, the JSON definition file needs to be parsed first to obtain the parsing result including the component category and component name. From the above analysis, it can be seen that the JSON definition file includes relevant component information. Therefore, by interpreting the JSON definition file, the number of the first component included in the JSON definition file, component category, component name and other information can be determined.

[0103] Among them, the component category refers to the category of the component. From the above analysis, we can know that the components in the JSON definition file are divided into three categories: step, segment and flow. Therefore, for a first component, its component category can only be one of step, segment and flow.

[0104] Component name, used to uniquely identify a component, such as "demo_step1" and "demo_step2" in the JOSN definition file in the previous example.

[0105] A component can be uniquely identified based on its type and name.

[0106] Optionally, in this embodiment, the parsing result can be presented in a table or a text file. For example, the parsing result of a JSON definition file can be shown in Table 1.

[0107] Table 1

[0108] Serial number Component Category Component Name 1 step demo_step1 2 step demo_step25 3 segment demo_segment3

[0109] S1022. Extract the first component from the component library according to the component category and component name of the first component.

[0110] In this step, after the component category and component name of the first component are determined, the corresponding first component can be easily found from the component library and extracted.

[0111] Optionally, the first component may be extracted from the component library by a key field screening method.

[0112] In this implementation, the JSON definition file is parsed and a parsing result is obtained, which includes the component category and component name of the first component. Based on the component category and component name of the first component, the first component is extracted from the component library, thereby automatically acquiring the first component corresponding to the component information in the JSON definition file, which is beneficial to improving the effectiveness of acquiring computing tasks and development efficiency.

[0113] S103: Encapsulate the first component to obtain a computing task.

[0114] In this step, after S102, the obtained first component is packaged to obtain a computing task that can be used in a computing engine to perform calculations to obtain computing results.

[0115] When there is one first component, the first component is packaged to obtain a computing task; when there are multiple first components, multiple first components are packaged together to obtain a computing task.

[0116] It should be noted that the encapsulation rules of the computing task are determined by the operating rules of the corresponding computing engine. For example, when the computing task is a Spark computing task, it is necessary to determine the encapsulation rules according to the operating rules of the Spark computing engine, and then encapsulate according to the encapsulation rules.

[0117] Optionally, after S103, the method of this embodiment further includes:

[0118] Submit the computing task to the computing engine for calculation.

[0119] In this step, after completing the encapsulation of the computing task, the system automatically submits the computing task to the computing engine so that the computing engine performs the calculation.

[0120] In this embodiment, by receiving a JSON definition file input by a user, the JSON definition file is a program file that describes the calculation process of a calculation task through component information, and obtaining a first component corresponding to the component information in the JSON definition file from a predefined component library, and encapsulating the first component to obtain the calculation task. Since the developer only needs to configure the JSON definition file within the agreed specifications, and the remaining supporting related operations such as calculation task encapsulation are automatically completed by the system, on the one hand, the programming workload of the developer is reduced, thereby improving the development efficiency of the calculation task. On the other hand, the development of the calculation task is carried out under a unified specification and within the system, which is convenient for later management and maintenance, thereby reducing maintenance costs.

[0121] Figure 2 The flowchart of the second embodiment of the computing task acquisition method provided in the embodiment of the present application is based on the above embodiment. Figure 2 As shown, in this embodiment, when the number of first components is at least two, the parsing result also includes a structural relationship between at least two first components, and the structural relationship is used to represent the hierarchical relationship and / or dependency relationship between the at least two first components. Accordingly, the first components are encapsulated to obtain a computing task, including:

[0122] The first component and the structural relationship are packaged to obtain a computing task.

[0123] It is understandable that when the number of first components is at least two, the JSON definition file also defines the hierarchical relationship and / or dependency relationship between different first components. To facilitate subsequent calculations by the calculation engine, when parsing the JSON definition file, the structural relationship between different first components can also be obtained. When generating a calculation task, the first components and the corresponding structural relationship are packaged to obtain a calculation task.

[0124] It should be noted that the structural relationship in this embodiment refers to a text file of the structural relationship between different first components.

[0125] Optionally, the first component and the structural relationship are packaged to obtain a computing task, including:

[0126] S201. Package the first component and the structural relationship with a component running toolkit to obtain a computing task.

[0127] In this step, the computing task is obtained by bundling and packaging the first component, the structural relationship, and the component running toolkit.

[0128] Among them, the component operation toolkit is pre-deployed in the development system, and is mainly used to assist the operation of the first component in the computing engine. The main functions of the component operation toolkit in this embodiment are as follows: (1) Enable different first components to execute in the computing engine according to the hierarchical relationship and / or dependency relationship specified by the structural relationship; (2) If there is a dependency relationship, it is responsible for passing the upstream operation results to the downstream; (3) After the operation is completed, all information during the operation is saved to a file. By packaging the component operation toolkit into the computing task, the orderliness of subsequent operation is guaranteed and the troubleshooting and tracking of problems are facilitated.

[0129] Accordingly, after S201, the method of this embodiment further includes:

[0130] S202: Submit the computing task to the computing engine for computing.

[0131] In this step, after obtaining the computing task by packaging in S201, the system automatically submits the computing task to the computing engine for execution of the computing so as to obtain the computing result.

[0132] S203: Obtain the calculation result of the calculation task through the component running toolkit.

[0133] In this step, when the computing engine performs calculations, the calculation results of the computing tasks are obtained through the component running toolkit, and saved or passed from the upstream to the downstream of the computing process to ensure that the calculations are carried out in an orderly manner. In addition, by obtaining and saving the results of the calculation process, subsequent maintenance is facilitated.

[0134] In this embodiment, a computing task is obtained by packaging the first component and the structural relationship with a component running toolkit, and the computing task is submitted to a computing engine for calculation. The computing result of the computing task is obtained through the component running toolkit. Since the computing tasks in the embodiment of the present application are componentized, when there are multiple first components, the component running toolkit is also encapsulated in the computing task, thereby ensuring the orderliness of the entire computing process in the computing engine and the transferability of the computing results, which is conducive to reducing the occurrence of problems in the computing process and the troubleshooting and tracking of problems when they occur, thereby reducing maintenance costs.

[0135] Figure 3 The flowchart of the third embodiment of the computing task acquisition method provided in the embodiment of the present application is based on the above embodiments. Figure 3 As shown, in this embodiment, before obtaining the first component corresponding to the component information in the JSON definition file from the predefined component library, the method of this embodiment further includes:

[0136] S302: Scan all registered components in the component library to obtain a component list.

[0137] In this step, in order to be able to obtain the first component in time later to save the time of generating computing tasks, before obtaining the first component, the system can first scan the component library to understand the status of all registered components in the component library and generate a component list, which includes the component categories and component names of all registered components.

[0138] All registered components include custom components and system default components. System default components are components that come with the system by default and do not need to be registered separately. Custom components are customized by users according to usage requirements. Therefore, they need to be registered to be recognized by the system. Only registered components are allowed to be used when encapsulating computing tasks.

[0139] After registration, the custom components will be saved in the component library together with the system default components for use in the subsequent development of computing tasks. All developers can use the components in the component library. It can be seen that component sharing can be achieved in the embodiment of the present application, and as more and more components are deposited in the component library, when users develop computing tasks, they can configure the JSON definition file according to the existing components of the system, which avoids developers from performing repetitive coding work, so that they can focus on the implementation of business logic, improving the efficiency and quality of obtaining computing tasks.

[0140] In addition, the registered custom components and system default components can be stored in different component libraries, such as a custom component library and a system default component library, or they can be stored in the same component library, or stored in other ways, such as dividing them into different component libraries according to component categories, etc., which is not limited here.

[0141] Accordingly, the component list may be differentiated according to the component libraries where the components are located. For example, in addition to the component category and the component name, the component list may also include information such as the component library where the component is located.

[0142] It should be noted that the obtained component list can also be updated in this step. In one possible implementation, all registered components in the component library are scanned at startup to obtain a component list. After that, always pay attention to whether there are new components added to the component library. If so, the component list is updated.

[0143] If the developer does not need to use a custom component that is not registered with the system when obtaining the computing task this time, before S302, the method of this embodiment further includes:

[0144] S300: Receive a custom component input by a user.

[0145] In this step, before registering the component, the developer will encode the custom component according to the agreed development specifications. After the encoding is completed, it will be uploaded to the system. Accordingly, the system will obtain the custom component according to the developer's operation.

[0146] S301. Register the custom component.

[0147] In this step, the system registers the received custom components, thereby achieving the "legalization" of the custom components.

[0148] Registration can be understood as the process of generating a component package from the coding file of the custom component, and naming and storing the component package, etc. The system can pre-set a special interface for custom component. In a possible implementation, the custom component is registered through an application programming interface (API).

[0149] Accordingly, after S302, in this embodiment, obtaining the first component corresponding to the component information in the JSON definition file from the predefined component library includes:

[0150] S303. According to the component list, obtain a first component corresponding to the component information in the JSON definition file from a predefined component library.

[0151] In this step, since the component list includes information such as component categories and component names of all registered components stored in the component library, the first component corresponding to the component information in the JSON definition file is obtained from the pre-defined component library according to the component list. On the one hand, it is possible to know in advance whether the first component corresponding to the component information has been registered and available. When there is a first component that is not included in the component list, the user can be notified in time to conduct error troubleshooting in time, such as troubleshooting the JSON definition file or custom components. On the other hand, when different component libraries are included, the component library where the first component is located can be promptly known according to the component list, and the corresponding first component can be loaded into the corresponding component library, which is conducive to improving the efficiency and accuracy of obtaining the first component.

[0152] In a possible implementation, the corresponding first component is obtained from the component library according to the component list and the component category and the component name of the first component in the parsing result.

[0153] In this embodiment, all registered components in the component library are scanned to obtain a component list, and based on the component list, the first component corresponding to the component information in the JSON definition file is obtained from the pre-defined component library, thereby improving the acquisition efficiency and accuracy of the first component and improving the acquisition efficiency of computing tasks. In addition, component resources can be shared through customized component registration and component library, which plays a positive role in improving the development efficiency of computing tasks and system maintenance.

[0154] To sum up, the beneficial effects of the text clustering method provided in the embodiment of the present application can be summarized as follows: it can develop computing tasks based on component configuration and expansion, which not only reduces the repetitive coding work of developers and improves the development efficiency of computing tasks, but also reduces the system maintenance cost and difficulty through standardized and streamlined development processes.

[0155] The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0156] Figure 4 A schematic diagram of the structure of an embodiment of a computing task acquisition device provided in an embodiment of the present application, such as Figure 4 As shown, in this embodiment, the computing task acquisition device 10 includes:

[0157] A receiving module 11 and a processing module 12 .

[0158] The receiving module 11 is used to receive the JS object notation JSON definition file input by the user, and the JSON definition file includes component information;

[0159] The processing module 12 is used to obtain the first component corresponding to the component information in the JSON definition file from a predefined component library; and encapsulate the first component to obtain a computing task.

[0160] In a possible implementation, the processing module 12 is specifically configured to:

[0161] Parsing the JSON definition file and obtaining a parsing result, where the parsing result includes a component category and a component name of the first component;

[0162] The first component is extracted from a component library according to the component category and the component name of the first component.

[0163] In another possible implementation, when the number of the first components is at least two, the parsing result also includes a structural relationship between the at least two first components, where the structural relationship is used to represent a hierarchical relationship and / or a dependency relationship between the at least two first components;

[0164] Accordingly, the processing module 12 is specifically used for:

[0165] The first component and the structural relationship are packaged to obtain the computing task.

[0166] Optionally, the processing module 12 is specifically used for:

[0167] The first component and the structural relationship are packaged with a component running toolkit to obtain a computing task.

[0168] Optionally, the processing module 12 is further used for:

[0169] Submit the computing task to the computing engine for calculation.

[0170] Optionally, the processing module 12 is further used for:

[0171] Get the calculation results of the calculation task through the component running toolkit.

[0172] In yet another possible implementation, the processing module 12 is further configured to:

[0173] Scan all registered components in the component library to obtain a component list, which includes component categories and component names of all registered components;

[0174] Accordingly, the processing module 12 is specifically used for:

[0175] According to the component list, a first component corresponding to the component information in the JSON definition file is obtained from a predefined component library.

[0176] Optionally, the registered components include custom components and system default components;

[0177] The receiving module 11 is also used to receive a custom component input by a user;

[0178] The processing module 12 is also used to register the custom component.

[0179] Optionally, the processing module 12 is specifically used for:

[0180] Register custom components through the application programming interface API.

[0181] The implementation principle and technical effects of this embodiment are similar to those of the method embodiment. Please refer to the method embodiment for details, and will not be described in detail here.

[0182] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0183] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (FPGA), etc. For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0184] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0185] Figure 5 A schematic diagram of the structure of an electronic device embodiment provided in the present application, such as Figure 5As shown, in this embodiment, the electronic device 20 may include: a processor 21, a memory 22, a communication interface 23 and a system bus 24. The memory 22 and the communication interface 23 are connected to the processor 21 through the system bus 24 and communicate with each other. The memory 22 is used to store computer execution instructions, and the communication interface 23 is used to communicate with other devices. When the processor 21 executes a computer program, a solution such as any of the above method embodiments is implemented.

[0186] In this Figure 5 In the embodiment, the processor 21 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components.

[0187] The memory 22 may include a random access memory (RAM), may include a read-only memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0188] The communication interface 23 is used to implement the communication between the database access device and other devices (such as a client, a read-write library, and a read-only library).

[0189] The system bus 24 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0190] Optionally, an embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes a method as described in any of the above method embodiments.

[0191] Optionally, an embodiment of the present application further provides a chip for executing instructions, wherein the chip is used to execute the method of any of the above method embodiments.

[0192] An embodiment of the present application also provides a program product, which includes a computer program, which is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when at least one processor executes the computer program, it can implement the method of any of the above method embodiments.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for obtaining a computing task, characterized in that: include: Receive a JS object notation JSON definition file input by a user, wherein the JSON definition file includes component information; Acquire a first component corresponding to the component information in the JSON definition file from a predefined component library; Encapsulating the first component to obtain a computing task; The obtaining, from a predefined component library, a first component corresponding to the component information in the JSON definition file comprises: parsing the JSON definition file and obtaining a parsing result, wherein the parsing result comprises a component category and a component name of the first component corresponding to the component information in the JSON definition file; and extracting the first component from the component library according to the component category and the component name of the first component; When the number of the first components is at least two, the parsing result also includes a structural relationship between the at least two first components, and the structural relationship is used to represent the hierarchical relationship and / or dependency relationship between the at least two first components; accordingly, the encapsulation of the first components to obtain a computing task includes: packaging the first components and the structural relationship with a component operation toolkit to obtain the computing task, wherein the component operation toolkit is pre-deployed in the development system, and is used to assist the first component in running in the computing engine according to the hierarchical relationship and / or the dependency relationship specified by the structural relationship.

2. The method according to claim 1, characterized in that Before obtaining the first component corresponding to the component information in the JSON definition file from the predefined component library, the method further includes: Scan all registered components in the component library to obtain a component list, wherein the component list includes component categories and component names of all registered components; Accordingly, obtaining the first component corresponding to the component information in the JSON definition file from the predefined component library includes: According to the component list, a first component corresponding to the component information in the JSON definition file is obtained from a predefined component library.

3. The method according to claim 2, characterized in that The registered components include custom components and system default components; before scanning all registered components in the component library and obtaining a component list, the method further includes: A custom component that receives user input; Register the custom component.

4. The method according to claim 3, characterized in that The registering of the custom component includes: The custom component is registered through an application programming interface (API).

5. The method according to claim 1, characterized in that The method further comprises: Submit the computing task to the computing engine for computing.

6. The method according to claim 5, characterized in that The method further comprises: The calculation result of the calculation task is obtained by running the toolkit through the component.

7. A computing task acquisition device, characterized in that: include: A receiving module, used for receiving a JS object notation JSON definition file input by a user, wherein the JSON definition file includes component information; A processing module, configured to obtain a first component corresponding to the component information in the JSON definition file from a predefined component library; and encapsulate the first component to obtain a computing task; The processing module is specifically used to parse the JSON definition file and obtain a parsing result, wherein the parsing result includes a component category and a component name of the first component corresponding to the component information in the JSON definition file; extracting the first component from the component library according to the component category and the component name of the first component; When the number of the first components is at least two, the parsing result also includes a structural relationship between the at least two first components, where the structural relationship is used to represent a hierarchical relationship and / or a dependency relationship between the at least two first components; Accordingly, the processing module is specifically used for: Packing the first component and the structural relationship to obtain the computing task; The processing module is specifically used to: package the first component and the structural relationship with a component operation toolkit to obtain a computing task, wherein the component operation toolkit is pre-deployed in the development system and is used to assist the first component in running in the computing engine according to the hierarchical relationship and / or the dependency relationship specified by the structural relationship.

8. A storage medium, characterized in that: The storage medium is used to store a computer program, and the computer program is used to implement the computing task acquisition method described in any one of claims 1 to 6.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program, and the processor executes the computer program to implement the computing task acquisition method according to any one of claims 1 to 6.

Images