Resource scheduling method and device based on execution component, equipment and medium

Abstract

The invention discloses a resource scheduling method and device based on an execution component, equipment and a medium, relates to the field of data security, is applied to a preset computing platform, and comprises the following steps: obtaining a resource scheduling task, and determining the execution component corresponding to the resource scheduling task in the preset computing platform; determining a computing framework corresponding to the resource scheduling task, and calling an execution component by utilizing an execution engine in a preset computing platform to establish a computing cluster corresponding to the resource scheduling task through the computing framework; and constructing a resource scheduling command corresponding to the resource scheduling task by utilizing an execution component, calling the resource scheduling command through an execution engine, and sending the resource scheduling command to a computing framework, so that the computing framework executes the resource scheduling task through a computing cluster according to the resource scheduling command. The execution component layer is extracted under the execution engine, and the component layer packages the algorithm and the execution steps into the corresponding command to be called by the engine, so that the execution component is separated from the engine, and the stability of the engine is ensured.

Description

Technical Field

[0001] The present invention relates to the field of data security, and in particular to a resource scheduling method, device, equipment and medium based on execution components. Background Art

[0002] Currently, the Data Security Law stipulates that "original data cannot leave the domain, and data is available but not visible", but traditional data sharing methods either require original data to leave the domain or require third-party applications to be placed in the domain where the original data is located for calculation. Therefore, since original data is not allowed to leave the domain, and placing third-party applications in the domain where the original data is located for calculation requires a comprehensive audit of the applications, it is difficult to eliminate malicious attacks by applications. For this reason, multi-party secure computing technology has become the only choice to solve the data sharing problem. However, due to the high technical complexity and strong algorithm interactivity of the multi-party secure computing platform, the underlying algorithm needs to be independent and self-contained. Therefore, the entire system is divided into two major parts: the management plane and the computing framework. And since multi-party secure computing is a natural distributed computing scenario, it is necessary to use a distributed scheduling engine to complete the scheduling logic.

[0003] There are currently two types of distributed schedulers on the market: one is based on the traditional distributed framework, using distributed computing frameworks such as spark, ray, eggroll, etc. to complete cross-node scheduling, and implement its own business logic under the framework; the other is to use self-developed scheduling logic to complete distributed scheduling tasks, but the current privacy and security computing manufacturers are more focused on "multi-party secure computing" itself, and the scheduling strategy is focused on completing the cross-node scheduling logic. The scheduling logic is mixed with calls to local algorithm components and cross-node task processing logic. The scheduling strategy and process are mixed together, making the scheduler an exclusive customized module for each privacy and security computing company, and it is difficult to expand to more business scenarios.

[0004] And because the management plane and the computing framework are separated, many manufacturers have open-sourced their own computing frameworks. Each computing framework has its own advantages. Different computing frameworks can be integrated under the same management plane, which will lead to the problem of inconsistent technical architecture and functions of different computing frameworks. It is necessary to be compatible with the processes, usage methods, and even underlying toolkits of each computing framework, which makes code maintenance difficult.

[0005] At present, the distributed schedulers of multi-party secure computing platforms on the market mostly use schedulers based on traditional distributed frameworks and self-developed ones that focus on solving privacy-safe computing tasks. However, since the distributed schedulers developed by various manufacturers directly call the computing framework in the management-side business code when integrating the computing framework, and do not extract an independent access layer, a lot of configuration and condition judgment are required when accessing different computing frameworks, which greatly reduces the maintainability of the code. In addition, when integrating the third-party frameworks of various manufacturers, the third-party code is directly modified to achieve the purpose of adaptation, which leads to the intrusion of the third-party code and inconvenience in maintenance. Therefore, how to improve the applicability of multi-party secure computing platforms is a problem to be solved in this field. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a resource scheduling method, device, equipment and storage medium based on execution components, by extracting the execution component layer under the execution engine, the component layer encapsulates the algorithm and execution steps into corresponding commands for the engine to call, thereby realizing the separation of the execution component and the engine and ensuring the stability of the engine. The specific scheme is as follows:

[0007] In a first aspect, the present application provides a resource scheduling method based on an execution component, which is applied to a preset computing platform, including:

[0008] Obtaining a resource scheduling task, and determining an execution component corresponding to the resource scheduling task in the preset computing platform;

[0009] Determine a computing framework corresponding to the resource scheduling task, and use an execution engine in the preset computing platform to call the execution component to establish a computing cluster corresponding to the resource scheduling task through the computing framework;

[0010] The execution component is used to construct a resource scheduling command corresponding to the resource scheduling task, and the resource scheduling command is called by the execution engine, and the resource scheduling command is sent to the computing framework, so that the computing framework executes the resource scheduling task according to the resource scheduling command through the computing cluster.

[0011] Optionally, before determining the execution component corresponding to the resource scheduling task in the preset computing platform, the method further includes:

[0012] An execution component layer is constructed according to the execution engine in the preset computing platform, so as to use the execution engine to determine the execution component corresponding to the resource scheduling task in the preset computing platform.

[0013] Optionally, after determining the execution component corresponding to the resource scheduling task in the preset computing platform, the method further includes:

[0014] The algorithm and execution steps corresponding to the execution component are encapsulated as corresponding first execution commands according to a preset operating system interface standard, so as to use the execution engine to call the corresponding first execution command through the interface corresponding to the execution component to establish the computing cluster.

[0015] Optionally, the using the execution engine in the preset computing platform to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework includes:

[0016] Using the execution engine in the preset computing platform to call the computing cluster establishment command corresponding to the execution component, and using the execution component to call the computing cluster establishment method corresponding to the computing cluster establishment command;

[0017] The computing cluster corresponding to the resource scheduling task is established through the computing framework based on the computing cluster establishment method.

[0018] Optionally, the resource scheduling method based on execution components further includes:

[0019] If a component access instruction from the user is received, encapsulating the component to be accessed corresponding to the component access instruction into a corresponding second execution command based on the preset operating system interface standard;

[0020] The execution logic of the component to be connected is added in the execution engine according to the second execution command, so that the execution engine is used to call the second execution command to execute corresponding execution steps according to the execution logic through the component to be connected.

[0021] Optionally, the using the execution engine in the preset computing platform to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework includes:

[0022] Determine the task execution logic of the computing framework, and construct a corresponding executable file through the task execution logic;

[0023] The execution engine in the preset computing platform is used to call the execution component, and the computing cluster corresponding to the resource scheduling task is established according to the executable file.

[0024] Optionally, establishing the computing cluster corresponding to the resource scheduling task through the computing framework according to the executable file includes:

[0025] Constructing an interface corresponding to the executable file based on the presentation layer state transfer principle, and standardizing the interface corresponding to the executable file;

[0026] The computing cluster corresponding to the resource scheduling task is established according to the executable file through a standardized interface.

[0027] In a second aspect, the present application provides a resource scheduling device based on an execution component, which is applied to a preset computing platform, including:

[0028] A component determination module, used to obtain a resource scheduling task and determine an execution component corresponding to the resource scheduling task in the preset computing platform;

[0029] A cluster establishment module, used to determine the computing framework corresponding to the resource scheduling task, and use the execution engine in the preset computing platform to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework;

[0030] The task execution module is used to use the execution component to construct the resource scheduling command corresponding to the resource scheduling task, and call the resource scheduling command through the execution engine, and send the resource scheduling command to the computing framework so that the computing framework executes the resource scheduling task according to the resource scheduling command through the computing cluster.

[0031] In a third aspect, the present application provides an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned resource scheduling method based on execution components.

[0032] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned resource scheduling method based on execution components.

[0033] In this application, the resource scheduling task is first obtained, and the execution component corresponding to the resource scheduling task in the preset computing platform is determined, and then the computing framework corresponding to the resource scheduling task is determined, and the execution engine in the preset computing platform is used to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework, and then the execution component is used to construct the resource scheduling command corresponding to the resource scheduling task, and the resource scheduling command is called through the execution engine, and the resource scheduling command is sent to the computing framework, so that the computing framework executes the resource scheduling task according to the resource scheduling command through the computing cluster. In this way, by using the execution engine to manage multi-party secure task scheduling, the distributed scheduling framework can be isolated from the multi-party secure computing platform, the scheduling logic can be made lightweight, and various business logics such as data sharing and auditing can be easily launched, and the management surface and the computing framework are isolated through the execution component so that each logic can be independently launched, which helps to reduce the configuration judgment logic and improve the maintainability of the computing platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0035] Figure 1 A flow chart of a resource scheduling method based on execution components provided in this application;

[0036] Figure 2 A schematic diagram of the engine and computing framework calling process provided by this application;

[0037] Figure 3 A schematic diagram of the calling process of an engine and a computing framework;

[0038] Figure 4 A schematic diagram of the technical architecture corresponding to a resource scheduling method based on execution components provided in this application;

[0039] Figure 5 A flow chart of a computing cluster establishment method provided in this application;

[0040] Figure 6 A schematic diagram of the structure of a resource scheduling device based on an execution component provided in this application;

[0041] Figure 7 A structural diagram of an electronic device provided for this application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] At present, the distributed schedulers of multi-party secure computing platforms on the market mostly use schedulers based on traditional distributed frameworks and self-developed ones that focus on solving privacy-sensitive and secure computing tasks. However, since the distributed schedulers developed by various manufacturers directly call the computing framework in the management-side business code to integrate the computing framework, and directly modify the third-party code to achieve the purpose of adaptation, it leads to inconvenience in maintenance. The present application uses an execution engine to manage multi-party secure task scheduling, which can isolate the distributed scheduling framework from the multi-party secure computing platform, and isolate the management side and the computing framework through execution components, which helps to independently develop the logic of each platform and improve the maintainability of the computing platform.

[0044] See also Figure 1 As shown, an embodiment of the present invention discloses a resource scheduling method based on an execution component, which is applied to a preset computing platform, including:

[0045] Step S11, obtaining a resource scheduling task, and determining an execution component corresponding to the resource scheduling task in the preset computing platform.

[0046] The preset computing platform in this embodiment is mainly used to execute the resource scheduling tasks (PSI, Pressure Stall Information) corresponding to each computing framework. Figure 2 As shown, the resource scheduling task can be obtained first, and the execution component corresponding to the resource scheduling task in the preset computing platform can be determined. It should be pointed out that before determining the execution component corresponding to the resource scheduling task in the preset computing platform, it is also necessary to build an execution component layer according to the execution engine in the preset computing platform, so as to use the execution engine to determine the execution component corresponding to the resource scheduling task in the preset computing platform.

[0047] After determining the execution component corresponding to the resource scheduling task in the preset computing platform, the algorithm and execution steps corresponding to the execution component are encapsulated as the corresponding first execution command according to the preset operating system interface standard, so as to use the execution engine to call the corresponding first execution command through the interface corresponding to the execution component to establish a computing cluster.

[0048] It should be pointed out that the preset computing platform in this embodiment is a multi-party secure computing platform that can meet the development needs of the multi-party secure computing platform, support better resource sharing, support the integration of multi-supplier multi-party secure computing frameworks, and support multi-role participation. Among them, since supporting the integration of multi-supplier multi-party secure computing frameworks is a technically important part of the entire platform, it determines the scalability and stability of the system, and plays a vital role in long-term business development and technological progress. Therefore, in this embodiment, under the technical requirements of flexibly supporting multiple multi-party secure computing frameworks and ensuring the stability of the platform, the multi-party secure computing platform is used to extract the following under the execution engine: Figure 2The execution component layer shown completely separates the component docking logic from the engine. The component layer uses command components to dock each algorithm, following the POSIX (Portable Operating System Interface) standard, encapsulating each algorithm and each necessary execution step into a shell command, which is provided to the engine for calling, thus achieving the separation of the execution component and the engine to the greatest extent.

[0049] Step S12: determine the computing framework corresponding to the resource scheduling task, and use the execution engine in the preset computing platform to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework.

[0050] In this embodiment, the computing framework corresponding to the resource scheduling task can be determined, and the execution engine in the preset computing platform can be used to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework. And when the execution engine in the preset computing platform is used to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework, the execution engine in the preset computing platform can be used to call the computing cluster establishment command corresponding to the execution component, and the execution component can be used to call the computing cluster establishment method corresponding to the computing cluster establishment command, and then the computing cluster corresponding to the resource scheduling task can be established through the computing framework based on the computing cluster establishment method. Figure 2 As shown, in this embodiment, after the PSI task is started through the execution engine of the computing platform, the command to establish a computing cluster is executed to establish the corresponding computing cluster. After that, the execution component can call the cluster establishment method corresponding to the current three-party framework, namely, computing framework A, and establish the corresponding computing cluster through the preset sidecar A (an executable file) and computing framework A.

[0051] Step S13: Use the execution component to construct a resource scheduling command corresponding to the resource scheduling task, and call the resource scheduling command through the execution engine, and send the resource scheduling command to the computing framework, so that the computing framework executes the resource scheduling task according to the resource scheduling command through the computing cluster.

[0052] In this embodiment, after the computing cluster is built, the execution component can be used to build the resource scheduling command corresponding to the resource scheduling task, and the resource scheduling command is called by the execution engine, and the resource scheduling command is sent to the computing framework, so that the computing framework executes the resource scheduling task according to the resource scheduling command through the computing cluster. Figure 2As shown, after computing framework A establishes a computing cluster, the engine executes the corresponding PSI command, and then the execution component can call A's PSI command, and then computing framework A starts to process the corresponding PSI process, and ends after PSI is completed. And in this embodiment, if the user's component access instruction is received, the component to be connected corresponding to the component access instruction is encapsulated as a corresponding second execution command based on the preset operating system interface standard, that is, the POSIX standard, and the execution logic of the component to be connected is added to the execution engine according to the second execution command, so as to use the execution engine to call the second execution command through the component to be connected to execute the corresponding execution steps according to the execution logic. In this way, when a new component needs to be connected, the corresponding component command is implemented according to the POSIX standard, and then the corresponding command execution logic is added to the plug-in layer of the execution engine. For the execution engine, it only needs to call the command, pass in the parameters and obtain the results. It is no longer necessary to pay attention to the specific logic of the component, and the calling logic is standardized, and the logic and code isolation are realized, which ensures the stability of the engine, and the execution component can also be reused in any required scenario.

[0053] In other words, for the multi-party secure computing platform, the focus is on resource sharing, multi-supplier framework integration, and multi-role participation. Among them, resource sharing is to solve the problems of data listing, data description information circulation, and data value circulation. It is necessary to complete the logic of data description information sharing, use application, review, etc. based on multi-party secure computing, and trade the data value. This part of the logic, namely resource sharing, is the business logic mixed in the scheduler, which needs to be extracted and operated separately to better meet future development needs. Multi-vendor framework integration is to complete the integration of multi-party secure computing frameworks of multiple vendors. Since each manufacturer's implementation has its own focus and advantages, the advantages of each framework can be utilized by integrating the frameworks of each manufacturer, thereby improving the platform capabilities while reducing repetitive development work. In this embodiment, the computing framework access capability can be extracted separately as an access layer for independent development and maintenance, and at the same time, it can be better compatible with different computing frameworks; multi-role participation is because in order to increase the value of data, data needs to be governed, among which data quality assessment, data asset assessment, data governance and other highly professional tasks require professional personnel to operate. Data providers usually do not have such capabilities, so it is necessary to introduce third-party assistance, and the platform needs to have role management, rapid application access and rapid process expansion capabilities. In this embodiment, by extracting the computing framework access layer, it is possible to quickly access the corresponding algorithm when there is a new demand, without affecting the original system function, thereby improving the development speed and stability, so as to improve the technology's response speed to the market. In this way, this embodiment focuses on data circulation rather than "multi-party secure computing" itself, focusing on resource sharing, multi-supplier framework integration, and multi-role participation, thereby opening up the platform's usage pattern, and the management plane and computing framework can be isolated through execution components so that their respective logics can be independently developed. Reduce configuration judgment logic and improve maintainability.

[0054] In this embodiment, based on Figure 3 The computing platform technology architecture shown in the figure can first obtain the resource scheduling task, and determine the execution component corresponding to the resource scheduling task in the preset computing platform, and then determine the computing framework corresponding to the resource scheduling task, and use the execution engine in the preset computing platform to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework, and then use the execution component to build the resource scheduling command corresponding to the resource scheduling task, and call the resource scheduling command through the execution engine, and send the resource scheduling command to the computing framework, so that the computing framework can execute the resource scheduling task according to the resource scheduling command through the computing cluster. In this way, compared with Figure 4The existing PSI engine and computing framework calling process shown in the figure can isolate the distributed scheduling framework from the multi-party secure computing platform by adopting an execution engine to manage multi-party secure task scheduling, making the scheduling logic lightweight and easily deploying various business logics such as data sharing and auditing. The management plane and the computing framework are isolated through the execution component so that each logic can be independently deployed, which helps to reduce the configuration judgment logic and improve the maintainability of the computing platform.

[0055] Based on the previous embodiment, it can be seen that the present application can use an execution engine to manage multi-party secure task scheduling, isolating the distributed scheduling framework from the multi-party secure computing platform. Next, the process of establishing a computing cluster will be described in detail in this embodiment. Figure 5 As shown, the embodiment of the present application discloses a specific method for establishing a computing cluster, which is applied to a preset computing platform, including:

[0056] Step S21: determine the task execution logic of the computing framework, and construct a corresponding executable file through the task execution logic.

[0057] In this embodiment, it is understandable that when accessing a third-party computing framework, it is often encountered that the third-party computing framework does not provide complete use of logical functions, such as not providing a multi-node cluster establishment interface, not providing a result data download interface, etc. Therefore, in this embodiment, in order to be able to follow up on the version iteration of the third party at the lowest cost for a long time, without modifying the other party's code, the corresponding logic can be implemented independently and packaged into an independent executable file, which is deployed together with the third-party framework. This independent executable file is called a sidecar. Therefore, in this embodiment, the task execution logic of the computing framework can be determined, and the corresponding executable file can be constructed through the task execution logic.

[0058] In this embodiment, the introduction of the sidecar ensures the standardization of the management of the third-party computing framework. The operation of similar functions of different computing frameworks can be realized through standard interfaces, and the implementation logic differences of the underlying computing framework are shielded at the sidecar level, which is more friendly to the engine call. On the other hand, it realizes zero-intrusion addition of functions to the third-party computing framework and compresses the code modification of the third-party framework to the greatest extent. Through the above technical solution, in this embodiment, the third-party framework can be adapted through the sidecar to achieve the purpose of reducing code intrusion. When the version needs to be updated, there is no need to do code comparison, and since there is no code intrusion, the stability of the third-party framework is also guaranteed.

[0059] Step S22: Using the execution engine in the preset computing platform to call the execution component, and establishing a computing cluster corresponding to the resource scheduling task according to the executable file.

[0060] Based on the above steps, it can be understood that the sidecar for tripartite framework integration is proposed and implemented in this embodiment, and the required capability expansion of the computing framework is reflected in the sidecar, and the sidecar adopts a restfull interface, and standardizes the interface for similar functions. The sidecar is also packaged, deployed, and managed separately, and exists in conjunction with the execution component; in this embodiment, the execution engine in the preset computing platform can be used to call the execution component, and the computing cluster corresponding to the resource scheduling task can be established according to the executable file. Specifically, the interface corresponding to the executable file, that is, the restfull interface, can be constructed based on the representational state transfer principle (REST, Representational State Transfer), and the interface corresponding to the executable file can be standardized, and then the computing cluster corresponding to the resource scheduling task can be established according to the executable file through the standardized interface.

[0061] In this way, based on the above technical solution combined with the above embodiment, the calling process of the computing framework can be extracted to form an access layer, and the corresponding commands are implemented using the POSIX standard, and provided to the engine for calling, thereby forming an execution component, and the execution component is packaged separately, deployed separately, and managed separately to form a self-contained system; afterwards, the engine accesses the execution component in the form of a command plug-in, and when there is a new execution component command access, the code is embedded into the engine in the form of a plug-in to ensure that the engine core code does not need to be modified when the algorithm is expanded. In the above process, it can be understood that the execution component and the sidecar adopt a stateless design, and all states are obtained from the engine and the computing framework, thereby ensuring the decoupling of the execution component and the sidecar logic. That is to say, in this embodiment, a self-developed lightweight process engine can be used to manage multi-party secure task scheduling, and traditional distributed scheduling frameworks such as spark, eggroll, and ray can be isolated from the multi-party secure computing platform, where the specific computing framework can choose or not choose spark, eggroll, ray and other frameworks to be platform-independent. In this way, the scheduling logic becomes lightweight, and various business logics such as data sharing and auditing can be easily deployed. It also uses a self-developed lightweight process engine to manage multi-party security task scheduling, replacing traditional distributed scheduling frameworks such as spark, eggroll, and ray for distributed scheduling, avoiding the management of complex and large spark, eggroll, and ray.

[0062] See also Figure 6 As shown, the embodiment of the present application also discloses a resource scheduling device based on an execution component, which is applied to a preset computing platform, including:

[0063] The component determination module 11 is used to obtain a resource scheduling task and determine an execution component corresponding to the resource scheduling task in the preset computing platform;

[0064] A cluster establishment module 12 is used to determine the computing framework corresponding to the resource scheduling task, and use the execution engine in the preset computing platform to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework;

[0065] The task execution module 13 is used to use the execution component to construct the resource scheduling command corresponding to the resource scheduling task, and call the resource scheduling command through the execution engine, and send the resource scheduling command to the computing framework so that the computing framework executes the resource scheduling task according to the resource scheduling command through the computing cluster.

[0066] This embodiment can obtain resource scheduling tasks, and determine the execution component corresponding to the resource scheduling task in the preset computing platform, and then determine the computing framework corresponding to the resource scheduling task, and use the execution engine in the preset computing platform to call the execution component to establish a computing cluster corresponding to the resource scheduling task through the computing framework, and then use the execution component to construct a resource scheduling command corresponding to the resource scheduling task, and call the resource scheduling command through the execution engine, and send the resource scheduling command to the computing framework, so that the computing framework executes the resource scheduling task according to the resource scheduling command through the computing cluster. In this way, by using the execution engine to manage multi-party security task scheduling, the distributed scheduling framework can be isolated from the multi-party security computing platform, the scheduling logic can be made lightweight, and various business logics such as data sharing and auditing can be easily deployed, and the management surface and the computing framework are isolated through the execution component so that each logic can be independently deployed, which helps to reduce the configuration judgment logic and improve the maintainability of the computing platform.

[0067] In some specific embodiments, the task processing device further includes:

[0068] The component layer construction module is used to construct an execution component layer according to the execution engine in the preset computing platform, so as to use the execution engine to determine the execution component corresponding to the resource scheduling task in the preset computing platform.

[0069] In some specific embodiments, the task processing device further includes:

[0070] The command encapsulation module is used to encapsulate the algorithm and execution steps corresponding to the execution component into a corresponding first execution command according to a preset operating system interface standard, so as to use the execution engine to call the corresponding first execution command through the interface corresponding to the execution component to establish the computing cluster.

[0071] In some specific embodiments, the cluster establishment module 12 specifically includes:

[0072] A command calling unit, used to use the execution engine in the preset computing platform to call the computing cluster establishment command corresponding to the execution component, and use the execution component to call the computing cluster establishment method corresponding to the computing cluster establishment command;

[0073] A cluster establishing unit is used to establish the computing cluster corresponding to the resource scheduling task through the computing framework based on the computing cluster establishing method.

[0074] In some specific embodiments, the task processing device further includes:

[0075] An instruction encapsulation module is used for encapsulating the component to be accessed corresponding to the component access instruction into a corresponding second execution command based on the preset operating system interface standard upon receiving a component access instruction from the user;

[0076] A logic adding module is used to add the execution logic of the component to be connected in the execution engine according to the second execution command, so as to use the execution engine to call the second execution command to execute the corresponding execution steps according to the execution logic through the component to be connected.

[0077] In some specific embodiments, the cluster establishment module 12 specifically includes:

[0078] A file construction unit, used to determine the task execution logic of the computing framework and construct a corresponding executable file through the task execution logic;

[0079] A component calling unit is used to call the execution component by using the execution engine in the preset computing platform, and to establish the computing cluster corresponding to the resource scheduling task according to the executable file.

[0080] In some specific embodiments, the cluster establishment module 12 specifically includes:

[0081] An interface construction unit, used to construct an interface corresponding to the executable file based on the presentation layer state transfer principle, and to perform interface standardization on the interface corresponding to the executable file;

[0082] A cluster establishing unit is used to establish the computing cluster corresponding to the resource scheduling task according to the executable file through a standardized interface.

[0083] Furthermore, the present application also discloses an electronic device. Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0084] Figure 7A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the resource scheduling method based on the execution component disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0085] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0086] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0087] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the resource scheduling method based on the execution component executed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0088] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the resource scheduling method based on the execution component disclosed above is implemented. For the specific steps of the method, reference can be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.

[0089] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0090] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0091] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0092] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0093] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A resource scheduling method based on execution components, characterized in that: Applies to pre-defined computing platforms, including: Obtaining a resource scheduling task, and determining an execution component corresponding to the resource scheduling task in the preset computing platform; Determine a computing framework corresponding to the resource scheduling task, and use an execution engine in the preset computing platform to call the execution component to establish a computing cluster corresponding to the resource scheduling task through the computing framework; The execution component is used to construct a resource scheduling command corresponding to the resource scheduling task, and the resource scheduling command is called by the execution engine, and the resource scheduling command is sent to the computing framework, so that the computing framework executes the resource scheduling task according to the resource scheduling command through the computing cluster.

2. The resource scheduling method based on execution components according to claim 1, characterized in that: Before determining the execution component corresponding to the resource scheduling task in the preset computing platform, the method further includes: An execution component layer is constructed according to the execution engine in the preset computing platform, so as to use the execution engine to determine the execution component corresponding to the resource scheduling task in the preset computing platform.

3. The resource scheduling method based on execution components according to claim 2, characterized in that: After determining the execution component corresponding to the resource scheduling task in the preset computing platform, the method further includes: The algorithm and execution steps corresponding to the execution component are encapsulated as corresponding first execution commands according to a preset operating system interface standard, so as to use the execution engine to call the corresponding first execution command through the interface corresponding to the execution component to establish the computing cluster.

4. The resource scheduling method based on execution components according to claim 3 is characterized in that: The using the execution engine in the preset computing platform to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework includes: Using the execution engine in the preset computing platform to call the computing cluster establishment command corresponding to the execution component, and using the execution component to call the computing cluster establishment method corresponding to the computing cluster establishment command; The computing cluster corresponding to the resource scheduling task is established through the computing framework based on the computing cluster establishment method.

5. The resource scheduling method based on execution components according to claim 3 is characterized in that: Also includes: If a component access instruction from the user is received, encapsulating the component to be accessed corresponding to the component access instruction into a corresponding second execution command based on the preset operating system interface standard; The execution logic of the component to be connected is added in the execution engine according to the second execution command, so that the execution engine is used to call the second execution command to execute corresponding execution steps according to the execution logic through the component to be connected.

6. The resource scheduling method based on execution components according to any one of claims 1 to 5, characterized in that: The using the execution engine in the preset computing platform to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework includes: Determine the task execution logic of the computing framework, and construct a corresponding executable file through the task execution logic; The execution engine in the preset computing platform is used to call the execution component, and the computing cluster corresponding to the resource scheduling task is established according to the executable file.

7. The resource scheduling method based on execution components according to claim 6, characterized in that: The step of establishing the computing cluster corresponding to the resource scheduling task through the computing framework according to the executable file includes: Constructing an interface corresponding to the executable file based on the presentation layer state transfer principle, and standardizing the interface corresponding to the executable file; The computing cluster corresponding to the resource scheduling task is established according to the executable file through a standardized interface.

8. A resource scheduling device based on execution components, characterized in that: Applies to pre-defined computing platforms, including: A component determination module, used to obtain a resource scheduling task and determine an execution component corresponding to the resource scheduling task in the preset computing platform; A cluster establishment module, used to determine the computing framework corresponding to the resource scheduling task, and use the execution engine in the preset computing platform to call the execution component to establish the computing cluster corresponding to the resource scheduling task through the computing framework; The task execution module is used to use the execution component to construct the resource scheduling command corresponding to the resource scheduling task, and call the resource scheduling command through the execution engine, and send the resource scheduling command to the computing framework so that the computing framework executes the resource scheduling task according to the resource scheduling command through the computing cluster.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the resource scheduling method based on the execution component as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the resource scheduling method based on execution components as described in any one of claims 1 to 7.

Images