Distributed task execution method and device based on alliance, equipment and medium
By adopting a federated distributed task execution method on a distributed platform, separating scheduling logic and business logic, the problem of poor stability of task execution of distributed platform is solved, and higher stability and scalability are achieved.
Patent Information
- Application Number
- CN202411273871.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-12-27
AI Technical Summary
The existing distributed platforms have poor stability when executing tasks, and the scheduling logic and business logic are mixed, resulting in performance problems and difficulty in solving distributed transactions and network communication problems.
Adopt a distributed task execution method based on alliances, obtain process status commands by receiving process forms, control the local computing framework to pull up algorithm components, and send process commands to the target alliance to pull up algorithm components of participants, separate scheduling logic and business Logic, centrally manage cross-node scheduling logic.
It improves the stability of task execution of distributed platform, isolates the impact of remote uncertainty, simplifies logic, improves the adaptability and scalability of computing frameworks, and reduces the workload of work nodes.
Smart Images
Figure CN119201389A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data security technology, and in particular to a distributed task execution method, device, equipment and medium based on an alliance. Background Art
[0002] With the development of computer technology, data has become more and more important. It contains huge value, and the need to share data value has become stronger and stronger. However, with the increasing emphasis on data security, there are more and more restrictions on data sharing, and gradually data islands have been formed. Data owners dare not share data with the outside world, and data demanders cannot use the required data. At present, the basic principle of data sharing is "original data does not leave the domain, and data is available but not visible", that is, the applications of the three parties are placed in the domain where the original data is located for calculation. Then, it is necessary to conduct a comprehensive audit of the application, and it is difficult to eliminate malicious attacks by the application, such as stealing data. For this reason, multi-party secure computing technology has become the only choice to solve the problem of data sharing. It not only meets the requirements of "original data does not leave the domain, and data is available but not visible", but also can perform secure data calculation. Since multi-party secure computing is a natural distributed computing scenario, it is necessary to use a distributed scheduling engine to complete the scheduling logic.
[0003] There are currently two types of distributed schedulers on the market. One is based on the traditional distributed framework and uses distributed computing frameworks such as spark, ray, and eggroll to complete cross-node scheduling and implement its own business logic under the framework; the other is to use a self-developed scheduler to complete distributed scheduling tasks. At present, privacy and security computing manufacturers focus more on "multi-party secure computing" itself, and the scheduling strategy focuses on completing cross-node scheduling logic. The scheduling logic is mixed with calls to local algorithm components and cross-node task processing logic. The scheduling logic and business logic are mixed together, which makes the distributed platform unstable when executing tasks.
[0004] From the above, it can be seen that how to improve the stability of distributed platform task execution is a problem to be solved in this field. Summary of the invention
[0005] In view of this, the purpose of the present invention is to provide a distributed task execution method, device, equipment and medium based on alliance to improve the stability of distributed platform task execution. The specific scheme is as follows:
[0006] In a first aspect, the present application discloses a distributed task execution method based on alliance, which is applied to an initiator working node of a distributed platform, including:
[0007] Receive a process form corresponding to the task to be executed issued by the first target alliance, obtain a first process status command based on the process form, and use the first process status command to control the local computing framework to pull up the first algorithm component;
[0008] Sending a process command carrying the process form to the second target alliance, so that the participant working node receives the process form issued by the second target alliance based on the process command, and uses the second process status command obtained based on the process form to pull up the second algorithm component;
[0009] The task to be performed is performed using the first algorithm component and the second algorithm component.
[0010] Optionally, the first process engine in the initiator's working node includes a first process scheduler, a first task scheduler, and a first executor, and the second process engine in the participant's working node includes a second process scheduler, a second task scheduler, and a second executor.
[0011] Optionally, the receiving of a process form corresponding to the task to be executed issued by the first target alliance includes:
[0012] Generate a process form corresponding to the task to be executed through the first target alliance;
[0013] The first process scheduler is used to receive the process form issued by the first target alliance.
[0014] Optionally, the acquiring the first process status command based on the process form includes:
[0015] Parsing the process form using the first process scheduler to generate a first task DSL, and sending the first task DSL to the first task scheduler;
[0016] A first process state command is generated based on the first task DSL using the first task scheduler.
[0017] Optionally, the using the first process state command to control the local computing framework to pull up the first algorithm component includes:
[0018] The first process status command issued by the first task scheduler is received through the first executor, and the first process status command is sent to the local computing framework, so that the local computing framework uses the first process status command to pull up the first algorithm component.
[0019] Optionally, the receiving the process form issued by the second target alliance based on the process command, and starting a second algorithm component by using a second process status command obtained based on the process form, includes:
[0020] receiving, through the second process scheduler, the process form issued by the second target alliance based on the process command, and parsing the process form to generate a second task DSL;
[0021] The second task scheduler is used to generate a second process status command based on the second task DSL, and the second executor is used to send the second process status command issued by the second task scheduler to the computing framework in the participant's working node, so that the computing framework in the participant's working node uses the second process status command to pull up the second algorithm component.
[0022] Optionally, the registration method of the first process engine and the registration method of the second process engine are both plug-in methods.
[0023] In a second aspect, the present application discloses a distributed task execution device based on alliance, which is applied to the initiator working node of the distributed platform, including:
[0024] A first component launching module, configured to receive a process form corresponding to a task to be executed issued by a first target alliance, obtain a first process status command based on the process form, and use the first process status command to control a local computing framework to launch a first algorithm component;
[0025] A second component pulling module is used to send a process command carrying the process form to a second target alliance, so that the participant working node receives the process form issued by the second target alliance based on the process command, and pulls up a second algorithm component using a second process status command obtained based on the process form;
[0026] A task execution module is used to execute the task to be executed by using the first algorithm component and the second algorithm component.
[0027] In a third aspect, the present application discloses an electronic device, comprising:
[0028] Memory, used to store computer programs;
[0029] A processor is used to execute the computer program to implement the steps of the aforementioned alliance-based distributed task execution method disclosed above.
[0030] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned disclosed alliance-based distributed task execution method are implemented.
[0031] The beneficial effects of the present application are as follows: the present application is applied to the initiator work node of the distributed platform, and the method includes: receiving the process form corresponding to the task to be executed issued by the first target alliance, and obtaining the first process status command based on the process form, and using the first process status command to control the local computing framework to pull up the first algorithm component; sending the process command carrying the process form to the second target alliance, so that the participant work node receives the process form issued by the second target alliance based on the process command, and uses the second process status command obtained based on the process form to pull up the second algorithm component; and using the first algorithm component and the second algorithm component to execute the task to be executed. It can be seen that in the present application, the initiator work node and the participant work node receive the process form, that is, the first target alliance and the second target alliance are responsible for sending the process form corresponding to the task to be executed, that is, the scheduling logic is the responsibility of the first target alliance and the second target alliance, and the initiator work node and the participant work node are responsible for completing the corresponding task to be executed based on the process form, that is, the task business logic is the responsibility of the initiator work node and the participant work node, so that the scheduling logic and the business logic are separated, thereby improving the stability of the distributed platform task execution. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0033] Figure 1 A distributed task execution flow chart based on alliance disclosed in this application;
[0034] Figure 2 A specific distributed task execution schematic diagram disclosed in this application;
[0035] Figure 3 This is a schematic diagram of the structure of a distributed task execution device based on alliance disclosed in this application;
[0036] Figure 4 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0038] There are currently two types of distributed schedulers on the market. One is based on the traditional distributed framework and uses distributed computing frameworks such as spark, ray, and eggroll to complete cross-node scheduling and implement its own business logic under the framework; the other is to use a self-developed scheduler to complete distributed scheduling tasks. At present, privacy and security computing manufacturers focus more on "multi-party secure computing" itself, and the scheduling strategy focuses on completing cross-node scheduling logic. The scheduling logic is mixed with calls to local algorithm components and cross-node task processing logic. The scheduling logic and business logic are mixed together, which makes the distributed platform unstable when executing tasks.
[0039] To this end, this application provides a distributed task execution solution based on an alliance to improve the stability of distributed platform task execution.
[0040] See also Figure 1 As shown, the embodiment of the present application discloses a distributed task execution method based on alliance, which is applied to the initiator working node of the distributed platform, including:
[0041] Step S11: receiving a process form corresponding to the task to be executed issued by the first target alliance, and obtaining a first process status command based on the process form, and using the first process status command to control the local computing framework to pull up the first algorithm component.
[0042] In this embodiment, the first process engine in the initiator working node includes a first process scheduler, a first task scheduler, and a first executor. It can be understood that the distributed platform includes multiple working nodes, and the process engine in each working node includes a process scheduler, a task scheduler, and an executor.
[0043] In this embodiment, the receiving of the process form corresponding to the task to be executed issued by the first target alliance includes: generating a process form corresponding to the task to be executed through the first target alliance; and receiving the process form issued by the first target alliance using the first process scheduler.
[0044] The platform needs to support better resource sharing, support the integration of multi-supplier and multi-party secure computing frameworks, support multi-role participation, and at the same time hope to improve the stability of the process scheduling engine and maintain its good scalability.
[0045] To achieve the above goals, we first need to separate the process scheduling logic from the business logic, so that the process scheduling engine can focus on scheduling itself. In order to further improve the stability of the scheduling engine, the logic of the scheduling engine is concentrated on local scheduling, without involving any cross-node scheduling logic. In this way, the scheduling engine does not need to solve distributed transaction problems, nor does it need to generate performance problems in network communication problems. At the same time, it isolates the impact of remote uncertainty on the scheduling engine.
[0046] Furthermore, since the cross-node scheduling logic and business logic are to be extracted from the scheduling engine, a module is needed to undertake the corresponding functions. In this embodiment, the business logic that does not involve cross-node calls is placed in the upper management plane, and the logic involved in scheduling, such as cross-node data retrieval, cross-node audit notification, cross-node process scheduling, etc., is centralized and placed in a new service, which is named the alliance. All cross-node call logic is centrally managed in the alliance, which can make the logic more centralized and facilitate the enhancement of the stability of cross-node scheduling. After separating the local process scheduling logic, the alliance logic focuses on parameter management and signaling distribution, and the logic becomes more focused and greatly simplified. In other words, each working node has its own corresponding alliance. When the same task to be executed requires multiple working nodes to complete together, in this embodiment, the alliance is responsible for the scheduling logic, that is, the alliance is responsible for sending the process form corresponding to the task to be executed to the corresponding working node, so that the process scheduler of the working node can receive the process form issued by the corresponding alliance.
[0047] In this embodiment, the acquisition of the first process status command based on the process form includes: using the first process scheduler to parse the process form to generate a first task DSL (Domain Specific Language), and sending the first task DSL to the first task scheduler; using the first task scheduler to generate a first process status command based on the first task DSL. The first process scheduler can call the process engine component plug-in to parse the process form to generate the first task DSL, and then send the first task DSL to the first task scheduler. The first task scheduler parses the first task DSL and assembles a PSI (Process Status Information) command to generate the first process status command.
[0048] In this embodiment, the use of the first process status command to control the local computing framework to pull up the first algorithm component includes: receiving the first process status command issued by the first task scheduler through the first executor, and sending the first process status command to the local computing framework, so that the local computing framework uses the first process status command to pull up the first algorithm component. The first task scheduler sends the first process status command to the first executor, that is, the first execution component. When the first executor receives the first process status command, it sends the first process status command to the local computing framework, and the local computing framework can pull up the first algorithm component, that is, pull up the PSI program.
[0049] Step S12: Send the process command carrying the process form to the second target alliance, so that the participant working node receives the process form issued by the second target alliance based on the process command, and uses the second process status command obtained based on the process form to pull up the second algorithm component.
[0050] It should be noted that the first target alliance of the initiator's working node will send the process command carrying the process form to the alliance of the participating working nodes, namely the second target alliance. In other words, each working node has a module specifically responsible for scheduling logic, namely the alliance of each working node, and the process engine of the working node does not need to be responsible for scheduling logic, but only needs to focus on completing the tasks to be executed based on the process form, reducing the workload of each process engine so that it can complete the tasks to be executed more stably.
[0051] In this embodiment, the second process engine in the participant's working node includes a second process scheduler, a second task scheduler, and a second executor. Similarly, the second process engine in the participant's working node also includes a second process scheduler, a second task scheduler, and a second executor.
[0052] In this embodiment, the receiving of the process form issued by the second target alliance based on the process command, and the use of the second process status command obtained based on the process form to pull up the second algorithm component, includes: receiving the process form issued by the second target alliance based on the process command through the second process scheduler, and parsing the process form to generate a second task DSL; using the second task scheduler to generate a second process status command based on the second task DSL, and using the second executor to send the second process status command issued by the second task scheduler to the computing framework in the participant's working node, so that the computing framework in the participant's working node uses the second process status command to pull up the second algorithm component.
[0053] The second target alliance sends the process form in the process command to the second process scheduler of the participant's working node. The second process scheduler parses the process form to determine the PSI to be executed, and calls the process engine component plug-in to generate the second task DSL, and then sends the second task DSL to the second task scheduler; after receiving the second task DSL, the second task scheduler parses it to obtain the second process status command, and then sends the second process status command to the second executor, that is, the second execution component; the second executor sends the second process status command to the computing framework in the participant's working node, and the computing framework in the participant's working node uses the second process status command to pull up the second algorithm component. It can be seen that the task scheduler does not submit tasks directly to the computing framework, but submits them to the execution component, which is sent to the computing framework by the execution component, ensuring the independence of the task scheduler. On the one hand, it can flexibly adapt to more computing frameworks, and on the other hand, when adapting the computing framework, there is no need to modify the scheduler code, which ensures the stability of the engine.
[0054] Step S13: Utilize the first algorithm component and the second algorithm component to execute the task to be executed.
[0055] In this embodiment, the registration method of the first process engine and the registration method of the second process engine are both plug-in methods. The first algorithm component of the initiator's working node and the second algorithm component of the participant's working node cooperate with each other to complete the task to be executed. It should be noted that the registration method of the first process engine and the registration method of the second process engine are both plug-in methods. In this way, the various components in the first process engine and the second process engine can be flexibly expanded. In other words, when a new algorithm component needs to be expanded, it is only necessary to implement the registration of a new DSL plug-in to the process engine. The task scheduler parses the DSL to generate task commands and also adopts a plug-in method, so that the algorithm component can be flexibly expanded, and when the algorithm component is expanded, there is no need to modify the engine code to ensure the stability of the engine. In the future, dynamic registration of plug-ins will be realized, and the registration interface will be opened to achieve dynamic updates.
[0056] The beneficial effects of the present application are as follows: the present application is applied to the initiator work node of the distributed platform, and the method includes: receiving the process form corresponding to the task to be executed issued by the first target alliance, and obtaining the first process status command based on the process form, and using the first process status command to control the local computing framework to pull up the first algorithm component; sending the process command carrying the process form to the second target alliance, so that the participant work node receives the process form issued by the second target alliance based on the process command, and uses the second process status command obtained based on the process form to pull up the second algorithm component; and using the first algorithm component and the second algorithm component to execute the task to be executed. It can be seen that in the present application, the initiator work node and the participant work node receive the process form, that is, the first target alliance and the second target alliance are responsible for sending the process form corresponding to the task to be executed, that is, the scheduling logic is the responsibility of the first target alliance and the second target alliance, and the initiator work node and the participant work node are responsible for completing the corresponding task to be executed based on the process form, that is, the task business logic is the responsibility of the initiator work node and the participant work node, so that the scheduling logic and the business logic are separated, thereby improving the stability of the distributed platform task execution.
[0057] Below Figure 2 Taking a specific distributed task execution diagram as an example, the present application is described accordingly. The cross-node task scheduling, resource sharing and collaboration logic are extracted and implemented in the alliance, so that the engine no longer pays attention to the cross-node information, the logic is more focused, and the stability and performance are improved. The alliance completes cross-node task scheduling, resource sharing and collaboration through a custom alliance protocol to realize the concept of alliance. A new alliance is created each time the collaboration is carried out. The alliance includes member nodes, alliance users, alliance data, and alliance tasks. The alliance manages and authenticates these resources in a unified manner. In the past, when cross-node task distribution was required, the engine needed to query the other party's data set according to the current task, and then query the other party's node information to obtain the other party's IP (Internet Protocol) and port, and then call the other party's interface to complete the task distribution. After the alliance, when the engine needs to distribute tasks, it only needs to call the alliance interface, provide the alliance id (Identity document, that is, identity identification number) and task parameters. The subsequent confirmation and call distribution by the participating parties are calculated and determined by the alliance according to the task configuration, and the distribution is completed. The implementation method of the engine supports mixed scheduling capabilities and can perform task scheduling across computing frameworks.
[0058] Determine work node A as the initiator work node, and determine work node B as the participant work node. The alliance of work node A generates a process form for the task to be executed, sends the process form to the process scheduler of the process engine in work node A, and sends the process command carrying the process form to the alliance of work node B, so that the alliance of work node B can send the process form carried by the process command to the process scheduler of the process engine in work node B.
[0059] Furthermore, the working nodes of both parties can synchronously or asynchronously pull up the corresponding algorithm components based on the process form. Specifically, the process schedulers of both parties receive process parameters and are responsible for process parsing, DSL generation, task generation, and process status monitoring. The task scheduler is responsible for DSL parsing, task submission, and task status monitoring. The execution component is responsible for submitting the commands from the task scheduler to the corresponding computing framework.
[0060] It can be seen that the implementation logic of the process engine of this application has become simpler. It only needs to consider the logical implementation of its own process and task scheduling. When other nodes need to cooperate, it only needs to send an application to the alliance, and there is no need to pay attention to who does it and how to do it. And it ensures the convenience of algorithm component expansion. When the algorithm component needs to be expanded, it only needs to implement the relevant plug-in code. At the same time, it ensures the convenience of expanding the computing framework. When it is necessary to support a new computing framework, it only needs to add the call logic of the new computing framework to the algorithm component. The whole process does not need to modify the process engine code. Finally, due to the high concentration of logic, it is easy to add process and task monitoring logic to ensure the controllability of the process and task execution process. The alliance centrally manages all cross-node resources and interaction logic, focusing on processing resource management and signaling transmission, and can more conveniently strengthen cross-node calls, such as unified distributed transactions. When the engine applies to the alliance for cross-node scheduling, the alliance obtains node information through its own resource information, performs parameter packaging, and then calls the signaling transmission related interface for cross-node calls, thereby realizing cross-node business process scheduling. When the signal received by the other node alliance needs to rely on the computing framework to perform specific operations, the alliance does not need to interact with the computing framework, but provides a unified command interface through the process engine to issue tasks, and the process engine handles the tasks uniformly. In this way, cross-node processes and tasks are still scheduled by the process engine, but the process engine does not focus on cross-node logic processing, which is more focused, simpler, and more stable. The alliance only manages signal transmission, has no state, and the logic encapsulation is simpler. Each performs its own duties, is fully decoupled, and uses simple logic combinations to implement complex cross-node call logic.
[0061] See also Figure 3As shown, the embodiment of the present application discloses a distributed task execution device based on alliance, which is applied to the initiator working node of the distributed platform, including:
[0062] The first component launching module 11 is used to receive a process form corresponding to the task to be executed issued by the first target alliance, obtain a first process status command based on the process form, and use the first process status command to control the local computing framework to launch the first algorithm component;
[0063] The second component pulling module 12 is used to send the process command carrying the process form to the second target alliance, so that the participant working node receives the process form issued by the second target alliance based on the process command, and pulls up the second algorithm component using the second process status command obtained based on the process form;
[0064] The task execution module 13 is used to execute the task to be executed by using the first algorithm component and the second algorithm component.
[0065] The beneficial effects of the present application are as follows: the present application is applied to the initiator work node of the distributed platform, and the method includes: receiving the process form corresponding to the task to be executed issued by the first target alliance, and obtaining the first process status command based on the process form, and using the first process status command to control the local computing framework to pull up the first algorithm component; sending the process command carrying the process form to the second target alliance, so that the participant work node receives the process form issued by the second target alliance based on the process command, and uses the second process status command obtained based on the process form to pull up the second algorithm component; and using the first algorithm component and the second algorithm component to execute the task to be executed. It can be seen that in the present application, the initiator work node and the participant work node receive the process form, that is, the first target alliance and the second target alliance are responsible for sending the process form corresponding to the task to be executed, that is, the scheduling logic is the responsibility of the first target alliance and the second target alliance, and the initiator work node and the participant work node are responsible for completing the corresponding task to be executed based on the process form, that is, the task business logic is the responsibility of the initiator work node and the participant work node, so that the scheduling logic and the business logic are separated, thereby improving the stability of the distributed platform task execution.
[0066] Furthermore, an embodiment of the present application also provides an electronic device. Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.
[0067] Figure 4A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the following steps:
[0068] Receive a process form corresponding to the task to be executed issued by the first target alliance, obtain a first process status command based on the process form, and use the first process status command to control the local computing framework to pull up the first algorithm component;
[0069] Sending a process command carrying the process form to the second target alliance, so that the participant working node receives the process form issued by the second target alliance based on the process command, and uses the second process status command obtained based on the process form to pull up the second algorithm component;
[0070] The task to be performed is performed using the first algorithm component and the second algorithm component.
[0071] In some specific implementations, the first process engine in the initiator's working node includes a first process scheduler, a first task scheduler, and a first executor, and the second process engine in the participant's working node includes a second process scheduler, a second task scheduler, and a second executor.
[0072] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:
[0073] Generate a process form corresponding to the task to be executed through the first target alliance;
[0074] The first process scheduler is used to receive the process form issued by the first target alliance.
[0075] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:
[0076] Parsing the process form using the first process scheduler to generate a first task DSL, and sending the first task DSL to the first task scheduler;
[0077] A first process state command is generated based on the first task DSL using the first task scheduler.
[0078] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:
[0079] The first process status command issued by the first task scheduler is received through the first executor, and the first process status command is sent to the local computing framework, so that the local computing framework uses the first process status command to pull up the first algorithm component.
[0080] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:
[0081] receiving, through the second process scheduler, the process form issued by the second target alliance based on the process command, and parsing the process form to generate a second task DSL;
[0082] The second task scheduler is used to generate a second process status command based on the second task DSL, and the second executor is used to send the second process status command issued by the second task scheduler to the computing framework in the participant's working node, so that the computing framework in the participant's working node uses the second process status command to pull up the second algorithm component.
[0083] In some specific implementations, the registration method of the first process engine and the registration method of the second process engine are both plug-in methods.
[0084] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0085] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0086] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.
[0087] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows, Unix, Linux, etc. In addition to including computer programs that can be used to complete the alliance-based distributed task execution method performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 222 can also further include computer programs that can be used to complete other specific tasks. In addition to data transmitted from an external device received by the electronic device, the data 223 can also include data collected by its own input and output interface 25.
[0088] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned alliance-based distributed task execution method is implemented. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiment, and will not be repeated here.
[0089] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0090] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly with hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable EPROM (Erasable Programmable Read Only Memory), an electrically erasable programmable EEPROM (Electrically Erasable Programmable read only memory), a register, a hard disk, a removable disk, a CD-ROM (CoMP23038002act Disc Read-Only Memory), or any other form of storage medium known in the technical field.
[0091] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0092] The above is a detailed introduction to the alliance-based distributed task execution method, device, equipment and medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A distributed task execution method based on alliance, characterized in that: The initiator working node applied to the distributed platform includes: Receive a process form corresponding to the task to be executed issued by the first target alliance, obtain a first process status command based on the process form, and use the first process status command to control the local computing framework to pull up the first algorithm component; Sending a process command carrying the process form to the second target alliance, so that the participant working node receives the process form issued by the second target alliance based on the process command, and uses the second process status command obtained based on the process form to pull up the second algorithm component; The task to be performed is performed using the first algorithm component and the second algorithm component.
2. The alliance-based distributed task execution method according to claim 1, characterized in that: The first process engine in the initiator's working node includes a first process scheduler, a first task scheduler, and a first executor, and the second process engine in the participant's working node includes a second process scheduler, a second task scheduler, and a second executor.
3. The alliance-based distributed task execution method according to claim 2, characterized in that: The receiving of the process form corresponding to the task to be executed issued by the first target alliance includes: Generate a process form corresponding to the task to be executed through the first target alliance; The first process scheduler is used to receive the process form issued by the first target alliance.
4. The alliance-based distributed task execution method according to claim 3, characterized in that: The acquiring the first process status command based on the process form includes: Parsing the process form using the first process scheduler to generate a first task DSL, and sending the first task DSL to the first task scheduler; A first process state command is generated based on the first task DSL using the first task scheduler.
5. The alliance-based distributed task execution method according to any one of claims 2 to 4, characterized in that: The using the first process state command to control the local computing framework to pull up the first algorithm component includes: The first process status command issued by the first task scheduler is received through the first executor, and the first process status command is sent to the local computing framework, so that the local computing framework uses the first process status command to pull up the first algorithm component.
6. The alliance-based distributed task execution method according to claim 2, characterized in that: The receiving the process form issued by the second target alliance based on the process command, and using the second process status command obtained based on the process form to pull up the second algorithm component, includes: receiving, through the second process scheduler, the process form issued by the second target alliance based on the process command, and parsing the process form to generate a second task DSL; The second task scheduler is used to generate a second process status command based on the second task DSL, and the second executor is used to send the second process status command issued by the second task scheduler to the computing framework in the participant's working node, so that the computing framework in the participant's working node uses the second process status command to pull up the second algorithm component.
7. The alliance-based distributed task execution method according to claim 2, characterized in that: The registration method of the first process engine and the registration method of the second process engine are both plug-in methods.
8. A distributed task execution device based on alliance, characterized in that: The initiator working node applied to the distributed platform includes: A first component launching module, configured to receive a process form corresponding to a task to be executed issued by a first target alliance, obtain a first process status command based on the process form, and use the first process status command to control a local computing framework to launch a first algorithm component; A second component pulling module is used to send a process command carrying the process form to a second target alliance, so that the participant working node receives the process form issued by the second target alliance based on the process command, and pulls up a second algorithm component using a second process status command obtained based on the process form; A task execution module is used to execute the task to be executed by using the first algorithm component and the second algorithm component.
9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the alliance-based distributed task execution method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the alliance-based distributed task execution method as described in any one of claims 1 to 7 are implemented.