A method, device, equipment and medium for cross-cloud resource operation based on task orchestration

Abstract

The present invention provides a method, apparatus, device and medium for cross-cloud resource operation based on task orchestration. The method includes: abstracting the received scheduling task request into task orchestration logic, and according to the task orchestration logic, calling the service interfaces of corresponding cloud providers to execute all operations; the task orchestration logic includes the order of main tasks, subtasks and subtask actions; recording all execution situations and recording them in a set log file, which can not only flexibly combine and schedule multi-cloud resources through task orchestration, but also ensure the traceability of all resource operations through a unified log tracking system, and quickly locate and solve problems when they occur.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-cloud operations, and particularly to a method, device, equipment, and medium for cross-cloud resource operations based on task orchestration. Background Art

[0002] With the development of cloud computing technology, enterprises widely adopt resources provided by different cloud service providers (such as AWS, Azure, Alibaba Cloud, etc.). Due to the different APIs and management processes of different cloud providers, cross-platform cloud resource management has become complex and time-consuming, and the APIs of each cloud provider are scattered and cannot be combined for use.

[0003] Existing technologies all perform targeted scheduling processing for a certain cloud provider, lacking an effective task orchestration mechanism to uniformly schedule cross-cloud resource operations, and lacking a unified log tracking system for operation traceability and problem location. This results in low efficiency and high operation and maintenance costs when performing resource operations in a multi-cloud environment. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, device, equipment, and medium for cross-cloud resource operations based on task orchestration, which greatly improves work efficiency through unified scheduling processing.

[0005] In a first aspect, the present invention provides a method for cross-cloud resource operations based on task orchestration, including the following steps:

[0006] Step 1: Abstract the received scheduling task request into task orchestration logic, and according to the task orchestration logic, call the service interface of the corresponding cloud provider to execute all operations; the task orchestration logic includes the order of main tasks, subtasks, and subtask actions;

[0007] Step 2: Record all execution situations and record them in a set log file.

[0008] In a second aspect, the present invention provides a device for cross-cloud resource operations based on task orchestration, including the following modules:

[0009] An execution operation module that abstracts the received scheduling task request into task orchestration logic, and according to the task orchestration logic, calls the service interface of the corresponding cloud provider to execute all operations; the task orchestration logic includes the order of main tasks, subtasks, and subtask actions;

[0010] A log recording module that records all execution situations and records them in a set log file.

[0011] In a third aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the first aspect is implemented.

[0012] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect is implemented.

[0013] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0014] Business process consistency: By implementing a unified scheduling mechanism across multiple cloud platforms, it is ensured that business processes executed in different cloud environments follow the same logic. This consistency enables developers to write the scheduling logic once and apply it to multiple cloud services, reducing the learning curve between different platforms.

[0015] Reduced operational complexity: Reduces the dependence on specific cloud services, making it easier to migrate business between different cloud platforms.

[0016] Transparency and traceability: The unified logging system ensures comprehensive monitoring of all task execution situations. Detailed logs of each operation can be obtained, enhancing the transparency of operations.

[0017] Improved analysis efficiency: Through detailed operation logs, troubleshooting becomes more efficient. Development and testing personnel can quickly locate the root cause of problems, analyze and optimize bottlenecks, ensuring business continuity and stability.

[0018] The above description is only an overview of the technical solution of the present invention. In order to be able to more clearly understand the technical means of the present invention, it can be implemented in accordance with the content of the description. And in order to make the above and other purposes, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are hereinafter specifically exemplified. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will be further described below with reference to the accompanying drawings in conjunction with embodiments.

[0020] Figure 1 It is a flowchart of the method in Embodiment 1 of the present invention.

[0021] Figure 2 It is a schematic structural diagram of the device in Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] Embodiments of the present application provide a method, apparatus, device, and medium for cross-cloud resource operations based on task orchestration, which can not only flexibly combine and schedule multi-cloud resources through task orchestration, but also ensure the traceability of all resource operations through a unified log tracking system, and quickly locate and solve problems when they occur.

[0023] The overall idea of the technical solution in the embodiments of the present application is as follows:

[0024] 1. Cloud resource operation process driven by task orchestration

[0025] The present invention abstracts the operations in business services into task orchestration logic and manages them through a scheduling service. The task orchestration logic includes the definition of main tasks, subtasks, and the order of each task action. When a business service needs to perform resource operations, the scheduling service calls the service interfaces of each cloud provider in sequence according to the predefined task orchestration logic. Decoupling from different cloud vendors is achieved through the adapter pattern to ensure that resource operations of different cloud vendors can be executed according to a unified process.

[0026] 2. Unified cloud resource operation log tracking

[0027] The present invention provides a unified log tracking system to record the execution status of each task, including the context information of the task and the vendor call log. In the log system, not only can the execution status of each main task be viewed, but also alarms can be set according to the log level. When the system operation fails, alarm mechanisms such as DingTalk are automatically triggered to improve the operation and maintenance efficiency and system reliability.

[0028] The implementation process is as follows:

[0029] 1. Define task orchestration logic in the scheduling service:

[0030] Define task orchestration logic in the system, including:

[0031] Main task: such as applying for an agent;

[0032] Subtask: Each subtask corresponds to the specific steps of the main task, such as creating an IP, creating an instance, creating a network card, binding an IP, deploying an agent, etc.;

[0033] Action order: Define the execution order of each subtask to ensure that all necessary operations are completed in sequence;

[0034] 2. Business service calls the scheduling service

[0035] The business service specifies the vendor and main task to be executed and calls the scheduling service. For example, select to execute the agent application task of Alibaba Cloud.

[0036] 3. The scheduling service receives and processes

[0037] Dispatch service receives: After receiving a new dispatch task request, the dispatch service creates a main task according to the configuration, extracts and stores the request parameters in Redis.

[0038] 3.1 Main task distribution (dispatch_main): During the main task management process, the dispatch service queries the main task in the initial state and generates and adds initial subtasks according to the defined task relationships;

[0039] 3.2 Subtask distribution (dispatch_item): During the subtask execution process, the dispatch service first checks whether the main task has an error. If there is an error, it cancels the execution of subsequent subtasks. If the concurrency threshold is reached, the subtask is delayed for execution. According to the subtask configuration, obtain the vendor adaptation service and actions to be sent, request parameters, asynchronously send the request, and update the subtask status to waiting for callback;

[0040] 4. Vendor adaptation service receives and processes

[0041] 4.1 First, according to the request parameters, which are the parameters set in 3.2, obtain the manufacturer and actions to be executed, find the adapter. The adapter is used to implement the interface calls of each vendor, complete the actual actions, then obtain the parameters from redis, execute the action call of the vendor. In the action call, sequentially request the interfaces of external cloud manufacturers according to the specific logic, and this logic is the execution steps of the specific actions of each subtask; then package the action return result and send it back to the dispatch service through a message;

[0042] 4.2 The system records the call information of external cloud manufacturers through data logging, generates sub-operation logs (the called method, status, start time, end time, execution times, parameters, return values, whether there is an exception), and records the operation results;

[0043] 5. Dispatch service receives and processes the callback

[0044] In the callback processing of the dispatch service manager, when receiving the callback message, at this time, perform a concurrent lock according to the main task ID to ensure the concurrency safety of the task, and ensure that during the operation of subtasks under the same main task, only one subtask is allowed to perform callback processing at the same time. If the lock operation fails, a message with a 10-second delay will be sent to retry the task later. If successful, first update the result to the Redis cache to provide necessary input parameter information for subsequent tasks. The callback processing is to determine whether to execute the next subtask when a subtask is completed;

[0045] Then first judge the execution situation of the subtask:

[0046] 5.1 If the subtask fails and the retry count has been reached, mark the main task as having an error, update the status of the subtask to error, and finally upload the main operation log (including: link ID, error reason (which subtask's exception caused the failure), exception information, occurrence time) to end the process;

[0047] 5.2 If the subtask fails but the retry count has not been reached, update the status of the subtask to in progress, and the system will re - execute step 3.2, that is, call the subtask again and continue to try;

[0048] 5.3 When the subtask is successful, the system will check whether all the previous tasks of the next step have been successfully executed. If the previous tasks are not completed, update the status of the subtask to successful and return directly; if all the previous tasks are completed, the system will add a new next subtask in the initial state and continue to execute step 3.2;

[0049] 5.4 If there are no subsequent subtasks to execute, mark the main task as completed and update the final status of the main task to successful to complete the entire process;

[0050] 6. Through the overall operation log, the root cause of the problem can be quickly traced to achieve efficient troubleshooting.

[0051] Example 1

[0052] As Figure 1 shown, this example provides a method for cross - cloud resource operation based on task orchestration, including the following steps:

[0053] Step 1: Abstract the received scheduling task request into task orchestration logic, and according to the task orchestration logic, call the service interface of the corresponding cloud provider to execute all operations; the task orchestration logic includes the order of the main task, subtasks, and subtask actions;

[0054] Step 2: Record all the execution situations and record them in a set log file.

[0055] In this example, preferably, step 1 is specifically as follows:

[0056] Step 11: According to the received scheduling task request, abstract the scheduling task request into task orchestration logic; the task orchestration logic includes the order of the main task, subtasks, and subtask actions;

[0057] Step 12: Generate and add initial subtasks according to the main task; set corresponding task configurations for each subtask, and the task configuration includes: the vendor - adapted service and action to be sent, request parameters, request sending method, and update the status of the subtask to waiting for callback;

[0058] Step 13: Execute the corresponding subtasks in sequence according to the execution order of the subtasks; obtain the actions corresponding to the execution of this task according to the task configuration, make action calls through the adapter of this cloud provider according to the request parameters, obtain the return results, and record the call information of the cloud provider in the sub-operation log;

[0059] Step 14: If the current subtask is executed successfully, proceed to the execution of the next subtask until all subtasks are executed, and record the execution status in the main task operation log. If the current subtask fails, end the execution, update the status of this subtask to error, mark the main task as having an error, and record the execution status in the main task operation log.

[0060] In this embodiment, preferably, Step 14 is specifically as follows: when the main task is being executed, only one subtask is allowed to perform callback processing at the same time; if the current subtask is executed successfully, perform a locking operation according to the main task ID. If the locking operation fails, delay for the first waiting time and then perform callback processing. If the locking is successful, perform callback processing; after completing the callback processing, perform an unlocking operation, and then proceed to the execution of the next subtask;

[0061] If the current subtask is executed successfully, proceed to the execution of the next subtask until all subtasks are executed, and record the execution status in the main task operation log; if the current subtask fails, re-execute the subtask after a set time. Only when the same subtask fails the set number of times, end the execution, update the status of this subtask to error, mark the main task as having an error, and record the execution status in the main task operation log.

[0062] In this embodiment, preferably, Step 2 is specifically as follows: record all execution statuses, including the context information of the task and the calls to the provider; record them in the set log files. The log files include: the main operation log and the sub-operation log. The main operation log includes: the link ID, the reason for the error, the exception information, and the occurrence time; the sub-operation log includes: the called method, the status, the start time, the end time, the execution times, the parameters, the return value, and whether there is an exception; when the main task or the sub-task operation fails, trigger the alarm mechanism according to the setting.

[0063] Based on the same inventive concept, the present application also provides a device corresponding to the method in Embodiment 1. For details, see Embodiment 2.

[0064] Embodiment 2

[0065] As Figure 2 shown, in this embodiment, a device for cross-cloud resource operation based on task orchestration is provided, including the following modules:

[0066] An operation execution module that abstracts the received scheduling task request into a task orchestration logic, and according to the task orchestration logic, calls the service interface of the corresponding cloud provider to execute all operations; the task orchestration logic includes the sequence of main tasks, subtasks, and subtask actions.

[0067] A log recording module that records all execution situations and records them in a set log file.

[0068] In this embodiment, preferably, the operation execution module is specifically:

[0069] A task orchestration unit that abstracts the received scheduling task request into a task orchestration logic according to the received scheduling task request; the task orchestration logic includes the sequence of main tasks, subtasks, and subtask actions.

[0070] A subtask setting unit that generates and adds initial subtasks according to the main task; each subtask is set with corresponding task configurations, and the task configurations include: the vendor adaptation service and actions to be sent, request parameters, the request sending method, and updates the subtask status to waiting for callback.

[0071] An execution recording unit that sequentially executes the corresponding subtasks according to the execution order of the subtasks; obtains the actions corresponding to the execution of the task according to the task configuration, calls the actions through the adapter of the cloud vendor according to the request parameters, obtains the return result, and records the call information of the cloud vendor in the sub-operation log.

[0072] A final execution unit that, if the current subtask is executed successfully, proceeds to execute the next subtask until all subtasks are executed, and records the execution situation in the main task operation log; if the current subtask is executed failed, ends the execution, updates the status of the subtask to error, marks the main task as in error, and records the execution situation in the main task operation log.

[0073] In this embodiment, preferably, the final execution unit is specifically: when the main task is being executed, only one subtask is allowed to perform callback processing at the same time; if the current subtask is executed successfully, a locking operation is performed according to the main task ID. If the locking operation fails, the first waiting time is delayed and then the callback processing is performed. If the locking is successful, the callback processing is performed; after the callback processing is completed, an unlocking operation is performed, and then the next subtask is executed.

[0074] If the current subtask is executed successfully, the next subtask is executed until all subtasks are executed, and the execution situation is recorded in the main task operation log; if the current subtask is executed failed, the subtask is re-executed after a set time. Only when the same subtask fails a set number of times, the execution ends, the status of the subtask is updated to error, the main task is marked as in error, and the execution situation is recorded in the main task operation log.

[0075] In this embodiment, preferably, the log recording module is specifically configured to: record all execution situations, including the context information of tasks and the calls by manufacturers; record them into a set log file, where the log file includes: a main operation log and a sub-operation log. The main operation log includes: a link ID, a reason for error, exception information, and an occurrence time; the sub-operation log includes: a called method, a status, a start time, an end time, an execution count, parameters, a return value, and whether there is an exception; when the main task or the sub-task operation fails, an alarm mechanism is triggered according to the setting.

[0076] Since the device introduced in the second embodiment of the present invention is the device adopted for implementing the method in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, those skilled in the art can understand the specific structure and variations of the device, so it will not be elaborated here. Any device adopted for the method in the first embodiment of the present invention belongs to the scope protected by the present invention.

[0077] Based on the same inventive concept, this application provides an electronic device embodiment corresponding to the first embodiment. For details, see the third embodiment.

[0078] The third embodiment

[0079] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, any implementation manner in the first embodiment can be realized.

[0080] Since the electronic device introduced in this embodiment is the device adopted for implementing the method in the first embodiment of this application, based on the method introduced in the first embodiment of this application, those skilled in the art can understand the specific implementation manners and various variations of the electronic device in this embodiment. Therefore, how the electronic device realizes the method in the embodiments of this application will not be described in detail here. Any device adopted by those skilled in the art for implementing the method in the embodiments of this application belongs to the scope protected by this application.

[0081] Based on the same inventive concept, this application provides a storage medium corresponding to the first embodiment. For details, see the fourth embodiment.

[0082] The fourth embodiment

[0083] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, any implementation manner in the first embodiment can be realized.

[0084] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0085] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0086] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means realizes the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0088] Although the specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments we described are illustrative rather than used to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered by the scope of the claims of the present invention.

Claims

1. A method for cross-cloud resource operation based on task orchestration, characterized in that: The steps include: Step 1: abstract the received scheduling task request into task scheduling logic, call the service interface of the corresponding cloud provider according to the task scheduling logic, and execute all operations; the task scheduling logic includes the main task, subtasks and the order of subtask actions; the specific step 1 is: Step 11: abstract the received scheduling task request into task scheduling logic; the task scheduling logic includes a main task, subtasks, and the order of subtask actions; Step 12: Generate and add initial subtasks based on the main task; Each subtask sets a corresponding task configuration, which includes: the supplier adaptation service and action to be sent, request parameters, the sending request method, and updates the subtask status to wait for callback; Step 13: Execute the corresponding subtasks in sequence according to the order in which the subtasks are executed; obtain the action corresponding to the task according to the task configuration, call the action through the cloud vendor's adapter according to the request parameters, obtain the return result, and record the cloud vendor's call information in the sub-operation log; Step 14, when the main task is being executed, only one subtask is allowed to perform callback processing at the same time; if the current subtask is successfully executed, the locking operation is performed according to the main task ID; if the locking operation fails, the first set time is delayed and the callback processing is performed again; if the locking is successful, the callback processing is performed; after the callback processing is completed, the unlocking operation is performed, and then the next subtask is executed; If the current subtask is successfully executed, the next subtask will be executed until all subtasks are completed, and the execution status will be recorded in the main task operation log; if the current subtask fails, the subtask will be re-executed after the set time. Only when the same subtask fails the set number of times, the execution will be terminated, the subtask status will be updated to error, and the main task will be marked as an error, and the execution status will be recorded in the main task operation log; Step 2: Record all execution status in the set log file.

2. The method for cross-cloud resource operation based on task orchestration according to claim 1, characterized in that: The step 2 specifically includes: recording all execution conditions, including task context information and manufacturer calls; Record in the set log file, the log file includes: main operation log and sub-operation log, the main operation log includes: link ID, error cause, exception information and occurrence time; the sub-operation log includes: calling method, status, start time, end time, number of executions, parameters, return value and whether it is abnormal; when the main task or sub-task operation fails, the alarm mechanism is triggered according to the setting.

3. A device for cross-cloud resource operation based on task orchestration, characterized in that: Includes the following modules: The execution operation module abstracts the received scheduling task request into task scheduling logic, calls the service interface of the corresponding cloud provider according to the task scheduling logic, and executes all operations; the task scheduling logic includes the main task, subtasks and the order of subtask actions; the execution operation module is specifically: The task scheduling unit abstracts the scheduling task request into task scheduling logic according to the received scheduling task request; the task scheduling logic includes the main task, subtasks and the order of subtask actions; A subtask setting unit generates and adds initial subtasks according to the main task; Each subtask sets a corresponding task configuration, which includes: the supplier adaptation service and action to be sent, request parameters, the sending request method, and updates the subtask status to wait for callback; The execution recording unit executes the corresponding subtasks in sequence according to the order in which the subtasks are executed; obtains the action corresponding to the task according to the task configuration, calls the action through the cloud vendor's adapter according to the request parameters, obtains the return result, and records the cloud vendor's call information in the sub-operation log; The final execution unit, when the main task is being executed, only one subtask is allowed to perform callback processing at the same time; if the current subtask is successfully executed, the locking operation is performed according to the main task ID; if the locking operation fails, the first set time is delayed and the callback processing is performed again; if the locking is successful, the callback processing is performed; after the callback processing is completed, the unlocking operation is performed, and then the next subtask is executed; If the current subtask is successfully executed, the next subtask will be executed until all subtasks are completed, and the execution status will be recorded in the main task operation log; if the current subtask fails, the subtask will be re-executed after the set time. Only when the same subtask fails the set number of times, the execution will be terminated, the subtask status will be updated to error, and the main task will be marked as an error, and the execution status will be recorded in the main task operation log; The logging module records all execution conditions in the set log file.

4. The device for cross-cloud resource operation based on task orchestration according to claim 3, characterized in that: The logging module specifically records all execution conditions, including task context information and vendor calls; Record in the set log file, the log file includes: main operation log and sub-operation log, the main operation log includes: link ID, error cause, exception information and occurrence time; the sub-operation log includes: calling method, status, start time, end time, number of executions, parameters, return value and whether it is abnormal; when the main task or sub-task operation fails, the alarm mechanism is triggered according to the setting.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 and 2 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 and 2 is implemented.

Images