Cross-cloud resource operation method, device and equipment based on task orchestration and medium

Abstract

The invention provides a cross-cloud resource operation method, device and equipment based on task orchestration and a medium, and the method comprises the steps: abstracting a received scheduling task request into task orchestration logic, calling a service interface of a corresponding cloud supplier according to the task orchestration logic, and executing all operations; the task arrangement logic comprises a main task, a subtask and a sequence of subtask actions; according to the method, all execution conditions are recorded and recorded into a set log file, multi-cloud resources can be flexibly combined and scheduled in a task arrangement mode, the traceability of all resource operations can be ensured through a unified log tracking system, and quick positioning and solution can be performed when a problem occurs.

Description

Technical Field

[0001] The present invention relates to the field of cross-cloud operation technology, and in particular to a method, device, equipment and medium for cross-cloud resource operation based on task orchestration. Background Art

[0002] With the development of cloud computing technology, enterprises widely use resources provided by different cloud service providers (such as AWS, Azure, Alibaba Cloud, etc.). Due to the different APIs and management processes of different cloud vendors, cross-platform cloud resource management becomes complicated and time-consuming, and the APIs of various cloud vendors are scattered and cannot be used in combination.

[0003] Existing technologies all perform targeted scheduling for a certain cloud vendor, lack an effective task scheduling mechanism to uniformly schedule cross-cloud resource operations, and lack a unified log tracking system to perform operation traceability and problem location. This leads to low efficiency and high operation and maintenance costs when performing resource operations in a multi-cloud environment. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method, device, equipment and medium for cross-cloud resource operation based on task orchestration, which greatly improves work efficiency through unified scheduling processing.

[0005] In a first aspect, the present invention provides a method for cross-cloud resource operation based on task orchestration, comprising the following steps:

[0006] Step 1: Abstract the received scheduling task request into task scheduling logic, and call the service interface of the corresponding cloud provider according to the task scheduling logic to execute all operations; the task scheduling logic includes the main task, subtasks and the order of subtask actions;

[0007] Step 2: Record all execution status in the set log file.

[0008] In a second aspect, the present invention provides a device for cross-cloud resource operation based on task orchestration, comprising the following modules:

[0009] The execution operation module abstracts the received scheduling task request into task scheduling logic, calls the service interface of the corresponding cloud provider according to the task scheduling logic, and executes all operations; the task scheduling logic includes the main task, subtasks and the order of subtask actions;

[0010] The logging module records all execution conditions in the set log file.

[0011] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0012] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0013] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0014] Business process consistency: By implementing a unified scheduling mechanism across multiple cloud platforms, we ensure that business processes executed in different cloud environments follow the same logic. This consistency allows developers to write scheduling logic once and apply it to multiple cloud services, reducing the learning curve between different platforms;

[0015] Reduce operational complexity: Reduce dependence on specific cloud services, making it easier to migrate services between different cloud platforms;

[0016] Transparency and traceability: A unified logging system ensures comprehensive monitoring of all task executions. Detailed logs of each operation can be obtained, which improves the transparency of operations;

[0017] Improved analysis efficiency: With detailed operation logs, troubleshooting becomes more efficient. Development and testing personnel can quickly locate the root cause of the problem, analyze and optimize the bottlenecks, and ensure business continuity and stability.

[0018] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention.

[0021] Figure 2 It is a schematic diagram of the structure of the device in the second embodiment of the present invention. DETAILED DESCRIPTION

[0022] The embodiments of the present application provide a method, apparatus, device and medium for cross-cloud resource operations based on task orchestration, which can not only flexibly combine and schedule multi-cloud resources through task orchestration, but also ensure the traceability of all resource operations through a unified log tracking system, and quickly locate and solve problems when they occur.

[0023] The technical solution in the embodiment of the present application has the following general idea:

[0024] 1. Task orchestration-driven cloud resource operation process

[0025] The present invention abstracts the operations in business services into task scheduling logic and manages them through scheduling services. Task scheduling logic includes the definition of the main task, subtasks, and the order of each task action. When a business service needs to perform resource operations, the scheduling service calls the service interfaces of each cloud provider in sequence according to the predefined task scheduling logic. Decoupling from different cloud vendors is achieved through the adapter mode, ensuring that resource operations of different cloud vendors can be executed according to a unified process.

[0026] 2. Unified cloud resource operation log tracking

[0027] The present invention provides a unified log tracking system to record the execution of each task, including the context information of the task and the manufacturer's call log. In the log system, not only can the execution status of each main task be viewed, but also alarms can be set according to the log level. When the system operation fails, the alarm mechanism such as DingTalk is automatically triggered, thereby improving the operation and maintenance efficiency and system reliability.

[0028] The implementation process is as follows:

[0029] 1. Define the task scheduling logic in the scheduling service:

[0030] Define the task scheduling logic in the system, including:

[0031] Main tasks: such as applying for agency;

[0032] Subtasks: Each subtask corresponds to the specific steps of the main task, such as creating an IP, creating an instance, creating a network card, binding an IP, deploying an agent, etc.

[0033] Action sequence: define the execution order of each subtask to ensure that all necessary operations are completed in sequence;

[0034] 2. Business service calls scheduling service

[0035] The business service specifies the supplier and main task to be executed, and calls the scheduling service. For example, choose to execute the Alibaba Cloud proxy application task.

[0036] 3. Scheduling service reception and processing

[0037] Scheduling service reception: After receiving the newly added scheduling task request, the scheduling service creates a new main task based on the configuration, extracts and stores the request parameters in Redis.

[0038] 3.1 Main task distribution (dispatch_main): In the main task management process, the scheduling service queries the main task in the initial state, and generates and adds initial subtasks based on the defined task relationship;

[0039] 3.2 Subtask dispatch (dispatch_item): During the execution of subtasks, the dispatch service first checks whether the main task has an error. If there is an error, the execution of subsequent subtasks is canceled. If the concurrency threshold is reached, the execution of the subtask is delayed. According to the subtask configuration, the supplier adaptation service and action to be sent are obtained, the request parameters are requested, the request is sent asynchronously, and the subtask status is updated to wait for callback;

[0040] 4. Supplier Adaptation Service Receiving and Processing

[0041] 4.1 First, according to the request parameters, which are the parameters set in 3.2, the vendor and action to be executed are obtained, and the adapter is found. The adapter is used to implement the interface call of each vendor and complete the actual action. Then the parameters are obtained from redis and the action call of the vendor is executed. In the action call, the interface of the external cloud vendor is requested in sequence according to the specific logic. The logic is the execution steps of the specific actions of each subtask; then the return result of the action is packaged and sent back to the scheduling service through a message;

[0042] 4.2 The system records the call information of external cloud vendors through tracking points, generates sub-operation logs (call method, status, start time, end time, execution times, parameters, return value, abnormality), and records the operation results;

[0043] 5. Scheduling service receiving callback processing

[0044] In the callback processing of the scheduling service manager, a callback message is received. At this time, concurrent locking is performed according to the main task ID to ensure the concurrent safety of the task and to ensure that only one subtask under the same main task is allowed to perform callback processing at the same time during the running process. If the locking operation fails, a message with a delay of 10 seconds will be sent so that the task can be retried later. If successful, the result is first updated to the Redis cache to provide the necessary input parameter information for subsequent tasks. Callback processing means that when a subtask is completed, it is necessary to determine whether to execute the next subtask;

[0045] Then determine the execution status of the subtask:

[0046] 5.1 If the subtask fails and the number of retries has been reached, the main task is marked as an error, and the subtask status is updated to an error. Finally, the main operation log (including: link ID, error cause (which subtask exception caused the failure), exception information, and occurrence time) is uploaded to end the process;

[0047] 5.2 If the subtask fails but the number of retries has not been reached, the subtask status is updated to "in progress", and the system will re-execute step 3.2, i.e. call the subtask again and continue trying;

[0048] 5.3 When a subtask succeeds, the system will check whether all the predecessor tasks of the next step have been successfully executed. If the predecessor task is not completed, the subtask status will be updated to success and return directly; if the predecessor tasks are all completed, the system will add a new next subtask in the initial state and continue to step 3.2;

[0049] 5.4 If there are no subsequent subtasks to be executed, mark the main task as completed and update the final status of the main task to success, completing the entire process;

[0050] 6. Through the overall operation log, you can quickly trace the root cause of the problem and achieve efficient troubleshooting.

[0051] Embodiment 1

[0052] like Figure 1 As shown, this embodiment provides a method for cross-cloud resource operation based on task orchestration, including the following steps:

[0053] Step 1: Abstract the received scheduling task request into task scheduling logic, and call the service interface of the corresponding cloud provider according to the task scheduling logic to execute all operations; the task scheduling logic includes the main task, subtasks and the order of subtask actions;

[0054] Step 2: Record all execution status in the set log file.

[0055] In this embodiment, preferably, the step 1 is specifically:

[0056] Step 11: abstract the received scheduling task request into task scheduling logic; the task scheduling logic includes a main task, subtasks, and the order of subtask actions;

[0057] Step 12: Generate and add initial subtasks according to the main task; set corresponding task configuration for each subtask, the task configuration includes: the supplier adaptation service and action to be sent, request parameters, sending request method, and update the subtask status to wait for callback;

[0058] Step 13: Execute the corresponding subtasks in sequence according to the order in which the subtasks are executed; obtain the action corresponding to the task according to the task configuration, call the action through the cloud vendor's adapter according to the request parameters, obtain the return result, and record the cloud vendor's call information in the sub-operation log;

[0059] Step 14: If the current subtask is successfully executed, the next subtask will be executed until all subtasks are completed, and the execution status will be recorded in the main task operation log. If the current subtask fails, the execution will be terminated, the subtask status will be updated to an error, the main task will be marked as an error, and the execution status will be recorded in the main task operation log.

[0060] In this embodiment, preferably, the step 14 is specifically as follows: when the main task is being executed, only one subtask is allowed to perform callback processing at the same time; if the current subtask is successfully executed, a locking operation is performed according to the main task ID; if the locking operation fails, the first time is delayed, and then the callback processing is performed; if the locking is successful, the callback processing is performed; after the callback processing is completed, an unlocking operation is performed, and then the next subtask is executed;

[0061] If the current subtask is executed successfully, the next subtask will be executed until all subtasks are completed, and the execution status will be recorded in the main task operation log; if the current subtask fails, the subtask will be re-executed after the set time. Only when the same subtask fails the set number of times, the execution will be terminated, the subtask status will be updated to error, and the main task will be marked as error, and the execution status will be recorded in the main task operation log.

[0062] In this embodiment, preferably, the step 2 is specifically as follows: recording all execution conditions, including the context information of the task and the manufacturer call; recording in a set log file, the log file including: a main operation log and a sub-operation log, the main operation log including: link ID, error cause, exception information and occurrence time; the sub-operation log including: the calling method, status, start time, end time, number of executions, parameters, return value and whether it is abnormal; when the main task or sub-task operation fails, an alarm mechanism is triggered according to the setting.

[0063] Based on the same inventive concept, the present application also provides a device corresponding to the method in Example 1, see Example 2 for details.

[0064] Embodiment 2

[0065] like Figure 2 As shown, in this embodiment, a device for cross-cloud resource operation based on task orchestration is provided, including the following modules:

[0066] The execution operation module abstracts the received scheduling task request into task scheduling logic, calls the service interface of the corresponding cloud provider according to the task scheduling logic, and executes all operations; the task scheduling logic includes the main task, subtasks and the order of subtask actions;

[0067] The logging module records all execution conditions in the set log file.

[0068] In this embodiment, preferably, the execution operation module is specifically:

[0069] The task scheduling unit abstracts the scheduling task request into task scheduling logic according to the received scheduling task request; the task scheduling logic includes the main task, subtasks and the order of subtask actions;

[0070] The subtask setting unit generates and adds initial subtasks according to the main task; each subtask sets a corresponding task configuration, and the task configuration includes: the supplier adaptation service and action to be sent, the request parameters, the sending request method, and the update subtask status to wait for callback;

[0071] The execution recording unit executes the corresponding subtasks in sequence according to the order in which the subtasks are executed; obtains the action corresponding to the task according to the task configuration, calls the action through the adapter of the cloud vendor according to the request parameters, obtains the return result, and records the call information of the cloud vendor in the sub-operation log;

[0072] The final execution unit, if the current subtask is executed successfully, will execute the next subtask until all subtasks are completed, and record the execution status in the main task operation log. If the current subtask fails, the execution ends, the subtask status is updated to an error, and the main task is marked as an error, and the execution status is recorded in the main task operation log.

[0073] In this embodiment, preferably, the final execution unit is specifically: when the main task is being executed, only one subtask is allowed to perform callback processing at the same time; if the current subtask is successfully executed, a locking operation is performed according to the main task ID; if the locking operation fails, the first time is delayed, and then the callback processing is performed; if the locking is successful, the callback processing is performed; after the callback processing is completed, the unlocking operation is performed, and then the next subtask is executed;

[0074] If the current subtask is executed successfully, the next subtask will be executed until all subtasks are completed, and the execution status will be recorded in the main task operation log; if the current subtask fails, the subtask will be re-executed after the set time. Only when the same subtask fails the set number of times, the execution will be terminated, the subtask status will be updated to error, and the main task will be marked as error, and the execution status will be recorded in the main task operation log.

[0075] In this embodiment, preferably, the logging module specifically records all execution conditions, including the context information of the task and the manufacturer call; records in a set log file, the log file includes: a main operation log and a sub-operation log, the main operation log includes: link ID, error cause, exception information and occurrence time; the sub-operation log includes: the calling method, status, start time, end time, number of executions, parameters, return value and whether it is abnormal; when the main task or sub-task operation fails, an alarm mechanism is triggered according to the setting.

[0076] Since the device introduced in the second embodiment of the present invention is a device used to implement the method of the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of the device based on the method introduced in the first embodiment of the present invention, so it is not described here in detail. All devices used in the method of the first embodiment of the present invention belong to the scope of protection of the present invention.

[0077] Based on the same inventive concept, the present application provides an electronic device embodiment corresponding to the first embodiment, see the third embodiment for details.

[0078] Embodiment 3

[0079] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any implementation method in the first embodiment can be implemented.

[0080] Since the electronic device introduced in this embodiment is a device used to implement the method in the first embodiment of the present application, based on the method introduced in the first embodiment of the present application, a person skilled in the art can understand the specific implementation of the electronic device of the present embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not described in detail here. As long as a person skilled in the art implements the device used by the method in the embodiment of the present application, it belongs to the scope of protection of the present application.

[0081] Based on the same inventive concept, the present application provides a storage medium corresponding to the first embodiment, see the fourth embodiment for details.

[0082] Embodiment 4

[0083] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any implementation method in the first embodiment can be implemented.

[0084] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0088] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for cross-cloud resource operation based on task orchestration, characterized in that: The steps include: Step 1: abstract the received scheduling task request into task scheduling logic, call the service interface of the corresponding cloud provider according to the task scheduling logic, and execute all operations; the task scheduling logic includes the main task, subtasks and the order of subtask actions; Step 2: Record all execution status in the set log file.

2. The method for cross-cloud resource operation based on task orchestration according to claim 1, characterized in that: The step 1 is specifically as follows: Step 11: abstract the received scheduling task request into task scheduling logic; the task scheduling logic includes a main task, subtasks, and the order of subtask actions; Step 12: Generate and add initial subtasks based on the main task; Each subtask sets a corresponding task configuration, which includes: the supplier adaptation service and action to be sent, request parameters, the sending request method, and updates the subtask status to wait for callback; Step 13: Execute the corresponding subtasks in sequence according to the order in which the subtasks are executed; obtain the action corresponding to the task according to the task configuration, call the action through the cloud vendor's adapter according to the request parameters, obtain the return result, and record the cloud vendor's call information in the sub-operation log; Step 14: If the current subtask is successfully executed, the next subtask will be executed until all subtasks are completed, and the execution status will be recorded in the main task operation log. If the current subtask fails, the execution will be terminated, the subtask status will be updated to an error, the main task will be marked as an error, and the execution status will be recorded in the main task operation log.

3. The method for cross-cloud resource operation based on task orchestration according to claim 2, characterized in that: The step 14 is specifically as follows: when the main task is being executed, only one subtask is allowed to perform callback processing at the same time; if the current subtask is successfully executed, a locking operation is performed according to the main task ID; if the locking operation fails, a first delay time is made before the callback processing is performed; if the locking is successful, a callback processing is performed; after the callback processing is completed, an unlocking operation is performed, and then the next subtask is executed; If the current subtask is executed successfully, the next subtask will be executed until all subtasks are completed and the execution status is recorded in the main task operation log; If the current subtask fails to execute, the subtask will be re-executed after the set time. Only when the same subtask fails the set number of times, the execution will be terminated, the subtask status will be updated to error, and the main task will be marked as an error, and the execution status will be recorded in the main task operation log.

4. The method for cross-cloud resource operation based on task orchestration according to claim 2, characterized in that: The step 2 specifically includes: recording all execution conditions, including task context information and manufacturer calls; Record in the set log file, the log file includes: main operation log and sub-operation log, the main operation log includes: link ID, error cause, exception information and occurrence time; the sub-operation log includes: calling method, status, start time, end time, number of executions, parameters, return value and whether it is abnormal; when the main task or sub-task operation fails, the alarm mechanism is triggered according to the setting.

5. A device for cross-cloud resource operation based on task orchestration, characterized in that: Includes the following modules: The execution operation module abstracts the received scheduling task request into task scheduling logic, calls the service interface of the corresponding cloud provider according to the task scheduling logic, and executes all operations; the task scheduling logic includes the main task, subtasks and the order of subtask actions; The logging module records all execution conditions in the set log file.

6. The device for cross-cloud resource operation based on task orchestration according to claim 5, characterized in that: The execution operation module is specifically: The task scheduling unit abstracts the scheduling task request into task scheduling logic according to the received scheduling task request; the task scheduling logic includes the main task, subtasks and the order of subtask actions; A subtask setting unit generates and adds initial subtasks according to the main task; Each subtask sets a corresponding task configuration, which includes: the supplier adaptation service and action to be sent, request parameters, the sending request method, and updates the subtask status to wait for callback; The execution recording unit executes the corresponding subtasks in sequence according to the order in which the subtasks are executed; obtains the action corresponding to the task according to the task configuration, calls the action through the adapter of the cloud vendor according to the request parameters, obtains the return result, and records the call information of the cloud vendor in the sub-operation log; The final execution unit, if the current subtask is executed successfully, will execute the next subtask until all subtasks are completed, and record the execution status in the main task operation log. If the current subtask fails, the execution ends, the subtask status is updated to an error, and the main task is marked as an error, and the execution status is recorded in the main task operation log.

7. The device for cross-cloud resource operation based on task orchestration according to claim 6, characterized in that: The final execution unit is specifically as follows: when the main task is being executed, only one subtask is allowed to perform callback processing at the same time; if the current subtask is successfully executed, a locking operation is performed according to the main task ID; if the locking operation fails, the first time is delayed and the callback processing is performed again; if the locking is successful, the callback processing is performed; after the callback processing is completed, an unlocking operation is performed, and then the next subtask is executed; If the current subtask is executed successfully, the next subtask will be executed until all subtasks are completed and the execution status is recorded in the main task operation log; If the current subtask fails to execute, the subtask will be re-executed after the set time. Only when the same subtask fails the set number of times, the execution will be terminated, the subtask status will be updated to error, and the main task will be marked as an error, and the execution status will be recorded in the main task operation log.

8. The device for cross-cloud resource operation based on task orchestration according to claim 6, characterized in that: The logging module specifically records all execution conditions, including task context information and vendor calls; Record in the set log file, the log file includes: main operation log and sub-operation log, the main operation log includes: link ID, error cause, exception information and occurrence time; the sub-operation log includes: calling method, status, start time, end time, number of executions, parameters, return value and whether it is abnormal; when the main task or sub-task operation fails, the alarm mechanism is triggered according to the setting.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images