Cross-data-center batch task evacuation method and device and electronic equipment

Abstract

The invention discloses a cross-data-center batch task evacuation method and device and electronic equipment, and relates to the field of distributed technologies or other related fields, and the method comprises the steps: responding to a task evacuation request, pausing the task execution operation of a fault data center, obtaining all unexecuted batch tasks of the fault data center, and sending the batch tasks to a server; and migrating unexecuted batch tasks to other data centers by adopting a load balancing strategy, obtaining all in-transit batch tasks from the in-transit batch task cache, and migrating the in-transit batch tasks to other data centers by adopting a hierarchical migration strategy based on task levels corresponding to the in-transit batch tasks. When it is detected that all unexecuted batch tasks and all in-transit batch tasks in the fault data center are migrated, the fault data center is isolated, and it is confirmed that the task evacuation request is executed. According to the invention, the technical problem of low evacuation efficiency caused by large cross-data center traffic when batch task evacuation is carried out in a multi-data center environment in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of distributed technologies or other related fields. Specifically, it relates to a method and device for evacuating batch tasks across data centers, and an electronic device. Background Art

[0002] In the context of digital development, enterprise applications and cloud services are expanding at an unprecedented rate, driving a sharp increase in data processing volume and computing requirements. Distributed systems are the cornerstone for processing large-scale data and providing highly available services, and their architecture design and operating efficiency are directly related to the continuity of business and the satisfaction of user experience. Massive batch tasks, such as data processing, report generation, data analysis, etc., are an essential part of distributed systems. To ensure the stable operation of distributed systems and the efficient execution of tasks, the management, scheduling, and evacuation mechanisms of batch tasks are one of the key points in the design of distributed systems.

[0003] In the related art, there are many problems in evacuating batch tasks in existing multi-data center distributed systems. First, when the load of the distributed system suddenly increases or a certain data center fails, the evacuation of batch tasks often requires large-scale data transmission between data centers, and the cross-data center traffic interaction is large. This not only consumes a large amount of network resources but also increases the system coupling degree between data centers, making the entire distributed system architecture more complex. Second, the existing batch task evacuation strategies have low processing efficiency for in-transit batch tasks. They often need to wait for the currently executing batch tasks to complete before evacuation, resulting in task delays and business impacts. Third, the performance problem of historical data query is particularly prominent in a multi-data center environment. Due to frequent data synchronization between data centers, the query response time is prolonged, the maintenance cost is increased, and the system stability and scalability are limited. Finally, with the increase in the number of data centers and the continuous update of application versions, the dynamic adjustment of load balancing becomes increasingly difficult. The existing load balancing strategies cannot adapt to the load changes between data centers in real time, easily leading to resource waste and system bottlenecks.

[0004] In response to the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present invention provide a method and device for evacuating batch tasks across data centers, and an electronic device, so as to at least solve the technical problem that in a multi-data center environment, when evacuating batch tasks, the large cross-data center traffic leads to low evacuation efficiency.

[0006] To achieve the above object, according to one aspect of the present application, there is provided a method for evacuating batch tasks across data centers, including: responding to a task evacuation request, suspending the task execution operation of the faulty data center; obtaining all unexecuted batch tasks of the faulty data center, and migrating the unexecuted batch tasks to other data centers by using a load balancing strategy; obtaining all in-transit batch tasks from the in-transit batch task cache of the faulty data center, and migrating the in-transit batch tasks to other data centers by using a hierarchical migration strategy based on the task levels corresponding to the in-transit batch tasks, where the in-transit batch tasks refer to the batch tasks that were being executed before the faulty data center failed; and isolating the faulty data center and confirming that the task evacuation request has been completed when it is detected that all unexecuted batch tasks and all in-transit batch tasks in the faulty data center have been migrated.

[0007] Optionally, the step of migrating the unexecuted batch tasks to other data centers by using a load balancing strategy includes: analyzing the application stickiness of all applications involved in the data center, where the application stickiness includes: the data flow direction and data traffic information of the application, the transmission link information, and the information on the dependent hardware resources and software resources; classifying the applications based on the application stickiness and assigning all applications to different categories; evaluating each other data center to determine the processing capabilities of the other data centers, where the processing capabilities include: computing power, storage capacity, and network status; specifying an evacuation order and a target data center for each batch task based on the application stickiness of the application and the processing capabilities of the data center, where the target data centers corresponding to the batch tasks of the same type of application are the same; and migrating all unexecuted batch tasks in the faulty data center to other data centers based on the evacuation order and the target data center.

[0008] Optionally, the step of migrating the unexecuted batch tasks to other data centers by using a load balancing strategy further includes: receiving parameter configurations input by an external terminal; when it is detected that there are equalization migration parameters specified for any batch task in the parameter configurations, specifying an evacuation order and a target data center for each batch task based on the application stickiness of the application, the processing capabilities of the data center, and all the equalization migration parameters, and migrating all unexecuted batch tasks in the faulty data center.

[0009] Optionally, the step of migrating the in-transit batch tasks to other data centers by using a hierarchical migration strategy includes: receiving parameter configurations input by an external terminal, where the parameter configurations at least include: the task levels of the in-transit batch tasks, and the task levels include the application levels and priorities of the in-transit batch tasks; and migrating the in-transit batch tasks to the corresponding target data centers by using a load balancing strategy based on the task levels.

[0010] Optionally, after migrating the in-transit batch tasks to other data centers, it further includes: other data centers receiving the in-transit batch tasks from the failed data center and synchronizing the received in-transit batch tasks to the in-transit batch task cache.

[0011] Optionally, after the task evacuation request is executed, it further includes: other data centers executing the batch tasks and in-transit batch tasks that the failed data center has not executed; other data centers saving the executed batch tasks to the historical batch database.

[0012] Optionally, before responding to the task evacuation request, it further includes: monitoring all data centers, and in the case of detecting that a data center fails, initiating a task evacuation request for the failed data center; or, in the case of detecting that any data center is in a load imbalance state, initiating a task evacuation request for that data center, where the load imbalance means that the data center is in an overloaded or idle state.

[0013] According to another aspect of the embodiments of the present invention, there is also provided a cross-data center batch task evacuation device, including: a response unit for responding to a task evacuation request and suspending the task execution operation of the failed data center; a first migration unit for obtaining all unexecuted batch tasks of the failed data center and migrating the unexecuted batch tasks to other data centers by using a load balancing strategy; a second migration unit for obtaining all in-transit batch tasks from the in-transit batch task cache of the failed data center and migrating the in-transit batch tasks to other data centers by using a hierarchical migration strategy based on the task levels corresponding to the in-transit batch tasks, where the in-transit batch tasks refer to the batch tasks that were being executed before the failed data center failed; an isolation unit for isolating the failed data center and confirming that the task evacuation request is executed after detecting that all unexecuted batch tasks and all in-transit batch tasks in the failed data center have been migrated.

[0014] Optionally, the first migration unit includes: an analysis module for analyzing the application stickiness of all applications involved in the data center, where the application stickiness includes: the data flow direction and data traffic information of the application, the transmission link information, and the information on the dependent hardware and software resources; a classification module for classifying the applications based on the application stickiness and categorizing all applications into different categories; an evaluation module for evaluating each other data center to determine the processing capacity of the other data center, where the processing capacity includes: computing power, storage capacity, and network status; a first designation module for designating an evacuation order and a target data center for each batch task based on the application stickiness of the application and the processing capacity of the data center, where the target data centers corresponding to the batch tasks of the same type of applications are the same; and a first evacuation module for migrating all unexecuted batch tasks in the faulty data center to other data centers based on the evacuation order and the target data center.

[0015] Optionally, the first migration unit further includes: a first receiving module for receiving parameter configurations input by an external terminal; and a second designation module for, when it is detected that there are load balancing parameters designated for any batch task in the parameter configurations, designating an evacuation order and a target data center for each batch task based on the application stickiness of the application, the processing capacity of the data center, and all the load balancing parameters, and migrating all unexecuted batch tasks in the faulty data center.

[0016] Optionally, the second migration unit includes: a second receiving module for receiving parameter configurations input by an external terminal, where the parameter configurations at least include: the task level of the in-transit batch task, and the task level includes the application level and the priority of the in-transit batch task; and a second evacuation module for migrating the in-transit batch task to the corresponding target data center based on the task level by using a load balancing strategy.

[0017] Optionally, the second migration unit further includes: a synchronization module for, after migrating the in-transit batch task to other data centers, the other data centers receiving the in-transit batch task from the faulty data center and synchronizing the received in-transit batch task to the in-transit batch task cache.

[0018] Optionally, the second migration unit further includes: an execution module for, after the task evacuation request is executed, the other data centers executing the unexecuted batch tasks and the in-transit batch tasks received from the faulty data center; and a saving module for the other data centers saving the executed batch tasks to the historical batch database.

[0019] Optionally, the response unit includes: a first initiation module for monitoring all data centers and initiating a task evacuation request for a faulty data center when a fault occurs in the data center; a second initiation module for initiating a task evacuation request for a data center when it is detected that any data center is in a load imbalance state, where the load imbalance means that the data center is in an overloaded or idle state.

[0020] According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, which includes a stored computer program, wherein when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the cross-data center batch task evacuation method of any one of the above.

[0021] According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including one or more processors and a memory, where the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the cross-data center batch task evacuation method of any one of the above.

[0022] According to another aspect of the embodiments of the present invention, there is also provided a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the cross-data center batch task evacuation method of any one of the above.

[0023] In the present disclosure, by responding to the task evacuation request, suspending the task execution operation of the faulty data center, then obtaining all unexecuted batch tasks of the faulty data center, migrating the unexecuted batch tasks to other data centers by using a load balancing strategy, then obtaining all in-transit batch tasks from the in-transit batch task cache of the faulty data center, migrating the in-transit batch tasks to other data centers by using a hierarchical migration strategy based on the task levels corresponding to the in-transit batch tasks, and finally, when it is detected that all unexecuted batch tasks and all in-transit batch tasks in the faulty data center have been migrated, isolating the faulty data center and confirming that the task evacuation request has been executed.

[0024] From the above disclosure, by responding to the task evacuation request and adopting the load balancing strategy to evacuate the unexecuted batch tasks, it is ensured that all unexecuted batch tasks are intelligently evacuated to the target data center, significantly reducing the traffic interaction between data centers and improving the efficiency of batch task evacuation; by introducing the in-transit batch task cache, the quick acquisition of in-transit batch tasks is realized, and the hierarchical migration strategy based on task levels is adopted to preferentially evacuate and process high-level in-transit batch tasks, significantly reducing the impact of failures on business continuity; by continuously and dynamically adjusting the distribution of batch tasks between data centers, the load balancing between data centers is achieved, thus solving the technical problem in the related art that when evacuating batch tasks in a multi-data center environment, the large cross-data center traffic leads to low evacuation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and the schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0026] Figure 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for evacuating batch tasks across data centers;

[0027] Figure 2 is a flowchart of an optional method for evacuating batch tasks across data centers according to an embodiment of the present invention;

[0028] Figure 3 is a structural diagram of an optional historical batch database according to an embodiment of the present invention;

[0029] Figure 4 is a structural diagram of an optional method for evacuating batch tasks across data centers according to an embodiment of the present invention;

[0030] Figure 5 is a schematic diagram of an optional device for evacuating batch tasks across data centers according to an embodiment of the present invention;

[0031] Figure 6 is a structural block diagram of an electronic device according to an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] It should be noted that the method and device for evacuating batch tasks across data centers in the present disclosure can be used in the case of evacuating batch tasks in the cross-data center scenario in the field of distributed technology, and can also be used in the case of evacuating batch tasks in the cross-data center scenario in any field other than the field of distributed technology. The application field of the method and device for evacuating batch tasks across data centers in the present disclosure is not limited.

[0035] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) collected in the present disclosure are information and data authorized by the user or fully authorized by all parties. And the processing of relevant data such as collection, storage, use, processing, transmission, provision, disclosure and application complies with the relevant laws, regulations and standards of the relevant regions, takes necessary confidentiality measures, does not violate public order and good customs, and provides corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set between the present system and relevant users or institutions. Before obtaining relevant information, a request for obtaining information needs to be sent to the aforementioned users or institutions through the interface, and after receiving the consent information feedback by the aforementioned users or institutions, the relevant information is obtained.

[0036] It should be noted that in the present disclosure, when collecting customer information, analyzing customer information, an operation entrance is provided for users to choose to agree or refuse the automated decision result; if the user chooses to refuse, the expert decision-making process will be entered.

[0037] The following embodiments of the present invention can be applied to various systems / applications / devices for evacuating batch tasks across data centers. By responding to task evacuation requests and adopting a load balancing strategy to evacuate unexecuted batch tasks, the present invention ensures that all unexecuted batch tasks are intelligently evacuated to the target data center, significantly reducing the traffic interaction between data centers and improving the efficiency of batch task evacuation. In addition, the present invention also realizes the rapid acquisition of in-transit batch tasks by introducing an in-transit batch task cache, and adopts a hierarchical migration strategy based on task levels to preferentially evacuate and process high-level in-transit batch tasks, significantly reducing the impact of failures on business continuity. Further, the present invention realizes load balancing between data centers by continuously and dynamically adjusting the distribution of batch tasks between data centers.

[0038] The present invention will be described in detail below in conjunction with each embodiment.

[0039] Embodiment 1

[0040] According to an embodiment of the present invention, an embodiment of a method for evacuating batch tasks across data centers is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0041] The embodiment of the method for evacuating batch tasks across data centers provided by the embodiment of the present application can be executed on a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the method for evacuating batch tasks across data centers is shown. As Figure 1 shown, the computer terminal 10 (or mobile device) may include one or more ( Figure 1 shown as 102a, 102b,..., 102n in Figure 1The structure shown is only schematic and does not limit the structure of the above electronic device. For example, computer terminal 10 may also include more or fewer components than those shown in Figure 1 or have a different configuration from that shown in Figure 1 .

[0042] It should be noted that one or more of the above processors 102 and / or other data processing circuits may generally be referred to as "data processing circuits" herein. The data processing circuit may be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any one of other elements in computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit is a processor control (such as the selection of a variable resistor terminal path connected to an interface).

[0043] Memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the cross-data center batch task evacuation method in the embodiments of the present application. Processor 102 executes various functional applications and data processing by running the software programs and modules stored in memory 104, that is, implements the above cross-data center batch task evacuation method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, memory 104 may further include a memory remotely located relative to processor 102, and these remote memories may be connected to computer terminal 10 through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0044] Transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by the communication provider of computer terminal 10. In one instance, transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0045] The display may be, for example, a touch-screen liquid crystal display (Liquid Crystal Display, LCD for short), and the liquid crystal display enables a user to interact with the user interface of computer terminal 10 (or mobile device).

[0046] Under the above operating environment, the present application provides a batch task evacuation method across data centers as Figure 2 shown.

[0047] First of all, it should be noted that in each data center of the embodiments of the present invention, a set of batch task evacuation systems across data centers is configured. Each set of batch task evacuation systems across data centers includes 6 modules, namely: D000 task configuration module, D001 monitoring module, D002 task scheduling module, D003 task execution module, D004 in-transit batch task cache module, and D005 historical batch database.

[0048] Among them, the D000 task configuration module is mainly used to record and store the batch tasks responsible for processing by this data center, and is the data source of the D002 task scheduling module. The D001 monitoring module is mainly used to receive task evacuation requests and forward the task evacuation requests to the D002 task scheduling module. The D002 task scheduling module is mainly used to analyze the batch tasks to be evacuated, generate task evacuation strategies, and transfer the task evacuation strategies to the D002 task scheduling modules of other data centers. The D003 task execution module is mainly used to execute the batch tasks issued by the D002 task scheduling module. The D004 in-transit batch task cache module is mainly used to store the in-transit batch tasks that have been executed but not yet completed. When the in-transit batch tasks are completed, they will be recorded and stored in the D005 historical batch database. Among them, the D005 historical batch database is mainly used to record and store the completed batch tasks.

[0049] Figure 2 is a flowchart of an optional batch task evacuation method across data centers according to the embodiments of the present invention, as Figure 2 shown. The method includes the following steps:

[0050] Step S201, in response to the task evacuation request, suspend the task execution operation of the faulty data center.

[0051] In the embodiments of the present invention, the D001 monitoring module of each data center monitors the operating status of this data center. When the D001 monitoring module monitors that the load of this data center is extremely high or a fault occurs, a task evacuation request for this data center will be triggered.

[0052] It should be noted that the triggering of the task evacuation request can be automatically triggered after monitoring an anomaly, or can be manually triggered by an operation and maintenance personnel.

[0053] After triggering a task evacuation request, the D001 monitoring module forwards the task evacuation request to the D002 task scheduling module in this data center (i.e., the faulty data center). After the D002 task scheduling module receives the task evacuation request, it will suspend the task execution operation in the faulty data center and no longer issue batch tasks to the D003 task execution module.

[0054] Step S202: Obtain all unexecuted batch tasks in the faulty data center, and use a load balancing strategy to migrate the unexecuted batch tasks to other data centers.

[0055] After suspending the task execution operation in the faulty data center, the D000 task configuration module will be updated, and the batch tasks originally executed by the faulty data center in the D000 task configuration module will be load balanced to other data centers. In this process, the D002 task scheduling module obtains all unexecuted batch tasks in the faulty data center from the D000 task configuration module, and uses a load balancing strategy to migrate the unexecuted batch tasks to other data centers.

[0056] It should be noted that the load balancing strategy adopted in this embodiment uses multiple algorithms, including a load balancing strategy based on application stickiness. This strategy preferentially concentrates batch tasks of the same type of application in the same data center, avoiding frequent access between data centers caused by the scheduling of batch tasks between applications to the greatest extent, and improving the efficiency and stability of task processing.

[0057] At the same time, the load balancing strategy also includes a custom strategy, that is, allowing operation and maintenance personnel to upload parameter configurations and specify the evacuation targets of certain batch tasks. By combining the load balancing strategy based on application stickiness and the custom strategy, the embodiments of the present invention effectively evacuate all unexecuted batch tasks in the faulty data center to other data centers, not only improving the efficiency of task processing and the stability of the system, but also reducing the traffic interaction between data centers and enhancing the scalability and reliability of the system.

[0058] Step S203: Obtain all in-transit batch tasks from the in-transit batch task cache in the faulty data center, and use a hierarchical migration strategy to migrate the in-transit batch tasks to other data centers based on the task levels corresponding to the in-transit batch tasks, where the in-transit batch tasks refer to the batch tasks that were being executed before the faulty data center failed.

[0059] In-transit batch tasks refer to the batch tasks that were in the execution state before the faulty data center failed. They may be interrupted due to sudden system failures, resource limitations, or network problems. These in-transit batch tasks are crucial for business continuity and data integrity.

[0060] For in-transit batch tasks, the D002 task scheduling module will read all in-transit batch tasks from the D004 in-transit batch task cache module and migrate these in-transit batch tasks to the D002 task scheduling modules of other data centers. For other data centers that are operating normally, after receiving the in-transit batch tasks, their D002 task scheduling modules will synchronize the in-transit batch tasks to the D004 in-transit batch task cache module of this data center, and then schedule the D003 task execution module of this data center to continue executing these evacuated in-transit batch tasks.

[0061] During the migration of in-transit batch tasks, the embodiment of the present invention adopts an intelligent hierarchical migration strategy. The hierarchical migration strategy is based on the task level corresponding to the in-transit batch tasks, and prioritizes the migration of high-level in-transit batch tasks. High-level batch tasks are usually closely related to key businesses or operations, and their completion directly affects the operating efficiency and business continuity of the entire system. Therefore, by adopting a hierarchical migration strategy, the business interruption time can be significantly reduced, ensuring the continuity of key tasks and the integrity of data.

[0062] Step S204, when it is detected that all unexecuted batch tasks and all in-transit batch tasks in the faulty data center have been migrated, the faulty data center is isolated and it is confirmed that the task evacuation request has been executed.

[0063] When it is detected that all unexecuted batch tasks in the faulty data center and all in-transit batch task migrations are completed, the embodiment of the present invention isolates the faulty data center, and the faulty data center no longer accepts any requests. At this time, it is confirmed that the task evacuation request has been executed.

[0064] When the faulty data center is repaired, the D000 task configuration module is updated through the D002 task scheduling module, and the data center can be restarted to continue providing services.

[0065] Optionally, the step of adopting a load balancing strategy to migrate unexecuted batch tasks to other data centers includes: analyzing application stickiness of all applications involved in the data center, wherein application stickiness includes: application data flow direction and data traffic information, transmission link information, and dependent hardware resources and software resource information; classifying applications based on application stickiness, and classifying all applications into different categories; evaluating each other data center to determine the processing capacity of other data centers, wherein processing capacity includes: computing power, storage capacity, and network status; specifying an evacuation order and a target data center for each batch task based on the application stickiness of the application and the processing capacity of the data center, wherein batch tasks of the same type of application have the same target data center; migrating all unexecuted batch tasks in the failed data center to other data centers based on the evacuation order and the target data center.

[0066] When migrating unexecuted batch tasks to other data centers using a load balancing strategy, by deeply analyzing application stickiness and reasonably evaluating the processing capabilities of data centers, intelligent classification and precise evacuation of batch tasks can be achieved. Specifically:

[0067] The D002 task scheduling module conducts stickiness analysis on all applications involved in the faulty data center. The stickiness analysis covers multiple key dimensions of the application, including: the data flow direction and data traffic information of the application, aiming to understand the frequency and scale of data interaction between different applications, as well as the dependence of different applications on different data centers; transmission link information, that is, the network paths and connection status relied on by the application, to evaluate the demand and dependence of different applications on network resources inside and outside the data center; information on the hardware resources and software resources relied on, including CPU, memory, storage devices, and specific software environments, to examine the resource consumption and requirements of the application for the data center. Through this series of detailed application stickiness analyses, the embodiments of the present invention can comprehensively grasp the running characteristics and resource requirements of each application.

[0068] Based on the results of the above stickiness analysis, the D002 task scheduling module classifies the applications, grouping applications with similar data flow patterns, network link requirements, and resource consumption into the same category. This classification process takes into account the data dependence relationships between applications, the efficiency of network communication, and the compatibility of hardware and software resources, ensuring that applications in the same category can be efficiently allocated to a unified and suitable resource environment during evacuation, reducing the coupling between data centers and the data synchronization cost.

[0069] At the same time, the D002 module carefully evaluates the processing capabilities of each other data center, including key indicators such as computing power, storage capacity, and network status. This evaluation aims to comprehensively understand the resource availability and performance status of different data centers, providing a basis for the subsequent evacuation and allocation of batch tasks. By real-time monitoring and collecting the resource usage of each data center, the embodiments of the present invention can quickly identify data centers with abundant resources and stable networks as the evacuation targets for unexecuted batch tasks in the faulty data center.

[0070] Based on the application stickiness of the application and the evaluation results of the processing capabilities of the data center, the D002 task scheduling module specifies the evacuation order and target data center for each unexecuted batch task. For batch tasks of the same type of application, they will be assigned to the same data center for execution. This approach not only reduces the data transmission between data centers, significantly reducing the cross-data center traffic interaction, but also fully utilizes the synergistic effect of the distributed architecture within the data center, improving the efficiency and stability of task execution.

[0071] Finally, the D002 task scheduling module begins to orderly migrate all unexecuted batch tasks in the faulty data center to other data centers according to the formulated evacuation strategy.

[0072] Through in-depth analysis of application stickiness and comprehensive evaluation of the processing capabilities of data centers, the embodiments of the present invention achieve the formulation and execution of intelligent evacuation strategies, effectively solve the problems of large traffic, high system coupling degree, and low evacuation efficiency during the evacuation of batch tasks across data centers, and achieve load balancing among multiple data centers in a distributed system.

[0073] Optionally, the step of migrating unexecuted batch tasks to other data centers using a load balancing strategy further includes: receiving parameter configurations input by an external terminal; when it is detected that there are equilibrium migration parameters specified for any batch task in the parameter configuration, based on the application stickiness of the application, the processing capabilities of the data centers, and all equilibrium migration parameters, specifying an evacuation order and a target data center for each batch task, and migrating all unexecuted batch tasks in the faulty data center.

[0074] When migrating unexecuted batch tasks to other data centers using a load balancing strategy, it not only covers the process of automatically analyzing application stickiness and data center processing capabilities, but also allows operation and maintenance personnel to specify and optimize the task evacuation strategy according to real-time business requirements and resource status, ensuring the efficiency and intelligence of the evacuation process. Specifically:

[0075] The D002 task scheduling module first receives parameter configurations from an external terminal. When it is detected that there are equilibrium migration parameters (such as specified target data centers, evacuation orders) specified for any batch task in the parameter configuration, the D002 task scheduling module comprehensively analyzes these parameters with the previously collected application stickiness data (including data flow direction and data traffic information of the application, transmission link information, and information on dependent hardware and software resources) and the evaluation results of data center processing capabilities (including computing power, storage capacity, and network status). This analysis process combines external parameters with internal data through a pre-constructed intelligent algorithm to generate a more accurate evacuation strategy.

[0076] Based on the above comprehensive analysis, the D002 task scheduling module upgrades the intelligent evacuation strategy, specifying a new evacuation order and a target data center for each batch task. This strategy fully considers the equilibrium migration parameters input externally, ensuring the optimization of the execution efficiency and resource utilization rate of batch tasks on the premise of meeting application stickiness and data center resource matching. After completing the upgrade of the intelligent evacuation strategy, the D002 task scheduling module migrates all unexecuted batch tasks in the faulty data center to the corresponding target data centers according to the specified evacuation order.

[0077] Optionally, the step of migrating the in-transit batch tasks to other data centers using a hierarchical migration strategy includes: receiving parameter configuration input by an external terminal, wherein the parameter configuration includes at least: a task level of the in-transit batch tasks, the task level including an application level and a priority level of the in-transit batch tasks; based on the task level, migrating the in-transit batch tasks to the corresponding target data center using a load balancing strategy.

[0078] When adopting the hierarchical migration strategy to migrate the in-transit batch tasks to other data centers, first, the D002 task scheduling module receives the parameter configuration input by the external terminal, which at least includes the task level information of the in-transit batch tasks. The task level is a comprehensive indicator that covers multiple dimensions such as application level and priority to measure the importance and urgency of the in-transit batch tasks. For example, in-transit batch tasks with high application levels are usually closely related to core business processes, while in-transit batch tasks with high priorities need to be completed within a limited time to avoid affecting business operations.

[0079] Based on the received task level information, the D002 task scheduling module will implement a hierarchical migration strategy. The core of this strategy is to migrate tasks to the corresponding target data center using a load balancing strategy based on the task level. The hierarchical migration strategy first sorts the in-transit batch tasks by level, divides the in-transit batch tasks according to their application level and priority, and then uses a load balancing strategy to migrate the in-transit batch tasks to the corresponding target data center based on the divided level, ensuring that the in-transit batch tasks with higher task levels are processed first and evacuated.

[0080] Optionally, after migrating the in-transit batch tasks to other data centers, the method further includes: the other data centers receive the in-transit batch tasks from the failed data center, and synchronize the received in-transit batch tasks to the in-transit batch task cache.

[0081] After migrating the in-transit batch tasks in the failed data center to other data centers, the D002 task scheduling module of other data centers is responsible for receiving these in-transit batch tasks. Other data centers synchronize the in-transit batch tasks to the D004 in-transit batch task cache module of this data center, and then schedule the D003 task execution module of this data center to continue executing these evacuated in-transit batch tasks to ensure that important in-transit batch tasks can be processed in a timely manner and reduce the impact of business interruptions.

[0082] Optionally, after the task evacuation request is executed, the method further includes: other data centers execute the received batch tasks that have not been executed by the faulty data center and the batch tasks in transit; and other data centers save the executed batch tasks to a historical batch database.

[0083] After the task evacuation request of the faulty data center is executed, other data centers will execute the batch tasks that the received faulty data center did not execute and the in-transit batch tasks, and save the executed batch tasks to the historical batch database.

[0084] It should be noted that since there is multi-data center task scheduling for batch tasks and query transactions also need to be executed across data centers, this will lead to a decrease in query efficiency. At the same time, in order to ensure data consistency, the database will also pay a huge cost in data synchronization. To improve data query efficiency and ensure system scalability, in the design of the historical batch database in the embodiments of the present invention, a sub-database query strategy is adopted, that is, the database is split according to the batch task id.

[0085] Figure 3 is a structural diagram of an optional historical batch database according to an embodiment of the present invention, as Figure 3 shown, in the embodiments of the present invention, logical sub-database and sub-table operations are performed according to the last digits of the batch task id. For example, transactions with the last digit 1, transactions with the last digit 2, and transactions with the last digit 3 are each divided into 3 parts, and a total of 9 parts of data are split. For the transactions with the last digit 1, the 3 parts of data are respectively stored in the db1 databases of data center A, data center B, and data center C. These 3 parts of data are not the same, and the db1 databases of data center A, data center B, and data center C do not need to perform data synchronization. When a user needs to query transactions with the last digit 1, only need to query the db1 databases of data center A, data center B, and data center C at the same time, and summarize the query results to obtain all query results. The historical batch database adopts the aforementioned sub-database design, which greatly improves data query efficiency and avoids a large amount of data synchronization at the cross-data center level.

[0086] Optionally, before responding to the task evacuation request, it further includes: monitoring all data centers, and in the case of monitoring that a data center fails, initiating a task evacuation request for the faulty data center; or, in the case of detecting that any data center is in a load imbalance state, initiating a task evacuation request for the data center, where the load imbalance means that the data center is in an overloaded or idle state.

[0087] In an embodiment of the present invention, the D001 monitoring module implements round-the-clock real-time monitoring of all data centers, and the monitoring content covers multiple dimensions such as the hardware status, software running status, network connection status, and task execution status of the data centers. Through continuous monitoring, the D001 monitoring module can quickly identify abnormal situations in the data centers, such as hardware failures, software crashes, network interruptions, or abnormal task executions. These abnormalities may affect the normal operation of the data centers. Once a failure occurs in the data center, the D001 monitoring module will automatically trigger a task evacuation request for the faulty data center to ensure that measures can be taken promptly when a failure occurs to prevent the spread of the failure or affect the stable operation of other data centers. Alternatively, after the D001 monitoring module detects a failure in the data center, it will send an alarm to the operation and maintenance terminal to remind the operation and maintenance personnel to manually trigger a task evacuation request for the faulty data center.

[0088] At the same time, during daily operation, the execution of batch tasks among different data centers will also change with the change of the application version. The originally load-balanced multi-data centers may also become load-imbalanced. At this time, when it is detected that any data center is in a load-imbalanced situation, a task evacuation request for this data center can also be initiated to achieve the redistribution of batch tasks across data centers. Since no system failure has occurred at this time and it is an actively triggered optimization adjustment, there is no need to isolate the data center.

[0089] Figure 4 It is a structural diagram of an optional method for evacuating batch tasks across data centers according to an embodiment of the present invention. As Figure 4 shown, data centers A, B, and C all include a D000 task configuration module (not shown in Figure 4), D001 monitoring module, D002 task scheduling module, D003 task execution module, D004 in-transit batch task cache module and D005 historical batch database. The D001 monitoring module forwards the task evacuation request to the D002 task scheduling module of this data center. The D002 task scheduling module no longer sends batch tasks to the D003 task execution module. It obtains all unexecuted batch tasks of the faulty data center from the D000 task configuration module and uses the load balancing strategy to migrate the unexecuted batch tasks to the D002 task scheduling module of other data centers. At the same time, the D002 task scheduling module reads all in-transit batch tasks from the D004 in-transit batch task cache module and migrates these in-transit batch tasks to the D002 task scheduling module of other data centers. After receiving the unexecuted and in-transit batch tasks, the D002 task scheduling module of other data centers will synchronize the in-transit batch tasks to the D004 in-transit batch task cache module of this data center, and then schedule the D003 task execution module of this data center to execute these unexecuted and in-transit batch tasks that have been evacuated.

[0090] The embodiment of the present invention realizes intelligent analysis and selection of data centers by adopting a load balancing strategy based on application stickiness, ensuring that unexecuted batch tasks are evacuated to the most suitable data center. When evacuating in-transit batch tasks, an intelligent hierarchical migration strategy is adopted to give priority to evacuating in-transit batch tasks with higher task levels, which helps to reduce the business impact caused by failures.

[0091] By introducing the in-transit batch cache module, large-volume data synchronization across data centers can be avoided. If all batch tasks are stored in the database, then the migration of in-transit batch tasks will involve database information synchronization, which will significantly increase the cost of data migration and make it difficult to ensure data consistency and real-time performance. After the introduction of the in-transit batch cache module, since the magnitude of in-transit batch tasks is much smaller than that of historical batch tasks, only a small amount of in-transit batch task information needs to be synchronized during each evacuation, which not only improves task timeliness and evacuation efficiency, but also reduces traffic costs between data centers.

[0092] In addition, the embodiment of the present invention realizes load balancing among data centers by continuously and dynamically adjusting the distribution of batch tasks among data centers. By real-time monitoring of data centers, it is ensured that task evacuation requests can be initiated immediately when a data center fails, thereby reducing the business impact caused by the failure.

[0093] In the normal operation state, the batch task evacuation systems of each data center operate independently without data interaction, thus significantly improving the operation speed of the batch task evacuation system. Since there is little data interaction between data centers, when the number of data centers increases, only this batch task evacuation system needs to be deployed in the corresponding data center without much modification, which has good scalability.

[0094] The steps provided by the above cross-data center batch task evacuation method, by responding to the task evacuation request and adopting the load balancing strategy to evacuate the unexecuted batch tasks, ensure that all unexecuted batch tasks are intelligently evacuated to the target data center, significantly reducing the traffic interaction between data centers and improving the batch task evacuation efficiency; by introducing the in-transit batch task cache, the quick acquisition of in-transit batch tasks is realized, and the hierarchical migration strategy based on the task level is adopted to preferentially evacuate and process high-level in-transit batch tasks, significantly reducing the impact of faults on business continuity; by continuously and dynamically adjusting the distribution of batch tasks between data centers, the load balancing between data centers is achieved, thus solving the technical problem of low evacuation efficiency caused by large cross-data center traffic when evacuating batch tasks in a multi-data center environment in the related technology.

[0095] The following will be described in detail in combination with another embodiment.

[0096] Embodiment 2

[0097] A cross-data center batch task evacuation device provided in this embodiment includes multiple implementation units, and each implementation unit corresponds to each implementation step in Embodiment 1 above.

[0098] Figure 5 It is a schematic diagram of an optional cross-data center batch task evacuation device according to an embodiment of the present invention. As Figure 5 shown, the cross-data center batch task evacuation device may include: a response unit 51, a first migration unit 52, a second migration unit 53, and an isolation unit 54.

[0099] Among them, the response unit 51 is used to respond to the task evacuation request and suspend the task execution operation of the faulty data center.

[0100] The first migration unit 52 is used to obtain all unexecuted batch tasks of the faulty data center and migrate the unexecuted batch tasks to other data centers by adopting the load balancing strategy.

[0101] A second migration unit 53 is configured to obtain all in-transit batch tasks from the in-transit batch task cache of the faulty data center, and based on the task levels corresponding to the in-transit batch tasks, migrate the in-transit batch tasks to other data centers by using a hierarchical migration strategy, where the in-transit batch tasks refer to the batch tasks being executed before the faulty data center fails.

[0102] An isolation unit 54 is configured to isolate the faulty data center and confirm that the task evacuation request is completed when it detects that all unexecuted batch tasks and all in-transit batch tasks in the faulty data center have been migrated.

[0103] The above-mentioned cross-data center batch task evacuation device can, by means of the response unit 51, respond to the task evacuation request, suspend the task execution operation of the faulty data center, then obtain all unexecuted batch tasks of the faulty data center through the first migration unit 52, migrate the unexecuted batch tasks to other data centers by using a load balancing strategy, then obtain all in-transit batch tasks from the in-transit batch task cache of the faulty data center through the second migration unit 53, and based on the task levels corresponding to the in-transit batch tasks, migrate the in-transit batch tasks to other data centers by using a hierarchical migration strategy. Finally, when the isolation unit 54 detects that all unexecuted batch tasks and all in-transit batch tasks in the faulty data center have been migrated, it isolates the faulty data center and confirms that the task evacuation request is completed.

[0104] In the embodiment of the present invention, by responding to the task evacuation request and using a load balancing strategy to evacuate unexecuted batch tasks, it is ensured that all unexecuted batch tasks are intelligently evacuated to the target data center, significantly reducing the traffic interaction between data centers and improving the batch task evacuation efficiency; by introducing an in-transit batch task cache, the rapid acquisition of in-transit batch tasks is realized, and a hierarchical migration strategy based on task levels is adopted to preferentially evacuate and process high-level in-transit batch tasks, significantly reducing the impact of faults on business continuity; by continuously and dynamically adjusting the distribution of batch tasks between data centers, load balancing between data centers is achieved, thus solving the technical problem in the related art that when evacuating batch tasks in a multi-data center environment, the cross-data center traffic is large, resulting in low evacuation efficiency.

[0105] Optionally, the first migration unit 52 includes: an analysis module for analyzing the application stickiness of all applications involved in the data center, where the application stickiness includes: the data flow direction and data traffic information of the application, the transmission link information, and the information on the dependent hardware and software resources; a classification module for classifying the applications based on the application stickiness and categorizing all applications into different categories; an evaluation module for evaluating each other data center to determine the processing capabilities of the other data centers, where the processing capabilities include: computing power, storage capacity, and network status; a first designation module for designating an evacuation order and a target data center for each batch task based on the application stickiness of the application and the processing capabilities of the data center, where the target data centers corresponding to the batch tasks of the same type of applications are the same; and a first evacuation module for migrating all unexecuted batch tasks in the failed data center to other data centers based on the evacuation order and the target data center.

[0106] Optionally, the first migration unit 52 further includes: a first receiving module for receiving parameter configurations input by an external terminal; and a second designation module for, when detecting that there are load balancing parameters designated for any batch task in the parameter configurations, designating an evacuation order and a target data center for each batch task based on the application stickiness of the application, the processing capabilities of the data center, and all the load balancing parameters, and migrating all unexecuted batch tasks in the failed data center.

[0107] Optionally, the second migration unit 53 includes: a second receiving module for receiving parameter configurations input by an external terminal, where the parameter configurations at least include: the task level of the in-transit batch task, and the task level includes the application level and the priority of the in-transit batch task; and a second evacuation module for migrating the in-transit batch tasks to the corresponding target data centers by using a load balancing strategy based on the task level.

[0108] Optionally, the second migration unit 53 further includes: a synchronization module for, after migrating the in-transit batch tasks to other data centers, the other data centers receiving the in-transit batch tasks from the failed data center and synchronizing the received in-transit batch tasks to the in-transit batch task cache.

[0109] Optionally, the second migration unit 53 further includes: an execution module for, after the task evacuation request is executed, the other data centers executing the unexecuted batch tasks and the in-transit batch tasks received from the failed data center; and a saving module for the other data centers saving the executed batch tasks to the historical batch database.

[0110] Optionally, the response unit 51 includes: a first initiation module for monitoring all data centers and initiating a task evacuation request for a faulty data center when a failure occurs in the data center; a second initiation module for initiating a task evacuation request for a data center when it is detected that any data center is in a load imbalance state, where the load imbalance means that the data center is in an overloaded or idle state.

[0111] The above-mentioned cross-data center batch task evacuation device may further include a processor and a memory. The above-mentioned response unit 51, the first migration unit 52, the second migration unit 53, the isolation unit 54, etc. are all stored in the memory as program units, and the corresponding functions are implemented by the processor executing the above program units stored in the memory.

[0112] The above-mentioned processor includes a kernel, and the kernel retrieves the corresponding program units from the memory. One or more kernels can be set, and the batch task evacuation is performed in the cross-data center scenario by adjusting the kernel parameters.

[0113] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip.

[0114] Embodiment III

[0115] An embodiment of the present application can provide an electronic device. Figure 6 It is a structural block diagram of an electronic device according to an embodiment of the present application. As Figure 6 shown, the electronic device may include: one or more ( Figure 6 only one is shown in the figure) processors 602, a memory 604, a storage controller, and a peripheral interface, where the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0116] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the above-mentioned methods are implemented. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely provided relative to the processor, and these remote memories can be connected to the terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise internal network, a local area network, a mobile communication network, and combinations thereof.

[0117] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: in response to a task evacuation request, suspend the task execution operation of the faulty data center; obtain all unexecuted batch tasks of the faulty data center, and use a load balancing strategy to migrate the unexecuted batch tasks to other data centers; obtain all in-transit batch tasks from the in-transit batch task cache of the faulty data center, and based on the task levels corresponding to the in-transit batch tasks, use a hierarchical migration strategy to migrate the in-transit batch tasks to other data centers, where the in-transit batch tasks refer to the batch tasks that were being executed before the faulty data center failed; in the case where it is detected that all unexecuted batch tasks and all in-transit batch tasks in the faulty data center have been migrated, isolate the faulty data center and confirm that the task evacuation request has been executed.

[0118] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: analyze the application stickiness of all applications involved in the data center, where the application stickiness includes: the data flow direction and data flow information of the application, the transmission link information, and the information on the hardware resources and software resources it depends on; classify the applications based on the application stickiness and assign all applications to different categories; evaluate each other data center to determine the processing capabilities of the other data centers, where the processing capabilities include: computing power, storage capacity, and network status; based on the application stickiness of the applications and the processing capabilities of the data centers, specify an evacuation order and a target data center for each batch task, where the target data centers corresponding to the batch tasks of the same type of application are the same; based on the evacuation order and the target data centers, migrate all unexecuted batch tasks in the faulty data center to other data centers.

[0119] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: receive the parameter configuration input by an external terminal; in the case where it is detected that there is an equalization migration parameter specified for any batch task in the parameter configuration, based on the application stickiness of the application, the processing capabilities of the data center, and all the equalization migration parameters, specify an evacuation order and a target data center for each batch task, and migrate all unexecuted batch tasks in the faulty data center.

[0120] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: receive the parameter configuration input by an external terminal, where the parameter configuration at least includes: the task levels of the in-transit batch tasks, and the task levels include the application levels and priorities of the in-transit batch tasks; based on the task levels, use a load balancing strategy to migrate the in-transit batch tasks to the corresponding target data centers.

[0121] The processor can also call the information and application programs stored in the memory through the transmission device to execute the following steps: Other data centers receive the in-transit batch tasks from the faulty data center and synchronize the received in-transit batch tasks to the in-transit batch task cache.

[0122] The processor can also call the information and application programs stored in the memory through the transmission device to execute the following steps: Other data centers execute the batch tasks and in-transit batch tasks that the received faulty data center has not executed; Other data centers save the completed batch tasks to the historical batch database.

[0123] The processor can also call the information and application programs stored in the memory through the transmission device to execute the following steps: Monitor all data centers, and in the case of detecting that a data center fails, initiate a task evacuation request for the faulty data center; or, in the case of detecting that any data center is in load imbalance, initiate a task evacuation request for that data center, where load imbalance means that the data center is in an overloaded or idle state.

[0124] Adopting the embodiment of the present application provides a solution for a cross-data center batch task evacuation method. By responding to the task evacuation request, suspending the task execution operation of the faulty data center, then obtaining all unexecuted batch tasks of the faulty data center, migrating the unexecuted batch tasks to other data centers using a load balancing strategy, then obtaining all in-transit batch tasks from the in-transit batch task cache of the faulty data center, and migrating the in-transit batch tasks to other data centers using a hierarchical migration strategy based on the task levels corresponding to the in-transit batch tasks, and finally, in the case of detecting that all unexecuted batch tasks and all in-transit batch tasks in the faulty data center have been migrated, isolating the faulty data center and confirming that the task evacuation request has been executed, thereby achieving the purpose of significantly reducing the traffic interaction between data centers and improving the batch task evacuation efficiency, and further solving the technical problem of low evacuation efficiency caused by large cross-data center traffic when evacuating batch tasks in a multi-data center environment in the related art.

[0125] Those of ordinary skill in the art can understand that Figure 6 The structure shown is only for illustration, and the electronic device can also be a terminal device such as a smart phone, a tablet computer, a handheld computer, and a Mobile Internet Device (MID), a PAD, etc. Figure 6 It does not limit the structure of the above electronic device. For example, the electronic device may further include more or fewer components (such as a network interface, a display device, etc.) than those shown Figure 6 in it, or have a different configuration from that shown Figure 6 in it.

[0126] Those of ordinary skill in the art can understand that all or part of the steps in the above-described various cross-data center batch task evacuation methods can be completed by a program instructing the relevant hardware of the terminal device. The program can be stored in a computer-readable storage medium, and the storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0127] Embodiment 4

[0128] An embodiment of the present application also provides a storage medium. Optionally, in this embodiment, the above storage medium can be used to store the program code executed by the cross-data center batch task evacuation method provided in Embodiment 1 above.

[0129] According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the cross-data center batch task evacuation method of any one of the above Embodiment 1.

[0130] Optionally, in this embodiment, the above storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0131] The present application also provides a computer program product, including a computer program, where the steps of the cross-data center batch task evacuation method described in various embodiments of the present application are implemented when the computer program is executed by a processor.

[0132] The present application also provides a computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and the steps of the cross-data center batch task evacuation method described in various embodiments of the present application are implemented when the computer program is executed by a processor.

[0133] The above serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0134] In the above embodiments of the present invention, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0135] In several embodiments provided by this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0136] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0138] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs and other various media that can store program codes.

[0139] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can still be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A batch task evacuation method across data centers, characterized in that: include: Respond to task evacuation requests and suspend task execution operations in the failed data center; Obtain all unexecuted batch tasks of the faulty data center, and migrate the unexecuted batch tasks to other data centers using a load balancing strategy; Obtaining all in-transit batch tasks from the in-transit batch task cache of the faulty data center, and migrating the in-transit batch tasks to the other data centers using a hierarchical migration strategy based on the task levels corresponding to the in-transit batch tasks, wherein the in-transit batch tasks refer to the batch tasks being executed before the faulty data center fails; When it is detected that all the unexecuted batch tasks and all the in-transit batch tasks in the faulty data center have been migrated, the faulty data center is isolated and it is confirmed that the task evacuation request has been executed.

2. The evacuation method according to claim 1, characterized in that: The step of adopting the load balancing strategy to migrate the unexecuted batch tasks to the other data center includes: Analyze the application stickiness of all applications involved in the data center, where the application stickiness includes: data flow direction and data traffic information, transmission link information, and dependent hardware and software resource information of the application; Based on the stickiness of the applications, the applications are classified into different categories; Evaluate each of the other data centers to determine the processing capacity of the other data centers, wherein the processing capacity includes: computing capacity, storage capacity, and network status; Based on the application stickiness of the application and the processing capacity of the data center, specifying an evacuation order and a target data center for each batch task, wherein the batch tasks of the same type of application have the same target data center; Based on the evacuation sequence and the target data center, all unexecuted batch tasks in the failed data center are migrated to the other data centers.

3. The evacuation method according to claim 2, characterized in that: The step of adopting the load balancing strategy to migrate the unexecuted batch tasks to the other data center also includes: Receive parameter configuration input from external terminal; When it is detected that the parameter configuration contains balanced migration parameters specified for any of the batch tasks, an evacuation order and a target data center are specified for each of the batch tasks based on the application stickiness of the application, the processing capacity of the data center, and all the balanced migration parameters, and all unexecuted batch tasks in the faulty data center are migrated.

4. The evacuation method according to claim 1, characterized in that: The step of migrating the in-transit batch tasks to the other data center by adopting the hierarchical migration strategy includes: Receiving parameter configuration input from an external terminal, wherein the parameter configuration at least includes: a task level of the in-transit batch task, the task level including an application level and a priority of the in-transit batch task; Based on the task level, the load balancing strategy is adopted to migrate the in-transit batch tasks to the corresponding target data center.

5. The evacuation method according to claim 1, characterized in that: After migrating the in-transit batch tasks to the other data center, the method further includes: The other data centers receive the in-transit batch tasks from the failed data center, and synchronize the received in-transit batch tasks to the in-transit batch task cache.

6. The evacuation method according to claim 1, characterized in that: After the task evacuation request is executed, the method further includes: The other data centers execute the received batch tasks that have not been executed by the faulty data center and the in-transit batch tasks; The other data centers save the executed batch tasks to the historical batch database.

7. The evacuation method according to claim 1, characterized in that: Before responding to the mission evacuation request, it also includes: Monitor all data centers, and if a failure occurs in the data center, initiate a task evacuation request for the failed data center; or, When it is detected that any data center is in a load imbalance, a task evacuation request for the data center is initiated, wherein the load imbalance refers to that the data center is in an overloaded or idle state.

8. A batch task evacuation device across data centers, characterized in that: include: A response unit, used to respond to the task evacuation request and suspend the task execution operation of the failed data center; A first migration unit is used to obtain all unexecuted batch tasks of the faulty data center, and migrate the unexecuted batch tasks to other data centers using a load balancing strategy; a second migration unit, configured to obtain all in-transit batch tasks from the in-transit batch task cache of the faulty data center, and migrate the in-transit batch tasks to the other data centers using a hierarchical migration strategy based on task levels corresponding to the in-transit batch tasks, wherein the in-transit batch tasks refer to batch tasks being executed before the faulty data center fails; The isolation unit is used to isolate the faulty data center and confirm that the task evacuation request has been executed when it is detected that all the unexecuted batch tasks and all the in-transit batch tasks in the faulty data center have been migrated.

9. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the batch task evacuation method across data centers as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the cross-data center batch task evacuation method described in any one of claims 1 to 7 are implemented.

Images