Task execution method, device and equipment for heterogeneous cluster hybrid scheduling and disaster recovery, and storage medium

Abstract

The application discloses a task execution method and device for heterogeneous cluster hybrid scheduling and disaster recovery, equipment and a storage medium, relates to the technical field of continuous integration, and comprises the following steps: acquiring a continuous integration task, an architecture type label corresponding to the continuous integration task, a plurality of deployed available build clusters, and a running state of a scheduling link; according to the architecture type label, distributing the continuous integration task to a corresponding logical task queue to obtain an isolated task; according to the architecture type label of the isolated task, determining a target build cluster from the available build clusters; distributing the isolated task to the target build cluster to obtain a distributed task; according to the running state, performing disaster recovery configuration on the distributed task to obtain a disaster recovery configuration result, and executing the distributed task according to the disaster recovery configuration result. The application avoids task interruption due to cluster abnormalities and guarantees continuous execution of continuous integration tasks.

Description

Technical Field

[0001] This application relates to the field of continuous integration technology, and in particular to a task execution method, apparatus, device, and storage medium for heterogeneous cluster hybrid scheduling and disaster recovery. Background Technology

[0002] Continuous integration requires developers to frequently merge code changes into the main branch and verify these changes through automated builds and tests to identify integration errors as early as possible. As enterprise software development scales up, the number and complexity of continuous integration tasks continue to rise, placing higher demands on the execution capabilities and stability of continuous integration systems.

[0003] The execution of continuous integration tasks relies on the computing resources and execution environment provided by the build cluster. Processors of different architectures differ fundamentally in instruction sets, execution efficiency, and ecosystem support; a build cluster based on a single architecture cannot meet the execution requirements of continuous integration tasks across all architecture types. Therefore, in practice, enterprises typically deploy build clusters of multiple architecture types, such as simultaneously deploying build clusters based on x86 architecture and build clusters based on ARM (Advanced RISC Machine) architecture, with each architecture-specific build cluster handling continuous integration tasks for its corresponding architecture type.

[0004] In heterogeneous cluster environments, multiple build clusters with different architectures run simultaneously, jointly undertaking the execution of continuous integration tasks. However, existing technologies, when allocating continuous integration tasks to build clusters, rely solely on preset static rules (such as round-robin or fixed weights) to select target clusters, lacking real-time awareness of the current operational status of the build clusters. When a build cluster becomes unable to execute tasks normally due to failure or excessive load, the scheduler fails to detect this abnormal state and continues to allocate new continuous integration tasks to the failed cluster. Consequently, these newly allocated tasks cannot be executed due to cluster unavailability, resulting in task interruption.

[0005] Therefore, the current solution has a technical problem where continuous integration tasks are easily interrupted due to abnormal situations when executing continuous integration tasks in a heterogeneous cluster environment. Summary of the Invention

[0006] The main purpose of this application is to provide a method, apparatus, device and storage medium for task execution with heterogeneous cluster hybrid scheduling and disaster recovery, which aims to solve the technical problem that tasks are easily interrupted due to abnormal conditions when executing continuous integration tasks in a heterogeneous cluster environment.

[0007] To achieve the above objectives, this application proposes a task execution method for heterogeneous cluster hybrid scheduling and disaster recovery, the task execution method for heterogeneous cluster hybrid scheduling and disaster recovery including: Obtain the continuous integration task, the architecture type label corresponding to the continuous integration task, the running status of multiple deployed available build clusters, and the scheduling link; Based on the architecture type label, the continuous integration task is assigned to the corresponding logical task queue to obtain isolated tasks; Based on the architecture type label of the isolation task, determine the target build cluster from the available build clusters; The isolated task is assigned to the target construction cluster to obtain the assigned task; Based on the operating status, disaster recovery configuration is performed on the assigned tasks to obtain the disaster recovery configuration result, and the assigned tasks are executed according to the disaster recovery configuration result.

[0008] In one embodiment, the step of configuring disaster recovery for the assigned tasks based on the running state to obtain a disaster recovery configuration result, and then executing the assigned tasks according to the disaster recovery configuration result, includes: Obtain the independent message queue, independent thread pool, cluster running status, middleware running status, and state machine running status from the running status; When the cluster is in an abnormal running state, the allocated tasks are migrated according to the available cluster construction to obtain the first configuration result; When the middleware is in a failed state, the allocated tasks are taken over by the available cluster to obtain a second configuration result. Based on the state machine's running state, the assigned tasks are processed through the independent message queue and the independent thread pool to obtain a third configuration result. Based on at least one of the first configuration result, the second configuration result, and the third configuration result, a disaster recovery configuration result is determined, and the assigned task is executed according to the disaster recovery configuration result.

[0009] In one embodiment, when the middleware is in a failed running state, obtaining a second configuration result by having the available cluster take over the assigned tasks includes: When the middleware is in a failed state, obtain the weight configuration of each available cluster to be built; Based on the weight configuration, a backup scheduling strategy is determined, and based on the backup scheduling strategy, a candidate building cluster is determined from the available building clusters; The assigned tasks are then distributed to the candidate building clusters to obtain the second configuration result.

[0010] In one embodiment, the step of processing the assigned tasks through the independent message queue and the independent thread pool according to the state machine's running state to obtain a third configuration result includes: Based on the state machine's running state, obtain the state machine events for each available cluster to be built; The state machine events are assigned to the corresponding independent message queues to obtain independent state machine events; The independent thread pool processes the independent state machine events to obtain the third configuration result.

[0011] In one embodiment, the step of allocating the continuous integration task to the corresponding logical task queue according to the architecture type label to obtain isolated tasks includes: Based on the architecture type label, determine the corresponding logical task queue; Obtain the trigger switch and rate limiting threshold corresponding to the logical task queue; When the trigger switch is in the on state, the continuous integration task is assigned to the logical task queue according to the rate limiting threshold to obtain an isolated task.

[0012] In one embodiment, determining the target build cluster from the available build clusters based on the architecture type label of the isolated task includes: Based on the architecture type label of the isolation task, determine the candidate build clusters that match the architecture type label from the available build clusters; Obtain the weight factors for each of the candidate clusters; The candidate building cluster with the largest weight factor is selected from each of the candidate building clusters as the target building cluster.

[0013] In one embodiment, obtaining the continuous integration task, the architecture type tag corresponding to the continuous integration task, the running status of multiple deployed available build clusters, and the scheduling link includes: Obtain the continuous integration request, and parse the continuous integration task and the architecture type tag corresponding to the continuous integration task from the continuous integration request; Obtain multiple deployed and available build clusters, and continuously obtain heartbeat information for each of the available build clusters, wherein the heartbeat information includes load status, health status, and availability zone identifier; When the health status is normal, the corresponding cluster running status of the available cluster to be built is marked as normal; When the health status is abnormal or the heartbeat information is not obtained within a preset time, the corresponding available cluster running status for building a cluster will be marked as abnormal. Obtain the middleware running status and the state machine running status, and use the cluster running status, the middleware running status, and the state machine running status as the running status of the scheduling link.

[0014] Furthermore, to achieve the above objectives, this application also proposes a heterogeneous cluster hybrid scheduling and disaster recovery task execution device, which includes: The data acquisition module is used to acquire the continuous integration task, the architecture type label corresponding to the continuous integration task, the running status of multiple deployed available build clusters, and the scheduling link; The task isolation module is used to allocate the continuous integration task to the corresponding logical task queue according to the architecture type label, so as to obtain isolated tasks; A cluster determination module is used to determine a target build cluster from the available build clusters based on the architecture type label of the isolation task; The task allocation module is used to allocate the isolated tasks to the target construction cluster to obtain the allocated tasks; The disaster recovery configuration module is used to perform disaster recovery configuration on the assigned tasks according to the running status, obtain the disaster recovery configuration result, and execute the assigned tasks according to the disaster recovery configuration result.

[0015] Furthermore, to achieve the above objectives, this application also proposes a task execution device for heterogeneous cluster hybrid scheduling and disaster recovery. The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the task execution method for heterogeneous cluster hybrid scheduling and disaster recovery as described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method described above.

[0017] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method described above.

[0018] One or more technical solutions proposed in this application have at least the following technical effects: This application, after assigning tasks to the target build cluster, does not simply wait for the tasks to complete. Instead, it further acquires the operational status of multiple components, including the build cluster, middleware, and state machine, and configures disaster recovery for the assigned tasks based on this acquired status. This disaster recovery configuration ensures that tasks, even after being assigned to the build cluster, still possess the ability to handle various abnormal situations during execution, preventing task interruption due to abnormal execution environments. Therefore, tasks can continue to execute until completion after being assigned to the build cluster, solving the technical problem of task interruption in multi-cluster environments in existing technologies. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating an embodiment of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method of this application. Figure 2 This is a schematic diagram of task queue splitting and independent rate limiting provided in an embodiment of the task execution method for heterogeneous cluster hybrid scheduling and disaster recovery in this application; Figure 3 This is a flowchart illustrating the second embodiment of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method of this application; Figure 4 This is a schematic diagram of cluster state management and state machine transition provided in an embodiment of the task execution method for heterogeneous cluster hybrid scheduling and disaster recovery in this application; Figure 5 A schematic diagram of asynchronous thread isolation and message queue architecture provided in an embodiment of the task execution method for heterogeneous cluster hybrid scheduling and disaster recovery in this application; Figure 6 A flowchart of load balancing and disaster recovery degradation decision-making provided in an embodiment of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method of this application; Figure 7 This is a schematic diagram of the module structure of the task execution device for heterogeneous cluster hybrid scheduling and disaster recovery in an embodiment of this application; Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the task execution method of heterogeneous cluster hybrid scheduling and disaster recovery in the embodiments of this application.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0025] The main solution of this application embodiment is as follows: Obtain the continuous integration task, the architecture type tag corresponding to the continuous integration task, the running status of multiple deployed available build clusters and scheduling links; allocate the continuous integration task to the corresponding logical task queue according to the architecture type tag to obtain isolated tasks; determine the target build cluster from the available build clusters according to the architecture type tag of the isolated tasks; allocate the isolated tasks to the target build cluster to obtain allocated tasks; perform disaster recovery configuration on the allocated tasks according to the running status to obtain disaster recovery configuration results, and execute the allocated tasks according to the disaster recovery configuration results.

[0026] Because existing technologies assign continuous integration tasks to the cluster for direct execution, there is a lack of safeguards for the task execution process. When cluster anomalies or middleware failures occur during task execution, the task is interrupted due to the unavailability of the execution environment.

[0027] This application provides a solution where, after a task is assigned to the target cluster, it does not immediately enter an execution waiting state. Instead, it further acquires the running status of the scheduling link and performs disaster recovery configuration on the assigned task based on the running status before executing the assigned task. Because the assigned task has undergone disaster recovery configuration before execution, it has the ability to cope with various abnormal situations during execution and will not be interrupted due to cluster anomalies or middleware failures, ensuring that the entire process of the assigned task from start to finish is uninterrupted.

[0028] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or processor capable of performing the above functions. The following description uses a processor as an example to illustrate this embodiment and the subsequent embodiments. All actions involving the acquisition of signals, information, or data in this application are performed in accordance with the relevant data protection regulations of the locality and with authorization from the owner of the corresponding device.

[0029] Based on this, embodiments of this application provide a task execution method for heterogeneous cluster hybrid scheduling and disaster recovery, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the task execution method for heterogeneous cluster hybrid scheduling and disaster recovery in this application.

[0030] In this embodiment, the heterogeneous cluster hybrid scheduling and disaster recovery task execution method includes steps S10~S50: Step S10: Obtain the continuous integration task, the architecture type label corresponding to the continuous integration task, the running status of multiple deployed available build clusters and scheduling links.

[0031] It should be noted that continuous integration tasks refer to automated build, compilation, and testing tasks triggered by code changes during software development. The architecture type tag identifies the processor architecture type required for the continuous integration task, including x86 and ARM architectures. The build cluster refers to the server cluster that provides computing resources and an execution environment for the continuous integration task. The scheduling chain refers to the various stages a continuous integration task goes through from entering the system to completion, including task queuing, task scheduling, task allocation, and task execution.

[0032] In one feasible implementation, step S10 may include: obtaining a continuous integration request, parsing a continuous integration task and an architecture type tag corresponding to the continuous integration task from the continuous integration request; obtaining multiple deployed available build clusters, and continuously obtaining heartbeat information of each available build cluster, wherein the heartbeat information includes load status, health status, and availability zone identifier; when the health status is normal, marking the cluster operation status of the corresponding available build cluster as normal; when the health status is abnormal or the heartbeat information is not obtained within a preset time, marking the cluster operation status of the corresponding available build cluster as abnormal; obtaining the middleware operation status and the state machine operation status, and using the cluster operation status, the middleware operation status, and the state machine operation status as the operation status of the scheduling link.

[0033] It should be noted that the cluster running status indicates whether each build cluster is currently available, including both normal and abnormal states. The middleware running status indicates whether the middleware that the scheduling service depends on (such as Redis (remote dictionary service)) is working properly. The state machine running status indicates whether the state machine event processing of each build cluster is normal. The running status of the scheduling link is composed of the above three states.

[0034] Specifically, continuous integration requests are received via the API gateway, originating from webhooks in the code repository or scheduled build triggers. The continuous integration task to be executed is parsed from the HTTP (Hypertext Transfer Protocol) request body, and the architecture type tag corresponding to the continuous integration task is extracted from the metadata fields of the request body. If the request does not explicitly carry an architecture type tag, the value of the architecture field is read from the Manifest file of the base image associated with the task to automatically infer and complete the architecture type tag.

[0035] At startup, an initial list of multiple deployed and available build clusters is loaded from the configuration center or database cluster registry. This list includes the name, cluster type, CPU (Central Processing Unit) architecture type, API Server (Application Programming Interface) address, authentication credentials, and availability zone identifier for each build cluster. This initial list is maintained in a local cache, and registry change events are monitored in real-time to dynamically update the list of multiple available build clusters.

[0036] The cluster registry is a relational database table with fields including cluster name, cluster type (OPENSHIFT or KUBERNETES), CPU architecture type (X86, C86, or ARM), API Server address, authentication credentials (stored in encrypted form), availability zone identifier, cluster status (ONLINE or OFFLINE), most recent heartbeat timestamp, current load value, and weight factor. When the scheduling service starts, it reads all cluster records with an ONLINE status from the cluster registry and constructs an initial list of available clusters. When operations personnel add or remove clusters through the low-code platform, the platform automatically updates the records in the cluster registry and sends a change notification to the scheduling service. Upon receiving the notification, the scheduling service reloads the cluster list, enabling dynamic cluster awareness and updates.

[0037] For each of the multiple available clusters, a separate health check coroutine is launched. The coroutine sends HTTP GET requests to the corresponding cluster's health check endpoint at fixed time intervals to retrieve heartbeat information. The heartbeat information includes load status, health status, and availability zone identifier. If the HTTP request returns a 200 status code within a preset timeout period and the response body indicates health, the heartbeat is considered successful; otherwise, it is considered a failure.

[0038] A sliding time window is maintained for each cluster. When the health status is normal (i.e., multiple consecutive heartbeats are successful within the sliding time window), the corresponding available cluster is marked as normal, and the cluster's most recent heartbeat timestamp is updated. When the health status is abnormal (i.e., multiple consecutive heartbeats fail, or no heartbeat information is obtained within a preset time), the corresponding available cluster is marked as abnormal, and a cluster offline alarm is triggered.

[0039] Send a probe command to the middleware to obtain its running status. If the probe command returns a normal response within a preset timeout period, the middleware running status is marked as normal; if the probe command times out or returns an abnormal response, the middleware running status is marked as failed.

[0040] The state machine's running status is obtained through the dedicated thread pool manager for each cluster. The monitoring interface of each thread pool is called to obtain the current number of active threads, the thread pool queue depth, and the average execution time of the most recent task. If the number of active threads does not exceed the maximum thread count threshold and the queue depth is less than a preset threshold, the state machine's running status is marked as normal; otherwise, it is marked as overloaded.

[0041] The cluster running status, middleware running status, and state machine running status are encapsulated into a state snapshot object, which serves as the running status of the scheduling link.

[0042] Understandably, existing technical solutions lack a mechanism for unified perception and comprehensive evaluation of the operational status of the building cluster, middleware, and state machine executors. When any link fails, the scheduling layer cannot obtain fault information in a timely manner, leading to tasks being incorrectly assigned to unavailable clusters or relying on faulty middleware to make scheduling decisions, which in turn causes global task blocking or allocation failure. Therefore, step S10, by continuously acquiring heartbeat information from each building cluster and simultaneously acquiring the middleware and state machine operational status, integrates these three into a unified operational status of the scheduling link. This avoids the scheduler making incorrect allocation decisions due to missing information in abnormal scenarios such as cluster failure, middleware failure, or state machine overload, thereby improving the accuracy of scheduling decisions and the overall disaster recovery response speed of the system, and providing an accurate data foundation for subsequent disaster recovery configuration.

[0043] Step S20: Based on the architecture type label, assign the continuous integration task to the corresponding logical task queue to obtain isolated tasks.

[0044] It's important to note that a logical task queue is a virtual queue divided according to processor architecture. Continuous integration tasks of different architecture types enter different logical task queues, and each logical task queue contains only continuous integration tasks with the same architecture type label. Logical task queues are implemented using Redis's List data structure or RabbitMQ's Queue component, and the queue name includes the architecture type identifier. An isolated task is a continuous integration task that, after being assigned to a logical task queue, is isolated from other tasks with different architecture type labels. These isolated tasks are queued independently in the logical task queue and are not affected by blocking or failures of other architecture type tasks.

[0045] In one feasible implementation, step S20 may include: determining the corresponding logical task queue according to the architecture type label; obtaining the trigger switch and rate limiting threshold corresponding to the logical task queue; and when the trigger switch is in the on state, allocating the continuous integration task to the logical task queue according to the rate limiting threshold to obtain isolated tasks.

[0046] It should be noted that the trigger switch is a Boolean configuration item used to control whether the logical task queue is allowed to perform dequeue operations, with a value of either on or off. The rate limiting threshold is a positive integer configuration item used to constrain the maximum number of tasks retrieved from the logical task queue in a single trigger. The rate limiting threshold is configured independently for logical task queues of different architecture types and in different environments.

[0047] Specifically, after parsing the architecture type tag from the HTTP request body, the pre-configured queue routing map is queried based on the architecture type tag to determine the corresponding logical task queue. The queue routing map maintains the correspondence between architecture type tags and logical task queue names; for example, a tag of X86 routes to `queue:x86:build`, a tag of C86 routes to `queue:c86:build`, and a tag of ARM routes to `queue:arm:build`. After determining the logical task queue, the trigger switch and rate limiting threshold corresponding to that logical task queue are obtained from the configuration center. The rate limiting threshold is configured differently based on the pipeline environment, architecture type, and business type. For example, for a logical task queue with a pipeline environment of DEV and an architecture type of X86, the maximum number of dequeues is configured to 100; for a logical task queue with a pipeline environment of ST and an architecture type of X86, the maximum number of dequeues is configured to 75; and for a logical task queue with a pipeline environment of UAT and an architecture type of ARM, the maximum number of dequeues is configured to 30. All trigger switches for the logical task queues mentioned above are enabled. This configuration can be dynamically adjusted based on the system's current operating status. When system resources are strained or a cluster of a certain architecture type fails, operations personnel can use the configuration center to lower the rate limiting threshold for the corresponding queue or disable the trigger switch to achieve rapid rate limiting and prevent task backlog from causing system overload.

[0048] The system checks the current state of the trigger switch. If the switch is off, the logical task queue is skipped, and no task is triggered this time. If the switch is on, the queue's batch dequeue operation is invoked, popping no more than the rate-limiting threshold of continuous integration tasks from the head of the queue and marking the popped tasks as isolated tasks. Isolated tasks have the same architecture type label as their queue, ensuring that they are only assigned to target build clusters matching that label during subsequent scheduling. The isolated tasks are then submitted in batches to the adaptation and conversion layer. This layer acts as an anti-corruption layer, decoupling the application-layer dequeue model from the scheduling layer's routing distribution, supporting rapid integration with new clusters. When a new cluster needs to be added, only the cluster's architecture type label, API Server address, and authentication credentials need to be configured in the adaptation and conversion layer. No modification to the application-layer dequeue logic or the scheduling layer's routing logic is required; the scheduling layer then performs the subsequent target cluster selection operation based on the isolated task's architecture type label.

[0049] Understandably, by assigning continuous integration tasks to corresponding logical task queues according to architecture type tags, and configuring independent trigger switches and rate limiting thresholds for each logical task queue, scheduling isolation between tasks of different architecture types is achieved. This avoids the impact of task queue blocking of one architecture type or cluster failure on the normal triggering of tasks of other architecture types, thereby improving the robustness of task scheduling in heterogeneous cluster environments and providing a task isolation foundation for subsequent target cluster construction based on architecture type.

[0050] Step S30: Determine the target build cluster from the available build clusters based on the architecture type label of the isolation task.

[0051] In one feasible implementation, step S30 may include: determining candidate build clusters that match the architecture type label from the available build clusters based on the architecture type label of the isolation task; obtaining the weight factor of each candidate build cluster; and selecting the candidate build cluster with the largest weight factor from each candidate build cluster as the target build cluster.

[0052] It should be noted that the weight factor is a numerical value used to measure the selection priority of each candidate cluster during task scheduling. The weight factor includes an initial static default weight and a dynamic current weight. The initial static default weight is manually configured and adjusted through the low-code platform. Operators can manually intervene and adjust the initial static default weight based on production traffic, cluster anomalies, and cluster canary upgrades. The weight factor is a dynamically changing current weight value. Each candidate cluster corresponds to one weight factor; the larger the weight factor, the higher the selection priority of the cluster in the current scheduling cycle. The weight factor is stored in an ordered set in Redis, existing in the form of a score, and is dynamically updated after each task scheduling using a smoothed weighted round-robin algorithm.

[0053] Specifically, after obtaining the isolated task, the architecture type tag is extracted from the isolated task's metadata. The list of available build clusters is traversed; tasks corresponding to the x86 architecture type tag are distributed to the x86 build cluster, and tasks corresponding to the ARM architecture type tag are distributed to the ARM build cluster. Clusters matching the architecture type tag are filtered based on their pre-defined CPU architecture types, selecting them as candidate build clusters. If no candidate build clusters are available, a heterogeneous compatibility degradation strategy is triggered, attempting to assign the task to a compatible simulator node or returning a task failure message.

[0054] After identifying candidate clusters, the target cluster is determined from these candidates using SWRR (Smooth Weighted Round Robin). SWRR stores the current weights of each node in a Redis sorted set and uses Lua scripts to ensure atomicity.

[0055] Define the candidate cluster node set as .

[0056] The static default weight of each node is .

[0057] The dynamic current weight of each node is .

[0058] The sum of the static weights of all nodes is .

[0059] The scheduler executes Lua scripts, with Redis serving as the single source of truth for the smooth weighted round-robin algorithm, storing the current weights of each candidate cluster. Redis uses sorted sets to store nodes and their current weights. The `ZREVRANGE` command retrieves the node with the highest score, the `ZINCRBY` command updates the weights, and the `ZRANGEBYSCORE` command (query by score range) retrieves a set of nodes within a specified weight range for auxiliary status verification or weight distribution queries. When the Lua script performs status verification, it uses the `ZRANGEBYSCORE` command to retrieve nodes in the current sorted set whose weights fall within a preset range, comparing them with the passed-in candidate cluster list to determine if the cluster list has changed. This perfectly aligns with Redis's data structure characteristics. Multiple sorted set key-value pairs achieve load balancing isolation for heterogeneous clusters across multiple availability zones; clusters of different architecture types and availability zones are stored in different sorted set keys. The Lua script has built-in status verification and initialization logic, automatically detecting node online / offline status or weight changes, avoiding frequent external system calls to Redis for cleanup and rebuilding, reducing network overhead and logical complexity.

[0060] First, check if the sorted set Key for the load balancing group exists in Redis, or if the number of nodes in the Key matches the passed-in candidate cluster list. If the Key does not exist or the node list has changed, delete the old Key and reinitialize it, setting the Score of each node to its respective static default weight. .

[0061] Next, select the node with the largest current weight. Use the ZREVRANGE command to obtain the member with the largest score; this member's score represents the node with the largest current weight. This member is the candidate with the highest current weight for building the cluster. .

[0062] Then, select the node Score minus total weight Execute the ZINCRBY command, that is:

[0063] Next, iterate through all candidate cluster building nodes. Add the corresponding static default weight to the Score of each node. Perform ZINCRBY on each node, that is:

[0064] Finally, return The target cluster is selected as the target cluster. If multiple candidate clusters with the same maximum weight factor exist, one of them is selected as the target cluster according to the lexicographical order of the cluster names or a preset priority rule.

[0065] Understandably, by filtering candidate clusters with matching architectures from available build clusters based on the architecture type label of the isolated task, it is ensured that tasks are only assigned to build clusters that support their instruction set architecture, avoiding task execution failures due to architecture mismatch. By obtaining the weight factors of each candidate cluster stored in a Redis sorted set and selecting the cluster with the largest weight factor as the target build cluster, combined with the dynamic weight update mechanism of the smooth weighted round-robin algorithm, load balancing can be achieved in a multi-availability zone heterogeneous cluster environment. This allows high-weight clusters to handle more scheduling traffic, while subsequent weight deduction and accumulation operations prevent high-weight clusters from being selected consecutively, thus achieving a uniform distribution of traffic over time.

[0066] Step S40: Assign the isolated task to the target building cluster to obtain the assigned task.

[0067] It should be noted that an assigned task refers to a continuous integration task that has been assigned to the target build cluster and is waiting or is being executed in the target build cluster. The assigned task includes metadata information such as task identifier, architecture type label, target build cluster identifier, assignment timestamp, and task status.

[0068] Specifically, after identifying the target build cluster, the isolation task is submitted to the target build cluster's API Server via an HTTP POST request. Upon receiving the request, the API Server writes the task to the cluster's internal build queue and initializes the task status to "submitted." Once the target cluster confirms that the task has been successfully enqueued, the isolation task is persisted to the task database, and the task status is updated to "assigned," thus obtaining the assigned task.

[0069] Understandably, by assigning isolated tasks to the target build cluster and generating assigned tasks, the continuous integration task complete flow from scheduling decision to execution is realized. The target build cluster identifier recorded in the assigned tasks provides a basis for subsequent task execution, status monitoring and fault migration. When the target cluster fails, the cluster identifier in the assigned tasks can be used to quickly locate and trigger the task migration process.

[0070] Step S50: Based on the running status, perform disaster recovery configuration on the assigned tasks to obtain the disaster recovery configuration result, and execute the assigned tasks according to the disaster recovery configuration result.

[0071] It should be noted that disaster recovery configuration refers to a combination of fault response strategies triggered to ensure the continued execution of assigned tasks when anomalies occur at different stages of the scheduling chain. The disaster recovery configuration is dynamically determined based on the operational status of the scheduling chain and includes at least one of cluster migration configuration, scheduling degradation configuration, and state machine isolation configuration. The disaster recovery configuration result refers to the final execution plan obtained after applying the disaster recovery configuration to the assigned tasks. The disaster recovery configuration result includes the target execution cluster of the task, the scheduling algorithm type, and the state machine processing mode.

[0072] Specifically, a corresponding disaster recovery configuration result is generated based on the operational status, and the assigned tasks are executed according to the disaster recovery configuration result. When the cluster operational status included in the scheduling link is abnormal, the assigned tasks are migrated from the abnormal cluster to another available cluster with the same architecture and normal status. When the cluster operational status is normal but the middleware operational status is faulty, the scheduling algorithm is switched from smooth weighted round-robin that depends on the middleware to a backup scheduling algorithm that does not depend on the middleware. The target cluster is re-determined and the assigned tasks are executed through the backup scheduling algorithm. When both the cluster operational status and the middleware operational status are normal but the state machine operational status is overloaded, the state machine processing mode of the assigned tasks is switched from synchronous processing to asynchronous isolation processing through independent message queues and independent thread pools. When there are multiple abnormal types in the operational status of the scheduling link, the cluster migration configuration, the scheduling degradation configuration, and the state machine isolation configuration are combined and superimposed according to a preset priority order to generate a disaster recovery configuration result containing multiple layers of disaster recovery measures. The assigned tasks are executed according to the target execution cluster, scheduling algorithm type, and state machine processing mode specified in the disaster recovery configuration result.

[0073] Understandably, by dynamically generating disaster recovery configuration results based on the running status of the scheduling link, the system can automatically trigger corresponding disaster recovery measures under different abnormal scenarios such as cluster failure, middleware failure, or state machine overload, avoiding the global blocking or execution interruption of tasks due to a single point of failure, thereby improving the high availability and fault self-healing capability of the continuous integration scheduling system in heterogeneous cluster environments.

[0074] This embodiment provides a task execution method for hybrid scheduling and disaster recovery in heterogeneous clusters. By acquiring the running status of the scheduling link and configuring disaster recovery for the assigned tasks, it solves the technical problem of task interruption caused by the scheduler's lack of real-time awareness of the running status of the cluster, which leads to the assignment of tasks to faulty clusters. It achieves the technical effect of automatically triggering disaster recovery configuration based on the running status and avoiding task interruption due to cluster anomalies. Furthermore, it achieves heterogeneous task isolation by allocating logical task queues according to architecture type tags, load balancing by selecting the candidate cluster with the largest weight factor using a smooth weighted round-robin algorithm, flow control for task dequeueing by configuring rate limiting thresholds, and state machine isolation by handling state machine events through independent message queues and independent thread pools. These measures collectively improve the stability and high availability of continuous integration task scheduling in heterogeneous cluster environments.

[0075] For example, to help understand the implementation process of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method obtained by combining the above embodiments, please refer to... Figure 2 , Figure 2 A task queue splitting and independent rate limiting diagram is provided for a task execution method that integrates heterogeneous cluster hybrid scheduling and disaster recovery. Specifically: Continuous integration tasks are enqueued from various triggers, including regular build triggers, dispatch pipeline triggers, change triggers, OPEN API calls, proxy build triggers, and step triggers. Continuous integration tasks generated by these triggers are all included in the task enqueuing process.

[0076] During the task queuing process, the scheduling service allocates tasks to the corresponding logical task queues based on the architecture type tag carried by each continuous integration task. Specifically, tasks with the architecture type tag "x86" are assigned to the x86 task queue, tasks with the architecture type tag "C86" are assigned to the C86 task queue, and tasks with the architecture type tag "ARM" are assigned to the ARM task queue. These logical task queues are isolated from each other, and tasks of different architecture types are queued independently in their respective queues without affecting each other.

[0077] Each logical task queue is configured with an independent rate-limiting threshold to constrain the maximum number of tasks retrieved from that queue in a single trigger. For example... Figure 2 As shown, the single-trigger scheduling quantity for the X86 task queue is configured to 75, for the C86 task queue to 50, and for the ARM task queue to 30. The scheduling service pops tasks in batches from each logical task queue according to the rate limiting threshold of each queue.

[0078] After a task is dequeued, the isolated task is submitted to the adaptation and conversion layer. The scheduling layer then performs the subsequent target cluster selection operation based on the architecture type label of each task, and finally distributes it to the corresponding build cluster for execution.

[0079] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Step S50 may include steps S51 to S55: Step S51: Obtain the independent message queue, independent thread pool, cluster running status, middleware running status, and state machine running status from the running status.

[0080] It should be noted that an independent message queue refers to a message queue created separately for each build cluster. Message queues in different clusters are isolated from each other, and the queue name includes the cluster identifier. They adopt a self-produced and self-consumed strategy. An independent thread pool refers to a thread pool initialized separately for each build cluster. Thread pools in different clusters are isolated from each other, and each thread pool has its own independent core thread count, maximum thread count, task queue, and rejection policy.

[0081] Specifically, the system loads pre-created independent message queue information for each build cluster from the configuration center or local cache. This independent message queue information includes the queue name, switch name, routing key, and connection factory configuration. It retrieves an independent thread pool instance initialized for each build cluster from the thread pool manager. In a specific configuration example, for an x86 architecture build cluster (resource specifications: 8-core CPU, 32GB memory), the independent thread pool is configured with 10 core threads, 50 maximum threads, a task queue capacity of 2000, and a rejection policy of CallerRunsPolicy. For an ARM architecture build cluster (resource specifications: 4-core CPU, 16GB memory), the independent thread pool is configured with 5 core threads, 25 maximum threads, a task queue capacity of 1000, and a rejection policy of CallerRunsPolicy. The rejection policy using CallerRunsPolicy ensures that when the thread pool task queue is full and the number of threads reaches the maximum, the caller thread executes the task synchronously, utilizing the execution time of the caller thread to form a natural rate limiting, preventing task loss. Each independent thread pool instance includes the core thread count, maximum thread count, task queue capacity, rejection policy, and current running metrics. The cluster running status, middleware running status, and state machine running status are extracted from the running status of the scheduling chain.

[0082] Understandably, existing technical solutions lack pre-prepared independent message queues and thread pools for scheduling services, and do not uniformly acquire and integrate the operating status of the cluster, middleware, and state machine. When disaster recovery measures need to be triggered, available isolation resources cannot be quickly obtained, and the current operating status of each component cannot be accurately perceived. This results in a lack of resource foundation and status basis for the execution of disaster recovery measures, affecting the timeliness and accuracy of disaster recovery response. Therefore, performing step S51 to acquire independent message queues, independent thread pools, and the operating status of the cluster, middleware, and state machine can prevent disaster recovery measures from failing to execute or executing incorrectly due to unprepared isolation resources or missing operating status information when disaster recovery configuration is triggered, thereby improving the resource readiness and status awareness capabilities of disaster recovery configuration.

[0083] Step S52: When the cluster is in an abnormal running state, migrate the assigned tasks according to the available cluster to obtain the first configuration result.

[0084] It should be noted that the first configuration result refers to the configuration information generated after applying the cluster migration strategy, including the target execution cluster identifier after migration, task reallocation status, and migration timestamp. Cluster lifecycle management refers to introducing a unified state management mechanism for each built cluster. The states include ONLINE, OFFLINE, and IDLE. A unified monitoring module manages the lifecycle states of heterogeneous clusters, improving cluster availability and disaster recovery capabilities. When the cluster state is ONLINE, the cluster normally receives and executes tasks; when the cluster state is OFFLINE, the cluster does not receive new tasks, and already assigned tasks trigger migration; when the cluster state is IDLE, the cluster retains tasks but does not actively assign new tasks, used for canary deployment verification or rolling upgrade scenarios.

[0085] Specifically, when the cluster's operating status is marked as abnormal, a cluster migration strategy is triggered. Abnormal cluster operating status includes the following situations: multiple consecutive heartbeat failures, failure to obtain heartbeat information within a preset time, the cluster health check endpoint returning a status code other than 200, the cluster API Server being unreachable, or the cluster actively reporting an OFFLINE status.

[0086] After triggering the cluster migration policy, the system filters other available clusters with the same architecture type tag as the abnormal cluster and whose cluster operation status is normal from the cluster registry. Filtering criteria include: cluster status is ONLINE, cluster health check passed, and current cluster load does not exceed a preset threshold. Clusters located in the same availability zone as the abnormal cluster are prioritized to reduce cross-availability zone network latency. If no available clusters meeting the criteria exist within the same availability zone, the filtering scope is expanded to available clusters with the same architecture type but located in different availability zones, and the cluster with the lowest load status is selected as the migration target cluster.

[0087] After identifying the target cluster for migration, withdraw the assigned tasks from the build queue of the faulty cluster. The withdrawal operation includes: calling the API Server of the faulty cluster to cancel the task queue; querying the task database for all task records in that cluster that are either assigned or submitted but not yet executed; and resetting the task status to pending assignment. Resubmit the withdrawn tasks to the API Server of the target cluster, including the task identifier, architecture type tag, task timeout, and priority in the submission request. Once the target cluster confirms successful task enqueueing, update the target build cluster identifier in the assigned tasks to the target cluster identifier, update the assignment timestamp to the migration completion time, and update the task status to assigned. If enqueueing to the target cluster fails during the migration process, record the migration failure event and trigger an alarm, while simultaneously attempting to assign the task to the next available cluster with the lowest load.

[0088] Cluster lifecycle management is implemented through a state machine. The states of each cluster include ONLINE, OFFLINE, and IDLE. The transition rules between states are as follows: When a cluster registers online for the first time or passes a health check after a fault recovery, its state switches from OFFLINE to ONLINE. While in ONLINE, if multiple heartbeat failures occur or no heartbeat information is received within a preset time, the cluster state switches from ONLINE to OFFLINE. When operations personnel mark the cluster as maintenance mode through the configuration center, the cluster state switches from ONLINE to IDLE. In this state, the cluster does not accept new tasks, and existing tasks continue to execute until completion. After all tasks are completed, the cluster enters the IDLE state, which is used for canary or rolling upgrade scenarios.

[0089] When the cluster state changes, the scheduling service performs the following operations: when switching from ONLINE to OFFLINE, the cluster migration policy is triggered; when switching from OFFLINE to ONLINE, the cluster is added back to the list of available build clusters and task reception is resumed; when switching from ONLINE to IDLE, the scheduling service stops assigning new tasks to the cluster, but does not trigger the migration of already assigned tasks, allowing existing tasks to complete naturally.

[0090] The configuration information, including the target cluster identifier, task redistribution status, and migration timestamp, will be used as the first configuration result.

[0091] Understandably, in existing technical solutions, when a build cluster fails, the tasks assigned to that cluster lack an automatic migration mechanism. This causes tasks to be blocked in the queue of the failed cluster for extended periods until they time out and fail, requiring manual intervention to recover, resulting in task execution interruptions and decreased service availability. Therefore, step S52, which automatically migrates assigned tasks to other available build clusters with the same architecture and normal status when the cluster is in an abnormal operating state, can avoid prolonged task blocking and delays in manual intervention caused by a single cluster failure, thereby improving the system's automatic fault recovery capability and task execution success rate.

[0092] Step S53: When the middleware is in a failed state, the available cluster takes over the assigned tasks to obtain a second configuration result.

[0093] It should be noted that the second configuration result refers to the configuration information generated after applying the scheduling degradation strategy, which includes the backup scheduling strategy type, the redefined target cluster identifier, and the allocation timestamp.

[0094] In one feasible implementation, step S53 may include: when the middleware running state is invalid, obtaining the weight configuration of each available build cluster; determining a backup scheduling strategy according to the weight configuration, and determining a candidate build cluster from the available build clusters according to the backup scheduling strategy; assigning the assigned task to the candidate build cluster to obtain a second configuration result.

[0095] It should be noted that the weight configuration in the backup scheduling strategy is a dynamic weight for each available cluster, calculated by querying the real-time idle resource quantity of each cluster in the relational database, and does not rely on any middleware storage. The backup scheduling strategy refers to a scheduling algorithm that does not rely on middleware and only relies on the capabilities of the relational database itself to select clusters. It is used to replace the smooth weighted round-robin algorithm that depends on middleware when the middleware fails.

[0096] Specifically, when the middleware's running status is marked as failed, a scheduling degradation strategy is triggered, switching the scheduling algorithm from a middleware-dependent smooth weighted round-robin to a middleware-independent backup scheduling strategy. A middleware running status of failed includes the following scenarios: a probe command sent to the middleware does not return a normal response within a preset timeout period, the middleware returns an abnormal response, the middleware connection times out, or the middleware actively reports unavailability. After triggering the scheduling degradation strategy, the backup scheduling strategy uses a SQL (Structured Query Language) weighted random algorithm. The backup scheduling strategy can also use a round-robin algorithm based on local caching. The scheduling service maintains a static weight list of each available building cluster in local memory. When the middleware fails, the scheduling service reads the static weights of each cluster from the local cache, generates a round-robin sequence according to the weight ratio, and selects clusters sequentially as candidate building clusters. This implementation does not rely on database queries and is suitable for scenarios with small cluster sizes and infrequent weight changes.

[0097] First, query the master_pool table (host pool), filter all records with an IDLE status, group by cluster name and count the number of idle Masters for each cluster, and use the number of idle Masters for each cluster as the dynamic weight of that cluster. The more idle Masters, the higher the weight.

[0098] Secondly, the `ORDER BY weight DESC LIMIT 1` command is executed. After sorting by dynamic weight from highest to lowest, the first record is selected, and the cluster with the highest weight is chosen as the candidate cluster for construction. This strategy has a negative feedback self-balancing characteristic: when a cluster is selected and assigned a task, the number of idle Masters in that cluster decreases. In the next query, the dynamic weight of that cluster automatically drops back, and the probability of other clusters being selected increases accordingly, automatically leading to a balance among the clusters.

[0099] Then, after locking the selected candidate clusters, execute ORDER BY RAND LIMIT 1. Sort the selected clusters in random order and take the first record. Randomly select an idle Master in the cluster to avoid always hitting the same Pod (container instance) and achieve Pod-level traffic distribution.

[0100] After determining the candidate cluster to build, the assigned tasks are distributed to that cluster to obtain the second configuration result.

[0101] Understandably, because existing scheduling algorithms heavily rely on middleware to store weight information and execute atomic operations, when the middleware fails, the scheduling service cannot obtain cluster weights or perform weighted selection, leading to an interruption in the task allocation chain and preventing all new tasks from being scheduled. Therefore, step S53 switches to an SQL-based weighted random algorithm when the middleware is in a failed state. This leverages the database's own capabilities to take over task allocation when the middleware is unavailable, preventing a complete interruption of the scheduling chain due to a single point of failure in the middleware and ensuring the continuous availability of the allocation chain. Furthermore, the dynamic weight calculation and negative feedback self-balancing characteristics of the SQL weighted random algorithm automatically balance the load across the cluster without manual intervention.

[0102] Step S54: Based on the state machine's running state, process the assigned tasks through the independent message queue and the independent thread pool to obtain a third configuration result.

[0103] It should be noted that the third configuration result refers to the configuration information generated after applying the state machine isolation strategy, which includes the state machine event handling mode, the corresponding independent message queue name, and the independent thread pool configuration.

[0104] In one feasible implementation, step S54 may include: obtaining state machine events for each available cluster based on the state machine running state; allocating the state machine events to the corresponding independent message queues to obtain independent state machine events; and processing the independent state machine events through the independent thread pool to obtain a third configuration result.

[0105] It should be noted that state machine events refer to lifecycle events generated during the execution of continuous integration tasks in the build cluster, including one or more of the following events: ADJUST_POOL_SIZE (adjust pool size), CHECK_MASTER_HEALTH (check Master health), CLEAR_JENKINS_MASTER (clean up Jenkins Master), CREATING_NEW_MASTER (create a new Master), POLLING_CREATING_MASTER (polling for Masters under creation), POLLING_TO_REUSE_MASTER (polling for Master reuse), RECYCLE_ABANDONED_MASTER (reclaim abandoned Masters), RECYCLE_LEAKED_MASTER (reclaim leaked Masters), REFRESH_MASTER_COMPUTER (refresh Master computer nodes), and SAMPLE_ACTIVE_JENKINS (sample active Jenkins instances). Independent state machine events refer to events formed after state machine events are assigned to independent message queues of the corresponding cluster. Each independent message queue only receives and processes state machine events of its own cluster.

[0106] Specifically, before the scheduling service was modified, the scheduling layer introduced a shared thread pool to manage the Master state machine executor, enabling asynchronous processing of state machine tasks. As platform build traffic increased and the number of connected build clusters grew, the scheduling service's creation, polling, and destruction operations on the Master surged. When the total resource pool exceeded 1200, the execution efficiency of Master creation, polling, and destruction significantly decreased, and single-cluster failures could cause state machine threads to block, impacting pipeline builds. Therefore, when the state machine's running state is marked as overloaded, a state machine isolation strategy is triggered. The state machine's running state being overloaded includes the following situations: the number of active threads exceeds the maximum thread count threshold, the thread pool queue depth is greater than a preset threshold, the average task execution time exceeds a preset time, or multiple consecutive probes mark it as overloaded.

[0107] After triggering the state machine isolation policy, an independent message queue is created for each build cluster. The queue name includes the cluster identifier, and the independent message queues of each build cluster are isolated from the independent message queues of other clusters. The system receives scheduled tasks and generates state machine events for each available build cluster. These state machine events include one or more of the following: creating a Master, polling the creation progress, checking Master health, refreshing Master nodes, reclaiming an obsolete Master, reclaiming a leaked Master, and sampling an active Master. The generated state machine events are distributed to the dedicated independent message queue for that cluster according to the cluster identifier corresponding to the event. Each independent message queue only receives and processes state machine events for its own cluster.

[0108] Each cluster is initialized with an independent thread pool. Each independent thread pool has its own core thread count configuration, maximum thread count configuration, task queue capacity configuration, and rejection policy configuration. The independent thread pools of different clusters are isolated from each other. Consumers of each cluster listen to the independent message queue dedicated to that cluster. When an independent state machine event exists in the independent message queue, the consumer pulls the independent state machine event and submits it to the independent thread pool of that cluster. The independent thread pool allocates threads to execute the operation corresponding to the state machine event.

[0109] When the state machine's running state is marked as overloaded, a synchronous degradation configuration is also triggered. A task type degradation switch is configured; this switch is a boolean configuration item used to control whether state machine events of a certain type skip the message queue and are directly executed synchronously by the local thread pool. The degradation switch is configured independently at the state machine event type level. When the degradation switch for a certain state machine event type is set to the enabled state, state machine events of that type are no longer delivered to the independent message queue, but are directly sent to the corresponding cluster's independent thread pool for synchronous execution.

[0110] When a consumption anomaly occurs during the execution of a state machine event, the unified logging platform collects error logs, configures corresponding alarm policies for these error logs, and sends alarm messages to operations and maintenance personnel when the log volume reaches a preset threshold. The state machine event execution scheme, after asynchronous isolation processing via independent message queues and independent thread pools, is used as the third configuration result.

[0111] Furthermore, independent message queues employ a self-produced, self-consumed strategy. The scheduling service acts as the producer, delivering state machine events to the queue. The same scheduling service instance acts as the consumer, pulling events from the queue and processing them, eliminating the need for additional message forwarding components. When message consumption fails, the unified log platform collects error logs and sends an alert to operations personnel. Operations personnel can quickly switch the degradation switch for the corresponding task type through the configuration center, downgrading the state machine event processing mode from asynchronous message queue mode to synchronous thread pool mode, achieving second-level failover. Independent message queues can also be implemented using Kafka's (distributed messaging system) Topic component. Each cluster corresponds to an independent Topic, with the Topic name containing the cluster identifier. The scheduling service, as the producer, sends state machine events to the corresponding cluster's Topic, and consumers in each cluster pull events from their respective Topics for processing.

[0112] Understandably, because in existing technical solutions all clusters share the same thread pool for state machine tasks, when a single cluster's state machine task is blocked or the thread pool resources are exhausted, the state machine processing of all clusters is affected, leading to global blocking. Therefore, step S54, by using independent message queues and independent thread pools to achieve isolated processing of state machine events across different clusters, can prevent the blocking or overload of a single cluster's state machine task from affecting the state machine processing of other clusters. Simultaneously, a degradation switch enables rapid switching in case of message queue anomalies, and combined with log alerts, real-time monitoring of the state machine's running status is achieved, thereby improving the high availability of the resource state machine.

[0113] Step S55: Determine the disaster recovery configuration result based on at least one of the first configuration result, the second configuration result, and the third configuration result, so as to execute the assigned task according to the disaster recovery configuration result.

[0114] Specifically, the first configuration result, the second configuration result, and the third configuration result are combined and superimposed according to a preset priority order to generate the disaster recovery configuration result.

[0115] First, determine if the first, second, and third configuration results exist. If the first configuration result exists, it indicates that the assigned tasks have been migrated from the abnormal cluster to the migration target cluster, and the target execution cluster for the assigned tasks has been redefined as the migration target cluster. If the second configuration result exists, it indicates that the scheduling algorithm has been switched from smooth weighted round-robin with middleware dependency to a backup scheduling strategy, and the assigned tasks have been reassigned to the candidate construction cluster through the backup scheduling strategy. The target execution cluster for the assigned tasks has been redefined as the candidate construction cluster, and the scheduling algorithm type has been updated to the backup scheduling strategy. If the third configuration result exists, it indicates that the state machine event handling mode has been switched to asynchronous isolation processing through independent message queues and independent thread pools, and the state machine handling mode has been updated to asynchronous isolation mode.

[0116] Secondly, the above configuration results are combined and superimposed according to a preset priority order. The preset priority order is: cluster migration strategy takes precedence over scheduling degradation strategy, and scheduling degradation strategy takes precedence over state machine isolation strategy. When both the first and second configuration results exist, the target execution cluster for the assigned tasks is the migration target cluster specified in the first configuration result, and the scheduling algorithm type is the backup scheduling strategy specified in the second configuration result. When both the first and third configuration results exist, the target execution cluster for the assigned tasks is the migration target cluster specified in the first configuration result, and the state machine processing mode is the asynchronous isolation mode specified in the third configuration result. When both the second and third configuration results exist, the scheduling algorithm type is the backup scheduling strategy specified in the second configuration result, and the state machine processing mode is the asynchronous isolation mode specified in the third configuration result. When all three configuration results exist, the target execution cluster for the assigned tasks is the migration target cluster specified in the first configuration result, the scheduling algorithm type is the backup scheduling strategy specified in the second configuration result, and the state machine processing mode is the asynchronous isolation mode specified in the third configuration result. When a certain configuration result does not exist, the corresponding configuration item retains its original configuration.

[0117] The final execution scheme generated after combining and overlaying is used as the disaster recovery configuration result. The disaster recovery configuration result includes the target execution cluster of the assigned tasks, the scheduling algorithm type, and the state machine processing mode.

[0118] Finally, the assigned tasks are executed according to the target execution cluster, scheduling algorithm type, and state machine processing mode specified in the disaster recovery configuration results.

[0119] It is understandable that by combining and superimposing at least one of the first configuration result, the second configuration result, and the third configuration result to generate the disaster recovery configuration result, the system can comprehensively apply multi-layer disaster recovery measures when multiple types of anomalies occur simultaneously. This avoids the limitation that a single disaster recovery strategy cannot cope with complex fault scenarios, thereby fully ensuring the continuous availability of the scheduling link under various abnormal scenarios.

[0120] This embodiment provides a task execution method for heterogeneous cluster hybrid scheduling and disaster recovery. By obtaining the cluster running status, middleware running status, and state machine running status from the running status of independent message queues, independent thread pools, and scheduling links, the method migrates the assigned tasks to other available building clusters with the same architecture when the cluster running status is abnormal. When the middleware running status is in failure, the method takes over task allocation through an SQL weighted random algorithm. When the state machine running status is overloaded, the method asynchronously isolates state machine events through independent message queues and independent thread pools. The method combines at least one of the above configuration results to generate a disaster recovery configuration result to execute the assigned tasks. This solves the technical problems in existing solutions where tasks cannot be automatically migrated when the building cluster fails, the scheduling link is completely interrupted when the middleware fails, and the state machine task single-point blocking leads to global blocking. It achieves automatic task migration and recovery within minutes when the cluster fails, seamless degradation and continuous availability of the scheduling link when the middleware fails, and cross-cluster isolation processing of state machine events without mutual interference.

[0121] For example, to help understand the implementation process of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method obtained by combining the above embodiments, please refer to... Figure 4 , Figure 4 A schematic diagram of cluster state management and state machine transition is provided for a task execution method with heterogeneous cluster hybrid scheduling and disaster recovery. Specifically: The cluster construction introduces a unified state management mechanism, including READY, ONLINE, RECYCLE, and OFFLINE states. In each state, the cluster supports executing corresponding state machine events, including creating a Master, polling the Master, destroying the Master, and scheduling the Master.

[0122] In cluster lifecycle management, the transition path between states is as follows: After the cluster completes initialization preparation, it switches from the READY state to the ONLINE state through an online operation; when the cluster is in the ONLINE state and running normally, it continuously receives and executes tasks; when the cluster is triggered by idle resource reclamation, it switches from the ONLINE state to the RECYCLE state; when the cluster experiences a failure, heartbeat loss, or actively goes offline, it switches from the ONLINE state or the RECYCLE state to the OFFLINE state.

[0123] In each state, the cluster executes the corresponding lifecycle events through a state machine. Specifically, the Create Master event is used to create a new Master node in the cluster; the Poll Master event is used to query the creation progress and running status of the Master node; the Destroy Master event is used to reclaim or release Master nodes that are no longer in use; and the Schedule Master event is used to assign tasks to specific Master nodes for execution.

[0124] For example, to help understand the implementation process of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method obtained by combining the above embodiments, please refer to... Figure 5 , Figure 5 A schematic diagram of an asynchronous thread isolation and message queue architecture is provided for a heterogeneous cluster hybrid scheduling and disaster recovery task execution method. Specifically: The scheduling service, acting as the producer of state machine events, internally contains multiple scheduled tasks, including a scheduled task for creating executors, a scheduled task for creating pollers, a scheduled task for reusing pollers, and a scheduled task for destroying executors. Each scheduled task generates corresponding state machine events through an API interface and delivers the generated state machine events to the corresponding independent message queue of the cluster being built.

[0125] At the message queue level, each cluster has one or more dedicated, independent message queues. Specifically, the X86-ocp01 cluster corresponds to X86-ocp01-queue, the X86-ocp02 cluster corresponds to X86-ocp02-queue, the Arm-k8s01 cluster corresponds to arm-k8s01-queue, and the Arm-k8s02 cluster corresponds to arm-k8s02-queue. These independent message queues are isolated from each other, and each queue stores multiple state machine events or tasks. Each independent message queue is configured with a corresponding consumer, which continuously listens for events in that queue.

[0126] At the cluster execution level, each build cluster listens to its own dedicated independent message queue. The X86-ocp01 and X86-ocp02 clusters are build clusters for the X86 architecture, while the Arm-k8s01 and Arm-k8s02 clusters are build clusters for the ARM architecture. Each cluster contains multiple Master nodes. When a state machine event exists in a consumer's independent message queue, the consumer pulls the state machine event and submits it to the corresponding cluster's independent thread pool for execution.

[0127] The producer of the scheduling service distributes state machine events to the corresponding independent message queues according to the cluster identifier. Consumers in each cluster only consume events in their own cluster's queue, and message queues and thread pools between different clusters are completely isolated.

[0128] For example, to help understand the implementation process of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method obtained by combining the above embodiments, please refer to... Figure 6 , Figure 6 A flowchart illustrating the load balancing and disaster recovery degradation decision-making process for a heterogeneous cluster hybrid scheduling and disaster recovery task execution method is provided, specifically: Retrieve the currently configured traffic mode, which includes manual mode and automatic mode. If the current traffic mode is manual mode, continue to determine whether the Redis cluster is accessible normally; if the current traffic mode is not manual mode, directly jump to the SQL fallback path.

[0129] If the current traffic pattern is manual, send a probe command to the Redis cluster to determine if the Redis cluster is accessible. If the Redis cluster is accessible, follow the normal path and execute the weighted round-robin algorithm based on Redis sorted sets for weight distribution; if the Redis cluster is accessing abnormally, redirect to the SQL fallback path.

[0130] Under normal Redis cluster access conditions, a smooth weighted round-robin algorithm based on Redis sorted sets is executed. The candidate cluster with the highest current weight is obtained atomically through Lua script, and the candidate cluster is locked as the cluster with the highest weight.

[0131] Obtain the execution result of the smooth weighted round-robin. If the execution is successful, lock the cluster with the highest weight and allocate a host instance within that cluster; if the execution fails, i.e., a smooth weighted round-robin execution exception is caught, then jump to the SQL degradation path.

[0132] In the SQL degradation path, the SQL weighted random algorithm is executed, which consists of three stages.

[0133] First, query the host pool table, filter all records that are in an idle state, group them by cluster name, count the number of idle hosts in each cluster, and use the statistical results as the dynamic weight of each cluster. The more idle hosts a cluster has, the higher its weight.

[0134] Secondly, after sorting the records by dynamic weight from high to low, the first record is selected, and the cluster with the highest weight is chosen as the candidate cluster. This strategy has a negative feedback self-balancing characteristic: when a cluster is selected and a host is allocated to it, the number of idle hosts in that cluster decreases, and the dynamic weight of that cluster automatically drops in the next query, while the probability of other clusters being selected increases accordingly, and the clusters automatically tend to reach equilibrium.

[0135] Finally, after locking in the selected candidate clusters, the first record is taken after being sorted in random order within the cluster, and an idle host is randomly selected to avoid always hitting the same container instance, thus achieving container instance-level traffic distribution.

[0136] After three stages of processing using the SQL weighted random algorithm, the selected host instance is returned.

[0137] Regardless of whether the normal path or the SQL fallback path is used, the final business caller obtains the selected host instance without needing to know whether the underlying path is the normal path or the fallback path, as the results returned by the two paths are consistent.

[0138] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the task execution method of heterogeneous cluster hybrid scheduling and disaster recovery in this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0139] This application also provides a task execution device for heterogeneous cluster hybrid scheduling and disaster recovery. Please refer to [link / reference]. Figure 7 The heterogeneous cluster hybrid scheduling and disaster recovery task execution device includes: The data acquisition module 10 is used to acquire the continuous integration task, the architecture type label corresponding to the continuous integration task, the running status of multiple deployed available build clusters and scheduling links; The task isolation module 20 is used to allocate the continuous integration task to the corresponding logical task queue according to the architecture type label, so as to obtain isolated tasks; Cluster determination module 30 is used to determine a target build cluster from the available build clusters based on the architecture type label of the isolated task; Task allocation module 40 is used to allocate the isolated task to the target construction cluster to obtain the allocated task; The disaster recovery configuration module 50 is used to perform disaster recovery configuration on the assigned tasks according to the running status, obtain the disaster recovery configuration result, and execute the assigned tasks according to the disaster recovery configuration result.

[0140] The heterogeneous cluster hybrid scheduling and disaster recovery task execution device provided in this application, employing the heterogeneous cluster hybrid scheduling and disaster recovery task execution method in the above embodiments, can solve the technical problem that tasks are easily interrupted due to abnormal conditions during continuous integration tasks in a heterogeneous cluster environment. Compared with the prior art, the beneficial effects of the heterogeneous cluster hybrid scheduling and disaster recovery task execution device provided in this application are the same as those of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method provided in the above embodiments, and other technical features in the heterogeneous cluster hybrid scheduling and disaster recovery task execution device are the same as those disclosed in the above embodiments, and will not be repeated here.

[0141] In one embodiment, the disaster recovery configuration module 50 is further configured to obtain independent message queues, independent thread pools, cluster running status, middleware running status, and state machine running status in the running status. When the cluster is in an abnormal running state, the allocated tasks are migrated according to the available cluster construction to obtain the first configuration result; When the middleware is in a failed state, the allocated tasks are taken over by the available cluster to obtain a second configuration result. Based on the state machine's running state, the assigned tasks are processed through the independent message queue and the independent thread pool to obtain a third configuration result. Based on at least one of the first configuration result, the second configuration result, and the third configuration result, a disaster recovery configuration result is determined, and the assigned task is executed according to the disaster recovery configuration result.

[0142] In one embodiment, the disaster recovery configuration module 50 is further configured to obtain the weight configuration of each available building cluster when the middleware is in a failed running state. Based on the weight configuration, a backup scheduling strategy is determined, and based on the backup scheduling strategy, a candidate building cluster is determined from the available building clusters; The assigned tasks are then distributed to the candidate building clusters to obtain the second configuration result.

[0143] In one embodiment, the disaster recovery configuration module 50 is further configured to obtain state machine events for each available cluster based on the state machine's operating state; The state machine events are assigned to the corresponding independent message queues to obtain independent state machine events; The independent thread pool processes the independent state machine events to obtain the third configuration result.

[0144] In one embodiment, the task isolation module 20 is further configured to determine the corresponding logical task queue based on the architecture type label; Obtain the trigger switch and rate limiting threshold corresponding to the logical task queue; When the trigger switch is in the on state, the continuous integration task is assigned to the logical task queue according to the rate limiting threshold to obtain an isolated task.

[0145] In one embodiment, the cluster determination module 30 is further configured to determine, based on the architecture type label of the isolation task, a candidate build cluster that matches the architecture type label from the available build clusters; Obtain the weight factors for each of the candidate clusters; The candidate building cluster with the largest weight factor is selected from each of the candidate building clusters as the target building cluster.

[0146] In one embodiment, the data acquisition module 10 is further configured to acquire a continuous integration request and parse the continuous integration task and the architecture type tag corresponding to the continuous integration task from the continuous integration request; Obtain multiple deployed and available build clusters, and continuously obtain heartbeat information for each of the available build clusters, wherein the heartbeat information includes load status, health status, and availability zone identifier; When the health status is normal, the corresponding cluster running status of the available cluster to be built is marked as normal; When the health status is abnormal or the heartbeat information is not obtained within a preset time, the corresponding available cluster running status for building a cluster will be marked as abnormal. Obtain the middleware running status and the state machine running status, and use the cluster running status, the middleware running status, and the state machine running status as the running status of the scheduling link.

[0147] This application provides a heterogeneous cluster hybrid scheduling and disaster recovery task execution device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the heterogeneous cluster hybrid scheduling and disaster recovery task execution method in the above embodiment 1.

[0148] The following is for reference. Figure 8 This document illustrates a structural diagram of a task execution device suitable for implementing heterogeneous cluster hybrid scheduling and disaster recovery in the embodiments of this application. The task execution device for heterogeneous cluster hybrid scheduling and disaster recovery in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The heterogeneous cluster hybrid scheduling and disaster recovery task execution device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0149] like Figure 8As shown, the heterogeneous cluster hybrid scheduling and disaster recovery task execution device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in ROM (Read Only Memory) 1002 or the program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the heterogeneous cluster hybrid scheduling and disaster recovery task execution device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the heterogeneous cluster hybrid scheduling and disaster recovery task execution equipment to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a heterogeneous cluster hybrid scheduling and disaster recovery task execution equipment with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0150] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0151] The heterogeneous cluster hybrid scheduling and disaster recovery task execution device provided in this application, employing the heterogeneous cluster hybrid scheduling and disaster recovery task execution method in the above embodiments, can solve the technical problem that tasks are easily interrupted due to abnormal conditions during continuous integration tasks in a heterogeneous cluster environment. Compared with the prior art, the beneficial effects of the heterogeneous cluster hybrid scheduling and disaster recovery task execution device provided in this application are the same as those of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method provided in the above embodiments, and other technical features in this heterogeneous cluster hybrid scheduling and disaster recovery task execution device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0152] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0154] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the heterogeneous cluster hybrid scheduling and disaster recovery task execution method in the above embodiments.

[0155] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0156] The aforementioned computer-readable storage medium may be included in the heterogeneous cluster hybrid scheduling and disaster recovery task execution device; or it may exist independently and not be assembled into the heterogeneous cluster hybrid scheduling and disaster recovery task execution device.

[0157] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the heterogeneous cluster hybrid scheduling and disaster recovery task execution device, the heterogeneous cluster hybrid scheduling and disaster recovery task execution device performs the following actions: It acquires a continuous integration task, the architecture type tag corresponding to the continuous integration task, multiple deployed available build clusters, and the running status of the scheduling link; it allocates the continuous integration task to the corresponding logical task queue according to the architecture type tag, obtaining an isolated task; it determines a target build cluster from the available build clusters according to the architecture type tag of the isolated task; it allocates the isolated task to the target build cluster, obtaining an assigned task; it performs disaster recovery configuration on the assigned task according to the running status, obtaining a disaster recovery configuration result, and then executes the assigned task according to the disaster recovery configuration result.

[0158] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0160] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0161] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described heterogeneous cluster hybrid scheduling and disaster recovery task execution method. This solves the technical problem that tasks are easily interrupted due to abnormal conditions during continuous integration execution in a heterogeneous cluster environment. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method provided in the above embodiments, and will not be repeated here.

[0162] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method described above.

[0163] The computer program product provided in this application can solve the technical problem that continuous integration tasks are easily interrupted due to abnormal conditions during execution in a heterogeneous cluster environment. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method provided in the above embodiments, and will not be repeated here.

[0164] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A task execution method for heterogeneous cluster hybrid scheduling and disaster recovery, characterized in that, The method includes: Obtain the continuous integration task, the architecture type label corresponding to the continuous integration task, the running status of multiple deployed available build clusters, and the scheduling link; Based on the architecture type label, the continuous integration task is assigned to the corresponding logical task queue to obtain isolated tasks; Based on the architecture type label of the isolation task, determine the target build cluster from the available build clusters; The isolated task is assigned to the target construction cluster to obtain the assigned task; Based on the operating status, disaster recovery configuration is performed on the assigned tasks to obtain the disaster recovery configuration result, and the assigned tasks are executed according to the disaster recovery configuration result.

2. The method as described in claim 1, characterized in that, The step of configuring disaster recovery for the assigned tasks based on the running status, obtaining a disaster recovery configuration result, and then executing the assigned tasks based on the disaster recovery configuration result includes: Obtain the independent message queue, independent thread pool, cluster running status, middleware running status, and state machine running status from the running status; When the cluster is in an abnormal running state, the allocated tasks are migrated according to the available cluster construction to obtain the first configuration result; When the middleware is in a failed state, the allocated tasks are taken over by the available cluster to obtain a second configuration result. Based on the state machine's running state, the assigned tasks are processed through the independent message queue and the independent thread pool to obtain a third configuration result. Based on at least one of the first configuration result, the second configuration result, and the third configuration result, a disaster recovery configuration result is determined, and the assigned task is executed according to the disaster recovery configuration result.

3. The method as described in claim 2, characterized in that, When the middleware is in a failed state, the second configuration result is obtained by taking over the assigned tasks based on the available cluster, including: When the middleware is in a failed state, obtain the weight configuration of each available cluster to be built; Based on the weight configuration, a backup scheduling strategy is determined, and based on the backup scheduling strategy, a candidate building cluster is determined from the available building clusters; The assigned tasks are then distributed to the candidate building clusters to obtain the second configuration result.

4. The method as described in claim 2, characterized in that, The process of processing the assigned tasks based on the state machine's running state, through the independent message queue and the independent thread pool, to obtain a third configuration result includes: Based on the state machine's running state, obtain the state machine events for each available cluster to be built; The state machine events are assigned to the corresponding independent message queues to obtain independent state machine events; The independent thread pool processes the independent state machine events to obtain the third configuration result.

5. The method as described in claim 1, characterized in that, The step of allocating the continuous integration task to the corresponding logical task queue according to the architecture type label to obtain isolated tasks includes: Based on the architecture type label, determine the corresponding logical task queue; Obtain the trigger switch and rate limiting threshold corresponding to the logical task queue; When the trigger switch is in the on state, the continuous integration task is assigned to the logical task queue according to the rate limiting threshold to obtain an isolated task.

6. The method as described in claim 1, characterized in that, The step of determining the target build cluster from the available build clusters based on the architecture type label of the isolation task includes: Based on the architecture type label of the isolation task, determine the candidate build clusters that match the architecture type label from the available build clusters; Obtain the weight factors for each of the candidate clusters; The candidate building cluster with the largest weight factor is selected from each of the candidate building clusters as the target building cluster.

7. The method as described in claim 1, characterized in that, The acquisition of the continuous integration task, the corresponding architecture type tag of the continuous integration task, the running status of multiple deployed available build clusters and scheduling links includes: Obtain the continuous integration request, and parse the continuous integration task and the architecture type tag corresponding to the continuous integration task from the continuous integration request; Obtain multiple deployed and available build clusters, and continuously obtain heartbeat information for each of the available build clusters, wherein the heartbeat information includes load status, health status, and availability zone identifier; When the health status is normal, the corresponding cluster running status of the available cluster to be built is marked as normal; When the health status is abnormal or the heartbeat information is not obtained within a preset time, the corresponding available cluster running status for building a cluster will be marked as abnormal. Obtain the middleware running status and the state machine running status, and use the cluster running status, the middleware running status, and the state machine running status as the running status of the scheduling link.

8. A task execution device for heterogeneous cluster hybrid scheduling and disaster recovery, characterized in that, The device includes: The data acquisition module is used to acquire the continuous integration task, the architecture type label corresponding to the continuous integration task, the running status of multiple deployed available build clusters, and the scheduling link; The task isolation module is used to allocate the continuous integration task to the corresponding logical task queue according to the architecture type label, so as to obtain isolated tasks; A cluster determination module is used to determine a target build cluster from the available build clusters based on the architecture type label of the isolation task; The task allocation module is used to allocate the isolated tasks to the target construction cluster to obtain the allocated tasks; The disaster recovery configuration module is used to perform disaster recovery configuration on the assigned tasks according to the running status, obtain the disaster recovery configuration result, and execute the assigned tasks according to the disaster recovery configuration result.

9. A task execution device for heterogeneous cluster hybrid scheduling and disaster recovery, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the heterogeneous cluster hybrid scheduling and disaster recovery task execution method as described in any one of claims 1 to 7.

Images