Task scheduling method, task scheduling system, electronic equipment and program product

Abstract

The embodiment of the invention provides a task scheduling method, a task scheduling system, electronic equipment and a program product. Relates to the technical field of computers. The method comprises the following steps: acquiring node state information of each control node in a control node cluster by adopting a preset heartbeat detection mechanism; updating a candidate control node set for task allocation according to the node state information of the control nodes; receiving a to-be-processed task, and obtaining operation state information of each control node in the candidate control node set; determining a target control node through a preset scheduling strategy according to the to-be-processed task and the operation state information of the control node; and sending the to-be-processed task to the target control node, so that the target control node schedules the service nodes in the preset service node cluster to execute the to-be-processed task. According to the method, the technical effects of improving the node availability and improving the scheduling success rate are achieved through state perception of multiple control nodes, dynamic maintenance of the candidate set and a strategy based on the task and the running state.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task scheduling method, a task scheduling system, an electronic device, and a program product. Background Technology

[0002] A continuous integration platform is a general-purpose infrastructure for building and managing automated pipelines, automating the entire process after code submission, including building, automated testing, packaging, static code analysis, and release orchestration.

[0003] In existing technologies, continuous integration platforms, such as Jenkins, typically employ a master-slave architecture. The master node is responsible for environment management and task scheduling, while the slave nodes are responsible for executing the actual build and testing tasks. The process is as follows: the master node receives task requests triggered externally or periodically, selects a target slave node according to a preset strategy, and issues instructions such as build, automated testing, documentation generation, packaging, and static code analysis to that slave node. The slave node executes these instructions and sends the results and logs back to the master node, which then summarizes the task status and presents it externally. To improve concurrency, existing technologies typically increase the number of slave nodes to execute more build tasks in parallel.

[0004] In the current deployment, the master node typically operates as a single node. The reception and distribution of all scheduling requests depend on this master node. When the master node fails or becomes unreachable, newly arriving scheduling requests cannot be accepted and assigned; even if slave nodes remain available, they cannot obtain new tasks due to the lack of scheduling instructions from the control plane, resulting in task interruption and a decrease in overall system availability. Summary of the Invention

[0005] This application provides a task scheduling method, a task scheduling system, an electronic device, and a program product to improve overall availability and scheduling efficiency.

[0006] In a first aspect, embodiments of this application provide a task scheduling method, including:

[0007] Based on the established control node cluster, a preset heartbeat detection mechanism is used to obtain the node status information of each control node in the control node cluster.

[0008] Update the set of candidate control nodes for task allocation based on the node status information of the control nodes;

[0009] Receive the task to be processed and obtain the running status information of each control node in the candidate control node set;

[0010] Based on the task to be processed and the running status information of the control node, the target control node is determined from the set of candidate control nodes through a preset scheduling strategy;

[0011] The task to be processed is sent to the target control node, so that the target control node schedules service nodes in the preset service node cluster to execute the task to be processed.

[0012] In one possible implementation, the step of using a preset heartbeat detection mechanism to obtain the node status information of each control node in the control node cluster includes:

[0013] According to the heartbeat detection cycle, a heartbeat detection request is sent to each control node in the control node cluster, and the heartbeat detection response information of each control node is received.

[0014] If a heartbeat detection response is received from the control node within a preset waiting time, the node status information of the control node is marked as online.

[0015] If no heartbeat response is received from the control node within the preset waiting time, the control node will resend the heartbeat request according to the preset retry strategy and the number of failures will be accumulated. When the accumulated number of failures is greater than or equal to the preset threshold, the node status of the control node will be marked as offline.

[0016] In one possible implementation, it also includes:

[0017] If a heartbeat response is received from the control node in a resend heartbeat request, the failure count of the control node is reset to zero, and the node status of the control node is marked as online.

[0018] In one possible implementation, constructing the control node cluster includes:

[0019] Construct a control node cluster containing at least two control nodes and assign a cluster identifier, and establish an association between the control node cluster and a pre-configured shared database;

[0020] The configuration file is stored in the shared database, and a version identifier is marked on the configuration file;

[0021] The current version identifier of the configuration file is published to the control node cluster so that each control node can read the current version of the configuration file from the shared database based on the version identifier.

[0022] In one possible implementation, updating the candidate control node set for task allocation based on the node state information of the control node includes:

[0023] Read the node status information of each of the control nodes;

[0024] Delete the control nodes marked as offline from the candidate control node set, and obtain the control nodes that are not in the candidate control node set but are marked as online, and add them to the candidate control node set to update the candidate control node set used for task allocation.

[0025] In one possible implementation, determining the target control node from the candidate control node set based on the task to be processed and the running status information of the control node, using a preset scheduling strategy, includes:

[0026] Based on the task to be processed, extract the task attribute information of the task to be processed;

[0027] Based on the task attribute information of the task to be processed, obtain the scheduling scoring rule corresponding to the task attribute information;

[0028] Based on the running status information of each control node in the candidate control node set, and based on the scheduling scoring rules, the scheduling score of the control node is calculated.

[0029] The control node is selected as the target control node based on the scheduling score, which is sorted from high to low according to the score.

[0030] In one possible implementation, it also includes:

[0031] Based on the task processing of the target control node and the service node, task execution records are generated and stored in the shared database.

[0032] Secondly, embodiments of this application provide a task scheduling system, including a task scheduling device, a control node cluster, and a service node cluster;

[0033] The task scheduling device is configured to: acquire node status information of each control node in the pre-built control node cluster using a preset heartbeat detection mechanism; update the candidate control node set for task allocation based on the node status information of the control nodes; receive tasks to be processed and acquire the running status information of each control node in the candidate control node set; determine a target control node from the candidate control node set based on the tasks to be processed and the running status information of the control nodes using a preset scheduling strategy; and send the tasks to be processed to the target control node so that the target control node schedules service nodes in a preset service node cluster to execute the tasks to be processed.

[0034] The control node cluster is used to receive tasks to be processed sent by the task scheduling device, and send scheduling execution instructions to the service node cluster according to the tasks to be processed; wherein, the control node cluster is in a multi-active mode.

[0035] The service node cluster is used to execute the tasks to be processed according to the scheduling execution instructions.

[0036] Thirdly, embodiments of this application provide a task scheduling device, including:

[0037] The status detection module is used to obtain the node status information of each control node in the control node cluster based on the constructed control node cluster and using a preset heartbeat detection mechanism.

[0038] The information update module is used to update the set of candidate control nodes for task allocation based on the node status information of the control nodes.

[0039] The information acquisition module is used to receive the task to be processed and acquire the running status information of each control node in the candidate control node set;

[0040] The node selection module is used to determine the target control node from the set of candidate control nodes based on the task to be processed and the running status information of the control node, through a preset scheduling strategy.

[0041] The scheduling and execution module is used to send the task to be processed to the target control node, so that the target control node can schedule service nodes in the preset service node cluster to execute the task to be processed.

[0042] Fourthly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0043] The memory stores computer-executed instructions;

[0044] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0045] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0046] In a sixth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0047] This application provides a task scheduling method, system, electronic device, and program product. By acquiring the node status information of each control node based on a pre-built control node cluster using a preset heartbeat detection mechanism, and by actively collecting node status, it achieves continuous awareness and timely updates of control node availability, improving the accuracy and timeliness of status determination and providing a reliable node availability foundation for task scheduling. Based on this, the candidate control node set is updated according to the control node status information. By dynamically removing unavailable nodes, it ensures that tasks are not scheduled to failed nodes, achieving dynamic screening and maintenance of scheduling participants and improving the stability of the scheduling link. By acquiring the running status information of each control node in the candidate control node set while receiving tasks to be processed, it achieves state-aware scheduling pre-positioning based on the current system load, making subsequent decisions more aligned with real-time load and reducing the probability of task queuing and resource congestion. By determining the target control node from the candidate control node set based on the running status information of the tasks to be processed and the control nodes, combined with a preset scheduling strategy, it achieves rule-driven comprehensive analysis and optimization, improving load balancing and task acceptance success rate. By sending pending tasks to the target control node to schedule service nodes in a pre-configured service node cluster to execute the tasks, the scheduling and execution are seamlessly integrated, improving resource utilization and enhancing task execution continuity. In summary, through multi-control node state awareness, dynamic maintenance of the candidate set, and strategy optimization based on task and runtime status, the technical effects of improving node availability and scheduling success rate are achieved. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] Figure 1 A flowchart illustrating the task scheduling method provided in this application embodiment;

[0050] Figure 2 This is a schematic diagram of the method for constructing a control node cluster provided in an embodiment of this application;

[0051] Figure 3 This is a schematic diagram of the structure of the task scheduling system provided in the embodiments of this application;

[0052] Figure 4 This is a schematic diagram of the structure of the task scheduling device provided in the embodiments of this application;

[0053] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0054] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0055] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0056] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0057] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0058] The inventive concept of this application aims to solve the problems of insufficient availability and untimely scheduling caused by the single-point deployment of control nodes in existing task scheduling. To this end, this application proposes a task scheduling method that constructs a cluster system composed of multiple control nodes. Through a heartbeat detection mechanism and scheduling strategy in synergy, it achieves high availability, dynamic awareness, and flexible decision-making in the scheduling process. First, by constructing a control node cluster containing multiple control nodes, the limitations of the traditional single-point control architecture are overcome, enabling the entire system to have redundant scheduling capabilities, providing structural guarantees for the continuity and stability of task scheduling. Under this cluster architecture, a preset heartbeat detection mechanism is used to periodically monitor the status of each control node, achieving real-time monitoring of the node's online status. Second, by analyzing the heartbeat detection results, the candidate control node set is dynamically updated, that is, only nodes currently in an available state are included in the scheduling candidate pool, reducing the allocation of tasks to disconnected or faulty nodes, thereby improving the reliability of task scheduling.

[0059] Subsequently, upon receiving the task to be processed, the running status information of the candidate control nodes is further obtained, providing comprehensive input for the scheduling strategy. Based on this, according to the task attributes and node running status, the scheduling suitability of each candidate node is comprehensively analyzed through a preset scheduling strategy to select the target control node. The task is then assigned to the target control node, which further coordinates the specific service nodes in the service node cluster to execute the task, thus realizing the execution of the scheduling instructions. Overall, this application, through its architecture of clustered control node deployment, real-time status monitoring, dynamic maintenance of candidate nodes, and flexible scheduling decisions and execution, forms a highly available and robust task scheduling framework, improving the stability and efficiency of task scheduling. It solves the technical problems of single point of failure and discontinuous task processing existing in traditional scheduling architectures.

[0060] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0061] Figure 1 This is a flowchart illustrating the task scheduling method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0062] S11, based on the established control node cluster, uses a preset heartbeat detection mechanism to obtain the node status information of each control node in the control node cluster.

[0063] In this embodiment, the control node cluster consists of multiple control nodes, which maintain consistency and collaborate on task scheduling through shared configuration and status information. To ensure the continuity and stability of the scheduling process, it is necessary to monitor the operating status of each control node in real time. Therefore, a preset heartbeat detection mechanism is used to detect the status and collect data from each control node in the cluster. The heartbeat detection mechanism is a periodic detection mechanism used to determine whether a control node is in an available state. A heartbeat probe request is sent to each control node through the communication interface, and the control node returns a heartbeat probe response after receiving the request. By comparing the round-trip time of the heartbeat request and response, the latency and reliability of the node communication link can be determined. In summary, by collecting this heartbeat probe response information, real-time monitoring of the availability and stability of control nodes is established, providing a reliable data foundation for the subsequent updating of the candidate control node set and the execution of task scheduling strategies.

[0064] S12, based on the node status information of the control nodes, update the set of candidate control nodes for task allocation.

[0065] In this embodiment, to ensure the stability and high availability of scheduling operations, the control nodes participating in task allocation need to be dynamically screened. Since some nodes in the control node cluster may be temporarily unavailable due to network failures, service anomalies, or system maintenance, including all control nodes in the scheduling selection scope could lead to task scheduling failures, delays, or incorrect assignments. Therefore, by updating the set of candidate control nodes for task allocation based on the node status information of the control nodes, the screening and dynamic maintenance of the control node scheduling capabilities are achieved.

[0066] The candidate control node set is the set of nodes used to undertake task allocation during the scheduling phase. Specifically, firstly, the node status information of each control node obtained by the heartbeat detection mechanism is read. For example, this information typically includes online status flags, response latency, the time of the most recent successful probe, and the count of failures. Subsequently, filtering is performed based on this status information to ensure that only control nodes capable of normally receiving and processing tasks are selected for scheduling.

[0067] To further enhance robustness, a time window for state changes can be incorporated to reduce task migration jitter caused by frequent switching. For example, a cooldown time threshold mechanism can be introduced to temporarily isolate nodes with frequent recent state changes, only re-including them in the candidate set after they have been continuously and stably online for a period of time. To adapt to large-scale control node deployment environments, this step can also adopt a partitioned state maintenance approach, maintaining candidate sets for control nodes in different logical partitions and merging them as needed during scheduling. In summary, this embodiment achieves the selection of a set of candidate nodes suitable for scheduling from all control nodes, improving the stability and accuracy of task scheduling and providing a reliable node foundation for the execution of subsequent scheduling strategies.

[0068] S13, receive the task to be processed and obtain the running status information of each control node in the candidate control node set.

[0069] In this embodiment, a task to be processed is received. This task can be submitted by a user interface, an external triggering system, a CI / CD process, or other upper-layer services, and includes task attributes such as task type, resource requirements, timeliness requirements, and dependency information. After the task is successfully received, the running status information of each control node in the current candidate control node set is obtained. The running status information is dynamic data reflecting the current load and resource status of the control node, including but not limited to: CPU utilization, memory usage, task queue length, task throughput, scheduling response latency, and service availability indicators. Furthermore, the range of collected indicators can be expanded, such as node temperature, hardware health status, and historical stability scores, to achieve more granular scheduling decisions.

[0070] Building upon this foundation, information collection can be achieved through centralized polling or distributed reporting. In centralized polling, operational metrics from each node are periodically retrieved; in distributed reporting, each control node periodically reports its own status via events or timed updates. Status information must be time-sensitive to ensure that the data used in scheduling decisions reflects the latest state. By acquiring the operational status information of each control node in the candidate control node set simultaneously with task reception, a foundation for the linkage between task and node status is established. This allows subsequent scheduling strategies to consider not only node availability but also their current operational load, thereby achieving dynamic resource balancing and rational task allocation. This bidirectional state-driven mechanism improves overall real-time performance, availability, and scheduling rationality.

[0071] S14. Based on the running status information of the task to be processed and the control node, the target control node is determined from the set of candidate control nodes through a preset scheduling strategy.

[0072] In this embodiment, to achieve task scheduling in a multi-control node architecture, the task to be processed is combined with the running status information of the control nodes. A preset scheduling strategy is used to select the most suitable target control node from the candidate control node set for the current task. The basic information of the task to be processed may include task type, resource requirements, task priority, and estimated execution time. The running status information of the control node may include real-time indicators such as the number of tasks currently being processed, CPU and memory usage, and response latency.

[0073] The pre-defined scheduling strategy maps and associates tasks with the running status information of control nodes to determine the target control node capable of handling the task from a set of candidate control nodes. The pre-defined scheduling strategy is a set of rules for task allocation decisions, a decision-making mechanism that comprehensively judges and ranks the attribute information of the tasks to be processed and the running status information of each control node. The scheduling strategy can be flexibly configured according to scale and business needs before deployment. For example, the scheduling strategy can be implemented based on various forms such as weight calculation, scoring sorting, round-robin priority, and minimum load priority; the specific implementation method is not limited to a fixed method. This step, through strategy-driven target node selection, ensures that each scheduling decision matches the task's execution requirements and the control node's carrying capacity to the greatest extent, improving task response efficiency. Through a joint analysis mechanism based on the tasks to be processed and their running status, more accurate control node selection is achieved, improving scheduling flexibility and task processing efficiency, and providing a reliable foundation for the subsequent execution and distribution of service nodes.

[0074] S15, the task to be processed is sent to the target control node so that the target control node can schedule the service nodes in the pre-set service node cluster to execute the task to be processed.

[0075] In this embodiment, the task to be processed is sent to the target control node to trigger the distribution and processing of the task execution flow. The task to be processed may include a task identifier, task attributes, execution parameters, required resource constraints, and execution environment information, and its format and content can be standardized and defined according to the scheduling architecture. The task transmission process is usually implemented through network communication protocols within the cluster, such as message queues, remote calls, or task dispatch interfaces. The specific implementation method can be flexibly configured according to the deployment environment.

[0076] Upon receiving a task, the target control node executes the scheduling process and, based on task requirements and its own logic, selects a suitable service node from a pre-configured service node cluster. The service node cluster is a resource pool composed of multiple nodes capable of task processing. These nodes can execute task code, scripts, or containerized task units issued by the control node. Through collaboration with the service node cluster, the target control node ensures the task is executed, such as performing a build process, running test tasks, or retrieving the compilation environment. By constructing a complete closed loop in the task scheduling chain, effective linkage between control layer decision-making and execution layer processing is achieved, ensuring consistency between the task allocation process and the actual execution process, thereby improving the processing efficiency, execution stability, and resource utilization of task scheduling.

[0077] It should be noted that the control node cluster in this application operates in a multi-active mode, meaning that multiple control nodes within the cluster can simultaneously accept different pending tasks from external sources and initiate their respective scheduling processes. Each control node performs task reception and task execution based on a shared configuration file and a consistent task view. This multi-active parallel processing mechanism improves the throughput and request processing concurrency of the scheduling entry point, reducing response latency caused by queuing at a single control node. Furthermore, when individual control nodes fail or become temporarily unavailable, other control nodes can continue to accept and schedule tasks, forming fault tolerance and degradation capabilities against single-point failures. This ensures the continuity of task scheduling while improving overall availability and stability.

[0078] In one embodiment, in step S11 above, a preset heartbeat detection mechanism is used to obtain the node status information of each control node in the control node cluster, which will be further explained here. Based on the above embodiment, it includes:

[0079] S111, according to the heartbeat detection cycle, send heartbeat detection requests to each control node in the control node cluster, and receive heartbeat detection response information from each control node;

[0080] S112, If a heartbeat detection response from the control node is received within a preset waiting time, the node status information of the control node is marked as online.

[0081] S113. If no heartbeat response information is received from the control node within the preset waiting time, the control node is resent the heartbeat request according to the preset retry strategy and the number of failures is accumulated. When the accumulated number of failures is greater than or equal to the preset threshold, the node status of the control node is marked as offline.

[0082] In this embodiment, a heartbeat detection mechanism is used to periodically probe the status of each control node in the control node cluster. This mechanism is typically triggered periodically based on a preset heartbeat detection cycle, sending heartbeat detection requests to each control node and listening for responses within a set waiting time. Each heartbeat detection request includes a unique identifier and timestamp information and is sent through an internal communication channel (such as a TCP connection, HTTP request, RPC call, etc.). When a control node successfully returns a response within the preset waiting time, it is marked as online, indicating that the node is currently operating normally and can participate in subsequent task scheduling processes.

[0083] If no heartbeat response is received within the waiting time, a retry probe is initiated for the control node according to the preset retry strategy. The retry strategy typically includes the number of retries, the retry interval, and timeout control parameters to ensure a certain degree of tolerance for momentary communication anomalies or minor system jitter. After each failed retry, the number of failures for the control node is accumulated and recorded. When the cumulative number of failures for a control node exceeds or equals a preset threshold (e.g., set to 3 times), the node is determined to be in an unreachable state, and its node status information is marked as offline. This embodiment achieves accurate identification of the online status of control nodes through the combination of periodic probing and retry fault tolerance mechanisms, ensuring that scheduling is always based on the latest and most accurate node status information for task allocation and resource scheduling.

[0084] Next, it also includes: if a heartbeat response is received from the control node in the resending heartbeat request, the failure count of the control node is reset to zero, and the node status of the control node is marked as online.

[0085] In this embodiment, when a control node fails to respond to the initial heartbeat probe request in a timely manner due to network latency or temporary failure, it will perform multiple subsequent probes according to a preset retry strategy to prevent misjudgment as offline. During these retry probes, once a heartbeat probe response is successfully received from the control node in a retry, the node is considered to still be available. In this case, to eliminate the impact of misidentification caused by temporary communication anomalies, the failure count counter of the control node is reset to zero. This operation ensures that the judgment of the node status is not affected by the cumulative deviation of historical failure records, thereby reflecting the current true connection status of the control node. At the same time, the node status of the control node is re-marked as online, ensuring that it can be reinstated into the set of candidate control nodes for task scheduling. This not only avoids the problem of node erroneous removal due to short-term communication anomalies, but also enhances the dynamic repair capability of node status management. By responding to heartbeat retry feedback in real time and resetting the failure status, the automatic recovery of the control node status is achieved, enhancing the robustness to transient network failures and ensuring the availability of the control node cluster and the continuous and stable operation of scheduling.

[0086] Furthermore, the heartbeat detection mechanism can employ a multi-level detection approach, combining TCP connection detection and business-level liveness detection, to comprehensively analyze the availability and service capabilities of the control node.

[0087] Specifically, the network connectivity status of the control node is first determined based on TCP connection detection. This involves proactively sending TCP handshake signals to the control node within a set detection period. If the handshake process is completed within a preset time window, the node's network link is considered to be intact; otherwise, it is considered to be in a preliminary disconnection state. Building upon the basic connection detection, a further business-level liveness detection is performed. Business-level liveness detection involves sending specific business interface call requests (such as status query APIs or internal RPC calls) to the control node. The validity and correctness of the response are verified to determine whether the node can provide scheduling services normally. This detection helps identify abnormal nodes, i.e., control nodes that are network reachable but whose internal services are suspended or threads are blocked. By combining TCP connection detection and business-level liveness detection, a progressive node status awareness mechanism is formed, achieving comprehensive awareness and accurate analysis of the control node's operational status. This improves the accuracy of determining the control node's availability and provides reliable basic information for subsequent candidate node selection and scheduling decisions.

[0088] Figure 2 This is a schematic flowchart illustrating a method for constructing a control node cluster provided in an embodiment of this application. Based on the above embodiments, as follows... Figure 2 As shown, it includes:

[0089] S21, construct a control node cluster containing at least two control nodes and assign a cluster identifier, and establish an association between the control node cluster and a pre-configured shared database;

[0090] S22, store the configuration file in the shared database and mark the version identifier of the configuration file;

[0091] S23, publish the current version identifier of the configuration file to the control node cluster so that each control node can read the current version of the configuration file from the shared database according to the version identifier.

[0092] In this embodiment, under a multi-control node architecture, to achieve consistency in node configuration and unified deployment of task scheduling logic within the cluster, a control node cluster needs to be constructed and its configuration shared synchronously. Specifically, an initialization process is used to construct a control node cluster containing at least two control nodes. Each control node is an independently running scheduling unit capable of collaboratively participating in task reception and distribution. After construction, a unique cluster identifier is assigned to the control node cluster to identify its affiliation during configuration synchronization, task coordination, and other processes, preventing information confusion between clusters.

[0093] Subsequently, the control node cluster is associated with a pre-configured shared database. This shared database serves as the storage and update center for unified cluster configurations, carrying information such as configuration files, scheduling policies, and task execution records, providing consistency support for the entire scheduling process. In terms of configuration management, scheduling logic, task forwarding rules, and service node binding relationships are encapsulated into configuration files and uploaded to the shared database for unified management. To achieve version control of the configurations and consistent node reading, a version identifier is appended to the configuration files during storage. Each change corresponds to a unique version number, supporting subsequent configuration difference comparisons and rollback operations.

[0094] After the control node cluster completes initialization, it pushes the current version configuration identifier to each control node, enabling them to identify the current configuration version to be used. Upon receiving the version identifier, each control node reads the corresponding configuration file from the shared database and updates the parameters and execution logic accordingly, ensuring consistency across the entire control node cluster in terms of configuration version and task scheduling strategy. This embodiment establishes a structural mapping relationship between the control node cluster and the shared database, constructing a configuration management system that supports collaborative work among multiple control nodes. It achieves centralized configuration, distributed reading, and consistent updates of scheduling strategies, thereby enhancing overall scalability and deployment flexibility.

[0095] In one embodiment, the step S12 described above, which involves updating the set of candidate control nodes for task allocation, will be further explained below. Based on the above embodiment, it includes:

[0096] S121, Read the node status information of each control node;

[0097] S122, delete the control nodes marked as offline from the candidate control node set, and obtain the control nodes that are not in the candidate control node set but are marked as online, and add them to the candidate control node set to update the candidate control node set used for task allocation.

[0098] In this embodiment, to ensure the dynamic accuracy of the candidate control node set, it needs to be updated. First, the pre-acquired node status information of each control node is read, and the candidate control node set is compared and maintained based on the current status information. Specifically, the current candidate control node set is traversed, and control nodes marked as offline are removed to avoid selecting unavailable nodes as task execution targets, which could cause task scheduling failure. Simultaneously, control nodes not in the candidate control node set but marked as online are identified from the list of all control nodes and added to the candidate control node set to ensure the integrity and real-time nature of the candidate set. This update mechanism ensures that the candidate control node set on which task scheduling depends always remains consistent with the current cluster operating state. By dynamically removing failed nodes and promptly adding newly online nodes, real-time maintenance of the candidate control node set is achieved, improving the adaptability of scheduling execution to changes in node status and the success rate of task allocation, thereby enhancing overall stability and high availability.

[0099] In one embodiment, the determination of the target control node in step S14 above is provided as an implementation method. Based on the above embodiment, it includes:

[0100] S1411, Extract the task attribute information of the task to be processed based on the task to be processed;

[0101] S1412, Based on the task attribute information of the task to be processed, obtain the scheduling scoring rules corresponding to the task attribute information;

[0102] S1413, Calculate the scheduling score of the control node based on the running status information of each control node in the candidate control node set and the scheduling scoring rules.

[0103] S1414: Sort the nodes according to their scheduling scores from high to low, and select the first node in the sorted list as the target control node.

[0104] In this embodiment, upon receiving a task to be processed, its task attribute information is extracted from the task description. Task attribute information refers to data fields that characterize the task type, execution requirements, or resource dependencies. Common fields include task type (e.g., build, test, deployment), resource requirements (e.g., CPU and memory usage), priority, and project identifier. After extracting the task attribute information, the corresponding scheduling scoring rules are queried based on the attribute information. The scheduling scoring rules are a pre-configured weighted parameter system used to handle the scheduling adaptability of each control node, and can adjust the processing focus of each running status indicator according to the task's requirements. For example, for resource-sensitive tasks, the scoring rules place higher weight on CPU load and memory usage; while for real-time tasks, they may focus more on the number of queued tasks or response latency of the control node.

[0105] Subsequently, the running status information of each control node in the candidate control node set is traversed, and a score is calculated for each node according to the scheduling scoring rules. Running status information typically includes parameters such as node CPU load, memory utilization, current task concurrency, average response latency, and resource availability ratio. These parameters are weighted and summed according to the weights of each indicator in the scoring rules to obtain the scheduling score value for each control node. After the scheduling scores of all control nodes are calculated, they are sorted from highest to lowest score, and the control node with the highest score is selected as the target control node. This node is considered the most suitable scheduler for the current task, possessing good resource availability and task matching. Furthermore, when scheduling scores are tied, priority is given in the following order: shorter queue length, shorter network round-trip latency, and smaller lexicographical order of the control node identifier. This scoring and sorting mechanism enables task attribute-driven control node selection, improving the rationality of scheduling decisions and overall operational efficiency.

[0106] In another embodiment, another method for determining the target control node is provided. Based on the above embodiments, it includes:

[0107] S1421, Based on the current and historical operating status information of each control node, a pre-trained prediction model is used to obtain the load prediction results of each control node.

[0108] S1422, Determine the target control node based on the load prediction results of each control node.

[0109] In this embodiment, to further improve the accuracy and foresight of task scheduling, historical operating status information is introduced as an auxiliary judgment criterion in addition to current operating status information when analyzing candidate control nodes. Historical operating status information includes time-series data of operating status information over multiple past periods, thus reflecting the load trends and periodic changes of the nodes. A pre-trained prediction model is used to analyze and process this operating status data. The prediction model can be a machine learning-based time-series prediction model, such as a Long Short-Term Memory (LSTM) network, a one-dimensional convolutional neural network (1D-CNN), or a gradient boosting tree, to model the load evolution trend of each control node. During the model training phase, a large number of collected historical task scheduling records and operating status samples are used, combined with task attributes and node load responses, to establish a prediction function.

[0110] Based on a set of candidate control nodes, the current operating status information of each control node is collected at a unified time segment. This information is then concatenated with the historical operating status information of each control node in chronological order to form an input sequence. This input sequence is fed into a pre-trained prediction model to obtain the load prediction results for each control node. By setting up a load prediction mechanism based on historical trends, dynamic analysis of the processing capacity of control nodes is achieved, reducing scheduling misjudgments caused by instantaneous index fluctuations. For example, if a control node currently has low CPU utilization, but the trend indicates that it is about to enter a high-load state, the prediction model can reduce the task allocation for that node in advance, improving the foresight and robustness of the overall scheduling strategy. Finally, based on the load prediction results of each control node and combined with the requirements of the current task, the node with the lowest predicted load and the best matching degree is selected as the target control node to complete the task scheduling decision, improving overall stability, task execution efficiency, and resource utilization.

[0111] In one embodiment, the task scheduling method further includes: generating task execution records and storing them in a shared database based on the task processing of the target control node and the service node.

[0112] To achieve unified task tracking and scheduling optimization analysis, corresponding task execution records are generated based on the execution status of the target control node and its scheduled service nodes. The task execution record includes fields such as task identifier, task reception time, target control node identifier, service node identifier, task start time, task completion time, execution status, resource usage, error code, and exception information (if any), forming a structured record format for easy subsequent querying and analysis.

[0113] During the record generation process, the entire execution process of the scheduled tasks is monitored and synchronized with the service nodes to ensure accurate collection of task execution progress and final results. For multi-stage tasks executed in parallel, the intermediate states of the sub-tasks can be broken down into stages, and then aggregated into a complete task execution record upon task completion. For failed or abnormally terminated tasks, the cause of failure, the error code returned by the service node, and the fault stack information are recorded to support fault location and recovery. This systematic recording of the task execution process enables traceability of the entire task execution flow, facilitating fault auditing, optimization, and scheduling strategy iteration. Furthermore, task execution records are uniformly stored in a shared database associated with the control node cluster. The shared database adopts a highly available architecture, supporting concurrent read / write and consistency checks across multiple control nodes, ensuring data consistency and availability of task records in multi-control node collaborative scenarios. All node information is also stored in the shared database, providing a reliable data foundation for subsequent task retries and historical analysis.

[0114] Figure 3 This is a schematic diagram of the structure of a task scheduling system provided in an embodiment of this application. Figure 3 As shown. The task scheduling system includes a task scheduling device, a control node cluster, and a service node cluster.

[0115] The task scheduling device is used to obtain the node status information of each control node in the pre-built control node cluster using a preset heartbeat detection mechanism; update the candidate control node set for task allocation based on the node status information of the control nodes; receive tasks to be processed and obtain the running status information of each control node in the candidate control node set; determine the target control node from the candidate control node set according to the tasks to be processed and the running status information of the control nodes, and through a preset scheduling strategy; and send the tasks to be processed to the target control node so that the target control node schedules service nodes in the preset service node cluster to execute the tasks to be processed.

[0116] The control node cluster receives tasks to be processed from the task scheduling device and, based on these tasks, sends scheduling execution instructions to the service node cluster. The control node cluster operates in a multi-active mode.

[0117] The service node cluster is used to execute pending tasks according to the scheduling execution instructions.

[0118] Taking Jenkins as an example, the task scheduling device in this embodiment works in conjunction with the Jenkins master cluster and the Jenkins slave cluster: the Jenkins master cluster, consisting of Jenkins master1, Jenkins master2, ... Jenkins masterN, corresponds to the control node cluster in this application; the Jenkins slave cluster, consisting of Jenkins slave1, Jenkins slave2, ... Jenkins slaveN, corresponds to the service node cluster in this application. The task scheduling device periodically interacts with each Jenkins master node in the Jenkins master cluster according to a preset heartbeat detection mechanism to obtain the node status information of each Jenkins master node, and maintains a set of candidate control nodes that can be used for task allocation based on this information.

[0119] When a task arrives, the task scheduler determines the target master node from the Jenkins master cluster based on the running status information of each Jenkins master node in the candidate control node set and the preset scheduling strategy, and assigns the task to that target master node. Upon receiving the task, the target master node further selects a suitable slave node from the Jenkins slave cluster and issues specific execution instructions to it, enabling the Jenkins slave node to execute the corresponding build, test, or other automated tasks. Through this structure, this embodiment demonstrates the hierarchical collaboration relationship and task scheduling process between the task scheduler, the control node cluster (Jenkins master cluster), and the service node cluster (Jenkins slave cluster) in a Jenkins scenario.

[0120] Figure 4 This is a schematic diagram of the structure of the task scheduling device provided in the embodiments of this application, as shown below. Figure 4 As shown, the task scheduling device 40 provided in this embodiment includes:

[0121] The status detection module 401 is used to obtain the node status information of each control node in the control node cluster based on the constructed control node cluster and using a preset heartbeat detection mechanism.

[0122] The information update module 402 is used to update the set of candidate control nodes for task allocation based on the node status information of the control nodes.

[0123] The information acquisition module 403 is used to receive the task to be processed and acquire the running status information of each control node in the candidate control node set;

[0124] The node selection module 404 is used to determine the target control node from the set of candidate control nodes based on the task to be processed and the running status information of the control node, through a preset scheduling strategy.

[0125] The scheduling and execution module 405 is used to send the task to be processed to the target control node, so that the target control node can schedule the service nodes in the preset service node cluster to execute the task.

[0126] In one possible implementation, the status detection module 401 is specifically used to: send heartbeat detection requests to each control node in the control node cluster according to the heartbeat detection cycle, and receive heartbeat detection response information from each control node; if a heartbeat detection response information from a control node is received within a preset waiting time, the node status information of the control node is marked as online; if no heartbeat detection response information from a control node is received within the preset waiting time, the heartbeat detection request is resent to the control node according to a preset retry strategy and the number of failures is accumulated; when the accumulated number of failures is greater than or equal to a preset threshold, the node status of the control node is marked as offline.

[0127] In one possible implementation, the status detection module 401 is further specifically used to: if a heartbeat detection response information of the control node is received in the resending heartbeat detection request, reset the failure count of the control node to zero and mark the node status of the control node as online.

[0128] In one possible implementation, a building module is also included, which is used to build a control node cluster containing at least two control nodes and assign a cluster identifier, and associate the control node cluster with a pre-configured shared database; store the configuration file in the shared database and mark the version identifier of the configuration file; and publish the current version identifier of the configuration file to the control node cluster so that each control node can read the current version of the configuration file from the shared database according to the version identifier.

[0129] In one possible implementation, the information update module 402 is specifically used to: read the node status information of each control node; delete control nodes marked as offline from the candidate control node set; and obtain control nodes that are not in the candidate control node set but are marked as online, and add them to the candidate control node set to update the candidate control node set used for task allocation.

[0130] In one possible implementation, the node selection module 404 is specifically used for: extracting task attribute information of the task to be processed based on the task to be processed; obtaining the scheduling scoring rules corresponding to the task attribute information based on the task attribute information; calculating the scheduling score of the control node based on the running status information of each control node in the candidate control node set and the scheduling scoring rules; sorting the control node from high to low according to the scheduling score, and selecting the control node with the highest ranking as the target control node.

[0131] One possible implementation also includes: generating task execution records and storing them in a shared database based on the task processing of the target control node and service node.

[0132] The task scheduling device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0133] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the electronic device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0134] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0135] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0136] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0137] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0138] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0139] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0140] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0141] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0142] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0143] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0144] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A task scheduling method, characterized in that, include: Based on the established control node cluster, a preset heartbeat detection mechanism is used to obtain the node status information of each control node in the control node cluster. Update the set of candidate control nodes for task allocation based on the node status information of the control nodes; Receive the task to be processed and obtain the running status information of each control node in the candidate control node set; Based on the task to be processed and the running status information of the control node, the target control node is determined from the set of candidate control nodes through a preset scheduling strategy; The task to be processed is sent to the target control node, so that the target control node schedules service nodes in the preset service node cluster to execute the task to be processed.

2. The method according to claim 1, characterized in that, The method employs a preset heartbeat detection mechanism to obtain the node status information of each control node in the control node cluster, including: According to the heartbeat detection cycle, a heartbeat detection request is sent to each control node in the control node cluster, and the heartbeat detection response information of each control node is received. If a heartbeat detection response is received from the control node within a preset waiting time, the node status information of the control node is marked as online. If no heartbeat response is received from the control node within the preset waiting time, the control node will resend the heartbeat request according to the preset retry strategy and the number of failures will be accumulated. When the accumulated number of failures is greater than or equal to the preset threshold, the node status of the control node will be marked as offline.

3. The method according to claim 2, characterized in that, Also includes: If a heartbeat response is received from the control node in a resend heartbeat request, the failure count of the control node is reset to zero, and the node status of the control node is marked as online.

4. The method according to claim 1, characterized in that, Constructing the control node cluster includes: Construct a control node cluster containing at least two control nodes and assign a cluster identifier, and establish an association between the control node cluster and a pre-configured shared database; The configuration file is stored in the shared database, and a version identifier is marked on the configuration file; The current version identifier of the configuration file is published to the control node cluster so that each control node can read the current version of the configuration file from the shared database based on the version identifier.

5. The method according to claim 2, characterized in that, The step of updating the candidate control node set for task allocation based on the node status information of the control node includes: Read the node status information of each of the control nodes; Delete the control nodes marked as offline from the candidate control node set, and obtain the control nodes that are not in the candidate control node set but are marked as online, and add them to the candidate control node set to update the candidate control node set used for task allocation.

6. The method according to claim 1, characterized in that, The step of determining the target control node from the candidate control node set based on the task to be processed and the running status information of the control node, through a preset scheduling strategy, includes: Based on the task to be processed, extract the task attribute information of the task to be processed; Based on the task attribute information of the task to be processed, obtain the scheduling scoring rule corresponding to the task attribute information; Based on the running status information of each control node in the candidate control node set, and based on the scheduling scoring rules, the scheduling score of the control node is calculated. The control node is selected as the target control node based on the scheduling score, which is sorted from high to low according to the score.

7. The method according to any one of claims 1 to 6, characterized in that, Also includes: Based on the task processing of the target control node and the service node, task execution records are generated and stored in the shared database.

8. A task scheduling system, characterized in that, This includes a task scheduling device, a control node cluster, and a service node cluster. The task scheduling device is used to obtain the node status information of each control node in the control node cluster based on the constructed control node cluster and using a preset heartbeat detection mechanism. Update the set of candidate control nodes for task allocation based on the node status information of the control nodes; Receive the task to be processed and obtain the running status information of each control node in the candidate control node set; determine the target control node from the candidate control node set according to the task to be processed and the running status information of the control node through a preset scheduling strategy. The task to be processed is sent to the target control node, so that the target control node schedules service nodes in the preset service node cluster to execute the task to be processed. The control node cluster is used to receive tasks to be processed sent by the task scheduling device, and send scheduling execution instructions to the service node cluster according to the tasks to be processed; wherein, the control node cluster is in a multi-active mode. The service node cluster is used to execute the tasks to be processed according to the scheduling execution instructions.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 7.

Images