Batch server resource capacity expansion and contraction processing method and system and related equipment
By controlling the head and tail nodes and combining them with the job schedule, the system enables timely scaling up and down of batch server resources, solving the problems of latency and flexibility, and improving scaling up and down efficiency and business stability.
Patent Information
- Application Number
- CN202511724254.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-23
- Publication Date
- 2026-02-17
AI Technical Summary
Existing technologies suffer from latency and poor flexibility in the process of scaling up and down server resources in batches. Especially in asynchronous batch job scenarios, the KPA strategy is highly sensitive but not applicable, while the HPA strategy has high latency and low efficiency.
The system uses a head node approach to request cluster expansion before the actual batch execution of business processes, and a tail node approach to request cluster reduction after execution. It combines the actual time of the job plan to expand or shrink resources, and uses the expansion/shrinkage check node to retry requests, thereby improving flexibility.
It reduces latency, improves the efficiency and flexibility of server resource scaling, ensures timely business execution, and has high fault tolerance.
Smart Images
Figure CN121542044A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of scaling up and down processing technology, and more specifically, to a method, system and related equipment for batch server resource scaling up and down processing. Background Technology
[0002] In real-world business scenarios, there are situations where a service needs to be scaled up or down, such as service load testing, e-commerce websites, live video streaming websites, and workload reduction. For these scaling scenarios, existing approaches typically involve either a Knative Pod Autoscaler (KPA) strategy based on the number of scheduled requests or a Horizontal Pod Autoscaler (HPA) strategy based on CPU and memory metrics, to scale up or down server resources in batches.
[0003] In the KPA strategy, the utilization of the Central Processing Unit (CPU) and memory is positively correlated with the number of requests. Therefore, a surge in requests can lead to CPU and memory exhaustion, ultimately causing service collapse. The KPA strategy is more sensitive to the number of requests and can scale up in a timely manner. However, it is not suitable for asynchronous batch jobs, resulting in poor flexibility in scaling up and down batch server resources.
[0004] HPA (Hyper-Scale-Up) strategies suffer from high response latency as soon as the metric rises. This means that the response is delayed after the metric detects that the threshold has been exceeded, and only then does scaling up to the smallest deployable unit (pod) begin. By the time the pod is ready to be used, the latency is already high. Therefore, this scaling-up / scaling approach has inherent latency, leading to low efficiency.
[0005] Therefore, how to reduce latency and improve flexibility and efficiency during the scaling up and down of server resources in batches is a problem that this application urgently needs to solve. Summary of the Invention
[0006] In view of this, this application discloses a method, system and related equipment for batch server resource scaling up and down, which aims to reduce latency and improve flexibility and efficiency during the batch server resource scaling up and down process.
[0007] To achieve the above objectives, the disclosed technical solution is as follows:
[0008] The first aspect of this application discloses a method for batch server resource scaling up and down, the method comprising:
[0009] Determine whether the communication ID used by the job node under the job plan in the batch scheduling scenario is unique; wherein, the job plan is a set of job nodes with sequential order and time requirements;
[0010] If the communication ID used by the job node under the job plan is unique, determine the type of the job node under the job plan;
[0011] If the job node under the job plan is the head node, send a batch server resource expansion request to the cluster;
[0012] When a response message indicating successful resource expansion is received from the cluster, the expansion check node is set to a successful state, and subsequent nodes are controlled to be started.
[0013] If the job node under the job plan is the tail node, send a batch server resource reduction request to the cluster;
[0014] When a response message indicating successful resource scaling-down is received from the cluster, the scaling-down check node is set to a successful state.
[0015] Preferably, the process of setting up a capacity expansion check node includes:
[0016] Identify all nodes in the job plan that do not have any preceding dependencies in the batch scheduling scenario;
[0017] Set up a resizing check node for all nodes that do not have any predecessor dependencies.
[0018] Preferably, the process of setting up a scaling down check node includes:
[0019] Identify all nodes in the job plan of the batch scheduling scenario that do not have subsequent dependencies;
[0020] Set up a shrink check node for all nodes that do not have subsequent dependencies.
[0021] Preferred options also include:
[0022] If, in a batch scheduling scenario, the communication IDs used by the job nodes under the job plan are not unique, the status of the job nodes under the job plan will be set to a failed state.
[0023] Preferred options also include:
[0024] When a response message indicating that resource expansion failed is received from the cluster, the status of the job node under the job plan is set to the failed state, subsequent nodes are prevented from starting, and the cluster expansion request is resent for retry.
[0025] Preferred options also include:
[0026] When a response message indicating that resource scaling down has failed is received from the cluster, the status of the job node under the job plan is set to the failed state, and the cluster scaling down request is resent for a retry.
[0027] A second aspect of this application discloses a system for batch server resource scaling up and down, the system comprising:
[0028] A determining unit is used to determine whether the communication ID used by the job node under the job plan in the batch scheduling scenario is unique; wherein, the job plan is a set of job nodes with sequential order and time requirements;
[0029] The judgment unit is used to determine the type of the job node under the job plan if the communication ID used by the job node under the job plan is unique;
[0030] The first sending unit is configured to send a batch server resource expansion request to the cluster if the job node under the job plan is the head node.
[0031] Configure a control unit to set the expansion check node to a successful state and control subsequent nodes to be invoked when the cluster returns a response message indicating that the resource expansion was successful.
[0032] The second sending unit is used to send a batch server resource reduction request to the cluster if the job node under the job plan is the tail node.
[0033] The first setting unit is used to set the scaling down check node to a successful state when it receives a response message from the cluster indicating that the resource scaling down was successful.
[0034] Preferably, the setting control unit for setting expansion check nodes includes:
[0035] The first determining module is used to determine all nodes in the job plan that do not have any preceding dependencies in the batch scheduling scenario;
[0036] The first configuration module is used to set up expansion check nodes for all nodes that do not have previous dependencies.
[0037] A third aspect of this application discloses a storage medium comprising stored instructions, wherein, when the instructions are executed, the device containing the storage medium is controlled to perform a batch server resource scaling method as described in any one of the first aspects.
[0038] The fourth aspect of this application discloses an electronic device, including a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described in any one of the first aspects of the batch server resource scaling method.
[0039] As can be seen from the above technical solution, this application discloses a batch server resource scaling method, system, and related equipment, which relates to the field of scaling processing technology. It determines whether the communication ID used by the job nodes under the job plan in a batch scheduling scenario is unique. The job plan is a set of job nodes with sequential and time requirements. If the communication ID used by the job nodes under the job plan is unique, the type of the job node under the job plan is determined. If the job node under the job plan is the head node, a batch server resource scaling request is sent to the cluster. When a response message indicating successful resource scaling is received from the cluster, the scaling check node is set to a successful state, and subsequent nodes are controlled to start. If the job node under the job plan is the tail node, a batch server resource scaling request is sent to the cluster. When a response message indicating successful resource scaling is received from the cluster, the scaling check node is set to a successful state.
[0040] The above solution addresses batch scheduling scenarios with a single job plan, single communication ID, and single cluster instance. It uses a head node to request cluster expansion control before the actual batch execution of business processes and a tail node to request cluster contraction control after the actual batch execution of business processes. Based on the actual start and execution times of the job plan, cluster resource expansion and contraction are performed on time, reducing latency and improving the efficiency of server resource expansion and contraction control during batch server resource expansion and contraction. Furthermore, cluster expansion and contraction requests can be retried by resubmitting expansion and contraction check nodes, enhancing the flexibility of the batch server resource expansion and contraction process. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0042] Figure 1 This is a flowchart illustrating a batch server resource scaling method disclosed in an embodiment of this application.
[0043] Figure 2 This is a structural example diagram of the node concurrent execution mode disclosed in the embodiments of this application;
[0044] Figure 3 This is a structural example diagram of the node-series execution mode disclosed in the embodiments of this application;
[0045] Figure 4 This is a structural example diagram of concurrent node execution with serial interleaved execution disclosed in the embodiments of this application;
[0046] Figure 5 This is a schematic diagram illustrating the setting of a scaling check node as disclosed in an embodiment of this application.
[0047] Figure 6 This is a flowchart illustrating another batch server resource scaling method disclosed in an embodiment of this application;
[0048] Figure 7 This is a schematic diagram of the structure of a batch server resource scaling system disclosed in an embodiment of this application;
[0049] Figure 8 This is a schematic diagram of the structure of the electronic device disclosed in the embodiments of this application. Detailed Implementation
[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0051] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0052] As the background technology indicates, the KPA strategy is more sensitive to the number of requests and can scale up promptly. However, it is not suitable for asynchronous batch jobs, resulting in poor flexibility during the scaling up and down of batch server resources. The HPA strategy's scaling up and down scheme has a certain degree of latency, leading to low efficiency.
[0053] To address the aforementioned issues, this application discloses a method, system, and related equipment for batch server resource scaling. For batch scheduling scenarios with a single job plan, single communication ID, and a single cluster instance, cluster scaling is requested and completed before the actual batch execution of business tasks using a head node approach, and cluster scaling is requested and completed after the actual batch execution of business tasks using a tail node approach. Cluster resource scaling is performed on time according to the actual start and execution times of the job plan, reducing latency and improving the efficiency of server resource scaling control during batch server resource scaling. Furthermore, cluster scaling requests can be retried by re-adding scaling and shrinking check nodes, improving the flexibility of the batch server resource scaling process. The specific implementation is described in detail in the following embodiments.
[0054] It should be noted that the batch server resource scaling method, system and related equipment provided in this application can be used in the technical field of scaling processing, etc. The above is only an example and does not limit the application field of the batch server resource scaling method, system and related equipment provided in this application.
[0055] refer to Figure 1 As shown in the figure, this application discloses a method for batch server resource scaling up and down, which mainly includes the following steps:
[0056] S101: Determine whether the communication ID used by the job node under the job plan in the batch scheduling scenario is unique; where the job plan is a set of job nodes with sequential order and time requirements.
[0057] Among them, batch scheduling scenarios can include single job plans, single communication IDs, and single cluster instances.
[0058] The current batch job execution modes under a single job plan include concurrent node execution, sequential node execution, and concurrent node execution combined with sequential cross-execution. A structural example of the specific concurrent node execution mode is shown below. Figure 2 As shown. An example of the node-series execution pattern structure is as follows. Figure 3 As shown. A specific example of a structure combining concurrent nodes and sequential interleaved execution is shown below. Figure 4 As shown.
[0059] A job plan is designed to complete a full batch of business processes, comprising a predefined set of job nodes with a clear sequence. A business system can have multiple job plans. Nodes within the same job plan can depend on each other to ensure the correct sequence.
[0060] A job node is a scheduling configuration unit for a batch of tasks, which typically includes batch tasks, scheduling time, execution parameters, scheduling rules, and other key configurations.
[0061] S102: If the communication ID used by the job node under the job plan is unique, determine the type of the job node under the job plan. If the job node under the job plan is the head node, execute S103; if the job node under the job plan is the tail node, execute S105.
[0062] Job nodes can be either head nodes or tail nodes.
[0063] The head node is the first node in the job plan execution process.
[0064] The tail node is the last node in the execution process of the job plan.
[0065] If, in a batch scheduling scenario, the communication IDs used by the job nodes under the job plan are not unique, the status of the job nodes under the job plan will be set to a failed state.
[0066] S103: Send a batch server resource expansion request to the cluster.
[0067] S104: When the cluster returns a response indicating that the resource expansion was successful, set the expansion check node to a successful state and control the subsequent nodes to be started.
[0068] The process of setting up the expansion check node is as follows:
[0069] Identify all nodes in the job plan that do not have any preceding dependencies in the batch scheduling scenario, and set up expansion check nodes for all nodes that do not have preceding dependencies.
[0070] When a response message indicating that resource expansion failed is received from the cluster, the status of the job node under the job plan is set to a failed state, preventing subsequent nodes from being started.
[0071] S105: Send a batch server resource reduction request to the cluster.
[0072] The head and tail node control mechanism in batch scheduling scenarios refers to the configuration relationship between business nodes and cluster expansion and contraction check nodes.
[0073] Resource scaling includes, but is not limited to, elastic scaling of servers, scaling based on system load metrics, scaling based on business logs, scaling based on resource requests, and scaling based on events.
[0074] Elastic scaling of servers refers to the ability of a server controller to automatically adjust the number of servers based on real-time load conditions without manual intervention. This function can dynamically increase or decrease server resources according to actual needs to cope with different load situations. Reducing the number of servers under low load saves costs; increasing the number of servers under high load ensures server stability, thereby improving resource utilization.
[0075] Scaling up or down based on system load metrics refers to setting metrics for CPU and memory utilization to automatically adjust the number of replicas to cope with different load conditions.
[0076] Scaling up and down based on business logs involves automatically scaling up and down according to the business logs, which requires log storage and analysis tools.
[0077] Scaling based on resource requests is a process of adjusting the amount of resources requested to achieve scaling.
[0078] Event-based scaling automatically adjusts resources based on specific events (such as tasks in a message queue or database tasks).
[0079] During peak traffic periods, batch server resource expansion requests are sent to the cluster to increase resource capacity and prevent system crashes; during off-peak periods, batch server resource reduction requests are sent to the cluster to reduce resources and save costs.
[0080] For batch scheduling scenarios with a single job plan, a single communication ID, and a single cluster instance, cluster expansion control is requested before the actual batch execution of business operations using a head node approach, and cluster shrinkage control is requested after the actual batch execution of business operations using a tail node approach. Based on the actual start and execution times of the job plan, cluster resource expansion and shrinkage are performed on time, reducing latency during batch server resource expansion and shrinkage and improving the efficiency of server resource expansion and shrinkage control.
[0081] S106: When a response message indicating successful resource scaling-down is received from the cluster, the scaling-down check node is set to a successful state.
[0082] It should be noted that the scaling check node can only be invoked if all its predecessor nodes have executed successfully. After successful scaling, the scaling check node is set to a success status, indicating that the job plan has been completed and executed successfully.
[0083] The process of setting up a scaling down check node is as follows:
[0084] Identify all nodes in the job plan that have no subsequent dependencies in the batch scheduling scenario, and set up a shrinkage check node for all nodes that have no subsequent dependencies.
[0085] The efficient server resource scaling control method during batch scheduling allows for timely scaling of cluster resources based on the actual start and execution times of the job plan.
[0086] After the scaling up or down of nodes is completed, the start and end execution times in the node execution information can explicitly indicate the start and end times of cluster resource scaling up or down.
[0087] When the scaling check node fails to execute, the scaling check node can be resubmitted. Resubmission means resending the cluster scaling request for a retry.
[0088] Modifications were made to the batch job execution mode. All nodes without prior dependencies under the job plan were uniformly assigned the aforementioned pre-expansion check node, and all nodes without subsequent dependencies were uniformly assigned a subsequent shrinkage check node. Specifically... Figure 5 As shown.
[0089] Add cluster expansion check nodes and shrinkage check nodes for batch servers, and configure the scheduling platform program for the expansion check nodes and shrinkage check nodes. The core function design of the scheduling platform program is shown in Figure 6.
[0090] Figure 6 In the process, the predecessor node of all head nodes (nodes without predecessors) in the original business node relationship diagram is set as the expansion check node of the cluster;
[0091] Set the successor node of all tail nodes (without successor nodes) in the original business node relationship graph as the cluster's shrinkage check node;
[0092] When a scaling check node is invoked, the first check is whether the communication ID used by the node under the job plan is unique.
[0093] If the communication ID is not unique, the job node under the job plan will be set to failure. If the expansion check node fails, subsequent nodes will not be started.
[0094] When the communication ID is unique, determine whether it is the head node or the tail node. If it is the head node, send a batch server resource expansion request to the cluster. If it is the tail node, send a batch server resource reduction request to the cluster.
[0095] If a request response indicating cluster scaling failure is received, the status of the job node under the job plan is set to a failed state, preventing subsequent nodes from starting; if scaling is successful, subsequent nodes are started.
[0096] Generally, business nodes have a requirement to be launched and executed on time. Therefore, container scaling needs to be completed before the business nodes are executed to ensure that the business nodes have the conditions to be launched and executed. So, for systems with time requirements, a launch start time can be configured for the scaling check node. This time should generally be earlier than the launch start time of subsequent business nodes, for example, the cluster resource scaling should be completed five minutes in advance.
[0097] This solution allows you to configure cluster resource scaling targets through node parameters. The number of cluster resources can be configured in the parameter settings of both the scaling and shrinking check nodes. Specifically, the target scaling target number can be set in the scaling check node parameter configuration. For example, configuring a target number of 10 with a specification of 4C8G will increase the number of servers in the cluster to 10 after scaling, with each server having a specification of 4C8G. Similarly, the target shrinking target number can be set in the shrinking check node parameter configuration. For instance, configuring a target number of 1 will reduce the number of Pods in the cluster to 1 after shrinking, serving as a standby resource.
[0098] The cluster expansion and contraction target values are configured through the node parameters, and cluster expansion can be controlled by scheduling the start time.
[0099] This application can retry cluster expansion and contraction requests by resubmitting expansion and contraction check nodes, thereby improving the flexibility of the process of expanding and contracting server resources in batches.
[0100] The cluster can be scaled up or down before and after the actual batch execution of business operations, avoiding errors caused by resource issues during batch execution. It has high fault tolerance and ensures normal business operation.
[0101] This solution is primarily used in batch scheduling scenarios. It leverages the node scheduling features of the batch scheduling platform to explicitly control the scaling of cluster resources before and after batch execution of services. In the batch scheduling platform, a node represents a service program instance, and executing a node signifies the execution of a specific service scenario.
[0102] In this embodiment, for batch scheduling scenarios with a single job plan, a single communication ID, and a single cluster instance, cluster expansion control is requested before the actual batch execution of business operations using a head node approach, and cluster shrinkage control is requested after the actual batch execution of business operations using a tail node approach. Cluster resource expansion and shrinkage are performed on time according to the actual start and execution times of the job plan, reducing latency during batch server resource expansion and shrinkage and improving the efficiency of server resource expansion and shrinkage control. Furthermore, cluster expansion and shrinkage requests can be retried by resubmitting expansion and shrinkage check nodes, improving the flexibility of the batch server resource expansion and shrinkage process.
[0103] Based on the above embodiments Figure 1This application discloses a method for batch server resource scaling up and down, and also provides a corresponding system for batch server resource scaling up and down. Figure 7 As shown, the batch server resource scaling system includes:
[0104] The determining unit 701 is used to determine whether the communication ID used by the job node under the job plan in the batch scheduling scenario is unique; wherein, the job plan is a set of job nodes with sequential order and time requirements;
[0105] The judgment unit 702 is used to determine the type of the job node under the job plan if the communication ID used by the job node under the job plan is unique;
[0106] The first sending unit 703 is used to send a batch server resource expansion request to the cluster if the job node under the job plan is the head node.
[0107] The control unit 704 is configured to set the expansion check node to a successful state and control subsequent nodes to be invoked when the cluster returns a response message indicating that the resource expansion was successful.
[0108] The second sending unit 705 is used to send a batch server resource reduction request to the cluster if the job node under the job plan is the tail node.
[0109] The first setting unit 706 is used to set the scaling-down check node to a successful state when it receives a response message from the cluster indicating that the resource scaling-down was successful.
[0110] Furthermore, the setting control unit 704 for setting expansion check nodes includes:
[0111] The first determination module is used to determine all nodes in the job plan that do not have any preceding dependencies in the batch scheduling scenario;
[0112] The first configuration module is used to set up expansion check nodes for all nodes that do not have previous dependencies.
[0113] Furthermore, the first setting unit 406 for setting the scaling down check node includes:
[0114] The second determination module is used to determine all nodes in the job plan that do not have subsequent dependencies in the batch scheduling scenario;
[0115] The second configuration module is used to set up a shrinkage check node for all nodes that do not have subsequent dependencies.
[0116] Furthermore, the batch server resource scaling up and down handling system also includes:
[0117] The second setting unit is used to set the status of the job node under the job plan to a failed state if the communication ID used by the job node under the job plan is not unique in a batch scheduling scenario.
[0118] Furthermore, the batch server resource scaling up and down handling system also includes:
[0119] The third setting unit is used to set the status of the job node under the job plan to a failed state when it receives a response message from the cluster indicating that the resource expansion failed, control the subsequent nodes not to start, and resend the cluster expansion request to retry the request.
[0120] Furthermore, the batch server resource scaling up and down handling system also includes:
[0121] The fourth setting unit is used to set the status of the job node under the job plan to a failed state and resend the cluster scaling-down request for retry when it receives a response message from the cluster indicating that the resource scaling-down failed.
[0122] In this embodiment, for batch scheduling scenarios with a single job plan, a single communication ID, and a single cluster instance, cluster expansion control is requested before the actual batch execution of business operations using a head node approach, and cluster shrinkage control is requested after the actual batch execution of business operations using a tail node approach. Cluster resource expansion and shrinkage are performed on time according to the actual start and execution times of the job plan, reducing latency during batch server resource expansion and shrinkage and improving the efficiency of server resource expansion and shrinkage control. Furthermore, cluster expansion and shrinkage requests can be retried by resubmitting expansion and shrinkage check nodes, improving the flexibility of the batch server resource expansion and shrinkage process.
[0123] This application embodiment also provides a storage medium, the storage medium including stored instructions, wherein, when the instructions are executed, the device where the storage medium is located is controlled to perform the batch server resource scaling method described above.
[0124] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 8 As shown, it specifically includes a memory 801 and one or more instructions 802, wherein one or more instructions 802 are stored in the memory 801 and are configured to be executed by one or more processors 803 to perform the above-mentioned batch server resource scaling method.
[0125] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0126] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system-type embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0127] The steps in the methods of the various embodiments of this application can be adjusted, combined, or deleted according to actual needs.
[0128] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.
[0129] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
[0130] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A batch server resource scaling handling method, characterized in that, The method includes: Determine whether the communication ID used by the job node under the job plan in the batch scheduling scenario is unique; wherein, the job plan is a set of job nodes with sequential order and time requirements; If the communication ID used by the job node under the job plan is unique, determine the type of the job node under the job plan; If the job node under the job plan is the head node, send a batch server resource expansion request to the cluster; When a response message indicating successful resource expansion is received from the cluster, the expansion check node is set to a successful state, and subsequent nodes are controlled to be started. If the job node under the job plan is the tail node, send a batch server resource reduction request to the cluster; When a response message indicating successful resource scaling-down is received from the cluster, the scaling-down check node is set to a successful state.
2. The method of claim 1, wherein, The process of setting up a capacity expansion check node includes: Identify all nodes in the job plan that do not have any preceding dependencies in the batch scheduling scenario; Set up a resizing check node for all nodes that do not have any predecessor dependencies.
3. The method of claim 1, wherein, The process of setting up a scaling down check node includes: Identify all nodes in the job plan of the batch scheduling scenario that do not have subsequent dependencies; Set up a shrink check node for all nodes that do not have subsequent dependencies.
4. The method of claim 1, wherein, Also includes: If, in a batch scheduling scenario, the communication IDs used by the job nodes under the job plan are not unique, the status of the job nodes under the job plan will be set to a failed state.
5. The method of claim 1, wherein, Also includes: When a response message indicating that resource expansion failed is received from the cluster, the status of the job node under the job plan is set to the failed state, subsequent nodes are prevented from starting, and the cluster expansion request is resent for retry.
6. The method of claim 1, wherein, Also includes: When a response message indicating that resource scaling down has failed is received from the cluster, the status of the job node under the job plan is set to the failed state, and the cluster scaling down request is resent for a retry.
7. A batch server resource scaling handling system, characterized in that, The system includes: A determining unit is used to determine whether the communication ID used by the job node under the job plan in the batch scheduling scenario is unique; wherein, the job plan is a set of job nodes with sequential order and time requirements; The judgment unit is used to determine the type of the job node under the job plan if the communication ID used by the job node under the job plan is unique; The first sending unit is configured to send a batch server resource expansion request to the cluster if the job node under the job plan is the head node. Configure a control unit to set the expansion check node to a successful state and control subsequent nodes to be invoked when the cluster returns a response message indicating that the resource expansion was successful. The second sending unit is used to send a batch server resource reduction request to the cluster if the job node under the job plan is the tail node. The first setting unit is used to set the scaling down check node to a successful state when it receives a response message from the cluster indicating that the resource scaling down was successful.
8. The system of claim 7, wherein, The configuration control unit for setting up capacity expansion check nodes includes: The first determining module is used to determine all nodes in the job plan that do not have any preceding dependencies in the batch scheduling scenario; The first configuration module is used to set up expansion check nodes for all nodes that do not have previous dependencies.
9. A storage medium, characterized in that, The storage medium includes stored instructions, wherein, when the instructions are executed, the device where the storage medium is located is controlled to perform the batch server resource scaling method as described in any one of claims 1 to 6.
10. An electronic device, characterized in that, It includes a memory, and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described in any one of claims 1 to 6.