Distributed computing task scheduling system and method based on Java

Abstract

The invention discloses a distributed computing task scheduling system and method based on Java, and relates to the technical field of distributed data management. Comprising the steps of 1, receiving a computing task submitted by a user, preprocessing the computing task, and generating task information, 2, monitoring and managing computing resources in the distributed computing system, and obtaining computing resource information which comprises state and performance parameter information of computing nodes, 3, selecting a proper preset scheduling strategy according to the task information and the computing resource information, and generating a task scheduling scheme, the scheduling strategy comprising a load balancing strategy, a priority strategy and a resource utilization rate maximization strategy, and 4, allocating the computing tasks to corresponding computing nodes for execution according to the task scheduling scheme, and the execution process of the calculation task is monitored, and an execution result is returned to the user.

Description

Technical Field

[0001] The present invention discloses a Java-based distributed computing task scheduling system and method, and relates to the technical field of distributed data management. Background Art

[0002] With the continuous development of computer technology, distributed computing has become an important means to solve large-scale computing problems. Distributed computing achieves the sharing and efficient use of computing resources by assigning computing tasks to multiple computing nodes. In distributed computing systems, task scheduling is one of the core issues, and its goal is to reasonably allocate computing tasks to improve the overall performance and computing efficiency of the system.

[0003] At present, the distributed computing task scheduling technology is mainly static scheduling. Static scheduling pre-assigns computing tasks before task execution, which is suitable for scenarios with small computing scale and clear dependencies between tasks. However, for large-scale, dynamically changing distributed computing tasks, static scheduling has problems such as low resource utilization and low computing efficiency. Dynamic scheduling is more flexible. As a cross-platform programming language, Java has a wide range of applications and a powerful ecosystem. The Java language has the characteristics of object-oriented, cross-platform, and multi-threaded, and is suitable for the development of distributed computing task scheduling systems. However, the existing Java-based distributed computing task scheduling system has problems such as single scheduling strategy, unbalanced resource allocation, and low computing efficiency, which makes it difficult to meet the needs of large-scale, dynamically changing distributed computing tasks. Summary of the invention

[0004] In view of the problems of the prior art, the present invention provides a Java-based distributed computing task scheduling system and method to solve the bottleneck problems existing in the current distributed computing field, such as rigid scheduling strategy, unbalanced resource allocation and low computing efficiency.

[0005] The specific scheme proposed by the present invention is:

[0006] The present invention provides a Java-based distributed computing task scheduling method, comprising:

[0007] Step 1: Receive the computing tasks submitted by the user, pre-process the computing tasks, and generate task information.

[0008] Step 2: Monitor and manage computing resources in the distributed computing system and obtain computing resource information, including the status and performance parameter information of computing nodes.

[0009] Step 3: Select an appropriate preset scheduling strategy according to the task information and computing resource information to generate a task scheduling plan. The scheduling strategy includes a load balancing strategy, a priority strategy, and a resource utilization maximization strategy. The load balancing strategy is used to assign computing tasks to nodes with lower loads for execution according to the load conditions of the computing nodes; the priority strategy is used to assign and execute tasks from high to low according to the priority of the computing tasks; and the resource utilization maximization strategy is used to assign computing tasks to nodes with lower resource utilization for execution according to the resource utilization conditions of the computing nodes;

[0010] Step 4: According to the task scheduling plan, the computing tasks are assigned to the corresponding computing nodes for execution, and the execution process of the computing tasks is monitored, and the execution results are returned to the user.

[0011] Furthermore, in step 1 of the Java-based distributed computing task scheduling method, the computing task is preprocessed, including: parsing the computing task, extracting the task type, input data, and computing amount information of the computing task, and generating task information based on the extracted information.

[0012] Furthermore, in step 3 of the Java-based distributed computing task scheduling method, the load conditions of the computing nodes are judged in turn, and if there is a load imbalance, a load balancing strategy is selected;

[0013] Determine the priority of the computing task. If there is a high-priority task, select the priority strategy;

[0014] Determine the resource utilization of the computing nodes. If there is an imbalance in resource utilization, select the resource utilization maximization strategy.

[0015] Furthermore, in step 4 of the Java-based distributed computing task scheduling method, monitoring the execution process of the computing task includes: monitoring the task execution status and execution time information.

[0016] The present invention also provides a Java-based distributed computing task scheduling system, including a task management module, a resource management module, a scheduling strategy module and a task execution module.

[0017] The task management module receives computing tasks submitted by users, pre-processes the computing tasks, generates task information and sends it to the scheduling strategy module.

[0018] The resource management module monitors and manages the computing resources in the distributed computing system, obtains computing resource information and sends it to the scheduling strategy module. The computing resource information includes the status and performance parameter information of the computing nodes.

[0019] The scheduling strategy module selects an appropriate preset scheduling strategy according to the task information and computing resource information, generates a task scheduling plan and sends it to the task execution module. The scheduling strategy includes a load balancing strategy, a priority strategy and a resource utilization maximization strategy. The load balancing strategy is used to assign computing tasks to nodes with lower loads for execution according to the load conditions of the computing nodes; the priority strategy is used to assign and execute tasks from high to low according to the priority of the computing tasks; the resource utilization maximization strategy is used to assign computing tasks to nodes with lower resource utilization for execution according to the resource utilization conditions of the computing nodes;

[0020] The task execution module assigns computing tasks to corresponding computing nodes for execution according to the task scheduling plan, monitors the execution process of computing tasks, and returns the execution results to the user.

[0021] Furthermore, the task management module of the Java-based distributed computing task scheduling system pre-processes the computing tasks, including: parsing the computing tasks, extracting the task type, input data, and computing amount information of the computing tasks, and generating task information based on the extracted information.

[0022] Furthermore, the scheduling strategy module of the Java-based distributed computing task scheduling system sequentially determines the load conditions of the computing nodes, and if there is a load imbalance, selects a load balancing strategy;

[0023] Determine the priority of the computing task. If there is a high-priority task, select the priority strategy;

[0024] Determine the resource utilization of the computing nodes. If there is an imbalance in resource utilization, select the resource utilization maximization strategy.

[0025] Furthermore, the task execution module of the Java-based distributed computing task scheduling system monitors the execution process of the computing task, including: monitoring the task execution status and execution time information.

[0026] The benefits of the present invention are:

[0027] 1. The present invention realizes the effective scheduling and management of distributed computing tasks. By real-time monitoring and management of computing resources, and selecting appropriate scheduling strategies based on task information and computing resource information, the resource utilization rate and computing efficiency of the system are improved.

[0028] 2. The present invention provides a variety of scheduling strategies, including load balancing strategy, priority strategy and resource utilization maximization strategy, which can be flexibly selected according to different task requirements and system status, thereby improving the flexibility and adaptability of the system.

[0029] 3. The present invention monitors and manages the execution process of computing tasks, can understand the execution status and progress of tasks in real time, promptly discover and handle problems that arise during the execution of tasks, and improve the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the system framework of the present invention.

[0031] Figure 2 It is a schematic diagram of the application process of the method of the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0033] Example 1

[0034] The present invention provides a Java-based distributed computing task scheduling method, comprising:

[0035] Step 1: Receive the computing tasks submitted by the user, preprocess the computing tasks, and generate task information.

[0036] Specifically, the steps include: receiving computing tasks submitted by users; parsing the computing tasks to extract information such as the task type, input data, and computational amount of the computing tasks; and generating task information based on the parsed computing task information.

[0037] Step 2: Monitor and manage computing resources in the distributed computing system and obtain computing resource information, which includes the status and performance parameter information of computing nodes.

[0038] Specifically, it may include: real-time monitoring of computing resources in a distributed computing system, including information such as the status and performance parameters of computing nodes.

[0039] Step 3: Select an appropriate preset scheduling strategy based on the task information and computing resource information to generate a task scheduling plan. The scheduling strategy includes a load balancing strategy, a priority strategy, and a resource utilization maximization strategy. The load balancing strategy is used to assign computing tasks to nodes with lower loads for execution based on the load conditions of the computing nodes; the priority strategy is used to assign and execute tasks from high to low according to the priority of the computing tasks; and the resource utilization maximization strategy is used to assign computing tasks to nodes with lower resource utilization for execution based on the resource utilization conditions of the computing nodes.

[0040] You can make judgments in turn to select a scheduling strategy. First, judge the load of the computing node. If there is a load imbalance, select a load balancing strategy.

[0041] Determine the priority of the computing task. If there is a high-priority task, select the priority strategy;

[0042] Determine the resource utilization of the computing nodes. If there is an imbalance in resource utilization, select the resource utilization maximization strategy.

[0043] Step 4: According to the task scheduling plan, the computing tasks are assigned to the corresponding computing nodes for execution, and the execution process of the computing tasks is monitored, and the execution results are returned to the user.

[0044] Example 2

[0045] The present invention also provides a Java-based distributed computing task scheduling system, including a task management module, a resource management module, a scheduling strategy module and a task execution module.

[0046] The task management module receives computing tasks submitted by users, preprocesses the computing tasks, generates task information and sends it to the scheduling strategy module.

[0047] The task management module may include a task receiving unit, a task parsing unit and a task information generating unit.

[0048] A task receiving unit, used for receiving computing tasks submitted by users;

[0049] A task parsing unit is used to parse the computing task and extract information such as the task type, input data, and computing amount of the computing task;

[0050] The task information generation unit is used to generate task information according to the parsed computing task information and send the task information to the scheduling strategy module.

[0051] The resource management module monitors and manages the computing resources in the distributed computing system, obtains computing resource information and sends it to the scheduling strategy module. The computing resource information includes the status and performance parameter information of the computing nodes.

[0052] The resource management module may include a resource monitoring unit and a resource information sending unit.

[0053] Resource monitoring unit, used to monitor computing resources in the distributed computing system in real time, including information such as the status and performance parameters of computing nodes;

[0054] The resource information sending unit is used to send the monitored computing resource information to the scheduling strategy module.

[0055] The scheduling strategy module selects an appropriate preset scheduling strategy based on the task information and computing resource information, generates a task scheduling plan and sends it to the task execution module. The scheduling strategy includes a load balancing strategy, a priority strategy and a resource utilization maximization strategy. The load balancing strategy is used to assign computing tasks to nodes with lower loads for execution based on the load conditions of the computing nodes; the priority strategy is used to assign and execute tasks from high to low according to the priority of the computing tasks; and the resource utilization maximization strategy is used to assign computing tasks to nodes with lower resource utilization for execution based on the resource utilization conditions of the computing nodes.

[0056] The scheduling strategy module may include a strategy selection unit, a strategy execution unit and a task scheduling solution generation unit.

[0057] The strategy selection unit is used to select an appropriate scheduling strategy according to the task information and the computing resource information; further, the scheduling strategy includes a load balancing strategy, a priority strategy and a resource utilization maximization strategy; the load balancing strategy is used to assign the computing tasks to the nodes with lower load for execution according to the load conditions of the computing nodes; the priority strategy is used to assign the high-priority tasks for execution according to the priorities of the computing tasks; the resource utilization maximization strategy is used to assign the computing tasks to the nodes with lower resource utilization for execution according to the resource utilization conditions of the computing nodes,

[0058] You can make judgments in turn to select a scheduling strategy. First, judge the load of the computing node. If there is a load imbalance, select a load balancing strategy.

[0059] Determine the priority of the computing task. If there is a high-priority task, select the priority strategy;

[0060] Determine the resource utilization of the computing nodes. If there is an imbalance in resource utilization, select the resource utilization maximization strategy.

[0061] The strategy execution unit is used to execute the selected scheduling strategy and generate a task scheduling plan based on task information and computing resource information.

[0062] The task scheduling scheme generating unit is used to send the generated task scheduling scheme to the task execution module.

[0063] The task execution module assigns computing tasks to corresponding computing nodes for execution according to the task scheduling plan, monitors the execution process of computing tasks, and returns the execution results to the user.

[0064] The task execution module may include a task allocation unit, a task execution monitoring unit and an execution result returning unit.

[0065] A task allocation unit is used to allocate computing tasks to corresponding computing nodes for execution according to a task scheduling scheme;

[0066] The task execution monitoring unit is used to monitor the execution process of the computing task, including task execution status, execution time and other information;

[0067] The execution result returning unit is used to return the execution result of the computing task to the user.

[0068] As the information interaction and execution process between the modules in the above-mentioned system are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0069] Likewise, the benefits of the system of the present invention are:

[0070] 1. Effective scheduling and management of distributed computing tasks is achieved. By real-time monitoring and management of computing resources, and selecting appropriate scheduling strategies based on task information and computing resource information, the system's resource utilization and computing efficiency are improved.

[0071] 2. It provides a variety of scheduling strategies, including load balancing strategy, priority strategy and resource utilization maximization strategy, which can be flexibly selected according to different task requirements and system status, improving the flexibility and adaptability of the system.

[0072] 3. By monitoring and managing the execution process of computing tasks, you can understand the execution status and progress of tasks in real time, promptly discover and handle problems that arise during task execution, and improve the stability and reliability of the system.

[0073] It should be noted that not all steps and modules in the above-mentioned processes and system structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0074] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A Java-based distributed computing task scheduling method, characterized by include: Step 1: Receive the computing tasks submitted by the user, pre-process the computing tasks, and generate task information. Step 2: Monitor and manage computing resources in the distributed computing system and obtain computing resource information, including the status and performance parameter information of computing nodes. Step 3: Select an appropriate preset scheduling strategy based on the task information and computing resource information to generate a task scheduling plan. The scheduling strategy includes a load balancing strategy, a priority strategy, and a resource utilization maximization strategy. The load balancing strategy is used to assign computing tasks to nodes with lower loads for execution based on the load conditions of the computing nodes; the priority strategy is used to assign and execute tasks from high to low according to the priority of the computing tasks; The resource utilization maximization strategy is used to assign computing tasks to nodes with lower resource utilization rates for execution according to the resource utilization rates of computing nodes; Step 4: According to the task scheduling plan, the computing tasks are assigned to the corresponding computing nodes for execution, and the execution process of the computing tasks is monitored, and the execution results are returned to the user.

2. A Java-based distributed computing task scheduling method according to claim 1, characterized in that In step 1, the computing task is preprocessed, including: parsing the computing task, extracting the task type, input data, and computing amount information of the computing task, and generating task information based on the extracted information.

3. The Java-based distributed computing task scheduling method according to claim 1, characterized in that In step 3, the load conditions of the computing nodes are determined in turn. If there is a load imbalance, a load balancing strategy is selected. Determine the priority of the computing task. If there is a high-priority task, select the priority strategy; Determine the resource utilization of the computing nodes. If there is an imbalance in resource utilization, select the resource utilization maximization strategy.

4. The Java-based distributed computing task scheduling method according to claim 1, characterized in that In step 4, the execution process of the computing task is monitored, including: monitoring the task execution status and execution time information.

5. A Java-based distributed computing task scheduling system, characterized by It includes task management module, resource management module, scheduling strategy module and task execution module. The task management module receives computing tasks submitted by users, pre-processes the computing tasks, generates task information and sends it to the scheduling strategy module. The resource management module monitors and manages the computing resources in the distributed computing system, obtains computing resource information and sends it to the scheduling strategy module. The computing resource information includes the status and performance parameter information of the computing nodes. The scheduling strategy module selects an appropriate preset scheduling strategy based on the task information and computing resource information, generates a task scheduling plan and sends it to the task execution module. The scheduling strategy includes a load balancing strategy, a priority strategy and a resource utilization maximization strategy. The load balancing strategy is used to assign computing tasks to nodes with lower loads for execution according to the load conditions of the computing nodes; the priority strategy is used to assign and execute tasks from high to low according to the priority of the computing tasks; The resource utilization maximization strategy is used to assign computing tasks to nodes with lower resource utilization rates for execution according to the resource utilization rates of computing nodes; The task execution module assigns computing tasks to corresponding computing nodes for execution according to the task scheduling plan, monitors the execution process of computing tasks, and returns the execution results to the user.

6. A Java-based distributed computing task scheduling system according to claim 5, characterized in that The task management module pre-processes the computing tasks, including: parsing the computing tasks, extracting the task type, input data, and computing amount information of the computing tasks, and generating task information based on the extracted information.

7. A Java-based distributed computing task scheduling system according to claim 5, characterized in that The scheduling strategy module determines the load conditions of the computing nodes in turn. If there is a load imbalance, a load balancing strategy is selected; Determine the priority of the computing task. If there is a high-priority task, select the priority strategy; Determine the resource utilization of the computing nodes. If there is an imbalance in resource utilization, select the resource utilization maximization strategy.

8. A Java-based distributed computing task scheduling system according to claim 5, characterized in that The task execution module monitors the execution process of the computing task, including: monitoring the task execution status and execution time information.

Images