Spark fine-grained job scheduling method based on DE-PPO hybrid algorithm

By adopting the fine-grained job scheduling method of DE-PPO hybrid algorithm in Spark, the problems of low efficiency and high cost of Spark job scheduling in hybrid cloud environments are solved. Through the combination of deep reinforcement learning and differential evolution algorithm, the job scheduling strategy is optimized. , realizes collaborative optimization of resource environment and operation characteristics, reduces costs and improves performance.

CN118963966APending Publication Date: 2024-11-15CHONGQING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411155101.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing Spark job scheduling method is difficult to fully consider heterogeneous clusters and job characteristics in hybrid cloud environments, resulting in low resource utilization efficiency and high cost, and lack of effective theoretical guidance on reward function design.

Method used

Using Spark fine-grained job scheduling method based on DE-PPO hybrid algorithm, the Spark fine-grained job scheduling model in a hybrid cloud environment is constructed and trained, and online data is obtained in real time through scheduler, cluster manager, reward generator and reward optimizer. Information, design fine-grained reward functions, optimize job scheduling strategies to reduce costs and improve performance.

Benefits of technology

Through the combination of deep reinforcement learning and differential evolution algorithms, adaptive cluster environment and workload characteristics, explore resource environment characteristics and job characteristics, optimize job executor placement, significantly reduce cluster operation costs and improve execution performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118963966A_ABST
    Figure CN118963966A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of reinforcement learning and big data processing, and particularly relates to a Spark fine-grained job scheduling method based on a DE-PPO hybrid algorithm, and the method comprises the steps: constructing and training a Spark fine-grained job scheduling model based on a hybrid cloud environment, obtaining online data information in real time, inputting the data information into the trained Spark fine-grained job scheduling model, and completing the scheduling of the Spark fine-grained job. The Spark fine-grained job scheduling model carries out job scheduling prediction on the data information, and the system selects a scheme with the lowest job scheduling cost to carry out job scheduling according to the job scheduling prediction; wherein the Spark fine-grained job scheduling model comprises a scheduler, a cluster manager, a reward generator and a reward optimizer; according to the method, the cluster performance is improved and the use cost of the cluster is reduced by learning the resource environment features and the job features.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of reinforcement learning and big data processing, and specifically relates to a Spark fine-grained job scheduling method based on a DE-PPO hybrid algorithm. Background Art

[0002] As one of the most popular big data processing frameworks, Spark is highly respected in both industry and academia for its excellent data processing capabilities and high scalability. However, in a distributed cluster environment, how to effectively schedule jobs and ensure the reasonable allocation and efficient use of resources has always been an important challenge facing Spark. This requires not only that the scheduling strategy can ensure the stable operation of jobs, but also that the cost of using the cluster be reduced while improving the execution performance of cluster jobs. The realization of this goal is of great significance to improving the efficiency of big data processing, optimizing resource utilization, and reducing enterprise operating costs.

[0003] With the rapid growth of data volume, more and more enterprises are trying to deploy Spark clusters in hybrid cloud environments. By combining local cloud environments and public cloud environments, enterprises can save costs as much as possible while meeting the growing demand for computing resources. However, this hybrid deployment method also brings greater challenges to Spark job scheduling. Existing heuristic-based scheduling methods are often designed for specific scenarios and goals, and it is difficult to fully consider the potential optimization of heterogeneous cluster characteristics and job characteristics to improve cluster execution performance and reduce cluster usage costs. It is complex and unrealistic to design and model all hybrid environments and multi-objective tasks. Deep reinforcement learning scheduling agents can capture various inherent cluster and workload characteristics by learning from experience, adapt to environmental characteristics, and optimize various objectives in the process of learning to maximize cumulative rewards. Reward function design plays a pivotal role in deep reinforcement learning and is directly related to the optimization results of the goal. However, there is currently no theory to guide how to design reward functions in hybrid cloud environments. Summary of the invention

[0004] In order to achieve the purpose of reducing the cost of Spark cluster during the operation service and improving the execution performance of cluster jobs, the present invention proposes a Spark fine-grained job scheduling method based on the DE-PPO hybrid algorithm, constructs and trains a Spark fine-grained job scheduling model based on a hybrid cloud environment, obtains online data information in real time, inputs the data information into the trained Spark fine-grained job scheduling model, and the Spark fine-grained job scheduling model performs job scheduling on the data information. The system selects the scheme with the lowest job scheduling cost to perform job scheduling according to the job scheduling result; wherein, the Spark fine-grained job scheduling model includes:

[0005] The scheduler is used to receive the cluster status information output by the cluster manager and output an action strategy according to the cluster status information output by the cluster manager;

[0006] The cluster manager receives the action strategy output by the scheduler and outputs the next cluster status information;

[0007] The reward generator is used to calculate rewards based on the action strategy and cluster status information, and combine the rewards within a cycle into a reward combination; combined with the reward optimizer, it feeds back the optimized reward combination to the scheduler;

[0008] The reward optimizer is used to collect and optimize the reward combination output by the reward generator. Furthermore, the specific processing flow of the Spark fine-grained job scheduling model includes:

[0009] S11. The scheduler reads the job information of the current job and outputs a policy action according to the cluster status information; if the policy action is to place an executor for the current job, then proceed to step S12; if the policy action is a waiting action, then proceed to step S14;

[0010] S12. The reward generator detects whether the node where the executor is placed in the strategy action meets the resource constraint. If so, step S13 is executed. If not, a basic negative reward is given and the process ends.

[0011] S13. The cluster manager executes the strategy action to complete an executor placement for the current job and updates the job information and cluster status information; the reward generator determines whether all executors of the current job have completed placement. If so, step S16 is executed. If not, a basic positive reward is given and the process directly returns to step S11;

[0012] S14. The reward generator checks whether the cluster environment has sufficient resources. If so, a basic negative reward is given and the process returns directly to step S11; if not, step S15 is executed;

[0013] S15. Check whether some executors of the current job have completed placement. If so, give a basic negative reward and end the process; if not, execute the strategy action to wait for the cluster to release resources, then give a basic negative reward and directly return to step S11;

[0014] S16. Perform fine-grained reward operations; then all executors of the current job execute and update the cluster status information;

[0015] S17. The reward generator determines whether the scheduling of all jobs is completed. If so, a plot reward is given and the process ends. If not, a basic positive reward is given, the next job in the job queue is made the current job, and then returns to step S11.

[0016] Furthermore, step S16 performs fine-grained reward operations including:

[0017] S161. If the job type of the current job is computationally intensive, determine whether all executors of the current job are placed on the same node. If so, give a basic negative reward and execute step S17. If not, execute step S162.

[0018] S162. Determine whether all executors of the current job are evenly placed on different nodes and no new nodes are started. If so, give a basic positive reward and execute step S17. If not, execute step S17;

[0019] S163. If the job type of the current job is network intensive, determine whether all executors of the current job are placed in different cloud environments. If so, give a basic negative reward and execute step S17; if not, execute step S164;

[0020] S164. Determine whether all executors of the current job are evenly placed on different nodes. If so, give a basic negative reward and execute step S17; if not, execute step S165;

[0021] S165. Determine whether all executors of the current job are placed on the same node. If so, give a basic positive reward and execute step S17. If not, execute step S17.

[0022] Furthermore, the calculation formula for the plot reward is:

[0023] R epi =R fixed ×[β×(1-Cost norm )+(1-β)×(1-Bsd norm )]

[0024] Among them, R epi Represents the episode reward value, R fixed represents a fixed reward value, β represents a balance parameter, and Cost norm Represents the normalized cluster usage cost, Bsd norm Represents the normalized cluster performance indicator.

[0025] Furthermore, the calculation formula of the normalized cluster performance index is:

[0026]

[0027]

[0028] Among them, Bsd avg Indicates the average bounded slowdown of cluster jobs, Bsd minIndicates the maximum bounded deceleration of cluster jobs, Bsd max Indicates the minimum bounded deceleration of cluster operations; bsd j represents the time ratio of job j, represents the waiting time of job j, represents the execution time of job j, τ represents a constant, and M represents the number of jobs.

[0029] Furthermore, the calculation formula for the normalized cluster usage cost is:

[0030]

[0031] Among them, Cost max Cost min Respectively represent the maximum cost of cluster usage and the minimum cost of cluster usage; Cost avg is the average usage cost of the cluster, N represents the number of nodes in the hybrid cloud environment, represents the total usage time of node i, represents the hourly price of node i.

[0032] Furthermore, the reward optimizer includes a reward function replayer, which is used to save the reward combination and its fitness output by the reward generator, and also save the reward combination and its fitness generated during the processing of the reward optimizer.

[0033] Furthermore, the specific processing flow of the reward optimizer includes:

[0034] S21. Receive a reward combination output by the reward generator, and use a reward function replay device to match an existing reward combination for the reward combination;

[0035] S22. Determine whether the reward combination matches an existing reward combination. If the match is successful, directly set the fitness of the reward combination to the fitness of the existing reward combination, and then execute step S23; if the match fails, directly execute step S23;

[0036] S23. Based on the 7 verified convergence reward combinations and the 16 randomly generated convergence reward combinations; each convergence reward combination is taken as an individual to form a population;

[0037] S24. Calculate the fitness of each individual in the population and store it in the reward function replay device;

[0038] S25. Perform mutation and crossover operations on each individual in the population as a parent individual to obtain a corresponding offspring individual to form a sub-population;

[0039] S26. Use the reward function replay device to match each offspring individual in the subpopulation. If the offspring individual is successfully matched, the offspring individual uses the fitness corresponding to the matched existing reward combination as its own fitness. If the individual fails to match, the fitness of the offspring individual itself is calculated and stored in the reward function replay device;

[0040] S27. Compare each offspring individual in the sub-population with its parent individual, retain individuals with high fitness to form a new population, and determine whether the maximum number of iterations has been reached. If so, output the optimal individual, if not, return to step S25.

[0041] Beneficial effects of the present invention:

[0042] The present invention performs Spark job scheduling in a hybrid cloud environment, utilizes deep reinforcement learning to adapt to the characteristics of cluster environment and workload, and the excellent global search capability of differential evolution (DE) algorithm, and reduces cluster operation costs and improves cluster execution performance by continuously exploring and optimizing reward function weight configuration.

[0043] In view of the problem that the configuration reward function only focuses on resource constraints in the existing deep reinforcement learning scheduling methods for Spark job scheduling, the present invention designs a fine-grained reward function based on cost-effectiveness to further guide the deep reinforcement learning scheduling agent to explore the potential optimization relationship between resource environment characteristics and job characteristics on job executor placement, so that the deep reinforcement learning scheduling agent can improve cluster performance and reduce cluster usage costs by learning resource environment characteristics and job characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A Spark fine-grained job scheduling model in a hybrid cloud environment of the present invention;

[0045] Figure 2 is a flowchart of the reward generator of the present invention;

[0046] Figure 3 This is the fine-grained operation process of the present invention;

[0047] Figure 4 This is the fine-grained reward optimizer model of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The present invention provides a Spark fine-grained job scheduling method based on the DE-PPO hybrid algorithm, including building and training a Spark fine-grained job scheduling model based on a hybrid cloud environment, acquiring online data information in real time, inputting the data information into the trained Spark fine-grained job scheduling model, and the Spark fine-grained job scheduling model predicts job scheduling for the data information. The system selects the solution with the lowest job scheduling cost to perform job scheduling based on the job scheduling prediction. Figure 1 As shown in the figure, the Spark fine-grained job scheduling model includes:

[0050] The scheduler is used to receive the cluster status information output by the cluster manager and output an action strategy according to the cluster status information output by the cluster manager;

[0051] The cluster manager receives the action strategy output by the scheduler and outputs the next cluster status information;

[0052] The reward generator is used to calculate rewards based on the action strategy and cluster status information, and combine the rewards within a cycle into a reward combination; combined with the reward optimizer, it feeds back the optimized reward combination to the scheduler;

[0053] The reward optimizer is used to collect and optimize the reward combination output by the reward generator.

[0054] In a hybrid cloud environment, users can easily share data and applications between local clouds and public clouds. Both local clouds and public clouds provide users with available resources, among which local clouds have lower operating costs and less expenses when processing simple tasks; public clouds usually have more resources and stronger computing power, enabling them to support large-scale task execution. In the present invention, when assigning tasks, the scheduler is no longer limited to simply considering the resource constraints between clusters, nodes and jobs. Instead, with the guidance provided by the reward generator, the scheduler begins to pay attention to the characteristics of the job type, the status of each node, and the cloud environment in which they are located, and formulates the best deployment strategy for the executors based on these factors.

[0055] Specifically, in a hybrid cloud environment, the scheduler should formulate an effective scheduling strategy based on the job type and cluster environment status. For example, when scheduling network-intensive jobs, the executors should be placed in the same cloud environment or even the same node to reduce transmission delays, improve performance and reduce costs; when scheduling compute-intensive jobs, the executors should be dispersed to different nodes to reduce interference between high-load executors in the same node and improve the execution performance of compute-intensive jobs. Considering the cost advantages of local cloud resources and the performance characteristics of public cloud resources, the DRL (Deep Reinforcement Learning) scheduling mechanism should prioritize the deployment of job executors on low-cost local nodes on the basis of meeting overall performance requirements, thereby reducing the total cost of the cluster.

[0056] To this end, the present invention uses a reward generator to provide feedback on whether the scheduling strategy complies with the resource constraints of the heterogeneous environment, and at the same time gives a finer-grained reward for the scheduling strategy and job characteristics, so as to guide the scheduler to make a finer-grained scheduling strategy. Therefore, after successfully completing the placement of all executors for a job, the reward generator will provide feedback based on the complete scheduling strategy of the job. When the job is network-intensive and the executors placed for the job are distributed in different cloud environments, a penalty is given to the scheduler; when the job is network-intensive and the executors placed for the job are evenly distributed in different nodes, a penalty is given; when the job is network-intensive and the executors placed for the job are distributed in the same node, a reward is given; when the job is compute-intensive and the executors placed for the job are distributed in the same node, a penalty is given; when the job is compute-intensive and the executors placed for the job are evenly distributed in different boot nodes (non-newly started nodes), a reward is given. There are multiple nodes in a cluster that can be used to place executors. When a node has no executors, that is, no resources are used, it can be regarded as a newly started node. Reducing the use of newly started nodes can save costs to a certain extent.

[0057] A job requires multiple executors to complete. In the present invention, the scheduler will form a policy action for each executor of the job. The reward generator has two parts for the action feedback of each job. The first part is the feedback on the policy action of each executor, and the second part is the feedback on the scheduling strategy after all executors of the job are placed.

[0058] Specifically, Figure 2 As shown, the specific processing flow of the reward generator includes:

[0059] S11. The scheduler reads the job information of the current job and outputs a policy action according to the cluster status information; if the policy action is to place an executor for the current job, then proceed to step S12; if the policy action is a waiting action, then proceed to step S14;

[0060] S12. The reward generator detects whether the node where the executor is placed in the strategy action meets the resource constraint. If so, step S13 is executed. If not, a basic negative reward is given and the process ends.

[0061] Specifically, it is detected whether the node where the executor is placed in the strategy action meets the resource constraint, that is, it is determined whether the current available memory and number of CPU cores of the node are greater than or equal to the available memory and number of CPU cores required by the executor; if the current available memory of the node is greater than or equal to the available memory required by the executor, and the current number of CPU cores of the node is greater than or equal to the number of CPU cores required by the executor, it means that the resource constraint is met, otherwise the resource constraint is not met.

[0062] S13. The cluster manager executes the strategy action to complete an executor placement for the current job and updates the job information and cluster status information; the reward generator determines whether all executors of the current job have completed placement. If so, step S16 is executed. If not, a basic positive reward is given and the process directly returns to step S11;

[0063] S14. The reward generator checks whether the cluster environment has sufficient resources. If so, a basic negative reward is given and the process returns directly to step S11; if not, step S15 is executed;

[0064] S15. Check whether some executors of the current job have completed placement. If so, give a basic negative reward and end the process; if not, execute the strategy action to wait for the cluster to release resources, then give a basic negative reward and directly return to step S11;

[0065] S16. Execute fine-grained reward operations; then all executors of the current job execute, and update the cluster status information;

[0066] S17. The reward generator determines whether the scheduling of all jobs is completed. If so, a plot reward is given and the process ends. If not, a basic positive reward is given, the next job in the job queue is made the current job, and then returns to step S11.

[0067] Specifically, Figure 3 As shown, step S16 performs fine-grained reward operations including:

[0068] S161. If the job type of the current job is computationally intensive, determine whether all executors of the current job are placed on the same node. If so, give a basic negative reward and execute step S17. If not, execute step S162.

[0069] S162. Determine whether all executors of the current job are evenly placed on different nodes and no new nodes are started. If so, give a basic positive reward and execute step S17. If not, execute step S17;

[0070] S163. If the job type of the current job is network intensive, determine whether all executors of the current job are placed in different cloud environments. If so, give a basic negative reward and execute step S17; if not, execute step S164;

[0071] S164. Determine whether all executors of the current job are evenly placed on different nodes. If so, give a basic negative reward and execute step S17; if not, execute step S165;

[0072] S165. Determine whether all executors of the current job are placed on the same node. If so, give a basic positive reward and execute step S17. If not, execute step S17.

[0073] Specifically, the calculation formula for plot rewards is:

[0074] R epi =R fixed ×[β×(1-Cost norm )+(1-β)×(1-Bsd norm )]

[0075] Among them, R epi Represents the episode reward value, R fixed Indicates a fixed reward value, Cost norm Represents the normalized cluster usage cost, Bsd norm Represents the normalized cluster performance indicator.

[0076] Giving a basic positive reward mentioned in the present invention means giving a reward once, and giving a basic negative reward means giving a punishment once, and the size of the reward or punishment depends on the circumstances.

[0077] Specifically, the ultimate goal of the present invention is to reduce the cost of the Spark cluster during the service operation and improve the execution performance of the cluster job, which is expressed as:

[0078] Target = β × Cost norm +(1-β)×Bsd norm

[0079] Target is the weighted sum of cluster usage cost and execution performance index. The smaller the value, the better the optimization effect. Among them, β represents the balance parameter for balancing cluster usage cost and cluster performance during the optimization process. If the value of β is large, it means that the total usage cost of the cluster is more inclined to be reduced during the optimization process; on the contrary, if the value of β is small, it reflects the optimization direction that focuses more on improving cluster performance. If β is assigned a value of 0, it means that the focus is entirely on optimizing cluster performance, while ignoring the optimization of the total usage cost of the cluster; when β = 0.5, it indicates that the cluster performance and total usage cost are balanced during the optimization process; if β is set to 1, it clearly indicates that the optimization goal is only to minimize the total usage cost of the cluster, without considering the impact on cluster performance. By changing the configuration of β, the optimization weights of cost targets and performance targets can be controlled.

[0080] The present invention introduces the average bounded slowdown (BSD) to describe the cluster performance index, which combines the waiting time and the execution time to provide a more comprehensive performance index. In order to balance the impact of the difference in the range of the cost index and the average bounded slowdown index on the final optimization goal, the present invention performs Min-Max normalization on the two goals.

[0081] The calculation formula of the normalized cluster performance index is:

[0082]

[0083] Among them, Bsd avg Indicates the average bounded slowdown of cluster jobs, Bsd min Indicates the maximum bounded deceleration of cluster jobs, Bsd max Indicates the minimum bounded deceleration of cluster operations; bsd j represents the time ratio of job j, represents the waiting time of job j, represents the execution time of job j, τ represents a constant, and M represents the number of jobs.

[0084] The calculation formula for the normalized cluster usage cost is:

[0085]

[0086] Among them, Cost max Cost min Respectively represent the maximum cost of cluster usage and the minimum cost of cluster usage; Cost avg is the average usage cost of the cluster, expressed as:

[0087]

[0088] Cost clusterIndicates the total cost of cluster usage. The total cost formula of the cluster is:

[0089]

[0090] Where N represents the number of nodes in the hybrid cloud environment. represents the total usage time of node i, represents the hourly price of node i.

[0091] In the present invention, the placement of all actuators that complete all jobs is regarded as a cycle. There are multiple strategic actions in each cycle, and the reward generator generates a reward for each strategic action (the reward is a basic positive reward, or a basic negative reward, or a plot reward), forming a reward combination, that is, the process of the previous steps S11-S17. The fitness corresponding to the reward combination is the negative value (opposite number) of the sum of all reward values ​​in the reward combination.

[0092] Specifically, each job needs to be allocated to multiple executors, and the allocation of each executor corresponds to a reward. The placement of all executors of all jobs in a complete scheduling plan will face a lot of rewards. For this reason, the present invention divides all rewards into 11 reward situations, and the reward value (reward weight) of the same reward situation is the same; in each cycle, the reward weights corresponding to the 11 reward situations calculated are combined into a reward combination for optimization. The division is shown in Table 1:

[0093] Table 1 Reward types

[0094]

[0095] Specifically, the reward optimizer also includes a reward function replayer, which is used to save the reward combination and its fitness output by the reward generator, and also saves the reward combination and its fitness generated during the processing of the reward optimizer.

[0096] The reward optimizer optimizes the reward combination output by the reward generator, such as Figure 4 As shown in the figure, the specific processing flow of the reward optimizer includes:

[0097] S21. Receive a reward combination output by the reward generator, and use a reward function replay device to match an existing reward combination for the reward combination;

[0098] S22. Determine whether the reward combination matches an existing reward combination. If the match is successful, directly set the fitness of the reward combination to the fitness of the existing reward combination, and then execute step S23 (that is, when calculating the individual fitness in the subsequent step S24, the fitness of the reward combination directly adopts the fitness of the matching existing reward combination to reduce time consumption); if the match fails, directly execute step S23;

[0099] S23. Based on the 7 verified convergence reward combinations and the 16 randomly generated convergence reward combinations (randomly generated in the reward space); each convergence reward combination is taken as an individual to form a population;

[0100] In order to reduce the training time, 8 convergence reward combinations that have been verified to converge are used. These 8 convergence reward combinations have passed experiments.

[0101] S24. Calculate the fitness of each individual in the population (here, mainly calculate the fitness of 16 randomly generated convergent reward combinations, and the fitness is the negative value of the sum of all reward values ​​in the convergent reward combination), and store it in the reward function replay device;

[0102] S25. Perform mutation and crossover operations on each individual in the population as a parent individual to obtain a corresponding offspring individual to form a sub-population;

[0103] S26. Use the reward function replay device to match each offspring individual in the subpopulation. If the offspring individual is successfully matched, the offspring individual uses the fitness corresponding to the matched existing reward combination as its own fitness. If the individual fails to match, the fitness of the offspring individual itself is calculated and stored in the reward function replay device;

[0104] S27. Compare each offspring individual in the sub-population with its parent individual, retain individuals with high fitness to form a new population, and determine whether the maximum number of iterations has been reached. If so, output the optimal individual (i.e., the optimal reward combination). If not, return to step S25.

[0105] In the present invention, unless otherwise clearly stipulated and limited, the terms such as "installation", "setting", "connection", "fixation" and "rotation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0106] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A Spark fine-grained job scheduling method based on DE-PPO hybrid algorithm, characterized in that: Build and train a Spark fine-grained job scheduling model based on a hybrid cloud environment, obtain online data information in real time, input the data information into the trained Spark fine-grained job scheduling model, and the Spark fine-grained job scheduling model schedules the data information. The system selects the solution with the lowest job scheduling cost for job scheduling based on the job scheduling results. The Spark fine-grained job scheduling model includes: The scheduler is used to receive the cluster status information output by the cluster manager and output an action strategy according to the cluster status information output by the cluster manager; The cluster manager receives the action strategy output by the scheduler and outputs the next cluster status information; The reward generator is used to calculate rewards based on the action strategy and cluster status information, and combine the rewards within a cycle into a reward combination; combined with the reward optimizer, it feeds back the optimized reward combination to the scheduler; The reward optimizer is used to collect and optimize the reward combination output by the reward generator.

2. According to claim 1, a Spark fine-grained job scheduling method based on DE-PPO hybrid algorithm is characterized in that: The specific processing flow of Spark's fine-grained job scheduling model includes: S11. The scheduler reads the job information of the current job and outputs a policy action according to the cluster status information; if the policy action is to place an executor for the current job, then proceed to step S12; if the policy action is a waiting action, then proceed to step S14; S12. The reward generator detects whether the node where the executor is placed in the strategy action meets the resource constraint. If so, step S13 is executed. If not, a basic negative reward is given and the process ends. S13. The cluster manager executes the strategy action to complete an executor placement for the current job and updates the job information and cluster status information; the reward generator determines whether all executors of the current job have completed placement. If so, step S16 is executed. If not, a basic positive reward is given and the process directly returns to step S11; S14. The reward generator checks whether the cluster environment has sufficient resources. If so, a basic negative reward is given and the process returns directly to step S11; if not, step S15 is executed; S15. Check whether some executors of the current job have completed placement. If so, give a basic negative reward and end the process; if not, execute the strategy action to wait for the cluster to release resources, then give a basic negative reward and directly return to step S11; S16. Perform fine-grained reward operations; then all executors of the current job execute and update the cluster status information; S17. The reward generator determines whether the scheduling of all jobs is completed. If so, a plot reward is given and the process ends. If not, a basic positive reward is given, the next job in the job queue is made the current job, and then returns to step S11.

3. According to claim 1, a Spark fine-grained job scheduling method based on DE-PPO hybrid algorithm is characterized in that: Step S16 performs fine-grained reward operations including: S161. If the job type of the current job is computationally intensive, determine whether all executors of the current job are placed on the same node. If so, give a basic negative reward and execute step S17. If not, execute step S162. S162. Determine whether all executors of the current job are evenly placed on different nodes and no new nodes are started. If so, give a basic positive reward and execute step S17. If not, execute step S17; S163. If the job type of the current job is network intensive, determine whether all executors of the current job are placed in different cloud environments. If so, give a basic negative reward and execute step S17; if not, execute step S164; S164. Determine whether all executors of the current job are evenly placed on different nodes. If so, give a basic negative reward and execute step S17; if not, execute step S165; S165. Determine whether all executors of the current job are placed on the same node. If so, give a basic positive reward and execute step S17. If not, execute step S17.

4. According to claim 2, a Spark fine-grained job scheduling method based on DE-PPO hybrid algorithm is characterized in that: The calculation formula for plot rewards is: R epi =R fixed ×[β×(1-Cost norm )+(1-β)×(1-Bsd norm )] Among them, R epi Represents the episode reward value, R fixed represents a fixed reward value, β represents a balance parameter, and Cost norm Represents the normalized cluster usage cost, Bsd norm Represents the normalized cluster performance indicator.

5. According to claim 4, a Spark fine-grained job scheduling method based on DE-PPO hybrid algorithm is characterized in that: The calculation formula of the normalized cluster performance index is: Among them, Bsd avg Indicates the average bounded slowdown of cluster jobs, Bsd min Indicates the maximum bounded deceleration of cluster jobs, Bsd max Indicates the minimum bounded deceleration of cluster operations; bsd j represents the time ratio of job j, represents the waiting time of job j, represents the execution time of job j, τ represents a constant, and M represents the number of jobs.

6. According to claim 4, a Spark fine-grained job scheduling method based on DE-PPO hybrid algorithm is characterized in that: The calculation formula for the normalized cluster usage cost is: Among them, Cost max Cost min Respectively represent the maximum cost of cluster usage and the minimum cost of cluster usage; Cost avg is the average usage cost of the cluster, N represents the number of nodes in the hybrid cloud environment, represents the total usage time of node i, represents the hourly price of node i.

7. The Spark fine-grained job scheduling method based on the DE-PPO hybrid algorithm according to claim 1 is characterized in that: The reward optimizer includes a reward function replayer, which is used to save the reward combination and its fitness output by the reward generator, and also saves the reward combination and its fitness generated during the reward optimizer processing.

8. The Spark fine-grained job scheduling method based on the DE-PPO hybrid algorithm according to claim 7 is characterized in that: The specific processing flow of the reward optimizer includes: S21. Receive a reward combination output by the reward generator, and use a reward function replay device to match an existing reward combination for the reward combination; S22. Determine whether the reward combination matches an existing reward combination. If the match is successful, directly set the fitness of the reward combination to the fitness of the existing reward combination, and then execute step S23; if the match fails, directly execute step S23; S23. Based on the 7 verified convergence reward combinations and the 16 randomly generated convergence reward combinations; each convergence reward combination is taken as an individual to form a population; S24. Calculate the fitness of each individual in the population and store it in the reward function replay device; S25. Perform mutation and crossover operations on each individual in the population as a parent individual to obtain a corresponding offspring individual to form a sub-population; S26. Use the reward function replay device to match each offspring individual in the subpopulation. If the offspring individual is successfully matched, the offspring individual uses the fitness corresponding to the matched existing reward combination as its own fitness. If the individual fails to match, the fitness of the offspring individual itself is calculated and stored in the reward function replay device; S27. Compare each offspring individual in the sub-population with its parent individual, retain individuals with high fitness to form a new population, and determine whether the maximum number of iterations has been reached. If so, output the optimal individual, if not, return to step S25.