The invention discloses a resource collaborative awareness-based
Storm flow calculation dynamic scheduling method, and provides the following technical scheme aiming at the problem that the performance of a
Storm default scheduling
algorithm is reduced in node abnormity, resource fluctuation and rescheduling scenes: firstly, dynamically sensing node busy,
downtime and data flow fluctuation events through a real-time monitoring module; triggering a rescheduling process; secondly, constructing a resource collaborative optimization model based on historical task instance resource requirements, node performance indexes and communication overhead, and generating a task instance allocation scheme by taking minimization of inter-node communication cost as a target and combining CPU / memory dynamic threshold constraints; further, a
greedy algorithm is adopted to sort high-relevance task instances, the high-relevance task instances are preferentially distributed to nodes with the optimal historical performance, and it is ensured that the node
resource utilization rate does not exceed a dynamic threshold value; meanwhile, node load balancing parameters are corrected in real time through time window
smoothing processing, and the remaining resource state is updated; and finally, outputting an optimized topology division result, so that the
system delay after rescheduling is remarkably reduced, and the
throughput is improved. According to the method, rapid
recovery and stable operation of the
heterogeneous cluster are realized through resource collaborative modeling, historical
data driven dynamic scheduling and load balancing optimization.