Cluster scheduling model construction method, scheduling model, scheduling method and system
A scheduling model and construction method technology, applied in the field of big data processing, can solve the problems of lack of scheduling decision-making and single scheduling basis of the cluster, and achieve the effect of ensuring stable and efficient operation and realizing elastic scaling
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0059] The cluster scheduling model construction method of the present invention is used to realize the elastic scaling of large data cluster resources, including the following steps:
[0060] S100. Collect sample data of optimal adjustment points through actual set resource scheduling, where the sample data includes big data cluster specifications, cluster load, expected cluster adjusted load, and optimal number of adjustment nodes;
[0061] S200. Perform normalization processing on the sample data to obtain preprocessed sample data;
[0062] S300. Construct a prediction model based on the deep neural network of backpropagation. The prediction model takes the big data cluster specification, cluster load and expected cluster adjustment load as input, and outputs the optimal number of adjustment nodes;
[0063] S400. Train the prediction model based on the preprocessed sample data, and adjust the neuron weights of each layer of the prediction model based on the backpropagation ...
Embodiment 2
[0084] The cluster scheduling model of the present invention is a post-training prediction model obtained through the cluster scheduling model construction method disclosed in Embodiment 1. The post-training prediction model includes an input layer, a hidden layer, and an output layer, and is used for large data cluster specifications, The cluster load and the expected cluster adjusted load are input, and the optimal number of adjusted nodes is predicted and output.
[0085] The trained model takes big data cluster specifications, cluster load, and expected cluster adjusted load as input, predicts and outputs the optimal number of adjustment nodes, and uses the optimal number of adjustment nodes as a scheduling strategy to perform elastic scaling of big data cluster resources.
Embodiment 3
[0087] The cluster scheduling method of the present invention is used to realize the scheduling of large data cluster resource elastic scaling, including the following steps:
[0088] S100. Acquire big data cluster specifications, cluster load, and expected cluster adjusted load as input data, and normalize the input data to obtain preprocessed input data;
[0089] S200. Using the preprocessed input data as input, obtain the number of nodes that need to be adjusted by predicting through the big data cluster elastic scaling scheduling model disclosed in Embodiment 2.
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com