Method for dynamically configuring distributed cluster node label
A distributed cluster, node label technology, applied in the field of dynamically configuring distributed cluster node labels, can solve the problem that the server can only be idle
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0044] [Example 1: Dynamic labeling cluster deployment method]
[0045] image 3 is a flowchart of the dynamic labeling cluster deployment method, such as image 3 shown, including:
[0046] Label setting step S1, YARN starts the node label function, adds the required labels in the cluster, and establishes a node pool;
[0047] The label activation step S1 also includes the mapping node mapping step S11, assigning corresponding labels to the cluster nodes that need to be physically isolated, and then configuring the mapping relationship between labels and queues in the resource queue of YARN.
[0048] As shown in 4, it is a schematic diagram of the mapping relationship between tags and queues in the tag startup step of the present invention. Figure 4 , the queue a (queue_a) contains two nodes (node_1 / node_2) to establish a mapping relationship with the corresponding label a (label_a), and the two nodes (node_3 / node_4) contained in the queue b (queue_b) also correspond to t...
Embodiment 2
[0064] Such as Figure 9 Shown is a schematic diagram of an embodiment of the present invention, such as Figure 9 As shown, the same set of clusters supports the different needs of multiple businesses, including data warehouse ETL jobs, machine learning, JAVA class engines, etc. Different jobs have very different resource requirements for clusters. All kinds of jobs (jobs) are classified by queue, submitted to the corresponding job (job), and then the queue is configured with a label, and this label is given to a specific node, so that the job can obtain the corresponding node resources.
[0065] This embodiment can avoid data synchronization among multiple clusters. For example, the current data mining needs to update data about 300G every day, and the synchronization speed is 30M / s, which takes 3h.
[0066] At the same time, this embodiment can improve resource utilization. For example, a business needs to apply for 4 nodes for daily data modeling, and the modeling time is...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com