Resource allocation method and device for Hadoop cluster, equipment and medium
A technology of hadoop clustering and resource allocation, applied in the direction of resource allocation, multi-programming device, program control design, etc., can solve the problems of incapable task queue resource allocation, limited ability of operation and maintenance personnel, low labor efficiency, etc., to reduce manual operation , the effect of improving utilization and improving efficiency
Pending Publication Date: 2022-03-25
CHINA PING AN PROPERTY INSURANCE CO LTD
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, due to the large number of task queues in the Hadoop cluster, manual resource allocation is inefficient
Or due to the limited ability of operation and maintenance personnel, it is impossible to quickly and accurately allocate resources to each task queue, resulting in waste of resources
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0023] The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.
[0024] In the field of big data technology, Hadoop clusters can provide massive data storage, query and analysis functions for various fields such as e-commerce, finance, industry, communication, and medical care. For example, Hadoop clusters can be used to store medical data in the field of medical technology, such as personal health records, prescriptions, inspection reports and other medical data. For another example, Hadoop clusters can be used to analyze financial data in the f...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The embodiment of the invention provides a Hadoop cluster resource allocation method and device, equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the feature extraction of task queue information of each task queue of a Hadoop cluster, obtaining a feature vector of each task queue, and obtaining a feature vector of each task queue; and calling a resource prediction model to perform prediction processing on the feature vector of each task queue to obtain a resource prediction result of each task queue, determining a target resource of each task queue according to the resource prediction result of each task queue, and performing resource allocation according to the target resource of each task queue. And resource allocation can be quickly and accurately carried out on each task queue in the Hadoop cluster. The present application can relate to a block chain technology, for example, a target resource of a task queue can be written into a block chain. The invention further relates to the technical field of digital medical treatment, for example, a Hadoop cluster can be used for executing tasks in the medical field.
Description
technical field [0001] The present application relates to the technical field of big data, and in particular to a resource allocation method, device, equipment and medium of a Hadoop cluster. Background technique [0002] In the field of big data technology, Hadoop clusters can provide massive data storage, query and analysis functions for various fields such as e-commerce, finance, industry, communication, and medical care. In order to meet the business requirements in various application scenarios, it is necessary to allocate resources to each task queue in the Hadoop cluster. In the existing method, resources can be allocated to each task queue in the Hadoop cluster manually. However, due to the large number of task queues in the Hadoop cluster, manual resource allocation is inefficient. Or due to the limited ability of operation and maintenance personnel, it is impossible to quickly and accurately allocate resources to each task queue, resulting in waste of resources. ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
CPCG06F9/5027G06F2209/5021
Inventor 王海昕刘竞迪
Owner CHINA PING AN PROPERTY INSURANCE CO LTD