An automatic data equalization method and tool based on hadoop
A data and balanced technology, applied in the direction of data exchange network, electronic digital data processing, digital data information retrieval, etc., can solve the problems of ineffective utilization, network bandwidth consumption, uneven data distribution, etc., and reduce the cost of manual operation and maintenance , data balance and reasonable, guaranteed performance effect
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment Construction
[0028] The present invention provides a hadoop-based automatic data balancing method, which calls hadoop API through a java application program, obtains the disk usage rate of each data node, calculates the standard deviation of the disk usage rate of each data node, and real-time and preset startup data balancing script Start threshold comparison, the current standard deviation is greater than the start threshold, obtain the current cluster load through the Hadoop API, set the data balancing bandwidth and number of threads according to the load value, and then start the data balancing script in the background through the java application to start data balancing, and Check the current equalization progress through the application.
[0029] At the same time, a hadoop-based automatic data equalization tool corresponding to the above method is provided, including a call unit, a calculation unit, an analysis unit, a setting unit and a startup unit, and the communication connection ...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com