Performance pre-evaluation based client cache distributing method and system

Abstract

The invention discloses a performance pre-evaluation based client cache distributing method. The performance pre-evaluation based client cache distributing method comprises the following procedures of: firstly, counting loads of different data nodes in a parallel file system and collecting information such as a network speed and a magnetic disk write-read speed in the parallel file system at the same time; performing performance pre-evaluation on different system client cache distribution strategies by using the counted and collected information; selecting a client cache distribution strategy capable of bringing maximum performances by the system based on a performance pre-evaluation result; giving different priorities to different write requests based on the selected client cache distribution strategy; distributing the client cache to the write requests with relatively high priorities; directly writing the write requests with relatively low priorities into a magnetic disk. The performance pre-evaluation based client cache distributing method can solve the problems of high priority and low efficiency existing in the client distribution strategy of the existing parallel file system and maximizes performance improvement which can be brought by the limited client cache.

Description

technical field [0001] The invention belongs to the field of distributed computing models, and more particularly relates to a performance estimation-based client buffer allocation method and system. Background technique [0002] In the era of big data, data-intensive computing is facing new opportunities and challenges. The traditional stand-alone file system is powerless in the face of the demand of big data, and the parallel file system can meet the high-speed read and write requirements brought by big data due to its high throughput. However, under high concurrent data requests, the load of data nodes will be unbalanced, resulting in a sharp decline in the performance of the parallel file system. Therefore, improving the performance of parallel file systems has become an important research topic. [0003] In a parallel file system, the file write strategy is the main factor to improve performance. Traditional distributed file system write strategies are mainly divided ...

AI Technical Summary

Problems solved by technology

[0004] Aiming at the above defects or improvement needs of the prior art, the present invention provides a client cache allocation method and system based on performance estimation, the purpose of which is to solve the problem of slow write operation and occupying a large amount of resources in the existing method. Caching resources and resulting in inefficient client-side caching technical issues

Images