Unlock instant, AI-driven research and patent intelligence for your innovation.

A file system management method that supports multi-job parallel IO performance isolation

A file system and management method technology, applied in the field of file system management that supports multi-job parallel IO performance isolation, can solve problems such as the inability to cope with multi-job interference and parallel IO balance requirements of high-performance computing platforms, and alleviate performance imbalance. , easy to implement, simple effect

Active Publication Date: 2022-07-12
JIANGNAN INST OF COMPUTING TECH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Data management methods in traditional parallel file systems cannot cope with the requirements of multi-job interference and parallel IO balance in high-performance computing platforms

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A file system management method that supports multi-job parallel IO performance isolation
  • A file system management method that supports multi-job parallel IO performance isolation
  • A file system management method that supports multi-job parallel IO performance isolation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0045] Embodiment: A file system management method that supports multi-job parallel IO performance isolation, including the following configurations:

[0046] Definition of file resource pool: including: list of service numbers, storage space quota on each server, maximum storage performance that can be used by the resource pool on each server, and system dynamic information. The resource pool has a unique resource pool identifier resource_id after allocation , the application in high-performance computing uses the resource_id to use the corresponding resource pool; the dynamic information includes information such as the current load, memory usage, and the number of allocated files on each storage server; multiple resource pools may contain A storage server, in this case, the server needs to provide a corresponding QoS mechanism to ensure that each resource pool can use the performance allocated to it.

[0047] File resource pool allocation: The parallel file system provides ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a file system management method supporting multi-job parallel IO performance isolation, including the following configuration: definition of file resource pool: including: service quantity list, storage space quota on each server, static information, The resource pool can use the maximum storage performance; file resource pool allocation: the parallel file system provides the file resource pool allocation interface. The file resource pool request includes the maximum performance of the requested resource pool, the IO usage mode of the resource pool, and the included data servers. Quantity, maximum allocated space; release of file resource pool: the parallel file system provides a release interface for file resource pool, through which the user or administrator releases resources in the file resource pool; load statistics in the resource pool: data in the file resource pool The evaluation of server load is achieved by monitoring the life cycle of files. The file system management method of the present invention alleviates the performance imbalance between processes in the parallel I / O process, and improves the parallel IO performance.

Description

technical field [0001] The invention belongs to the technical field of computer file system management, and in particular relates to a file system management method supporting multi-job parallel IO performance isolation. Background technique [0002] The application execution process in a high-performance computer system can typically be divided into two parts: computing and I / O. The ratio of computing and IO determines the efficiency of computing resources, and applications in high-performance computing are mostly performed in parallel IO for file systems. access. With the increasing computing scale, high-performance computing platforms often run a large number of concurrent applications at the same time. These concurrent applications share the entire storage system; IO accesses between applications will cause interference and competition in the storage system, causing fluctuations in the performance obtained by each job in the storage system. The current mainstream paral...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/11G06F16/18G06F9/50
CPCG06F16/122G06F16/1858G06F9/505G06F9/5022Y02D10/00
Inventor 陈起何晓斌魏巍余婷王红艳王涛肖伟
Owner JIANGNAN INST OF COMPUTING TECH