Method and system for reading and writing data of distributed file system

Abstract

The invention discloses a method for reading and writing data of a distributed file system. The method comprises the following steps of: firstly, acquiring data block meta information of a file to be read and written by a client side, and an IP address of the client side; secondly, analyzing a data block IP address included in the data block meta information; and finally, comparing the data block IP address with the IP address of the client side, and calling a local file system I / O (Input / Output) to read and write the file data when the data block IP address and the IP address of the client side are the same. Due to the adoption of the reading and writing method of the distributed file system, provided by the invention, when a judgment shows that the data block and the client side are located at the same data node, the local file system I / O of the client side is called for reading and writing the file data in the data block, so that the interactive speed of the client side and the data node is improved, the interactive time is shortened effectively, the reading and writing efficiency of the file and the calculation performance of an I / O interface are improved.

Description

technical field [0001] The present application relates to the field of distributed computing, in particular to a method and system for reading and writing data in a distributed file system. Background technique [0002] In a distributed file system, the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the node through a computer network. [0003] The I / O (input / output port, input / output) applied in the distributed file system includes L-I / O and D-I / O; where L-I / O refers to the I / O of the local file system, and D-I / O refers to the distributed file System I / O. Both L-I / O and D-I / O provide file read and write interfaces through which files can be read and written. The difference is that the operation object of L-I / O is located on the local disk, while the operation object of D-I / O may be Distributed on disks in different physical locations, they are connected through the network, so the task o...

AI Technical Summary

Problems solved by technology

This method shields the fact that the data is distributed and stored, which is transparent to users, but in terms of performance, the SOCKET method based on TCP / IP is much slower than L-I / O, because it is very difficult to establish a TCP / IP link and maintain it. Time-consuming, and also time-consuming to transfer over the network

[0005] On the basis of the above description, the inventor found through research that during the process of data interaction between D-I / O and data nodes through TCP / IP SOCKET mode, when the client and the data blocks it needs are located in the same data node, the client The terminal uses D-I / O to interact with the data node through TCP / IP SOCKET, which consumes more time on the basis of the original interaction process.

Images