Small computer system interface (SCSI) fault-tolerant optimization method and device based on hadoop distributed file system (HDFS)

An optimization method and a technology for optimizing devices, which are applied in the Internet field, can solve problems such as not considering other copies of HDFS data blocks, and achieve the effect of improving IO efficiency and reducing failure rates
CN103220162AActive Publication Date: 2013-07-24BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
5 Cites 6 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
Publication Date
2013-07-24

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention provides a small computer system interface (SCSI) fault-tolerant optimization method and device based on a hadoop distributed file system (HDFS). The method comprises the following steps: a request which is sent by a client end and is about a first data block duplicate is received; the request is sent to an SCSI bottom layer through an SCSI middle layer; the SCSI bottom layer processes the request and returns a processed result to the SCSI middle layer; the SCSI middle layer executes calling-back according to the processed result returned by the SCSI bottom layer so as to judge whether the request is successful or not, and the SCSI bottom layer needs to be retried or is failed; and retrying times are set or retrying is given up when the fact that the SCSI bottom layer needs to be retried is judged. According to the SCSI fault-tolerant optimization method and device based on the HDFS, when the request of the client end is judged to be the fact that the SCSI bottom layer needs to be retried according to the result returned by the SCSI bottom layer, the retrying times are set or giving-up retrying is set, and the error processing method which does not influence the actual request of the client end is optimized. Thus, IO efficiency is improved, and hardware failure rate is lowered.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of the Internet, in particular to an HDFS-based SCSI fault-tolerant optimization method and device. Background technique

[0002] HDFS (Hadoop Distributed File System, a distributed file system) usually stores a copy of each data block, of which the default is 3, and the client client of HADOOP (a software framework for distributed processing of large amounts of data) is in When reading a data block, if HDFS reads another copy of the data block from other places when an error is detected, the whole process is transparent to the user, but for HDFS-based SCSI (Small Computer Systems Interface, small computer system Interface), if an error is detected, an error will be reported because the bottom layer of SCSI fails to read data.

[0003] In view of the above problems, in the prior art, it is judged whether there is an error through the IO request processing result returned by the bottom layer of the SCSI. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More