Small computer system interface (SCSI) fault-tolerant optimization method and device based on hadoop distributed file system (HDFS)

An optimization method and a technology for optimizing devices, which are applied in the Internet field, can solve problems such as not considering other copies of HDFS data blocks, and achieve the effect of improving IO efficiency and reducing failure rates

Active Publication Date: 2013-07-24
BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
View PDF5 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The problem existing in the existing technology is that starting from the bottom layer of SCSI and the IO request processing results, other copies of the data blocks stored in HDFS are not considered. When the client’s IO request has actually been satisfied, HDFS-based SCSI still proceeds. Unnecessary error reporting or retry mechanism

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Small computer system interface (SCSI) fault-tolerant optimization method and device based on hadoop distributed file system (HDFS)
  • Small computer system interface (SCSI) fault-tolerant optimization method and device based on hadoop distributed file system (HDFS)
  • Small computer system interface (SCSI) fault-tolerant optimization method and device based on hadoop distributed file system (HDFS)

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention. On the contrary, the embodiments of the present invention include all changes, modifications and equivalents coming within the spirit and scope of the appended claims.

[0025] In the description of the present invention, it should be understood that the terms "first", "second" and so on are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance. In the description of the present invention, it should be noted that unless otherwise specified and limited, the terms "connected" and "connect...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a small computer system interface (SCSI) fault-tolerant optimization method and device based on a hadoop distributed file system (HDFS). The method comprises the following steps: a request which is sent by a client end and is about a first data block duplicate is received; the request is sent to an SCSI bottom layer through an SCSI middle layer; the SCSI bottom layer processes the request and returns a processed result to the SCSI middle layer; the SCSI middle layer executes calling-back according to the processed result returned by the SCSI bottom layer so as to judge whether the request is successful or not, and the SCSI bottom layer needs to be retried or is failed; and retrying times are set or retrying is given up when the fact that the SCSI bottom layer needs to be retried is judged. According to the SCSI fault-tolerant optimization method and device based on the HDFS, when the request of the client end is judged to be the fact that the SCSI bottom layer needs to be retried according to the result returned by the SCSI bottom layer, the retrying times are set or giving-up retrying is set, and the error processing method which does not influence the actual request of the client end is optimized. Thus, IO efficiency is improved, and hardware failure rate is lowered.

Description

technical field [0001] The invention relates to the technical field of the Internet, in particular to an HDFS-based SCSI fault-tolerant optimization method and device. Background technique [0002] HDFS (Hadoop Distributed File System, a distributed file system) usually stores a copy of each data block, of which the default is 3, and the client client of HADOOP (a software framework for distributed processing of large amounts of data) is in When reading a data block, if HDFS reads another copy of the data block from other places when an error is detected, the whole process is transparent to the user, but for HDFS-based SCSI (Small Computer Systems Interface, small computer system Interface), if an error is detected, an error will be reported because the bottom layer of SCSI fails to read data. [0003] In view of the above problems, in the prior art, it is judged whether there is an error through the IO request processing result returned by the bottom layer of the SCSI. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/24H04L29/08
Inventor 胡殿明魏伟王志谦
Owner BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap