System and method for downloading data

Abstract

The invention discloses a system and a method for downloading data. The system comprises a plurality of downloading groups, wherein each downloading group comprises an Linux virtual server LVS, at least two downloading nodes Nginx and a distributed file system HDFS; an Nginx operation system is used for mounting storage service of the HDFS through a user space file system FUSE; the LVS is used for receiving a download request from the user, performing scheduling selection for each Nginx and forwarding the download request to the selected Nginx; the Nginx are used for accessing data stored in the HDFS through the FUSE after receiving the download request forwarded by the LVS, and responding to the download request of the user; the HDFS is used for storing data. According to the method, the demand on download service under large storage data quantity, large file quantity and non-concentrated access hot spots of some service users can be met.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a data downloading system and method. Background technique [0002] With the development of information technology, people are increasingly accustomed to obtaining various data through the network. For example, a common way is to download desired content through a data download system. [0003] Currently, there are mainly two download architectures that are widely used, one is the traditional CDN (Content Delivery Network, content distribution network) multi-layer cache architecture, and the other is the download architecture that is commonly used by general video websites. [0004] The schematic diagram of CDN multi-layer cache architecture is as follows: figure 1 As shown, in this architecture, the global data is only stored in the top-level source site device, and each edge service node is reverse-proxyed by Nginx, which forwards the request to the back-end Squid, and Squi...

Images