Hadoop distributed file system (HDFS) based file tracing file transfer protocol (FTP) system

A file system and file technology, applied in transmission systems, electrical components, energy reduction, etc., can solve problems such as inability to trace the source, inability to maintain and view, and achieve the effect of good real-time performance and high work efficiency

Active Publication Date: 2016-02-24
FUJIAN NEWLAND SOFTWARE ENGINEERING CO LTD
View PDF2 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The technical problem to be solved by the present invention is to provide a file traceability FTP system based on HDFS, which solves the problem that in the prior art, after uploading files to the HDFS file system, the later period cannot be maintained and checked, and the problem of traceability cannot be performed

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Hadoop distributed file system (HDFS) based file tracing file transfer protocol (FTP) system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] In order to describe the technical content, structural features, achieved goals and effects of the present invention in detail, the following will be described in detail in conjunction with the embodiments and accompanying drawings.

[0021] HDFS-OVER-FTP is an open source, easy-to-use FTP server 200 for uploading and downloading HDFS file systems. The present invention uses the FTP server 200 to realize functions such as uploading, downloading, and traceability of files. The present invention discloses an HDFS-based file traceability FTP system, which specifically includes: FTP server 200, file upload module 300, and historical records. Module 400, HDFS file system and file flow pool 500; FTP service end 200 initiates N concurrent threads after receiving N uploading file requests from client 100, and each thread calls uploading file module 300 to upload the file that client 100 sends, every One thread corresponds to a file; Upload file module 300 uploads file to HDFS f...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a hadoop distributed file system (HDFS) based file tracing file transfer protocol (FTP) system. The system comprises an FTP server, a file uploading module, a historical recording module, a file stream pool and an HDFS file system. The FTP server receives N file uploading requests from a client side, and then initiates N concurrent threads. Each thread invokes a file uploading module to upload a file sent by the client side. Each thread is corresponding to a file. The file uploading module uploads the file to the HDFS file system. The file uploading module determines that the file is uploaded to the HDFS file system, and then invokes the historical recording module. The historical recording module acquires an idle file stream from the file stream pool, and writes uploading information of the file in a historical recoding file by use of the file stream. The uploading information is used for file tracing. The file stream pool stores mediums of a plurality of file streams. Each file stream manages a historical recording file. The HDFS file system stores the file system of the uploaded file. When the FTP server is used for uploading files to the HDFS file system, the file uploading information can be recorded in the historical recording file, thereby being convenient to check and trace for maintaining in a later period.

Description

technical field [0001] The invention relates to a distributed file system, in particular to an HDFS-based file tracing FTP system. Background technique [0002] In the era of mobile Internet, the amount of user behavior data in the mobile communication industry has surged, and the field of data analysis uses advanced big data technology for data analysis and data access. [0003] The Hadoop distributed file system is designed as a distributed file system suitable for running on general-purpose hardware, and it has many similarities with existing distributed file systems. HDFS is a highly fault-tolerant system suitable for deployment on cheap machines. HDFS can provide high-throughput data access and is very suitable for applications on large-scale data sets. HDFS relaxes some POSIX constraints to achieve the purpose of streaming file system data. [0004] HDFS-OVER-FTP is an open source, easy-to-use FTP server that implements upload and download of the HDFS file system. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/08G06F17/30
CPCG06F16/1805H04L67/06H04L67/1097Y02D30/50
Inventor 张强
Owner FUJIAN NEWLAND SOFTWARE ENGINEERING CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap