A data placement method for a distributed file system supporting deep learning
A distributed file and deep learning technology, applied in the field of data placement in distributed file systems, can solve problems such as large network overhead and inability to support the locality of upper-layer computing data, so as to improve operating efficiency and reduce network overhead. , improve the effect of data locality
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment Construction
[0039] like figure 1Shown is a schematic diagram of a distributed file system framework that supports deep learning based on HDFS. The overall structure of the system includes Namenode for managing metadata of the entire system, Datanode for actual data storage, and DFS Client for interacting with users. Among them, the DFS Client can perform file query and storage operations on the Namenode, and the Namenode returns the storage location and other information to the DFS Client. Namenode can control DFS Client to perform file storage and read operations to Datanode. At the same time, Namenode can obtain storage status or issue related instructions from Datanode. There are system namespaces and upper-layer logical storage in the entire Namenode. The system namespace is the mechanism used by HDFS to display data files to users, and the upper-layer logical storage is used to consider the storage method and placement strategy of data files. Namenode is also responsible for obtain...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com