Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Adaptive reading optimization method and system for mass data under cloud storage environment

A mass data, self-adaptive technology, applied in special data processing applications, electrical digital data processing, instruments, etc., can solve the problem of long data reading time, and achieve the effect of increasing the reading speed and reducing the time

Inactive Publication Date: 2017-04-19
COMP NETWORK INFORMATION CENT CHINESE ACADEMY OF SCI
View PDF6 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

When a user reads a large amount of data, due to the delay of network transmission, the data reading time will be very long

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Adaptive reading optimization method and system for mass data under cloud storage environment
  • Adaptive reading optimization method and system for mass data under cloud storage environment
  • Adaptive reading optimization method and system for mass data under cloud storage environment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] In the following, the present invention will be further described through a specific implementation case, a massive data adaptive reading system in a cloud storage environment, and accompanying drawings.

[0024] The massive data adaptive reading system in the cloud storage environment in this embodiment includes three parts: user program, virtual file system and distributed file system, wherein the virtual file system consists of file access middleware, cache, log database and statistical analysis service consists of four modules, such as figure 1 shown.

[0025] The file access middleware adopts the user space file system (FUSE technology) to realize the file interface conforming to POSIX, and is responsible for receiving the data object reading instruction of the user program, reading the data object from the distributed file The file system pre-reads the data objects into the cache; the log database stores the data access log information; the statistical analysis s...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an adaptive reading optimization method for mass data under cloud storage environment. The method comprises the following steps of: recording the data access log information of a local user program; carrying out statistical analysis on the above log information in fixed time to obtain an incidence relationship among data objects; according to the data access request of the user grogram, obtaining a data object set related to the data object which is accessed at present, and pre-reading the data object set to a local cache; and for the data object access request of the user program, firstly reading the data object access request from the local cache, and if a data object to be accessed is in the absence in the local cache, reading the data object access request from the far-end storage node of a distributed file system. The method also comprises the steps of pre-reading the related data object set to the local cache from the far-end storage node of the distributed file system and updating the local cache. By use of the method, according to a data object access log subjected to statistics and analysis, the incidence relationship of the data object is established, and the incidence data object of the current read data object is pre-read to the local cache to improve data reading speed.

Description

technical field [0001] The invention relates to a distributed file system data reading optimization method in the field of computer technology, in particular to a massive data adaptive reading optimization method and system in a cloud storage environment. Background technique [0002] With the advent of the information society, all kinds of data are growing rapidly in an explosive manner. With the massive data generated, the requirements for data storage, reading, and management are getting higher and higher. At this stage, distributed data storage has become mainstream. A distributed file system usually stores data on multiple nodes (different servers), each node is connected through a network, and users can access the data between each node through the interface of the distributed file system. When a user reads a large amount of data, due to the delay of network transmission, the data reading time will be very long. Therefore, how to quickly read massive data has become...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/172G06F16/182
Inventor 王学志赵江华林青慧周园春黎建辉
Owner COMP NETWORK INFORMATION CENT CHINESE ACADEMY OF SCI
Features
  • Generate Ideas
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com