Method for increasing read-write speed of spatial big data files

A technology for data files and acceleration space, which is used in electrical digital data processing, special data processing applications, instruments, etc. It can solve the problem of time-consuming uploading of large files, and achieve the effect of fast speed.

Active Publication Date: 2013-11-27
ZHEJIANG UNIV
View PDF3 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In the traditional file upload mode, no matter what the size, content and characteristics of the file are, the file wi...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for increasing read-write speed of spatial big data files
  • Method for increasing read-write speed of spatial big data files
  • Method for increasing read-write speed of spatial big data files

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0062] Zero copy steps:

[0063] like figure 1 As shown here, the data flow in the traditional network file transfer scheme is shown here. The data is first loaded into the read buffer area of ​​the kernel state from the disk memory of the system kernel state through DMA, and then transferred to the application buffer area of ​​the user state through the CPU. Then transfer it to the Socket buffer area of ​​the kernel state through the CPU, and finally transfer it to the NIC buffer area through DMA to transfer the file.

[0064] like figure 2 As shown in , it shows the process of context switching in the traditional network file transfer scheme. Since the data is transmitted between the kernel state and the user state, the whole process has four context switches.

[0065] In contrast, image 3 It shows the data flow direction of the zero-copy step. The data is first read into the read buffer area of ​​the kernel state through the disk, and then directly copied into the Sock...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of computer data transmission and discloses a method for increasing the read-write speed of spatial big data files. The method for increasing the read-write speed of spatial big data files comprises a zero-copy step, a file buffering step and a digest checking step and particularly comprises the steps of when a read-write access is performed on a spatial big data file, obtaining a message digest of the spatial big data file through the digest checking step and comparing the message digest of the spatial big data file with a message digest of a file in a cache system; if the spatial big data file has the same copy in the cache system, updating cache hit information of the copy, updating the weight W of the copy, adjusting the link position of the copy in a sequential linked list according to the weight W of the updated copy, and transmitting the copy through the zero-copy step; if the spatial big data file does not have the same copy in the cache system, creating the copy of the spatial big data file in the cache system, updating the weight W of the copy, and transmitting the copy through the zero-copy step. The method for increasing the read-write speed of spatial big data files has the advantages of overcoming data read-write bottlenecks and being beneficial to increasing data read-write speed on the whole.

Description

technical field [0001] The invention relates to the field of computer data transmission, in particular to a method for accelerating the reading and writing speed of large spatial data files. Background technique [0002] With the continuous development of information technology, the storage system of massive data occupies a pivotal position. In today's era, the amount of data is increasing geometrically every minute and every second, and being able to cope with the rapid reading and writing of massive data has become the most critical topic of discussion. In a high-speed mass data reading and writing environment, how to ensure reliable data storage capabilities, privacy security protection capabilities, and mobile scalability has become a particularly important topic. [0003] In the storage system of spatial big data, its read and write speed directly determines the overall response time of the system. The most direct impact on the reading and writing speed of large files...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 陈华钧陈云路张军权郑国轴杨建华吴朝晖
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap