Small file combination method for spatio-temporal data in smart city

A spatiotemporal data, small file technology, applied in file system, data mining, file access structure, etc., can solve the problem of not using user access information in depth.

Active Publication Date: 2018-08-28
CHONGQING UNIV OF POSTS & TELECOMM
View PDF5 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0012] However, the existing small file merging strategies all use the characteristics of the file itself (in the above, the reading time of small files, user access characteristics (mainly referring to the number of user visits, stay time, etc.), are directly related to specific files, and belong to the same category. The characteristics of the file itself), can only merge files that have been accessed in history, or simply merge temporally adjacent files, and does not use user access information at a deep level

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Small file combination method for spatio-temporal data in smart city
  • Small file combination method for spatio-temporal data in smart city
  • Small file combination method for spatio-temporal data in smart city

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0065] In order to provide users with convenient and fast spatiotemporal data services, smart cities often provide spatiotemporal data application services to the public in the form of predefined applications through network platforms. After the system receives the user's application access request, it will access the data within a certain space-time range according to the predefined application and time-space parameters selected by the user, and then use the function function to analyze and process the accessed data in the background, and finally Return the result to the user.

[0066] Obviously, if small files within a certain space and time span are frequently accessed by users, it means that the access of small files within this space-time range is related. Inspired by this, this embodiment considers using a hierarchical clustering algorithm to dig out the space-time range with access correlation from the historical small file access information, and use this range to guid...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a small file combination method for spatio-temporal data in a smart city. A small file is combined to serve as a microscopic data layout mechanism, system I / O performance of the system can be effectively improved, and a user access time delay is reduced. The characteristics of the spatio-temporal data are analyzed, historical user access information is subjected to parametric representation and spatio-temporal attribute extraction, the access information is subjected to hierarchy clustering by an AGNES (AGglomerative NESting) algorithm in a spatio-temporal attribute domain, a clustering result is subjected to weighted calculation based on access density, and a relevant spatio-temporal range is found. Finally, the spatio-temporal range is used for guiding the combination of small files. An experiment result indicates that the algorithm is simple and efficient, and the access efficiency of the spatio-temporal data small file in the system is greatly improved.

Description

technical field [0001] The invention relates to the research field of merging strategies for spatio-temporal data small files in smart cities. Background technique [0002] In a smart city based on the Internet of Things and cloud computing, ubiquitous sensors always generate sensing data containing three inherent attributes of time, space, and type. These data are small in size (usually tens to hundreds of KB) , a wide variety, a large number, high redundancy and dynamic growth over time, which are typical small spatio-temporal data files. [0003] The current mainstream distributed file system focuses on large files in terms of metadata management, data layout, stripe design, cache management and other implementation strategies. At present, the more common distributed file systems include Google GFS, Hadoop HDFS, PVFS, and Luster. These file systems all adopt a centralized data management mechanism with a master-slave structure, and store metadata (data describing data, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06K9/62
CPCG06F16/13G06F16/16G06F16/1734G06F16/182G06F2216/03G06F18/231
Inventor 熊炼熊珊国代新
Owner CHONGQING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap