Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Storage method of very large data and distributed database system and retrieval method thereof

A data storage and database technology, applied in the database field, to achieve the effects of small footprint, reduced consumption, and improved processing speed

Active Publication Date: 2013-12-25
POWER DISPATCHING CONTROL CENT OF GUANGDONG POWER GRID CO LTD
View PDF4 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] At present, distributed architectures commonly used in the industry, such as hadoop, mongodb, and mysql, are all implemented using share nothing. Although they can meet the application requirements of super-large data from the perspective of data storage and query, their consumption of storage resources is several times the size of the original data, so there is an urgent need for a data storage solution that consumes less hardware resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Storage method of very large data and distributed database system and retrieval method thereof
  • Storage method of very large data and distributed database system and retrieval method thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] The present invention will be described in further detail below in conjunction with the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0023] Such as figure 1 As shown, it is a schematic flow chart of the ultra-large data storage method of the present invention in an embodiment, including the following steps:

[0024] S11. Divide each piece of data to be stored according to a preset segmentation attribute to obtain active data and dead data of each piece of data;

[0025] S12. Compress the dead data after storing it;

[0026] For the data to be stored, in general, in addition to attributes that may be applied to query conditions, statistics, and association classes, there are often a large number of inactive data attributes (not applied to query conditions, statistics, and attributes of association classes) , that is, dead data. If these attribute data adopt the storage mode of active data uniformly, it w...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a storage method of very large data. The method includes the steps of segmenting each stored piece of data according to predetermined attributes to obtain active data and dead data of the piece of data; storing and compressing the dead data; generating a database table of the active data, and storing the database table into different databases in a classified manner according to predetermined distribution strategies. The invention further provides a distributed database system and a retrieval method thereof. The storage method of very large data and the distributed database system and the retrieval method thereof have the advantages that the storage problem of large structured data is solved, consumption of storage resources is low, and the data can be retrieved fast.

Description

technical field [0001] The invention relates to the technical field of databases, in particular to a super-large data storage method, a distributed database system, and a retrieval method of the distributed database system. Background technique [0002] The 21st century is an era of data explosion, especially the gradual objectification and refinement of data definition, and more and more structured data will be generated accordingly. In particular, the current communication operators and the Internet industry are paying more and more attention to user behavior analysis, and the amount of data that needs to be stored, queried and analyzed is also increasing. [0003] For example, the data volume of a provincial-level telecom operator’s online list is more than 1 billion records per day, and the data volume of a single table is more than 300G per day, and there are dozens of similar list data types. Generally, it needs to be kept for three months. It takes half a year, so th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 林斌李星南杨德强余锦业包达志姜绍艳李溢杰李伟坚蒋康明
Owner POWER DISPATCHING CONTROL CENT OF GUANGDONG POWER GRID CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com