Large data inquiring method based on distribution relation-object mapping processing

A query method and mapping processing technology are applied in the network field to achieve the effect of ensuring query efficiency and data integrity

Active Publication Date: 2014-03-26
COMP NETWORK INFORMATION CENT CHINESE ACADEMY OF SCI
View PDF7 Cites 29 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] For the distributed storage and query problems of big data in relational databases, the purpos

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Large data inquiring method based on distribution relation-object mapping processing
  • Large data inquiring method based on distribution relation-object mapping processing
  • Large data inquiring method based on distribution relation-object mapping processing

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0057] 1. Configuration work

[0058] Before loading Hibernate for the first time, you need to initialize the Session selector according to the configuration information of the configuration file. The format of the configuration file is as follows

[0059]

[0060] The configuration file mainly stores two types of configuration information: table attribute configuration and partition strategy configuration. The table attribute configuration is a Map collection. The elements in this collection are key-value pairs with the table name TableName as the Key and L or S as the Value. This Map can provide the basis for the table attributes for the Session selector and the parser. , the L-type table is a table with a large amount of data, and its data is stored in blocks on each data node according to the division strategy. The S-type table is a table with a small amount of data, and the table data is stored on each data node. full backup.

[0061] The second type of configuration...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a large data inquiring method based on distribution relation-object mapping processing. The method includes the following steps that firstly, m data processing nodes are selected, a Hibernate Session object is constructed for each data processing node, a distributed Hibernate framework is obtained and a Session resource queue is generated; secondly, a configuration file is set, a storage strategy and a routing strategy corresponding to a list of each type are set, an S list with a data volume is backed up at each data processing node, and an L list with a large data volume is stored to the m data processing nodes in a blocking mode; thirdly, an input inquiring request is analyzed and a corresponding processor is selected according to the type of the inquiring request; fourthly, the selected processor selects the corresponding node from the resource queue to process the inquiring request according to the inquiring request, the corresponding routing strategy, and a processing result is protocoled. The large data inquiring method based on distribution relation-object mapping processing can obviously improve the rate of inquiring large data lists.

Description

technical field [0001] The invention relates to a large data storage and query method, in particular to a method for processing large data storage and query using a distributed Hibernate architecture, and belongs to the network technology field. technical background [0002] Since the beginning of the new century, with the development and popularization of the network, the amount of data generated and processed by applications has increased. Take the data processed by Google every day as an example. In 2004, it needed to process 100TB of data per day (Jeffrey Dean and Sanjay Ghemawat.MapReduce:Simplified data processing on large clusters.In Proceedings of the6 th Symposium on Operating System Design and Implementation (OSDI2004), 137-150, 2004), and its daily data volume reached 20PB by 2008 (Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified data processing on large clusters. Communications of the ACM ,51(1):107-113,2008), it can be seen that with the explosive growt...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/245G06F16/284
Inventor 王鹏尧崔建业杨风雷黎建辉
Owner COMP NETWORK INFORMATION CENT CHINESE ACADEMY OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap