Merging and storing method and system for massive data

A data fusion and mass data technology, applied in the field of communication, can solve the problems of data redundancy, difficulty in realization, and inability to meet flexible and diverse business needs, so as to achieve the effect of reducing redundancy and reducing operation and maintenance costs

Inactive Publication Date: 2015-06-10
ZTE CORP
View PDF8 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

For the same data, depending on the needs of the business, it may not only need to perform massive data query, statistics, grouping, and sorting operations through SQL, but also need to efficiently query and intelligently scan through key-value methods. It is difficult to implement in the system
It can be seen that the current massive data processing system mainly has the following problems: (1) The structured storage engine and the NoSQL storage engine are independent of each other in terms of physical data storage. Diverse business needs; (2) If a structured stora

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Merging and storing method and system for massive data
  • Merging and storing method and system for massive data
  • Merging and storing method and system for massive data

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0026] Hereinafter, the present invention will be described in detail with reference to the drawings and in conjunction with the embodiments. It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other if there is no conflict.

[0027] The embodiment of the present invention provides a method for fusion storage of massive data. figure 1 It is a flowchart of a method for fusion storage of massive data according to an embodiment of the present invention, such as figure 1 As shown, the method mainly includes the following steps (step S102-step S104):

[0028] Step S102: The data fusion subsystem receives a write operation request sent by a structured storage engine or a NoSQL storage engine, where the structured storage engine supports SQL structured data read and write operations, and the NoSQL storage engine supports key-value NoSQL data read and write operations ;

[0029] Step S104: The data fusion subsystem perform...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Disclosed is a mass data fusion storage method and system. The method comprises: a data fusion subsystem receives a write operation request sent by a structured storage engine or by an NoSQL storage engine, wherein the structured storage engine supports SQL structured data read and write operations, and the NoSQL storage engine supports key value NoSQL data read and write operations; the data fusion subsystem, in accordance with the write operation request, engages in a write operation in regard to a distributed file subsystem, wherein the distributed file subsystem is additionally used to receive a read operation request sent by a structured storage engine or by an NoSQL storage engine, and, in accordance with the read operation request, to execute the read operation. Achieved is a reduction in data redundancy and decreases system operation and maintenance costs.

Description

technical field [0001] The present invention relates to the communication field, in particular to a massive data fusion storage method and system. Background technique [0002] With the rapid development of the Internet, massive amounts of data are generated every day. The US Internet Data Center pointed out that the data on the Internet will increase by 50% every year, doubling every two years, and more than 90% of the data in the world is only generated in recent years. How to efficiently process these massive amounts of data has become a hot topic in recent years. [0003] In the existing mass data processing systems, there are structured storage engines that support SQL (Structured Query Language, Structured Query Language) for processing relational data, and NoSQL (Not Only SQL, unstructured query language) for processing non-relational data. query language) storage engine. Among them, the structured storage engine mainly completes the functions of structured data de...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/2471G06F16/182G06F16/215G06F16/2365G06F16/256G06F16/27
Inventor 汪峰来朱科支杨庆平陈福忠
Owner ZTE CORP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products