Unlock instant, AI-driven research and patent intelligence for your innovation.

Data editing method based on HDFS big data storage

A big data storage and data editing technology, applied in the field of big data storage, can solve the problem of not supporting data update operation, and achieve the effect of solving the problem of not supporting data update operation and improving data access efficiency

Pending Publication Date: 2022-02-01
TIANJIN RICHSOFT ELECTRIC POWER INFORMATION TECH +1
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Many non-relational database (NoSQL) underlying storage uses HDFS, which supports large data storage, fast access and data analysis, but most of them do not support data update operations

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data editing method based on HDFS big data storage
  • Data editing method based on HDFS big data storage

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] In order to further understand the content, characteristics and effects of the present invention, the following examples are given, and detailed descriptions are given below with reference to the accompanying drawings. It should be noted that this embodiment is descriptive, not restrictive, and cannot thereby limit the protection scope of the present invention.

[0031] A data editing method based on HDFS big data storage, such as figure 1 As shown, data insertion or update is stored in hot storage first, and then synchronized to distributed storage under certain conditions. Both hot storage and distributed storage are divided into two storage spaces to store updated data and new data respectively. Data in storage is automatically merged based on policies. Such as figure 2 As shown in the data query request, the hot storage will be queried first, and the hit will be returned, otherwise the distributed storage will be queried. Including the following steps:

[0032]...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data editing method based on HDFS big data storage. The method comprises the following steps that 1, a storage strategy, a synchronization strategy and a data merging strategy are configured; 2, a data receiving server stores the data in a period of time in a hot data storage memory and a disk according to the storage strategy; 3, a data synchronization server synchronizes the data in the hot storage to different storage spaces in the distributed storage according to the synchronization strategy and whether the data is an update operation; 4, a data merging server performs data merging on the data of the two storage spaces in the distributed storage according to the data merging strategy; and 5, a data query server must query the hot storage firstly when querying the data, returns the data if the data is hit, otherwise, separately queries the two storage spaces of the distributed storage, merges and returns a query result. By means of the method, the data updatable function in HDFS storage is achieved, and meanwhile the access efficiency is improved through the data in the hot storage.

Description

technical field [0001] The invention relates to the technical field of big data storage, in particular to a data editing method based on HDFS big data storage. Background technique [0002] Hadoop Distributed File System, HDFS for short, is a distributed file system. HDFS has the characteristics of high fault tolerance (fault-tolerant) and is designed to be deployed on low-cost (low-cost) hardware. And it provides high throughput (highthroughput) to access application data, suitable for those applications with very large data sets (largedataset). Many non-relational databases (NoSQL) use HDFS as the underlying storage. They support large data storage, fast access and data analysis, but most of them do not support data update operations. Contents of the invention [0003] The purpose of the present invention is to overcome the deficiencies of the prior art, and provide a data editing method based on HDFS big data storage, which realizes the data update function in HDFS st...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/14G06F16/172G06F16/178G06F16/182
CPCG06F16/148G06F16/172G06F16/178G06F16/182
Inventor 胡浩瀚郭正雄王汝英张立杨少春张海涛朱传晶刘德强张志陶
Owner TIANJIN RICHSOFT ELECTRIC POWER INFORMATION TECH