An object storage metadata management device and method based on Kudu

A technology of object data and object storage, applied in the database field, can solve problems such as inability to realize massive structured storage, data delay, etc., and achieve the effects of linear expansion of storage capacity, simplified hybrid architecture, and efficient support for large files

Inactive Publication Date: 2019-04-26
INSPUR SOFTWARE CO LTD
View PDF6 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, this technical solution cannot achieve the goal of mass structured storage, and at the same time solve the problem of data delay

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • An object storage metadata management device and method based on Kudu
  • An object storage metadata management device and method based on Kudu
  • An object storage metadata management device and method based on Kudu

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0073] as attached figure 1 As shown, the Kudu-based object storage metadata management device of the present invention has a structure including a metadata management module and an object data storage module; the metadata management module is used to store and manage metadata information of object data; the metadata management module passes The OSMetadata (metadata) daemon module works; the object data storage module is used to store and manage object data; the object data storage module uses CMSP (cloud message service platform) to send and cache messages, and finally store them, and finally store the data to file system or raw device.

[0074] Metadata information is stored in the metadata table, which is a built-in kudu table; metadata information includes object data name, object data storage id, object data length, object data version and timestamp; metadata information is stored in metadata table , the table is a built-in kudu table named kudu_os_metadata, and the stru...

Embodiment 2

[0085] The Kudu-based object storage metadata management method of the present invention, the method is to use the metadata management module to complete the insertion, acquisition, update and deletion of object data, and simultaneously store and manage the metadata information of the object data; reuse the object data storage module The storage of object data is completed, and non-object storage is expanded on the basis of kudu structured storage. The object data storage module uses CMSP to send and cache messages, and finally stores them. Finally, the data is stored in the file system or raw devices. The metadata management module works through the OSMetadata daemon process. The OSMetadata daemon process mainly includes the following functions: object data insertion operations, object data acquisition operations, and object data update and deletion operations.

[0086] as attached figure 2 As shown, the object data insertion operation is to obtain the corresponding metadata...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an object storage metadata management device and method based on Kudu and belongs to the database field, The technical problem to be solved by the invention is how to achieve the goal of mass structured storage. According to the technical scheme, (1) the Kudu-based object storage metadata management device comprises a metadata management module and an object data storage module which are respectively used for storing and managing metadata information of object data and storing management object data; (2) a Kudu-based object storage metadata management method, which comprises the following steps of: inserting, acquiring, updating and deleting object data by utilizing a metadata management module, and storing and managing metadata information of the object data at thesame time; The object data storage module is used for completing storage of the object data, non-object storage is expanded on the basis of kudu structured storage, the object data storage module isused for sending and caching messages through a CMSP, and finally the data is stored in a file system or bare equipment.

Description

technical field [0001] The invention relates to the field of databases, in particular to a Kudu-based object storage metadata management device and method. Background technique [0002] The Hadoop ecosystem has many components, each with different functions. In real scenarios, users often need to deploy many Hadoop tools at the same time to solve a problem. For example, users need to use Hbase's fast insert and fast read random access features to import data, and users use HDFS / Parquet+Impala / Hive to query and analyze very large data sets. Many companies have successfully deployed the HDFS / Parquet+HBase hybrid architecture. However, this architecture is more complicated, and it is also very difficult to maintain, and it will also cause data delays. Therefore, how to achieve the goal of massive structured storage while solving data delay is a technical problem that needs to be solved urgently. Mass structured storage uses a simple architecture to store structured data, ach...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/23
Inventor 邓光超李朝铭
Owner INSPUR SOFTWARE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap