Unlock instant, AI-driven research and patent intelligence for your innovation.

Object metadata aggregation method and device, object metadata reading method and device, equipment and storage medium

An aggregation method and metadata technology, which are applied in the directions of metadata multimedia retrieval, multimedia data retrieval, and multimedia data indexing to achieve the effect of reducing size, increasing speed, and increasing synchronization speed.

Active Publication Date: 2021-07-13
SHANDONG YINGXIN COMP TECH CO LTD
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In response to the above problems, the current solution in the industry is to increase the read and write speed of metadata by increasing the SSD capacity, but this solution cannot fundamentally solve the performance bottleneck problem caused by metadata

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Object metadata aggregation method and device, object metadata reading method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0025] In general: For buckets, each bucket has multiple index shards, and object metadata is stored on each index shard through hash calculation. As more and more objects in the bucket reach hundreds of millions, more and more object metadata information will be recorded on each bucket index shard. The metadata is getting bigger and bigger, which will slow down the reconstruction speed during failure recovery (after a cluster failure, the stored data and metadata will be reconstructed during recovery, the more data and metadata, the greater the amount of data, the reconstruction The slower the speed), it will affect the synchronization speed between sites (synchronization between sites, data and metadata need to be synchronized, the more data and metadata, the greater the amount of data, the slower the synchronization speed). Therefore, this design sc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an object metadata aggregation method and device, a reading method and device, equipment and a storage medium, and the object metadata aggregation method comprises the steps that the object number N of metadata aggregation and a metadata aggregation switch are configured; it is judged whether to start a storage bucket metadata aggregation scanning thread or not according to the metadata aggregation switch; if the metadata aggregation scanning thread is started, metadata aggregation is carried out on each bucket index fragment according to the set object number N, and then the bucket index fragments are stored on the bucket index fragments, and otherwise, the process is ended. In the embodiment of the invention, the index fragments in each storage bucket are scanned in the metadata aggregation thread, the multiple pieces of object metadata information are aggregated into one piece of data, and the data are stored in the index fragments of the buckets, so that the reconstruction speed of fault recovery and the synchronization speed between sites can be improved.

Description

technical field [0001] The present invention relates to the technical field of object metadata storage, in particular to an object metadata aggregation method, reading method, device, equipment and storage medium. Background technique [0002] In the era of big data, Object Storage Service (OSS) is an emerging massive, safe, low-cost, and highly reliable cloud storage service, suitable for storing unstructured data, such as: video, pictures, documents, etc. . One of the characteristics of the above-mentioned unstructured data is the large amount of data, and the generated metadata is also getting larger and larger. Metadata is data about data and is important information about an object. After the object is stored on the disk, more and more object metadata is operated. In many scenarios, such as reconstruction and synchronization, it gradually becomes the bottleneck of performance. In response to the above problems, the current solution in the industry is to increase the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/41G06F16/48
CPCG06F16/41G06F16/48
Inventor 陶桐桐胡永刚
Owner SHANDONG YINGXIN COMP TECH CO LTD