Unlock instant, AI-driven research and patent intelligence for your innovation.

Storage method, device and system for dynamic data fragmentation

A technology of data dynamics and storage devices, which is applied in the storage field of data dynamic sharding, which can solve the problems of low I/O performance, low reading efficiency, and data sharding placement in the storage system, so as to improve placement and reading efficiency, The effect of improving I/O performance

Active Publication Date: 2022-02-01
CHINA TELECOM CORP LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Existing data sharding technology solutions use fixed parameters to set, or set the number of shards by setting the file size threshold, which will lead to problems such as low I / O performance of the storage system, low data shard placement and read efficiency, etc.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Storage method, device and system for dynamic data fragmentation
  • Storage method, device and system for dynamic data fragmentation
  • Storage method, device and system for dynamic data fragmentation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. The following description of at least one exemplary embodiment is merely illustrative in nature and in no way taken as limiting the invention, its application or uses. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0044] The relative arrangements of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0045] At the same time, it should be understood that, for the convenience of d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a storage method, device and system for data dynamic fragmentation. The method includes: responding to an access request initiated by a client, performing business analysis, and obtaining node parameters and storage parameters of configuration management templates; collecting network bandwidth of each storage node in real time; according to the node parameters, storage parameters and network bandwidth of each storage node Determine the number of current file fragments; fragment or merge data according to the current number of file fragments, so that the routing controller can access the data slices of each corresponding storage node. The present invention can dynamically adjust the number of file fragments by introducing the real-time available bandwidth of the storage node as a parameter, thereby improving the I / O performance of the storage node and improving the efficiency of data fragment placement and reading.

Description

technical field [0001] The invention relates to the field of data storage, in particular to a storage method, device and system for dynamic data fragmentation. Background technique [0002] With the rapid development of information technology, the amount of data in the information system database is increasing. In order to meet the storage requirements of large amounts of data, distributed storage systems running on multiple servers have been widely used. [0003] In a distributed storage system, multiple database systems run on multiple servers. When data is stored, it is necessary to shard the data first, and then assign different data shards to different servers for storage. Fragmentation is a horizontal scaling method that distributes a large data set to multiple data nodes, and all data nodes will form a logical database to store this large data set. Fragmentation is transparent to the user (application layer), and the user will not know which slice server the data i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06G06F16/22H04L43/0894H04L43/16
CPCH04L43/0894H04L43/16G06F3/061G06F3/067
Inventor 王海霞李先绪陈泳黄植勤邱红飞吴家隐郑文武
Owner CHINA TELECOM CORP LTD