Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Distributed storage method and transmission decoding method

A distributed storage and distributed transmission technology, applied in the field of distributed storage, can solve the problems of high code storage redundancy, codec complexity reduction, transmission bandwidth redundancy, etc., to reduce storage redundancy, improve storage efficiency, The effect of reducing storage redundancy

Active Publication Date: 2020-07-24
SHENZHEN RES INST OF BIG DATA
View PDF8 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In a distributed system, there are n nodes in the network, the data is divided into k blocks, and the encoded n data packets are stored on n nodes respectively. If the data can be recovered by any k nodes, the encoding is ( n, k) MDS code, compared with the traditional backup mechanism, the encoding and decoding efficiency of MDS code is greatly improved, but the MDS encoding and decoding complexity is higher
In 2013, Sung et al. proposed an MDS encoding and decoding scheme based on XOR operation (that is, XOR operation). This scheme is suitable for networks of any scale. It adopts ZigZag decoding, and the complexity of encoding and decoding is greatly reduced. However, the More transmission bandwidth redundancy
In 2014, Fu et al. proposed a non-redundant transmission and decoding scheme based on XOR operation, which reduced bandwidth transmission redundancy to zero and eliminated decoding space redundancy, but its encoding and storage redundancy is still high
High redundancy will increase transmission costs, and as the number of network nodes increases, its storage redundancy will increase accordingly

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed storage method and transmission decoding method
  • Distributed storage method and transmission decoding method
  • Distributed storage method and transmission decoding method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0055]Embodiment 1 of the present invention provides a distributed storage method. In the existing coding schemes of Sung and Fu, the information stored by each node has redundancy. For example, among n nodes, the storage redundancy of node i is i(k-1), and the maximum redundancy can reach n (k-1). As the number of nodes in the network increases, storage redundancy increases. However, the distributed storage method of this embodiment can reduce the storage redundancy of each node, and has high coding efficiency at the same time.

[0056] figure 1 A schematic flow diagram of a distributed storage method provided by an embodiment of the present invention, such as figure 1 As shown, the method includes the following steps:

[0057] S11: Divide the original data into blocks to obtain a first preset number k of data blocks, and the data lengths of the data blocks are the same.

[0058] S12: Encode the data block according to the generator matrix, and generate a second preset n...

Embodiment 2

[0086] This embodiment provides a distributed transmission decoding method, which is used to decode the data stored in the distributed system nodes in Embodiment 1, such as figure 2 As shown, it is a schematic flow chart of the distributed transmission decoding method of this embodiment, including:

[0087] S21: Connect to and acquire a first preset number k of node data, wherein the node data is stored in the nodes using a distributed storage method as described in any one of the embodiments.

[0088] That is, in the transmission phase, the user can randomly select k nodes to obtain the stored data of their nodes, and the nodes are denoted as i 1 ,...,i k , satisfy the condition: n≥i 1 >...>i k ≥ 1, the node packet in the node transmitted to the user.

[0089] S22: Perform local decoding on the node data to obtain original data.

[0090] In this embodiment, local decoding is adopted, which can eliminate extra decoding space consumption.

[0091] The specific decoding...

Embodiment 3

[0105] This embodiment proposes a specific application scenario for Embodiment 1 and Embodiment 2, provides a method for sharing secret data, and performs secret data sharing among a second preset number of devices, including:

[0106] The secret data is distributedly encoded according to the distributed storage method according to any one of the embodiments to obtain a second preset number of data packets, and each device includes one data packet.

[0107] Randomly select the data packets of the first preset number of devices to perform local decoding according to a distributed transmission decoding method according to any one of the second embodiment to obtain secret data, wherein the secret data also includes, for example, permissions, secret keys, etc.

[0108]That is, in a secret sharing scheme involving n users, the secret data is divided into k data blocks and encoded with the method described in Embodiment 1 to obtain n data packets, and each user holds one data packet,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a distributed storage method and a transmission decoding method, and relates to the field of distributed storage. The distributed storage method comprises the following steps:partitioning original data to obtain a first preset number of data blocks, then encoding the data blocks according to a generation matrix to generate a second preset number of data packets, performingdata interception on the data packets by utilizing an interception scheme to obtain intercepted data packets, and storing the intercepted data packets into the corresponding distributed nodes. According to the embodiment of the invention, higher coding efficiency can be ensured by coding according to the generation matrix, and the storage redundancy of the nodes can be reduced when the data packets are intercepted and stored through the interception scheme, so that the storage redundancy of the whole distributed system is reduced, the storage space is saved, and the storage efficiency is improved.

Description

technical field [0001] The invention relates to the field of distributed storage, in particular to a distributed storage method and a transmission decoding method. Background technique [0002] In distributed storage system, MDS (Maximum Distance Separable) code is an effective storage coding scheme, which mainly solves the problem of data recovery. In a distributed system, there are n nodes in the network, the data is divided into k blocks, and the encoded n data packets are stored on n nodes respectively. If the data can be recovered by any k nodes, the encoding is ( n, k) MDS code, compared with the traditional backup mechanism, the encoding and decoding efficiency of the MDS code is greatly improved, but the MDS encoding and decoding complexity is relatively high. In 2013, Sung et al. proposed an MDS encoding and decoding scheme based on XOR operation (that is, XOR operation). This scheme is suitable for networks of any scale. It adopts ZigZag decoding, and the complexi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): H04L1/00H04L29/08G06F3/06
CPCH04L67/1097H04L1/0076H04L1/0057G06F3/0608G06F3/064G06F3/067
Inventor 付希明郭沅鑫杨升浩
Owner SHENZHEN RES INST OF BIG DATA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com