Unlock instant, AI-driven research and patent intelligence for your innovation.

Vector processing for partitioned hash calculations

A technique of hash value and vector, applied in the field of segmentation of input data stream

Active Publication Date: 2020-03-10
HUAWEI TECH CO LTD
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] One of the main challenges is to efficiently segment data streams such that changes to a segment's data content affect the segment as little as possible

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Vector processing for partitioned hash calculations
  • Vector processing for partitioned hash calculations
  • Vector processing for partitioned hash calculations

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] Some embodiments of the invention relate to segmentation of input data streams, more particularly but not exclusively, to segmentation of input data streams using vectors.

[0032] The present invention provides a system and method for segmenting an input data stream using vectors as part of a deduplication process applied to the input data stream to reduce the data volume of the input data stream by removing redundant (duplicate) data segments. Performing a deduplication process for reducing the data volume of an input data stream, eg, data files, media files, streaming data, etc., in order to reduce storage space and / or network bandwidth required for storing and / or transmitting the input data stream. Segmentation is achieved by concurrent or simultaneous processing of multiple partial rolling sequences that make up a rolling sequence that progressively shifts in a sequence of consecutive data items, such as bytes, words, doublewords, and / or pixels. Simultaneously pro...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A system for segmenting an input data stream using vectors, comprising: a processor for repeating the following steps throughout the input data stream to create a segmented data stream comprising a plurality of segments: a sequence of consecutive data items in the input data stream applying a rolling sequence on the above, wherein the rolling sequence includes a subset of consecutive data items of the sequence; the processing pipeline of the processor calculates partial hash values ​​simultaneously for a plurality of partial rolling sequences, wherein each part The rolling sequence includes evenly spaced data items of the subset; determining compliance of each of the plurality of partial hashes with one or more corresponding partial partitioning criteria; The sequence is designated as a variable length segment when at least some meet the corresponding partial segmentation criteria.

Description

technical field [0001] Some embodiments of the invention relate to segmentation of input data streams, more particularly but not exclusively, to segmentation of input data streams using vectors. Background technique [0002] Today is the information age, and the amount of data that needs to be stored and / or transmitted between locations is increasing rapidly. The sheer volume of data may primarily present challenges of cost and / or complexity in terms of storage space for storing the data and / or network bandwidth for transmitting the data. [0003] One solution commonly used to reduce the amount of data stored and / or transmitted is data deduplication (commonly known as "smart compression" or "single instance storage"), which is a method of reducing data volume by eliminating redundant data . While methods exist for file deduplication, block deduplication may present better results considering data compression. In block deduplication, only a single instance of a data segmen...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H03M7/30G06F16/901
CPCH03M7/3095G06F16/3347G06F16/2365G06F16/24568H03M7/6023G06F9/30036G06F9/3867
Inventor 大卫·叶赫那丹亚伊尔·托弗迈克尔·赫希
Owner HUAWEI TECH CO LTD