Data segmentation method

Abstract

The invention provides a data segmentation method. The method comprises the steps that an index data string Ck is obtained from a starting point position of input data S in a sequential increment mode; the index data string Ck is used for scanning the remaining bits of the input data S, and the starting point position Lm of the data segment identical to the Ck, the data length, the number of repetition times and the Ck are recorded; and finally, a set of the positions of the optimal repeated data is acquired, and the data in the set is determined as a data segmentation point.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a data segmentation method. Background technique [0002] At all levels of data transfer, reducing the transmission of duplicate data is of great significance. The key to reducing repeated data transmission is how to know which data is repeated, and how to quickly and flawlessly restore the original data at the data sending end at the data receiving end. For static data, that is, a data set whose content is determined, if it is possible to know the data or data set that needs to be transmitted repeatedly, and assign a label or identifier to the repeated data, the label can be used to replace the repeated data for transmission, thereby reducing data traffic. Retransmit and restore the original data at the data receiving end. Moreover, the smaller the ratio of the tag length to the repeated data length, the higher the data transmission efficiency. For dynamic data, that ...

AI Technical Summary

Problems solved by technology

It is common knowledge that the scanning results or data preprocessing results obtained at the end are very different when the starting point of the scanning operation is different, that is to say, the existing data preprocessing methods are difficult to obtain the best data preprocessing results

Images