Data processing method and device in trunking system

Abstract

In the invention, when a received datastream is subjected to repeated data query, a first physical node in the trunking system corresponding to every first characteristic value is determined based on the first characteristic value representing the datastream, and then the first characteristic value representing the datastream is transmitted to the determined physical node to be repeated data query, the query process of repeated data is not changed because of the increasing of the nodes in the trunking system, and the calculated amount is not increased because of the increasing of the nodes in the trunking system.

Description

technical field [0001] Embodiments of the present invention relate to storage technology, and in particular to a data processing method and device in a cluster system. Background technique [0002] Data deduplication (abbreviated as deduplication), also known as intelligent compression or single instance storage, is a method that can automatically search for duplicate data, keep only one copy of the same data, and replace other duplicate copies with pointers to the single copy , to achieve a storage technology that eliminates redundant data and reduces storage capacity requirements. [0003] Cluster deduplication (referred to as cluster deduplication) refers to a technology that organizes multiple deduplication physical nodes to improve deduplication performance and capacity. In the cluster deduplication technology in the prior art, the physical node receiving the data stream usually divides the data stream into blocks to obtain several data blocks, and groups the obtained ...

AI Technical Summary

Problems solved by technology

[0004] The inventor found through research that in the prior art cluster deduplication technology, the sampled metadata information needs to be sent to all physical nodes for query, resulting in too many interactions between physical nodes during the deduplication process. When the cluster system When there are many physical nodes, when each physical node performs deduplication, the amount of calculation will increase with the increase of physical nodes in the cluster system, resulting in a decrease in deduplication performance of the system

Images