A method and system for deduplication in network based on SDN

Abstract

The invention discloses an intranet duplicated data deletion method and system based on an SDN (Software Defined Network) and belongs to the technical field of computer network and storage. According to the method and the system, advantages such as isolation of control and forwarding, expandability, programmability, of the SDN, and the advantages in aspects such as reduction of storage costs and reduction of network overheads, of a duplicated data deletion technology are fully utilized, and redundancy elimination service in a network is achieved. Storage overheads of a destination side can be reduced, moreover, network link loads can be greatly reduced, the response time of a user is shortened, the greater flexibility is provided for a traffic engineering goal, and a compromise between source deduplication and destination deduplication is realized. An SDN controller communicates with an openflow switch in a bottom layer through an openflow protocol, thereby controlling data forwarding. File fingerprint information transmitted in the network is stored, thereby controlling deduplication of data. By taking full advantage of a locality principle, communication overheads between the SDN controller and the switch are reduced, and preliminary redundancy elimination is carried out by utilization of file information cached on the switch.

Description

technical field [0001] The invention belongs to the field of computer network and storage technologies, and more specifically relates to a method and system for deduplicating data in a network based on SDN. Background technique [0002] With the explosive growth of data in the information age, a large amount of duplicate data will appear in information processing, communication and storage, and may be transmitted through Internet links. Especially in the process of using cloud backup and cloud sharing services, a large amount of duplicate data will be uploaded, which will affect user experience, increase network load, and reduce the storage efficiency of the storage system. Therefore, data deduplication technology is required. Data deduplication technology is a lossless data compression technology, which is widely accepted as an effective technology to reduce data transmission and storage overhead. According to where the duplicate data is deleted, it can be divided into sou...

AI Technical Summary

Problems solved by technology

[0003] The existing data deduplication method has the following problems. Although the deduplication at the source end also determines whether to transmit the entire file by querying the server, the transmission delay of this method is very large, and it needs to go through a lot of wide area networks.

However, deduplication at the destination is to transfer the entire file to the server, and then perform duplicate detection. Obviously, this method greatly consumes network bandwidth resources.

Images