Middleware-based repeated data deletion method and related device

Abstract

The invention relates to data transmission optimization, and discloses a middleware-based repeated data deletion method and a related device, and the method comprises the steps: obtaining a data processing message sent by a client through middleware, the data processing message carrying a data identifier and first data; obtaining N pieces of second data matched with the data identifier from a database to determine the repetition rate of the first data and each piece of second data in the N pieces of second data so as to obtain N repetition rates, wherein N is a positive integer; comparing theN repetition rates with a preset repetition rate; and if M repetition rates in the N repetition rates exceed the preset repetition rate, deleting M second data corresponding to the M repetition rates,where 0 < M < = N and M is an integer. By implementing the embodiment of the invention, a high-requirement transmission bandwidth is not needed, the operation load of a source end machine is favorably reduced, and the storage utilization rate is also improved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a middleware-based de-duplication method and a related device. Background technique [0002] With the rapid development of the new generation of information technology, the development of virtualization technology is becoming more and more mature. In a virtualization scenario, in order to prevent data loss caused by system operation errors or system failures, memory virtualization technology is used to back up data, which generally includes full backup and incremental backup, so a large amount of duplicate data will be generated. These duplicate data take up a lot of hard disk space, resulting in low storage utilization and high storage costs. In order to improve storage utilization and reduce storage costs, people have proposed the technology of data deduplication to save data storage space. [0003] Currently, in most storage systems, source-side deduplication or target-sid...

AI Technical Summary

Problems solved by technology

Among them, the source refers to the sender of the data, and the deduplication at the source refers to the deduplication of data before the data is transmitted through the network. However, deduplication at the source may cause the operating burden of the source machine. Heavy

The target end refers to the receiver of the data. The deduplication at the target end refers to deduplication after the data reaches the target server. However, deduplication at the target end requires high transmission bandwidth.

Images