A method and device for deleting duplicate data
A deletion method and technology for duplicating data, which is applied in the fields of electronic digital data processing, digital data information retrieval, special data processing applications, etc., can solve problems affecting system performance, consuming seek time, system response delay, etc., and achieve reasonable deletion of duplication Data, reduce seek time, reduce the effect of disk fragmentation
Active Publication Date: 2019-03-15
INSPUR BEIJING ELECTRONICS INFORMATION IND
View PDF4 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, this online deduplication method has many problems at the same time, the most important one is that it affects the performance of the system, especially when the amount of data is very large, it will take a lot of time to find the duplicate data.
At the same time, due to the repeated data using the index method, the file will have more fragments, resulting in more seek time in the process of data re-reading.
All of these cause delays in system response, and in severe cases, the cost of delay is higher than the cost of data redundancy
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
[0047] The embodiment of the invention discloses a method and device for deleting duplicate data, so as to realize reasonable deletion of duplicate data.
[0048] see figure 1 , a method for deduplicating data provided by an embodiment of the present invention includes:
[0049] S101. Query the read-write frequency of a file similar to the target file to be written in the file read-write frequency table;
[0050] Specifically, the method for deduplicating data de...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a repeated data deleting method and device.The method includes the steps of inquiring about the read-write frequency of a file similar to a to-be-written-in object file in a file read-write frequency table, judging whether the read-write frequency is larger than a preset threshold or not, if yes, writing the object file in a newly-distributed magnetic disk space, and if not, writing the object file in the newly-distributed magnetic disk space through a repeated data deleting strategy.It can be seen that through the combination of the executing of the repeated data deleting strategy and the read-write frequency of the file, repeated data deleting operation is not conducted on a file with high read-write frequency, and therefore magnetic disk segments are reduced, seek time is shortened, system performance is improved, and repeated data is deleted more reasonably.
Description
technical field [0001] The present invention relates to the technical field of computer storage, and more specifically, to a method and device for deleting duplicate data. Background technique [0002] With the continuous development of IT technology, many industries are showing a trend of rapid digitalization, and the application fields of information storage are becoming more and more extensive. Coupled with the application of cloud technology and cloud storage, the storage demand of enterprise data centers is increasing. The amount of data It has grown exponentially, and has risen from the previous TB level to the PB level, or even the EB level. At the same time, studies have shown that in the data stored in the application system, a large amount of duplicate data has caused a serious waste of storage resources. Therefore, the problem of high data redundancy in the storage system has received more and more attention. How to reduce the data storage capacity of the storage ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/16G06F16/174
CPCG06F16/162G06F16/1748
Inventor 刘相乐杨敏
Owner INSPUR BEIJING ELECTRONICS INFORMATION IND