A sliding block repeating data deleting method based on edge calculation

A technology of deduplication and edge computing, applied in computing, digital data processing, special data processing applications, etc., can solve the problems of poor deletion effect and coarse detection granularity, and achieve good effect, fine granularity, and elimination of sensitivity. Effect

Inactive Publication Date: 2019-04-26
ELECTRIC POWER RESEARCH INSTITUTE, CHINA SOUTHERN POWER GRID CO LTD +1
View PDF3 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The original complete file detection technology is a method of finding duplicate data at the granularity of files. The Single Instance Storage (SIS) of Windows 2000 uses this technology to delete. This method is simple to implement and fast in calculation speed, but the detection granularity is relatively coarse , th

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A sliding block repeating data deleting method based on edge calculation
  • A sliding block repeating data deleting method based on edge calculation
  • A sliding block repeating data deleting method based on edge calculation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0036] The present invention provides a sliding block deduplication method based on edge computing, including:

[0037] S1, calculate the summation of each overlapping block of the file object according to the Rsync summation check function and the sliding window, and obtain the value of the checksum;

[0038] S2. For each block, compare the value of the checksum with the previously stored value;

[0039] S3. If the checksum value matches the previously store...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of data deduplication, and particularly relates to a sliding block duplication data deletion method based on edge calculation, which comprises the following steps: calculating the summation of each overlapped block of a file object according to an Rsync summation check function and a sliding window to obtain the value of a check sum; Comparing, for eachblock, the value of the checksum to a previously stored value; If the value of the checksum is matched with the value stored in advance, comparing the calculated hash value of the SHA-1 with a previously stored value so as to carry out redundancy detection, and deleting a data block represented by the same data if the same data is detected; According to the sliding block repeated data deleting technology based on edge computing, the problems that traditional data de-duplication is poor in granularity and high in cost are solved, the defect that inserting and deleting of data are sensitive isovercome. A check method, a weak check detection method and a thought of manually setting the size of a window are added on the basis of an SHA algorithm 1, so that a better duplicate removal effect is achieved.

Description

technical field [0001] The invention belongs to the technical field of data deduplication, and in particular relates to a method for deduplicating data by sliding blocks based on edge computing. Background technique [0002] With the rapid development of the Internet of Things technology and 5G network architecture, new equipment and new technologies in the fields of equipment intelligence and wireless communication continue to emerge, the amount of digital information is growing rapidly, and the storage space required for data is increasing. Management data costs and The energy consumption of the central space is also becoming more and more serious. The traditional cloud computing model cannot effectively solve problems such as cloud center load and transmission bandwidth. Edge computing emerged as the times require. Edge computing refers to a new service model in which data or tasks can be calculated and executed on the edge of the network close to the source of the data....

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/174
Inventor 明哲吴涛许爱东李锦涛蒋屹新先兴平
Owner ELECTRIC POWER RESEARCH INSTITUTE, CHINA SOUTHERN POWER GRID CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap