Data deduplication method and system, FPGA and data processing system
A data processing system and data processing technology, applied in the computer field, can solve the problems of occupation, affecting the computing performance of cloud computing data centers, low data deduplication computing efficiency, etc., to shorten the time, simplify the research and development cycle, and improve the throughput performance. Effect
Inactive Publication Date: 2019-05-10
ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
View PDF0 Cites 1 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0002] In the era of cloud computing, a large amount of data transmission requires data deduplication calculations, resulting in the occupation of a large amount of CPU (full name in Chinese: Central Processing Unit, full name in English: Central Processing Unit) or computing resources, affecting the computing performance of the cloud computing data center, and The data deduplication calculation efficiency in the prior art is low
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0039] The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.
[0040] The embodiment of the present application discloses a data deduplication method, which improves the efficiency of data deduplication calculation.
[0041] see figure 1 , a flow chart of a data deduplication method shown according to an exemplary embodiment, as shown in figure 1 shown, including:
[0042] S101: When receiving the data processing command sent by the host, read the data to be processed in the cache according to the target number of bytes;
[0043] The execution subje...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a data deduplication method and system, an FPGA, a computer readable storage medium and a data processing system, and the method comprises the steps: reading to-be-processed data in a cache according to a target byte number when a data processing command sent by a host is received; performing data de-duplication processing on the to-be-processed data by using a de-duplication formula to obtain de-duplication data; and when the to-be-processed data in the cache is completely processed, returning the duplicate removal data to the host. The invention provides a data deduplication method. A general processor CPU is used as a host end to operate a main program, an FPGA is used as a hardware acceleration unit for a coprocessor to perform a data deduplication algorithm, the parallelism degree of the algorithm is improved through a parallel pipeline optimization mode, the execution performance of the data deduplication algorithm can be effectively improved, and the throughput rate performance during algorithm execution is improved.
Description
technical field [0001] The present application relates to the field of computer technology, and more specifically, to a data deduplication method and system, an FPGA, a computer-readable storage medium, and a data processing system. Background technique [0002] In the era of cloud computing, a large amount of data transmission requires data deduplication calculations, resulting in the occupation of a large amount of CPU (full name in Chinese: Central Processing Unit, full name in English: Central Processing Unit) or computing resources, affecting the computing performance of the cloud computing data center, and The data deduplication calculation efficiency in the prior art is low. [0003] Therefore, how to improve the efficiency of data deduplication operation is a technical problem to be solved by those skilled in the art. Contents of the invention [0004] The purpose of this application is to provide a data deduplication method and system, an FPGA, a computer-readabl...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/38G06F9/50G06F13/42
Inventor 徐翔宇
Owner ZHENGZHOU YUNHAI INFORMATION TECH CO LTD