Data storage method and device
A data storage device and data storage technology, applied in the computer field, can solve problems such as low effective data rate, achieve the effects of reducing storage pressure, increasing effective data rate, and improving reading performance
Active Publication Date: 2015-07-01
HUAWEI TECH CO LTD
View PDF5 Cites 9 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
When the number of files is large, the data blocks of most files will be scattered in multiple different containers, resulting in a low effective data rate (ratio of useful data blocks to all data blocks read) when reading files
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
[0057] An embodiment of the present invention provides a data storage method, such as figure 2 As shown, the method includes:
[0058] S101. Receive a file to be stored, divide the file to be stored into multiple data blocks of a preset size, and determine a first data block set from the multiple data blocks.
[0059] Wherein, the size of the data block can be set as required, for example, the size of the data block can be set to 4KB, and the embodiment of t...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The embodiment of the invention discloses a data storage method and device, and relates to the field of computers. The storage pressure of a disk can be reduced, the valid data rate of a read file is increased, and reading performance is improved. According to the specific scheme, the method includes the steps that a file to be stored is received, the file is divided into multiple data blocks of the preset size, and a first data block set is determined from the data blocks; when the ratio of the number of the data blocks in the first data block set to the number of the data blocks is larger than a first threshold value, if a second data block set with the number of the data blocks in the second data block set larger than a second threshold value exists, the data blocks in a third data block set are deleted from the first data block set; the remaining data blocks are written in a quick non-volatile storage device; when the ratio of the storage space occupied by the data blocks stored in the quick non-volatile storage device to the storage space of the quick non-volatile storage device is larger than a third threshold value, the data blocks stored in the quick non-volatile storage device are written in the disk.
Description
technical field [0001] The invention relates to the field of computers, in particular to a data storage method and device. Background technique [0002] With the advent of the big data era, the total amount of data continues to increase, and the storage pressure on storage devices is also increasing, and a large part of the storage pressure on storage devices comes from duplicate data. In order to reduce the storage pressure of the storage device, it is necessary to delete duplicate data when storing data. [0003] like figure 1 Shown is a schematic diagram of a common data storage system. The file is divided into data blocks of a fixed size (such as 4KB, that is, 4 kilobytes), each data block corresponds to a fingerprint information (English: fingerprint, abbreviation: FP, FP is a short feature used to uniquely identify a data block value). Data blocks are stored on the disk in units of containers (English: container), and each container has a fixed size (for example, 4...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G06F3/06G06F17/30
Inventor 柴云鹏孙东旺
Owner HUAWEI TECH CO LTD