Method and device for data compression
A data compression and data technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve the problems of low cardinality, low near-line data compression, etc., and achieve the effect of improving the compression ratio
Inactive Publication Date: 2015-03-25
HUAWEI TECH CO LTD
View PDF1 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0006] 2. The attribute value is an optional value with low cardinality, and the cardinality of the combination relationship between attributes is also low
[0011] 4. There are logical associations between many records stored non-continuously, and there are only a few differences in attributes, while most of the attributes are the same
However, the compression ratio of near-line data in the prior art is low
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
[0029] The mainstream relational databases in the prior art basically store data in row mode, and the compression method is as follows: after scanning the entire table in order to find repeated characters and strings, store the above characters and strings in a compression dictionary, and then The actual data in the table is replaced by a substitute symbol stored in the dictionary corresponding to the data string, thereby achieving the purpose of c...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The embodiment of the invention provides a method and a device for data compression. The method comprises the following steps of: generating a digital fingerprint for data similarity detection according to the raw data; determining a data block with data similarity to the raw data according to the digital fingerprint, routing the raw data to a compression process of the determined data block; and executing the compression process of the determined data block, and finishing the compression of the raw data. The device corresponding to the method comprises a digital fingerprint generator, a data routing module, a compression module and a data block. The above scheme can compress the raw data to the specified data block based on the similarity of the digital fingerprint generated by the raw data, thereby improving the compression ratio of the data compression.
Description
technical field [0001] Embodiments of the present invention relate to the field of data storage, and more specifically, to a data compression method and device. Background technique [0002] After years of system construction and optimization, the business analysis system in the telecommunications field has established a data information platform for the operation of telecommunications companies. However, with the passage of time and the accelerated growth of business volume, the data stored in the data warehouse expands rapidly, and the storage requirements must be met through continuous expansion, resulting in high costs and low returns for the data warehouse, and eventually no expansion. In order to meet the above challenges, the data warehouse system of the telecom industry has gradually introduced a hierarchical storage strategy. According to the response requirements of data creation time, access frequency and access time, data is divided into online, near-line, archiv...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 陈系华张亮
Owner HUAWEI TECH CO LTD