Unlock instant, AI-driven research and patent intelligence for your innovation.

A Compression Optimization Method Applied to Massive Incomplete Data

An optimization method and a complete data technology, applied in the database field, can solve problems such as compressed data redundancy, reduced compression efficiency, and increased compression costs, and achieve the effects of reducing redundancy, improving compression efficiency, and reducing storage space

Active Publication Date: 2019-04-09
LIAONING UNIVERSITY
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The AQ-MI method realizes the compression of massive incomplete data according to the user's frequent query conditions, but due to the design of the basic data structure and in order to obtain more accurate query results, there is a problem of compressed data redundancy during compression, that is, the existence of attributes The problem that tuples with missing values ​​will be compressed multiple times
This will increase the size of the compressed file, resulting in waste of storage space and reduced compression efficiency, greatly increasing the cost of compression
None of these methods are suitable for efficient compression of massive incomplete data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Compression Optimization Method Applied to Massive Incomplete Data
  • A Compression Optimization Method Applied to Massive Incomplete Data
  • A Compression Optimization Method Applied to Massive Incomplete Data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0052] The present invention will be further described below in conjunction with the accompanying drawings. Such as figure 1 Shown is an example diagram of the temperature partial data of an environmental test in a certain place. Here, only a few pieces of data are selected to illustrate the method in the present invention. The massive data represented by the data in the diagram is missing. If the previous mass incomplete data compression method is used for compression, tuple 2 and tuple 3 will be compressed multiple times, resulting in the problem of compressed data redundancy. In the hard optimization algorithm used in the present invention, when the attribute value field corresponding to the deterministic query condition is missing, the field is represented by * when encoding it. At this time, no matter how many attribute values ​​​​are missing in the tuple, the tuple can only calculate the unique Def_Query value, so that the tuple is written into the cache block Block to ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a compression optimization method applied to mass incomplete data. The compression optimization method comprises a hard optimization method and a soft optimization method. According to the method, by combining compression of the mass incomplete data with a traditional rough theory and improving the methods therein, the method is utilized to calculate property importance and property synthetic weight for properties of the incomplete data in the compression process. Meanwhile, a new encoding mode for property value fields in an incomplete data set is designed on the basis of the property synthetic weight. Through the method, the compression efficiency of the mass incomplete data is improved, the storage space of the mass incomplete data is reduced, and high-efficiency compression of the mass incomplete data can be realized on the premise of reducing redundancy. The method is adaptive to redundancy reduction compression of the mass incomplete data.

Description

technical field [0001] The invention relates to a compression optimization method applied to massive incomplete data, which belongs to the field of databases. Background technique [0002] With the advent of the era of big data, the scale of data has increased exponentially, and various forms of massive data are constantly being generated. Data compression is crucial to the storage, query, and application of massive data. In order to effectively manage massive data, various data compression technologies have been proposed. In massive data, data loss due to network errors, collection errors, and human errors is a very common phenomenon. Unreasonable compression methods for massive incompleteness will cause serious losses. Therefore, the research on massive incomplete data compression method is of great practical significance. [0003] At present, the research on massive data compression methods mainly focuses on the compression of complete data, such as index-based compres...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06G06F16/13G06F16/172
CPCG06F3/0608G06F3/064G06F16/134G06F16/172
Inventor 王妍孙凌峰李玉诺王俊陆宋宝燕
Owner LIAONING UNIVERSITY