Data storage method based on data filtering

Abstract

The present invention relates to a data storage method based on data screening. The method includes obtaining data to be stored, the data to be stored includes multiple levels of data, and data information is also included under each level of data; a central processing unit is also provided with Hierarchical matrix G (G1, G2, G3, G4, G5) and data similarity coefficient matrix K (K1, K2, K3, K4, K5), when comparing data similarity, if they are two data of any level If the similarity is greater than or equal to the coefficient corresponding to this level, it means that the two data at this level have a high similarity and are determined to be duplicate data; otherwise, no data screening will be performed, and the filtered level data and the corresponding data information under the level data will be stored. , as the remote backup data of the current data level. By dividing the stored data according to a specific format, and setting different similarity coefficients for each level, it is convenient to accurately judge the similarity of the data. It is necessary to use parameters with higher similarity coefficients to improve the accuracy of data comparison. .

Description

technical field [0001] The invention relates to the field of data storage, in particular to a data storage method based on data screening. Background technique [0002] At present, in the process of informatization development, a large amount of information data is generated rapidly, and people can analyze the actual operating conditions of various enterprises and mine high-value information based on these data. How to realize the efficient management of the above-mentioned types of data has therefore become a key research content at this stage. Even if a sufficient number of storage devices are built to complete the data storage process, a large amount of network bandwidth will be occupied when the data is transmitted, causing network congestion. [0003] When storing data, there will be many similar data that are repeated with each other. This kind of repeated data is usually a backup part to ensure data stability and avoid loss. Some data is caused by misoperation or som...

AI Technical Summary

Problems solved by technology

[0004] However, it is usually necessary to divide the data into blocks, but the rules of the blocks usually divide the content into blocks. However, due to the blockization according to the content, the data blocks are too long or too short, which is still not conducive to data storage and restricts data storage. speed, affecting the integrity of the data content

Images