Duplication eliminating method based on multidimensional lattice data spatial model

A technology of lattice data and spatial model, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as reduced efficiency, slow retrieval speed, and increased resource consumption, and achieve serious resource consumption and deduplication high efficiency effect
CN102708148AInactive Publication Date: 2012-10-03苏州云端信息科技有限公司
4 Cites 11 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
苏州云端信息科技有限公司
Publication Date
2012-10-03
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a duplication eliminating method based on a multidimensional lattice data spatial model. The method includes following steps: loading local cache data and building the multidimensional lattice data spatial model; transferring the data into customized data format and cutting the data into data points; searching the data one by one, positioning coordinates of each data point on dimensions corresponding to the data model, searching each data point from a first digit down digit by digit, characterizing each data point if the same does not exist in the data model, and marking the data as absence; and traversing the data points of the data, outputting the cache if the data is marked as absence, and searching next data until all the data is searched. The duplication eliminating method based on the multidimensional lattice data spatial model is suitable for filtration and duplication eliminating of various data, high in duplication eliminating efficiency and has fine application value in engineering. In addition, by the method, the problem of severe resource consumption caused by length difference of the data is solved.
Need to check novelty before this filing date? Find Prior Art

Description

[technical field]

[0001] The invention relates to data deduplication, in particular to a deduplication method based on a multidimensional lattice data space model. [Background technique]

[0002] There are many deduplication technologies. The most commonly used deduplication technologies such as hash and Bloom process the content that needs to be deduplicated and then match them one by one. This collection processing method is feasible, but it is cumbersome in massive amounts of data, and when the quantity is to a certain extent, the data repetition rate will increase rapidly, so data deduplication is meaningless. In the process of deduplication, a series of issues such as how to save deduplicated data and cache loading also need to be considered. If the deduplicated data cannot be saved and cached, restarting the deduplication server will start a new deduplication work, and it will be a repeated work for the processed data, which will reduce the accuracy of the data virtua...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More