Method and device for processing data fragments and deleting garbage files

A data sharding and data technology, applied in the computer field, can solve problems such as the inability to effectively realize sharding and merging, and achieve the effect of optimizing the processing mechanism and efficient processing requirements

Active Publication Date: 2015-09-02
BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
View PDF10 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, it needs to rely on the link function of the file

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for processing data fragments and deleting garbage files
  • Method and device for processing data fragments and deleting garbage files
  • Method and device for processing data fragments and deleting garbage files

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0044] figure 1 It is a flow chart of a method for processing data fragmentation in a distributed total order storage system provided in the first embodiment of the present invention. The method of this embodiment can be executed by a processing device for data fragmentation in a distributed total order storage system , the device can be realized by means of hardware and / or software, and generally can be integrated into the slice server in the distributed total order storage system, and used in cooperation with the management server in the distributed total order storage system. The method of this embodiment specifically includes:

[0045] 110. Acquire at least one piece of attribute description information corresponding to the data fragment during the process of generating the total order data fragment by the distributed total order storage system, wherein the attribute description information includes data iteration information.

[0046]As mentioned above, it is difficult t...

no. 2 example

[0064] figure 2 It is a flowchart of a method for processing data fragmentation in a distributed total order storage system according to the second embodiment of the present invention. This embodiment is optimized on the basis of the above-mentioned embodiments. In this embodiment, the processing instruction is specifically optimized as a split instruction for data fragmentation; correspondingly, after receiving at least one target data fragmentation processing When indicated, process the data iteration information in the file meta-information corresponding to the target data fragment, so as to realize the processing of the target data fragment. A split instruction; according to the split instruction, obtain target file meta information corresponding to the target data fragment; perform split processing on the data iteration information in the target file meta information to generate at least two split file meta information; The split result is returned to the management ser...

no. 3 example

[0086] image 3 It is a flowchart of a method for processing data fragmentation in a distributed total order storage system according to the third embodiment of the present invention. This embodiment is optimized on the basis of the above-mentioned embodiments. In this embodiment, the processing instruction is specifically optimized as a merging instruction for data fragments; correspondingly, after receiving the processing instruction for at least one target data fragment When indicated, process the data iteration information in the file meta information corresponding to the target data fragments, so as to realize the processing of the target data fragments. The specific optimization is as follows: receiving at least two target data Merge instructions of fragments; according to the merge instructions, at least two target file meta-information corresponding to the at least two target data fragments are obtained; and the data iteration information in the at least two target fil...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a method and device for processing data fragments and deleting garbage files. The method for processing the data fragments comprises the steps that in the process that a distributed total order storage system generates total order data fragments, and at least one piece of attribute description information corresponding to the data fragments is obtained, wherein the attribute description information comprises data iterative information; the attribute description information is written in file meta information corresponding to the data fragments; when an instruction for processing at least one target data fragment is received, the data iterative information in the file meta information corresponding to the target data fragments is processed to achieve processing on the target data fragments. According to the technical scheme, the technical effect for processing the target data fragments completely can be achieved without the needs for moving or modifying the data files, the processing mechanism of data fragments in an existing distributed total order storage system is optimized, and the ever-growing convenient and efficient processing demands of people for the data fragments are met.

Description

technical field [0001] The embodiment of the present invention relates to computer technology, in particular to a method and device for processing data fragments and deleting junk files. Background technique [0002] Generally speaking, data is stored in the database mainly through Key-Value (key-value pairs). Each key name (Key) stores a corresponding key value (Value), and the corresponding key value can be found through the key name, and then certain data operations can be performed on the key value. In addition, in order to achieve fast reading and writing of data in the database, the data stored in the database is generally fully sequenced data. [0003] Totally sequenced data is logically a super-large data set sorted by key (the number of data rows is more than one trillion). Due to the huge amount of data, it is impossible to completely store the super-large data set through only one or a few servers. of. Therefore, in the existing distributed total order storage ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/162G06F16/182G06F16/215
Inventor 徐佩林颜世光覃安李康梁栋
Owner BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap