Mixed file storage method and device

A technology for mixing files and storage devices, applied in file systems, file access structures, special data processing applications, etc., can solve problems such as low storage efficiency and waste of storage space

CN109684293AInactive Publication Date: 2019-04-26THE PEOPLES BANK OF CHINA NAT CLEARING CENT
4 Cites 1 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Publication Date
2019-04-26
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention provides a mixed file storage method and device. The method comprises the following steps: reading data in a to-be-stored file; Analyzing the data and generating data details; Generatingpage group element information according to the data details; And storing the data and the page group meta-information into a corresponding storage space according to a preset storage rule. Accordingto the method, the page group original information corresponding to the data is obtained by analyzing the file data, so that the data storage is more perfect and convenient, and in addition, the purposes of meeting the requirement of detail query and meeting the requirement of statistical query are achieved.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of hybrid file storage, in particular to a hybrid file storage method and device. Background technique

[0002] In big data application scenarios, there are usually two types of application requirements: detailed query and statistical query. Detailed query filters qualified multi-row data from structured data by specifying conditions. For example, taking Hbase as an example, by establishing a row key index, you can quickly query qualified data. Data is usually logically in the form of complete rows. return. Statistical queries usually filter data by specifying conditions, and only need to return the data of certain columns, plus operator operations such as aggregation to complete the final return of statistical data, such as performing groupby operations through hive to complete data aggregation for parquet format . Hbase's row key design and data organization and storage in the form of column families, a...

Examples

Embodiment Construction

[0034] Embodiments of the present invention provide a hybrid file storage method and device.

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0036] Usually, the demand analysis for data mainly has the following four requirements: 1. Support massive data scanning and select several columns; 2. Support search based on primary key and respond at sub-second level; 3. Support OLAP-like processing on massive data Interactive query, and many filter conditions are involved in the query; 4. It supports rapid extraction...