Data processing method and device for distributed database and storage medium

A data processing and database technology, applied in the database field, can solve the problem of low performance of full table scan, and achieve the effect of speeding up the query speed, reducing the operation time, and quickly generating

Active Publication Date: 2019-11-12
北京航天智造科技发展有限公司
View PDF5 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Queries are generally conditional retrievals. When searching through the API provided by HBase itself, if you cannot set the relevant range of Rowkey or directly specify Rowkey, then HBase will scan the entire table, and HBase is designed for large data volumes. The total amount of data in a table can reach billions or even tens of billions of data, and the performance of full table scanning is very low.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device for distributed database and storage medium
  • Data processing method and device for distributed database and storage medium
  • Data processing method and device for distributed database and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangements of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0028] At the same time, it should be understood that, for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0029] The following description of at least one exemplary embodiment is merely illustrative in nature and in no way taken as limiting the invention, its application or uses.

[0030] Techniques, methods and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, techniques, methods and devices should be considered part of the description. ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method and device for a distributed database and a storage medium. The method comprises the steps of generating a service feature bit sequence based on a service field value corresponding to a service field, generating an identification feature bit sequence corresponding to to-be-processed data according to current time, and splicing the service feature bit sequence and the identification feature bit sequence to generate a row key Rowkey of the to-be-processed data. According to the method, the device and the storage medium, the service and the query performance are integrated, the Rowkey can be quickly generated, the Rowkey can be used for query, full-table scanning is avoided, and the query speed is increased. By using the Rowkey generated by theself-defined 64-system mask, it can be guaranteed that the binary sequence of Rowkey generation is from small to large, the sorting rule of the HBase insertion sequence is met, and the Rowkey generation range can be rapidly determined.

Description

technical field [0001] The present invention relates to the technical field of databases, in particular to a data processing method, device and storage medium of a distributed database. Background technique [0002] A distributed database is logically a unified whole, but physically it is stored on different physical nodes. There are many kinds of distributed databases, such as HBase and so on. HBase is a distributed column-oriented database built on the Hadoop file system. HBase is a database designed for large tables in enterprises with tens of billions of records. It has the characteristics of strong fault tolerance, high data reliability, and high performance. For columnar databases, data is stored according to columns. Generally, there is a concept of rowkey. Rowkey can uniquely identify a piece of data in a columnar database, and a piece of data can be quickly located by searching for a rowkey. Since HBase writes data in the ascending order of Rowkey, if the Rowkey o...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/27G06F16/22G06F16/2453
CPCG06F16/2255G06F16/2282G06F16/2453G06F16/27
Inventor 邵永安刘亚军贾庚泉翟双庆关鸿立
Owner 北京航天智造科技发展有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap