Data processing method, device and storage medium of distributed database

A database and distributed technology, applied in the database field, can solve problems such as low performance of full table scan, and achieve the effect of speeding up query speed, avoiding full table scan, and reducing operation time.

Active Publication Date: 2021-11-23
北京航天智造科技发展有限公司
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Queries are generally conditional retrievals. When searching through the API provided by HBase itself, if the relevant range of Rowkey cannot be set or the Rowkey can be specified directly, then HBase will scan the entire table, and HBase is designed for large data volumes. The total amount of data in a table can reach billions or even tens of billions of data, and the performance of full table scanning is very low.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method, device and storage medium of distributed database
  • Data processing method, device and storage medium of distributed database
  • Data processing method, device and storage medium of distributed database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangements of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0028] At the same time, it should be understood that, for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0029] The following description of at least one exemplary embodiment is merely illustrative in nature and in no way taken as limiting the invention, its application or uses.

[0030] Techniques, methods and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, techniques, methods and devices should be considered part of the description. ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method, device and storage medium of a distributed database, wherein the method includes: generating a service characteristic bit sequence based on a service field value corresponding to the service field, and generating a bit sequence corresponding to the data to be processed according to the current time Identify the characteristic bit sequence, splicing the service characteristic bit sequence and the identification characteristic bit sequence to generate the row key Rowkey of the data to be processed. The method, device and storage medium of the present invention integrate business and query performance into one, can realize rapid generation of Rowkey, can use Rowkey to query, avoid full table scanning, and speed up query; use self-defined 64 base mask to generate Rowkey can ensure that the binary order generated by Rowkey is from small to large, which conforms to the sorting rules of HBase insertion order, and can quickly determine the range of Rowkey generation.

Description

technical field [0001] The present invention relates to the technical field of databases, in particular to a data processing method, device and storage medium of a distributed database. Background technique [0002] A distributed database is logically a unified whole, but physically it is stored on different physical nodes. There are many kinds of distributed databases, such as HBase and so on. HBase is a distributed column-oriented database built on the Hadoop file system. HBase is a database designed for large tables in enterprises with tens of billions of records. It has the characteristics of strong fault tolerance, high data reliability, and high performance. For columnar databases, data is stored according to columns. Generally, there is a concept of rowkey. Rowkey can uniquely identify a piece of data in a columnar database, and a piece of data can be quickly located by searching for a rowkey. Since HBase writes data in the ascending order of Rowkey, if the Rowkey o...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/27G06F16/22G06F16/2453
CPCG06F16/2255G06F16/2282G06F16/2453G06F16/27
Inventor 邵永安刘亚军贾庚泉翟双庆关鸿立
Owner 北京航天智造科技发展有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap