A coding and compression method for HBase row keys in medical big data storage

A technology of big data storage and compression method, which is applied in the direction of database indexing, structured data retrieval, database distribution/replication, etc. It can solve the problems that the memory space cannot be adapted and the length of the row key cannot be effectively controlled.

Active Publication Date: 2020-01-03
ENJOYOR COMPANY LIMITED
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] In order to overcome the shortcomings of existing medical data storage methods that the length of the row key cannot be effectively controlled and the memory space cannot adapt to the large increase in the amount of data, the present invention provides a method that can effectively control the length of the row key and adapt to the large increase in the amount of data. Encoding and compression method of Hbase row key in medical big data storage

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A coding and compression method for HBase row keys in medical big data storage
  • A coding and compression method for HBase row keys in medical big data storage
  • A coding and compression method for HBase row keys in medical big data storage

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] The present invention will be further described below in conjunction with the accompanying drawings.

[0038] refer to Figure 1 ~ Figure 3, a method for encoding and compressing Hbase row keys in medical big data storage, said method comprising:

[0039] First, the encoding and compression of query conditions is as follows:

[0040] Step 1.1, according to the query condition used, judge whether the value range used by the query condition is fixed, if it is a fixed value range, perform step 1.2, otherwise perform steps 1.3 and 1.4;

[0041] Step 1.2. Check whether the corresponding code exists in the public dictionary table, and return the corresponding ID compression code if it exists, otherwise, send the dictionary category to the ID generation service, generate the ID code from the ID code returned by the ID generation service, and store it in the public Dictionary, return ID compression code;

[0042] Step 1.3. Split the value domain into the form of prefix + suf...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

A method for encoding and compressing Hbase row keys in big medical data storage comprises the steps that 1, query conditions are encoded and compressed, whether a value domain used by the query conditions is fixed or not is judged according to the used query conditions, the query conditions are respectively encoded till encoding of all the query conditions is completed, all output compressed codes are spliced into new characters serving as the row keys of service data, the service data is stored in a Hbase table; 2, the query process is executed, wherein whether the value domain used by the query conditions is fixed or not is judged according to the used query conditions, the query conditions are respectively encoded and are converted into query service data in the Hbase. The length of the row keys is effectively controlled, adaptive data volume is remarkably increased, and certain multi-condition-based query is achieved.

Description

technical field [0001] The invention belongs to the field of medical data storage, in particular to a method for encoding and compressing Hbase row keys in medical big data storage. Background technique [0002] With the rapid development of cloud storage and cloud computing technology, the technical research on medical big data storage is becoming more and more popular. In the process of integrating and storing the historical data of hospitals in Hbase, the primary problem we must face is how to The unique identifier of the hospital data is the primary key, and use certain coding rules to generate a unique identifier that meets the requirements of the Hbase row key specification. The reason is that the length of the row key Rowkey of Hbase cannot be too long. If it is too long, such as 100 bytes, then the The row keys of 10 million pieces of data will consume nearly 1G of memory space. At the same time, Hbase can only return results efficiently through row keys. In view of ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/22G06F16/27
CPCG06F16/2282G06F16/27
Inventor 于海龙李建元温晓岳
Owner ENJOYOR COMPANY LIMITED
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap