Unlock instant, AI-driven research and patent intelligence for your innovation.

Method, controller and system for data storage

A data storage and controller technology, applied in the storage field, can solve problems affecting user experience, HBase performance degradation, and consumption of computing power

Active Publication Date: 2021-07-09
深圳华为云计算技术有限公司
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Due to the large bandwidth pressure and short-term IO pressure and calculation pressure during the execution of compaction, the performance of HBase has dropped significantly.
The main cluster provides read and write services to user devices, but the main cluster will bring a lot of bandwidth pressure and short-term IO pressure during the execution of compaction, and the execution of compaction will consume computing power, resulting in a significant drop in HBase performance and affecting user experience

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, controller and system for data storage
  • Method, controller and system for data storage
  • Method, controller and system for data storage

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] The technical solution in this application will be described below with reference to the accompanying drawings.

[0046] figure 1 It is a system architecture diagram 100 of a data storage method and a controller according to the present application. The system is a cross-AZ active-active architecture system, such as figure 1 As shown, the system 100 includes user equipment 10 , an active cluster controller 11 and a standby cluster controller 12 . The active cluster controller 11 and the standby cluster controller 12 may be a computing device, such as a server, a desktop computer, and the like. A writing system and application programs are installed on the active cluster controller 11 and the standby cluster controller 12 . The master cluster controller 11 may receive input output (I / O) requests from user equipment. The master cluster controller 11 may also store the data carried in the I / O request, and write the data into a persistent storage device. The primary cl...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present application provides a data storage method, so that the controller of the main cluster does not need to perform a merge operation, avoiding a large bandwidth pressure and IO pressure during the merge process, and improving system performance. The method includes: receiving the merged file, the first serial number and the first identification information sent by the second controller, the first serial number is the serial number of the first data in the merged file, and the first identification information is used for Indicates that the area number of the merged file in the standby cluster is the first area number; determine the serial number of the last written data in the merged file according to the first serial number; compare the last written data in the merged file The serial number of the data and the serial number of the last written data in the unmerged file; when the serial number of the last written data in the unmerged file is less than the serial number of the last written data in the merged file When the number is , delete the unmerged files; store the merged files.

Description

technical field [0001] The present application relates to the field of storage, and more particularly, to a data storage method, controller and system. Background technique [0002] Under the cross-availability zone (Availability Zone, AZ) active-active architecture, the primary cluster provides read and write services for user devices, and the standby cluster is only used to back up the data of the primary The cluster sends the backup data to restore the data, and the standby cluster does not provide read and write services for user devices. Between the active cluster and the standby cluster, data synchronization is achieved by synchronizing data write-ahead logging (WAL). Both the active and standby clusters use a distributed storage system (Hadoop Database, HBase). The active and standby clusters generate their own meta and HFile according to the data WAL, and perform the merge compaction operation of HFile respectively. Due to the huge bandwidth pressure and short-term...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/182G06F16/11G06F16/27G06F11/14
CPCG06F11/1464G06F16/11G06F16/182G06F16/27
Inventor 徐振鑫
Owner 深圳华为云计算技术有限公司