Unlock instant, AI-driven research and patent intelligence for your innovation.

Coarse-grained process-level parallel optimization method and system for compaction for key-value systems

An optimization method and process-level technology, applied in multi-programming devices, program control design, program startup/switching, etc., can solve problems such as insufficient utilization of CPU computing power and coarse-grained process-level parallelism potential of compaction.

Active Publication Date: 2020-07-10
ANHUI UNIVERSITY
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Although these research works can improve the performance of Key-Value storage, they have not tapped the coarse-grained process-level parallelism potential of compaction, and therefore have not fully utilized the computing power of the CPU.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Coarse-grained process-level parallel optimization method and system for compaction for key-value systems
  • Coarse-grained process-level parallel optimization method and system for compaction for key-value systems
  • Coarse-grained process-level parallel optimization method and system for compaction for key-value systems

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] Such as Figure 1-Figure 4 as shown, Figure 1-Figure 4 The present invention proposes a compaction coarse-grained process-level parallel optimization method and system for Key-Value systems.

[0051] refer to figure 1 , image 3 , Figure 4 , the present invention proposes a compaction coarse-grained process-level parallel optimization method for Key-Value systems, comprising the following steps:

[0052] S1. After the main process receives the read and write request, it judges whether the compaction thread needs to be scheduled. When the judgment result is yes, the main process divides the task information in the compaction thread into the main process subtask and the slave process subtask;

[0053] In this embodiment, after the main process receives a read request or a write request, it judges whether the Key-Value system needs to schedule a compaction thread according to the file capacity threshold that each layer of the Key-Value system can accommodate;

[005...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention discloses a compaction coarse-grained process-level parallel optimization method and system for Key-Value systems. The method includes: after receiving a read-write request, the main process judges whether a compaction thread needs to be scheduled; The task information is divided into subtasks of the master process and subtasks of the slave process; the master process and the slave process perform compaction operations on the subtasks of the master process and the subtasks of the slave process respectively; the master process completes the compaction operation of the master process subtasks in the master process and the slave process completes Optimize the files of the Key-Value system after the process subtask compaction operation. The present invention utilizes the master process to complete the dynamic segmentation of compaction tasks in the Key-Value system, and realizes a new mode of collaborative compaction processing with slave processes, taps the process-level parallelism of compaction tasks, fully utilizes the computing resources of the CPU, and improves the The utilization rate of CPU resources and the time consumed by compaction are optimized, which not only improves the real-time performance of data processing, but also improves system performance and throughput.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to a compaction coarse-grained process-level parallel optimization method and system for Key-Value systems. Background technique [0002] In the environment of data-intensive applications such as search engines, big data, cloud computing, cloud storage, e-commerce and social networks, persistent Key-Value storage is increasingly playing an irreplaceable role. For write-intensive loads, LSM-tree-based Key-Value storage is currently the most advanced technology. Among them, widely used Key-Value storage includes Google's BigTable, LevelDB, and FaceBook's Cassandra. The LSM-tree data organization consists of a C0 component in the memory and C1~Ck in the disk. Among them, C0 is used as a buffer in the memory. Once C0 reaches the threshold, it will perform a merge sort operation with C1. This operation is called compaction. In addition, when the component Ci in the disk reaches t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/4881G06F9/5038
Inventor 孙辉刘伟施巍松
Owner ANHUI UNIVERSITY
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com