Supercharge Your Innovation With Domain-Expert AI Agents!

RDD persistence method based on SSD and HDD hybrid storage system

A hybrid storage and persistence technology, applied in the field of data processing, can solve the problem of inability to achieve on-demand persistence, and achieve the effect of on-demand persistence

Active Publication Date: 2017-09-22
POWERLEADER TELECOM TECH
View PDF2 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The present invention aims to solve the technical problem that the on-demand persistence cannot be realized in the prior art, and provides an RDD persistence method based on an SSD and HDD hybrid storage system that cannot realize on-demand persistence

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • RDD persistence method based on SSD and HDD hybrid storage system
  • RDD persistence method based on SSD and HDD hybrid storage system
  • RDD persistence method based on SSD and HDD hybrid storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0018] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0019] Specifically, the emergence of solid-state drives (Solid-State Drive, SSD for short) has brought new opportunities for improving the performance of storage systems. SSDs have advantages such as low power consumption, low latency, and small size. Different from the traditional enterprise hard disk (Hard DiskDrive, referred to as HDD) which is addressed by moving the mechanical arm, the SSD is completely built on the semiconductor chip, so it has random access performance. However, due to the high cost of SSD capacity and li...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an RDD persistence method based on an SSD and HDD hybrid storage system. The method comprises the steps that an RDD module transmits a block identifier in the RDD module and a preset persistence level of data in the RDD module to a block manager; a disk block manager transmits the preset persistence level to a device adapter; the device adapter receives the preset persistence level of the data and reads two directory management variables in a configuration file, the preset persistence level is matched with a temporary file directory in the corresponding directory management variable according to the preset persistence level of the data, and the temporary file directory obtained through matching is returned to the disk block manager; the disk block manager obtains a file name according to the block identifier, obtains a data storage address according to the temporary file directory obtained through matching and the file name and returns the data storage address to the block manager; and the block manager stores the data in the RDD module into an SSD or an HDD according to the data storage address.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to an RDD persistence method based on an SSD and HDD hybrid storage system. Background technique [0002] In the current era of big data, in the face of massive data, how to manage, analyze and extract valuable information in an effective time has become an urgent problem that people need to solve. However, regardless of size, variety or structure, big data poses a huge challenge to people's ability to manage data. [0003] Spark is currently an efficient and widely used big data computing framework in the industry. It is a general and fast large-scale data processing engine. First of all, Spark provides a unified solution that can be used for complex tasks such as interactive query, real-time stream processing, and machine learning; secondly, Spark divides stages and tasks through Resilient Distributed Dataset (RDD). The Directed Acyclic Graph (DAG for short) execution en...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F17/30G06F11/30G06F11/32
CPCG06F3/064G06F3/0643G06F3/068G06F11/3034G06F11/325G06F16/182
Inventor 陆克中黄泽成毛睿廖好朱金彬隋秀峰
Owner POWERLEADER TELECOM TECH
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com