Supercharge Your Innovation With Domain-Expert AI Agents!

A rdd persistence method based on ssd and hdd hybrid storage system

A hybrid storage and persistence technology, applied in the field of data processing, can solve the problem of inability to achieve on-demand persistence, and achieve the effect of on-demand persistence

Active Publication Date: 2020-05-12
POWERLEADER TELECOM TECH
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The present invention aims to solve the technical problem that the on-demand persistence cannot be realized in the prior art, and provides an RDD persistence method based on an SSD and HDD hybrid storage system that cannot realize on-demand persistence

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A rdd persistence method based on ssd and hdd hybrid storage system
  • A rdd persistence method based on ssd and hdd hybrid storage system
  • A rdd persistence method based on ssd and hdd hybrid storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0018] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, in which the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention, but should not be construed as limiting the present invention.

[0019] Specifically, the emergence of solid-state drives (Solid-State Drive, SSD for short) has brought new opportunities to improve storage system performance. SSDs have the advantages of low power consumption, low latency, and small size. Unlike traditional enterprise-level hard disks (Hard DiskDrive, HDD for short) that are addressed through mobile robotic arms, SSDs are completely built on semiconductor chips, so they have random access performance. However, due to the high cost of SSD capacity an...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an RDD persistence method based on an SSD and HDD hybrid storage system. The method comprises the steps that an RDD module transmits a block identifier in the RDD module and a preset persistence level of data in the RDD module to a block manager; a disk block manager transmits the preset persistence level to a device adapter; the device adapter receives the preset persistence level of the data and reads two directory management variables in a configuration file, the preset persistence level is matched with a temporary file directory in the corresponding directory management variable according to the preset persistence level of the data, and the temporary file directory obtained through matching is returned to the disk block manager; the disk block manager obtains a file name according to the block identifier, obtains a data storage address according to the temporary file directory obtained through matching and the file name and returns the data storage address to the block manager; and the block manager stores the data in the RDD module into an SSD or an HDD according to the data storage address.

Description

Technical field [0001] The invention relates to the technical field of data processing, in particular to an RDD persistence method based on an SSD and HDD hybrid storage system. Background technique [0002] In the current era of big data, in the face of massive data, how to manage, analyze, and extract valuable information in an effective time has become an urgent problem for people to solve. However, regardless of scale, type or structure, big data poses a huge challenge to people's ability to control data. [0003] Spark is currently an efficient and widely used big data computing framework in the industry. It is a universal and fast large-scale data processing engine. First of all, Spark provides a unified solution that can be used for complex tasks such as interactive query, real-time stream processing, and machine learning. Second, Spark divides phases and tasks through Resilient Distributed Dataset (RDD). The Directed Acyclic Graph (DAG) execution engine optimizes the exec...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06G06F16/182G06F11/30G06F11/32
CPCG06F3/064G06F3/0643G06F3/068G06F11/3034G06F11/325G06F16/182
Inventor 陆克中黄泽成毛睿廖好朱金彬隋秀峰
Owner POWERLEADER TELECOM TECH
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com