Supercharge Your Innovation With Domain-Expert AI Agents!

Parallel processing big data storage system and method based on memory calculation mode

A big data storage and parallel processing technology, applied in computing, electrical digital data processing, data processing input/output process, etc., can solve problems such as permanent failure, and achieve energy consumption advantages, high throughput, and improved timeliness. Effect

Inactive Publication Date: 2018-11-30
HUNAN MECHANICAL & ELECTRICAL POLYTECHNIC
View PDF6 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Because SCM only has the same reading speed as DRAM, but the writing speed is more than 10 to 100 times different, and SCM will cause permanent failure when writing millions of times

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Parallel processing big data storage system and method based on memory calculation mode
  • Parallel processing big data storage system and method based on memory calculation mode
  • Parallel processing big data storage system and method based on memory calculation mode

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0049] The present invention aims to solve the problem that the current computer system cannot effectively process large data above the PB level; MapReduce needs to obtain data from the disk, and then write the intermediate result data back to the disk, and the design based on the disk makes its efficiency low and the I / O The overhead is huge, and it is not suitable for applications with online and real-time requirements. From the point of view of the system structure, the emergence of the memory computing mode brings the possibility of providing strong timeliness, high performance and high throughput system structure suppo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of information retrieval and database structures, and discloses a parallel processing big data storage system and method based on a memory calculation mode. A novel hybrid memory system is designed on the basis of a novel storage-level memory and a traditional DRAM, so that the memory capacity is greatly expanded on the premise of keeping the advantagesin cost and energy consumption, and the calculation can be carried out in not only the DRAM but also an SCM; and a data-centric processing mode based on a hybrid memory architecture is provided for big data processing, so that the timeliness of big data processing is greatly improved. A high-capacity hybrid memory system is constructed on the basis of a novel nonvolatile storage device to accelerate data processing, so that the timeliness of large data processing is greatly improved, and the processing is called memory calculation. From the system structure, the occurrence of the memory calculation mode brings the possibility for providing high-timeliness, high-performance and high-throughput system structure support for big data processing.

Description

technical field [0001] The invention belongs to the technical field of information retrieval and its database structure, and in particular relates to a parallel processing large data storage system and method based on an in-memory computing mode. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: Big data poses 4V challenges: scale, the volume of data is getting bigger and bigger, from terabytes to petabytes or even ten trillions of bytes; types, there are many types of data, including traditional Structured data includes unstructured data such as text, video, pictures, and audio, and the proportion of unstructured data is increasing rapidly; value, data value density is low, and it is difficult to perform calculations such as predictive analysis, operational intelligence, and decision support. ; Speed, the speed of big data processing is becoming more and more prominent, and the timeliness is difficult to guara...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F3/06
CPCG06F3/0614G06F3/0631G06F3/0634G06F3/0644G06F3/067
Inventor 吴勇
Owner HUNAN MECHANICAL & ELECTRICAL POLYTECHNIC
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com