Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data processing system and method

A data processing system and data processing technology, which are used in electrical digital data processing, special data processing applications, error detection of redundant data in operations, etc., can solve problems such as data migration reducing data processing efficiency, and improve efficiency. and speed, improve processing efficiency

Inactive Publication Date: 2013-04-24
DAWNING INFORMATION IND BEIJING
View PDF4 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, expansion nodes must consider expansion while not generating too much data migration and not reducing the efficiency of data processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing system and method
  • Data processing system and method
  • Data processing system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0040] figure 1 is a schematic structural diagram of a data processing system according to an embodiment of the present invention. Such as figure 1 As shown, the data processing system includes multiple storage nodes. The plurality of storage nodes may be divided into multiple levels, for example, at least including a first-level storage node 20A and a second-level storage node 20B. Each level of storage nodes may include one or more storage nodes.

[0041] The data processing system further includes a data distribution device 40, configured to classify the data stored in each storage node according to data classification rules, so as to obtain data of different levels. Data can be divided into first-level data, second-level data, third-level data, and so on.

[0042] For the data stored in each storage node, if the data stored in a storage node is class...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data distribution device which comprises a data classification unit and a data migration unit, wherein the data classification unit is used for classifying data stored in a plurality of storage nodes of a data processing system to obtain data of various stages according to a set data classification rule, wherein the data of various stages at least comprise first stage data and second stage data, the plurality of storage nodes at least comprise a first stage storage node and a second storage node; and the data migration unit is used for migrating the first stage data to the first storage node when the first stage data are not stored in the first storage node of the plurality of storage nodes as well as migrating the second stage data to the second storage node when the second stage data are not stored in the second storage node of the plurality of storage nodes. The invention also discloses a data processing method and system. Therefore, the efficiency of accessing data from mass data is improved.

Description

technical field [0001] The present invention relates to the technical field of computer data processing, and more specifically, to a data processing system, a data processing method, and a data distribution device in a large-scale data environment. Background technique [0002] The typical hierarchical storage strategy adopted by the existing storage system is to store frequently accessed data in the memory, and use different cache replacement algorithms to update the data stored in the memory. are stored on disk. This facilitates quick positioning during data query and analysis, thereby reducing maintenance and processing overhead. [0003] In addition, many existing distributed storage systems distribute and store massive data on multiple storage nodes according to a certain distribution strategy. However, as the amount of data increases and the number of nodes remains unchanged, memory will become a bigger bottleneck, and the overhead of data location will also increase...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30G06F11/14
Inventor 王颖宋怀明狄静舒苗艳超刘新春邵宗有
Owner DAWNING INFORMATION IND BEIJING
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com