Data processing method and system

A data processing and data technology, applied in the field of communications, can solve problems such as energy waste and equipment depreciation, save energy, slow down depreciation, and improve system performance

Active Publication Date: 2009-08-19
CHENGDU HUAWEI TECH
View PDF0 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0013] 2. Files are distributed in each storage node of the entire system. Even if only a small amount of data is read

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and system
  • Data processing method and system
  • Data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0060] Example one

[0061] The data processing system in this embodiment, such as Figure 4 As shown, it includes the main control node (namely the M node in the figure), the client node (namely the C node in the figure) and the storage node. The storage node includes the full data node (namely the B node in the figure) and the performance node (namely D node in the figure).

[0062] The method of data storage in this embodiment, such as Figure 5 Shown, including:

[0063] Step S401, the client node sends a data write request to the master control node;

[0064] Step S402, the master control node assigns an ID to the file to be stored, and sends the file ID and the address of the storage node to the client node;

[0065] Step S403, the client node divides the file to be stored into fragments and writes them to multiple storage nodes;

[0066] In step S404, a full data node among the multiple storage nodes obtains a fragmented file distributed in the multiple storage nodes corres...

Example Embodiment

[0099] Example two

[0100] In this embodiment, the main control node and the full data node can be combined and set, then the data processing system in this embodiment is as follows: Figure 7 As shown, the M'node is the merged main control node and the full data node, which is called the merged node, the D node is the performance node, and the C is the client node.

[0101] The method of data storage in this embodiment, such as Figure 8 Shown, including:

[0102] Step S601, the client node sends a data write request to the merging node;

[0103] Step S602, the merging node allocates an ID to the file to be stored, and sends the file ID and the address of the performance node to the client node;

[0104] Step S603: The client node divides the file to be stored into fragments and writes them to each performance node and merge node respectively;

[0105] In step S604, the merging node obtains fragment files distributed in multiple storage nodes corresponding to a storage file, int...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method and a data processing system which are used for saving energy and lowering equipment depreciation speed. The data processing method comprises the following steps: a global data node in a plurality of storage nodes obtains fragmented files corresponding to a storage file and distributed in the plurality of storage nodes, merges the fragmented files, and stores the merged file; and when a client node request for reading the stored file is learnt, the merged file is sent to the client node. By adopting the technical proposal provided by the embodiment of the data processing method and the data processing system, the merged file is stored, and consequently only one storage node needs to be started in file reading, thus saving energy and lowering equipment depreciation speed.

Description

Technical field [0001] The present invention relates to the field of communication technology, in particular to a data processing method and system. Background technique [0002] In order to increase the speed of accessing files, a common method is to use a higher-performance CPU (Central Processing Unit, central processing unit) and a larger bandwidth interface. These solutions are limited by the physical bandwidth of the interface and the maximum speed of the CPU. In recent years, a new method has been developed to combine multiple CPUs and multiple interfaces to provide higher speeds. [0003] In the above method, the method for the client node to read data includes the following steps: [0004] 1. The client node sends a data read request to the master node; [0005] 2. The master node obtains a globally unique file ID (identification) from the database and the distribution of the read file in each storage node according to the request, and returns it to the client node; [0...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCY02B60/188G06F17/30135G06F16/1724Y02D10/00
Inventor 文海
Owner CHENGDU HUAWEI TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap