Data processing method and device

A data processing and data technology, applied in the direction of electrical digital data processing, data processing input/output process, instruments, etc., to facilitate data query and avoid writing hot spots

Inactive Publication Date: 2020-04-07
HUAWEI TECH CO LTD
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] This application provides a data processing method and device to solve the problem of data writing hotspots and ensure the data reading rate

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device
  • Data processing method and device
  • Data processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] The following describes the embodiments of the present application in conjunction with the accompanying drawings.

[0031] Such as figure 1 As shown, the embodiment of the present application provides a distributed storage system 100 , and the distributed storage system 100 may include at least computing nodes 102 and storage nodes 103 . Optionally, the distributed storage system 100 may further include a client (client) 104 .

[0032] Wherein, the computing node 102 may include one or more computing nodes, and the computing node may be a carrier having a data processing function, for example, the computing node may be a server or the like.

[0033] The storage node 103 may include one or more storage nodes, and the storage node may be a carrier with data storage space, for example, the storage node may be a disk in a server or a computer, or the storage node It may be a logical storage space, such as a logical unit (logical unit, LU).

[0034] Based on the above sce...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method and device. The method may include: providing a plurality of sensors; the first calculation node receives a write-in command sent by the client, the first calculation node computes a corresponding relation between the calculation node and a file according to a routing view of a range partitioning algorithm; determining a first file corresponding tothe first data, the first calculation node calculates the first data according to the first data, storing to the first file; the first calculation node computes a file according to the corresponding relationship between the calculation node and the file; determining a second file corresponding to the first calculation node, and the first calculation node acquires second files stored by other calculation nodes except the first calculation node, and the first calculation node stores the acquired second files of the other calculation nodes, so that the problem that the calculation nodes write hotspots can be avoided, and the data reading rate is ensured.

Description

technical field [0001] The present application relates to the technical field of data storage, and in particular to a data processing method and device. Background technique [0002] Such as figure 1 As shown, the distributed storage system may include the following parts: client, computing node and storage node. The process of writing data is as follows: the user inputs data to the client, and the client uses the range partitioning algorithm or hash partitioning algorithm to allocate the input data to the corresponding computing nodes, and then the corresponding computing nodes The node executes the write operation and writes to the corresponding storage node. [0003] If the range partitioning algorithm is used, the written data is from A0 to A99, the number of computing nodes is 3, and the routing view of the range partitioning is: A0—A32 → computing node 1, A33—A65 → computing node 2, A66—A99 → computing node 3. Using the range partitioning algorithm may cause write ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/0611G06F3/067
Inventor 钟延辉张程伟
Owner HUAWEI TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap