Data storage device and method

A data storage and data technology, applied in the field of network applications, can solve the problems of taking up a lot of time and reducing the speed of data storage, and achieve the effects of saving time, improving processing speed, and shortening time

Active Publication Date: 2013-04-10
BEIJING QIHOO TECH CO LTD
View PDF7 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] When there is a lot of input data, it will take a lot of time to id each data (such as the key in each key-value pair), which will reduce the data storage speed

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data storage device and method
  • Data storage device and method
  • Data storage device and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0116] In this embodiment, the original data (vectors) in Table 1 on the left are continuously idized and converted into the form in Table 2 on the right.

[0117] Table 1 Table 2

[0118]

[0119] In this example, 2 nodes (Node0, Node1) are used, and each node has 2 threads (Thread0, Thread1) inside. According to the number of nodes, the original data can be divided as shown in Table 3:

[0120] Table three

[0121]

[0122] Referring to Table 3, we can see that each node processes 5 pieces of data. by the formula It can be calculated that the starting number of Node0 is 0, and the starting number of Node1 is 5.

[0123] Each Thread sequentially accesses the data to be processed by the Thread, and uses the atomic operation to increment the start number each time one is processed. For example, Thread0 of Node0 processes the first data (10, 3), then the number is (0, 3), and the number is incremented by atomic_inc to 1. Thread1 of Node0 processes the first data (40,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data storage method, which comprises the following steps of: grouping input data, wherein the group number of the data is determined by the number of nodes processing the data; sequentially allocating a group of data to be processed to each node, and performing parallel continuous identification (ID) processing on corresponding data by utilizing each node, wherein the continuous ID processing refers to the replacement of a keyword in the data into an integer from 0 to N; and storing the data obtained by the continuous ID processing. By the method, the data can be subjected to the parallel ID processing, so that the storage time is saved. The invention also discloses a corresponding device.

Description

technical field [0001] The invention relates to the field of network applications, in particular to a data storage method and a data storage device. Background technique [0002] In massively parallel computing, the input data is usually a series of key-value (key-value) pairs. After a series of calculations, the new value corresponding to each key is output. In order to facilitate addressing and save storage space, the input data can be continuously idized, that is, the key in each key-value pair is replaced by an integer of 0-N. [0003] When there is a lot of input data, it will take a lot of time to id each data (such as the key in each key-value pair), which will reduce the data storage speed. Contents of the invention [0004] In view of the above problems, the present invention is proposed to provide a data storage device and a corresponding data storage method that overcome the above problems or at least partially solve the above problems. [0005] According to ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 齐路何锐邦唐会军
Owner BEIJING QIHOO TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap