Storage apparatus and data processing method

A storage device and data processing technology, which is applied in the direction of electrical digital data processing, input/output process of data processing, instruments, etc., can solve the problems of high cost, high power consumption, high cost, waste of power consumption, etc., to improve performance, The effect of mitigating the workload

Inactive Publication Date: 2016-03-23
SHANGHAI XINCHU INTEGRATED CIRCUIT
View PDF4 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, when the CPU of the terminal system is doing some light-weight data processing operations, the main microcontrollers in these storage devices are in an idle state when not doing data processing. device, which is a waste in performance and power consumption
[0006] As can be seen from the brief introduction above, the tasks processed by the above-mentioned large-scale cluster composed of thousands of commercial machines are all run and processed by the CPU of the commercial machines, and these tasks are some lightweight tasks that do not require arithmetic and logic operations. Unit (Arithmetic and Logic Unit, referred to as ALU) involved in a lot of work (such as mapping tasks and reduction tasks), first of all, it is wasteful in terms of power consumption, and secondly, it is also wasteful in terms of performance (high-performance ALU does not participate in a large number of calculations), so The power consumption and cost of the whole cluster are very high
[0007] Traditionally, in the method of data processing in the storage device, all data processing operations are completed by the main microcontroller in the storage device, the load is heavy, and the performance of the storage device will be affected
If a programmable logic array (Field Programmable Gate Array, referred to as FPGA) is used to implement operations such as data search, identification, comparison or matching, the efficiency is low and it is a waste of performance; if an application specific integrated circuit (Application Specific Integrated Circuit, referred to as ASIC) is used to implement, The cost is relatively high

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Storage apparatus and data processing method
  • Storage apparatus and data processing method
  • Storage apparatus and data processing method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0042] Such as Figure 5In the shown storage device 18 (the storage device 18 may be a magnetic disk, a solid state drive, a network attached storage or a redundant array of independent disks, or other storage devices that can be used in the present invention, this is not a limitation), The main micro-controller 20 is respectively connected with the upper system and the storage array 21, and is used to receive the data processing command issued by the upper system, and can also manage and control the storage unit, such as wear leveling, module selection, error checking and error correction, Data reading and writing etc.; Comparison matching module 19 is connected with main micro-controller 20 and memory array 21, to according to data processing command (data processing command can be data search command, data identification command, data comparison command, data matching command or other data Processing command) processes the data to be processed stored in the storage array 21...

Embodiment 2

[0044] Such as Figure 7 As shown, the present invention provides a data processing method applied to Figure 4 As shown in the storage device 18, the method includes:

[0045] S1: The upper-level system sends a data processing command (the data processing command may be a data search command, a data identification command, a data comparison command, a data matching command or other related commands) to the storage device 18 through the external interface 22;

[0046] S2: The main microcontroller 20 in the storage device 18 receives the data processing command, and judges whether the data in the storage array 21 needs to be processed according to the data processing command; if the judgment result is yes, execute S3; if the judgment result is no , execute S5;

[0047] S3: the main microcontroller 20 sends the data processing command to the comparison and matching module 19 (the comparison and matching module 19 can be a content-addressable memory or a triple content-addressa...

Embodiment 3

[0051] Google File System (google File System Google File System, referred to as GFS), a proprietary distributed file system, developed by Google, runs on the Linux platform. It provides massive storage for Google cloud computing, and is closely integrated with technologies such as Chubby, MapReduce, and Bigtable, and is at the bottom of all core technologies. GFS includes a master node (main server), multiple chunkservers (data servers) and multiple clients (clients running various applications). Figure 8 It is a schematic diagram of the GFS architecture. Each node is an ordinary Linux server. The job of GFS is to coordinate hundreds of servers to provide services for various applications. Chunkservers provide storage. GFS will divide the file into fixed-length data blocks, each data block has a globally unique and immutable ID (data block handle: chunk_handle), and the data blocks are stored on the chunkserver in the form of ordinary Linux files, for reliability considerat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention relates to the technical field of data storage, and particularly relates to a storage apparatus and a data processing method. A comparing and matching module is added in a storage apparatus, so that tasks such as retrieval, identifying and comparing or matching and the like of data in the storage apparatus are accelerated, the work load of a central processing unit is reduced, and the performance of the storage apparatus on data processing is improved.

Description

technical field [0001] The present invention relates to the technical field of data storage, in particular to a storage device and a data processing method. Background technique [0002] In the era of data and information, the Internet has become an indispensable part of people's daily life. There are many types of Internet applications, including social networks represented by Facebook, MySpace, and QQ, video sharing applications represented by Youtube, and Google, Yahoo, and Baidu. search engine applications, etc. The ever-increasing number of Internet users brings an explosion of data (videos, pictures, blogs, etc.). The technical route for processing these data can be divided into two aspects in essence: storage management technology for massive data and processing technology for massive data (log analysis, search engine application, etc.). [0003] The huge amount of data on the Internet makes us think of applying the idea of ​​parallel computing to commercial cluster...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/0689
Inventor 景蔚亮陈邦明
Owner SHANGHAI XINCHU INTEGRATED CIRCUIT
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap