Blockchain-based data processing device and method

Abstract

To provide a block chain-based high-speed data processing device and a method, which do not affect the efficiency of performance even when data increases as the data is processed by utilizing a material structure for maximizing performance in a block chain system.SOLUTION: The present invention relates to a data processing technique, and more particularly to a block chain-based high-speed data processing device and a method, for performing management by a material structure format capable of maximizing performance when data is processed in a block chain system. When <Key>:<Value> is stored, a final value is stored in a Log Structured Merge (LSM) tree material structure format, and the key used for storing the final value is separately stored as a B+ tree material structure format, so that storage performance and reference (read) performance can be improved as compared to those of conventional formats.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a data processing technology, and more particularly to a blockchain-based high-speed data processing device and method that manages data in a data structure type manner that can maximize performance when processing data in a blockchain system. [Background technology]

[0002] The database used by the blockchain system is LevelDB, also known as a KV (Key-Value) database, rather than the relational database management system (RDBMS) used in existing systems.

[0003] LevelDB is a database that allows for quick storage and retrieval of key-value pairs.

[0004] If LevelDB is applied to a blockchain system, the block ID is set as the key, and the value is set to the block information value corresponding to the block ID. Therefore, it becomes possible to quickly refer to the block value stored in the blockchain based on the block ID.

[0005] When a KV (Key-Value) database stores KV pairs in a file, it stores them as a tree-type data structure, and in almost all LevelDBs, data is stacked and stored in the commonly used B+ Tree format.

[0006] The B+ tree data structure type is a structure that is commonly and frequently used as a data storage structure not only in the file system of the Windows operating system (OS), but also in commercial relational database management systems such as MySQL / Oracle, and is said to be a method that can efficiently store large amounts of data.

[0007] However, when considering storage performance (speed) and reference performance (read speed), it is said that the performance efficiency decreases to O(nlogn) as the amount of data to be stored increases.

[0008] Since blockchain data is not deleted and only append-only operations are performed, it is believed that the performance of LevelDB will inevitably decrease as the number of blocks increases. Since LevelDB stacks blocks using a B+ tree data structure type method, it can be determined that the performance efficiency drops sharply in direct proportion to the efficiency of a B+ tree (O(logN) when storing, O(logN) when reading). Summary of the Invention [Problem to be solved by the invention]

[0009] The present invention provides a high-speed data processing device and method based on blockchain, which processes data using a data structure that maximizes the performance of the blockchain system, so that performance efficiency is not affected even when data increases.

[0010] The technical problems that the present invention aims to solve are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be apparent to a person having ordinary skill in the art to which the present invention pertains from the following description. [Means for solving the problem]

[0011] According to one aspect of the present invention, there is provided a blockchain-based data high speed processing device.

[0012] A blockchain-based high-speed data processing device according to one embodiment of the present invention may include a key management unit that manages the key of a <key>:<value> pair, and a value management unit that manages the value of the <key>:<value> pair.

[0013] According to another aspect of the present invention, there is provided a high-speed data processing method based on a blockchain and a computer program for executing the same.

[0014] A high-speed data processing method based on a blockchain according to an embodiment of the present invention and a computer program for executing the same may include a step of reading configuration information for operation, a step of checking whether a WAL file in an active state exists, a step of checking whether a WAL file in an immutable state exists, and a step of operating a flush channel. Effect of the Invention

[0015] According to an embodiment of the present invention, when storing <Key>:<Value>, the final value is stored using a Log Structured Merge (LSM) tree data structure type method, and the key used when storing is stored separately as a B+ tree data structure type, thereby improving storage performance and reference (read) performance compared to conventional methods.

[0016] The effects of the present invention are not limited to the effects described above, but include all effects that can be inferred from the configuration of the invention described in the detailed description of the present invention or the claims. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram for explaining a high-speed data processing device based on a blockchain according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram for explaining a high-speed data processing device based on a blockchain according to an embodiment of the present invention. [Diagram 3] FIG. 2 is an exemplary diagram showing a method and structure for high-speed data processing based on a blockchain according to an embodiment of the present invention. [Figure 4] FIG. 2 is an exemplary diagram showing a method and structure for high-speed data processing based on a blockchain according to an embodiment of the present invention. [Diagram 5] FIG. 2 is an exemplary diagram showing a method and structure for high-speed data processing based on a blockchain according to an embodiment of the present invention. [Figure 6] FIG. 2 is an exemplary diagram showing a method and structure for high-speed data processing based on a blockchain according to an embodiment of the present invention. [Figure 7] FIG. 2 is an exemplary diagram showing a method and structure for high-speed data processing based on a blockchain according to an embodiment of the present invention. [Figure 8] FIG. 2 is an exemplary diagram showing a method and structure for high-speed data processing based on a blockchain according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Since the present invention can be modified in various ways and can have various embodiments, a specific embodiment is illustrated in the drawings and described in detail in the detailed description of the invention. However, this is not intended to limit the present invention to a specific embodiment, and includes all modifications, equivalents, or alternatives within the spirit and technical scope of the present invention. In describing the present invention, if a specific description of a known technology related to the present invention is deemed to be likely to unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the singular expression used in this specification and claims generally means "one or more" unless the context clearly indicates otherwise.

[0019] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same or corresponding components are designated by the same reference numerals, and redundant description thereof will be omitted.

[0020] Instead of the relational database management system (RDBMS) used in existing systems, the blockchain system applies and uses a key-value database that was created to quickly store and reference a key and value as a single <key>:<value> pair. When storing a <key>:<value> pair, the key-value database universally stores data in the form of a B+ tree data structure.

[0021] In the blockchain-based high-speed data processing device 10 according to an embodiment of the present invention, when storing key-values ​​in a blockchain system, the final value is stored in a Log Structured Merge (LSM) tree data structure type method, and the key used for storing is stored separately as a B+ tree data structure type, thereby improving storage performance and reference (read) performance compared to conventional methods.

[0022] 1 and 2 are diagrams for explaining a high-speed data processing device based on a blockchain according to one embodiment of the present invention.

[0023] Referring to FIG. 1 , a blockchain-based data processing device 10 may include a key management unit 100 and a value management unit 200.

[0024] The key management unit 100 can manage the key of a <Key>:<Value> pair. The key management unit 100 can store the key as a B+ tree data structure type. The key management unit 100 can register (store) the key and the index (Internal index, IX) in a Direct Key Index (DKI) file for quick search of the key. The key management unit 100 can quickly search the index (Internal index, IX) for the <Key> in the process of reading <Key>:<Value>.

[0025] The value management unit 200 can manage the value of a <key>:<value> pair.

[0026] The value management unit 200 can store the values ​​in a Log Structured Merge (LSM) tree-structured format. For example, the value management unit 200 can store the values ​​in a Value Log file.

[0027] The blockchain-based data high-speed processing device 10 can register <Key>:<Index (Internal index, IX)> in a DKI file. For example, the DKI file may be of a B+ tree data structure type.

[0028] The blockchain-based data high-speed processing device 10 can register an <Internal index, IX> in a Value Index file. For example, the Value Index file may be of a B+ tree data structure type.

[0029] The blockchain-based data high-speed processing device 10 can store the <value> in a Value Log file. For example, the Value Log file may be a Log file based on a Log Structured Merge (LSM) tree-structured method.

[0030] The blockchain-based high-speed data processing device 10 uses the input key Key to search for index IX information from the DKI file, and uses the index IX information to search for location information of the value value from the Value Index file, and e This can be used to query the location information of the value from the Value Log file.

[0031] When a key Key is input to the blockchain-based high-speed data processor 10 in order to inquire about a value, the blockchain-based high-speed data processor 10 searches for the key Key from a DKI file (Direct Key Index File) of a B+ tree file structure. The blockchain-based high-speed data processor 10 can obtain location information (Fid, Offset, EntrySize) of the value Value corresponding to the key Key searched for from a Log file. The blockchain-based high-speed data processor 10 searches for a Log file corresponding to the obtained Fid (file id), calculates an offset Offset in a manner determined in the Log file, and then reads data corresponding to the entry size EntrySize, thereby being able to inquire about the value Value in the data structure.

[0032] 2, the blockchain-based data high-speed processing device 10 registers <Key>:<Index Internal index, IX> in a DKI file (Direct Key Index File), and in the process of reading <Key>:<Value>, reads the index Internal index, IX for the key to quickly query the value. For example, the index Internal index, IX may include location information (Fid, Offset, EnteySize) of the value corresponding to the key.

[0033] The blockchain-based high-speed data processor 10 can hold information in any one of the states of Active, Immutable, and Flushed in the process of writing / reading <Key>:<Value>. The blockchain-based high-speed data processor 10 can store and hold the same data in a write-ahead log (WAL) file and memory (MEMTable) in the process of writing / reading <Key>:<Value>.

[0034] The blockchain-based data high-speed processing device 10 can set the active MEMTable and WAL file to a certain size and use them to write / read real-time input values ​​<index Internal index, IX>:<value>.

[0035] When the set size of the MEMTable in the active state reaches the upper limit (or becomes full, crammed, or filled up), the blockchain-based high-speed data processing device 10 registers the MEMTable and WAL file in the immutable MEMTable and sets the state to immutable.

[0036] The blockchain-based high-speed data processing device 10 can asynchronously transfer data from the immutable MEMTable and WAL files to the Value Index file and Value Log file, and then delete them.

[0037] The blockchain-based high-speed data processing device 10 can store a value (Value) in a Value Log file that is in a flushed state, and can store an entry (Entry), which is search information for the value, in a Value Index file together with an index IX.

[0038] 3 to 8 are exemplary diagrams showing a method and structure of a blockchain-based high-speed data processing device according to an embodiment of the present invention. Each process described below is performed by each functional unit constituting a blockchain-based high-speed data processing device, but for a simple and clear explanation of the present invention, the subject of each step will be collectively referred to as a blockchain-based high-speed data processing device.

[0039] Referring to FIG. 3, the blockchain-based data processing device 10 can perform initialization work for operating the blockchain system.

[0040] In step S310, the blockchain-based data processing device 10 reads configuration information for operating the blockchain system.

[0041] In step S320, the blockchain-based data high-speed processor 10 can ascertain whether or not there is an active WAL file. If there is an active WAL file, the blockchain-based data high-speed processor 10 can generate an active MEMTable by utilizing the active WAL file information. If there is no active WAL file, the blockchain-based data high-speed processor 10 performs step S330.

[0042] In step S330, the blockchain-based high-speed data processor 10 can check whether there is an immutable WAL file. If there is an immutable WAL file, the blockchain-based high-speed data processor 10 can generate an immutable MEMTable by utilizing the immutable WAL file information. If there is no immutable WAL file, the blockchain-based high-speed data processor 10 performs step S340.

[0043] In step S340, the blockchain-based data processor 10 can drive the Flush channel.

[0044] 4, the blockchain-based data high-speed processing device 10 can store the key in a DKI file (Direct Key Index File). For example, the blockchain-based data high-speed processing device 10 can store [DID]:[BlockHeight] information in a DKI file (Direct Key Index File).

[0045] When the blockchain-based data processing device 10 is running, it can generate an Active MEMTable using WAL file information in an active state.

[0046] The blockchain-based high-speed data processing device 10 can set active WAL files and MEMTables to a certain size and use them to write / read input values ​​<index Internal index, IX>:<value> in real time.

[0047] When the set size of the active state MEMTable reaches the upper limit (or becomes full), the blockchain-based high-speed data processing device 10 can switch to the immutable state, generate a new active state WAL file, and use the WAL file information to generate a new active MEMTable.

[0048] The blockchain-based data high-speed processing device 10 can store and retain the same data [BlockHeight]:[Block] redundantly in a file (WAL file) and in memory (MEMTable).

[0049] When the blockchain-based data processing device 10 is running, it can generate an Immutable MEMTable using immutable WAL file information.

[0050] The blockchain-based data processing device 10 can generate a Value Log file corresponding to an immutable state and update a Value Index file through periodic flushing.

[0051] In the blockchain-based high-speed data processing device 10, a value is stored in a Value Log file in a flushed state, and an entry, which is search information for the value, can be stored in a Value Index file together with an index IX.

[0052] The blockchain-based high-speed data processing device 10 can store meta information [BlockHeight]:[METAINFO] of a block in a Value Index file that is a B+ tree data structure type. For example, the meta information ([METAINFO]={Fid, Offset, EntrySize}) may be information on the physical storage location (Offset, EntrySize) of a block in a Value Log file (Fid).

[0053] The blockchain-based high-speed data processing device 10 can asynchronously delete the immutable MEMTable and WAL files after transferring the data as a Value Index file and a Value Log file.

[0054] FIG. 5 is a diagram illustrating a method for storing (Put) data by the blockchain-based data high speed processing device 10 according to one embodiment of the present invention.

[0055] Here, Put, Write, and Save are all interpreted as having the same meaning, and will be expressed as "storing data in a file."

[0056] Referring to FIG. 5, in step S510, the blockchain-based data high speed processing device 10 can write <Key>:<Index IX> to a DKI file (Direct Key Index File).

[0057] In step S520, the blockchain-based data processing device 10 stores (Puts) <Index IX>:<Value> in MEMTable / WAL.

[0058] In step S530, the blockchain-based data high-speed processing device 10 can determine whether the Active MEMTable is full. If there is space to store (write), the blockchain-based data high-speed processing device 10 performs step S540.

[0059] In step S540, the blockchain-based data processing device 10 writes <index IX>:<value> to the Active WAL file.

[0060] In step S550, the blockchain-based data processing device 10 writes <Index IX>:<Value> to the Active MEMTable.

[0061] In step S530, if the Active MEMTable is full and there is no storage space, the blockchain-based data high speed processing device 10 performs step S560.

[0062] In step S560, the blockchain-based data high speed processing device 10 determines whether to process with a new MEMTable, and if a new MEMTable is required, can perform step S5610.

[0063] In step S5610, the blockchain-based data processing device 10 can switch the state of the MEMTable from Active to Immutable and add it to the Immutable MEMTable pool.

[0064] In step S5620, the blockchain-based data processing device 10 can also switch the state of the WAL file to an immutable state and specify it as an Immutable WAL file.

[0065] In step S5630, the blockchain-based data processing device 10 can generate a new Active WAL file.

[0066] In step S5640, the blockchain-based data processing device 10 can generate a new Active MEMTable.

[0067] In addition, the blockchain-based data high speed processing device 10 performs step S 5 30 and Steps 5 40 can be done at the same time.

[0068] The blockchain-based data high-speed processing device 10 may perform a step of storing information in each file (S540) after generating new Active MEMTble and WAL files.

[0069] If a new MEMTable is not needed in step S560, step S570 can be performed.

[0070] In step S570, the blockchain-based data processing device 10 can transmit the Flush channel to Immutable.

[0071] The blockchain-based data high-speed processing device 10 can check in step S580 whether the MEMTable has been transmitted to the Flush channel, and can check in step S5810 whether the MEMTable has been received.

[0072] In step S5810, if the MEMTable is received, the blockchain-based data processing device 10 can write the value in the Value Log file in step S5820.

[0073] In step S5830, the blockchain-based data processing device 10 can write <Index IX>:<Value> to the Value Index file.

[0074] In step S5840, the blockchain-based data processing device 10 can delete the Immutable MEMTable and the Immutable WAL file.

[0075] FIG. 6 is an example of a structural diagram in which the blockchain-based data high-speed processing device 10 stores (puts) data according to one embodiment of the present invention.

[0076] The blockchain-based data high-speed processing device 10 can store the key in a DKI file (Direct Key Index File). For example, the blockchain-based data high-speed processing device 10 can store [DID]:[BlockHeight] information in a DKI file (Direct Key Index File).

[0077] The blockchain-based high-speed data processing device 10 can store and retain the same data [BlockHeight]:[Block] in a file (WAL file) and in memory (MEMTable) in a duplicated manner.

[0078] The blockchain-based data processing device 10 can switch the state of the MEMTable from Active to Immutable and add it to the Immutable MEMTable pool.

[0079] The blockchain-based data processing device 10 can also switch the state of the WAL file to an immutable state and specify it as an Immutable WAL file.

[0080] The blockchain-based data high-speed processing device 10 can generate a Value Log file corresponding to an immutable state and update a Value Index file through periodic flushing.

[0081] In the blockchain-based high-speed data processing device 10, a value (value) is stored in a Value Log file in a flushed state, and an entry (Entry), which is search information for the value (value), can be stored in a Value Index file together with an internal index (IX).

[0082] The blockchain-based data high-speed processing device 10 can store meta information [BlockHeight]:[METAINFO] of a block in a Value Index file that is a B+ tree data structure type. For example, the information ([METAINFO]={Fid, Offset, EntrySize}) is the physical storage position (Offset, EntrySize) information of a block in a Value Log file (Fid).

[0083] FIG. 7 is a diagram illustrating a method for the blockchain-based high-speed data processing device 10 according to one embodiment of the present invention to read data.

[0084] Referring to FIG. 7, the blockchain-based high speed data processing device 10 can query (Get) the value corresponding to the <Key>:<Value> pair using the key as an input value.

[0085] In step S710, the blockchain-based data processing device 10 finds the index IX of the input key Key from the DKI file.

[0086] In step S710, the blockchain-based high-speed data processor 10 determines that the process is an error because it cannot find the value without the index IX, and ends the process. If the blockchain-based high-speed data processor 10 has an index IX corresponding to the input key Key, it performs step S730.

[0087] In step S730, if the MEMTable and WAL file in the active state have <index IX>, the blockchain-based high-speed data processing device 10 can read <index IX>:<value>. If the MEMTable and WAL file in the active state do not have index IX, the blockchain-based high-speed data processing device 10 performs step S740.

[0088] In step S740, if the MEMTable and WAL file in the immutable state have <Index IX>, the blockchain-based high-speed data processing device 10 can read <Index IX>:<Value>. If the MEMTable and WAL file in the immutable state do not have <Index IX>, the blockchain-based high-speed data processing device 10 performs step S750.

[0089] In step S750, if the Value Index file has an index IX, the blockchain-based data high speed processing device 10 can read the value Value corresponding to the index IX from the Value Log file.

[0090] 8, the blockchain-based high-speed data processing device 10 can search for a key Key from a DKI file, which is a B+ tree file, and obtain location information (Fid, Offset, EntrySize) of a value corresponding to the key Key from a Value Index file. The blockchain-based high-speed data processing device 10 can search for a Log file corresponding to the obtained location information Fid from a Value Log file, calculate an offset Offset in a predetermined manner from the file, and then read data corresponding to the entry size EntrySize, thereby referencing a value Value corresponding to the input key Key in the data structure.

[0091] The above-mentioned high-speed data processing method based on the blockchain may be realized by computer-readable code on a computer-readable medium. The above-mentioned computer-readable recording medium may be, for example, a portable recording medium (compact disc (CD), digital versatile disc (DVD), Blu-ray disc, universal serial bus (USB) storage device, portable hard disk) or a fixed recording medium (read-only memory (ROM), random access memory (RAM), hard disk built into a personal computer). The computer program recorded on the computer-readable recording medium may be transmitted to another computing device via a network such as the Internet and installed in the other computing device, thereby becoming available in the other computing device.

[0092] Although it has been described above that all components constituting the embodiments of the present invention are combined together or operate in combination, the present invention is not necessarily limited to such an embodiment, that is, all components may be selectively combined with one or more components and operate within the scope of the present invention.

[0093] Although the figures show operations in a particular order, it is not intended that the operations must be performed in the particular order shown, or in a sequential order, or that all of the illustrated operations must be performed to achieve desired results. In certain situations, multitasking and parallel processing may be advantageous. Moreover, the separation of various components in the above-described embodiments does not necessarily require such separation, and the program components and systems described may generally be integrated together in a single software product or packaged as multiple software products.

[0094] The present invention has been described above with reference to its embodiment. Those skilled in the art will appreciate that the present invention may be realized in modified forms without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments are to be considered from an explanatory perspective, not a restrictive one. The scope of the present invention is disclosed in the claims, not the above description, and all differences within the scope of the equivalents are included in the present invention. [Explanation of symbols]

[0095] 10. Blockchain-based data processing device 100 Key Management Unit 200 Value Management Department

Claims

1. In a blockchain-based data high-speed processing device, a key management unit that manages the keys of the <key>:<value> pairs; a value management unit that stores and manages the value of a <key>:<value> pair in a Value Log file; The key management unit Register the <key> and <index (IX)> in the DKI (direct key index) file, The <Index (IX)> is stored in a Value Index file, The <Index (IX)> includes Fid, Offset, and EntrySize, which are position information of a <Value> corresponding to a <Key>, Use the <key> to find the <index (IX)> in the DKI (direct key index) file, Using the <Index (IX)>, find the location information of the <Value> from the Value Index file; A Value Log file corresponding to the Fid of the location information of the <Value> is found, an offset in a manner determined in the Value Log file is calculated, and then data corresponding to the entry size EntrySize is read to query the <Value>; The DKI (direct key index) file and the Value Index file are B+ tree structure type. The Value Log file is an LSM tree structure. A blockchain-based high-speed data processing device.

2. 1. A method for implementing a blockchain-based data processor comprising: When writing / reading <Key>:<Value>, The state is one of Active, Immutable, and Flushed. The same data is stored in duplicate in the write-ahead log (WAL) file and in memory (MEMTable), Set the active MEMTable and WAL files to a certain size. When the size of the active MEMTable reaches its upper limit, the MEMTable and WAL files are registered in the immutable MEMTable and the state is set to immutable. Asynchronously move the immutable MEMTable and WAL files to the Value Index and Value Log files, then delete them. The Value Log file in the flushed state stores the Value, and the Entry, which is search information for the Value, is stored in the Value Index file together with the Index IX. <Key>:<Index IX> is registered in the DKI file (Direct Key Index File), The DKI file (Direct Key Index File) and the Value Index file are B+ tree structure type. The Value Log file is an LSM tree structure. A high-speed data processing method based on a blockchain.

Images